Method: Utilizes internal weight activations of multilingual LLMs, computes weight importance scores via adapted pruning algorithm to derive high-dimensional vector representations for languages.

Result: Validated across 106 languages, results align with established linguistic families while revealing unexpected inter-language connections suggesting historical contact or evolution.

Conclusion: The framework successfully captures linguistic phenomena through LLM weight analysis, providing new insights into language relationships with publicly available tools and data.

Abstract: We introduce a novel framework that utilizes the internal weight activations of modern Large Language Models (LLMs) to construct a metric space of languages. Unlike traditional approaches based on hand-crafted linguistic features, our method automatically derives high-dimensional vector representations by computing weight importance scores via an adapted pruning algorithm. Our approach captures intrinsic language characteristics that reflect linguistic phenomena. We validate our approach across diverse datasets and multilingual LLMs, covering 106 languages. The results align well with established linguistic families while also revealing unexpected inter-language connections that may indicate historical contact or language evolution. The source code, computed language latent vectors, and visualization tool are made publicly available at https://github.com/mshamrai/deep-language-geometry.

Method: Conducted two experiments: (1) varying retriever parameters while keeping generator fixed, and (2) varying generator architectures while keeping retriever parameters fixed. Tested across four datasets (two open-domain, two proprietary) to assess reliability of synthetic benchmarks compared to human-labeled baselines.

Result: Synthetic benchmarks reliably rank RAG systems with different retriever configurations, aligning well with human-labeled benchmarks. However, they fail to produce consistent rankings when comparing different generator architectures.

Conclusion: While synthetic QA data can serve as a proxy for human benchmarks in evaluating retriever configurations, it is unreliable for comparing generator architectures due to task mismatch and stylistic biases that favor certain generators.

Abstract: We investigate whether synthetic question-answer (QA) data generated by large language models (LLMs) can serve as an effective proxy for human-labeled benchmarks when such data is unavailable. We assess the reliability of synthetic benchmarks across two experiments: one varying retriever parameters while keeping the generator fixed, and another varying the generator with fixed retriever parameters. Across four datasets, of which two open-domain and two proprietary, we find that synthetic benchmarks reliably rank the RAGs varying in terms of retriever configuration, aligning well with human-labeled benchmark baselines. However, they fail to produce consistent RAG rankings when comparing generator architectures. The breakdown possibly arises from a combination of task mismatch between the synthetic and human benchmarks, and stylistic bias favoring certain generators.

Method: Applied imitation learning to conversation tasks, creating both a policy that generates responses given prompts and a discriminator that distinguishes between expert and synthetic conversations.

Result: Successfully recovered an effective conversation policy, but the discriminator revealed limitations in dialog models, showing this approach can identify adverse behaviors in data models.

Conclusion: Imitation learning is effective for conversation tasks, and the discriminator-based approach provides valuable insights into model limitations and can help identify problematic behaviors in dialog-oriented models.

Abstract: Imitation learning is a proven method for creating a policy in the absence of rewards, by leveraging expert demonstrations. In this work, we apply imitation learning to conversation. In doing so, we recover a policy capable of talking to a user given a prompt (input state), and a discriminator capable of classifying between expert and synthetic conversation. While our policy is effective, we recover results from our discriminator that indicate the limitations of dialog models. We argue that this technique can be used to identify adverse behavior of arbitrary data models common for dialog oriented tasks.

Method: Used a small hand-constructed lexicon to analyze transcription inconsistencies and tested bigram word-based language modeling vs lexicon-constrained decoding approaches.

Result: Found that transcription inconsistencies exist but are not the main challenge. Bigram word-based language modeling provided no benefit, but lexicon-constrained decoding showed some improvement.

Conclusion: The Faetar ASR task remains extremely difficult despite addressing transcription issues, suggesting other fundamental challenges beyond data quality.

Abstract: We examine the role of transcription inconsistencies in the Faetar Automatic Speech Recognition benchmark, a challenging low-resource ASR benchmark. With the help of a small, hand-constructed lexicon, we conclude that find that, while inconsistencies do exist in the transcriptions, they are not the main challenge in the task. We also demonstrate that bigram word-based language modelling is of no added benefit, but that constraining decoding to a finite lexicon can be beneficial. The task remains extremely difficult.

Method: Organized four tasks (content reproduction, comparison, scoring, reflection) requiring specific LLM roles, using first-rate Information Systems articles from top journals and multiple text metrics for rigorous evaluation of Google’s Gemini.

Result: Compromised performance across all tasks: acceptable reliability in summarization/paraphrasing, faint scalability in text comparison, poor discrimination in grading, self-consistent but uninsightful qualitative reflection. Consistent negative evidence across linguistic assessment, ground truth comparison, and human evaluation.

Conclusion: Do not recommend unchecked use of LLMs in constructing peer reviews due to significant limitations in text-processing capabilities for scholarly applications.

Abstract: How much large language models (LLMs) can aid scientific discovery, notably in assisting academic peer review, is in heated debate. Between a literature digest and a human-comparable research assistant lies their practical application potential. We organize individual tasks that computer science studies employ in separate terms into a guided and robust workflow to evaluate LLMs’ processing of academic text input. We employ four tasks in the assessment: content reproduction/comparison/scoring/reflection, each demanding a specific role of the LLM (oracle/judgmental arbiter/knowledgeable arbiter/collaborator) in assisting scholarly works, and altogether testing LLMs with questions that increasingly require intellectual capabilities towards a solid understanding of scientific texts to yield desirable solutions. We exemplify a rigorous performance evaluation with detailed instructions on the prompts. Adopting first-rate Information Systems articles at three top journals as the input texts and an abundant set of text metrics, we record a compromised performance of the leading LLM - Google’s Gemini: its summary and paraphrase of academic text is acceptably reliable; using it to rank texts through pairwise text comparison is faintly scalable; asking it to grade academic texts is prone to poor discrimination; its qualitative reflection on the text is self-consistent yet hardly insightful to inspire meaningful research. This evidence against an endorsement of LLMs’ text-processing capabilities is consistent across metric-based internal (linguistic assessment), external (comparing to the ground truth), and human evaluation, and is robust to the variations of the prompt. Overall, we do not recommend an unchecked use of LLMs in constructing peer reviews.

Method: Developed a lightweight analysis framework using statistical tools and training-free tasks to investigate SFM layers. Contributed spoken NER and NEL tasks to SLU benchmark, and developed SFM-based end-to-end approaches comparing them against traditional cascaded methods.

Result: The analysis provides insights into acoustic and linguistic knowledge encoded in SFM layers with implications for downstream performance. End-to-end models leveraging SFMs surpassed traditional cascaded approaches on spoken NER and NEL tasks.

Conclusion: This thesis addresses key gaps in SFM understanding by providing analytical tools and datasets, enabling informed design choices for future model development and demonstrating the superiority of end-to-end SFM approaches for complex spoken language understanding tasks.

Abstract: Speech foundation models (SFMs) are designed to serve as general-purpose representations for a wide range of speech-processing tasks. The last five years have seen an influx of increasingly successful self-supervised and supervised pre-trained models with impressive performance on various downstream tasks. Although the zoo of SFMs continues to grow, our understanding of the knowledge they acquire lags behind. This thesis presents a lightweight analysis framework using statistical tools and training-free tasks to investigate the acoustic and linguistic knowledge encoded in SFM layers. We conduct a comparative study across multiple SFMs and statistical tools. Our study also shows that the analytical insights have concrete implications for downstream task performance. The effectiveness of an SFM is ultimately determined by its performance on speech applications. Yet it remains unclear whether the benefits extend to spoken language understanding (SLU) tasks that require a deeper understanding than widely studied ones, such as speech recognition. The limited exploration of SLU is primarily due to a lack of relevant datasets. To alleviate that, this thesis contributes tasks, specifically spoken named entity recognition (NER) and named entity localization (NEL), to the Spoken Language Understanding Evaluation benchmark. We develop SFM-based approaches for NER and NEL, and find that end-to-end (E2E) models leveraging SFMs can surpass traditional cascaded (speech recognition followed by a text model) approaches. Further, we evaluate E2E SLU models across SFMs and adaptation strategies to assess the impact on task performance. Collectively, this thesis tackles previously unanswered questions about SFMs, providing tools and datasets to further our understanding and to enable the community to make informed design choices for future model development and adoption.

Method: Uses LLMs to generate structured plans for sentence simplification and concise summaries for document simplification, then performs plan-driven and summary-guided simplification

Result: Enables more coherent and contextually faithful simplifications of scientific text

Conclusion: The two-stage LLM-based framework effectively handles scientific text simplification at both sentence and document levels

Abstract: In this paper, we present our approach for the CLEF 2025 SimpleText Task 1, which addresses both sentence-level and document-level scientific text simplification. For sentence-level simplification, our methodology employs large language models (LLMs) to first generate a structured plan, followed by plan-driven simplification of individual sentences. At the document level, we leverage LLMs to produce concise summaries and subsequently guide the simplification process using these summaries. This two-stage, LLM-based framework enables more coherent and contextually faithful simplifications of scientific text.

Method: Modifies existing non-causal encoder to causal encoder by fine-tuning both encoder and decoder using Low-Rank Adaptation (LoRA) with weakly aligned dataset. Implements updated inference mechanism for greedy and beam-search decoding.

Result: Outperforms existing non-fine-tuned streaming approaches on low-latency chunk sizes (<300 msec) in most cases with lower complexity. Achieves better alignment enabling simple word-level timestamp extraction.

Conclusion: Successfully transforms transformer encoder-decoder models into efficient streaming ASR systems through causal encoder modification and LoRA fine-tuning, providing a practical solution for real-time transcription applications.

Abstract: Automatic Speech Recognition (ASR) has seen remarkable progress, with models like OpenAI Whisper and NVIDIA Canary achieving state-of-the-art (SOTA) performance in offline transcription. However, these models are not designed for streaming (online or real-time) transcription, due to limitations in their architecture and training methodology. We propose a method to turn the transformer encoder-decoder model into a low-latency streaming model that is careless about future context. We present an analysis explaining why it is not straightforward to convert an encoder-decoder transformer to a low-latency streaming model. Our proposed method modifies the existing (non-causal) encoder to a causal encoder by fine-tuning both the encoder and decoder using Low-Rank Adaptation (LoRA) and a weakly aligned dataset. We then propose an updated inference mechanism that utilizes the fine-tune causal encoder and decoder to yield greedy and beam-search decoding, and is shown to be locally optimal. Experiments on low-latency chunk sizes (less than 300 msec) show that our fine-tuned model outperforms existing non-fine-tuned streaming approaches in most cases, while using a lower complexity. Additionally, we observe that our training process yields better alignment, enabling a simple method for extracting word-level timestamps. We release our training and inference code, along with the fine-tuned models, to support further research and development in streaming ASR.

Method: Constructed ensemble framework integrating multiple strategies: BERT-based classifier, semantic similarity measures, natural language inference model, and LLM reasoning. Used meta-classifiers to combine diverse signals. Implemented LLM-based post-editing system for grounded generation that revises simplifications based on original texts.

Result: Developed a comprehensive solution that enhances robustness in detecting spurious content and information distortion through multi-strategy ensemble approach.

Conclusion: The integrated ensemble approach with diverse detection signals and LLM-based post-editing provides an effective methodology for addressing creative generation detection and information distortion evaluation in scientific text simplification.

Abstract: In this paper, we describe our methodology for the CLEF 2025 SimpleText Task 2, which focuses on detecting and evaluating creative generation and information distortion in scientific text simplification. Our solution integrates multiple strategies: we construct an ensemble framework that leverages BERT-based classifier, semantic similarity measure, natural language inference model, and large language model (LLM) reasoning. These diverse signals are combined using meta-classifiers to enhance the robustness of spurious and distortion detection. Additionally, for grounded generation, we employ an LLM-based post-editing system that revises simplifications based on the original input texts.

Method: Develop a framework using system prompts generated from periodic functions modeling key hormones (estrogen, testosterone, cortisol) to embed simulated menstrual and circadian cycles into Large Language Models.

Result: Linguistic analysis shows emotional/stylistic variations tracking biological phases (sadness peaks during menstruation, happiness dominates ovulation). Benchmarking reveals subtle but consistent performance variations across SQuAD, MMLU, Hellaswag, and AI2-ARC, with optimal function in moderate hormonal ranges.

Conclusion: The methodology provides a novel approach to contextual AI while revealing how societal biases regarding gender and biology are embedded within language models.

Abstract: Despite significant advances, AI systems struggle with the frame problem: determining what information is contextually relevant from an exponentially large possibility space. We hypothesize that biological rhythms, particularly hormonal cycles, serve as natural relevance filters that could address this fundamental challenge. We develop a framework that embeds simulated menstrual and circadian cycles into Large Language Models through system prompts generated from periodic functions modeling key hormones including estrogen, testosterone, and cortisol. Across multiple state-of-the-art models, linguistic analysis reveals emotional and stylistic variations that track biological phases; sadness peaks during menstruation while happiness dominates ovulation and circadian patterns show morning optimism transitioning to nocturnal introspection. Benchmarking on SQuAD, MMLU, Hellaswag, and AI2-ARC demonstrates subtle but consistent performance variations aligning with biological expectations, including optimal function in moderate rather than extreme hormonal ranges. This methodology provides a novel approach to contextual AI while revealing how societal biases regarding gender and biology are embedded within language models.

Method: Systematic review and analysis of 53 datasets from psycholinguistics and computational linguistics, examining their content, form, intended use, annotation practices, coverage, and task framing.

Result: Psycholinguistic resources contain normed ratings (familiarity, transparency, compositionality) while computational datasets support tasks like idiomaticity detection, paraphrasing, and cross-lingual modeling. Recent efforts expanded language coverage and task diversity.

Conclusion: Despite recent progress in both fields, there is currently no established relationship between psycholinguistic and computational research on idioms, indicating a gap that needs to be addressed for more comprehensive idiom understanding.

Abstract: Idioms are figurative expressions whose meanings often cannot be inferred from their individual words, making them difficult to process computationally and posing challenges for human experimental studies. This survey reviews datasets developed in psycholinguistics and computational linguistics for studying idioms, focusing on their content, form, and intended use. Psycholinguistic resources typically contain normed ratings along dimensions such as familiarity, transparency, and compositionality, while computational datasets support tasks like idiomaticity detection/classification, paraphrasing, and cross-lingual modeling. We present trends in annotation practices, coverage, and task framing across 53 datasets. Although recent efforts expanded language coverage and task diversity, there seems to be no relation yet between psycholinguistic and computational research on idioms.

Method: Proposed Speech-First Multimodal Training (SFMT) using curriculum learning to establish robust speech modeling foundations before cross-modal fusion. Systematic study of MLLM for comprehensive ASA.

Result: MLLM-based systems elevated holistic assessment performance from PCC 0.783 to 0.846. SFMT achieved 4% absolute accuracy improvement in delivery aspect evaluation over conventional approaches.

Conclusion: MLLMs show superior performance for ASA, particularly in content and language aspects. Delivery assessment requires specialized training like SFMT, which paves new avenue for comprehensive automated speaking assessment.

Abstract: Traditional Automated Speaking Assessment (ASA) systems exhibit inherent modality limitations: text-based approaches lack acoustic information while audio-based methods miss semantic context. Multimodal Large Language Models (MLLM) offer unprecedented opportunities for comprehensive ASA by simultaneously processing audio and text within unified frameworks. This paper presents a very first systematic study of MLLM for comprehensive ASA, demonstrating the superior performance of MLLM across the aspects of content and language use . However, assessment on the delivery aspect reveals unique challenges, which is deemed to require specialized training strategies. We thus propose Speech-First Multimodal Training (SFMT), leveraging a curriculum learning principle to establish more robust modeling foundations of speech before cross-modal synergetic fusion. A series of experiments on a benchmark dataset show MLLM-based systems can elevate the holistic assessment performance from a PCC value of 0.783 to 0.846. In particular, SFMT excels in the evaluation of the delivery aspect, achieving an absolute accuracy improvement of 4% over conventional training approaches, which also paves a new avenue for ASA.

Method: Compared sequential fine-tuning with monolingual and simultaneous fine-tuning using XLM-R and mBERT across five languages (English, Spanish, Chinese, Turkish, Yoruba), analyzing performance based on language pairings, typological features, and pretraining coverage.

Result: Sequential fine-tuning with high-resource L1 improves L2 performance, especially for low-resource languages like Yoruba and Turkish. XLM-R achieves larger gains but is more sensitive to pretraining gaps and catastrophic forgetting, while mBERT yields more stable though lower results.

Conclusion: Sequential fine-tuning is a simple yet effective strategy for improving euphemism detection in multilingual models, particularly beneficial for low-resource languages.

Abstract: Euphemisms are culturally variable and often ambiguous, posing challenges for language models, especially in low-resource settings. This paper investigates how cross-lingual transfer via sequential fine-tuning affects euphemism detection across five languages: English, Spanish, Chinese, Turkish, and Yoruba. We compare sequential fine-tuning with monolingual and simultaneous fine-tuning using XLM-R and mBERT, analyzing how performance is shaped by language pairings, typological features, and pretraining coverage. Results show that sequential fine-tuning with a high-resource L1 improves L2 performance, especially for low-resource languages like Yoruba and Turkish. XLM-R achieves larger gains but is more sensitive to pretraining gaps and catastrophic forgetting, while mBERT yields more stable, though lower, results. These findings highlight sequential fine-tuning as a simple yet effective strategy for improving euphemism detection in multilingual models, particularly when low-resource languages are involved.

Method: SupraTok extends Byte-Pair Encoding with three innovations: cross-boundary pattern learning for multi-word semantic units, entropy-driven data curation for optimal training corpus quality, and multi-phase curriculum learning for stable convergence.

Result: Achieves 31% improvement in English tokenization efficiency (5.91 vs 4.51 chars/token) vs OpenAI’s tokenizer and 30% vs Google’s Gemma 3. When integrated with GPT-2 scale model, yields 8.4% improvement on HellaSWAG and 9.5% on MMLU benchmarks.

Conclusion: Efficient tokenization can complement architectural innovations as a path to improved language model performance, though further validation at larger scales is needed.

Abstract: Tokenization remains a fundamental yet underexplored bottleneck in natural language processing, with strategies largely static despite remarkable progress in model architectures. We present SupraTok, a novel tokenization architecture that reimagines subword segmentation through three innovations: cross-boundary pattern learning that discovers multi-word semantic units, entropy-driven data curation that optimizes training corpus quality, and multi-phase curriculum learning for stable convergence. Our approach extends Byte-Pair Encoding by learning “superword” tokens, coherent multi-word expressions that preserve semantic unity while maximizing compression efficiency. SupraTok achieves 31% improvement in English tokenization efficiency (5.91 versus 4.51 characters per token) compared to OpenAI’s o200k tokenizer and 30% improvement over Google’s Gemma 3 tokenizer (256k vocabulary), while maintaining competitive performance across 38 languages. When integrated with a GPT-2 scale model (124M parameters) trained on 10 billion tokens from the FineWeb-Edu dataset, SupraTok yields 8.4% improvement on HellaSWAG and 9.5% on MMLU benchmarks without architectural modifications. While these results are promising at this scale, further validation at larger model scales is needed. These findings suggest that efficient tokenization can complement architectural innovations as a path to improved language model performance.

Method: InitERC uses a one-stage in-context instruction tuning framework with four components: demonstration pool construction, in-context example selection, prompt template design, and in-context instruction tuning. It explores retrieval strategies, example ordering, and number of examples to optimize performance.

Result: Extensive experiments on three widely used datasets demonstrate that InitERC achieves substantial improvements over state-of-the-art baselines in emotion recognition performance.

Conclusion: The proposed one-stage in-context instruction tuning framework effectively learns speaker-context-emotion alignment and outperforms existing multi-stage approaches, providing a more unified and effective solution for emotion recognition in conversations.

Abstract: Emotion recognition in conversation (ERC) aims to identify the emotion of each utterance in a conversation, playing a vital role in empathetic artificial intelligence. With the growing of large language models (LLMs), instruction tuning has emerged as a critical paradigm for ERC. Existing studies mainly focus on multi-stage instruction tuning, which first endows LLMs with speaker characteristics, and then conducts context-aware instruction tuning to comprehend emotional states. However, these methods inherently constrains the capacity to jointly capture the dynamic interaction between speaker characteristics and conversational context, resulting in weak alignment among speaker identity, contextual cues, and emotion states within a unified framework. In this paper, we propose InitERC, a simple yet effective one-stage in-context instruction tuning framework for ERC. InitERC adapts LLMs to learn speaker-context-emotion alignment from context examples via in-context instruction tuning. Specifically, InitERC comprises four components, i.e., demonstration pool construction, in-context example selection, prompt template design, and in-context instruction tuning. To explore the impact of in-context examples, we conduct a comprehensive study on three key factors: retrieval strategy, example ordering, and the number of examples. Extensive experiments on three widely used datasets demonstrate that our proposed InitERC achieves substantial improvements over the state-of-the-art baselines.

Method: Developed CORE metric integrating cluster entropy, lexical repetition, and semantic similarity. Applied to pairwise LLM dialogs in competitive, cooperative, and neutral settings, analyzed through Zipf’s and Heaps’ Laws for word frequency distributions and vocabulary growth.

Result: Cooperative settings exhibit steeper Zipf distributions and higher Heap exponents (more repetition with greater vocabulary expansion), while competitive interactions show lower exponents (less repetition with constrained vocabularies).

Conclusion: Social incentives significantly influence language adaptation in multi-agent systems, and CORE serves as a robust diagnostic tool for measuring linguistic robustness in LLM-based interactions.

Abstract: Game-theoretic interactions between agents with Large Language Models (LLMs) have revealed many emergent capabilities, yet the linguistic diversity of these interactions has not been sufficiently quantified. In this paper, we present the Conversational Robustness Evaluation Score: CORE, a metric to quantify the effectiveness of language use within multi-agent systems across different game-theoretic interactions. CORE integrates measures of cluster entropy, lexical repetition, and semantic similarity, providing a direct lens of dialog quality. We apply CORE to pairwise LLM dialogs across competitive, cooperative, and neutral settings, further grounding our analysis in Zipf’s and Heaps’ Laws to characterize word frequency distributions and vocabulary growth. Our findings show that cooperative settings exhibit both steeper Zipf distributions and higher Heap exponents, indicating more repetition alongside greater vocabulary expansion. In contrast, competitive interactions display lower Zipf and Heaps exponents, reflecting less repetition and more constrained vocabularies. These results provide new insights into how social incentives influence language adaptation, and highlight CORE as a robust diagnostic for measuring linguistic robustness in multi-agent LLM systems. Our code is available at https://github.com/psyonp/core.

Method: Three strategies: 1) Privacy-preserved text-truth sample generation using ASR-transcribed calls with TTS regeneration, 2) LLM-based self-instruction sampling for semantic enhancement, 3) Multi-agent adversarial synthesis simulating fraud tactics.

Result: Created TeleAntiFraud-28k dataset with 28,511 speech-text pairs and detailed fraud reasoning annotations, plus TeleAntiFraud-Bench evaluation benchmark and production-optimized SFT model.

Conclusion: Establishes foundational framework for multimodal anti-fraud research while addressing data privacy and scenario diversity challenges, with open-source release to enable community-driven expansion.

Abstract: The detection of telecom fraud faces significant challenges due to the lack of high-quality multimodal training data that integrates audio signals with reasoning-oriented textual analysis. To address this gap, we present TeleAntiFraud-28k, the first open-source audio-text slow-thinking dataset specifically designed for automated telecom fraud analysis. Our dataset is constructed through three strategies: (1) Privacy-preserved text-truth sample generation using automatically speech recognition (ASR)-transcribed call recordings (with anonymized original audio), ensuring real-world consistency through text-to-speech (TTS) model regeneration; (2) Semantic enhancement via large language model (LLM)-based self-instruction sampling on authentic ASR outputs to expand scenario coverage; (3) Multi-agent adversarial synthesis that simulates emerging fraud tactics through predefined communication scenarios and fraud typologies. The generated dataset contains 28,511 rigorously processed speech-text pairs, complete with detailed annotations for fraud reasoning. The dataset is divided into three tasks: scenario classification, fraud detection, fraud type classification. Furthermore, we construct TeleAntiFraud-Bench, a standardized evaluation benchmark comprising proportionally sampled instances from the dataset, to facilitate systematic testing of model performance on telecom fraud detection tasks. We also contribute a production-optimized supervised fine-tuning (SFT) model trained on hybrid real/synthetic data, while open-sourcing the data processing framework to enable community-driven dataset expansion. This work establishes a foundational framework for multimodal anti-fraud research while addressing critical challenges in data privacy and scenario diversity. The project will be released at https://github.com/JimmyMa99/TeleAntiFraud.

Method: Constructed linguistically motivated, template-based NLI dataset with 1,350 sentence pairs per language (Chinese and Japanese) focusing on perfect aspect temporal relationships.

Result: Advanced LLMs struggle significantly with temporal inference, particularly in detecting subtle tense and reference-time shifts, revealing limitations in cross-linguistic temporal semantics understanding.

Conclusion: Findings highlight critical model limitations and underscore the importance of cross-linguistic evaluation for temporal semantics, with dataset made publicly available for further research.

Abstract: Unlike English, which uses distinct forms (e.g., had, has, will have) to mark the perfect aspect across tenses, Chinese and Japanese lack separate grammatical forms for tense within the perfect aspect, which complicates Natural Language Inference (NLI). Focusing on the perfect aspect in these languages, we construct a linguistically motivated, template-based NLI dataset (1,350 pairs per language). Experiments reveal that even advanced LLMs struggle with temporal inference, particularly in detecting subtle tense and reference-time shifts. These findings highlight model limitations and underscore the need for cross-linguistic evaluation in temporal semantics. Our dataset is available at https://github.com/Lujie2001/CrossNLI.

Method: Systematic study of four compression methods: weight pruning, head pruning, low-rank approximation, and knowledge distillation on self-supervised speech Transformers. Evaluation under three key metrics: parameter count, multiply-accumulate operations, and real-time factor.

Result: Each compression method offers distinct advantages. The study provides comparative analysis of recent techniques including DistilHuBERT, FitHuBERT, LightHuBERT, ARMHuBERT, and STaRHuBERT under a unified evaluation framework.

Conclusion: The research offers practical guidance for compression deployment in speech Transformers, establishing a consistent evaluation framework that enables meaningful comparison of different compression techniques for real-world applications.

Abstract: Transformer-based self-supervised models have achieved remarkable success in speech processing, but their large size and high inference cost present significant challenges for real-world deployment. While numerous compression techniques have been proposed, inconsistent evaluation metrics make it difficult to compare their practical effectiveness. In this work, we conduct a comprehensive study of four common compression methods, including weight pruning, head pruning, low-rank approximation, and knowledge distillation on self-supervised speech Transformers. We evaluate each method under three key metrics: parameter count, multiply-accumulate operations, and real-time factor. Results show that each method offers distinct advantages. In addition, we contextualize recent compression techniques, comparing DistilHuBERT, FitHuBERT, LightHuBERT, ARMHuBERT, and STaRHuBERT under the same framework, offering practical guidance on compression for deployment.

Method: CAMF uses multiple LLM-based agents in a three-phase process: Multi-dimensional Linguistic Feature Extraction, Adversarial Consistency Probing, and Synthesized Judgment Aggregation to analyze subtle textual incongruities.

Result: Empirical evaluations show CAMF significantly outperforms state-of-the-art zero-shot MGT detection techniques.

Conclusion: The collaborative adversarial multi-agent framework provides a more effective approach for detecting machine-generated text by deeply analyzing cross-dimensional linguistic inconsistencies.

Abstract: Detecting machine-generated text (MGT) from contemporary Large Language Models (LLMs) is increasingly crucial amid risks like disinformation and threats to academic integrity. Existing zero-shot detection paradigms, despite their practicality, often exhibit significant deficiencies. Key challenges include: (1) superficial analyses focused on limited textual attributes, and (2) a lack of investigation into consistency across linguistic dimensions such as style, semantics, and logic. To address these challenges, we introduce the \textbf{C}ollaborative \textbf{A}dversarial \textbf{M}ulti-agent \textbf{F}ramework (\textbf{CAMF}), a novel architecture using multiple LLM-based agents. CAMF employs specialized agents in a synergistic three-phase process: \emph{Multi-dimensional Linguistic Feature Extraction}, \emph{Adversarial Consistency Probing}, and \emph{Synthesized Judgment Aggregation}. This structured collaborative-adversarial process enables a deep analysis of subtle, cross-dimensional textual incongruities indicative of non-human origin. Empirical evaluations demonstrate CAMF’s significant superiority over state-of-the-art zero-shot MGT detection techniques.

Method: Formally defined singing style captioning task and created S2Cap dataset with detailed descriptions, then developed an efficient baseline algorithm for the task.

Result: Created a comprehensive dataset of singing voices with diverse characteristics and developed a working baseline approach for singing style captioning.

Conclusion: S2Cap dataset fills the gap in singing voice datasets and enables better singing style captioning through detailed acoustic and vocal attribute descriptions.

Abstract: Singing voices contain much richer information than common voices, including varied vocal and acoustic properties. However, current open-source audio-text datasets for singing voices capture only a narrow range of attributes and lack acoustic features, leading to limited utility towards downstream tasks, such as style captioning. To fill this gap, we formally define the singing style captioning task and present S2Cap, a dataset of singing voices with detailed descriptions covering diverse vocal, acoustic, and demographic characteristics. Using this dataset, we develop an efficient and straightforward baseline algorithm for singing style captioning. The dataset is available at https://zenodo.org/records/15673764.

Method: Splits training and memory data into correct/incorrect parts, uses dual-task fine-tuning, and employs instruction-based contrastive tuning to continuously correct cognitive biases using previous data in instruction-tuning manner.

Result: Achieves new state-of-the-art performance on TACRED and FewRel datasets with significant improvements.

Conclusion: Specializing in exploiting error cases is crucial for effective continual relation extraction in LLMs, and the proposed approach effectively mitigates the gap between old and new relations.

Abstract: Continual Relation Extraction (CRE) aims to continually learn new emerging relations while avoiding catastrophic forgetting. Existing CRE methods mainly use memory replay and contrastive learning to mitigate catastrophic forgetting. However, these methods do not attach importance to the error cases that can reveal the model’s cognitive biases more effectively. To address this issue, we propose an instruction-based continual contrastive tuning approach for Large Language Models (LLMs) in CRE. Different from existing CRE methods that typically handle the training and memory data in a unified manner, this approach splits the training and memory data of each task into two parts respectively based on the correctness of the initial responses and treats them differently through dual-task fine-tuning. In addition, leveraging the advantages of LLM’s instruction-following ability, we propose a novel instruction-based contrastive tuning strategy for LLM to continuously correct current cognitive biases with the guidance of previous data in an instruction-tuning manner, which mitigates the gap between old and new relations in a more suitable way for LLMs. We experimentally evaluate our model on TACRED and FewRel, and the results show that our model achieves new state-of-the-art CRE performance with significant improvements, demonstrating the importance of specializing in exploiting error cases.

Method: Developed a systematic benchmark framework with automatic metrics for consistent and reproducible assessment of key interactive behaviors in full-duplex spoken dialogue systems.

Result: Created Full-Duplex-Bench, which provides a fair and fast evaluation setup for measuring pause handling, backchanneling, turn-taking, and interruption management capabilities.

Conclusion: The benchmark and released code aim to advance spoken dialogue modeling and foster development of more natural and engaging spoken dialogue models by providing comprehensive evaluation tools.

Abstract: Spoken dialogue modeling poses challenges beyond text-based language modeling, requiring real-time interaction, turn-taking, and backchanneling. While most Spoken Dialogue Models (SDMs) operate in half-duplex mode-processing one turn at a time - emerging full-duplex SDMs can listen and speak simultaneously, enabling more natural conversations. However, current evaluations remain limited, focusing mainly on turn-based metrics or coarse corpus-level analyses. To address this, we introduce Full-Duplex-Bench, a benchmark that systematically evaluates key interactive behaviors: pause handling, backchanneling, turn-taking, and interruption management. Our framework uses automatic metrics for consistent, reproducible assessment and provides a fair, fast evaluation setup. By releasing our benchmark and code, we aim to advance spoken dialogue modeling and foster the development of more natural and engaging SDMs.

Method: Developed a pipeline for constructing training data capturing LLM response distributions, trained a supervised model to predict confidence scores, proposed Backward Confidence Integration (BCI) strategy using subsequent text information, and introduced three strategies for optimal confidence estimation positions.

Result: Extensive experiments on multiple benchmark datasets demonstrate that FineCE consistently outperforms existing classical confidence estimation methods.

Conclusion: FineCE provides accurate, fine-grained confidence estimation during text generation, enhancing the trustworthiness and reliability of LLM-generated outputs with superior performance over existing methods.

Abstract: While large language models (LLMs) have demonstrated remarkable performance across diverse tasks, they fundamentally lack self-awareness and frequently exhibit overconfidence, assigning high confidence scores to incorrect predictions. Accurate confidence estimation is therefore critical for enhancing the trustworthiness and reliability of LLM-generated outputs. However, existing approaches suffer from coarse-grained scoring mechanisms that fail to provide fine-grained, continuous confidence estimates throughout the generation process. To address these limitations, we introduce FineCE, a novel confidence estimation method that delivers accurate, fine-grained confidence scores during text generation. Specifically, we first develop a comprehensive pipeline for constructing training data that effectively captures the underlying probabilistic distribution of LLM responses, and then train a model to predict confidence scores for arbitrary text sequences in a supervised manner. Furthermore, we propose a Backward Confidence Integration (BCI) strategy that leverages information from the subsequent text to enhance confidence estimation for the current sequence during inference. We also introduce three strategies for identifying optimal positions to perform confidence estimation within the generation process. Extensive experiments on multiple benchmark datasets demonstrate that FineCE consistently outperforms existing classical confidence estimation methods. Our code and all baselines used in the paper are available on GitHub.

Method: Proposes J6, a structured Jacobian-based method that decomposes the gradient interaction matrix into six interpretable components, enabling both hard decision-making (argmax) and soft strategies (softmax weighting) for dynamic update framework adaptation.

Result: The method provides interpretable structure for parameter attribution, task interference analysis, and geometry-aligned adaptation in multi-objective prompt optimization.

Conclusion: J6 introduces a principled and extensible mechanism for conflict-aware prompt optimization and opens new avenues for incorporating structured Jacobian reasoning into multi-objective neural tuning.

Abstract: In large language model (LLM) adaptation, balancing multiple optimization objectives such as improving factuality (heat) and increasing confidence (via low entropy) poses a fundamental challenge, especially when prompt parameters (e.g., hidden-layer insertions h and embedding modifications w) interact in non-trivial ways. Existing multi-objective optimization strategies often rely on scalar gradient aggregation, ignoring the deeper geometric structure between objectives and parameters. We propose J6, a structured Jacobian-based method that decomposes the gradient interaction matrix into six interpretable components. This decomposition enables both hard decision-making (e.g., choosing the dominant update direction via argmax) and soft strategies (e.g., attention-style weighting via softmax over J6), forming a dynamic update framework that adapts to local conflict and synergy. Moreover, the interpretable structure of J6 provides insight into parameter attribution, task interference, and geometry-aligned adaptation. Our work introduces a principled and extensible mechanism for conflict-aware prompt optimization, and opens a new avenue for incorporating structured Jacobian reasoning into multi-objective neural tuning.

Method: Identifies significant transition samples (STS) by analyzing performance transitions among LLMs of same architecture but varying parameter scales, then uses these samples to estimate capability positions of unknown models.

Result: STEM reliably captures performance trends and aligns with ground-truth rankings of model capability across six diverse benchmarks using the Qwen3 model family.

Conclusion: STEM provides a practical and scalable method for fine-grained, architecture-agnostic evaluation of LLMs, offering an efficient alternative to full benchmark evaluations.

Abstract: Evaluating large language models (LLMs) has become increasingly challenging as model capabilities advance rapidly. While recent models often achieve higher scores on standard benchmarks, these improvements do not consistently reflect enhanced real-world reasoning capabilities. Moreover, widespread overfitting to public benchmarks and the high computational cost of full evaluations have made it both expensive and less effective to distinguish meaningful differences between models. To address these challenges, we propose the \textbf{S}tructured \textbf{T}ransition \textbf{E}valuation \textbf{M}ethod (STEM), a lightweight and interpretable evaluation framework for efficiently estimating the relative capabilities of LLMs. STEM identifies \textit{significant transition samples} (STS) by analyzing consistent performance transitions among LLMs of the same architecture but varying parameter scales. These samples enable STEM to effectively estimate the capability position of an unknown model. Qwen3 model family is applied to construct the STS pool on six diverse and representative benchmarks. To assess generalizability. Experimental results indicate that STEM reliably captures performance trends, aligns with ground-truth rankings of model capability. These findings highlight STEM as a practical and scalable method for fine-grained, architecture-agnostic evaluation of LLMs.

Method: Systematic evaluation of Qwen3 (1.7B-235B) and DeepSeek-R1 (1.5B-70B) models across 15 medical datasets with controlled thinking budgets from zero to unlimited tokens.

Result: Identified three efficiency regimes: high-efficiency (0-256 tokens), balanced (256-512 tokens), and high-accuracy (>512 tokens). Smaller models showed 15-20% improvements vs 5-10% for larger models. Domain-specific patterns emerged with neurology/gastroenterology requiring deeper reasoning.

Conclusion: Thinking budget control is critical for optimizing medical AI systems, enabling dynamic resource allocation aligned with clinical needs while maintaining transparency for healthcare deployment.

Abstract: This study presents the first comprehensive evaluation of thinking budget mechanisms in medical reasoning tasks, revealing fundamental scaling laws between computational resources and reasoning quality. We systematically evaluated two major model families, Qwen3 (1.7B to 235B parameters) and DeepSeek-R1 (1.5B to 70B parameters), across 15 medical datasets spanning diverse specialties and difficulty levels. Through controlled experiments with thinking budgets ranging from zero to unlimited tokens, we establish logarithmic scaling relationships where accuracy improvements follow a predictable pattern with both thinking budget and model size. Our findings identify three distinct efficiency regimes: high-efficiency (0 to 256 tokens) suitable for real-time applications, balanced (256 to 512 tokens) offering optimal cost-performance tradeoffs for routine clinical support, and high-accuracy (above 512 tokens) justified only for critical diagnostic tasks. Notably, smaller models demonstrate disproportionately larger benefits from extended thinking, with 15 to 20% improvements compared to 5 to 10% for larger models, suggesting a complementary relationship where thinking budget provides greater relative benefits for capacity-constrained models. Domain-specific patterns emerge clearly, with neurology and gastroenterology requiring significantly deeper reasoning processes than cardiovascular or respiratory medicine. The consistency between Qwen3 native thinking budget API and our proposed truncation method for DeepSeek-R1 validates the generalizability of thinking budget concepts across architectures. These results establish thinking budget control as a critical mechanism for optimizing medical AI systems, enabling dynamic resource allocation aligned with clinical needs while maintaining the transparency essential for healthcare deployment.

Method: Conducted a comprehensive study with 10 datasets, 13 different LLMs, and 677 human participants to compare LLM privacy evaluations against human perceptions of text sensitivity.

Result: LLMs can accurately model a global human privacy perspective, though privacy proves difficult to measure empirically with generally low inter-human agreement rates.

Conclusion: LLM-as-a-Judge shows promise for privacy evaluation in textual data, paving the way for using LLMs as privacy evaluators to address core privacy challenges with innovative technical solutions.

Abstract: Despite advances in the field of privacy-preserving Natural Language Processing (NLP), a significant challenge remains the accurate evaluation of privacy. As a potential solution, using LLMs as a privacy evaluator presents a promising approach $\unicode{x2013}$ a strategy inspired by its success in other subfields of NLP. In particular, the so-called $\textit{LLM-as-a-Judge}$ paradigm has achieved impressive results on a variety of natural language evaluation tasks, demonstrating high agreement rates with human annotators. Recognizing that privacy is both subjective and difficult to define, we investigate whether LLM-as-a-Judge can also be leveraged to evaluate the privacy sensitivity of textual data. Furthermore, we measure how closely LLM evaluations align with human perceptions of privacy in text. Resulting from a study involving 10 datasets, 13 LLMs, and 677 human survey participants, we confirm that privacy is indeed a difficult concept to measure empirically, exhibited by generally low inter-human agreement rates. Nevertheless, we find that LLMs can accurately model a global human privacy perspective, and through an analysis of human and LLM reasoning patterns, we discuss the merits and limitations of LLM-as-a-Judge for privacy evaluation in textual data. Our findings pave the way for exploring the feasibility of LLMs as privacy evaluators, addressing a core challenge in solving pressing privacy issues with innovative technical solutions.

Method: Developed a novel taxonomy to categorize Arabic MML research into four key topics: datasets, applications, approaches, and challenges. Analyzed existing literature through this structured framework.

Result: Provides a structured overview of current Arabic MML state, identifies unexplored areas and critical research gaps, and offers insights to guide future research directions.

Conclusion: This survey empowers researchers to build on identified opportunities and address challenges to advance Arabic MML field, serving as a foundational reference for future work in Arabic multimodal learning.

Abstract: Multimodal Machine Learning (MML) aims to integrate and analyze information from diverse modalities, such as text, audio, and visuals, enabling machines to address complex tasks like sentiment analysis, emotion recognition, and multimedia retrieval. Recently, Arabic MML has reached a certain level of maturity in its foundational development, making it time to conduct a comprehensive survey. This paper explores Arabic MML by categorizing efforts through a novel taxonomy and analyzing existing research. Our taxonomy organizes these efforts into four key topics: datasets, applications, approaches, and challenges. By providing a structured overview, this survey offers insights into the current state of Arabic MML, highlighting areas that have not been investigated and critical research gaps. Researchers will be empowered to build upon the identified opportunities and address challenges to advance the field.

Method: Created SEA-BED benchmark with 169 datasets (71% human-formulated) across 9 tasks and 10 SEA languages, then evaluated 17 embedding models across six studies analyzing task challenges, cross-benchmark comparisons, and translation effects.

Result: Results show sharp ranking shifts, inconsistent model performance among SEA languages, and the critical importance of human-curated datasets for low-resource languages like Burmese.

Conclusion: Human-curated benchmarks are essential for accurate evaluation of SEA languages, as machine-translated datasets fail to capture linguistic nuances, leading to unreliable performance assessments in this diverse linguistic region.

Abstract: Sentence embeddings are essential for NLP tasks such as semantic search, re-ranking, and textual similarity. Although multilingual benchmarks like MMTEB broaden coverage, Southeast Asia (SEA) datasets are scarce and often machine-translated, missing native linguistic properties. With nearly 700 million speakers, the SEA region lacks a region-specific embedding benchmark. We introduce SEA-BED, the first large-scale SEA embedding benchmark with 169 datasets across 9 tasks and 10 languages, where 71% are formulated by humans, not machine generation or translation. We address three research questions: (1) which SEA languages and tasks are challenging, (2) whether SEA languages show unique performance gaps globally, and (3) how human vs. machine translations affect evaluation. We evaluate 17 embedding models across six studies, analyzing task and language challenges, cross-benchmark comparisons, and translation trade-offs. Results show sharp ranking shifts, inconsistent model performance among SEA languages, and the importance of human-curated datasets for low-resource languages like Burmese.

Method: Systematic review examining text-to-structure techniques, challenges, current datasets, assessment criteria, and introducing a universal evaluation framework for structured outputs.

Result: Establishes text-to-structure as foundational infrastructure for next-generation AI systems through comprehensive analysis of existing approaches and evaluation methodologies.

Conclusion: Text-to-structure conversion is critical infrastructure for advanced AI systems, and the review provides synthesis of current state while outlining future research directions and evaluation standards.

Abstract: The evolution of AI systems toward agentic operation and context-aware retrieval necessitates transforming unstructured text into structured formats like tables, knowledge graphs, and charts. While such conversions enable critical applications from summarization to data mining, current research lacks a comprehensive synthesis of methodologies, datasets, and metrics. This systematic review examines text-to-structure techniques and the encountered challenges, evaluates current datasets and assessment criteria, and outlines potential directions for future research. We also introduce a universal evaluation framework for structured outputs, establishing text-to-structure as foundational infrastructure for next-generation AI systems.

Method: Proposed a novel taxonomy with two dimensions: fast/slow boundary (intuitive vs deliberative) and internal/external boundary (parameter-based vs tool-augmented reasoning), followed by systematic survey of adaptive reasoning methods.

Result: Developed a comprehensive categorization framework for LLM reasoning strategies that systematically organizes recent work based on key decision factors.

Conclusion: Highlights open challenges and future directions for developing more adaptive, efficient, and reliable LLMs through better reasoning strategy selection.

Abstract: Large Language Models (LLMs) have demonstrated remarkable progress in reasoning across diverse domains. However, effective reasoning in real-world tasks requires adapting the reasoning strategy to the demands of the problem, ranging from fast, intuitive responses to deliberate, step-by-step reasoning and tool-augmented thinking. Drawing inspiration from cognitive psychology, we propose a novel taxonomy of LLM reasoning strategies along two knowledge boundaries: a fast/slow boundary separating intuitive from deliberative processes, and an internal/external boundary distinguishing reasoning grounded in the model’s parameters from reasoning augmented by external tools. We systematically survey recent work on adaptive reasoning in LLMs and categorize methods based on key decision factors. We conclude by highlighting open challenges and future directions toward more adaptive, efficient, and reliable LLMs.

Method: Introduced the Self-Execution Benchmark to measure model’s ability to predict properties of its own responses, including question difficulty, refusal behavior, and likely associations.

Result: Models generally perform poorly on this benchmark, and increased model size or capability does not consistently lead to better performance.

Conclusion: There is a fundamental limitation in how LLMs represent and reason about their own behavior, suggesting current architectures lack self-awareness capabilities.

Abstract: Large language models (LLMs) are commonly evaluated on tasks that test their knowledge or reasoning abilities. In this paper, we explore a different type of evaluation: whether an LLM can predict aspects of its own responses. Since LLMs lack the ability to execute themselves, we introduce the Self-Execution Benchmark, which measures a model’s ability to anticipate properties of its output, such as whether a question will be difficult for it, whether it will refuse to answer, or what kinds of associations it is likely to produce. Our experiments show that models generally perform poorly on this benchmark, and that increased model size or capability does not consistently lead to better performance. These results suggest a fundamental limitation in how LLMs represent and reason about their own behavior.

Method: Two-stage approach: (1) distills latent reasoning capabilities from DeepSeek-R1 (Large Reasoning Model), (2) refines reasoning quality via differential comparisons with dual-mode input (direct answer vs reasoning-augmented) and multidimensional reward mechanism assessing structural coherence and legal-domain specificity.

Result: Outperforms strong baselines on multiple legal reasoning tasks in both accuracy and interpretability, consistently producing more robust and trustworthy legal judgments without relying on labeled preference data.

Conclusion: LegalΔ successfully addresses the interpretability challenge in legal AI by encouraging meaningful reasoning patterns through information gain maximization, demonstrating significant improvements in legal reasoning quality and reliability.

Abstract: Legal Artificial Intelligence (LegalAI) has achieved notable advances in automating judicial decision-making with the support of Large Language Models (LLMs). However, existing legal LLMs still struggle to generate reliable and interpretable reasoning processes. They often default to fast-thinking behavior by producing direct answers without explicit multi-step reasoning, limiting their effectiveness in complex legal scenarios that demand rigorous justification. To address this challenge, we propose Legal$\Delta$, a reinforcement learning framework designed to enhance legal reasoning through chain-of-thought guided information gain. During training, Legal$\Delta$ employs a dual-mode input setup-comprising direct answer and reasoning-augmented modes-and maximizes the information gain between them. This encourages the model to acquire meaningful reasoning patterns rather than generating superficial or redundant explanations. Legal$\Delta$ follows a two-stage approach: (1) distilling latent reasoning capabilities from a powerful Large Reasoning Model (LRM), DeepSeek-R1, and (2) refining reasoning quality via differential comparisons, combined with a multidimensional reward mechanism that assesses both structural coherence and legal-domain specificity. Experimental results on multiple legal reasoning tasks demonstrate that Legal$\Delta$ outperforms strong baselines in both accuracy and interpretability. It consistently produces more robust and trustworthy legal judgments without relying on labeled preference data. All code and data will be released at https://github.com/NEUIR/LegalDelta.

Method: Constructed from over 300,000 news articles (2019-2024) with 5,176 high-quality questions covering absolute, aggregate, and relative temporal types. Used multi-stage validation including rule-based, LLM-based, and human evaluation to ensure data quality.

Result: Created ChronoQA - a dynamic, reliable, and scalable benchmark dataset with comprehensive structural annotations that supports both single- and multi-document scenarios for temporal alignment and logical consistency evaluation.

Conclusion: ChronoQA serves as a robust benchmark for advancing time-sensitive retrieval-augmented question answering systems by enabling structured evaluation across a wide range of temporal tasks.

Abstract: We introduce ChronoQA, a large-scale benchmark dataset for Chinese question answering, specifically designed to evaluate temporal reasoning in Retrieval-Augmented Generation (RAG) systems. ChronoQA is constructed from over 300,000 news articles published between 2019 and 2024, and contains 5,176 high-quality questions covering absolute, aggregate, and relative temporal types with both explicit and implicit time expressions. The dataset supports both single- and multi-document scenarios, reflecting the real-world requirements for temporal alignment and logical consistency. ChronoQA features comprehensive structural annotations and has undergone multi-stage validation, including rule-based, LLM-based, and human evaluation, to ensure data quality. By providing a dynamic, reliable, and scalable resource, ChronoQA enables structured evaluation across a wide range of temporal tasks, and serves as a robust benchmark for advancing time-sensitive retrieval-augmented question answering systems.

Method: Three-stage approach: 1) Construct specialized probation dataset with fact descriptions and probation legal elements (PLEs); 2) Design Multi-Task Dual-Theory Probation Prediction Model (MT-DT) grounded in legal logic and Dual-Track Theory of Punishment; 3) Experimental validation on probation dataset.

Result: MT-DT model outperforms baseline models, and legal logic analysis validates the effectiveness of the proposed approach.

Conclusion: Integrating legal logic into deep learning models provides a more effective framework for probation prediction that aligns with judicial decision-making principles, addressing the limitations of purely data-driven methods in judicial assistance systems.

Abstract: Probation is a crucial institution in modern criminal law, embodying the principles of fairness and justice while contributing to the harmonious development of society. Despite its importance, the current Intelligent Judicial Assistant System (IJAS) lacks dedicated methods for probation prediction, and research on the underlying factors influencing probation eligibility remains limited. In addition, probation eligibility requires a comprehensive analysis of both criminal circumstances and remorse. Much of the existing research in IJAS relies primarily on data-driven methodologies, which often overlooks the legal logic underpinning judicial decision-making. To address this gap, we propose a novel approach that integrates legal logic into deep learning models for probation prediction, implemented in three distinct stages. First, we construct a specialized probation dataset that includes fact descriptions and probation legal elements (PLEs). Second, we design a distinct probation prediction model named the Multi-Task Dual-Theory Probation Prediction Model (MT-DT), which is grounded in the legal logic of probation and the \textit{Dual-Track Theory of Punishment}. Finally, our experiments on the probation dataset demonstrate that the MT-DT model outperforms baseline models, and an analysis of the underlying legal logic further validates the effectiveness of the proposed approach.

Method: Extended conflict-aware MAQA setting requiring models to identify all valid answers and detect conflicting pairs. Used cost-effective methodology leveraging fact-checking datasets to construct NATCONFQA benchmark with detailed conflict labels.

Result: Evaluation of eight high-end LLMs revealed fragility in handling various conflict types and flawed resolution strategies, demonstrating significant challenges in realistic MAQA scenarios.

Conclusion: LLMs exhibit substantial weaknesses in conflict-aware multi-answer question answering, highlighting the need for improved benchmarks and model capabilities in handling realistic conflicting information scenarios.

Abstract: Large Language Models (LLMs) have demonstrated strong performance in question answering (QA) tasks. However, Multi-Answer Question Answering (MAQA), where a question may have several valid answers, remains challenging. Traditional QA settings often assume consistency across evidences, but MAQA can involve conflicting answers. Constructing datasets that reflect such conflicts is costly and labor-intensive, while existing benchmarks often rely on synthetic data, restrict the task to yes/no questions, or apply unverified automated annotation. To advance research in this area, we extend the conflict-aware MAQA setting to require models not only to identify all valid answers, but also to detect specific conflicting answer pairs, if any. To support this task, we introduce a novel cost-effective methodology for leveraging fact-checking datasets to construct NATCONFQA, a new benchmark for realistic, conflict-aware MAQA, enriched with detailed conflict labels, for all answer pairs. We evaluate eight high-end LLMs on NATCONFQA, revealing their fragility in handling various types of conflicts and the flawed strategies they employ to resolve them.

Method: Uses Multi-Route Process Verification to contrast positive/negative reasoning paths, Enabling Autonomy via Asymptotic Induction to fade external signals gradually, and guided chain-of-thought distillation to encode domain knowledge.

Result: Extensive experiments show ReaLM significantly improves SLM performance across reasoning capability, autonomy, and generalization in both vertical and general reasoning tasks.

Conclusion: ReaLM provides a comprehensive framework for robust and self-sufficient reasoning in small language models, addressing key limitations without sacrificing important performance aspects.

Abstract: Small Language Models (SLMs) are a cost-effective alternative to Large Language Models (LLMs), but often struggle with complex reasoning due to their limited capacity and a tendency to produce mistakes or inconsistent answers during multi-step reasoning. Existing efforts have improved SLM performance, but typically at the cost of one or more of three key aspects: (1) reasoning capability, due to biased supervision that filters out negative reasoning paths and limits learning from errors; (2) autonomy, due to over-reliance on externally generated reasoning signals; and (3) generalization, which suffers when models overfit to teacher-specific patterns. In this paper, we introduce ReaLM, a reinforcement learning framework for robust and self-sufficient reasoning in vertical domains. To enhance reasoning capability, we propose Multi-Route Process Verification (MRPV), which contrasts both positive and negative reasoning paths to extract decisive patterns. To reduce reliance on external guidance and improve autonomy, we introduce Enabling Autonomy via Asymptotic Induction (EAAI), a training strategy that gradually fades external signals. To improve generalization, we apply guided chain-of-thought distillation to encode domain-specific rules and expert knowledge into SLM parameters, making them part of what the model has learned. Extensive experiments on both vertical and general reasoning tasks demonstrate that ReaLM significantly improves SLM performance across aspects (1)-(3) above.

Method: Uses two specialized agents (Extractor and Constructor) powered by Qwen2.5-32B-Instruct to incrementally build KGs day-by-day from 10M+ PubMed abstracts (1975-2023), with sampling-based confidence scoring and temporal integration.

Result: Constructed KG with 156,275 entities and 2,971,384 relational triples; 90% accuracy validated by SOTA LLMs and domain experts; significant improvements in RAG performance across 7 medical QA benchmarks using 5 leading LLMs.

Conclusion: MedKGent successfully addresses temporal dynamics in medical knowledge, providing high-quality evolving KGs that enhance downstream applications like drug repurposing through confidence-aware causal inference.

Abstract: The rapid expansion of medical literature presents growing challenges for structuring and integrating domain knowledge at scale. Knowledge Graphs (KGs) offer a promising solution by enabling efficient retrieval, automated reasoning, and knowledge discovery. However, current KG construction methods often rely on supervised pipelines with limited generalizability or naively aggregate outputs from Large Language Models (LLMs), treating biomedical corpora as static and ignoring the temporal dynamics and contextual uncertainty of evolving knowledge. To address these limitations, we introduce MedKGent, a LLM agent framework for constructing temporally evolving medical KGs. Leveraging over 10 million PubMed abstracts published between 1975 and 2023, we simulate the emergence of biomedical knowledge via a fine-grained daily time series. MedKGent incrementally builds the KG in a day-by-day manner using two specialized agents powered by the Qwen2.5-32B-Instruct model. The Extractor Agent identifies knowledge triples and assigns confidence scores via sampling-based estimation, which are used to filter low-confidence extractions and inform downstream processing. The Constructor Agent incrementally integrates the retained triples into a temporally evolving graph, guided by confidence scores and timestamps to reinforce recurring knowledge and resolve conflicts. The resulting KG contains 156,275 entities and 2,971,384 relational triples. Quality assessments by two SOTA LLMs and three domain experts demonstrate an accuracy approaching 90%, with strong inter-rater agreement. To evaluate downstream utility, we conduct RAG across seven medical question answering benchmarks using five leading LLMs, consistently observing significant improvements over non-augmented baselines. Case studies further demonstrate the KG’s value in literature-based drug repurposing via confidence-aware causal inference.

Method: Developed comprehensive PASC lexicon with specialists, created hybrid NLP pipeline integrating rule-based named entity recognition with BERT-based assertion detection modules, validated on 160 notes from 11 health systems.

Result: Achieved F1 score of 0.82 (internal) and 0.76 (external validation), processed notes in 2.448±0.812 seconds, strong Spearman correlations (ρ>0.83 positive, ρ>0.72 negative mentions, p<0.0001).

Conclusion: The hybrid NLP pipeline demonstrates effectiveness and efficiency for PASC symptom extraction, showing potential to improve PASC diagnosis through automated clinical note analysis.

Abstract: Accurately and efficiently diagnosing Post-Acute Sequelae of COVID-19 (PASC) remains challenging due to its myriad symptoms that evolve over long- and variable-time intervals. To address this issue, we developed a hybrid natural language processing pipeline that integrates rule-based named entity recognition with BERT-based assertion detection modules for PASC-symptom extraction and assertion detection from clinical notes. We developed a comprehensive PASC lexicon with clinical specialists. From 11 health systems of the RECOVER initiative network across the U.S., we curated 160 intake progress notes for model development and evaluation, and collected 47,654 progress notes for a population-level prevalence study. We achieved an average F1 score of 0.82 in one-site internal validation and 0.76 in 10-site external validation for assertion detection. Our pipeline processed each note at $2.448\pm 0.812$ seconds on average. Spearman correlation tests showed $\rho >0.83$ for positive mentions and $\rho >0.72$ for negative ones, both with $P <0.0001$. These demonstrate the effectiveness and efficiency of our models and their potential for improving PASC diagnosis.

Method: Designs a criterion to exclusively group retrieval heads or streaming heads in unique layers rather than mixing them, eliminating the extra latency from decomposed attention computations while maintaining KV cache optimization.

Result: The method reduces latency significantly while incurring only negligible performance degradation, making it competitive with existing baselines.

Conclusion: ZigzagAttention provides an effective approach to optimize KV cache memory footprint in LLMs by strategically grouping attention head types, achieving better latency-performance tradeoffs for long-context handling.

Abstract: With the rapid development of large language models (LLMs), handling long context has become one of the vital abilities in LLMs. Such long-context ability is accompanied by difficulties in deployment, especially due to the increased consumption of KV cache. There is certain work aiming to optimize the memory footprint of KV cache, inspired by the observation that attention heads can be categorized into retrieval heads that are of great significance and streaming heads that are of less significance. Typically, identifying the streaming heads and and waiving the KV cache in the streaming heads would largely reduce the overhead without hurting the performance that much. However, since employing both retrieval and streaming heads in one layer decomposes one large round of attention computation into two small ones, it may unexpectedly bring extra latency on accessing and indexing tensors. Based on this intuition, we impose an important improvement to the identification process of retrieval and streaming heads, in which we design a criterion that enforces exclusively retrieval or streaming heads gathered in one unique layer. In this way, we further eliminate the extra latency and only incur negligible performance degradation. Our method named \textsc{ZigzagAttention} is competitive among considered baselines owing to reduced latency and comparable performance.

Method: Created a Cultural Probe Dataset of 200 prompts targeting Individualism-Collectivism and Power Distance dimensions. Used standardized zero-shot prompts to compare GPT-4 (Western) and ERNIE Bot (Eastern), with human annotation and statistical analysis including Cultural Alignment Index against Hofstede’s national scores.

Result: Significant divergence found: GPT-4 shows individualistic/low-power-distance tendencies (IDV ~1.21, PDI ~-1.05), ERNIE Bot shows collectivistic/high-power-distance tendencies (IDV ~-0.89, PDI ~0.76). GPT-4 aligns with USA values (IDV CAI ~0.91, PDI CAI ~0.88), ERNIE Bot aligns with China (IDV CAI ~0.85, PDI CAI ~0.81). All differences statistically significant (p < 0.001).

Conclusion: LLMs function as statistical mirrors of their cultural training corpora, demonstrating the need for culturally aware evaluation and deployment to prevent algorithmic cultural hegemony in global AI systems.

Abstract: Large language models (LLMs) are deployed globally, yet their underlying cultural and ethical assumptions remain underexplored. We propose the notion of a “cultural gene” – a systematic value orientation that LLMs inherit from their training corpora – and introduce a Cultural Probe Dataset (CPD) of 200 prompts targeting two classic cross-cultural dimensions: Individualism-Collectivism (IDV) and Power Distance (PDI). Using standardized zero-shot prompts, we compare a Western-centric model (GPT-4) and an Eastern-centric model (ERNIE Bot). Human annotation shows significant and consistent divergence across both dimensions. GPT-4 exhibits individualistic and low-power-distance tendencies (IDV score approx 1.21; PDI score approx -1.05), while ERNIE Bot shows collectivistic and higher-power-distance tendencies (IDV approx -0.89; PDI approx 0.76); differences are statistically significant (p < 0.001). We further compute a Cultural Alignment Index (CAI) against Hofstede’s national scores and find GPT-4 aligns more closely with the USA (e.g., IDV CAI approx 0.91; PDI CAI approx 0.88) whereas ERNIE Bot aligns more closely with China (IDV CAI approx 0.85; PDI CAI approx 0.81). Qualitative analyses of dilemma resolution and authority-related judgments illustrate how these orientations surface in reasoning. Our results support the view that LLMs function as statistical mirrors of their cultural corpora and motivate culturally aware evaluation and deployment to avoid algorithmic cultural hegemony.

Method: Used dynamics forecasting tasks in physical systems to evaluate ICL, analyzed model activations with sparse autoencoders (SAEs) to identify encoded physical concepts.

Result: Performance improves with longer contexts, and SAE features correlate with key physical variables like energy, showing meaningful physical concepts are encoded during ICL.

Conclusion: Physics tasks provide valuable insights into LLM reasoning, demonstrating that meaningful physical representations emerge during in-context learning, broadening understanding of LLM capabilities.

Abstract: Large language models (LLMs) exhibit impressive in-context learning (ICL) abilities, enabling them to solve wide range of tasks via textual prompts alone. As these capabilities advance, the range of applicable domains continues to expand significantly. However, identifying the precise mechanisms or internal structures within LLMs that allow successful ICL across diverse, distinct classes of tasks remains elusive. Physics-based tasks offer a promising testbed for probing this challenge. Unlike synthetic sequences such as basic arithmetic or symbolic equations, physical systems provide experimentally controllable, real-world data based on structured dynamics grounded in fundamental principles. This makes them particularly suitable for studying the emergent reasoning behaviors of LLMs in a realistic yet tractable setting. Here, we mechanistically investigate the ICL ability of LLMs, especially focusing on their ability to reason about physics. Using a dynamics forecasting task in physical systems as a proxy, we evaluate whether LLMs can learn physics in context. We first show that the performance of dynamics forecasting in context improves with longer input contexts. To uncover how such capability emerges in LLMs, we analyze the model’s residual stream activations using sparse autoencoders (SAEs). Our experiments reveal that the features captured by SAEs correlate with key physical variables, such as energy. These findings demonstrate that meaningful physical concepts are encoded within LLMs during in-context learning. In sum, our work provides a novel case study that broadens our understanding of how LLMs learn in context.

Method: Proposes M3PO method that selects learning-valuable preference pairs using M3P-Score combining Multimodal Alignment Score (external quality) and Self-Consistency/Confidence (internal belief), then applies Direct Preference Optimization with LoRA on base LVLMs.

Result: M3PO consistently outperforms baselines (SFT, simulated RLHF, vanilla DPO, RM-DPO) across multiple multimodal benchmarks including MME-Bench, POPE, IFT, and Human Preference Score.

Conclusion: M3PO provides a data-efficient approach for enhancing LVLM capabilities in visual instruction following by intelligently selecting challenging preference pairs from model-generated candidates, eliminating the need for costly human annotation.

Abstract: Large Vision-Language Models (LVLMs) hold immense potential for complex multimodal instruction following, yet their development is often hindered by the high cost and inconsistency of human annotation required for effective fine-tuning and preference alignment. Traditional supervised fine-tuning (SFT) and existing preference optimization methods like RLHF and DPO frequently struggle to efficiently leverage the model’s own generation space to identify highly informative “hard negative” samples. To address these challenges, we propose Multimodal-Model-Guided Preference Optimization (M3PO), a novel and data-efficient method designed to enhance LVLMs’ capabilities in visual instruction following. M3PO intelligently selects the most “learning-valuable” preference sample pairs from a diverse pool of LVLM-generated candidates. This selection is driven by a sophisticated mechanism that integrates two crucial signals: a Multimodal Alignment Score (MAS) to assess external quality and the model’s Self-Consistency / Confidence (log-probability) to gauge internal belief. These are combined into a novel M3P-Score, which specifically identifies preferred responses and challenging dispreferred responses that the model might confidently generate despite being incorrect. These high-quality preference pairs are then used for efficient Direct Preference Optimization (DPO) fine-tuning on base LVLMs like LLaVA-1.5 (7B/13B) using LoRA. Our extensive experiments demonstrate that M3PO consistently outperforms strong baselines, including SFT, simulated RLHF, vanilla DPO, and RM-DPO, across a comprehensive suite of multimodal instruction following benchmarks (MME-Bench, POPE, IFT, Human Pref. Score).

Method: Created LoraxBench covering 20 Indonesian languages with 6 tasks: reading comprehension, open-domain QA, language inference, causal reasoning, translation, and cultural QA. Included two formality registers for three languages. Evaluated diverse multilingual and region-focused LLMs.

Result: Benchmark proved challenging with visible performance discrepancy between Indonesian and other low-resource languages. No clear advantage for region-specific models over general multilingual models. Register changes (especially high-politeness forms like Krama Javanese) significantly affected model performance.

Conclusion: LoraxBench highlights the need for better NLP support for Indonesia’s linguistic diversity, showing current models struggle with low-resource languages and formal registers not commonly found in social media data.

Abstract: As one of the world’s most populous countries, with 700 languages spoken, Indonesia is behind in terms of NLP progress. We introduce LoraxBench, a benchmark that focuses on low-resource languages of Indonesia and covers 6 diverse tasks: reading comprehension, open-domain QA, language inference, causal reasoning, translation, and cultural QA. Our dataset covers 20 languages, with the addition of two formality registers for three languages. We evaluate a diverse set of multilingual and region-focused LLMs and found that this benchmark is challenging. We note a visible discrepancy between performance in Indonesian and other languages, especially the low-resource ones. There is no clear lead when using a region-specific model as opposed to the general multilingual model. Lastly, we show that a change in register affects model performance, especially with registers not commonly found in social media, such as high-level politeness `Krama’ Javanese.

Method: Evaluated GPT-OSS 20B and 120B models against six contemporary open-source LLMs (14.7B-235B parameters) across ten benchmarks covering general knowledge, math reasoning, code generation, multilingual understanding, and conversational ability. Used standardized inference settings with statistical validation via McNemar’s test and effect size analysis.

Result: GPT-OSS-20B consistently outperformed GPT-OSS-120B on several benchmarks (including HumanEval and MMLU) despite requiring substantially less memory and energy. Both models showed mid-tier overall performance with strengths in code generation and weaknesses in multilingual tasks.

Conclusion: Scaling sparse architectures may not yield proportional performance gains, highlighting the need for better optimization strategies and more efficient model selection for open-source deployments.

Abstract: In August 2025, OpenAI released GPT-OSS models, its first open weight large language models since GPT-2 in 2019, comprising two mixture of experts architectures with 120B and 20B parameters. We evaluated both variants against six contemporary open source large language models ranging from 14.7B to 235B parameters, representing both dense and sparse designs, across ten benchmarks covering general knowledge, mathematical reasoning, code generation, multilingual understanding, and conversational ability. All models were tested in unquantised form under standardised inference settings, with statistical validation using McNemars test and effect size analysis. Results show that gpt-oss-20B consistently outperforms gpt-oss-120B on several benchmarks, such as HumanEval and MMLU, despite requiring substantially less memory and energy per response. Both models demonstrate mid-tier overall performance within the current open source landscape, with relative strength in code generation and notable weaknesses in multilingual tasks. These findings provide empirical evidence that scaling in sparse architectures may not yield proportional performance gains, underscoring the need for further investigation into optimisation strategies and informing more efficient model selection for future open source deployments.

Method: Trained RoBERTa and GPT-2 on perturbed datasets where syntactic information was ablated, comparing effects on verb representations when syntax vs co-occurrence information was removed.

Result: Models’ verb representation degraded more when syntactic cues were removed than when co-occurrence information was removed. Mental verbs were more negatively impacted than physical verbs. Noun representations were more affected by co-occurrence distortion than syntax distortion.

Conclusion: Results reinforce the important role of syntactic bootstrapping in verb learning and demonstrate the viability of testing developmental hypotheses at scale through manipulating LLM learning environments.

Abstract: Syntactic bootstrapping (Gleitman, 1990) is the hypothesis that children use the syntactic environments in which a verb occurs to learn its meaning. In this paper, we examine whether large language models exhibit a similar behavior. We do this by training RoBERTa and GPT-2 on perturbed datasets where syntactic information is ablated. Our results show that models’ verb representation degrades more when syntactic cues are removed than when co-occurrence information is removed. Furthermore, the representation of mental verbs, for which syntactic bootstrapping has been shown to be particularly crucial in human verb learning, is more negatively impacted in such training regimes than physical verbs. In contrast, models’ representation of nouns is affected more when co-occurrences are distorted than when syntax is distorted. In addition to reinforcing the important role of syntactic bootstrapping in verb learning, our results demonstrated the viability of testing developmental hypotheses on a larger scale through manipulating the learning environments of large language models.

Method: Supervised fine-tuning framework that trains LLMs to construct variable-level directed acyclic graphs (DAGs) and perform reasoning over them, using a dataset of 25,368 samples with input questions, explicit causal DAGs, graph-based reasoning traces, and validated answers.

Result: Achieves 95.33% accuracy on CLADDER (surpassing human performance of 94.8% for the first time) and reduces hallucination on HaluEval with 10% improvements across four LLMs and eight tasks.

Conclusion: Explicit causal structure modeling in LLMs can effectively mitigate logical inconsistencies in LLM outputs, demonstrating that causal reasoning capabilities are crucial for reducing hallucinations.

Abstract: Large language models (LLMs) exhibit logically inconsistent hallucinations that appear coherent yet violate reasoning principles, with recent research suggesting an inverse relationship between causal reasoning capabilities and such hallucinations. However, existing reasoning approaches in LLMs, such as Chain-of-Thought (CoT) and its graph-based variants, operate at the linguistic token level rather than modeling the underlying causal relationships between variables, lacking the ability to represent conditional independencies or satisfy causal identification assumptions. To bridge this gap, we introduce causal-DAG construction and reasoning (CDCR-SFT), a supervised fine-tuning framework that trains LLMs to explicitly construct variable-level directed acyclic graph (DAG) and then perform reasoning over it. Moreover, we present a dataset comprising 25,368 samples (CausalDR), where each sample includes an input question, explicit causal DAG, graph-based reasoning trace, and validated answer. Experiments on four LLMs across eight tasks show that CDCR-SFT improves the causal reasoning capability with the state-of-the-art 95.33% accuracy on CLADDER (surpassing human performance of 94.8% for the first time) and reduces the hallucination on HaluEval with 10% improvements. It demonstrates that explicit causal structure modeling in LLMs can effectively mitigate logical inconsistencies in LLM outputs. Code is available at https://github.com/MrLYG/CDCR-SFT.

Method: CorrSteer selects features by correlating sample correctness with SAE activations from generated tokens at inference time, using only inference-time activations to avoid spurious correlations and obtaining steering coefficients from average activations.

Result: Improved task performance on QA, bias mitigation, jailbreaking prevention, and reasoning benchmarks on Gemma 2 2B and LLaMA 3.1 8B, with +4.1% MMLU improvement and +22.9% HarmBench improvement using only 4000 samples.

Conclusion: Correlation-based selection is an effective and scalable approach for automated SAE steering across language model applications, with selected features showing semantically meaningful patterns aligned with task requirements.

Abstract: Sparse Autoencoders (SAEs) can extract interpretable features from large language models (LLMs) without supervision. However, their effectiveness in downstream steering tasks is limited by the requirement for contrastive datasets or large activation storage. To address these limitations, we propose CorrSteer, which selects features by correlating sample correctness with SAE activations from generated tokens at inference time. This approach uses only inference-time activations to extract more relevant features, thereby avoiding spurious correlations. It also obtains steering coefficients from average activations, automating the entire pipeline. Our method shows improved task performance on QA, bias mitigation, jailbreaking prevention, and reasoning benchmarks on Gemma 2 2B and LLaMA 3.1 8B, notably achieving a +4.1% improvement in MMLU performance and a +22.9% improvement in HarmBench with only 4000 samples. Selected features demonstrate semantically meaningful patterns aligned with each task’s requirements, revealing the underlying capabilities that drive performance. Our work establishes correlationbased selection as an effective and scalable approach for automated SAE steering across language model applications.

Method: Hybrid agentic memory architecture that enriches vector-based storage with explicit linguistic cues through dependency parsing, discourse relation tagging, and coreference resolution to create structured memory entries.

Result: 18% improvement in factual recall and discourse coherence over strong RAG baselines on adapted long-term dialogue datasets, with ablation studies and human evaluations confirming robustness.

Conclusion: Semantic Anchoring effectively bridges the gap between semantic similarity and linguistic structure, significantly enhancing LLM memory performance in multi-session interactions.

Abstract: Large Language Models (LLMs) have demonstrated impressive fluency and task competence in conversational settings. However, their effectiveness in multi-session and long-term interactions is hindered by limited memory persistence. Typical retrieval-augmented generation (RAG) systems store dialogue history as dense vectors, which capture semantic similarity but neglect finer linguistic structures such as syntactic dependencies, discourse relations, and coreference links. We propose Semantic Anchoring, a hybrid agentic memory architecture that enriches vector-based storage with explicit linguistic cues to improve recall of nuanced, context-rich exchanges. Our approach combines dependency parsing, discourse relation tagging, and coreference resolution to create structured memory entries. Experiments on adapted long-term dialogue datasets show that semantic anchoring improves factual recall and discourse coherence by up to 18% over strong RAG baselines. We further conduct ablation studies, human evaluations, and error analysis to assess robustness and interpretability.

Method: Embeds and clusters incoming queries, then routes each to the most suitable model based on a performance-efficiency score, ensembling LLMs of varying capacities and efficiencies.

Result: Surpasses strongest single model (GPT-5-medium) by +7% average accuracy, matches strongest model accuracy at 27% lower cost, achieves 90% performance at 63% lower cost, and establishes Pareto frontier dominance.

Conclusion: Avengers-Pro provides an effective framework for optimizing LLM performance-efficiency tradeoffs through intelligent test-time routing, delivering superior results across multiple benchmarks and models.

Abstract: Balancing performance and efficiency is a central challenge in large language model (LLM) advancement. GPT-5 addresses this with test-time routing, dynamically assigning queries to either an efficient or a high-capacity model during inference. In this work, we present Avengers-Pro, a test-time routing framework that ensembles LLMs of varying capacities and efficiencies, providing a unified solution for all performance-efficiency tradeoffs. The Avengers-Pro embeds and clusters incoming queries, then routes each to the most suitable model based on a performance-efficiency score. Across 6 challenging benchmarks and 8 leading models – including GPT-5-medium, Gemini-2.5-pro, and Claude-opus-4.1 – Avengers-Pro achieves state-of-the-art results: by varying a performance-efficiency trade-off parameter, it can surpass the strongest single model (GPT-5-medium) by +7% in average accuracy. Moreover, it can match the average accuracy of the strongest single model at 27% lower cost, and reach ~90% of that performance at 63% lower cost. Last but not least, it achieves a Pareto frontier, consistently yielding the highest accuracy for any given cost, and the lowest cost for any given accuracy, among all single models. Code is available at https://github.com/ZhangYiqun018/AvengersPro.

Method: Proposes Linguistic Fingerprints Extraction (LIFE) that reconstructs word-level probability distributions to find discriminative patterns. Uses distributional divergence analysis to uncover statistically distinct probability shifts between real and fake news, and leverages key-fragment techniques to amplify subtle linguistic differences.

Result: LIFE achieves state-of-the-art performance in detecting LLM-generated fake news and maintains high performance in human-written fake news detection.

Conclusion: The method successfully identifies prompt-induced linguistic fingerprints that serve as reliable indicators for fake news detection, providing an effective solution to the growing threat of AI-generated misinformation.

Abstract: With the rapid development of large language models, the generation of fake news has become increasingly effortless, posing a growing societal threat and underscoring the urgent need for reliable detection methods. Early efforts to identify LLM-generated fake news have predominantly focused on the textual content itself; however, because much of that content may appear coherent and factually consistent, the subtle traces of falsification are often difficult to uncover. Through distributional divergence analysis, we uncover prompt-induced linguistic fingerprints: statistically distinct probability shifts between LLM-generated real and fake news when maliciously prompted. Based on this insight, we propose a novel method named Linguistic Fingerprints Extraction (LIFE). By reconstructing word-level probability distributions, LIFE can find discriminative patterns that facilitate the detection of LLM-generated fake news. To further amplify these fingerprint patterns, we also leverage key-fragment techniques that accentuate subtle linguistic differences, thereby improving detection reliability. Our experiments show that LIFE achieves state-of-the-art performance in LLM-generated fake news and maintains high performance in human-written fake news. The code and data are available at https://anonymous.4open.science/r/LIFE-E86A.

Method: Fine-tuning LLMs on synthetic code-switched text generated using controlled language-mixing methods, creating a new dataset from CommonSenseQA with three distinct language ratio configurations.

Result: Substantial improvements in low-resource language model performance while preserving or enhancing performance in high-resource languages.

Conclusion: Synthetic code-switched fine-tuning effectively bridges the performance gap between high and low-resource languages in LLM common sense reasoning tasks.

Abstract: Cutting-edge LLMs have emerged as powerful tools for multilingual communication and understanding. However, LLMs perform worse in Common Sense Reasoning (CSR) tasks when prompted in low-resource languages (LRLs) like Hindi or Swahili compared to high-resource languages (HRLs) like English. Equalizing this inconsistent access to quality LLM outputs is crucial to ensure fairness for speakers of LRLs and across diverse linguistic communities. In this paper, we propose an approach to bridge this gap in LLM performance. Our approach involves fine-tuning an LLM on synthetic code-switched text generated using controlled language-mixing methods. We empirically demonstrate that fine-tuning LLMs on synthetic code-switched datasets leads to substantial improvements in LRL model performance while preserving or enhancing performance in HRLs. Additionally, we present a new dataset of synthetic code-switched text derived from the CommonSenseQA dataset, featuring three distinct language ratio configurations.

Method: Framed as regression problem (0-100 scores), evaluated multiple prompting strategies: zero-shot, fixed-context few-shot, and retrieval-based prompting using BERT embeddings. Introduced “Misery Game Show” - a gamified framework with structured rounds for ordinal comparison, binary classification, scalar estimation, and feedback-driven reasoning.

Result: Few-shot approaches consistently outperformed zero-shot baselines, demonstrating the value of contextual examples in affective prediction. The gamified evaluation showed LLMs’ potential in dynamic emotional reasoning tasks.

Conclusion: LLMs show promise in predicting human misery scores, with few-shot learning being particularly effective. The gamified framework provides a more comprehensive evaluation of LLM capabilities in emotional reasoning beyond traditional regression metrics.

Abstract: This study investigates the use of Large Language Models (LLMs) for predicting human-perceived misery scores from natural language descriptions of real-world scenarios. The task is framed as a regression problem, where the model assigns a scalar value from 0 to 100 to each input statement. We evaluate multiple prompting strategies, including zero-shot, fixed-context few-shot, and retrieval-based prompting using BERT sentence embeddings. Few-shot approaches consistently outperform zero-shot baselines, underscoring the value of contextual examples in affective prediction. To move beyond static evaluation, we introduce the “Misery Game Show”, a novel gamified framework inspired by a television format. It tests LLMs through structured rounds involving ordinal comparison, binary classification, scalar estimation, and feedback-driven reasoning. This setup enables us to assess not only predictive accuracy but also the model’s ability to adapt based on corrective feedback. The gamified evaluation highlights the broader potential of LLMs in dynamic emotional reasoning tasks beyond standard regression. Code and data link: https://github.com/abhi1nandy2/Misery_Data_Exps_GitHub

Method: Three-stage framework: 1) Coarse-grained initialization builds structurally complete dialogue skeleton, 2) Iterative refinement adds realistic complexities via mask-and-fill operations, 3) Offline verification ensures correctness via rule- and model-based checks.

Result: ToolACE-MT enables efficient, effective and generalizable agentic data generation, offering a new paradigm for high-quality data construction in tool-augmented LLM scenarios.

Conclusion: The proposed non-autoregressive iterative generation framework provides a more efficient alternative to costly autoregressive methods for generating high-quality multi-turn agentic dialogues.

Abstract: Agentic task-solving with Large Language Models (LLMs) requires multi-turn, multi-step interactions, often involving complex function calls and dynamic user-agent exchanges. Existing simulation-based data generation methods for such scenarios rely heavily on costly autoregressive interactions between multiple LLM agents, thereby limiting real-world performance of agentic tasks. In this paper, we propose a novel Non-Autoregressive Iterative Generation framework, called ToolACE-MT, for constructing high-quality multi-turn agentic dialogues. ToolACE-MT generates full conversational trajectories through three stages: coarse-grained initialization, iterative refinement, and offline verification. The initialization phase builds a structurally complete yet semantically coarse dialogue skeleton; the iterative refinement phase introduces realistic complexities and continued refinement via mask-and-fill operations; and the offline verification phase ensures correctness and coherence via rule- and model-based checks. Experiments demonstrate that ToolACE-MT enables efficient, effective and generalizable agentic data generation, offering a new paradigm for high-quality data construction in tool-augmented LLM scenarios.

Method: Reverse-engineer over 120,000 design logics from existing questions using LLMs, then match these logics with disciplinary source materials (book and web corpora) to synthesize challenging reasoning questions.

Result: Created two large datasets: DLR-Book (3.04M questions) and DLR-Web (1.66M questions) spanning 75 disciplines. Questions show substantially greater difficulty and diversity than baseline datasets. SFT experiments show models trained on these datasets outperform existing datasets and even surpass official Qwen3 models’ multidisciplinary reasoning performance.

Conclusion: The DESIGNER pipeline successfully generates high-quality, challenging reasoning questions at scale, significantly advancing LLM reasoning capabilities across diverse disciplines through improved training data.

Abstract: Large language models (LLMs) have achieved remarkable success in many natural language tasks but still struggle with complex, multi-step reasoning, particularly across diverse disciplines. Existing reasoning datasets often either lack disciplinary breadth or the structural depth necessary to elicit robust reasoning behaviors. We propose DESIGNER: a DESIGN-logic-guidEd Reasoning data synthesis pipeline that leverages naturally available, extensive raw documents (book corpus and web corpus) to generate multidisciplinary challenging questions. A core innovation of our approach is the introduction of a Design Logic concept, which mimics the question-creation process of human educators. We use LLMs to reverse-engineer and abstract over 120,000 design logics from existing questions across various disciplines. By matching these design logics with disciplinary source materials, we are able to create reasoning questions that far surpass the difficulty and diversity of existing datasets. Based on this pipeline, we synthesized two large-scale reasoning datasets that span 75 disciplines: Design-Logic-Reasoning-Book (DLR-Book), containing 3.04 million challenging questions synthesized from the book corpus, and Design-Logic-Reasoning-Web (DLR-Web), with 1.66 million challenging questions from the web corpus. Our data analysis demonstrates that the questions synthesized by our method exhibit substantially greater difficulty and diversity than those in the baseline datasets. We validate the effectiveness of these datasets by conducting SFT experiments on the Qwen3-8B-Base and Qwen3-4B-Base models. The results show that our dataset significantly outperforms existing multidisciplinary datasets of the same volume. Training with the full datasets further enables the models to surpass the multidisciplinary reasoning performance of the official Qwen3-8B and Qwen3-4B models.

Method: Created LinguaSafe dataset with 45k entries in 12 languages using translated, transcreated, and natively-sourced data, featuring multidimensional evaluation framework with direct/indirect safety assessments and oversensitivity evaluations.

Result: Safety and helpfulness evaluations vary significantly across different domains and languages, even among languages with similar resource levels, highlighting the importance of thorough multilingual safety assessment.

Conclusion: LinguaSafe provides comprehensive metrics for in-depth safety evaluation and underscores the critical need for balanced multilingual safety alignment in LLMs, with dataset and code released publicly to advance research.

Abstract: The widespread adoption and increasing prominence of large language models (LLMs) in global technologies necessitate a rigorous focus on ensuring their safety across a diverse range of linguistic and cultural contexts. The lack of a comprehensive evaluation and diverse data in existing multilingual safety evaluations for LLMs limits their effectiveness, hindering the development of robust multilingual safety alignment. To address this critical gap, we introduce LinguaSafe, a comprehensive multilingual safety benchmark crafted with meticulous attention to linguistic authenticity. The LinguaSafe dataset comprises 45k entries in 12 languages, ranging from Hungarian to Malay. Curated using a combination of translated, transcreated, and natively-sourced data, our dataset addresses the critical need for multilingual safety evaluations of LLMs, filling the void in the safety evaluation of LLMs across diverse under-represented languages from Hungarian to Malay. LinguaSafe presents a multidimensional and fine-grained evaluation framework, with direct and indirect safety assessments, including further evaluations for oversensitivity. The results of safety and helpfulness evaluations vary significantly across different domains and different languages, even in languages with similar resource levels. Our benchmark provides a comprehensive suite of metrics for in-depth safety evaluation, underscoring the critical importance of thoroughly assessing multilingual safety in LLMs to achieve more balanced safety alignment. Our dataset and code are released to the public to facilitate further research in the field of multilingual LLM safety.

Method: CRED-SQL uses cluster-based large-scale schema retrieval to identify relevant tables/columns, then introduces Execution Description Language (EDL) as intermediate representation, decomposing the task into Text-to-EDL and EDL-to-SQL stages.

Result: Achieves state-of-the-art performance on SpiderUnion and BirdUnion benchmarks, demonstrating effectiveness and scalability for large-scale cross-domain databases.

Conclusion: The framework successfully bridges the semantic gap in Text-to-SQL systems through innovative retrieval and intermediate representation techniques, enabling more accurate SQL generation for complex databases.

Abstract: Recent advances in large language models (LLMs) have significantly improved the accuracy of Text-to-SQL systems. However, a critical challenge remains: the semantic mismatch between natural language questions (NLQs) and their corresponding SQL queries. This issue is exacerbated in large-scale databases, where semantically similar attributes hinder schema linking and semantic drift during SQL generation, ultimately reducing model accuracy. To address these challenges, we introduce CRED-SQL, a framework designed for large-scale databases that integrates Cluster Retrieval and Execution Description. CRED-SQL first performs cluster-based large-scale schema retrieval to pinpoint the tables and columns most relevant to a given NLQ, alleviating schema mismatch. It then introduces an intermediate natural language representation-Execution Description Language (EDL)-to bridge the gap between NLQs and SQL. This reformulation decomposes the task into two stages: Text-to-EDL and EDL-to-SQL, leveraging LLMs’ strong general reasoning capabilities while reducing semantic deviation. Extensive experiments on two large-scale, cross-domain benchmarks-SpiderUnion and BirdUnion-demonstrate that CRED-SQL achieves new state-of-the-art (SOTA) performance, validating its effectiveness and scalability. Our code is available at https://github.com/smduan/CRED-SQL.git

Method: Two-step training: continual pre-training on parallel data followed by supervised fine-tuning on high-quality instructions. For WMT25, vocabulary adaptation and additional training phases were added. Used Minimum Bayes Risk Decoding and Tuned Re-ranking with COMET metrics.

Result: Developed SALAMANDRATA models in 2B and 7B parameter versions, with an additional SALAMANDRATA-V2 model, all publicly released on Hugging Face.

Conclusion: The SALAMANDRATA family provides improved translation capabilities for European languages and represents BSC’s submission to WMT25 General Machine Translation shared task with optimized performance across translation directions.

Abstract: In this paper, we present the SALAMANDRATA family of models, an improved iteration of SALAMANDRA LLMs (Gonzalez-Agirre et al., 2025) specifically trained to achieve strong performance in translation-related tasks for 38 European languages. SALAMANDRATA comes in two scales: 2B and 7B parameters. For both versions, we applied the same training recipe with a first step of continual pre-training on parallel data, and a second step of supervised fine-tuning on high-quality instructions. The BSC submission to the WMT25 General Machine Translation shared task is based on the 7B variant of SALAMANDRATA. We first adapted the model vocabulary to support the additional non-European languages included in the task. This was followed by a second phase of continual pre-training and supervised fine-tuning, carefully designed to optimize performance across all translation directions for this year’s shared task. For decoding, we employed two quality-aware strategies: Minimum Bayes Risk Decoding and Tuned Re-ranking using COMET and COMET-KIWI respectively. We publicly release both the 2B and 7B versions of SALAMANDRATA, along with the newer SALAMANDRATA-V2 model, on Hugging Face1

Method: Constructed MedAtlas with multimodal report repositories and text corpora. Developed HeteroRAG framework with Modality-specific CLIPs for report retrieval and Multi-corpora Query Generator for dynamic query construction. Used Heterogeneous Knowledge Preference Tuning for cross-modality and multi-source knowledge alignment.

Result: Achieved state-of-the-art performance across 12 datasets and 3 modalities in medical vision language benchmarks. Significantly improved factual accuracy and reliability of Med-LVLMs.

Conclusion: HeteroRAG framework successfully bridges the gap in heterogeneous knowledge retrieval for medical applications, providing a robust solution to enhance the factuality and reliability of medical vision-language models in clinical settings.

Abstract: Medical large vision-language Models (Med-LVLMs) have shown promise in clinical applications but suffer from factual inaccuracies and unreliable outputs, posing risks in real-world diagnostics. While retrieval-augmented generation has emerged as a potential solution, current medical multimodal RAG systems are unable to perform effective retrieval across heterogeneous sources. The irrelevance of retrieved reports affects the factuality of analysis, while insufficient knowledge affects the credibility of clinical decision-making. To bridge the gap, we construct MedAtlas, which includes extensive multimodal report repositories and diverse text corpora. Based on it, we present HeteroRAG, a novel framework that enhances Med-LVLMs through heterogeneous knowledge sources. The framework introduces Modality-specific CLIPs for effective report retrieval and a Multi-corpora Query Generator for dynamically constructing queries for diverse corpora. Incorporating knowledge from such multifaceted sources, Med-LVLM is then trained with Heterogeneous Knowledge Preference Tuning to achieve cross-modality and multi-source knowledge alignment. Extensive experiments across 12 datasets and 3 modalities demonstrate that the proposed HeteroRAG achieves state-of-the-art performance in most medical vision language benchmarks, significantly improving factual accuracy and reliability of Med-LVLMs.

Method: Proposes Atomic Thought paradigm that breaks reasoning into functional units supervised by Reasoning Reward Models (RRMs) providing Atomic Thought Rewards (ATR), integrated with a curriculum-inspired reward schedule that prioritizes process-level rewards early and transitions to outcome rewards.

Result: Experiments on seven benchmarks show consistent improvements over state-of-the-art methods, with advantages including scalable computation, better supervision bridging, and more interpretable human-like reasoning patterns.

Conclusion: Atom-Searcher effectively addresses RL training challenges in agentic deep research by providing fine-grained reward guidance through atomic reasoning decomposition, leading to superior performance and more interpretable reasoning processes.

Abstract: Large language models (LLMs) exhibit remarkable problem-solving abilities, but struggle with complex tasks due to static internal knowledge. Retrieval-Augmented Generation (RAG) enhances access to external information, yet remains limited in multi-hop reasoning and strategic search due to rigid workflows. Recent advancements in agentic deep research empower LLMs to autonomously reason, search, and synthesize information. However, current approaches relying on outcome-based reinforcement learning (RL) face critical issues such as conflicting gradients and reward sparsity, limiting performance gains and training efficiency. To address these, we first propose Atomic Thought, a novel LLM thinking paradigm that decomposes reasoning into fine-grained functional units. These units are supervised by Reasoning Reward Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained guidance. Building on this, we propose Atom-Searcher, a novel RL framework for agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher uses a curriculum-inspired reward schedule, prioritizing process-level ATR early and transitioning to outcome rewards, accelerating convergence on effective reasoning paths. Experiments on seven benchmarks show consistent improvements over the state-of-the-art. Key advantages include: (1) Atom-Searcher scales computation at test-time. (2) Atomic Thought provides supervision anchors for RRMs, bridging deep research tasks and RRMs. (3) Atom-Searcher exhibits more interpretable, human-like reasoning patterns.

Method: Developed an online variational probing framework that continuously estimates the subspace of the standard variety during fine-tuning, enabling projection-based decoupling. Used Arabic with 25 dialects as a case study due to rich parallel resources, treating dialectal MT as a controlled proxy for generative tasks.

Result: Intervention improved generation quality across 25 dialects by up to +4.9 chrF++ and +2.0 on average compared to standard fine-tuning, despite a measured tradeoff in standard-language performance. Provides causal evidence that subspace dominance restricts generative capacity.

Conclusion: Subspace dominance by high-resource varieties can restrict generative modeling of related varieties. The study unifies geometric and information-theoretic probing with subspace-level causal interventions, offering practical tools for controlling representational allocation in multilingual and multi-domain LLMs.

Abstract: Alignment with high-resource standard languages is often assumed to aid the modeling of related low-resource varieties. We challenge this assumption by demonstrating that excessive representational entanglement with a dominant variety, such as Modern Standard Arabic (MSA) in relation to Arabic dialects, can actively hinder generative modeling. We present the first comprehensive causal study of this phenomenon by analyzing and directly intervening in the internal representation geometry of large language models (LLMs). Our key contribution is an online variational probing framework that continuously estimates the subspace of the standard variety during fine-tuning, enabling projection-based decoupling from this space. While our study uses Arabic as a case due to its unusually rich parallel resources across 25 dialects, the broader motivation is methodological: dialectal MT serves as a controlled proxy for generative tasks where comparable multi-variety corpora are unavailable. Across 25 dialects, our intervention improves generation quality by up to +4.9 chrF++ and +2.0 on average compared to standard fine-tuning, despite a measured tradeoff in standard-language performance. These results provide causal evidence that subspace dominance by high-resource varieties can restrict generative capacity for related varieties. More generally, we unify geometric and information-theoretic probing with subspace-level causal interventions, offering practical tools for improving generative modeling in closely related language families and, more broadly, for controlling representational allocation in multilingual and multi-domain LLMs. Code will be released.

Method: Annotated the DinG corpus (French Catan game dialogues) using extended AMR framework, created annotation guidelines, trained and evaluated an AMR parser for assistance annotation.

Result: Created and published a French semantic dialogue corpus under CC-SA-BY license, developed an AMR parser that can provide initial annotations for human refinement.

Conclusion: This work contributes to French semantic resource development and provides tools for consistent annotation of spontaneous French dialogues using extended AMR framework.

Abstract: We present our work to build a French semantic corpus by annotating French dialogue in Abstract Meaning Representation (AMR). Specifically, we annotate the DinG corpus, consisting of transcripts of spontaneous French dialogues recorded during the board game Catan. As AMR has insufficient coverage of the dynamics of spontaneous speech, we extend the framework to better represent spontaneous speech and sentence structures specific to French. Additionally, to support consistent annotation, we provide an annotation guideline detailing these extensions. We publish our corpus under a free license (CC-SA-BY). We also train and evaluate an AMR parser on our data. This model can be used as an assistance annotation tool to provide initial annotations that can be refined by human annotators. Our work contributes to the development of semantic resources for French dialogue.

Method: Tested four classification models on parent-reply tweet pairs, comparing content-based and account-based features from context versus reply-only features.

Result: Incorporating contextual features led to substantial improvements over reply-only features, with content-based features contributing most to performance.

Conclusion: Contextual information from parent tweets significantly enhances abusive language detection, and combining multiple content-based features yields best results for realistic conversation settings.

Abstract: Abusive language detection has become an increasingly important task as a means to tackle this type of harmful content in social media. There has been a substantial body of research developing models for determining if a social media post is abusive or not; however, this research has primarily focused on exploiting social media posts individually, overlooking additional context that can be derived from surrounding posts. In this study, we look at conversational exchanges, where a user replies to an earlier post by another user (the parent tweet). We ask: does leveraging context from the parent tweet help determine if a reply post is abusive or not, and what are the features that contribute the most? We study a range of content-based and account-based features derived from the context, and compare this to the more widely studied approach of only looking at the features from the reply tweet. For a more generalizable study, we test four different classification models on a dataset made of conversational exchanges (parent-reply tweet pairs) with replies labeled as abusive or not. Our experiments show that incorporating contextual features leads to substantial improvements compared to the use of features derived from the reply tweet only, confirming the importance of leveraging context. We observe that, among the features under study, it is especially the content-based features (what is being posted) that contribute to the classification performance rather than account-based features (who is posting it). While using content-based features, it is best to combine a range of different features to ensure improved performance over being more selective and using fewer features. Our study provides insights into the development of contextualized abusive language detection models in realistic settings involving conversations.

Method: Using stylometric techniques including HTR pipeline, analysis of most frequent words, POS tags, and pseudo-affixes following Camps, Clérice, and Pinche (2021) methodology.

Result: The study will compare performance on manually composed vs automatically extracted data and test transformer-based OCR/transcription alignment validity for scholastic Latin corpora.

Conclusion: If successful, this research will provide a reusable template for analyzing collaborative literary production from medieval universities using computational methods.

Abstract: While the indirect evidence suggests that already in the early scholastic period the literary production based on records of oral teaching (so-called reportationes) was not uncommon, there are very few sources commenting on the practice. This paper details the design of a study applying stylometric techniques of authorship attribution to a collection developed from reportationes – Stephen Langton’s Quaestiones Theologiae – aiming to uncover layers of editorial work and thus validate some hypotheses regarding the collection’s formation. Following Camps, Cl'erice, and Pinche (2021), I discuss the implementation of an HTR pipeline and stylometric analysis based on the most frequent words, POS tags, and pseudo-affixes. The proposed study will offer two methodological gains relevant to computational research on the scholastic tradition: it will directly compare performance on manually composed and automatically extracted data, and it will test the validity of transformer-based OCR and automated transcription alignment for workflows applied to scholastic Latin corpora. If successful, this study will provide an easily reusable template for the exploratory analysis of collaborative literary production stemming from medieval universities.

Method: Extracted token embedding space of RoBERTa-base, performed k-means clustering into 200 clusters, manually inspected clusters for semantic sensitivity, and tested sensitivity to five psycholinguistic measures (valence, concreteness, iconicity, taboo, age of acquisition).

Result: Positive findings showing wide variety of semantic information encoded within token embedding space, with clusters sensitive to semantic information and psycholinguistic measures.

Conclusion: Transformer LLMs do encode semantic information in their representations, ruling out meaning eliminativist hypotheses about how they process semantic information.

Abstract: We investigate how word meanings are represented in the transformer language models. Specifically, we focus on whether transformer models employ something analogous to a lexical store - where each word has an entry that contains semantic information. To do this, we extracted the token embedding space of RoBERTa-base and k-means clustered it into 200 clusters. In our first study, we then manually inspected the resultant clusters to consider whether they are sensitive to semantic information. In our second study, we tested whether the clusters are sensitive to five psycholinguistic measures: valence, concreteness, iconicity, taboo, and age of acquisition. Overall, our findings were very positive - there is a wide variety of semantic information encoded within the token embedding space. This serves to rule out certain “meaning eliminativist” hypotheses about how transformer LLMs process semantic information.

Method: Proposes an LLM-based agent approach with five external tools using ReAct framework, enabling dynamic selection of annotation strategies based on table characteristics. Uses Levenshtein distance to reduce redundant annotations.

Result: Outperforms existing approaches across various metrics on Tough Tables and BiodivTab datasets from SemTab challenge. Achieves 70% reduction in time costs and 60% reduction in LLM token usage.

Conclusion: Provides an efficient and cost-effective solution for semantic table annotation that handles complex table challenges while significantly reducing computational resources.

Abstract: The Semantic Table Annotation (STA) task, which includes Column Type Annotation (CTA) and Cell Entity Annotation (CEA), maps table contents to ontology entities and plays important roles in various semantic applications. However, complex tables often pose challenges such as semantic loss of column names or cell values, strict ontological hierarchy requirements, homonyms, spelling errors, and abbreviations, which hinder annotation accuracy. To address these issues, this paper proposes an LLM-based agent approach for CTA and CEA. We design and implement five external tools with tailored prompts based on the ReAct framework, enabling the STA agent to dynamically select suitable annotation strategies depending on table characteristics. Experiments are conducted on the Tough Tables and BiodivTab datasets from the SemTab challenge, which contain the aforementioned challenges. Our method outperforms existing approaches across various metrics. Furthermore, by leveraging Levenshtein distance to reduce redundant annotations, we achieve a 70% reduction in time costs and a 60% reduction in LLM token usage, providing an efficient and cost-effective solution for STA.

Method: ProActive Self-Refinement (PASR) allows LLMs to decide whether, when, and how to refine during generation using internal state and context.

Result: On Qwen3-8B, PASR reduces average token consumption by 41.6% compared to standard generation and achieves 8.2% accuracy improvement across 10 tasks.

Conclusion: PASR demonstrates significant efficiency and performance gains by enabling proactive, context-aware refinement during the generation process.

Abstract: Recent advances in self-refinement have demonstrated significant potential for improving the outputs of large language models (LLMs) through iterative refinement. However, most existing self-refinement methods rely on a reactive process with a fixed number of iterations, making it difficult to determine the optimal timing and content of refinement based on the evolving generation context. Inspired by the way humans dynamically refine their thoughts during execution, we propose ProActive Self-Refinement (PASR), a novel method that enables LLMs to refine their outputs during the generation process. Unlike methods that regenerate entire responses, PASR proactively decides whether, when, and how to refine based on the model’s internal state and evolving context. We conduct extensive experiments on a diverse set of 10 tasks to evaluate the effectiveness of PASR. Experimental results show that PASR significantly enhances problem-solving performance. In particular, on Qwen3-8B, PASR reduces average token consumption by 41.6 percent compared to standard generation, while also achieving an 8.2 percent improvement in accuracy. Our code and all baselines used in the paper are available in the GitHub.

Method: Constructed LLM-based multi-agent system for travel planning with two key mechanisms: notebook for information sharing and orchestrator agent for coordination in free-form conversations

Result: Notebook reduced hallucination errors by 18%, orchestrator reduced errors by up to 13.5% in focused areas, combined system achieved 25% final pass rate (17.5% improvement over 7.5% single-agent baseline)

Conclusion: Structured information sharing and reflective orchestration are key components for effective multi-agent systems in long-horizon planning with LLMs

Abstract: Multi-agent systems (MASs) have pushed the boundaries of large language model (LLM) agents in domains such as web research and software engineering. However, long-horizon, multi-constraint planning tasks involve conditioning on detailed information and satisfying complex interdependent constraints, which can pose a challenge for these systems. In this study, we construct an LLM-based MAS for a travel planning task which is representative of these challenges. We evaluate the impact of a notebook to facilitate information sharing, and evaluate an orchestrator agent to improve coordination in free form conversation between agents. We find that the notebook reduces errors due to hallucinated details by 18%, while an orchestrator directs the MAS to focus on and further reduce errors by up to 13.5% within focused sub-areas. Combining both mechanisms achieves a 25% final pass rate on the TravelPlanner benchmark, a 17.5% absolute improvement over the single-agent baseline’s 7.5% pass rate. These results highlight the potential of structured information sharing and reflective orchestration as key components in MASs for long horizon planning with LLMs.

Method: Created four simulated online shops populated with authentic product offers from Common Crawl, developed 91 cross-shop tasks including basic (product finding, price comparison, checkout) and advanced tasks (vague requirements, substitutes, compatibility). Evaluated eight baseline agents with different observation modalities, memory utilization, and LLMs (GPT 4.1 and Claude Sonnet 4).

Result: Best-performing configurations achieved 75% completion rate and 87% F1 score on basic tasks, and 53% completion rate with 63% F1 score on advanced tasks. Tasks require longer interaction trajectories than WebShop while maintaining real-world relevance.

Conclusion: WebMall provides a more realistic benchmark for web agents with heterogeneous product data from multiple shops, enabling better evaluation of comparison-shopping capabilities. The benchmark is publicly released to advance research in web navigation, reasoning, and efficiency for e-commerce applications.

Abstract: LLM-based web agents have the potential to automate long-running web tasks, such as finding offers for specific products in multiple online shops and subsequently ordering the cheapest products that meet the users needs. This paper introduces WebMall, a multi-shop online shopping benchmark for evaluating the effectiveness and efficiency of web agents for comparison-shopping. WebMall consists of four simulated online shops populated with authentic product offers sourced from the Common Crawl, alongside a suite of 91 cross-shop tasks. These tasks include basic tasks such as finding specific products in multiple shops, performing price comparisons, adding items to the shopping cart, and completing checkout. Advanced tasks involve searching for products based on vague requirements, identifying suitable substitutes, and finding compatible products. Compared to existing e-commerce benchmarks, such as WebShop or ShoppingBench, WebMall introduces comparison-shopping tasks across multiple shops. Furthermore, the product offers are more heterogeneous, as they originate from hundreds of distinct real-world shops. The tasks in WebMall require longer interaction trajectories than those in WebShop, while remaining representative of real-world shopping behaviors. We evaluate eight baseline agents on WebMall, varying in observation modality, memory utilization, and underlying large language model (GPT 4.1 and Claude Sonnet 4). The best-performing configurations achieve completion rates of 75% and 53%, and F1 scores of 87% and 63%, on the basic and advanced task sets, respectively. WebMall is publicly released to facilitate research on web agents and to promote advancements in navigation, reasoning, and efficiency within e-commerce scenarios.

Method: Integrates feedback loss from bi-modal sarcasm detection model into TTS training, plus two-stage fine-tuning: first on diverse speech styles, then specifically on sarcastic speech dataset.

Result: Objective and subjective evaluations show improved quality, naturalness, and sarcasm-awareness of synthesized speech.

Conclusion: The proposed methods effectively enhance sarcastic speech synthesis by addressing data limitations and capturing nuanced prosodic features through multi-modal feedback and targeted fine-tuning.

Abstract: Sarcastic speech synthesis, which involves generating speech that effectively conveys sarcasm, is essential for enhancing natural interactions in applications such as entertainment and human-computer interaction. However, synthesizing sarcastic speech remains a challenge due to the nuanced prosody that characterizes sarcasm, as well as the limited availability of annotated sarcastic speech data. To address these challenges, this study introduces a novel approach that integrates feedback loss from a bi-modal sarcasm detection model into the TTS training process, enhancing the model’s ability to capture and convey sarcasm. In addition, by leveraging transfer learning, a speech synthesis model pre-trained on read speech undergoes a two-stage fine-tuning process. First, it is fine-tuned on a diverse dataset encompassing various speech styles, including sarcastic speech. In the second stage, the model is further refined using a dataset focused specifically on sarcastic speech, enhancing its ability to generate sarcasm-aware speech. Objective and subjective evaluations demonstrate that our proposed methods improve the quality, naturalness, and sarcasm-awareness of synthesized speech.

Method: Proposes LoRID method with multi-LoRA interaction: 1) Create knowledge-enhanced datasets using LLMs, 2) Train Intuitive Reasoner (IR) LoRA block for direct Chain-of-Thought generation (System 1), 3) Train Knowledge Generator (KG) and Deep Reasoner (DR) separately (System 2) - KG outputs knowledge, DR uses knowledge for reasoning, 4) Implement iterative inference with consistency checking between IR and DR outputs for mutual feedback enhancement.

Result: Achieves state-of-the-art performance, especially on GSM8K dataset where it outperforms second-best method by 2.3%, 16.1%, 2.4%, 12.3%, and 1.8% accuracy across five different base models respectively.

Conclusion: LoRID successfully enhances mathematical reasoning in SLMs by mimicking both System 1 and System 2 human thinking processes through multi-LoRA interaction and iterative consistency checking, demonstrating significant performance improvements over existing methods.

Abstract: Recent studies have demonstrated that Large Language Models (LLMs) have strong mathematical reasoning abilities but rely on hundreds of billions of parameters. To tackle the challenge of poor reasoning in Small Language Models (SLMs), existing methods typically leverage LLMs to generate massive amounts of data for cramming training. In psychology, they are akin to System 1 thinking, which resolves reasoning problems rapidly based on experience and intuition. However, human learning also requires System 2 thinking, where knowledge is first acquired and then reinforced through practice. Inspired by such two distinct modes of thinking, we propose a novel method based on the multi-LoRA Interaction for mathematical reasoning Distillation (LoRID). First, we input the question and reasoning of each sample into an LLM to create knowledge-enhanced datasets. Subsequently, we train a LoRA block on the student model as an Intuitive Reasoner (IR), which directly generates Chain-of-Thoughts for problem-solving. Then, to imitate System 2 thinking, we train the Knowledge Generator (KG) and Deep Reasoner (DR), respectively. The former outputs only knowledge after receiving problems, while the latter uses that knowledge to perform reasoning. Finally, to address the randomness in the generation of IR and DR, we evaluate whether their outputs are consistent, and the inference process needs to be iterated if not. This step can enhance the mathematical reasoning ability of SLMs through mutual feedback. Experimental results show that LoRID achieves state-of-the-art performance, especially on the GSM8K dataset, where it outperforms the second-best method by 2.3%, 16.1%, 2.4%, 12.3%, and 1.8% accuracy across the five base models, respectively.

Method: Created TR-MMLU benchmark based on a meticulously curated dataset of 6,200 multiple-choice questions covering 62 sections within the Turkish education system.

Result: Evaluated state-of-the-art LLMs on TR-MMLU, identifying areas for improvement in model design for Turkish language processing.

Conclusion: TR-MMLU establishes a new standard for Turkish NLP research and provides a framework to inspire future innovations in Turkish language model evaluation.

Abstract: Language models have made significant advancements in understanding and generating human language, achieving remarkable success in various applications. However, evaluating these models remains a challenge, particularly for resource-limited languages like Turkish. To address this issue, we introduce the Turkish MMLU (TR-MMLU) benchmark, a comprehensive evaluation framework designed to assess the linguistic and conceptual capabilities of large language models (LLMs) in Turkish. TR-MMLU is based on a meticulously curated dataset comprising 6,200 multiple-choice questions across 62 sections within the Turkish education system. This benchmark provides a standard framework for Turkish NLP research, enabling detailed analyses of LLMs’ capabilities in processing Turkish text. In this study, we evaluated state-of-the-art LLMs on TR-MMLU, highlighting areas for improvement in model design. TR-MMLU sets a new standard for advancing Turkish NLP research and inspiring future innovations.

Method: Used Turkish MMLU dataset (6,200 questions) to evaluate tokenizers based on vocabulary size, token count, processing time, language-specific token percentages (%TR), and token purity (%Pure) metrics.

Result: Language-specific token percentages showed stronger correlation with downstream performance than token purity. Increasing model parameters alone doesn’t improve linguistic performance.

Conclusion: The framework establishes robust tokenization standards for morphologically complex languages, emphasizing the importance of tailored, language-specific tokenization methods.

Abstract: Tokenization is a fundamental preprocessing step in Natural Language Processing (NLP), significantly impacting the capability of large language models (LLMs) to capture linguistic and semantic nuances. This study introduces a novel evaluation framework addressing tokenization challenges specific to morphologically-rich and low-resource languages such as Turkish. Utilizing the Turkish MMLU (TR-MMLU) dataset, comprising 6,200 multiple-choice questions from the Turkish education system, we assessed tokenizers based on vocabulary size, token count, processing time, language-specific token percentages (%TR), and token purity (%Pure). These newly proposed metrics measure how effectively tokenizers preserve linguistic structures. Our analysis reveals that language-specific token percentages exhibit a stronger correlation with downstream performance (e.g., MMLU scores) than token purity. Furthermore, increasing model parameters alone does not necessarily enhance linguistic performance, underscoring the importance of tailored, language-specific tokenization methods. The proposed framework establishes robust and practical tokenization standards for morphologically complex languages.

Method: Developed the Simon Fraser University Speech Error Database (SFUSED) with comprehensive annotations including linguistic hierarchical level, contextual sensitivity, degraded words, word corrections, and error positioning at word and syllable levels. Evaluated WhisperX transcription accuracy across 5,300 documented word and phonological errors.

Result: The database proved effective as a diagnostic tool for assessing ASR system performance, providing valuable insights into model capabilities through systematic error analysis.

Conclusion: SFUSED serves as a valuable resource for linguistic and psycholinguistic research and provides an effective framework for evaluating and improving speech recognition model performance through detailed error analysis.

Abstract: The Simon Fraser University Speech Error Database (SFUSED) is a public data collection developed for linguistic and psycholinguistic research. Here we demonstrate how its design and annotations can be used to test and evaluate speech recognition models. The database comprises systematically annotated speech errors from spontaneous English speech, with each error tagged for intended and actual error productions. The annotation schema incorporates multiple classificatory dimensions that are of some value to model assessment, including linguistic hierarchical level, contextual sensitivity, degraded words, word corrections, and both word-level and syllable-level error positioning. To assess the value of these classificatory variables, we evaluated the transcription accuracy of WhisperX across 5,300 documented word and phonological errors. This analysis demonstrates the atabase’s effectiveness as a diagnostic tool for ASR system performance.

Method: Reinforcement learning framework that self-supervises by estimating token prediction difficulty and adaptively selects the next token during training and inference.

Result: Superior performance on challenging reasoning and planning datasets, sometimes outperforming oracle models with ground-truth order supervision.

Conclusion: Adaptive token generation orders learned through reinforcement learning significantly improve model performance on complex reasoning tasks compared to fixed-order approaches.

Abstract: Modern causal language models, followed by rapid developments in discrete diffusion models, can now produce a wide variety of interesting and useful content. However, these families of models are predominantly trained to output tokens with a fixed (left-to-right) or random order, which may deviate from the logical order in which tokens are generated originally. In this paper, we observe that current causal and diffusion models encounter difficulties in problems that require adaptive token generation orders to solve tractably, which we characterize with the $\mathcal{V}$-information framework. Motivated by this, we propose Reinforced Context Order Recovery (ReCOR), a reinforcement-learning-based framework to extract adaptive, data-dependent token generation orders from text data without annotations. Self-supervised by token prediction statistics, ReCOR estimates the hardness of predicting every unfilled token and adaptively selects the next token during both training and inference. Experiments on challenging reasoning and planning datasets demonstrate the superior performance of ReCOR compared with baselines, sometimes outperforming oracle models supervised with the ground-truth order.

Method: Modified existing web extraction pipeline to preserve complete document integrity from source, retaining all content including unaligned portions. Identified optimal training context strategy through preliminary experiments.

Result: LLMs fine-tuned on DocHPLT substantially outperform off-the-shelf instruction-tuned baselines, with particularly dramatic improvements for under-resourced languages.

Conclusion: DocHPLT provides essential infrastructure for advancing multilingual document-level translation and is open-sourced under a permissive license.

Abstract: Existing document-level machine translation resources are only available for a handful of languages, mostly high-resourced ones. To facilitate the training and evaluation of document-level translation and, more broadly, long-context modeling for global communities, we create DocHPLT, the largest publicly available document-level translation dataset to date. It contains 124 million aligned document pairs across 50 languages paired with English, comprising 4.26 billion sentences, with further possibility to provide 2500 bonus pairs not involving English. Unlike previous reconstruction-based approaches that piece together documents from sentence-level data, we modify an existing web extraction pipeline to preserve complete document integrity from the source, retaining all content including unaligned portions. After our preliminary experiments identify the optimal training context strategy for document-level translation, we demonstrate that LLMs fine-tuned on DocHPLT substantially outperform off-the-shelf instruction-tuned baselines, with particularly dramatic improvements for under-resourced languages. We open-source the dataset under a permissive license, providing essential infrastructure for advancing multilingual document-level translation.

Method: End-to-end RAG pipeline with three enhancements: context-aware query translator that handles document references and adapts retrieval parameters, open-source retrieval using SBERT and GTE embeddings, and comprehensive evaluation framework combining RAGAS, BERTScore-F1, and ROUGE-Recall metrics.

Result: Substantial performance gains with 30-95% improvement in Recall@K and ~2.5x Precision@K for K>4. Open-source pipelines rival or outperform proprietary approaches in retrieval quality. Custom legal-grounded prompts produce more faithful and contextually relevant answers than baseline prompting.

Conclusion: Task-aware, component-level tuning enables legally grounded, reproducible, and cost-effective RAG systems for legal research, demonstrating the potential of carefully designed open-source approaches to match or exceed proprietary solutions.

Abstract: Retrieval-Augmented Generation (RAG) mitigates hallucinations by grounding large language model outputs in cited sources, a capability that is especially critical in the legal domain. We present an end-to-end RAG pipeline that revisits and extends the LegalBenchRAG baseline with three targeted enhancements: (i) a context-aware query translator that disentangles document references from natural-language questions and adapts retrieval depth and response style based on expertise and specificity, (ii) open-source retrieval strategies using SBERT and GTE embeddings that achieve substantial performance gains (improving Recall@K by 30-95% and Precision@K by $\sim$2.5$\times$ for $K>4$) while remaining cost-efficient, and (iii) a comprehensive evaluation and generation framework that combines RAGAS, BERTScore-F1, and ROUGE-Recall to assess semantic alignment and faithfulness across models and prompt designs. Our results show that carefully designed open-source pipelines can rival or outperform proprietary approaches in retrieval quality, while a custom legal-grounded prompt consistently produces more faithful and contextually relevant answers than baseline prompting. Taken together, these contributions demonstrate the potential of task-aware, component-level tuning to deliver legally grounded, reproducible, and cost-effective RAG systems for legal research assistance.

Method: Extends AutoBnB framework with RAG capabilities in Backdoors & Breaches tabletop environment. Introduces two retrieval settings: RAG-Wiki (technical documentation) and RAG-News (narrative incident reports). Evaluates eight team structures including argumentative configurations for critical reasoning.

Result: Retrieval augmentation improves decision quality and success rates across diverse organizational models. System demonstrates ability to reconstruct complex multi-stage attacks based on real-world cyber incidents.

Conclusion: Integration of retrieval mechanisms into LLM-based multi-agent systems provides significant value for cybersecurity decision-making, enhancing autonomous incident response capabilities.

Abstract: Incident response (IR) requires fast, coordinated, and well-informed decision-making to contain and mitigate cyber threats. While large language models (LLMs) have shown promise as autonomous agents in simulated IR settings, their reasoning is often limited by a lack of access to external knowledge. In this work, we present AutoBnB-RAG, an extension of the AutoBnB framework that incorporates retrieval-augmented generation (RAG) into multi-agent incident response simulations. Built on the Backdoors & Breaches (B&B) tabletop game environment, AutoBnB-RAG enables agents to issue retrieval queries and incorporate external evidence during collaborative investigations. We introduce two retrieval settings: one grounded in curated technical documentation (RAG-Wiki), and another using narrative-style incident reports (RAG-News). We evaluate performance across eight team structures, including newly introduced argumentative configurations designed to promote critical reasoning. To validate practical utility, we also simulate real-world cyber incidents based on public breach reports, demonstrating AutoBnB-RAG’s ability to reconstruct complex multi-stage attacks. Our results show that retrieval augmentation improves decision quality and success rates across diverse organizational models. This work demonstrates the value of integrating retrieval mechanisms into LLM-based multi-agent systems for cybersecurity decision-making.

Method: Introduces BlindSpot framework with taxonomy of 15 operational bias dimensions, uses LLM as zero-shot classifier to derive categorical distributions, and quantifies bias using Fidelity Gap (JS Divergence) and Coverage metrics.

Result: Empirical study with 2500 real call transcripts and summaries from 20 LLMs shows biases are systemic and present across all models regardless of size or family.

Conclusion: Operational biases in LLM-generated call summaries are widespread and systematic, requiring frameworks like BlindSpot for identification and quantification to address these issues in contact center applications.

Abstract: Abstractive summarization is a core application in contact centers, where Large Language Models (LLMs) generate millions of summaries of call transcripts daily. Despite their apparent quality, it remains unclear whether LLMs systematically under- or over-attend to specific aspects of the transcript, potentially introducing biases in the generated summary. While prior work has examined social and positional biases, the specific forms of bias pertinent to contact center operations - which we term Operational Bias - have remained unexplored. To address this gap, we introduce BlindSpot, a framework built upon a taxonomy of 15 operational bias dimensions (e.g., disfluency, speaker, topic) for the identification and quantification of these biases. BlindSpot leverages an LLM as a zero-shot classifier to derive categorical distributions for each bias dimension in a pair of transcript and its summary. The bias is then quantified using two metrics: Fidelity Gap (the JS Divergence between distributions) and Coverage (the percentage of source labels omitted). Using BlindSpot, we conducted an empirical study with 2500 real call transcripts and their summaries generated by 20 LLMs of varying scales and families (e.g., GPT, Llama, Claude). Our analysis reveals that biases are systemic and present across all evaluated models, regardless of size or family.

Method: Introduce MuDRiC dataset with multiple Arabic dialects and propose a novel Graph Convolutional Network (GCN) adaptation for Arabic commonsense reasoning to enhance semantic relationship modeling.

Result: The approach achieves superior performance in Arabic commonsense validation compared to existing methods.

Conclusion: This work enhances Arabic natural language understanding by providing both a foundational multi-dialect dataset and a novel method for handling Arabic’s complex linguistic variations.

Abstract: Commonsense validation evaluates whether a sentence aligns with everyday human understanding, a critical capability for developing robust natural language understanding systems. While substantial progress has been made in English, the task remains underexplored in Arabic, particularly given its rich linguistic diversity. Existing Arabic resources have primarily focused on Modern Standard Arabic (MSA), leaving regional dialects underrepresented despite their prevalence in spoken contexts. To bridge this gap, we present two key contributions: (i) we introduce MuDRiC, an extended Arabic commonsense dataset incorporating multiple dialects, and (ii) a novel method adapting Graph Convolutional Networks (GCNs) to Arabic commonsense reasoning, which enhances semantic relationship modeling for improved commonsense validation. Our experimental results demonstrate that this approach achieves superior performance in Arabic commonsense validation. Our work enhances Arabic natural language understanding by providing both a foundational dataset and a novel method for handling its complex variations. To the best of our knowledge, we release the first Arabic multi-dialect commonsense reasoning dataset.

Method: Developed hybrid schemes that combine watermark detectors with non-watermark detectors, testing various experimental conditions to evaluate performance improvements.

Result: Observed significant performance gains over using either watermark or non-watermark detectors alone across a wide range of experimental conditions.

Conclusion: Combining watermark and non-watermark detection methods provides superior detection capabilities for LLM-generated content, especially in practical low-entropy scenarios.

Abstract: Watermarking has recently emerged as an effective strategy for detecting the generations of large language models (LLMs). The strength of a watermark typically depends strongly on the entropy afforded by the language model and the set of input prompts. However, entropy can be quite limited in practice, especially for models that are post-trained, for example via instruction tuning or reinforcement learning from human feedback (RLHF), which makes detection based on watermarking alone challenging. In this work, we investigate whether detection can be improved by combining watermark detectors with non-watermark ones. We explore a number of hybrid schemes that combine the two, observing performance gains over either class of detector under a wide range of experimental conditions.

Method: Created two sub-benchmarks: OverthinkingBench with 72 domains of simple queries, and UnderthinkingBench with 11 challenging reasoning tasks. Used novel thinking-adjusted accuracy metrics to evaluate 33 different models.

Result: No model achieved optimal thinking - thinking models overthink simple queries without performance gains, while large non-thinking models underthink and perform worse than smaller thinking models on complex tasks.

Conclusion: Current approaches improve one aspect at the expense of the other, highlighting the need for better unified models that can balance performance and efficiency across different query complexities.

Abstract: Thinking LLMs solve complex tasks at the expense of increased compute and overthinking on simpler problems, while non-thinking LLMs are faster and cheaper but underthink on harder reasoning problems. This has led to the development of separate thinking and non-thinking LLM variants, leaving the onus of selecting the optimal model for each query on the end user. In this work, we introduce OptimalThinkingBench, a unified benchmark that jointly evaluates overthinking and underthinking in LLMs and also encourages the development of optimally-thinking models that balance performance and efficiency. Our benchmark comprises two sub-benchmarks: OverthinkingBench, featuring simple queries in 72 domains, and UnderthinkingBench, containing 11 challenging reasoning tasks. Using novel thinking-adjusted accuracy metrics, we perform extensive evaluation of 33 different thinking and non-thinking models and show that no model is able to optimally think on our benchmark. Thinking models often overthink for hundreds of tokens on the simplest user queries without improving performance. In contrast, large non-thinking models underthink, often falling short of much smaller thinking models. We further explore several methods to encourage optimal thinking, but find that these approaches often improve on one sub-benchmark at the expense of the other, highlighting the need for better unified and optimal models in the future.

Method: Introduced two key metrics (signal and noise) to evaluate benchmark quality, tested 30 benchmarks with 375 language models (60M-32B parameters), and proposed three interventions: switching to better metrics like perplexity, filtering noisy subtasks, and averaging intermediate checkpoints.

Result: Benchmarks with better signal-to-noise ratio are more reliable for small-scale decisions and have lower scaling law prediction error. The interventions consistently improved reliability across different benchmarks.

Conclusion: Benchmark creators should aim for high signal and low noise, using interventions like better metrics, subtask filtering, and checkpoint averaging to create more reliable evaluation benchmarks for LLM development.

Abstract: Developing large language models is expensive and involves making decisions with small experiments, typically by evaluating on large, multi-task evaluation suites. In this work, we analyze specific properties which make a benchmark more reliable for such decisions, and interventions to design higher-quality evaluation benchmarks. We introduce two key metrics that show differences in current benchmarks: signal, a benchmark’s ability to separate better models from worse models, and noise, a benchmark’s sensitivity to random variability between training steps. We demonstrate that benchmarks with a better signal-to-noise ratio are more reliable when making decisions at small scale, and those with less noise have lower scaling law prediction error. These results suggest that improving signal or noise will lead to more useful benchmarks, so we introduce three interventions designed to directly affect signal or noise. For example, we propose that switching to a metric that has better signal and noise (e.g., perplexity rather than accuracy) leads to better reliability and improved scaling law error. We also find that filtering noisy subtasks, to improve an aggregate signal-to-noise ratio, leads to more reliable multi-task evaluations. We also find that averaging the output of a model’s intermediate checkpoints to reduce noise leads to consistent improvements. We conclude by recommending that those creating new benchmarks, or selecting which existing benchmarks to use, aim for high signal and low noise. We use 30 benchmarks for these experiments, and 375 open-weight language models from 60M to 32B parameters, resulting in a new, publicly available dataset of 900K evaluation benchmark results, totaling 200M instances.

Method: RepreGuard employs a surrogate model to collect representations of both LLM-generated and human-written texts, extracts distinct activation features that identify machine-generated content, and classifies texts by calculating projection scores along these feature directions compared to precomputed thresholds.

Result: The method achieves 94.92% average AUROC across both in-distribution and out-of-distribution scenarios, outperforming all baselines while demonstrating robustness to various text sizes and mainstream attacks.

Conclusion: Internal LLM representations provide superior features for detecting generated content, enabling RepreGuard to achieve state-of-the-art performance with strong generalization capabilities across diverse scenarios and attack resilience.

Abstract: Detecting content generated by large language models (LLMs) is crucial for preventing misuse and building trustworthy AI systems. Although existing detection methods perform well, their robustness in out-of-distribution (OOD) scenarios is still lacking. In this paper, we hypothesize that, compared to features used by existing detection methods, the internal representations of LLMs contain more comprehensive and raw features that can more effectively capture and distinguish the statistical pattern differences between LLM-generated texts (LGT) and human-written texts (HWT). We validated this hypothesis across different LLMs and observed significant differences in neural activation patterns when processing these two types of texts. Based on this, we propose RepreGuard, an efficient statistics-based detection method. Specifically, we first employ a surrogate model to collect representation of LGT and HWT, and extract the distinct activation feature that can better identify LGT. We can classify the text by calculating the projection score of the text representations along this feature direction and comparing with a precomputed threshold. Experimental results show that RepreGuard outperforms all baselines with average 94.92% AUROC on both in-distribution (ID) and OOD scenarios, while also demonstrating robust resilience to various text sizes and mainstream attacks. Data and code are publicly available at: https://github.com/NLP2CT/RepreGuard

Method: Large-scale experiment with 8000 simulations, manipulating target culture (US vs. Korean), inventory language (English vs. Korean), and language model (GPT-4 vs. GPT-3.5) using the Ten-Item Personality Inventory.

Result: GPT-4 successfully replicated cross-cultural differences for all Big Five factors, but exhibited upward bias in mean ratings, lower variation than human samples, and reduced structural validity.

Conclusion: LLMs show promise for aiding cross-cultural research and practice, though current limitations in bias, variation, and structural validity need to be addressed for more accurate cultural simulations.

Abstract: We use a large-scale experiment (N=8000) to determine whether GPT-4 can replicate cross-cultural differences in the Big Five, measured using the Ten-Item Personality Inventory. We used the US and South Korea as the cultural pair, given that prior research suggests substantial personality differences between people from these two countries. We manipulated the target of the simulation (US vs. Korean), the language of the inventory (English vs. Korean), and the language model (GPT-4 vs. GPT-3.5). Our results show that GPT-4 replicated the cross-cultural differences for each factor. However, mean ratings had an upward bias and exhibited lower variation than in the human samples, as well as lower structural validity. We provide preliminary evidence that LLMs can aid cross-cultural researchers and practitioners.

Method: Created the Mostly Hard Python Problems (MHPP) dataset with 210 unique human-curated problems that focus on natural language and code reasoning, requiring comprehension of specifications, multi-step reasoning, and coding knowledge application.

Result: Evaluation of 26 LLMs showed that many high-performing models on HumanEval failed to achieve similar success on MHPP, revealing previously undiscovered limitations in various LLMs.

Conclusion: MHPP provides a more comprehensive benchmark that can better assess LLMs’ true code generation capabilities and limitations, paving the way for improved understanding of their performance.

Abstract: Recent advancements in large language models (LLMs) have greatly improved code generation, specifically at the function level. For instance, GPT-4o has achieved a 91.0% pass rate on HumanEval. However, this draws into question the adequacy of existing benchmarks in thoroughly assessing function-level code generation capabilities. Our study analyzed two common benchmarks, HumanEval and MBPP, and found that these might not thoroughly evaluate LLMs’ code generation capacities due to limitations in quality, difficulty, and granularity. To resolve this, we introduce the Mostly Hard Python Problems (MHPP) dataset, consisting of 210 unique human-curated problems. By focusing on the combination of natural language and code reasoning, MHPP gauges LLMs' abilities to comprehend specifications and restrictions, engage in multi-step reasoning, and apply coding knowledge effectively. Initial evaluations of 26 LLMs using MHPP showed many high-performing models on HumanEval failed to achieve similar success on MHPP. Moreover, MHPP highlighted various previously undiscovered limitations within various LLMs, leading us to believe that it could pave the way for a better understanding of LLMs’ capabilities and limitations. MHPP, evaluation pipeline, and leaderboard can be found in https://github.com/SparksofAGI/MHPP.

Method: Proposes FacLens - a lightweight model that probes hidden representations of fact-seeking questions to predict non-factuality before response generation, leveraging cross-model pattern similarities.

Result: Extensive experiments show FacLens achieves superior effectiveness and efficiency in non-factuality prediction, with demonstrated transferability across different LLMs.

Conclusion: FacLens provides an efficient and transferable solution for predicting LLM non-factuality, reducing development costs while maintaining high performance across various language models.

Abstract: Despite advancements in large language models (LLMs), non-factual responses still persist in fact-seeking question answering. Unlike extensive studies on post-hoc detection of these responses, this work studies non-factuality prediction (NFP), predicting whether an LLM will generate a non-factual response prior to the response generation. Previous NFP methods have shown LLMs’ awareness of their knowledge, but they face challenges in terms of efficiency and transferability. In this work, we propose a lightweight model named Factuality Lens (FacLens), which effectively probes hidden representations of fact-seeking questions for the NFP task. Moreover, we discover that hidden question representations sourced from different LLMs exhibit similar NFP patterns, enabling the transferability of FacLens across different LLMs to reduce development costs. Extensive experiments highlight FacLens’s superiority in both effectiveness and efficiency.

Method: Created RATS dataset with assembly task instructions and code examples, then evaluated state-of-the-art LLMs for conversational code generation using in-context learning.

Result: LLMs successfully generated accurate first-order instruction sequences but had difficulties producing higher-order code abstractions like functions and loops.

Conclusion: LLMs show promise for basic robotic programming but need improvement for complex programming constructs in industrial assembly scenarios.

Abstract: While there has been a lot of research recently on robots in household environments, at the present time, most robots in existence can be found on shop floors, and most interactions between humans and robots happen there. Collaborative robots'' (cobots) designed to work alongside humans on assembly lines traditionally require expert programming, limiting ability to make changes, or manual guidance, limiting expressivity of the resulting programs. To address these limitations, we explore using Large Language Models (LLMs), and in particular, their abilities of doing in-context learning, for conversational code generation. As a first step, we define RATS, the Repetitive Assembly Task’’, a 2D building task designed to lay the foundation for simulating industry assembly scenarios. In this task, a programmer' instructs a cobot, using natural language, on how a certain assembly is to be built; that is, the programmer induces a program, through natural language. We create a dataset that pairs target structures with various example instructions (human-authored, template-based, and model-generated) and example code. With this, we systematically evaluate the capabilities of state-of-the-art LLMs for synthesising this kind of code, given in-context examples. Evaluating in a simulated environment, we find that LLMs are capable of generating accurate first order code’ (instruction sequences), but have problems producing `higher-order code’ (abstractions such as functions, or use of loops).

Method: Used contextual bandit framework for in-context reinforcement learning (ICRL), experimenting with classification tasks across LLM sizes from 500M to 70B parameters, addressing instability issues and testing both semantic and abstract labels.

Result: LLMs effectively demonstrated in-context reinforcement learning capabilities, showing learning from external rewards with identified scaling trends, though limitations were found in their implicit error reasoning.

Conclusion: LLMs possess significant ICRL capabilities that extend beyond supervised in-context learning, but fundamental limitations exist in how they reason about and learn from errors in reinforcement learning settings.

Abstract: Large Language Models (LLMs) excel at in-context learning (ICL), a supervised learning technique that relies on adding annotated examples to the model context. We investigate a contextual bandit version of in-context reinforcement learning (ICRL), where models learn in-context, online, from external reward, instead of supervised data. We show that LLMs effectively demonstrate such learning, and provide a detailed study of the phenomena, experimenting with challenging classification tasks and models of sizes from 500M to 70B parameters. This includes identifying and addressing the instability of the process, demonstrating learning with both semantic and abstract labels, and showing scaling trends. Our findings highlight ICRL capabilities in LLMs, while also underscoring fundamental limitations in their implicit reasoning about errors.

Method: Step-grained reinforcement learning framework with two components: Step-grained Reward Shaping (rewards per tool interaction based on success and contribution) and Step-grained Optimization (policy gradient methods across multiple decision steps).

Result: Consistently outperforms both SFT-based and RL-based baselines in task Pass Rate and Recall of relevant tools across diverse benchmarks. Helps models discover new tool-use strategies rather than just re-weighting prior knowledge.

Conclusion: StepTool demonstrates the importance of fine-grained decision modeling in tool learning and provides a general, robust solution for enhancing multi-step tool use in LLMs.

Abstract: Despite their powerful text generation capabilities, large language models (LLMs) still struggle to effectively utilize external tools to solve complex tasks, a challenge known as tool learning. Existing methods primarily rely on supervised fine-tuning, treating tool learning as a text generation problem while overlooking the decision-making complexities inherent in multi-step contexts. In this work, we propose modeling tool learning as a dynamic decision-making process and introduce StepTool, a novel step-grained reinforcement learning framework that enhances LLMs’ capabilities in multi-step tool use. StepTool comprises two key components: Step-grained Reward Shaping, which assigns rewards to each tool interaction based on its invocation success and contribution to task completion; and Step-grained Optimization, which applies policy gradient methods to optimize the model across multiple decision steps. Extensive experiments across diverse benchmarks show that StepTool consistently outperforms both SFT-based and RL-based baselines in terms of task Pass Rate and Recall of relevant tools. Furthermore, our analysis suggests that StepTool helps models discover new tool-use strategies rather than merely re-weighting prior knowledge. These results highlight the importance of fine-grained decision modeling in tool learning and establish StepTool as a general and robust solution for enhancing multi-step tool use in LLMs. Code and data are available at https://github.com/yuyq18/StepTool.

Method: Leverages in-context learning to systematically insert emojis into text before evaluation by Judge LLMs, inducing embedding distortions that lower the likelihood of detecting unsafe content while introducing semantic ambiguity.

Result: Emoji Attack substantially reduces the unsafe prediction rate in state-of-the-art Judge LLMs, successfully bypassing existing safeguards and demonstrating significant vulnerability in current defense mechanisms.

Conclusion: Current Judge LLM-based defenses are critically vulnerable to token segmentation bias attacks using emojis, highlighting the need for more robust safety evaluation methods that account for tokenization artifacts and semantic manipulation.

Abstract: Jailbreaking techniques trick Large Language Models (LLMs) into producing restricted output, posing a potential threat. One line of defense is to use another LLM as a Judge to evaluate the harmfulness of generated text. However, we reveal that these Judge LLMs are vulnerable to token segmentation bias, an issue that arises when delimiters alter the tokenization process, splitting words into smaller sub-tokens. This alters the embeddings of the entire sequence, reducing detection accuracy and allowing harmful content to be misclassified as safe. In this paper, we introduce Emoji Attack, a novel strategy that amplifies existing jailbreak prompts by exploiting token segmentation bias. Our method leverages in-context learning to systematically insert emojis into text before it is evaluated by a Judge LLM, inducing embedding distortions that significantly lower the likelihood of detecting unsafe content. Unlike traditional delimiters, emojis also introduce semantic ambiguity, making them particularly effective in this attack. Through experiments on state-of-the-art Judge LLMs, we demonstrate that Emoji Attack substantially reduces the unsafe prediction rate, bypassing existing safeguards.

Method: Two variants of Number Token Loss (NTL) that minimize either L_p norm or Wasserstein distance between numerical values of real and predicted number tokens, added to cross-entropy objective during training without runtime overhead.

Result: NTL consistently improves performance on various mathematical datasets, matches regression head performance on regression tasks, and scales effectively to 3B parameter models with improved quantitative reasoning capabilities.

Conclusion: NTL provides a lightweight, easily integrable solution to enhance language models’ numerical reasoning without architectural changes, demonstrating potential for seamless integration into LLM pretraining objectives.

Abstract: While language models have exceptional capabilities at text generation, they lack a natural inductive bias for emitting numbers and thus struggle in tasks involving quantitative reasoning, especially arithmetic. One fundamental limitation is the nature of the cross-entropy (CE) loss, which assumes a nominal scale and thus cannot convey proximity between generated number tokens. In response, we here present a regression-like loss that operates purely on token level. Our proposed Number Token Loss (NTL) comes in two flavors and minimizes either the $L_p$ norm or the Wasserstein distance between the numerical values of the real and predicted number tokens. NTL can easily be added to any language model and extend the CE objective during training without runtime overhead. We evaluate the proposed scheme on various mathematical datasets and find that it consistently improves performance in math-related tasks. In a direct comparison on a regression task, we find that NTL can match the performance of a regression head, despite operating on token level. Finally, we scale NTL up to 3B parameter models and observe improved performance, demonstrating its potential for seamless integration into LLMs. We hope to inspire LLM developers to improve their pretraining objectives and distribute NTL as a minimalistic and lightweight PyPI package $ntloss$: https://github.com/ai4sd/number-token-loss. Development code for full paper reproduction is available separately.

Method: The method analyzes input and output representations of tokens, leveraging the observation that word embedding norms capture token importance during pre-training and that token importance correlates with how closely token representations resemble the model’s final prediction.

Result: Extensive analyses show NormXLogit outperforms existing gradient-based methods in faithfulness and achieves competitive performance in layer-wise explanations compared to leading architecture-specific techniques.

Conclusion: NormXLogit provides an effective, model-agnostic approach for token importance assessment that is both faithful and computationally efficient, addressing limitations of current interpretability methods.

Abstract: With new large language models (LLMs) emerging frequently, it is important to consider the potential value of model-agnostic approaches that can provide interpretability across a variety of architectures. While recent advances in LLM interpretability show promise, many rely on complex, model-specific methods with high computational costs. To address these limitations, we propose NormXLogit, a novel technique for assessing the significance of individual input tokens. This method operates based on the input and output representations associated with each token. First, we demonstrate that during the pre-training of LLMs, the norms of word embeddings effectively capture token importance. Second, we reveal a significant relationship between a token’s importance and the extent to which its representation can resemble the model’s final prediction. Extensive analyses reveal that our approach outperforms existing gradient-based methods in terms of faithfulness and offers competitive performance in layer-wise explanations compared to leading architecture-specific techniques.

Method: Combines five SOTA NER models with ChatGPT using custom prompt engineering as an arbitrator within an ensemble method, tested on three medical benchmark datasets.

Result: Outperforms SOTA models, individual GPT-3.5/GPT-4 applications, and voting ensemble on CADEC, ShARe13, and ShARe14 medical datasets.

Conclusion: ChatGPT integration as arbitrator in ensemble learning effectively enhances DNER performance, showing promise for healthcare NLP applications.

Abstract: Named Entity Recognition has traditionally been a key task in natural language processing, aiming to identify and extract important terms from unstructured text data. However, a notable challenge for contemporary deep-learning NER models has been identifying discontinuous entities, which are often fragmented within the text. To date, methods to address Discontinuous Named Entity Recognition have not been explored using ensemble learning to the best of our knowledge. Furthermore, the rise of large language models, such as ChatGPT in recent years, has shown significant effectiveness across many NLP tasks. Most existing approaches, however, have primarily utilized ChatGPT as a problem-solving tool rather than exploring its potential as an integrative element within ensemble learning algorithms. In this study, we investigated the integration of ChatGPT as an arbitrator within an ensemble method, aiming to enhance performance on DNER tasks. Our method combines five state-of-the-art NER models with ChatGPT using custom prompt engineering to assess the robustness and generalization capabilities of the ensemble algorithm. We conducted experiments on three benchmark medical datasets, comparing our method against the five SOTA models, individual applications of GPT-3.5 and GPT-4, and a voting ensemble method. The results indicate that our proposed fusion of ChatGPT with the ensemble learning algorithm outperforms the SOTA results in the CADEC, ShARe13, and ShARe14 datasets, showcasing its potential to enhance NLP applications in the healthcare domain.

Method: Developed a dataset of 10,580 sentences containing 101 Sorani Kurdish idioms, then trained and evaluated three deep learning models: KuBERT-based transformer sequence classification, Recurrent Convolutional Neural Network (RCNN), and BiLSTM with attention mechanism.

Result: The transformer model (fine-tuned BERT) achieved the best performance with nearly 99% accuracy, followed by RCNN (96.5%) and BiLSTM (80%).

Conclusion: Transformer-based architectures are highly effective for idiom detection in low-resource languages like Kurdish, and this research provides foundational resources including a dataset and optimized models for advancing Kurdish NLP.

Abstract: Idiom detection using Natural Language Processing (NLP) is the computerized process of recognizing figurative expressions within a text that convey meanings beyond the literal interpretation of the words. While idiom detection has seen significant progress across various languages, the Kurdish language faces a considerable research gap in this area despite the importance of idioms in tasks like machine translation and sentiment analysis. This study addresses idiom detection in Sorani Kurdish by approaching it as a text classification task using deep learning techniques. To tackle this, we developed a dataset containing 10,580 sentences embedding 101 Sorani Kurdish idioms across diverse contexts. Using this dataset, we developed and evaluated three deep learning models: KuBERT-based transformer sequence classification, a Recurrent Convolutional Neural Network (RCNN), and a BiLSTM model with an attention mechanism. The evaluations revealed that the transformer model, the fine-tuned BERT, consistently outperformed the others, achieving nearly 99% accuracy while the RCNN achieved 96.5% and the BiLSTM 80%. These results highlight the effectiveness of Transformer-based architectures in low-resource languages like Kurdish. This research provides a dataset, three optimized models, and insights into idiom detection, laying a foundation for advancing Kurdish NLP.

Method: Two-stage approach: 1) Width pruning removes neurons based on importance scores to preserve connectivity, 2) Depth pruning iteratively removes attention submodules with minimum impact on perplexity, with a novel sparsity balancing mechanism.

Result: Outperforms five state-of-the-art competitors across three language modeling datasets and six downstream tasks, with up to two-order-of-magnitude faster pruning time, tested on four LLM families at 25%, 37.5%, and 50% sparsity rates.

Conclusion: 2SSP provides an effective and efficient structured pruning framework for LLMs that combines width and depth pruning strategies, achieving superior performance with significantly reduced computational time compared to existing methods.

Abstract: We propose a novel Two-Stage framework for Structured Pruning (\textsc{2SSP}) for pruning Large Language Models (LLMs), which combines two different strategies of pruning, namely Width and Depth Pruning. The first stage (Width Pruning) removes entire neurons, hence their corresponding rows and columns, aiming to preserve the connectivity among the pruned structures in the intermediate state of the Feed-Forward Networks in each Transformer block. This is done based on an importance score measuring the impact of each neuron on the output magnitude. The second stage (Depth Pruning), instead, removes entire Attention submodules. This is done by applying an iterative process that removes the Attention with the minimum impact on a given metric of interest (in our case, perplexity). We also propose a novel mechanism to balance the sparsity rate of the two stages w.r.t. to the desired global sparsity. We test \textsc{2SSP} on four LLM families and three sparsity rates (25%, 37.5%, and 50%), measuring the resulting perplexity over three language modeling datasets as well as the performance over six downstream tasks. Our method consistently outperforms five state-of-the-art competitors over three language modeling and six downstream tasks, with an up to two-order-of-magnitude gain in terms of pruning time. The code is available at https://github.com/FabrizioSandri/2SSP.

Method: Proposes VisualSpeech model that incorporates both visual and textual information for prosody generation in TTS systems.

Result: Empirical results show that incorporating visual features improves prosodic modeling and enhances the expressiveness of synthesized speech.

Conclusion: Visual context can significantly enhance prosody prediction in TTS systems, making synthesized speech more expressive and natural.

Abstract: Text-to-Speech (TTS) synthesis faces the inherent challenge of producing multiple speech outputs with varying prosody given a single text input. While previous research has addressed this by predicting prosodic information from both text and speech, additional contextual information, such as video, remains under-utilized despite being available in many applications. This paper investigates the potential of integrating visual context to enhance prosody prediction. We propose a novel model, VisualSpeech, which incorporates visual and textual information for improving prosody generation in TTS. Empirical results indicate that incorporating visual features improves prosodic modeling, enhancing the expressiveness of the synthesized speech. Audio samples are available at https://ariameetgit.github.io/VISUALSPEECH-SAMPLES/.

Method: The paper examines various automatic approaches for aggregating multiple annotations in the context of hate speech classification, specifically focusing on Turkish tweets and strategies to address annotator disagreement.

Result: The work provides state-of-the-art benchmark results for hate speech detection in online discourse, demonstrating the effectiveness of different annotation aggregation methods for Turkish language content.

Conclusion: Addressing annotator disagreement through automatic annotation aggregation is essential for improving hate speech detection models, and this research provides valuable insights and benchmarks specifically for Turkish social media content.

Abstract: Hate speech detection is a crucial task, especially on social media, where harmful content can spread quickly. Implementing machine learning models to automatically identify and address hate speech is essential for mitigating its impact and preventing its proliferation. The first step in developing an effective hate speech detection model is to acquire a high-quality dataset for training. Labeled data is essential for most natural language processing tasks, but categorizing hate speech is difficult due to the diverse and often subjective nature of hate speech, which can lead to varying interpretations and disagreements among annotators. This paper examines strategies for addressing annotator disagreement, an issue that has been largely overlooked. In particular, we evaluate various automatic approaches for aggregating multiple annotations, in the context of hate speech classification in Turkish tweets. Our work highlights the importance of the problem and provides state-of-the-art benchmark results for the detection and understanding of hate speech in online discourse.

Method: LIDDIA leverages large language models’ reasoning capabilities to serve as an autonomous agent for in silico drug discovery, intelligently balancing exploration and exploitation in chemical space.

Result: LIDDIA generated molecules meeting key pharmaceutical criteria for over 70% of 30 clinically relevant targets, identified promising novel candidates for AR/NR3C4 (prostate and breast cancer target), and demonstrated intelligent chemical space navigation.

Conclusion: LIDDIA represents a low-cost, highly-adaptable tool for autonomous drug discovery that can significantly accelerate the drug development process by intelligently navigating chemical space and identifying promising therapeutic candidates.

Abstract: Drug discovery is a long, expensive, and complex process, relying heavily on human medicinal chemists, who can spend years searching the vast space of potential therapies. Recent advances in artificial intelligence for chemistry have sought to expedite individual drug discovery tasks; however, there remains a critical need for an intelligent agent that can navigate the drug discovery process. Towards this end, we introduce LIDDIA, an autonomous agent capable of intelligently navigating the drug discovery process in silico. By leveraging the reasoning capabilities of large language models, LIDDIA serves as a low-cost and highly-adaptable tool for autonomous drug discovery. We comprehensively examine LIDDIA , demonstrating that (1) it can generate molecules meeting key pharmaceutical criteria on over 70% of 30 clinically relevant targets, (2) it intelligently balances exploration and exploitation in the chemical space, and (3) it identifies one promising novel candidate on AR/NR3C4, a critical target for both prostate and breast cancers. Code and dataset are available at https://github.com/ninglab/LIDDiA

Method: Developed an information-theoretic algorithm leveraging weighted self-information distributions (extending classical discrete measures with continuous differential self-information) to detect structured patterns in text, avoiding instability issues of covariance-based methods in high-dimensional settings.

Result: Successfully isolated stylistic layers in hypothesized authorial divisions of the Hebrew Bible, providing quantitative framework for textual stratification and compositional pattern analysis.

Conclusion: The method enhances ability to analyze compositional patterns and offers deeper insights into literary and cultural evolution of texts shaped by complex authorship and editorial processes, applicable across different textual representations including neural embeddings.

Abstract: Texts, whether literary or historical, exhibit structural and stylistic patterns shaped by their purpose, authorship, and cultural context. Formulaic texts, characterized by repetition and constrained expression, tend to have lower variability in self-information compared to more dynamic compositions. Identifying such patterns in historical documents, particularly multi-author texts like the Hebrew Bible provides insights into their origins, purpose, and transmission. This study aims to identify formulaic clusters – sections exhibiting systematic repetition and structural constraints – by analyzing recurring phrases, syntactic structures, and stylistic markers. However, distinguishing formulaic from non-formulaic elements in an unsupervised manner presents a computational challenge, especially in high-dimensional textual spaces where patterns must be inferred without predefined labels. To address this, we develop an information-theoretic algorithm leveraging weighted self-information distributions to detect structured patterns in text, unlike covariance-based methods, which become unstable in small-sample, high-dimensional settings, our approach directly models variations in self-information to identify formulaicity. By extending classical discrete self-information measures with a continuous formulation based on differential self-information, our method remains applicable across different types of textual representations, including neural embeddings under Gaussian priors. Applied to hypothesized authorial divisions in the Hebrew Bible, our approach successfully isolates stylistic layers, providing a quantitative framework for textual stratification. This method enhances our ability to analyze compositional patterns, offering deeper insights into the literary and cultural evolution of texts shaped by complex authorship and editorial processes.

Method: Studied 31 diverse languages, proposed interlingual representation framework identifying shared semantic subspace and fragmented components. Introduced ILO score to quantify alignment by comparing local neighborhood structures of high-dimensional representations.

Result: Multilingual LLMs exhibit inconsistent cross-lingual alignments. Single-language fine-tuning disrupts alignment in early layers, while freezing these layers preserves interlingual alignment and improves cross-lingual generalization.

Conclusion: Interlingual alignment is crucial for scalable multilingual learning. The proposed framework and ILO metric effectively evaluate interlingual representations, showing that preserving early layer alignment enhances cross-lingual performance.

Abstract: Large language models (LLMs) trained on massive multilingual datasets hint at the formation of interlingual constructs–a shared subspace in the representation space. However, evidence regarding this phenomenon is mixed, leaving it unclear whether these models truly develop unified interlingual representations, or present a partially aligned constructs. We explore 31 diverse languages varying on their resource-levels, typologies, and geographical regions; and find that multilingual LLMs exhibit inconsistent cross-lingual alignments. To address this, we propose an interlingual representation framework identifying both the shared interlingual semantic subspace and fragmented components, existed due to representational limitations. We introduce Interlingual Local Overlap (ILO) score to quantify interlingual alignment by comparing the local neighborhood structures of high-dimensional representations. We utilize ILO to investigate the impact of single-language fine-tuning on the interlingual representations in multilingual LLMs. Our results indicate that training exclusively on a single language disrupts the alignment in early layers, while freezing these layers preserves the alignment of interlingual representations, leading to improved cross-lingual generalization. These results validate our framework and metric for evaluating interlingual representation, and further underscore that interlingual alignment is crucial for scalable multilingual learning.

Method: Novel evaluation framework using free-form storytelling to surface biases; systematic analysis of ten prominent LLMs; comparison with real-world labor data and human stereotypes

Result: Consistent pattern of overrepresenting female characters across occupations due to SFT and RLHF; occupational gender distributions align more closely with human stereotypes than real-world data

Conclusion: Highlights challenge of implementing balanced mitigation measures to promote fairness and prevent establishment of new biases; releases prompts and generated stories for further research

Abstract: Large Language Models (LLMs) have revolutionized natural language processing, yet concerns persist regarding their tendency to reflect or amplify social biases. This study introduces a novel evaluation framework to uncover gender biases in LLMs: using free-form storytelling to surface biases embedded within the models. A systematic analysis of ten prominent LLMs shows a consistent pattern of overrepresenting female characters across occupations, likely due to supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). Paradoxically, despite this overrepresentation, the occupational gender distributions produced by these LLMs align more closely with human stereotypes than with real-world labor data. This highlights the challenge and importance of implementing balanced mitigation measures to promote fairness and prevent the establishment of potentially new biases. We release the prompts and LLM-generated stories at GitHub.

Method: Proposed FastCuRL, a curriculum RL framework with stage-wise context scaling that controls context length and curates training data based on input prompt length.

Result: FastCuRL-1.5B-V3 outperforms state-of-the-art reasoning models on five benchmarks with 49.6% accuracy on AIME 2024. FastCuRL-1.5B-Preview surpasses DeepScaleR-1.5B-Preview using only 50% training steps and single node with 8 GPUs.

Conclusion: Proper context length scaling and data curation significantly improve RL training efficiency and reasoning performance while reducing computational requirements.

Abstract: Improving training efficiency continues to be one of the primary challenges in large-scale Reinforcement Learning (RL). In this paper, we investigate how context length and the complexity of training data influence the RL scaling training process of R1-distilled reasoning models, e.g., DeepSeek-R1-Distill-Qwen-1.5B. Our experimental results reveal that: (1) simply controlling the context length and curating the training data based on the input prompt length can effectively improve the training efficiency of RL scaling, achieving better performance with more concise CoT; (2) properly scaling the context length helps mitigate entropy collapse; and (3) carefully choosing the context length facilitates achieving efficient LLM training and reasoning. Inspired by these insights, we propose FastCuRL, a curriculum RL framework with stage-wise context scaling to achieve efficient LLM training and reasoning. Extensive experimental results demonstrate that FastCuRL-1.5B-V3 significantly outperforms state-of-the-art reasoning models on five competition-level benchmarks and achieves 49.6% accuracy on AIME 2024. Furthermore, FastCuRL-1.5B-Preview surpasses DeepScaleR-1.5B-Preview on five benchmarks while only using a single node with 8 GPUs and a total of 50% of training steps.

Method: Introduces Feather-SQL framework with schema pruning/linking and multi-path generation, plus a 1+1 Model Collaboration Paradigm pairing a general chat model with a SQL specialist model.

Result: Achieves ~10% performance improvement for SLMs without fine-tuning on BIRD benchmark, with accuracy ceiling raised to 54.76% for SLMs.

Conclusion: Feather-SQL effectively bridges the performance gap for small language models in NL2SQL tasks while maintaining privacy and deployability advantages over large models.

Abstract: Natural Language to SQL (NL2SQL) has seen significant advancements with large language models (LLMs). However, these models often depend on closed-source systems and high computational resources, posing challenges in data privacy and deployment. In contrast, small language models (SLMs) struggle with NL2SQL tasks, exhibiting poor performance and incompatibility with existing frameworks. To address these issues, we introduce Feather-SQL, a new lightweight framework tailored for SLMs. Feather-SQL improves SQL executability and accuracy through 1) schema pruning and linking, 2) multi-path and multi-candidate generation. Additionally, we introduce the 1+1 Model Collaboration Paradigm, which pairs a strong general-purpose chat model with a fine-tuned SQL specialist, combining strong analytical reasoning with high-precision SQL generation. Experimental results on BIRD demonstrate that Feather-SQL improves NL2SQL performance on SLMs, with around 10% boost for models without fine-tuning. The proposed paradigm raises the accuracy ceiling of SLMs to 54.76%, highlighting its effectiveness.

Method: Proposes SCORE framework that tracks key item statuses, generates episode summaries, and uses Retrieval-Augmented Generation (RAG) with TF-IDF and cosine similarity to identify related episodes and enhance story structure.

Result: Testing shows SCORE significantly improves narrative coherence consistency and stability compared to baseline GPT models in multiple LLM-generated stories.

Conclusion: SCORE provides a robust method for evaluating and refining AI-generated narratives by enhancing coherence through systematic tracking and retrieval mechanisms.

Abstract: Large Language Models (LLMs) can generate creative and engaging narratives from user-specified input, but maintaining coherence and emotional depth throughout these AI-generated stories remains a challenge. In this work, we propose SCORE, a framework for Story Coherence and Retrieval Enhancement, designed to detect and resolve narrative inconsistencies. By tracking key item statuses and generating episode summaries, SCORE uses a Retrieval-Augmented Generation (RAG) approach, incorporating TF-IDF and cosine similarity to identify related episodes and enhance the overall story structure. Results from testing multiple LLM-generated stories demonstrate that SCORE significantly improves the consistency and stability of narrative coherence compared to baseline GPT models, providing a more robust method for evaluating and refining AI-generated narratives.

Method: SPECTR approach for dynamic composition of expert models at each time step during inference, requiring no additional training and enabling flexible token- and layer-wise model combinations.

Result: Experimental results show SPECTR improves routing accuracy over alternative training-free methods and increases task performance across expert domains.

Conclusion: SPECTR provides an effective training-free solution for leveraging existing expert models through dynamic composition during inference, demonstrating improved performance across specialized domains.

Abstract: Training large, general-purpose language models poses significant challenges. The growing availability of specialized expert models, fine-tuned from pretrained models for specific tasks or domains, offers a promising alternative. Leveraging the potential of these existing expert models in real-world applications requires effective methods to select or merge the models best suited for a given task. This paper introduces SPECTR, an approach for dynamically composing expert models at each time step during inference. Notably, our method requires no additional training and enables flexible, token- and layer-wise model combinations. Our experimental results demonstrate that SPECTR improves routing accuracy over alternative training-free methods, increasing task performance across expert domains.

Method: Evaluated quantization (weight, KV cache, activation) on DeepSeek-R1-Distilled Qwen, LLaMA families (1.5B-70B), QwQ-32B, and Qwen3-8B using state-of-the-art algorithms at varying bit-widths across mathematical, scientific, and programming reasoning benchmarks.

Result: Lossless quantization achievable with W8A8 or W4A16, but lower bit-widths introduce significant accuracy risks. Quantized models don’t show increased output lengths. Strategic scaling of model sizes or reasoning steps can enhance performance.

Conclusion: Quantization is viable for reasoning models with careful bit-width selection, and model size/origin/task difficulty are critical performance determinants. All models and code open-sourced.

Abstract: Recent advancements in reasoning language models have demonstrated remarkable performance in complex tasks, but their extended chain-of-thought reasoning process increases inference overhead. While quantization has been widely adopted to reduce the inference cost of large language models, its impact on reasoning models remains understudied. In this paper, we conduct the first systematic study on quantized reasoning models, evaluating the open-sourced DeepSeek-R1-Distilled Qwen and LLaMA families ranging from 1.5B to 70B parameters, QwQ-32B, and Qwen3-8B. Our investigation covers weight, KV cache, and activation quantization using state-of-the-art algorithms at varying bit-widths, with extensive evaluation across mathematical (AIME, MATH-500), scientific (GPQA), and programming (LiveCodeBench) reasoning benchmarks. Our findings reveal that while lossless quantization can be achieved with W8A8 or W4A16 quantization, lower bit-widths introduce significant accuracy risks. We further identify model size, model origin, and task difficulty as critical determinants of performance. Contrary to expectations, quantized models do not exhibit increased output lengths. In addition, strategically scaling the model sizes or reasoning steps can effectively enhance the performance. All quantized models and codes are open-sourced in https://github.com/ruikangliu/Quantized-Reasoning-Models.

Method: Proposes adaptive rejection sampling that requires fewer constraint evaluations and extends it to produce low-variance, unbiased importance weights for use in sequential Monte Carlo algorithms.

Result: Superior to state-of-the-art baselines across multiple domains (text-to-SQL, molecular synthesis, goal inference, pattern matching, JSON), supporting broader constraint classes and improving both runtime and performance.

Conclusion: The method’s runtime efficiency scales with divergence between unconstrained and constrained LMs, meaning better models see greater improvements, making it a scalable solution for constrained generation.

Abstract: The dominant approach to generating from language models subject to some constraint is locally constrained decoding (LCD), incrementally sampling tokens at each time step such that the constraint is never violated. Typically, this is achieved through token masking: looping over the vocabulary and excluding non-conforming tokens. There are two important problems with this approach. (i) Evaluating the constraint on every token can be prohibitively expensive – LM vocabularies often exceed $100,000$ tokens. (ii) LCD can distort the global distribution over strings, sampling tokens based only on local information, even if they lead down dead-end paths. This work introduces a new algorithm that addresses both these problems. First, to avoid evaluating a constraint on the full vocabulary at each step of generation, we propose an adaptive rejection sampling algorithm that typically requires orders of magnitude fewer constraint evaluations. Second, we show how this algorithm can be extended to produce low-variance, unbiased estimates of importance weights at a very small additional cost – estimates that can be soundly used within previously proposed sequential Monte Carlo algorithms to correct for the myopic behavior of local constraint enforcement. Through extensive empirical evaluation in text-to-SQL, molecular synthesis, goal inference, pattern matching, and JSON domains, we show that our approach is superior to state-of-the-art baselines, supporting a broader class of constraints and improving both runtime and performance. Additional theoretical and empirical analyses show that our method’s runtime efficiency is driven by its dynamic use of computation, scaling with the divergence between the unconstrained and constrained LM, and as a consequence, runtime improvements are greater for better models.

Method: EvalAgent mines expert-authored online guidance to propose diverse, grounded evaluation criteria from reliable external sources.

Result: EvalAgent produces criteria that are often implicit, specific, and actionable - not satisfied by initial responses but can be used to refine them. Combining LLM-generated and EvalAgent criteria uncovers more human-valued criteria.

Conclusion: EvalAgent successfully identifies nuanced evaluation criteria that go beyond basic requirements, enabling better assessment and improvement of language model outputs on structured writing tasks.

Abstract: Evaluation of language model outputs on structured writing tasks is typically conducted with a number of desirable criteria presented to human evaluators or large language models (LLMs). For instance, on a prompt like “Help me draft an academic talk on coffee intake vs research productivity”, a model response may be evaluated for criteria like accuracy and coherence. However, high-quality responses should do more than just satisfy basic task requirements. An effective response to this query should include quintessential features of an academic talk, such as a compelling opening, clear research questions, and a takeaway. To help identify these implicit criteria, we introduce EvalAgent, a novel framework designed to automatically uncover nuanced and task-specific criteria. EvalAgent first mines expert-authored online guidance. It then uses this evidence to propose diverse, long-tail evaluation criteria that are grounded in reliable external sources. Our experiments demonstrate that the grounded criteria produced by EvalAgent are often implicit (not directly stated in the user’s prompt), yet specific (high degree of lexical precision). Further, EvalAgent criteria are often not satisfied by initial responses but they are actionable, such that responses can be refined to satisfy them. Finally, we show that combining LLM-generated and EvalAgent criteria uncovers more human-valued criteria than using LLMs alone.

Method: Interfaces LMs with symbolic environment as world model, uses policy model to propose actions, symbolic execution for verification, iterative correction to refine actions, and contrastive ranking to compare candidate plans.

Result: Outperforms pure natural language baselines on PlanBench, producing more coherent, diverse, and verifiable plans.

Conclusion: Combining symbolic environments with language models through structured interfaces significantly improves planning capabilities and robustness.

Abstract: Planning remains a core challenge for language models (LMs), particularly in domains that require coherent multi-step action sequences grounded in external constraints. We introduce SymPlanner, a novel framework that equips LMs with structured planning capabilities by interfacing them with a symbolic environment that serves as an explicit world model. Rather than relying purely on natural language reasoning, SymPlanner grounds the planning process in a symbolic state space, where a policy model proposes actions and a symbolic environment deterministically executes and verifies their effects. To enhance exploration and improve robustness, we introduce Iterative Correction (IC), which refines previously proposed actions by leveraging feedback from the symbolic environment to eliminate invalid decisions and guide the model toward valid alternatives. Additionally, Contrastive Ranking (CR) enables fine-grained comparison of candidate plans by evaluating them jointly. We evaluate SymPlanner on PlanBench, demonstrating that it produces more coherent, diverse, and verifiable plans than pure natural language baselines.

Method: Leverages Q-learning on large language models to bootstrap planning, determine optimal strategies based on long-term returns, and guide LLM responses

Result: Outperforms baselines including direct inference, self-refine, chain of thought, finetuning, and finite state machines on ESC datasets

Conclusion: The straQ* framework effectively addresses long-term satisfaction in emotional support conversations through Q-learning enhanced LLM planning

Abstract: Emotional support conversation (ESC) aims to alleviate the emotional distress of individuals through effective conversations. Although large language models (LLMs) have obtained remarkable progress on ESC, most of these studies might not define the diagram from the state model perspective, therefore providing a suboptimal solution for long-term satisfaction. To address such an issue, we leverage the Q-learning on LLMs, and propose a framework called straQ*. Our framework allows a plug-and-play LLM to bootstrap the planning during ESC, determine the optimal strategy based on long-term returns, and finally guide the LLM to response. Substantial experiments on ESC datasets suggest that straQ* outperforms many baselines, including direct inference, self-refine, chain of thought, finetuning, and finite state machines.

Method: Fine-tuned LLMs using only synthetic instructions and evaluated performance on benchmark dataset containing both synthetic and human-written instructions for spatial grounding tasks on 2.5D grid.

Result: Models generalize well on simple tasks but performance degrades significantly on more complex tasks when handling human-authored instructions.

Conclusion: There are significant gaps in instruction generalization that need to be addressed, particularly for complex spatial grounding tasks with human-written instructions.

Abstract: Instruction-tuned large language models (LLMs) have shown strong performance on a variety of tasks; however, generalizing from synthetic to human-authored instructions in grounded environments remains a challenge for them. In this work, we study generalization challenges in spatial grounding tasks where models interpret and translate instructions for building object arrangements on a $2.5$D grid. We fine-tune LLMs using only synthetic instructions and evaluate their performance on a benchmark dataset containing both synthetic and human-written instructions. Our results reveal that while models generalize well on simple tasks, their performance degrades significantly on more complex tasks. We present a detailed error analysis of the gaps in instruction generalization.

Method: Micro token-level Accept-Reject Aligning (MARA) decomposes sentence-level preference learning into token-level binary classification using a compact three-layer fully-connected network to determine if candidate tokens should be accepted or rejected in responses.

Result: Extensive experiments across seven different LLMs and three open-source datasets show MARA achieves significant improvements in alignment performance while reducing computational costs.

Conclusion: MARA provides an efficient and effective alternative to traditional alignment methods, operating independently of language models and achieving better performance with lower computational requirements.

Abstract: With the rapid development of Large Language Models (LLMs), aligning these models with human preferences and values is critical to ensuring ethical and safe applications. However, existing alignment techniques such as RLHF or DPO often require direct fine-tuning on LLMs with billions of parameters, resulting in substantial computational costs and inefficiencies. To address this, we propose Micro token-level Accept-Reject Aligning (MARA) approach designed to operate independently of the language models. MARA simplifies the alignment process by decomposing sentence-level preference learning into token-level binary classification, where a compact three-layer fully-connected network determines whether candidate tokens are “Accepted” or “Rejected” as part of the response. Extensive experiments across seven different LLMs and three open-source datasets show that MARA achieves significant improvements in alignment performance while reducing computational costs. The source code and implementation details are publicly available at https://github.com/IAAR-Shanghai/MARA, and the trained models are released at https://huggingface.co/IAAR-Shanghai/MARA_AGENTS.

Method: Developed intent-hiding adversarial prompting strategy that composes skills to conceal malicious intent. Created game-theoretic framework to model attack-defense interactions with prompt and response filtering. Proposed tailored defense mechanism against intent-hiding attacks.

Result: Attack strategy proved effective across multiple real-world LLMs for various malicious behaviors, showing clear advantages over existing adversarial prompting techniques. Game-theoretic analysis identified equilibrium points and structural advantages for attackers.

Conclusion: Intent-hiding adversarial prompting presents a scalable threat to LLM safety. The proposed defense mechanism offers a countermeasure, but structural advantages remain with attackers, highlighting the need for continued security research.

Abstract: As large language models (LLMs) grow more capable, concerns about their safe deployment have also grown. Although alignment mechanisms have been introduced to deter misuse, they remain vulnerable to carefully designed adversarial prompts. In this work, we present a scalable attack strategy: intent-hiding adversarial prompting, which conceals malicious intent through the composition of skills. We develop a game-theoretic framework to model the interaction between such attacks and defense systems that apply both prompt and response filtering. Our analysis identifies equilibrium points and reveals structural advantages for the attacker. To counter these threats, we propose and analyze a defense mechanism tailored to intent-hiding attacks. Empirically, we validate the attack’s effectiveness on multiple real-world LLMs across a range of malicious behaviors, demonstrating clear advantages over existing adversarial prompting techniques.

Method: Extends SMILE methodology with controlled prompt perturbations, Wasserstein distance metrics, and weighted linear surrogates to identify input tokens with significant impact on output, generating intuitive heatmaps.

Result: Evaluated across GPT-3.5-turbo-instruct, LLaMA 3.1 Instruct Turbo, and Claude 2.1, gSMILE delivers reliable human-aligned attributions with Claude 2.1 excelling in attention fidelity and GPT-3.5 achieving highest output consistency.

Conclusion: gSMILE balances model performance and interpretability, enabling more transparent and trustworthy AI systems through reliable token-level attribution analysis.

Abstract: Large Language Models (LLMs) such as GPT, LLaMA, and Claude achieve remarkable performance in text generation but remain opaque in their decision-making processes, limiting trust and accountability in high-stakes applications. We present gSMILE (generative SMILE), a model-agnostic, perturbation-based framework for token-level interpretability in LLMs. Extending the SMILE methodology, gSMILE uses controlled prompt perturbations, Wasserstein distance metrics, and weighted linear surrogates to identify input tokens with the most significant impact on the output. This process enables the generation of intuitive heatmaps that visually highlight influential tokens and reasoning paths. We evaluate gSMILE across leading LLMs (OpenAI’s gpt-3.5-turbo-instruct, Meta’s LLaMA 3.1 Instruct Turbo, and Anthropic’s Claude 2.1) using attribution fidelity, attribution consistency, attribution stability, attribution faithfulness, and attribution accuracy as metrics. Results show that gSMILE delivers reliable human-aligned attributions, with Claude 2.1 excelling in attention fidelity and GPT-3.5 achieving the highest output consistency. These findings demonstrate gSMILE’s ability to balance model performance and interpretability, enabling more transparent and trustworthy AI systems.

Method: Analysis and reflection based on recent studies and user experiences, proposing a “duality” hypothesis that LLMs’ translation abilities originate from two types of pre-training data internalized differently.

Result: The paper suggests LLMs can leverage semantically similar monolingual content across different languages that wouldn’t fit in a single context window, indicating sophisticated cross-lingual alignment capabilities.

Conclusion: LLMs’ translation abilities likely emerge from incidental bilingual content in training data, with implications for reconceptualizing both human and machine translation in the deep learning era.

Abstract: Large Language Models (LLMs) excel in translation among other things, demonstrating competitive performance for many language pairs in zero- and few-shot settings. But unlike dedicated neural machine translation models, LLMs are not trained on any translation-related objective. What explains their remarkable translation abilities? Are these abilities grounded in “incidental bilingualism” (Briakou et al. 2023) in training data? Does instruction tuning contribute to it? Are LLMs capable of aligning and leveraging semantically identical or similar monolingual contents from different corners of the internet that are unlikely to fit in a single context window? I offer some reflections on this topic, informed by recent studies and growing user experience. My working hypothesis is that LLMs’ translation abilities originate in two different types of pre-training data that may be internalized by the models in different ways. I discuss the prospects for testing the “duality” hypothesis empirically and its implications for reconceptualizing translation, human and machine, in the age of deep learning.

Method: 1) Analyze cosine similarity between normalized hidden states of sink tokens and other tokens across layers 2) Propose OrthoRank - dynamic token selection method that defines token importance by speed of movement toward sink token, converted into orthogonality with sink token

Result: Experiments show OrthoRank achieves lower perplexity, higher zero-shot accuracy compared to layer pruning methods at same sparsity ratio with comparable throughput, and superior performance on LongBench benchmark

Conclusion: Sink tokens consistently attract other tokens throughout layers, and leveraging this phenomenon through orthogonality-based token selection (OrthoRank) provides an effective method for improving LLM performance while maintaining efficiency

Abstract: Attention mechanisms are central to the success of large language models (LLMs), enabling them to capture intricate token dependencies and implicitly assign importance to each token. Recent studies have revealed the sink token, which receives disproportionately high attention despite their limited semantic role. In this paper, we first expand the relationship between the sink token and other tokens, moving beyond attention to explore their similarity in hidden states, considering the layer depth. We observe that as the layers get deeper, the cosine similarity between the normalized hidden states of the sink token and those of other tokens increases, and that the normalized hidden states of the sink token exhibit negligible changes. These imply that other tokens consistently are directed toward the sink token throughout the layers. Next, we propose a dynamic token selection method, called OrthoRank, using these findings to select important tokens. Specifically, in a certain layer, we define token importance by the speed at which the token moves toward the sink token. This is converted into orthogonality with the sink token, meaning that tokens that are more orthogonal to the sink token are assigned greater importance. Finally, through extensive experiments, we demonstrated that our method results in lower perplexity and higher zero-shot accuracy compared to layer pruning methods at the same sparsity ratio with comparable throughput, while also achieving superior performance on LongBench.

Method: Proposes LoRA-Augmented Generation (LAG) which filters, retrieves, and applies LoRA adapters on a per-token and layer basis without requiring additional training or data access.

Result: Achieves superior performance over existing data-free methods on various knowledge-intensive tasks and demonstrates compatibility with retrieval-augmented generation (RAG) when additional data is available.

Conclusion: LAG provides an effective solution for leveraging large libraries of LoRA adapters, offering strong performance in both data-free and data-available scenarios while maintaining compatibility with existing augmentation methods.

Abstract: The proliferation of fine-tuned language model experts for specific tasks and domains signals the need for efficient selection and combination methods. We propose LoRA-Augmented Generation (LAG) for leveraging large libraries of knowledge and task-specific LoRA adapters. LAG requires no additional training or access to data, and efficiently filters, retrieves, and applies experts on a per-token and layer basis. We evaluate LAG on various knowledge-intensive tasks, achieving superior performance over existing data-free methods. We explore scenarios where additional data is available, demonstrating LAG’s compatibility with alternative solutions such as retrieval-augmented generation (RAG).

Method: PromptSuite uses a modular prompt design that allows controlled perturbations to each component, is extensible to support new components and perturbation types, and works out-of-the-box across various tasks and benchmarks.

Result: Case studies demonstrate that PromptSuite provides meaningful prompt variations that support strong evaluation practices, making robust multi-prompt evaluation more accessible.

Conclusion: PromptSuite offers both a Python API and web interface to facilitate automatic generation of diverse prompts, enabling more reliable and comprehensive evaluation of LLM performance across different tasks.

Abstract: Evaluating LLMs with a single prompt has proven unreliable, with small changes leading to significant performance differences. However, generating the prompt variations needed for a more robust multi-prompt evaluation is challenging, limiting its adoption in practice. To address this, we introduce PromptSuite, a framework that enables the automatic generation of various prompts. PromptSuite is flexible - working out of the box on a wide range of tasks and benchmarks. It follows a modular prompt design, allowing controlled perturbations to each component, and is extensible, supporting the addition of new components and perturbation types. Through a series of case studies, we show that PromptSuite provides meaningful variations to support strong evaluation practices. It is available through both a Python API: https://github.com/eliyahabba/PromptSuite, and a user-friendly web interface: https://promptsuite.streamlit.app/

Method: Two-phase training-free denoising strategy: 1) Before denoising - iteratively expand from short initial length to coarse task-appropriate length using sequence completion metric; 2) During denoising - dynamically expand insufficient generation regions through mask token insertion.

Result: DAEDAL achieves performance comparable or superior to fixed-length baselines while enhancing computational efficiency through higher effective token ratio. It resolves the static length constraint without additional training.

Conclusion: DAEDAL unlocks new potential for Diffusion LLMs by bridging a critical gap with Autoregressive counterparts, enabling more efficient and capable generation through dynamic length adaptation.

Abstract: Diffusion Large Language Models (DLLMs) are emerging as a powerful alternative to the dominant Autoregressive Large Language Models, offering efficient parallel generation and capable global context modeling. However, the practical application of DLLMs is hindered by a critical architectural constraint: the need for a statically predefined generation length. This static length allocation leads to a problematic trade-off: insufficient lengths cripple performance on complex tasks, while excessive lengths incur significant computational overhead and sometimes result in performance degradation. While the inference framework is rigid, we observe that the model itself possesses internal signals that correlate with the optimal response length for a given task. To bridge this gap, we leverage these latent signals and introduce DAEDAL, a novel training-free denoising strategy that enables Dynamic Adaptive Length Expansion for Diffusion Large Language Models. DAEDAL operates in two phases: 1) Before the denoising process, DAEDAL starts from a short initial length and iteratively expands it to a coarse task-appropriate length, guided by a sequence completion metric. 2) During the denoising process, DAEDAL dynamically intervenes by pinpointing and expanding insufficient generation regions through mask token insertion, ensuring the final output is fully developed. Extensive experiments on DLLMs demonstrate that DAEDAL achieves performance comparable, and in some cases superior, to meticulously tuned fixed-length baselines, while simultaneously enhancing computational efficiency by achieving a higher effective token ratio. By resolving the static length constraint, DAEDAL unlocks new potential for DLLMs, bridging a critical gap with their Autoregressive counterparts and paving the way for more efficient and capable generation.

Method: Proposes Dynamic Entropy Weighting with two approaches: 1) GTPO assigns entropy-weighted rewards to individual tokens, 2) GRPO-S assigns entropy-weighted rewards to sequences based on average token entropy.

Result: Significantly outperforms the strong DAPO baseline, demonstrating superior performance in reasoning tasks.

Conclusion: Entropy-weighting mechanism is the key driver for performance improvement, offering a better approach to enhance deep reasoning capabilities in language models.

Abstract: Reinforcement learning (RL) with algorithms like Group Relative Policy Optimization (GRPO) improves Large Language Model (LLM) reasoning, but is limited by a coarse-grained credit assignment that applies a uniform reward to all tokens in a sequence. This is a major flaw in long-chain reasoning tasks. This paper solves this with \textbf{Dynamic Entropy Weighting}. Our core idea is that high-entropy tokens in correct responses can guide the policy toward a higher performance ceiling. This allows us to create more fine-grained reward signals for precise policy updates via two ways: 1) \textbf{Group Token Policy Optimization} (\textbf{GTPO}), we assigns a entropy-weighted reward to each token for fine-grained credit assignment. 2) \textbf{Sequence-Level Group Relative Policy Optimization} (\textbf{GRPO-S}), we assigns a entropy-weighted reward to each sequence based on its average token entropy. Experiments show our methods significantly outperform the strong DAPO baseline. The results confirm that our entropy-weighting mechanism is the key driver of this performance boost, offering a better path to enhance deep reasoning in models.

Method: Finetuned four distinct LLM architectures (Phi 3.5, Gemma 2, LLaMA 3.2, DeepSeek R1 LLaMA Distilled) using the LDC2020T02 Gold AMR3.0 test set for AMR parsing evaluation.

Result: LLaMA 3.2 achieved SMATCH F1: 0.804, on par with APT + Silver (IBM) and approaching Graphene Smatch (MBSE) at 0.854. LLaMA 3.2 led in semantic performance while Phi 3.5 excelled in structural validity.

Conclusion: Straightforward finetuning of decoder-only LLMs can achieve comparable performance to complex SOTA AMR parsers, making this a promising direction for semantic parsing.

Abstract: AMR (Abstract Meaning Representation) is a semantic formalism that encodes sentence meaning as rooted, directed, acyclic graphs, where nodes represent concepts and edges denote semantic relations. Finetuning decoder only Large Language Models (LLMs) represent a promising novel straightfoward direction for AMR parsing. This paper presents a comprehensive evaluation of finetuning four distinct LLM architectures, Phi 3.5, Gemma 2, LLaMA 3.2, and DeepSeek R1 LLaMA Distilled using the LDC2020T02 Gold AMR3.0 test set. Our results have shown that straightfoward finetuning of decoder only LLMs can achieve comparable performance to complex State of the Art (SOTA) AMR parsers. Notably, LLaMA 3.2 demonstrates competitive performance against SOTA AMR parsers given a straightforward finetuning approach. We achieved SMATCH F1: 0.804 on the full LDC2020T02 test split, on par with APT + Silver (IBM) at 0.804 and approaching Graphene Smatch (MBSE) at 0.854. Across our analysis, we also observed a consistent pattern where LLaMA 3.2 leads in semantic performance while Phi 3.5 excels in structural validity.

Method: Matrix-Driven Instant Review (MDIR) leverages matrix analysis and Large Deviation Theory to accurately reconstruct weight relationships, provide rigorous p-value estimation, and focus exclusively on weight similarity without requiring full model inference.

Result: MDIR reliably detects plagiarism even after extensive transformations including random permutations and continual pretraining with trillions of tokens. All detections can be performed on a single PC within an hour, making it both efficient and accessible.

Conclusion: MDIR addresses critical limitations of existing LLM plagiarism detection methods by providing accurate, statistically rigorous, and efficient detection capabilities that work even after significant model transformations.

Abstract: In recent years, concerns about intellectual property (IP) in large language models (LLMs) have grown significantly. Plagiarizing other LLMs (through direct weight copying, upcycling, pruning, or continual pretraining) and claiming authorship without properly attributing to the original license, is a serious misconduct that can lead to significant financial and reputational harm to the original developers. However, existing methods for detecting LLM plagiarism fall short in key areas. They fail to accurately reconstruct weight correspondences, lack the ability to compute statistical significance measures such as $p$-values, and may mistakenly flag models trained on similar data as being related. To address these limitations, we propose Matrix-Driven Instant Review (MDIR), a novel method that leverages matrix analysis and Large Deviation Theory. MDIR achieves accurate reconstruction of weight relationships, provides rigorous $p$-value estimation, and focuses exclusively on weight similarity without requiring full model inference. Experimental results demonstrate that MDIR reliably detects plagiarism even after extensive transformations, such as random permutations and continual pretraining with trillions of tokens. Moreover, all detections can be performed on a single PC within an hour, making MDIR both efficient and accessible.

Method: Investigates four aspects: exploration space shaping with quantitative metrics, entropy-performance exchange analysis across training stages and token patterns, and RL performance optimization methods.

Result: Develops quantitative metrics for LLM capability boundaries and examines how exploration gains translate into measurable improvements.

Conclusion: Provides foundational framework unifying previous insights with new evidence to advance RLVR systems and understanding of LLM exploration behaviors.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs). Unlike traditional RL approaches, RLVR leverages rule-based feedback to guide LLMs in generating and refining complex reasoning chains – a process critically dependent on effective exploration strategies. While prior work has demonstrated RLVR’s empirical success, the fundamental mechanisms governing LLMs’ exploration behaviors remain underexplored. This technical report presents a systematic investigation of exploration capacities in RLVR, covering four main aspects: (1) exploration space shaping, where we develop quantitative metrics to characterize LLMs’ capability boundaries; (2) entropy-performance exchange, analyzed across training stages, individual instances, and token-level patterns; and (3) RL performance optimization, examining methods to effectively translate exploration gains into measurable improvements. By unifying previously identified insights with new empirical evidence, this work aims to provide a foundational framework for advancing RLVR systems.

Method: Used contrastive stimulus sets to localize top-activated units across 11 LLMs and 5 VLMs (3B-90B parameters), then performed targeted ablations to assess causal role. Compared effects of lesioning functionally selected units against low-activation and randomly selected units on downstream task accuracy.

Result: Contrary to expectations, low-activation units sometimes produced larger performance drops than highly activated ones. Units from mathematical localizer often impaired ToM performance more than those from ToM localizer. Results question causal relevance of contrast-based localizers.

Conclusion: The findings challenge the assumption that contrast-based localizers accurately identify task-specific causal units, highlighting the need for broader stimulus sets and better methods to capture truly task-relevant units in large models.

Abstract: This work adapts a neuroscientific contrast localizer to pinpoint causally relevant units for Theory of Mind (ToM) and mathematical reasoning tasks in large language models (LLMs) and vision-language models (VLMs). Across 11 LLMs and 5 VLMs ranging in size from 3B to 90B parameters, we localize top-activated units using contrastive stimulus sets and assess their causal role via targeted ablations. We compare the effect of lesioning functionally selected units against low-activation and randomly selected units on downstream accuracy across established ToM and mathematical benchmarks. Contrary to expectations, low-activation units sometimes produced larger performance drops than the highly activated ones, and units derived from the mathematical localizer often impaired ToM performance more than those from the ToM localizer. These findings call into question the causal relevance of contrast-based localizers and highlight the need for broader stimulus sets and more accurately capture task-specific units.

Method: Conducted a comprehensive survey of clinical mental health datasets, categorizing them by mental disorders (e.g., depression, schizophrenia), data modalities (text, speech, physiological signals), task types (diagnosis prediction, symptom severity estimation), accessibility (public/restricted/private), and sociocultural context (language, cultural background). Also investigated synthetic datasets.

Result: Identified critical gaps including lack of longitudinal data, limited cultural and linguistic representation, inconsistent collection and annotation standards, and insufficient modalities in synthetic data. The survey provides a structured categorization of existing datasets.

Conclusion: Outlines key challenges in curating and standardizing future datasets and provides actionable recommendations to facilitate development of more robust, generalizable, and equitable mental health AI systems.

Abstract: Mental health disorders are rising worldwide. However, the availability of trained clinicians has not scaled proportionally, leaving many people without adequate or timely support. To bridge this gap, recent studies have shown the promise of Artificial Intelligence (AI) to assist mental health diagnosis, monitoring, and intervention. However, the development of efficient, reliable, and ethical AI to assist clinicians is heavily dependent on high-quality clinical training datasets. Despite growing interest in data curation for training clinical AI assistants, existing datasets largely remain scattered, under-documented, and often inaccessible, hindering the reproducibility, comparability, and generalizability of AI models developed for clinical mental health care. In this paper, we present the first comprehensive survey of clinical mental health datasets relevant to the training and development of AI-powered clinical assistants. We categorize these datasets by mental disorders (e.g., depression, schizophrenia), data modalities (e.g., text, speech, physiological signals), task types (e.g., diagnosis prediction, symptom severity estimation, intervention generation), accessibility (public, restricted or private), and sociocultural context (e.g., language and cultural background). Along with these, we also investigate synthetic clinical mental health datasets. Our survey identifies critical gaps such as a lack of longitudinal data, limited cultural and linguistic representation, inconsistent collection and annotation standards, and a lack of modalities in synthetic data. We conclude by outlining key challenges in curating and standardizing future datasets and provide actionable recommendations to facilitate the development of more robust, generalizable, and equitable mental health AI systems.

Method: Proposes a neural contextual bandit-based algorithm that trains neural networks to model LLM success on each subtask in an online manner, learning to guide LLM selections for different subtasks without requiring historical performance data.

Result: Experiments on telecommunications question answering and medical diagnosis prediction datasets show the approach is effective compared to other LLM selection algorithms.

Conclusion: The proposed algorithm successfully addresses the complex problem of selecting optimal LLM sequences for multi-step tasks by learning performance dependencies in real-time, outperforming existing single-LLM selection methods.

Abstract: With the increasing popularity of large language models (LLMs) for a variety of tasks, there has been a growing interest in strategies that can predict which out of a set of LLMs will yield a successful answer at low cost. This problem promises to become more and more relevant as providers like Microsoft allow users to easily create custom LLM “assistants” specialized to particular types of queries. However, some tasks (i.e., queries) may be too specialized and difficult for a single LLM to handle alone. These applications often benefit from breaking down the task into smaller subtasks, each of which can then be executed by a LLM expected to perform well on that specific subtask. For example, in extracting a diagnosis from medical records, one can first select an LLM to summarize the record, select another to validate the summary, and then select another, possibly different, LLM to extract the diagnosis from the summarized record. Unlike existing LLM selection or routing algorithms, this setting requires that we select a sequence of LLMs, with the output of each LLM feeding into the next and potentially influencing its success. Thus, unlike single LLM selection, the quality of each subtask’s output directly affects the inputs, and hence the cost and success rate, of downstream LLMs, creating complex performance dependencies that must be learned and accounted for during selection. We propose a neural contextual bandit-based algorithm that trains neural networks that model LLM success on each subtask in an online manner, thus learning to guide the LLM selections for the different subtasks, even in the absence of historical LLM performance data. Experiments on telecommunications question answering and medical diagnosis prediction datasets illustrate the effectiveness of our proposed approach compared to other LLM selection algorithms.

Method: Used DementiaBank cookie-theft task transcripts (n=237), evaluated 10 transformer models with 3 fine-tuning strategies, fused top transformer embeddings with 110 linguistic features, generated synthetic speech using 5 LLMs for data augmentation, and tested 3 multimodal models for speech-text classification.

Result: Fusion model achieved F1=83.3 (AUC=89.5), outperforming baseline methods. Data augmentation with MedAlpaca-7B synthetic speech increased F1 to 85.7. Fine-tuning improved unimodal LLM classifiers significantly (MedAlpaca: F1=47.3→78.5). Multimodal models showed lower performance (GPT-4o=70.2 F1).

Conclusion: Integration of transformer embeddings with linguistic features enhances ADRD detection. Clinically tuned LLMs effectively support classification and data augmentation, but multimodal modeling requires further advancement.

Abstract: Alzheimer’s disease and related dementias (ADRD) affect approximately five million older adults in the U.S., yet over half remain undiagnosed. Speech-based natural language processing (NLP) offers a promising, scalable approach to detect early cognitive decline through linguistic markers. To develop and evaluate a screening pipeline that (i) fuses transformer embeddings with handcrafted linguistic features, (ii) tests data augmentation using synthetic speech generated by large language models (LLMs), and (iii) benchmarks unimodal and multimodal LLM classifiers for ADRD detection. Transcripts from the DementiaBank “cookie-theft” task (n = 237) were used. Ten transformer models were evaluated under three fine-tuning strategies. A fusion model combined embeddings from the top-performing transformer with 110 lexical-derived linguistic features. Five LLMs (LLaMA-8B/70B, MedAlpaca-7B, Ministral-8B, GPT-4o) were fine-tuned to generate label-conditioned synthetic speech, which was used to augment training data. Three multimodal models (GPT-4o, Qwen-Omni, Phi-4) were tested for speech-text classification in zero-shot and fine-tuned settings. The fusion model achieved F1 = 83.3 (AUC = 89.5), outperforming linguistic or transformer-only baselines. Augmenting training data with 2x MedAlpaca-7B synthetic speech increased F1 to 85.7. Fine-tuning significantly improved unimodal LLM classifiers (e.g., MedAlpaca: F1 = 47.3 -> 78.5 F1). Current multimodal models demonstrated lower performance (GPT-4o = 70.2 F1; Qwen = 66.0). Performance gains aligned with the distributional similarity between synthetic and real speech. Integrating transformer embeddings with linguistic features enhances ADRD detection from speech. Clinically tuned LLMs effectively support both classification and data augmentation, while further advancement is needed in multimodal modeling.

Method: Leverages foundation model representations to compute influence scores through a closed-form expression based on a single forward pass, eliminating gradient computations while capturing alignment in hidden representations and prediction errors.

Result: For-Value matches or outperforms gradient-based baselines in identifying impactful fine-tuning examples and effectively detects mislabeled data while being computationally efficient.

Conclusion: The framework provides scalable and efficient influence estimation for large models, enhancing transparency and accountability without the computational burden of traditional methods.

Abstract: Quantifying the influence of individual training samples is essential for enhancing the transparency and accountability of large language models (LLMs) and vision-language models (VLMs). However, existing data valuation methods often rely on Hessian information or model retraining, making them computationally prohibitive for billion-parameter models. In this work, we introduce For-Value, a forward-only data valuation framework that enables scalable and efficient influence estimation for both LLMs and VLMs. By leveraging the rich representations of modern foundation models, For-Value computes influence scores using a simple closed-form expression based solely on a single forward pass, thereby eliminating the need for costly gradient computations. Our theoretical analysis demonstrates that For-Value accurately estimates per-sample influence by capturing alignment in hidden representations and prediction errors between training and validation samples. Extensive experiments show that For-Value matches or outperforms gradient-based baselines in identifying impactful fine-tuning examples and effectively detecting mislabeled data.

Method: Three-stage structured approach: content extraction from submissions, retrieval and synthesis of related work, and structured comparison for evidence-based assessment, informed by large-scale analysis of human novelty reviews.

Result: Achieves 86.5% alignment with human reasoning and 75.3% agreement on novelty conclusions on 182 ICLR 2025 submissions, substantially outperforming existing LLM baselines while producing detailed literature-aware analyses.

Conclusion: Structured LLM-assisted approaches can support more rigorous and transparent peer review without displacing human expertise, demonstrating significant potential for automated novelty assessment systems.

Abstract: Novelty assessment is a central yet understudied aspect of peer review, particularly in high volume fields like NLP where reviewer capacity is increasingly strained. We present a structured approach for automated novelty evaluation that models expert reviewer behavior through three stages: content extraction from submissions, retrieval and synthesis of related work, and structured comparison for evidence based assessment. Our method is informed by a large scale analysis of human written novelty reviews and captures key patterns such as independent claim verification and contextual reasoning. Evaluated on 182 ICLR 2025 submissions with human annotated reviewer novelty assessments, the approach achieves 86.5% alignment with human reasoning and 75.3% agreement on novelty conclusions - substantially outperforming existing LLM based baselines. The method produces detailed, literature aware analyses and improves consistency over ad hoc reviewer judgments. These results highlight the potential for structured LLM assisted approaches to support more rigorous and transparent peer review without displacing human expertise. Data and code are made available.

Method: Proposed a verifier-based inference-time scaling method that uses off-the-shelf pre-trained embedding models as simple soft-value-based verifiers to guide the denoising process and find better candidate generations

Result: Experiments show significant gains in generation quality for text-style transfer tasks, establishing MDMs as better alternatives to autoregressive language models, even when used on top of existing classifier-free guidance setups

Conclusion: Verifier-based inference-time scaling effectively enhances MDM performance, making them superior non-autoregressive generators for discrete data with improved scalability and training ease

Abstract: Masked diffusion language models (MDMs) have recently gained traction as a viable generative framework for natural language. This can be attributed to its scalability and ease of training compared to other diffusion model paradigms for discrete data, establishing itself as the state-of-the-art non-autoregressive generator for discrete data. Diffusion models, in general, have shown excellent ability to improve the generation quality by leveraging inference-time scaling either by increasing the number of denoising steps or by using external verifiers on top of the outputs of each step to guide the generation. In this work, we propose a verifier-based inference-time scaling method that aids in finding a better candidate generation during the denoising process of the MDM. Our experiments demonstrate the application of MDMs for standard text-style transfer tasks and establish MDMs as a better alternative to autoregressive language models. Additionally, we show that a simple soft-value-based verifier setup for MDMs using off-the-shelf pre-trained embedding models leads to significant gains in generation quality even when used on top of typical classifier-free guidance setups in the existing literature.

Method: Reframe detection as a signal processing problem by analyzing token log-probability sequences in frequency domain using global Discrete Fourier Transform (DFT) and local Short-Time Fourier Transform (STFT), finding human text has higher spectral energy.

Result: SpecDetect (using DFT total energy) and enhanced SpecDetect++ outperform state-of-the-art models while running in nearly half the time, demonstrating superior efficiency and performance.

Conclusion: Classical signal processing techniques provide an efficient, interpretable pathway for LLM-generated text detection, with spectral energy analysis offering a powerful solution to modern detection challenges.

Abstract: The proliferation of high-quality text from Large Language Models (LLMs) demands reliable and efficient detection methods. While existing training-free approaches show promise, they often rely on surface-level statistics and overlook fundamental signal properties of the text generation process. In this work, we reframe detection as a signal processing problem, introducing a novel paradigm that analyzes the sequence of token log-probabilities in the frequency domain. By systematically analyzing the signal’s spectral properties using the global Discrete Fourier Transform (DFT) and the local Short-Time Fourier Transform (STFT), we find that human-written text consistently exhibits significantly higher spectral energy. This higher energy reflects the larger-amplitude fluctuations inherent in human writing compared to the suppressed dynamics of LLM-generated text. Based on this key insight, we construct SpecDetect, a detector built on a single, robust feature from the global DFT: DFT total energy. We also propose an enhanced version, SpecDetect++, which incorporates a sampling discrepancy mechanism to further boost robustness. Extensive experiments demonstrate that our approach outperforms the state-of-the-art model while running in nearly half the time. Our work introduces a new, efficient, and interpretable pathway for LLM-generated text detection, showing that classical signal processing techniques offer a surprisingly powerful solution to this modern challenge.

Method: Evaluated YOLOv8, YOLOv11, and YOLOv12 models, then developed a custom YOLOv8-based model with additional structures for challenging surveillance contexts. Used dataset of 8,124 images from 20 scenarios with 2,708 low-light samples. Tested on edge devices with multithreaded operations.

Result: Proposed custom model outperformed others with 78.90% recall and 83.70% mAP@50. Jetson Xavier NX achieved 52-97ms per inference, suitable for real-time operations.

Conclusion: The system provides a robust and adaptable platform for real-time smoking detection in surveillance contexts, effectively addressing safety concerns in fire exit areas.

Abstract: A deep learning real-time smoking detection system for CCTV surveillance of fire exit areas is proposed due to critical safety requirements. The dataset contains 8,124 images from 20 different scenarios along with 2,708 raw samples demonstrating low-light areas. We evaluated three advanced object detection models: YOLOv8, YOLOv11, and YOLOv12, followed by development of a custom model derived from YOLOv8 with added structures for challenging surveillance contexts. The proposed model outperformed the others, achieving a recall of 78.90 percent and mAP at 50 of 83.70 percent, delivering optimal object detection across varied environments. Performance evaluation on multiple edge devices using multithreaded operations showed the Jetson Xavier NX processed data at 52 to 97 milliseconds per inference, establishing its suitability for time-sensitive operations. This system offers a robust and adaptable platform for monitoring public safety and enabling automatic regulatory compliance.

Method: Train representation models exclusively on procedural data and apply them to visual tasks using visual memory - an explicit database of reference image embeddings, without further training on real images.

Result: Performs within 1% on NIGHTS visual similarity, outperforms by 8% and 15% on CUB200 and Flowers102 classification, within 10% on ImageNet-1K, and achieves strong zero-shot segmentation (R² on COCO within 10% of real-data models).

Conclusion: Procedural models can achieve competitive performance while being fully compartmentalized from real images, with remaining performance gaps attributed to dissimilar representations of object parts in procedural models causing incorrect memory searches.

Abstract: We train representation models with procedural data only, and apply them on visual similarity, classification, and semantic segmentation tasks without further training by using visual memory – an explicit database of reference image embeddings. Unlike prior work on visual memory, our approach achieves full compartmentalization with respect to all real-world images while retaining strong performance. Compared to a model trained on Places, our procedural model performs within $1%$ on NIGHTS visual similarity, outperforms by $8%$ and $15%$ on CUB200 and Flowers102 fine-grained classification, and is within $10%$ on ImageNet-1K classification. It also demonstrates strong zero-shot segmentation, achieving an $R^2$ on COCO within $10%$ of the models trained on real data. Finally, we analyze procedural versus real data models, showing that parts of the same object have dissimilar representations in procedural models, resulting in incorrect searches in memory and explaining the remaining performance gap.

Method: Proposed FusionFM evaluation suite with two fusion approaches to integrate different ophthalmic FMs. Evaluated on both ophthalmic disease detection (glaucoma, diabetic retinopathy, AMD) and systemic disease prediction (diabetes, hypertension) using retinal imaging from standardized multi-country datasets. Used AUC and F1 metrics for benchmarking.

Result: DINORET and RetiZero achieved superior performance in both ophthalmic and systemic disease tasks. RetiZero showed stronger generalization on external datasets. Gating-based fusion provided modest improvements for glaucoma, AMD, and hypertension prediction. Systemic disease prediction, especially hypertension in external cohorts, remains challenging.

Conclusion: This study provides evidence-based evaluation of ophthalmic FMs, demonstrates benefits of model fusion, and identifies strategies for enhancing clinical applicability, though systemic disease prediction remains a challenge requiring further improvement.

Abstract: Foundation models (FMs) have shown great promise in medical image analysis by improving generalization across diverse downstream tasks. In ophthalmology, several FMs have recently emerged, but there is still no clear answer to fundamental questions: Which FM performs the best? Are they equally good across different tasks? What if we combine all FMs together? To our knowledge, this is the first study to systematically evaluate both single and fused ophthalmic FMs. To address these questions, we propose FusionFM, a comprehensive evaluation suite, along with two fusion approaches to integrate different ophthalmic FMs. Our framework covers both ophthalmic disease detection (glaucoma, diabetic retinopathy, and age-related macular degeneration) and systemic disease prediction (diabetes and hypertension) based on retinal imaging. We benchmarked four state-of-the-art FMs (RETFound, VisionFM, RetiZero, and DINORET) using standardized datasets from multiple countries and evaluated their performance using AUC and F1 metrics. Our results show that DINORET and RetiZero achieve superior performance in both ophthalmic and systemic disease tasks, with RetiZero exhibiting stronger generalization on external datasets. Regarding fusion strategies, the Gating-based approach provides modest improvements in predicting glaucoma, AMD, and hypertension. Despite these advances, predicting systemic diseases, especially hypertension in external cohort remains challenging. These findings provide an evidence-based evaluation of ophthalmic FMs, highlight the benefits of model fusion, and point to strategies for enhancing their clinical applicability.

Method: UniDCF uses multimodal fusion encoding of point clouds and multi-view images, leveraging complementary strengths of each modality with a score-based denoising module to refine surface smoothness. The framework was trained on the largest multimodal dataset with 54,555 annotated instances from 6,609 patients.

Result: UniDCF outperforms state-of-the-art methods in geometric precision, structural completeness, and spatial accuracy. It reduces reconstruction design time by 99% and achieves clinician-rated acceptability exceeding 94%.

Conclusion: UniDCF enables rapid, automated, high-fidelity reconstruction of multiple dentocraniofacial tissues, supporting personalized restorative treatments, streamlining clinical workflows, and enhancing patient outcomes.

Abstract: Dentocraniofacial hard tissue defects profoundly affect patients' physiological functions, facial aesthetics, and psychological well-being, posing significant challenges for precise reconstruction. Current deep learning models are limited to single-tissue scenarios and modality-specific imaging inputs, resulting in poor generalizability and trade-offs between anatomical fidelity, computational efficiency, and cross-tissue adaptability. Here we introduce UniDCF, a unified framework capable of reconstructing multiple dentocraniofacial hard tissues through multimodal fusion encoding of point clouds and multi-view images. By leveraging the complementary strengths of each modality and incorporating a score-based denoising module to refine surface smoothness, UniDCF overcomes the limitations of prior single-modality approaches. We curated the largest multimodal dataset, comprising intraoral scans, CBCT, and CT from 6,609 patients, resulting in 54,555 annotated instances. Evaluations demonstrate that UniDCF outperforms existing state-of-the-art methods in terms of geometric precision, structural completeness, and spatial accuracy. Clinical simulations indicate UniDCF reduces reconstruction design time by 99% and achieves clinician-rated acceptability exceeding 94%. Overall, UniDCF enables rapid, automated, and high-fidelity reconstruction, supporting personalized and precise restorative treatments, streamlining clinical workflows, and enhancing patient outcomes.

Method: Integrates native-resolution vision transformer for variable-resolution image processing, implements reflection-based reasoning with self-checking and revision, uses a five-phase curriculum training (foundational pretraining, instruction tuning, alignment with DPO/GRPO), and employs multimodal data packing with hybrid parallelism for efficiency.

Result: Ovis2.5-9B scores 78.3 on OpenCompass leaderboard (substantial improvement over Ovis2-8B), Ovis2.5-2B scores 73.9 (SOTA for its size), achieves leading results on STEM benchmarks, strong grounding/video capabilities, and SOTA for complex chart analysis at its scale.

Conclusion: Ovis2.5 successfully addresses native-resolution visual perception challenges and enhances multimodal reasoning through reflection, establishing new state-of-the-art performance for open-source models in the sub-40B parameter range while maintaining efficiency for resource-constrained applications.

Abstract: We present Ovis2.5, a successor to Ovis2 designed for native-resolution visual perception and strong multimodal reasoning. Ovis2.5 integrates a native-resolution vision transformer that processes images at their native, variable resolutions, avoiding the degradation from fixed-resolution tiling and preserving both fine detail and global layout – crucial for visually dense content like complex charts. To strengthen reasoning, we train the model to move beyond linear chain-of-thought and perform reflection – including self-checking and revision. This advanced capability is exposed as an optional “thinking mode” at inference time, allowing users to trade latency for enhanced accuracy on difficult inputs. The model is trained via a comprehensive five-phase curriculum that progressively builds its skills. The process begins with foundational visual and multimodal pretraining, advances through large-scale instruction tuning, and culminates in alignment and reasoning enhancement using DPO and GRPO. To scale these upgrades efficiently, we employ multimodal data packing and hybrid parallelism, yielding a significant end-to-end speedup. We release two open-source models: Ovis2.5-9B and Ovis2.5-2B. The latter continues the “small model, big performance” philosophy of Ovis2, making it ideal for resource-constrained, on-device scenarios. On the OpenCompass multimodal leaderboard, Ovis2.5-9B averages 78.3, marking a substantial improvement over its predecessor, Ovis2-8B, and achieving state-of-the-art results among open-source MLLMs in the sub-40B parameter range; Ovis2.5-2B scores 73.9, establishing SOTA for its size. Beyond aggregate scores, Ovis2.5 achieves leading results on STEM benchmarks, exhibits strong capabilities on grounding and video tasks, and achieves open-source SOTA at its scale for complex chart analysis.

Method: 1) Prompt optimization framework varying prompt structure, supervision granularity, and training modality; 2) Multimodal data augmentation pipeline generating 2,479 counterfactually neutral memes using multi-agent LLM-VLM setup to isolate and rewrite hateful modality.

Result: Structured prompts improve robustness even in small models, InternVL2 achieves best F1-scores across binary and scaled settings, and the augmentation pipeline successfully reduces spurious correlations and improves classifier generalization.

Conclusion: Prompt structure and data composition are as critical as model size, and targeted augmentation can support more trustworthy and context-sensitive hate detection, inspiring new directions for building synthetic data to train robust vision-language models.

Abstract: The modern web is saturated with multimodal content, intensifying the challenge of detecting hateful memes, where harmful intent is often conveyed through subtle interactions between text and image under the guise of humor or satire. While recent advances in Vision-Language Models (VLMs) show promise, these models lack support for fine-grained supervision and remain susceptible to implicit hate speech. In this paper, we present a dual-pronged approach to improve multimodal hate detection. First, we propose a prompt optimization framework that systematically varies prompt structure, supervision granularity, and training modality. We show that prompt design and label scaling both influence performance, with structured prompts improving robustness even in small models, and InternVL2 achieving the best F1-scores across binary and scaled settings. Second, we introduce a multimodal data augmentation pipeline that generates 2,479 counterfactually neutral memes by isolating and rewriting the hateful modality. This pipeline, powered by a multi-agent LLM-VLM setup, successfully reduces spurious correlations and improves classifier generalization. Our approaches inspire new directions for building synthetic data to train robust and fair vision-language models. Our findings demonstrate that prompt structure and data composition are as critical as model size, and that targeted augmentation can support more trustworthy and context-sensitive hate detection.

Method: Created VideoAVE dataset with CLIP-based Mixture of Experts filtering to remove mismatched video-product pairs, then benchmarked state-of-the-art video VLMs on attribute-conditioned and open attribute-value extraction tasks.

Result: Video-to-text AVE remains challenging, especially in open settings, with current VLMs showing room for improvement in leveraging temporal information effectively.

Conclusion: VideoAVE provides a valuable benchmark for advancing video VLMs in e-commerce applications, demonstrating the need for more sophisticated temporal modeling in video-based attribute extraction.

Abstract: Attribute Value Extraction (AVE) is important for structuring product information in e-commerce. However, existing AVE datasets are primarily limited to text-to-text or image-to-text settings, lacking support for product videos, diverse attribute coverage, and public availability. To address these gaps, we introduce VideoAVE, the first publicly available video-to-text e-commerce AVE dataset across 14 different domains and covering 172 unique attributes. To ensure data quality, we propose a post-hoc CLIP-based Mixture of Experts filtering system (CLIP-MoE) to remove the mismatched video-product pairs, resulting in a refined dataset of 224k training data and 25k evaluation data. In order to evaluate the usability of the dataset, we further establish a comprehensive benchmark by evaluating several state-of-the-art video vision language models (VLMs) under both attribute-conditioned value prediction and open attribute-value pair extraction tasks. Our results analysis reveals that video-to-text AVE remains a challenging problem, particularly in open settings, and there is still room for developing more advanced VLMs capable of leveraging effective temporal information. The dataset and benchmark code for VideoAVE are available at: https://github.com/gjiaying/VideoAVE

Method: Using three handcrafted feature maps (planar curvature magnitude, curvature sign, and gradient orientation) as inputs to a multilayer perceptron (MLP) classifier for handwritten character recognition tasks.

Result: The curvature-orientation MLP achieved 97% accuracy on MNIST digits and 89% accuracy on EMNIST letters.

Conclusion: Curvature-based representations have strong discriminative power for handwritten character images, and the advantages of deep learning can be realized with interpretable, hand-engineered features rather than complex CNN architectures.

Abstract: This study investigates whether second-order geometric cues - planar curvature magnitude, curvature sign, and gradient orientation - are sufficient on their own to drive a multilayer perceptron (MLP) classifier for handwritten character recognition (HCR), offering an alternative to convolutional neural networks (CNNs). Using these three handcrafted feature maps as inputs, our curvature-orientation MLP achieves 97 percent accuracy on MNIST digits and 89 percent on EMNIST letters. These results underscore the discriminative power of curvature-based representations for handwritten character images and demonstrate that the advantages of deep learning can be realized even with interpretable, hand-engineered features.

Method: Systematic categorization and evaluation of recent Transformer architectures for UAVs, including creation of unified taxonomy, comparative analyses through structured tables and performance benchmarks, and review of datasets, simulators, and evaluation metrics.

Result: Presents a comprehensive synthesis of Transformer-based UAV models, highlights emerging applications like precision agriculture and autonomous navigation, identifies literature gaps, and outlines computational efficiency and real-time deployment challenges.

Conclusion: This review provides a structured framework to understand and advance Transformer-driven UAV technologies, offering critical insights into current limitations and future research directions for the field.

Abstract: The rapid advancement of Transformer-based models has reshaped the landscape of uncrewed aerial vehicle (UAV) systems by enhancing perception, decision-making, and autonomy. This review paper systematically categorizes and evaluates recent developments in Transformer architectures applied to UAVs, including attention mechanisms, CNN-Transformer hybrids, reinforcement learning Transformers, and large language models (LLMs). Unlike previous surveys, this work presents a unified taxonomy of Transformer-based UAV models, highlights emerging applications such as precision agriculture and autonomous navigation, and provides comparative analyses through structured tables and performance benchmarks. The paper also reviews key datasets, simulators, and evaluation metrics used in the field. Furthermore, it identifies existing gaps in the literature, outlines critical challenges in computational efficiency and real-time deployment, and offers future research directions. This comprehensive synthesis aims to guide researchers and practitioners in understanding and advancing Transformer-driven UAV technologies.

Method: The study investigates Gaussian curvature’s properties using the Middlebury stereo dataset, analyzing its role as a sparse descriptor, implicit prior in state-of-the-art methods, and potential as an unsupervised metric.

Result: Gaussian curvature offers a compact 3D surface description, is implicitly considered by current monocular and stereo methods, serves as an effective geometric prior for reconstruction, and shows promise as an unsupervised evaluation metric.

Conclusion: Explicit modeling of Gaussian curvature provides valuable geometric insights that complement data-driven approaches, offering analyzable, transferable, and modifiable geometric representations for 3D computer vision tasks.

Abstract: Recent advances in computer vision have predominantly relied on data-driven approaches that leverage deep learning and large-scale datasets. Deep neural networks have achieved remarkable success in tasks such as stereo matching and monocular depth reconstruction. However, these methods lack explicit models of 3D geometry that can be directly analyzed, transferred across modalities, or systematically modified for controlled experimentation. We investigate the role of Gaussian curvature in 3D surface modeling. Besides Gaussian curvature being an invariant quantity under change of observers or coordinate systems, we demonstrate using the Middlebury stereo dataset that it offers: (i) a sparse and compact description of 3D surfaces, (ii) state-of-the-art monocular and stereo methods seem to implicitly consider it, but no explicit module of such use can be extracted, (iii) a form of geometric prior that can inform and improve 3D surface reconstruction, and (iv) a possible use as an unsupervised metric for stereo methods.

Method: Multi-source Multimodal Progressive Domain Adaptation (MMPDA) framework that gradually aligns source and target domains at both feature and decision levels to bridge domain shifts

Result: Achieved Top-2 place in competition with 60.43% accuracy and 56.99% F1-score on stage 2, surpassing 1st place by 5.59% on F1-score and 3rd place by 6.75% on accuracy

Conclusion: The proposed MMPDA framework effectively handles domain shifts in multimodal deception detection, demonstrating superior performance in cross-domain adaptation and securing top ranking in the competition

Abstract: This paper presents the winning approach for the 1st MultiModal Deception Detection (MMDD) Challenge at the 1st Workshop on Subtle Visual Computing (SVC). Aiming at the domain shift issue across source and target domains, we propose a Multi-source Multimodal Progressive Domain Adaptation (MMPDA) framework that transfers the audio-visual knowledge from diverse source domains to the target domain. By gradually aligning source and the target domain at both feature and decision levels, our method bridges domain shifts across diverse multimodal datasets. Extensive experiments demonstrate the effectiveness of our approach securing Top-2 place. Our approach reaches 60.43% on accuracy and 56.99% on F1-score on competition stage 2, surpassing the 1st place team by 5.59% on F1-score and the 3rd place teams by 6.75% on accuracy. Our code is available at https://github.com/RH-Lin/MMPDA.

Method: A novel visual token pruning method called EVTP-IV that builds upon k-center algorithm by integrating spatial information to ensure better coverage of compact yet representative token subsets. The method includes information-theoretic analysis to support the design.

Result: Achieves up to 5X speed-up on video tasks and 3.5X on image tasks while maintaining comparable accuracy using only 20% of tokens. Consistently outperforms state-of-the-art pruning baselines across varying pruning ratios.

Conclusion: EVTP-IV provides an effective solution to reduce inference costs in MLLMs for visual segmentation tasks through intelligent token pruning that preserves spatial representativeness and maintains performance.

Abstract: Instructed Visual Segmentation (IVS) tasks require segmenting objects in images or videos based on natural language instructions. While recent multimodal large language models (MLLMs) have achieved strong performance on IVS, their inference cost remains a major bottleneck, particularly in video. We empirically analyze visual token sampling in MLLMs and observe a strong correlation between subset token coverage and segmentation performance. This motivates our design of a simple and effective token pruning method that selects a compact yet spatially representative subset of tokens to accelerate inference. In this paper, we introduce a novel visual token pruning method for IVS, called EVTP-IV, which builds upon the k-center by integrating spatial information to ensure better coverage. We further provide an information-theoretic analysis to support our design. Experiments on standard IVS benchmarks show that our method achieves up to 5X speed-up on video tasks and 3.5X on image tasks, while maintaining comparable accuracy using only 20% of the tokens. Our method also consistently outperforms state-of-the-art pruning baselines under varying pruning ratios.

Method: Systematically evaluated multiple segmentation schemes, edge construction strategies, and node feature sets (color, texture, shape descriptors) for image-derived graph representations. Conducted experiments on dermoscopic images using state-of-the-art GLAD models in unsupervised, weakly supervised, and fully supervised regimes.

Result: Color descriptors provided best standalone performance, while incorporating shape and texture features consistently enhanced detection efficacy. Best unsupervised configuration achieved AUC-ROC of 0.805, increasing to 0.872 with sparse labels and 0.914 with full supervision.

Conclusion: Comprehensive analysis demonstrates that carefully designed image-to-graph transformations enable competitive graph-level anomaly detection performance without relying on pretrained backbones, with significant performance gains from incorporating multiple feature types and supervision.

Abstract: Graph Neural Networks (GNNs) have emerged as a powerful approach for graph-based machine learning tasks. Previous work applied GNNs to image-derived graph representations for various downstream tasks such as classification or anomaly detection. These transformations include segmenting images, extracting features from segments, mapping them to nodes, and connecting them. However, to the best of our knowledge, no study has rigorously compared the effectiveness of the numerous potential image-to-graph transformation approaches for GNN-based graph-level anomaly detection (GLAD). In this study, we systematically evaluate the efficacy of multiple segmentation schemes, edge construction strategies, and node feature sets based on color, texture, and shape descriptors to produce suitable image-derived graph representations to perform graph-level anomaly detection. We conduct extensive experiments on dermoscopic images using state-of-the-art GLAD models, examining performance and efficiency in purely unsupervised, weakly supervised, and fully supervised regimes. Our findings reveal, for example, that color descriptors contribute the best standalone performance, while incorporating shape and texture features consistently enhances detection efficacy. In particular, our best unsupervised configuration using OCGTL achieves a competitive AUC-ROC score of up to 0.805 without relying on pretrained backbones like comparable image-based approaches. With the inclusion of sparse labels, the performance increases substantially to 0.872 and with full supervision to 0.914 AUC-ROC.

Method: Proposes LKMN with two core components: Enhanced Partial Large Kernel Block (EPLKB) for non-local feature extraction using channel shuffle, attention, and large kernel strip convolutions; and Cross-Gate Feed-Forward Network (CGFN) for dynamic feature fusion and modulation.

Result: Outperforms state-of-the-art lightweight SR models, achieving 0.23 dB PSNR improvement over DAT-light on Manga109 dataset at 4× upscale with 4.8× faster inference.

Conclusion: LKMN successfully balances quality and efficiency in image super-resolution, demonstrating that pure CNN architectures can achieve superior non-local feature modeling while maintaining computational efficiency.

Abstract: Image super-resolution (SR) in resource-constrained scenarios demands lightweight models balancing performance and latency. Convolutional neural networks (CNNs) offer low latency but lack non-local feature capture, while Transformers excel at non-local modeling yet suffer slow inference. To address this trade-off, we propose the Large Kernel Modulation Network (LKMN), a pure CNN-based model. LKMN has two core components: Enhanced Partial Large Kernel Block (EPLKB) and Cross-Gate Feed-Forward Network (CGFN). The EPLKB utilizes channel shuffle to boost inter-channel interaction, incorporates channel attention to focus on key information, and applies large kernel strip convolutions on partial channels for non-local feature extraction with reduced complexity. The CGFN dynamically adjusts discrepancies between input, local, and non-local features via a learnable scaling factor, then employs a cross-gate strategy to modulate and fuse these features, enhancing their complementarity. Extensive experiments demonstrate that our method outperforms existing state-of-the-art (SOTA) lightweight SR models while balancing quality and efficiency. Specifically, LKMN-L achieves 0.23 dB PSNR improvement over DAT-light on the Manga109 dataset at $\times$4 upscale, with nearly $\times$4.8 times faster. Codes are in the supplementary materials. The code is available at https://github.com/Supereeeee/LKMN.

Method: The attack exploits standard 3DGS shading methods to create viewpoint-specific camouflage - colors and textures that change with viewing angle to embed adversarial content visible only from specific viewpoints, operating in a black-box manner without requiring model architecture or weights.

Result: Extensive experiments show ComplicitSplat generalizes to successfully attack various popular detectors including single-stage, multi-stage, and transformer-based models on both real-world physical object captures and synthetic scenes.

Conclusion: This exposes a novel safety risk for mission-critical applications, representing the first black-box attack on downstream object detectors using 3DGS, highlighting significant security vulnerabilities in safety-critical systems.

Abstract: As 3D Gaussian Splatting (3DGS) gains rapid adoption in safety-critical tasks for efficient novel-view synthesis from static images, how might an adversary tamper images to cause harm? We introduce ComplicitSplat, the first attack that exploits standard 3DGS shading methods to create viewpoint-specific camouflage

• colors and textures that change with viewing angle - to embed adversarial content in scene objects that are visible only from specific viewpoints and without requiring access to model architecture or weights. Our extensive experiments show that ComplicitSplat generalizes to successfully attack a variety of popular detector - both single-stage, multi-stage, and transformer-based models on both real-world capture of physical objects and synthetic scenes. To our knowledge, this is the first black-box attack on downstream object detectors using 3DGS, exposing a novel safety risk for applications like autonomous navigation and other mission-critical robotic systems.

Method: Proposed a comprehensive taxonomy of spatial tasks unifying existing benchmarks, evaluated state-of-the-art proprietary and open-source models on eight key benchmarks using over one billion total tokens, and conducted qualitative evaluation on diverse scenarios.

Result: GPT-5 demonstrates unprecedented strength in spatial intelligence but still falls short of human performance across broad spectrum of tasks. Identified more challenging spatial problems where proprietary models don’t show decisive advantages.

Conclusion: Despite GPT-5’s significant progress, multi-modal models still struggle with spatial intelligence tasks that are intuitive for humans, indicating substantial room for improvement in spatial reasoning capabilities.

Abstract: Multi-modal models have achieved remarkable progress in recent years. Nevertheless, they continue to exhibit notable limitations in spatial understanding and reasoning, which are fundamental capabilities to achieving artificial general intelligence. With the recent release of GPT-5, allegedly the most powerful AI model to date, it is timely to examine where the leading models stand on the path toward spatial intelligence. First, we propose a comprehensive taxonomy of spatial tasks that unifies existing benchmarks and discuss the challenges in ensuring fair evaluation. We then evaluate state-of-the-art proprietary and open-source models on eight key benchmarks, at a cost exceeding one billion total tokens. Our empirical study reveals that (1) GPT-5 demonstrates unprecedented strength in spatial intelligence, yet (2) still falls short of human performance across a broad spectrum of tasks. Moreover, we (3) identify the more challenging spatial intelligence problems for multi-modal models, and (4) proprietary models do not exhibit a decisive advantage when facing the most difficult problems. In addition, we conduct a qualitative evaluation across a diverse set of scenarios that are intuitive for humans yet fail even the most advanced multi-modal models.

Method: Systematically varied high-/low-quality image ratios in finetuning and evaluation sets using ProFound, a domain-specific vision foundation model pretrained on large-scale prostate MRI datasets, testing on tasks like automated radiology reporting and prostate cancer detection.

Result: Image quality distribution and finetune-test mismatch significantly affect performance. Consistent quality ratios enable far less labeled data than training from scratch, but insufficient high-quality finetuning data can cause pretrained models to underperform non-pretrained ones.

Conclusion: Assessing and aligning quality distributions between finetuning and deployment is crucial, highlighting the need for quality standards in finetuning data to fully realize foundation model efficiency benefits.

Abstract: Foundation models in medical imaging have shown promising label efficiency, achieving high downstream performance with only a fraction of annotated data. Here, we evaluate this in prostate multiparametric MRI using ProFound, a domain-specific vision foundation model pretrained on large-scale prostate MRI datasets. We investigate how variable image quality affects label-efficient finetuning by measuring the generalisability of finetuned models. Experiments systematically vary high-/low-quality image ratios in finetuning and evaluation sets. Our findings indicate that image quality distribution and its finetune-and-test mismatch significantly affect model performance. In particular: a) Varying the ratio of high- to low-quality images between finetuning and test sets leads to notable differences in downstream performance; and b) The presence of sufficient high-quality images in the finetuning set is critical for maintaining strong performance, whilst the importance of matched finetuning and testing distribution varies between different downstream tasks, such as automated radiology reporting and prostate cancer detection.When quality ratios are consistent, finetuning needs far less labeled data than training from scratch, but label efficiency depends on image quality distribution. Without enough high-quality finetuning data, pretrained models may fail to outperform those trained without pretraining. This highlights the importance of assessing and aligning quality distributions between finetuning and deployment, and the need for quality standards in finetuning data for specific downstream tasks. Using ProFound, we show the value of quantifying image quality in both finetuning and deployment to fully realise the data and compute efficiency benefits of foundation models.

Method: Proposes cross-layer tensor ring decomposition (TRD) to formulate adapters as layer-shared tensor cores and layer-specific slices, exploiting tensor-level low-rankness to remove redundancy. Uses generalization-aware fine-tuning with diverse rank-driven adapters for collaborative task handling.

Result: Achieves state-of-the-art performance while reducing average training parameters by 90% across various tasks.

Conclusion: AdaRing provides ultra-light parameter-efficient adaptation of vision-language models through effective cross-layer redundancy removal and diverse adapter collaboration.

Abstract: Adapter-based fine-tuning has gained remarkable attention in adapting large pre-trained vision language models (VLMs) for a wide range of downstream tasks efficiently. In this paradigm, only the inserted adapters are fine-tuned, without the need for training the original VLM backbone. Existing works scale adapters by integrating them into every layer of VLMs to increase the capacity of adapters. However, these methods face two primary limitations: 1) limited compression rate due to ignoring cross-layer redundancy, and 2) limited representational capacity across homogeneous adapters. In this paper, we propose a novel vision-language fine-tuning framework based on cross-layer tensor ring decomposition (TRD) with the integration and collaboration of diverse adapters, called AdaRing, achieving ultra-light parameter-efficient adaptation of VLMs on various tasks. To remove the high redundancy that exists among adapters across layers, we exploit the tensor-level low-rankness to formulate adapters as layer-shared tensor cores and layer-specific slices. Moreover, guided by generalization-aware fine-tuning, diverse rank-driven adapters cooperate to handle tasks that require different representations. Our experiments show that the proposed AdaRing achieves the state-of-the-art performance while reducing average training parameters by 90%.

Method: Train a multilayer perceptron (MLP) using only horizontal and vertical Sobel derivatives as input features on MNIST and EMNIST Letters datasets.

Result: The MLP achieved 98% accuracy on MNIST digits and 92% on EMNIST letters, approaching CNN performance while offering smaller memory footprint and more transparent features.

Conclusion: First-order gradients capture most class-discriminative information in handwritten characters, making edge-aware MLPs a compelling alternative to CNNs for HCR tasks.

Abstract: We revisit the classical Sobel operator to ask a simple question: Are first-order edge maps sufficient to drive an all-dense multilayer perceptron (MLP) for handwritten character recognition (HCR), as an alternative to convolutional neural networks (CNNs)? Using only horizontal and vertical Sobel derivatives as input, we train an MLP on MNIST and EMNIST Letters. Despite its extreme simplicity, the resulting network reaches 98% accuracy on MNIST digits and 92% on EMNIST letters – approaching CNNs while offering a smaller memory footprint and transparent features. Our findings highlight that much of the class-discriminative information in handwritten character images is already captured by first-order gradients, making edge-aware MLPs a compelling option for HCR.

Method: Integrates continuous-time camera trajectory into a unified physical imaging model, jointly optimizing exposure times, camera trajectory, and camera response function (CRF) from auto-exposure videos with motion blur.

Result: Outperforms existing methods in robustness and rendering quality on both synthetic and real-world datasets, demonstrating effective HDR reconstruction from casual video input.

Conclusion: Casual3DHDR provides a practical solution for 3D HDR scene reconstruction from casually-captured videos, overcoming limitations of traditional methods that require controlled capture conditions.

Abstract: Photo-realistic novel view synthesis from multi-view images, such as neural radiance field (NeRF) and 3D Gaussian Splatting (3DGS), has gained significant attention for its superior performance. However, most existing methods rely on low dynamic range (LDR) images, limiting their ability to capture detailed scenes in high-contrast environments. While some prior works address high dynamic range (HDR) scene reconstruction, they typically require multi-view sharp images with varying exposure times captured at fixed camera positions, which is time-consuming and impractical. To make data acquisition more flexible, we propose \textbf{Casual3DHDR}, a robust one-stage method that reconstructs 3D HDR scenes from casually-captured auto-exposure (AE) videos, even under severe motion blur and unknown, varying exposure times. Our approach integrates a continuous-time camera trajectory into a unified physical imaging model, jointly optimizing exposure times, camera trajectory, and the camera response function (CRF). Extensive experiments on synthetic and real-world datasets demonstrate that \textbf{Casual3DHDR} outperforms existing methods in robustness and rendering quality. Our source code and dataset will be available at https://lingzhezhao.github.io/CasualHDRSplat/

Method: Proposed OVG-HQ-Unify framework with Parametric Memory Block to retain learned knowledge and cross-modal distillation to balance modality learning. Constructed QVHighlights-Unify dataset and adapted online evaluation metrics.

Result: OVG-HQ-Unify outperforms existing models, providing robust performance for online hybrid-modal video grounding with both accuracy and efficiency.

Conclusion: The framework successfully addresses online video grounding challenges with hybrid queries through memory retention and modality balancing, offering a comprehensive solution with new dataset and evaluation metrics.

Abstract: Video grounding (VG) task focuses on locating specific moments in a video based on a query, usually in text form. However, traditional VG struggles with some scenarios like streaming video or queries using visual cues. To fill this gap, we present a new task named Online Video Grounding with Hybrid-modal Queries (OVG-HQ), which enables online segment localization using text, images, video segments, and their combinations. This task poses two new challenges: limited context in online settings and modality imbalance during training, where dominant modalities overshadow weaker ones. To address these, we propose OVG-HQ-Unify, a unified framework featuring a Parametric Memory Block (PMB) that retain previously learned knowledge to enhance current decision and a cross-modal distillation strategy that guides the learning of non-dominant modalities. This design enables a single model to effectively handle hybrid-modal queries. Due to the lack of suitable datasets, we construct QVHighlights-Unify, an expanded dataset with multi-modal queries. Besides, since offline metrics overlook prediction timeliness, we adapt them to the online setting, introducing oR@n, IoU=m, and online mean Average Precision (omAP) to evaluate both accuracy and efficiency. Experiments show that our OVG-HQ-Unify outperforms existing models, offering a robust solution for online, hybrid-modal video grounding. Source code and datasets are available at https://github.com/maojiaqi2324/OVG-HQ.

Method: SafeCtrl uses a detect-then-suppress paradigm: first precisely localizes unsafe content, then suppresses harmful semantics allowing natural resolution to safe alternatives. Trained with DPO using image-level preference data without needing pixel-level annotations.

Result: Extensive experiments show SafeCtrl significantly outperforms state-of-the-art methods in both safety efficacy and fidelity preservation.

Conclusion: Decoupled, suppression-based control is an effective and scalable direction for building more responsible generative models.

Abstract: The widespread deployment of text-to-image models is challenged by their potential to generate harmful content. While existing safety methods, such as prompt rewriting or model fine-tuning, provide valuable interventions, they often introduce a trade-off between safety and fidelity. Recent localization-based approaches have shown promise, yet their reliance on explicit ``concept replacement" can sometimes lead to semantic incongruity. To address these limitations, we explore a more flexible detect-then-suppress paradigm. We introduce SafeCtrl, a lightweight, non-intrusive plugin that first precisely localizes unsafe content. Instead of performing a hard A-to-B substitution, SafeCtrl then suppresses the harmful semantics, allowing the generative process to naturally and coherently resolve into a safe, context-aware alternative. A key aspect of our work is a novel training strategy using Direct Preference Optimization (DPO). We leverage readily available, image-level preference data to train our module, enabling it to learn nuanced suppression behaviors and perform region-guided interventions at inference without requiring costly, pixel-level annotations. Extensive experiments show that SafeCtrl significantly outperforms state-of-the-art methods in both safety efficacy and fidelity preservation. Our findings suggest that decoupled, suppression-based control is a highly effective and scalable direction for building more responsible generative models.

Method: Leverages spectral and temporal information from Sentinel-2 imagery of single pixels combined with cross-view learning using geo-tagged ground-level photos from LUCAS and Sen4Map datasets, minimizing caption-based training requirements.

Result: Demonstrates that single pixel inputs with temporal and spectral cues are sufficient for thematic mapping tasks including LULC, crop type, and ecosystem type classification, providing scalable and efficient alternative.

Conclusion: TimeSenCLIP offers a lightweight framework that reevaluates spatial context importance, showing that temporal-spectral single-pixel analysis can effectively replace large tile approaches while maintaining semantic alignment between satellite and ground perspectives.

Abstract: Vision-language models have shown significant promise in remote sensing applications, particularly for land-use and land-cover (LULC) via zero-shot classification and retrieval. However, current approaches face two key challenges: reliance on large spatial tiles that increase computational cost, and dependence on text-based supervision, which is often not readily available. In this work, we present TimeSenCLIP, a lightweight framework that reevaluate the role of spatial context by evaluating the effectiveness of a single pixel by leveraging its temporal and spectral dimensions, for classifying LULC and ecosystem types. By leveraging spectral and temporal information from Sentinel-2 imagery and cross-view learning with geo-tagged ground-level photos, we minimises the need for caption-based training while preserving semantic alignment between overhead (satellite) and ground perspectives. Our approach is grounded in the LUCAS and Sen4Map datasets, and evaluated on classification tasks including LULC, crop type, and ecosystem type. We demonstrate that single pixel inputs, when combined with temporal and spectral cues, are sufficient for thematic mapping, offering a scalable and efficient alternative for large-scale remote sensing applications. Code is available at https://github.com/pallavijain-pj/TimeSenCLIP

Method: Audio-Visual Differentiable Room Acoustic Rendering (AV-DAR) framework that leverages visual cues extracted from multi-view images and acoustic beam tracing for physics-based room acoustic rendering.

Result: Experiments across six real-world environments show AV-DAR significantly outperforms prior methods, achieving comparable performance to models trained on 10x more data and delivering relative gains of 16.6% to 50.9% when trained at same scale.

Conclusion: AV-DAR provides an efficient, interpretable, and accurate multimodal physics-based approach for room acoustic rendering that bridges the gap between data efficiency and computational performance.

Abstract: An immersive acoustic experience enabled by spatial audio is just as crucial as the visual aspect in creating realistic virtual environments. However, existing methods for room impulse response estimation rely either on data-demanding learning-based models or computationally expensive physics-based modeling. In this work, we introduce Audio-Visual Differentiable Room Acoustic Rendering (AV-DAR), a framework that leverages visual cues extracted from multi-view images and acoustic beam tracing for physics-based room acoustic rendering. Experiments across six real-world environments from two datasets demonstrate that our multimodal, physics-based approach is efficient, interpretable, and accurate, significantly outperforming a series of prior methods. Notably, on the Real Acoustic Field dataset, AV-DAR achieves comparable performance to models trained on 10 times more data while delivering relative gains ranging from 16.6% to 50.9% when trained at the same scale.

Method: Used U-Net segmentation network trained on: 1) real BraTS 2020 data, 2) synthetic data from medigan GAN, 3) hybrid datasets with varying real/synthetic proportions (40% real + 60% synthetic performed best).

Result: Quantitative performance (Dice, IoU, precision, recall, accuracy) was comparable between real-only and hybrid models. Qualitative analysis showed improved whole tumor boundary delineation with hybrid datasets, but tumor core and enhancing tumor regions still had lower accuracy due to class imbalance.

Conclusion: Synthetic data is feasible for brain tumor segmentation augmentation, but needs larger-scale experiments, volumetric consistency, and better class imbalance mitigation strategies.

Abstract: Manual brain tumor segmentation from MRI scans is challenging due to tumor heterogeneity, scarcity of annotated data, and class imbalance in medical imaging datasets. Synthetic data generated by generative models has the potential to mitigate these issues by improving dataset diversity. This study investigates, as a proof of concept, the impact of incorporating synthetic MRI data, generated using a pre-trained GAN model, into training a U-Net segmentation network. Experiments were conducted using real data from the BraTS 2020 dataset, synthetic data generated with the medigan library, and hybrid datasets combining real and synthetic samples in varying proportions. While overall quantitative performance (Dice coefficient, IoU, precision, recall, accuracy) was comparable between real-only and hybrid-trained models, qualitative inspection suggested that hybrid datasets, particularly with 40% real and 60% synthetic data, improved whole tumor boundary delineation. However, region-wise accuracy for the tumor core and the enhancing tumor remained lower, indicating a persistent class imbalance. The findings support the feasibility of synthetic data as an augmentation strategy for brain tumor segmentation, while highlighting the need for larger-scale experiments, volumetric data consistency, and mitigating class imbalance in future work.

Method: Two-module approach: 1) Unmixing autoencoder module that shifts generation from image space to low-dimensional abundance space using unmixing guidance, 2) Abundance diffusion module that generates samples with non-negativity and unity constraints for physical consistency.

Result: The model generates high-quality and diverse hyperspectral images with significantly reduced computational complexity while preserving high fidelity and physical consistency.

Conclusion: The proposed hyperspectral unmixing-guided diffusion model advances hyperspectral data generation by addressing dimensionality challenges while ensuring physical constraints and diversity, with new evaluation metrics tailored for hyperspectral data.

Abstract: Recently, hyperspectral image generation has received increasing attention, but existing generative models rely on conditional generation schemes, which limits the diversity of generated images. Diffusion models are popular for their ability to generate high-quality samples, but adapting these models from RGB to hyperspectral data presents the challenge of high dimensionality and physical constraints. To address these challenges, we propose a novel diffusion model guided by hyperspectral unmixing. Our model comprises two key modules: an unmixing autoencoder module and an abundance diffusion module. The unmixing autoencoder module leverages unmixing guidance to shift the generative task from the image space to the low-dimensional abundance space, significantly reducing computational complexity while preserving high fidelity. The abundance diffusion module generates samples that satisfy the constraints of non-negativity and unity, ensuring the physical consistency of the reconstructed HSIs. Additionally, we introduce two evaluation metrics tailored to hyperspectral data. Empirical results, evaluated using both traditional metrics and our proposed metrics, indicate that our model is capable of generating high-quality and diverse hyperspectral images, offering an advancement in hyperspectral data generation.

Method: The paper formulates point cloud denoising as a two-step process: outlier removal and surface noise restoration. It creates a taxonomy for denoising tasks, compares methods based on similarities/differences/advantages, and analyzes research limitations.

Result: The survey provides a comprehensive framework for understanding DL-based point cloud denoising, categorizing existing methods, and identifying key challenges in the field.

Conclusion: This systematic survey fills the research gap by offering insights into DL-based PCD developments, proposing a tailored taxonomy, and discussing future research directions to advance point cloud denoising technology.

Abstract: Real-world environment-derived point clouds invariably exhibit noise across varying modalities and intensities. Hence, point cloud denoising (PCD) is essential as a preprocessing step to improve downstream task performance. Deep learning (DL)-based PCD models, known for their strong representation capabilities and flexible architectures, have surpassed traditional methods in denoising performance. To our best knowledge, despite recent advances in performance, no comprehensive survey systematically summarizes the developments of DL-based PCD. To fill the gap, this paper seeks to identify key challenges in DL-based PCD, summarizes the main contributions of existing methods, and proposes a taxonomy tailored to denoising tasks. To achieve this goal, we formulate PCD as a two-step process: outlier removal and surface noise restoration, encompassing most scenarios and requirements of PCD. Additionally, we compare methods in terms of similarities, differences, and respective advantages. Finally, we discuss research limitations and future directions, offering insights for further advancements in PCD.

Method: Three synergistic components: (1) Visual-inertial odometry compensates for camera motion ROI shifts, (2) Depth-informed 2D tracker corrects ROI deviations from object translation, (3) VIO-guided Kalman filter predicts rotation and refines poses hierarchically in a closed-loop system.

Result: The method achieves real-time and robust 6D pose tracking for fast-moving cameras and objects, as demonstrated through both simulation and real-world experiments.

Conclusion: DynamicPose successfully overcomes the limitations of previous methods by providing accurate pose initialization and precise tracking in dynamic scenarios without requiring retraining.

Abstract: We present DynamicPose, a retraining-free 6D pose tracking framework that improves tracking robustness in fast-moving camera and object scenarios. Previous work is mainly applicable to static or quasi-static scenes, and its performance significantly deteriorates when both the object and the camera move rapidly. To overcome these challenges, we propose three synergistic components: (1) A visual-inertial odometry compensates for the shift in the Region of Interest (ROI) caused by camera motion; (2) A depth-informed 2D tracker corrects ROI deviations caused by large object translation; (3) A VIO-guided Kalman filter predicts object rotation, generates multiple candidate poses, and then obtains the final pose by hierarchical refinement. The 6D pose tracking results guide subsequent 2D tracking and Kalman filter updates, forming a closed-loop system that ensures accurate pose initialization and precise pose tracking. Simulation and real-world experiments demonstrate the effectiveness of our method, achieving real-time and robust 6D pose tracking for fast-moving cameras and objects.

Method: The method approximates multi-scale features from a single neighborhood using knowledge distillation, employs class-aware statistics as transferable features to compensate for diversity loss, and introduces central weighted intersection over union to address center offset misalignment in optimization.

Result: Extensive experiments on public datasets demonstrate the effectiveness of the proposed method in achieving competitive object detection performance while significantly reducing computational costs.

Conclusion: The proposed approach successfully addresses the computational burden of traditional multi-scale feature learning in point cloud object detection through knowledge distillation and transferable feature embedding, making it suitable for resource-constrained environments.

Abstract: This paper investigates multi-scale feature approximation and transferable features for object detection from point clouds. Multi-scale features are critical for object detection from point clouds. However, multi-scale feature learning usually involves multiple neighborhood searches and scale-aware layers, which can hinder efforts to achieve lightweight models and may not be conducive to research constrained by limited computational resources. This paper approximates point-based multi-scale features from a single neighborhood based on knowledge distillation. To compensate for the loss of constructive diversity in a single neighborhood, this paper designs a transferable feature embedding mechanism. Specifically, class-aware statistics are employed as transferable features given the small computational cost. In addition, this paper introduces the central weighted intersection over union for localization to alleviate the misalignment brought by the center offset in optimization. Note that the method presented in this paper saves computational costs. Extensive experiments on public datasets demonstrate the effectiveness of the proposed method.

Method: The framework employs an LLM to comprehend and decode sentences and 3D representations, with a core spatial decoder using latent diffusion model for 3D generation. It also includes a geometric-semantic learning strategy to pretrain the vision encoder for joint capture of semantic and geometric cues.

Result: Extensive experimental results demonstrate superiority in visual representation, spatial understanding, and 3D generation. The method supports 3D scene generation from reference images with arbitrary view transformations while maintaining spatial VQA capabilities.

Conclusion: UniUGG successfully addresses the integration challenge of 3D tasks within unified architectures, providing a comprehensive solution for both 3D understanding and generation with demonstrated superior performance across multiple tasks.

Abstract: Despite the impressive progress on understanding and generating images shown by the recent unified architectures, the integration of 3D tasks remains challenging and largely unexplored. In this paper, we introduce UniUGG, the first unified understanding and generation framework for 3D modalities. Our unified framework employs an LLM to comprehend and decode sentences and 3D representations. At its core, we propose a spatial decoder leveraging a latent diffusion model to generate high-quality 3D representations. This allows for the generation and imagination of 3D scenes based on a reference image and an arbitrary view transformation, while remaining supports for spatial visual question answering (VQA) tasks. Additionally, we propose a geometric-semantic learning strategy to pretrain the vision encoder. This design jointly captures the input’s semantic and geometric cues, enhancing both spatial understanding and generation. Extensive experimental results demonstrate the superiority of our method in visual representation, spatial understanding, and 3D generation. The source code will be released upon paper acceptance.

Method: Local Prompt Adaptation (LPA) splits prompts into content and style tokens, then selectively injects them into U-Net’s attention layers at specific timesteps - conditioning object tokens early and style tokens later in the denoising process.

Result: LPA improves CLIP-prompt alignment by +0.41% over SDXL and +0.34% over SD1.5, with +0.09% CLIP-prompt and +0.08% CLIP-style gains on style-rich benchmarks, maintaining diversity without additional training cost.

Conclusion: LPA provides a practical, model-agnostic solution for controllable style-consistent multi-object generation through a single configuration change, offering improved prompt alignment and style uniformity.

Abstract: Diffusion models have become a powerful backbone for text-to-image generation, producing high-quality visuals from natural language prompts. However, when prompts involve multiple objects alongside global or local style instructions, the outputs often drift in style and lose spatial coherence, limiting their reliability for controlled, style-consistent scene generation. We present Local Prompt Adaptation (LPA), a lightweight, training-free method that splits the prompt into content and style tokens, then injects them selectively into the U-Net’s attention layers at chosen timesteps. By conditioning object tokens early and style tokens later in the denoising process, LPA improves both layout control and stylistic uniformity without additional training cost. We conduct extensive ablations across parser settings and injection windows, finding that the best configuration – lpa late only with a 300-650 step window – delivers the strongest balance of prompt alignment and style consistency. On the T2I benchmark, LPA improves CLIP-prompt alignment over vanilla SDXL by +0.41% and over SD1.5 by +0.34%, with no diversity loss. On our custom 50-prompt style-rich benchmark, LPA achieves +0.09% CLIP-prompt and +0.08% CLIP-style gains over baseline. Our method is model-agnostic, easy to integrate, and requires only a single configuration change, making it a practical choice for controllable, style-consistent multi-object generation.

Method: Proposes SAMDWICH framework with Moment-guided Dual-path Propagation (MDP) for moment-aware object tracking and Object-level Selective Supervision (OSS) for filtering irrelevant objects. Uses newly annotated MeViS-M dataset with temporal moment annotations.

Result: Achieves state-of-the-art performance on challenging MeViS benchmark, particularly excelling in complex scenarios with diverse expressions.

Conclusion: The moment-aware approach with selective supervision significantly enhances video-text alignment and referential understanding in RVOS tasks.

Abstract: Referring Video Object Segmentation (RVOS) aims to segment and track objects in videos based on natural language expressions, requiring precise alignment between visual content and textual queries. However, existing methods often suffer from semantic misalignment, largely due to indiscriminate frame sampling and supervision of all visible objects during training – regardless of their actual relevance to the expression. To address this, we introduce a moment-aware RVOS framework named SAMDWICH, along with a newly annotated dataset, MeViS-M, built upon the challenging MeViS benchmark. We manually annotate temporal moments indicating when each object is referred to by the expression, enabling semantically grounded supervision that strengthens video-text alignment. SAMDWICH leverages these aligned text-to-clip pairs to guide training, significantly enhancing referential understanding. Building upon this framework, we propose Moment-guided Dual-path Propagation (MDP), a moment-aware propagation strategy that improves both object grounding and tracking by training on both relevant and irrelevant frames through a moment-centric memory mechanism. In addition, we introduce Object-level Selective Supervision (OSS), an object-level filtering strategy that supervises only the objects temporally aligned with the expression in each training clip. This selective supervision reduces semantic noise and reinforces language-conditioned learning. Extensive experiments show that SAMDWICH achieves state-of-the-art performance on challenging MeViS benchmark, particularly excelling in complex scenarios involving diverse expressions.

Method: Proposes PEdger++ framework using collaborative learning that extracts cross-information from heterogeneous architectures, diverse training moments, and multiple parameter samplings to enhance learning from an ensemble perspective.

Result: Extensive experiments on BSDS500, NYUD and Multicue datasets show clear quantitative and qualitative improvements over existing methods, with multiple model versions available for different computational requirements.

Conclusion: PEdger++ successfully balances accuracy and efficiency in edge detection, demonstrating adaptability to various resource constraints while maintaining competitive performance across multiple benchmark datasets.

Abstract: Edge detection serves as a critical foundation for numerous computer vision applications, including object detection, semantic segmentation, and image editing, by extracting essential structural cues that define object boundaries and salient edges. To be viable for broad deployment across devices with varying computational capacities, edge detectors shall balance high accuracy with low computational complexity. While deep learning has evidently improved accuracy, they often suffer from high computational costs, limiting their applicability on resource-constrained devices. This paper addresses the challenge of achieving that balance: \textit{i.e.}, {how to efficiently capture discriminative features without relying on large-size and sophisticated models}. We propose PEdger++, a collaborative learning framework designed to reduce computational costs and model sizes while improving edge detection accuracy. The core principle of our PEdger++ is that cross-information derived from heterogeneous architectures, diverse training moments, and multiple parameter samplings, is beneficial to enhance learning from an ensemble perspective. Extensive experimental results on the BSDS500, NYUD and Multicue datasets demonstrate the effectiveness of our approach, both quantitatively and qualitatively, showing clear improvements over existing methods. We also provide multiple versions of the model with varying computational requirements, highlighting PEdger++’s adaptability with respect to different resource constraints. Codes are accessible at https://github.com/ForawardStar/EdgeDetectionviaPEdgerPlus/.

Method: Created a multi-resolution, multi-modal dataset with synchronized RGB and event cameras under variable lighting conditions. Evaluated two baseline tasks: Action Unit classification using Spiking Neural Networks and frame reconstruction using Conditional Variational Autoencoders.

Result: Event-based data achieved 51.23% accuracy for Action Unit classification vs 23.12% with RGB data. Frame reconstruction achieved SSIM = 0.8513 and PSNR = 26.89 dB with high-resolution event input.

Conclusion: Event cameras show promising results for micro-expression recognition and frame reconstruction, outperforming traditional RGB cameras due to their microsecond-level precision, high dynamic range, and low latency.

Abstract: Micro-expression analysis has applications in domains such as Human-Robot Interaction and Driver Monitoring Systems. Accurately capturing subtle and fast facial movements remains difficult when relying solely on RGB cameras, due to limitations in temporal resolution and sensitivity to motion blur. Event cameras offer an alternative, with microsecond-level precision, high dynamic range, and low latency. However, public datasets featuring event-based recordings of Action Units are still scarce. In this work, we introduce a novel, preliminary multi-resolution and multi-modal micro-expression dataset recorded with synchronized RGB and event cameras under variable lighting conditions. Two baseline tasks are evaluated to explore the spatial-temporal dynamics of micro-expressions: Action Unit classification using Spiking Neural Networks (51.23% accuracy with events vs. 23.12% with RGB), and frame reconstruction using Conditional Variational Autoencoders, achieving SSIM = 0.8513 and PSNR = 26.89 dB with high-resolution event input. These promising results show that event-based data can be used for micro-expression recognition and frame reconstruction.

Method: Proposes MOON model with: (1) guided Mixture-of-Experts module for multimodal and aspect-specific content modeling, (2) core semantic region detection to mitigate background noise, (3) specialized negative sampling strategy for increased difficulty and diversity of negative samples.

Result: Demonstrates competitive zero-shot performance on both the proposed MBE benchmark and public datasets, showing strong generalization across cross-modal retrieval, product classification, and attribute prediction tasks. Case studies and visualizations confirm effectiveness.

Conclusion: MOON successfully addresses key challenges in product representation learning through generative MLLM approach, providing a robust solution for product understanding with strong generalization capabilities across multiple downstream tasks.

Abstract: With the rapid advancement of e-commerce, exploring general representations rather than task-specific ones has attracted increasing research attention. For product understanding, although existing discriminative dual-flow architectures drive progress in this field, they inherently struggle to model the many-to-one alignment between multiple images and texts of products. Therefore, we argue that generative Multimodal Large Language Models (MLLMs) hold significant potential for improving product representation learning. Nevertheless, achieving this goal still remains non-trivial due to several key challenges: the lack of multimodal and aspect-aware modeling modules in typical LLMs; the common presence of background noise in product images; and the absence of a standard benchmark for evaluation. To address these issues, we propose the first generative MLLM-based model named MOON for product representation learning. Our method (1) employs a guided Mixture-of-Experts (MoE) module for targeted modeling of multimodal and aspect-specific product content; (2) effectively detects core semantic regions in product images to mitigate the distraction and interference caused by background noise; and (3) introduces the specialized negative sampling strategy to increase the difficulty and diversity of negative samples. In addition, we release a large-scale multimodal benchmark MBE for various product understanding tasks. Experimentally, our model demonstrates competitive zero-shot performance on both our benchmark and the public dataset, showcasing strong generalization across various downstream tasks, including cross-modal retrieval, product classification, and attribute prediction. Furthermore, the case study and visualization illustrate the effectiveness of MOON for product understanding.

Method: Uses SAM-generated masks as pseudo ground-truth for 2D feature learning via contrastive loss and pseudo-supervised objectives. Introduces 3D regularization to implicitly encode instance identities with voxel-based loss consistency. Employs a lightweight static codebook to bridge continuous features and discrete identities without preprocessing.

Result: Quantitative and qualitative experiments demonstrate effectiveness. First framework to achieve 3D instance segmentation in dynamic, open-world driving scenes.

Conclusion: InstDrive successfully addresses instance-aware reconstruction in dynamic driving scenes without complex preprocessing, enabling better instance-level understanding and flexible scene editing capabilities.

Abstract: Reconstructing dynamic driving scenes from dashcam videos has attracted increasing attention due to its significance in autonomous driving and scene understanding. While recent advances have made impressive progress, most methods still unify all background elements into a single representation, hindering both instance-level understanding and flexible scene editing. Some approaches attempt to lift 2D segmentation into 3D space, but often rely on pre-processed instance IDs or complex pipelines to map continuous features to discrete identities. Moreover, these methods are typically designed for indoor scenes with rich viewpoints, making them less applicable to outdoor driving scenarios. In this paper, we present InstDrive, an instance-aware 3D Gaussian Splatting framework tailored for the interactive reconstruction of dynamic driving scene. We use masks generated by SAM as pseudo ground-truth to guide 2D feature learning via contrastive loss and pseudo-supervised objectives. At the 3D level, we introduce regularization to implicitly encode instance identities and enforce consistency through a voxel-based loss. A lightweight static codebook further bridges continuous features and discrete identities without requiring data pre-processing or complex optimization. Quantitative and qualitative experiments demonstrate the effectiveness of InstDrive, and to the best of our knowledge, it is the first framework to achieve 3D instance segmentation in dynamic, open-world driving scenes.More visualizations are available at our project page.

Method: Proposes contour-aware scanline placement using weakly supervised B-mode landmark detection to infer LV long axis and basal level. Builds on EnLVAM’s approach but adds full automation by generating AMM images and predicting landmarks along automatically placed scanlines.

Result: The method enables fully automated yet manually adaptable LV linear measurements that mimic clinical guidelines, combining structure awareness from B-mode images with motion awareness from AMM mode for enhanced robustness and accuracy.

Conclusion: WiseLVAM provides a practical solution for routine clinical application by automating the entire LV measurement process while maintaining clinical reliability through its dual awareness approach and adherence to clinical guidelines.

Abstract: Clinical guidelines recommend performing left ventricular (LV) linear measurements in B-mode echocardiographic images at the basal level – typically at the mitral valve leaflet tips – and aligned perpendicular to the LV long axis along a virtual scanline (SL). However, most automated methods estimate landmarks directly from B-mode images for the measurement task, where even small shifts in predicted points along the LV walls can lead to significant measurement errors, reducing their clinical reliability. A recent semi-automatic method, EnLVAM, addresses this limitation by constraining landmark prediction to a clinician-defined SL and training on generated Anatomical Motion Mode (AMM) images to predict LV landmarks along the same. To enable full automation, a contour-aware SL placement approach is proposed in this work, in which the LV contour is estimated using a weakly supervised B-mode landmark detector. SL placement is then performed by inferring the LV long axis and the basal level-mimicking clinical guidelines. Building on this foundation, we introduce \textit{WiseLVAM} – a novel, fully automated yet manually adaptable framework for automatically placing the SL and then automatically performing the LV linear measurements in the AMM mode. \textit{WiseLVAM} utilizes the structure-awareness from B-mode images and the motion-awareness from AMM mode to enhance robustness and accuracy with the potential to provide a practical solution for the routine clinical application.

Method: Uses Fast Fourier Transform to shift medical image and text features into frequency domain for noise filtering, combined with quantum-inspired retrieval system to fetch relevant medical facts from external sources using quantum-based similarity techniques.

Result: Outperforms previous models on VQA-RAD dataset, particularly excelling on complex cases requiring image-text reasoning, with improved performance and explainability.

Conclusion: The integration of frequency processing and quantum information retrieval provides a promising approach for developing intelligent, transparent, and helpful AI tools for medical professionals.

Abstract: Solving tough clinical questions that require both image and text understanding is still a major challenge in healthcare AI. In this work, we propose Q-FSRU, a new model that combines Frequency Spectrum Representation and Fusion (FSRU) with a method called Quantum Retrieval-Augmented Generation (Quantum RAG) for medical Visual Question Answering (VQA). The model takes in features from medical images and related text, then shifts them into the frequency domain using Fast Fourier Transform (FFT). This helps it focus on more meaningful data and filter out noise or less useful information. To improve accuracy and ensure that answers are based on real knowledge, we add a quantum-inspired retrieval system. It fetches useful medical facts from external sources using quantum-based similarity techniques. These details are then merged with the frequency-based features for stronger reasoning. We evaluated our model using the VQA-RAD dataset, which includes real radiology images and questions. The results showed that Q-FSRU outperforms earlier models, especially on complex cases needing image-text reasoning. The mix of frequency and quantum information improves both performance and explainability. Overall, this approach offers a promising way to build smart, clear, and helpful AI tools for doctors.

Method: Develops a motion-centered video retrieval model (Gemini Motion Video Retriever) and a Motion-centric Dual-alignment DPO Trainer to address retrieval effectiveness and mitigate error propagation from suboptimal retrieval results.

Result: Experimental results demonstrate that VimoRAG significantly boosts the performance of motion LLMs that are constrained to text-only input.

Conclusion: The framework successfully leverages in-the-wild video databases to enhance 3D motion generation by effectively retrieving and utilizing 2D human motion signals, overcoming the data limitations of traditional motion LLMs.

Abstract: This paper introduces VimoRAG, a novel video-based retrieval-augmented motion generation framework for motion large language models (LLMs). As motion LLMs face severe out-of-domain/out-of-vocabulary issues due to limited annotated data, VimoRAG leverages large-scale in-the-wild video databases to enhance 3D motion generation by retrieving relevant 2D human motion signals. While video-based motion RAG is nontrivial, we address two key bottlenecks: (1) developing an effective motion-centered video retrieval model that distinguishes human poses and actions, and (2) mitigating the issue of error propagation caused by suboptimal retrieval results. We design the Gemini Motion Video Retriever mechanism and the Motion-centric Dual-alignment DPO Trainer, enabling effective retrieval and generation processes. Experimental results show that VimoRAG significantly boosts the performance of motion LLMs constrained to text-only input.

Method: Proposes PCR metric that measures spatial consistency between boxes before/after NMS and reliability via confidence scores of overlapping boxes. Uses meta-dataset with varying image corruptions for realistic evaluation.

Result: PCR provides more accurate performance estimates than existing AutoEval methods, and the meta-dataset covers wider performance range.

Conclusion: The AutoEval framework with PCR enables efficient and accurate object detector evaluation without ground-truth labels, with practical applications for real-world deployment.

Abstract: Recent advances in computer vision have made training object detectors more efficient and effective; however, assessing their performance in real-world applications still relies on costly manual annotation. To address this limitation, we develop an automated model evaluation (AutoEval) framework for object detection. We propose Prediction Consistency and Reliability (PCR), which leverages the multiple candidate bounding boxes that conventional detectors generate before non-maximum suppression (NMS). PCR estimates detection performance without ground-truth labels by jointly measuring 1) the spatial consistency between boxes before and after NMS, and 2) the reliability of the retained boxes via the confidence scores of overlapping boxes. For a more realistic and scalable evaluation, we construct a meta-dataset by applying image corruptions of varying severity. Experimental results demonstrate that PCR yields more accurate performance estimates than existing AutoEval methods, and the proposed meta-dataset covers a wider range of detection performance. The code is available at https://github.com/YonseiML/autoeval-det.

Method: A diffusion-based model that encodes frame changes via temporal self-similarity, then iteratively decodes random noise into plausible boundaries using classifier-free guidance to control diversity.

Result: Achieves strong performance on Kinetics-GEBD and TAPOS benchmarks, generating diverse and plausible event boundaries.

Conclusion: The generative diffusion approach effectively addresses the subjectivity problem in GEBD by producing multiple plausible boundary solutions rather than single deterministic predictions.

Abstract: Generic event boundary detection (GEBD) aims to identify natural boundaries in a video, segmenting it into distinct and meaningful chunks. Despite the inherent subjectivity of event boundaries, previous methods have focused on deterministic predictions, overlooking the diversity of plausible solutions. In this paper, we introduce a novel diffusion-based boundary detection model, dubbed DiffGEBD, that tackles the problem of GEBD from a generative perspective. The proposed model encodes relevant changes across adjacent frames via temporal self-similarity and then iteratively decodes random noise into plausible event boundaries being conditioned on the encoded features. Classifier-free guidance allows the degree of diversity to be controlled in denoising diffusion. In addition, we introduce a new evaluation metric to assess the quality of predictions considering both diversity and fidelity. Experiments show that our method achieves strong performance on two standard benchmarks, Kinetics-GEBD and TAPOS, generating diverse and plausible event boundaries.

Method: Multi-stage CNN approach that pairs high-accuracy scanners as references with low-accuracy scanners to quantify error patterns, combining geometric processing with neural network refinement.

Result: 70% MSE reduction and approximately 6dB PSNR improvement, enabling low-end devices to approach high-end measurement accuracy without hardware changes.

Conclusion: The method successfully transforms systematic error quantification into supervised learning, providing precise correction while preserving geometric features for improved spatial measurements.

Abstract: We propose a multi-stage convolutional neural network (MSCNN) based integrated method for reducing uncertainty of 3D point accuracy of lasar scanner (LS) in rough indoor rooms, providing more accurate spatial measurements for high-precision geometric model creation and renovation. Due to different equipment limitations and environmental factors, high-end and low-end LS have positional errors. Our approach pairs high-accuracy scanners (HAS) as references with corresponding low-accuracy scanners (LAS) of measurements in identical environments to quantify specific error patterns. By establishing a statistical relationship between measurement discrepancies and their spatial distribution, we develop a correction framework that combines traditional geometric processing with targeted neural network refinement. This method transforms the quantification of systematic errors into a supervised learning problem, allowing precise correction while preserving critical geometric features. Experimental results in our rough indoor rooms dataset show significant improvements in measurement accuracy, with mean square error (MSE) reductions exceeding 70% and peak signal-to-noise ratio (PSNR) improvements of approximately 6 decibels. This approach enables low-end devices to achieve measurement uncertainty levels approaching those of high-end devices without hardware modifications.

Method: Developed a theoretical framework that mathematically formulates error propagation, derived per-step quantization error propagation equations, established closed-form solution for cumulative error, and proposed timestep-aware cumulative error compensation scheme.

Result: Extensive experiments across multiple image datasets show the compensation strategy effectively mitigates error propagation and significantly enhances existing PTQ methods to achieve state-of-the-art performance on low-precision diffusion models.

Conclusion: The proposed theoretical framework and compensation scheme successfully address quantization error accumulation in diffusion models, enabling more efficient deployment while maintaining high output quality.

Abstract: Diffusion models have transformed image synthesis by establishing unprecedented quality and creativity benchmarks. Nevertheless, their large-scale deployment faces challenges due to computationally intensive iterative denoising processes. Although post-training quantization(PTQ) provides an effective pathway for accelerating sampling, the iterative nature of diffusion models causes stepwise quantization errors to accumulate progressively during generation, inevitably compromising output fidelity. To address this challenge, we develop a theoretical framework that mathematically formulates error propagation in Diffusion Models (DMs), deriving per-step quantization error propagation equations and establishing the first closed-form solution for cumulative error. Building on this theoretical foundation, we propose a timestep-aware cumulative error compensation scheme. Extensive experiments across multiple image datasets demonstrate that our compensation strategy effectively mitigates error propagation, significantly enhancing existing PTQ methods to achieve state-of-the-art(SOTA) performance on low-precision diffusion models.

Method: Proposes VELVET-Med framework with: 1) Uni-modal self-supervised learning integration, 2) TriBERT language encoder for multi-level textual semantics, 3) Hierarchical contrastive learning for multi-level vision-language correspondence. Uses only 38,875 scan-report pairs.

Result: Achieves state-of-the-art performance across multiple downstream tasks including 3D segmentation, cross-modal retrieval, visual question answering, and report generation. The encoders exhibit strong transferability.

Conclusion: The framework successfully uncovers rich spatial and semantic relationships in volumetric medical images and clinical narratives, enhancing generalization ability without requiring large-scale data collection.

Abstract: Vision-and-language models (VLMs) have been increasingly explored in the medical domain, particularly following the success of CLIP in general domain. However, unlike the relatively straightforward pairing of 2D images and text, curating large-scale paired data in the medical field for volumetric modalities such as CT scans remains a challenging and time-intensive process. This difficulty often limits the performance on downstream tasks. To address these challenges, we propose a novel vision-language pre-training (VLP) framework, termed as \textbf{VELVET-Med}, specifically designed for limited volumetric data such as 3D CT and associated radiology reports. Instead of relying on large-scale data collection, our method focuses on the development of effective pre-training objectives and model architectures. The key contributions are: 1) We incorporate uni-modal self-supervised learning into VLP framework, which are often underexplored in the existing literature. 2) We propose a novel language encoder, termed as \textbf{TriBERT}, for learning multi-level textual semantics. 3) We devise the hierarchical contrastive learning to capture multi-level vision-language correspondence. Using only 38,875 scan-report pairs, our approach seeks to uncover rich spatial and semantic relationships embedded in volumetric medical images and corresponding clinical narratives, thereby enhancing the generalization ability of the learned encoders. The resulting encoders exhibit strong transferability, achieving state-of-the-art performance across a wide range of downstream tasks, including 3D segmentation, cross-modal retrieval, visual question answering, and report generation.

Method: Proposes Simple o3 framework with scalable data synthesis pipeline generating high-quality interleaved vision-language reasoning chains via ‘observe-reason-act’ cycle. Uses supervised fine-tuning to integrate dynamic tool interactions and creates TWI-Tools-146K dataset with executable visual operations and rigorous verification.

Result: Demonstrates superior performance on diverse benchmarks, outperforming existing approaches. Found that reusing and magnifying original images improves visual reasoning, while image cropping based on precise visual grounding enhances focus on key entities/regions.

Conclusion: Simple o3 establishes a powerful yet computationally affordable paradigm for advancing multimodal reasoning, providing first in-depth analysis of different interleaved reasoning strategies and their impact on model performance.

Abstract: Multimodal Large Language Models (MLLMs) have shown impressive performance on vision-language tasks, but their long Chain-of-Thought (CoT) capabilities in multimodal scenarios remain underexplored. Inspired by OpenAI’s o3 model, which emulates human-like ‘’thinking with image’’ through iterative visual transformations and linguistic reasoning, we propose Simple o3, an end-to-end framework that integrates dynamic tool interactions (e.g., cropping, zooming, and reusing) into interleaved vision-language reasoning via supervised fine-tuning (SFT). Our approach features a scalable data synthesis pipeline that generates high-quality interleaved vision-language reasoning chains via an ‘‘observe-reason-act’’ cycle, complete with executable visual operations and rigorous verification, yielding the open-source TWI-Tools-146K dataset. Experimental results demonstrate Simple o3’s superior performance on diverse benchmarks, outperforming existing approaches. By combining enhanced reasoning capabilities, Simple o3 establishes a powerful yet computationally affordable paradigm for advancing multimodal reasoning. Remarkably, we provide the first in-depth analysis of different interleaved reasoning strategies, offering insights into their impact on model performance. We found that by introducing additional visual tokens for interleaved vision-language reasoning, reusing and magnifying the original image significantly improves the model’s visual reasoning and fine-grained perception, while image cropping based on precise visual grounding allows the model to effectively focus on key entities or regions, further enhancing its capabilities.

Method: Two-stage hybrid pipeline: 1) Warps target garment using learned flow field for high-fidelity preservation, 2) Uses fidelity-preserving try-on module with preserved-region input and inpainting mask to blend warped garment while retaining key areas and regenerating only necessary regions.

Result: Extensive qualitative results show visually seamless try-on results while faithfully maintaining high-frequency garment details, achieving effective balance between reconstruction accuracy and perceptual realism.

Conclusion: DualFit successfully addresses the limitation of detail preservation in virtual try-on by combining warping for fidelity with diffusion for realism, particularly preserving critical brand elements like logos and text.

Abstract: Virtual Try-On technology has garnered significant attention for its potential to transform the online fashion retail experience by allowing users to visualize how garments would look on them without physical trials. While recent advances in diffusion-based warping-free methods have improved perceptual quality, they often fail to preserve fine-grained garment details such as logos and printed text elements that are critical for brand integrity and customer trust. In this work, we propose DualFit, a hybrid VTON pipeline that addresses this limitation by two-stage approach. In the first stage, DualFit warps the target garment to align with the person image using a learned flow field, ensuring high-fidelity preservation. In the second stage, a fidelity-preserving try-on module synthesizes the final output by blending the warped garment with preserved human regions. Particularly, to guide this process, we introduce a preserved-region input and an inpainting mask, enabling the model to retain key areas and regenerate only where necessary, particularly around garment seams. Extensive qualitative results show that DualFit achieves visually seamless try-on results while faithfully maintaining high-frequency garment details, striking an effective balance between reconstruction accuracy and perceptual realism.

Method: TriQDef consists of three components: (1) Feature Disalignment Penalty (FDP) that enforces semantic inconsistency in intermediate representations, (2) Gradient Perceptual Dissonance Penalty (GPDP) that misaligns input gradients across bit-widths using Edge IoU and HOG Cosine metrics, and (3) Joint Quantization-Aware Training Protocol that unifies these penalties in a shared-weight training scheme across multiple quantization levels.

Result: Extensive experiments on CIFAR-10 and ImageNet show that TriQDef reduces Attack Success Rates (ASR) by over 40% on unseen patch and quantization combinations while preserving high clean accuracy.

Conclusion: The findings highlight the importance of disrupting both semantic and perceptual gradient alignment to effectively mitigate patch transferability vulnerabilities in Quantized Neural Networks.

Abstract: Quantized Neural Networks (QNNs) are increasingly deployed in edge and resource-constrained environments due to their efficiency in computation and memory usage. While shown to distort the gradient landscape and weaken conventional pixel-level attacks, it provides limited robustness against patch-based adversarial attacks-localized, high-saliency perturbations that remain surprisingly transferable across bit-widths. Existing defenses either overfit to fixed quantization settings or fail to address this cross-bit generalization vulnerability. We introduce \textbf{TriQDef}, a tri-level quantization-aware defense framework designed to disrupt the transferability of patch-based adversarial attacks across QNNs. TriQDef consists of: (1) a Feature Disalignment Penalty (FDP) that enforces semantic inconsistency by penalizing perceptual similarity in intermediate representations; (2) a Gradient Perceptual Dissonance Penalty (GPDP) that explicitly misaligns input gradients across bit-widths by minimizing structural and directional agreement via Edge IoU and HOG Cosine metrics; and (3) a Joint Quantization-Aware Training Protocol that unifies these penalties within a shared-weight training scheme across multiple quantization levels. Extensive experiments on CIFAR-10 and ImageNet demonstrate that TriQDef reduces Attack Success Rates (ASR) by over 40% on unseen patch and quantization combinations, while preserving high clean accuracy. Our findings underscore the importance of disrupting both semantic and perceptual gradient alignment to mitigate patch transferability in QNNs.

Method: Systematic analysis of regularization techniques from continual learning, combined with careful validation set design and hyperparameter tuning to retain multimodal knowledge during domain-specific fine-tuning.

Result: Consistently strong results on fine-grained and coarse-grained retrieval benchmarks, retaining visual-text alignment without using text data or original text encoder during fine-tuning.

Conclusion: The proposed method effectively balances domain adaptation and knowledge retention, achieving optimal performance for fine-grained retrieval while preserving the VLM’s broad multimodal capabilities.

Abstract: Large-scale contrastive pre-training produces powerful Vision-and-Language Models (VLMs) capable of generating representations (embeddings) effective for a wide variety of visual and multimodal tasks. However, these pretrained embeddings remain suboptimal for fine-grained open-set visual retrieval, where state-of-the-art results require fine-tuning the vision encoder using annotated domain-specific samples. Naively performing such fine-tuning typically leads to catastrophic forgetting, severely diminishing the model’s general-purpose visual and cross-modal capabilities. In this work, we propose a fine-tuning method explicitly designed to achieve optimal balance between fine-grained domain adaptation and retention of the pretrained VLM’s broad multimodal knowledge. Drawing inspiration from continual learning literature, we systematically analyze standard regularization techniques aimed at knowledge retention and propose an efficient and effective combination strategy. Additionally, we address the commonly overlooked yet critical aspects of validation set design and hyperparameter tuning to ensure reproducibility and robust generalization across datasets and pretrained models. We extensively evaluate our method on both fine-grained and coarse-grained image-image and image-text retrieval benchmarks. Our approach consistently achieves strong results, notably retaining the visual-text alignment without utilizing any text data or the original text encoder during fine-tuning. Code and model checkpoints: https://github.com/nikosips/infusing .

Method: Proposed KP-INR with dual branches: one processes positional embeddings of k-space coordinates, the other learns from local multi-scale k-space feature representations at those coordinates, with cross-branch interaction.

Result: Experiments on CMRxRecon2024 dataset show improved performance over baseline models, confirming strong performance on challenging Cartesian k-space data.

Conclusion: KP-INR demonstrates potential in cardiac cine MRI reconstruction by effectively combining positional and feature information, offering better image recovery from undersampled data.

Abstract: Cardiac Magnetic Resonance (CMR) imaging is a non-invasive method for assessing cardiac structure, function, and blood flow. Cine MRI extends this by capturing heart motion, providing detailed insights into cardiac mechanics. To reduce scan time and breath-hold discomfort, fast acquisition techniques have been utilized at the cost of lowering image quality. Recently, Implicit Neural Representation (INR) methods have shown promise in unsupervised reconstruction by learning coordinate-to-value mappings from undersampled data, enabling high-quality image recovery. However, current existing INR methods primarily focus on using coordinate-based positional embeddings to learn the mapping, while overlooking the feature representations of the target point and its neighboring context. In this work, we propose KP-INR, a dual-branch INR method operating in k-space for cardiac cine MRI reconstruction: one branch processes the positional embedding of k-space coordinates, while the other learns from local multi-scale k-space feature representations at those coordinates. By enabling cross-branch interaction and approximating the target k-space values from both branches, KP-INR can achieve strong performance on challenging Cartesian k-space data. Experiments on the CMRxRecon2024 dataset confirms its improved performance over baseline models and highlights its potential in this field.

Method: Proposed Foreground Background Memorization (FB-Mem), a novel segmentation-based metric that classifies and quantifies memorized regions within generated images using a clustering approach.

Result: Reveals memorization is more pervasive: (1) individual generations link to clusters of similar training images showing complex patterns, (2) existing mitigation methods fail to eliminate local memorization especially in foreground regions.

Conclusion: Establishes an effective framework for measuring memorization in diffusion models, demonstrates inadequacy of current mitigation approaches, and proposes a stronger clustering-based mitigation method.

Abstract: Diffusion models (DMs) memorize training images and can reproduce near-duplicates during generation. Current detection methods identify verbatim memorization but fail to capture two critical aspects: quantifying partial memorization occurring in small image regions, and memorization patterns beyond specific prompt-image pairs. To address these limitations, we propose Foreground Background Memorization (FB-Mem), a novel segmentation-based metric that classifies and quantifies memorized regions within generated images. Our method reveals that memorization is more pervasive than previously understood: (1) individual generations from single prompts may be linked to clusters of similar training images, revealing complex memorization patterns that extend beyond one-to-one correspondences; and (2) existing model-level mitigation methods, such as neuron deactivation and pruning, fail to eliminate local memorization, which persists particularly in foreground regions. Our work establishes an effective framework for measuring memorization in diffusion models, demonstrates the inadequacy of current mitigation approaches, and proposes a stronger mitigation method using a clustering approach.

Method: Uses VAE to generate 3D facial landmarks from audio, concatenates with emotion-label embeddings via ResNet-based landmark deformation model, then conditions a novel tri-plane attention NeRF with landmarks and facial blendshape coefficients to synthesize emotional talking heads.

Result: Extensive experiments show RealTalk outperforms existing methods in emotion accuracy, controllability, and identity preservation.

Conclusion: RealTalk advances the development of socially intelligent AI systems by enabling high-quality emotional talking head synthesis with precise emotion control and identity preservation.

Abstract: Emotion is a critical component of artificial social intelligence. However, while current methods excel in lip synchronization and image quality, they often fail to generate accurate and controllable emotional expressions while preserving the subject’s identity. To address this challenge, we introduce RealTalk, a novel framework for synthesizing emotional talking heads with high emotion accuracy, enhanced emotion controllability, and robust identity preservation. RealTalk employs a variational autoencoder (VAE) to generate 3D facial landmarks from driving audio, which are concatenated with emotion-label embeddings using a ResNet-based landmark deformation model (LDM) to produce emotional landmarks. These landmarks and facial blendshape coefficients jointly condition a novel tri-plane attention Neural Radiance Field (NeRF) to synthesize highly realistic emotional talking heads. Extensive experiments demonstrate that RealTalk outperforms existing methods in emotion accuracy, controllability, and identity preservation, advancing the development of socially intelligent AI systems.

Method: Uses language-guided 4D world generator with state-aware causal transformer for human motion generation, and phase-coherent ray tracing simulator for accurate RF signal simulation.

Result: Effective conditioned human motion generation, successful application of phase coherence to beamforming and respiration monitoring, and performance gains in both data-limited and data-adequate scenarios for RF imaging and activity recognition.

Conclusion: WaveVerse enables RF imaging data generation for the first time and provides a scalable framework for realistic RF signal simulation that consistently improves performance across various RF sensing applications.

Abstract: Radio Frequency (RF) sensing has emerged as a powerful, privacy-preserving alternative to vision-based methods for indoor perception tasks. However, collecting high-quality RF data in dynamic and diverse indoor environments remains a major challenge. To address this, we introduce WaveVerse, a prompt-based, scalable framework that simulates realistic RF signals from generated indoor scenes with human motions. WaveVerse introduces a language-guided 4D world generator, which includes a state-aware causal transformer for human motion generation conditioned on spatial constraints and texts, and a phase-coherent ray tracing simulator that enables the simulation of accurate and coherent RF signals. Experiments demonstrate the effectiveness of our approach in conditioned human motion generation and highlight how phase coherence is applied to beamforming and respiration monitoring. We further present two case studies in ML-based high-resolution imaging and human activity recognition, demonstrating that WaveVerse not only enables data generation for RF imaging for the first time, but also consistently achieves performance gain in both data-limited and data-adequate scenarios.

Method: Formulates feature lifting as a sparse linear inverse problem solvable in closed form, with Tikhonov Guidance for numerical stability and Post-Lifting Aggregation for noise filtering through feature clustering.

Result: Achieves state-of-the-art performance on open-vocabulary 3D segmentation benchmarks, outperforming training-based, grouping-based, and heuristic baselines while generating lifted features in minutes.

Conclusion: The proposed unified framework provides efficient, high-quality feature lifting with provable error bounds and effective regularization strategies for handling multi-view inconsistencies.

Abstract: Feature lifting has emerged as a crucial component in 3D scene understanding, enabling the attachment of rich image feature descriptors (e.g., DINO, CLIP) onto splat-based 3D representations. The core challenge lies in optimally assigning rich general attributes to 3D primitives while addressing the inconsistency issues from multi-view images. We present a unified, kernel- and feature-agnostic formulation of the feature lifting problem as a sparse linear inverse problem, which can be solved efficiently in closed form. Our approach admits a provable upper bound on the global optimal error under convex losses for delivering high quality lifted features. To address inconsistencies and noise in multi-view observations, we introduce two complementary regularization strategies to stabilize the solution and enhance semantic fidelity. Tikhonov Guidance enforces numerical stability through soft diagonal dominance, while Post-Lifting Aggregation filters noisy inputs via feature clustering. Extensive experiments demonstrate that our approach achieves state-of-the-art performance on open-vocabulary 3D segmentation benchmarks, outperforming training-based, grouping-based, and heuristic-forward baselines while producing the lifted features in minutes. Code is available at \href{https://github.com/saliteta/splat-distiller.git}{\textbf{github}}. We also have a \href{https://splat-distiller.pages.dev/}

Method: Developed C2PSA module for enhanced small-target feature extraction, implemented dynamic category weighting to handle sample imbalance, and improved data augmentation via Mosaic-MixUp scaling. Deployed on mobile system for real-time monitoring.

Result: Experimental results on 4,078-image dataset show: mAP50: 0.820 (+8.0% improvement); mAP50-95: 0.705 (+10.5% improvement); Inference speed: 158 FPS.

Conclusion: The optimized YOLOv11 system enables effective real-time cotton disease monitoring and precision treatment in agricultural applications with significant performance improvements over baseline methods.

Abstract: This study presents a deep learning-based optimization of YOLOv11 for cotton disease detection, developing an intelligent monitoring system. Three key challenges are addressed: (1) low precision in early spot detection (35% leakage rate for sub-5mm2 spots), (2) performance degradation in field conditions (25% accuracy drop), and (3) high error rates (34.7%) in multi-disease scenarios. The proposed solutions include: C2PSA module for enhanced small-target feature extraction; Dynamic category weighting to handle sample imbalance; Improved data augmentation via Mosaic-MixUp scaling. Experimental results on a 4,078-image dataset show: mAP50: 0.820 (+8.0% improvement); mAP50-95: 0.705 (+10.5% improvement); Inference speed: 158 FPS. The mobile-deployed system enables real-time disease monitoring and precision treatment in agricultural applications.

Method: Proposes a generative neural physics framework that couples generative networks with physics-informed neural simulation. Learns a compact surrogate model of ultrasonic wave propagation from only dozens of cross-modality images, merging wave modeling accuracy with deep learning efficiency.

Result: Achieves accurate quantitative imaging of in vivo musculoskeletal tissues, producing 3D spatial maps of acoustic properties. Reconstructs tissue parameter maps in under 10 minutes on synthetic and in vivo data (breast, arm, leg), with sensitivity to biomechanical properties and resolution comparable to MRI.

Conclusion: The approach overcomes computational bottlenecks in strongly scattering regimes, advancing USCT toward routine clinical assessment of musculoskeletal disease by providing fast, high-fidelity quantitative imaging capabilities.

Abstract: Ultrasound computed tomography (USCT) is a radiation-free, high-resolution modality but remains limited for musculoskeletal imaging due to conventional ray-based reconstructions that neglect strong scattering. We propose a generative neural physics framework that couples generative networks with physics-informed neural simulation for fast, high-fidelity 3D USCT. By learning a compact surrogate of ultrasonic wave propagation from only dozens of cross-modality images, our method merges the accuracy of wave modeling with the efficiency and stability of deep learning. This enables accurate quantitative imaging of in vivo musculoskeletal tissues, producing spatial maps of acoustic properties beyond reflection-mode images. On synthetic and in vivo data (breast, arm, leg), we reconstruct 3D maps of tissue parameters in under ten minutes, with sensitivity to biomechanical properties in muscle and bone and resolution comparable to MRI. By overcoming computational bottlenecks in strongly scattering regimes, this approach advances USCT toward routine clinical assessment of musculoskeletal disease.

Method: Created WXSOD dataset with synthesized and real weather noise. Proposed WFANet - a two-branch network with weather prediction branch and saliency detection branch that fuses semantic features with weather features.

Result: WFANet achieves superior performance compared to 17 existing SOD methods on the new WXSOD benchmark dataset.

Conclusion: The WXSOD dataset fills a critical gap in weather-affected SOD research, and the proposed WFANet demonstrates effective weather-aware feature fusion for improved salient object detection in adverse weather conditions.

Abstract: Salient object detection (SOD) in complex environments remains a challenging research topic. Most existing methods perform well in natural scenes with negligible noise, and tend to leverage multi-modal information (e.g., depth and infrared) to enhance accuracy. However, few studies are concerned with the damage of weather noise on SOD performance due to the lack of dataset with pixel-wise annotations. To bridge this gap, this paper introduces a novel Weather-eXtended Salient Object Detection (WXSOD) dataset. It consists of 14,945 RGB images with diverse weather noise, along with the corresponding ground truth annotations and weather labels. To verify algorithm generalization, WXSOD contains two test sets, i.e., a synthesized test set and a real test set. The former is generated by adding weather noise to clean images, while the latter contains real-world weather noise. Based on WXSOD, we propose an efficient baseline, termed Weather-aware Feature Aggregation Network (WFANet), which adopts a fully supervised two-branch architecture. Specifically, the weather prediction branch mines weather-related deep features, while the saliency detection branch fuses semantic features extracted from the backbone with weather features for SOD. Comprehensive comparisons against 17 SOD methods shows that our WFANet achieves superior performance on WXSOD. The code and benchmark results will be made publicly available at https://github.com/C-water/WXSOD

Method: Proposes superpixel-informed continuous tensor representation with asymmetric low-rank tensor factorization using shared neural network with specialized heads to capture both global patterns and local variations.

Result: Achieves 3-5 dB PSNR improvements over existing LRTR-based methods across multispectral images, videos, and color images on benchmark datasets.

Conclusion: SCTR provides a flexible continuous modeling framework that effectively handles spatial variations and diverse data forms while maintaining compact and expressive representations.

Abstract: Low-rank tensor representation (LRTR) has emerged as a powerful tool for multi-dimensional data processing. However, classical LRTR-based methods face two critical limitations: (1) they typically assume that the holistic data is low-rank, this assumption is often violated in real-world scenarios with significant spatial variations; and (2) they are constrained to discrete meshgrid data, limiting their flexibility and applicability. To overcome these limitations, we propose a Superpixel-informed Continuous low-rank Tensor Representation (SCTR) framework, which enables continuous and flexible modeling of multi-dimensional data beyond traditional grid-based constraints. Our approach introduces two main innovations: First, motivated by the observation that semantically coherent regions exhibit stronger low-rank characteristics than holistic data, we employ superpixels as the basic modeling units. This design not only encodes rich semantic information, but also enhances adaptability to diverse forms of data streams. Second, we propose a novel asymmetric low-rank tensor factorization (ALTF) where superpixel-specific factor matrices are parameterized by a shared neural network with specialized heads. By strategically separating global pattern learning from local adaptation, this framework efficiently captures both cross-superpixel commonalities and within-superpixel variations. This yields a representation that is both highly expressive and compact, balancing model efficiency with adaptability. Extensive experiments on several benchmark datasets demonstrate that SCTR achieves 3-5 dB PSNR improvements over existing LRTR-based methods across multispectral images, videos, and color images.

Method: Proposed Region-level Context-aware Visual Instruction Tuning (RCVIT) that incorporates object information and bounding box coordinates into model input, created RCMU dataset for training, and developed RC-Qwen2-VL models through RCVIT on Qwen2-VL.

Result: RC-Qwen2-VL models achieve outstanding performance on multiple RCMU tasks and demonstrate successful applications in multimodal RAG and personalized conversation.

Conclusion: The proposed RCMU framework effectively enhances MLLMs’ ability to integrate visual content with object textual context, providing comprehensive region-level context-aware multimodal understanding capabilities.

Abstract: Despite significant progress, existing research on Multimodal Large Language Models (MLLMs) mainly focuses on general visual understanding, overlooking the ability to integrate textual context associated with objects for a more context-aware multimodal understanding – an ability we refer to as Region-level Context-aware Multimodal Understanding (RCMU). To address this limitation, we first formulate the RCMU task, which requires models to respond to user instructions by integrating both image content and textual information of regions or objects. To equip MLLMs with RCMU capabilities, we propose Region-level Context-aware Visual Instruction Tuning (RCVIT), which incorporates object information into the model input and enables the model to utilize bounding box coordinates to effectively associate objects’ visual content with their textual information. To address the lack of datasets, we introduce the RCMU dataset, a large-scale visual instruction tuning dataset that covers multiple RCMU tasks. We also propose RC&P-Bench, a comprehensive benchmark that can evaluate the performance of MLLMs in RCMU and multimodal personalized understanding tasks. Additionally, we propose a reference-free evaluation metric to perform a comprehensive and fine-grained evaluation of the region-level context-aware image descriptions. By performing RCVIT on Qwen2-VL models with the RCMU dataset, we developed RC-Qwen2-VL models. Experimental results indicate that RC-Qwen2-VL models not only achieve outstanding performance on multiple RCMU tasks but also demonstrate successful applications in multimodal RAG and personalized conversation. Our data, model and benchmark are available at https://github.com/hongliang-wei/RC-MLLM

Method: Proposed SNNSIR with three key components: 1) Spike Residual Basic Block (SRBB) for enhanced information flow via spike-compatible residual learning, 2) Spike Stereo Convolutional Modulation (SSCM) module with simplified nonlinearity and cross-view-aware modulation, 3) Spike Stereo Cross-Attention (SSCA) module for efficient bidirectional feature interaction across views.

Result: Extensive experiments on stereo image restoration tasks (rain streak removal, raindrop removal, low-light enhancement, super-resolution) show competitive restoration performance while significantly reducing computational overhead.

Conclusion: The model demonstrates potential for real-time, low-power stereo vision applications with a fully spike-driven architecture that maintains performance while being hardware-friendly.

Abstract: Spiking Neural Networks (SNNs), characterized by discrete binary activations, offer high computational efficiency and low energy consumption, making them well-suited for computation-intensive tasks such as stereo image restoration. In this work, we propose SNNSIR, a simple yet effective Spiking Neural Network for Stereo Image Restoration, specifically designed under the spike-driven paradigm where neurons transmit information through sparse, event-based binary spikes. In contrast to existing hybrid SNN-ANN models that still rely on operations such as floating-point matrix division or exponentiation, which are incompatible with the binary and event-driven nature of SNNs, our proposed SNNSIR adopts a fully spike-driven architecture to achieve low-power and hardware-friendly computation. To address the expressiveness limitations of binary spiking neurons, we first introduce a lightweight Spike Residual Basic Block (SRBB) to enhance information flow via spike-compatible residual learning. Building on this, the Spike Stereo Convolutional Modulation (SSCM) module introduces simplified nonlinearity through element-wise multiplication and highlights noise-sensitive regions via cross-view-aware modulation. Complementing this, the Spike Stereo Cross-Attention (SSCA) module further improves stereo correspondence by enabling efficient bidirectional feature interaction across views within a spike-compatible framework. Extensive experiments on diverse stereo image restoration tasks, including rain streak removal, raindrop removal, low-light enhancement, and super-resolution demonstrate that our model achieves competitive restoration performance while significantly reducing computational overhead. These results highlight the potential for real-time, low-power stereo vision applications. The code will be available after the article is accepted.

Method: Three-tier control mechanism (width multiplier, classifier depth, classifier kernel) for fine-grained model adaptation, combined with Bayesian Optimization for efficient hyperparameter search under tight computational budgets.

Result: Enables broad model scaling, targeted refinement of final layers, and scenario-specific optimization, leading to improved resource allocation and performance with task-specific learning adaptation.

Conclusion: The approach successfully addresses scenario-specific requirements through automatic parameter search, maximizing computational capacity and model accuracy while optimizing hardware utilization for diverse self-driving tasks.

Abstract: Autonomous driving platforms encounter diverse driving scenarios, each with varying hardware resources and precision requirements. Given the computational limitations of embedded devices, it is crucial to consider computing costs when deploying on target platforms like the NVIDIA\textsuperscript{\textregistered} DRIVE PX 2. Our objective is to customize the semantic segmentation network according to the computing power and specific scenarios of autonomous driving hardware. We implement dynamic adaptability through a three-tier control mechanism – width multiplier, classifier depth, and classifier kernel – allowing fine-grained control over model components based on hardware constraints and task requirements. This adaptability facilitates broad model scaling, targeted refinement of the final layers, and scenario-specific optimization of kernel sizes, leading to improved resource allocation and performance. Additionally, we leverage Bayesian Optimization with surrogate modeling to efficiently explore hyperparameter spaces under tight computational budgets. Our approach addresses scenario-specific and task-specific requirements through automatic parameter search, accommodating the unique computational complexity and accuracy needs of autonomous driving. It scales its Multiply-Accumulate Operations (MACs) for Task-Specific Learning Adaptation (TSLA), resulting in alternative configurations tailored to diverse self-driving tasks. These TSLA customizations maximize computational capacity and model accuracy, optimizing hardware utilization.

Method: Proposes CLAIR with confidence scoring for pseudo-label refinement, inter-instance/inter-cluster contrastive losses, cross-domain mapping function using CLIP text embeddings, and learnable prompts for zero-shot generalization

Result: Superior performance on TUBerlin, Sketchy, Quickdraw, and DomainNet zero-shot datasets compared to state-of-the-art methods

Conclusion: CLAIR effectively handles noisy pseudo-labels from foundation models and achieves strong cross-domain retrieval performance through confidence scoring, contrastive learning, and domain alignment techniques

Abstract: The recent growth of large foundation models that can easily generate pseudo-labels for huge quantity of unlabeled data makes unsupervised Zero-Shot Cross-Domain Image Retrieval (UZS-CDIR) less relevant. In this paper, we therefore turn our attention to weakly supervised ZS-CDIR (WSZS-CDIR) with noisy pseudo labels generated by large foundation models such as CLIP. To this end, we propose CLAIR to refine the noisy pseudo-labels with a confidence score from the similarity between the CLIP text and image features. Furthermore, we design inter-instance and inter-cluster contrastive losses to encode images into a class-aware latent space, and an inter-domain contrastive loss to alleviate domain discrepancies. We also learn a novel cross-domain mapping function in closed-form, using only CLIP text embeddings to project image features from one domain to another, thereby further aligning the image features for retrieval. Finally, we enhance the zero-shot generalization ability of our CLAIR to handle novel categories by introducing an extra set of learnable prompts. Extensive experiments are carried out using TUBerlin, Sketchy, Quickdraw, and DomainNet zero-shot datasets, where our CLAIR consistently shows superior performance compared to existing state-of-the-art methods.

Method: Proposes Edge-Aware Score for candidate selection, Long-Axis Split strategy to reduce geometric distortions, and overfitting mitigation techniques including Recovery-Aware Pruning, Multi-step Update, and Growth Control.

Result: Achieves state-of-the-art performance with enhanced rendering fidelity using fewer Gaussians, without additional training or inference overhead.

Conclusion: The comprehensive improvements to 3DGS densification pipeline significantly enhance reconstruction quality while maintaining computational efficiency.

Abstract: Although 3D Gaussian Splatting (3DGS) has achieved impressive performance in real-time rendering, its densification strategy often results in suboptimal reconstruction quality. In this work, we present a comprehensive improvement to the densification pipeline of 3DGS from three perspectives: when to densify, how to densify, and how to mitigate overfitting. Specifically, we propose an Edge-Aware Score to effectively select candidate Gaussians for splitting. We further introduce a Long-Axis Split strategy that reduces geometric distortions introduced by clone and split operations. To address overfitting, we design a set of techniques, including Recovery-Aware Pruning, Multi-step Update, and Growth Control. Our method enhances rendering fidelity without introducing additional training or inference overhead, achieving state-of-the-art performance with fewer Gaussians.

Method: Proposes NCA-WSS (neural cellular automata for weakly supervised segmentation) that leverages feature maps generated by neural cellular automata during classification to extract segmentation masks without retraining with segmentation labels.

Result: The method significantly outperforms existing weakly supervised approaches on three white blood cell microscopy datasets, demonstrating superior segmentation performance.

Conclusion: NCA-WSS shows strong potential for both classification and segmentation in weakly supervised frameworks, providing a scalable and efficient solution for medical image analysis tasks.

Abstract: The detection and segmentation of white blood cells in blood smear images is a key step in medical diagnostics, supporting various downstream tasks such as automated blood cell counting, morphological analysis, cell classification, and disease diagnosis and monitoring. Training robust and accurate models requires large amounts of labeled data, which is both time-consuming and expensive to acquire. In this work, we propose a novel approach for weakly supervised segmentation using neural cellular automata (NCA-WSS). By leveraging the feature maps generated by NCA during classification, we can extract segmentation masks without the need for retraining with segmentation labels. We evaluate our method on three white blood cell microscopy datasets and demonstrate that NCA-WSS significantly outperforms existing weakly supervised approaches. Our work illustrates the potential of NCA for both classification and segmentation in a weakly supervised framework, providing a scalable and efficient solution for medical image analysis.

Method: Combined attention pooling mechanism with Neural Cellular Automata to enhance feature extraction by focusing on informative regions.

Result: Significantly outperformed existing NCA methods on eight microscopy datasets, achieved better performance than lightweight CNNs and vision transformers with lower parameter count.

Conclusion: NCA-based models with attention pooling offer a promising alternative for explainable image classification, balancing performance and interpretability.

Abstract: Neural Cellular Automata (NCA) offer a robust and interpretable approach to image classification, making them a promising choice for microscopy image analysis. However, a performance gap remains between NCA and larger, more complex architectures. We address this challenge by integrating attention pooling with NCA to enhance feature extraction and improve classification accuracy. The attention pooling mechanism refines the focus on the most informative regions, leading to more accurate predictions. We evaluate our method on eight diverse microscopy image datasets and demonstrate that our approach significantly outperforms existing NCA methods while remaining parameter-efficient and explainable. Furthermore, we compare our method with traditional lightweight convolutional neural network and vision transformer architectures, showing improved performance while maintaining a significantly lower parameter count. Our results highlight the potential of NCA-based models an alternative for explainable image classification.

Method: Points from previous frames are shifted radially using their dynamic Doppler component to eliminate radial scatter, with unique aggregation durations assigned based on Doppler and angle to minimize tangential scatter.

Result: DoppDrive significantly improves object detection performance across various detectors and datasets as a pre-detection enhancement step.

Conclusion: The proposed Doppler-driven temporal aggregation method effectively enhances radar point cloud density while minimizing scatter, making it compatible with any detector and beneficial for autonomous driving applications.

Abstract: Radar-based object detection is essential for autonomous driving due to radar’s long detection range. However, the sparsity of radar point clouds, especially at long range, poses challenges for accurate detection. Existing methods increase point density through temporal aggregation with ego-motion compensation, but this approach introduces scatter from dynamic objects, degrading detection performance. We propose DoppDrive, a novel Doppler-Driven temporal aggregation method that enhances radar point cloud density while minimizing scatter. Points from previous frames are shifted radially according to their dynamic Doppler component to eliminate radial scatter, with each point assigned a unique aggregation duration based on its Doppler and angle to minimize tangential scatter. DoppDrive is a point cloud density enhancement step applied before detection, compatible with any detector, and we demonstrate that it significantly improves object detection performance across various detectors and datasets.

Method: Integrates GAN-based video inpainting guided by dense facial landmarks and a reference frame, followed by SynergyNet-based 3DMM parameter regression with dense landmark optimization throughout the pipeline.

Result: Successfully removes HMDs while maintaining facial identity and realism, producing photorealistic 3D face geometry outputs. Robust across different landmark densities with minor quality degradation under sparse configurations.

Conclusion: The framework effectively addresses HMD occlusion issues in XR applications, enabling complete facial reconstruction for improved social interaction and immersive experiences.

Abstract: Head-mounted displays (HMDs) are essential for experiencing extended reality (XR) environments and observing virtual content. However, they obscure the upper part of the user’s face, complicating external video recording and significantly impacting social XR applications such as teleconferencing, where facial expressions and eye gaze details are crucial for creating an immersive experience. This study introduces a geometry-aware learning-based framework to jointly remove HMD occlusions and reconstruct complete 3D facial geometry from RGB frames captured from a single viewpoint. The method integrates a GAN-based video inpainting network, guided by dense facial landmarks and a single occlusion-free reference frame, to restore missing facial regions while preserving identity. Subsequently, a SynergyNet-based module regresses 3D Morphable Model (3DMM) parameters from the inpainted frames, enabling accurate 3D face reconstruction. Dense landmark optimization is incorporated throughout the pipeline to improve both the inpainting quality and the fidelity of the recovered geometry. Experimental results demonstrate that the proposed framework can successfully remove HMDs from RGB facial videos while maintaining facial identity and realism, producing photorealistic 3D face geometry outputs. Ablation studies further show that the framework remains robust across different landmark densities, with only minor quality degradation under sparse landmark configurations.

Method: Uses semantic token sampling to mitigate irrelevant feature shifts, concept-level forgery discrepancy learning through visual reconstruction, and low-level forgery feature enhancement.

Result: Achieves superior results on two standard image forgery datasets compared to existing methods.

Conclusion: SDD effectively aligns semantic and forgery spaces, demonstrating strong performance in detecting fake images through semantic-guided discrepancy learning.

Abstract: With the rapid advancement of image generation techniques, robust forgery detection has become increasingly imperative to ensure the trustworthiness of digital media. Recent research indicates that the learned semantic concepts of pre-trained models are critical for identifying fake images. However, the misalignment between the forgery and semantic concept spaces hinders the model’s forgery detection performance. To address this problem, we propose a novel Semantic Discrepancy-aware Detector (SDD) that leverages reconstruction learning to align the two spaces at a fine-grained visual level. By exploiting the conceptual knowledge embedded in the pre-trained vision language model, we specifically design a semantic token sampling module to mitigate the space shifts caused by features irrelevant to both forgery traces and semantic concepts. A concept-level forgery discrepancy learning module, built upon a visual reconstruction paradigm, is proposed to strengthen the interaction between visual semantic concepts and forgery traces, effectively capturing discrepancies under the concepts’ guidance. Finally, the low-level forgery feature enhancemer integrates the learned concept level forgery discrepancies to minimize redundant forgery information. Experiments conducted on two standard image forgery datasets demonstrate the efficacy of the proposed SDD, which achieves superior results compared to existing methods. The code is available at https://github.com/wzy1111111/SSD.

Method: A multi-scale feature enhancement network trained end-to-end with the detector’s loss function, ensuring enhancement is guided to refine features most relevant to detection tasks. Integrated with YOLOv8m.

Result: Achieves state-of-the-art Precision (0.877) and Recall (0.624), with competitive mAP scores (mAP@0.5 of 0.677 and mAP@[0.5:0.95] of 0.421) while maintaining 46.5 FPS processing speed.

Conclusion: AquaFeat provides an effective and computationally efficient solution for real-world underwater applications like marine ecosystem monitoring and infrastructure inspection.

Abstract: The severe image degradation in underwater environments impairs object detection models, as traditional image enhancement methods are often not optimized for such downstream tasks. To address this, we propose AquaFeat, a novel, plug-and-play module that performs task-driven feature enhancement. Our approach integrates a multi-scale feature enhancement network trained end-to-end with the detector’s loss function, ensuring the enhancement process is explicitly guided to refine features most relevant to the detection task. When integrated with YOLOv8m on challenging underwater datasets, AquaFeat achieves state-of-the-art Precision (0.877) and Recall (0.624), along with competitive mAP scores (mAP@0.5 of 0.677 and mAP@[0.5:0.95] of 0.421). By delivering these accuracy gains while maintaining a practical processing speed of 46.5 FPS, our model provides an effective and computationally efficient solution for real-world applications, such as marine ecosystem monitoring and infrastructure inspection.

Method: Propose MBMamba with two key components: 1) memory buffer mechanism to preserve historical information for fusion and model relevance between adjacent features, 2) Ising-inspired regularization loss that simulates energy minimization of pixel mutual attraction to maintain image structure.

Result: Outperforms state-of-the-art approaches on widely used benchmarks.

Conclusion: The proposed MBMamba effectively addresses Mamba’s limitations for image deblurring without modifying the original architecture, achieving superior performance while maintaining computational efficiency.

Abstract: The Mamba architecture has emerged as a promising alternative to CNNs and Transformers for image deblurring. However, its flatten-and-scan strategy often results in local pixel forgetting and channel redundancy, limiting its ability to effectively aggregate 2D spatial information. Although existing methods mitigate this by modifying the scan strategy or incorporating local feature modules, it increase computational complexity and hinder real-time performance. In this paper, we propose a structure-aware image deblurring network without changing the original Mamba architecture. Specifically, we design a memory buffer mechanism to preserve historical information for later fusion, enabling reliable modeling of relevance between adjacent features. Additionally, we introduce an Ising-inspired regularization loss that simulates the energy minimization of the physical system’s “mutual attraction” between pixels, helping to maintain image structure and coherence. Building on this, we develop MBMamba. Experimental results show that our method outperforms state-of-the-art approaches on widely used benchmarks.

Method: Proposes EgoLoc with hand-dynamics-guided sampling to generate visual prompts, uses vision-language models to identify contact/separation attributes and localize timestamps, and employs closed-loop feedback for refinement.

Result: Achieves plausible temporal interaction localization on public datasets and novel benchmarks, effectively facilitating downstream applications in egocentric vision and robotic manipulation tasks.

Conclusion: EgoLoc provides a generalizable zero-shot solution for fine-grained hand-object interaction timing that eliminates dependency on object masks and verb-noun taxonomies, demonstrating strong performance across multiple applications.

Abstract: Analyzing hand-object interaction in egocentric vision facilitates VR/AR applications and human-robot policy transfer. Existing research has mostly focused on modeling the behavior paradigm of interactive actions (i.e., how to interact''). However, the more challenging and fine-grained problem of capturing the critical moments of contact and separation between the hand and the target object (i.e., when to interact’’) is still underexplored, which is crucial for immersive interactive experiences in mixed reality and robotic motion planning. Therefore, we formulate this problem as temporal interaction localization (TIL). Some recent works extract semantic masks as TIL references, but suffer from inaccurate object grounding and cluttered scenarios. Although current temporal action localization (TAL) methods perform well in detecting verb-noun action segments, they rely on category annotations during training and exhibit limited precision in localizing hand-object contact/separation moments. To address these issues, we propose a novel zero-shot approach dubbed EgoLoc to localize hand-object contact and separation timestamps in egocentric videos. EgoLoc introduces hand-dynamics-guided sampling to generate high-quality visual prompts. It exploits the vision-language model to identify contact/separation attributes, localize specific timestamps, and provide closed-loop feedback for further refinement. EgoLoc eliminates the need for object masks and verb-noun taxonomies, leading to generalizable zero-shot implementation. Comprehensive experiments on the public dataset and our novel benchmarks demonstrate that EgoLoc achieves plausible TIL for egocentric videos. It is also validated to effectively facilitate multiple downstream applications in egocentric vision and robotic manipulation tasks. Code and relevant data will be released at https://github.com/IRMVLab/EgoLoc.

Method: Two-step process: first augmenting original offline data to introduce diversity, then using diffusion models to generate additional synthetic data in latent space.

Result: Significantly improves generalization across continuous (Visual D4RL) and discrete (Procgen) action spaces without requiring algorithmic changes to existing model-free offline RL methods.

Conclusion: The approach increases training data diversity, reduces generalization gap, maintains computational efficiency, and could fuel progress in synthetic data generation for training more general agents.

Abstract: Offline reinforcement learning (RL) offers a promising framework for training agents using pre-collected datasets without the need for further environment interaction. However, policies trained on offline data often struggle to generalise due to limited exposure to diverse states. The complexity of visual data introduces additional challenges such as noise, distractions, and spurious correlations, which can misguide the policy and increase the risk of overfitting if the training data is not sufficiently diverse. Indeed, this makes it challenging to leverage vision-based offline data in training robust agents that can generalize to unseen environments. To solve this problem, we propose a simple approach generating additional synthetic training data. We propose a two-step process, first augmenting the originally collected offline data to improve zero-shot generalization by introducing diversity, then using a diffusion model to generate additional data in latent space. We test our method across both continuous action spaces (Visual D4RL) and discrete action spaces (Procgen), demonstrating that it significantly improves generalization without requiring any algorithmic changes to existing model-free offline RL methods. We show that our method not only increases the diversity of the training data but also significantly reduces the generalization gap at test time while maintaining computational efficiency. We believe this approach could fuel additional progress in generating synthetic data to train more general agents in the future.

Method: Proposes Interpretable Pathology Graph-Transformer (IPGPhormer) that captures tumor microenvironment characteristics and models spatial dependencies across tissue, providing interpretability at both tissue and cellular levels without post-hoc annotations.

Result: Comprehensive evaluations on four public benchmark datasets show IPGPhormer outperforms state-of-the-art methods in both predictive accuracy and interpretability.

Conclusion: IPGPhormer offers a promising tool for cancer prognosis assessment, enabling more reliable and interpretable decision-support systems in pathology.

Abstract: Pathological images play an essential role in cancer prognosis, while survival analysis, which integrates computational techniques, can predict critical clinical events such as patient mortality or disease recurrence from whole-slide images (WSIs). Recent advancements in multiple instance learning have significantly improved the efficiency of survival analysis. However, existing methods often struggle to balance the modeling of long-range spatial relationships with local contextual dependencies and typically lack inherent interpretability, limiting their clinical utility. To address these challenges, we propose the Interpretable Pathology Graph-Transformer (IPGPhormer), a novel framework that captures the characteristics of the tumor microenvironment and models their spatial dependencies across the tissue. IPGPhormer uniquely provides interpretability at both tissue and cellular levels without requiring post-hoc manual annotations, enabling detailed analyses of individual WSIs and cross-cohort assessments. Comprehensive evaluations on four public benchmark datasets demonstrate that IPGPhormer outperforms state-of-the-art methods in both predictive accuracy and interpretability. In summary, our method, IPGPhormer, offers a promising tool for cancer prognosis assessment, paving the way for more reliable and interpretable decision-support systems in pathology. The code is publicly available at https://anonymous.4open.science/r/IPGPhormer-6EEB.

Method: Applies adversarial augmentation to surrogate ViTs using three strategies: Multi-head dropping, Attention score scaling, and MLP feature mixing with Bayesian optimization. Includes Automatic Reweighting and Step Size Enlargement modules.

Result: Significantly enhances adversarial transferability of ensemble-based attacks on ViTs, outperforming existing methods by a substantial margin in extensive experiments.

Conclusion: The proposed ViT-EnsembleAttack successfully addresses the gap in ensemble model exploration for adversarial attacks, particularly for Vision Transformers, demonstrating superior transferability performance.

Abstract: Ensemble-based attacks have been proven to be effective in enhancing adversarial transferability by aggregating the outputs of models with various architectures. However, existing research primarily focuses on refining ensemble weights or optimizing the ensemble path, overlooking the exploration of ensemble models to enhance the transferability of adversarial attacks. To address this gap, we propose applying adversarial augmentation to the surrogate models, aiming to boost overall generalization of ensemble models and reduce the risk of adversarial overfitting. Meanwhile, observing that ensemble Vision Transformers (ViTs) gain less attention, we propose ViT-EnsembleAttack based on the idea of model adversarial augmentation, the first ensemble-based attack method tailored for ViTs to the best of our knowledge. Our approach generates augmented models for each surrogate ViT using three strategies: Multi-head dropping, Attention score scaling, and MLP feature mixing, with the associated parameters optimized by Bayesian optimization. These adversarially augmented models are ensembled to generate adversarial examples. Furthermore, we introduce Automatic Reweighting and Step Size Enlargement modules to boost transferability. Extensive experiments demonstrate that ViT-EnsembleAttack significantly enhances the adversarial transferability of ensemble-based attacks on ViTs, outperforming existing methods by a substantial margin. Code is available at https://github.com/Trustworthy-AI-Group/TransferAttack.

Method: Two-stage framework: 1) Complex Instruction Decomposition and Semantic Enhancement using LLMs to break down instructions into structured semantic units, 2) Multi-Stage Prompt Integration and Adaptive Generation to transform units into hierarchical or optimized prompts for T2I models.

Result: DeCoT consistently improves T2I model performance across all dimensions, achieving average score of 3.52 vs baseline 3.44 on LongBench-T2I, with significant gains in challenging aspects like “Text” and “Composition”. Human evaluations confirm superior perceptual quality and instruction fidelity.

Conclusion: DeCoT effectively bridges the gap between high-level user intent and T2I model requirements, leading to more faithful and accurate image generation through sophisticated LLM prompting and instruction decomposition.

Abstract: Despite remarkable advancements, current Text-to-Image (T2I) models struggle with complex, long-form textual instructions, frequently failing to accurately render intricate details, spatial relationships, or specific constraints. This limitation is highlighted by benchmarks such as LongBench-T2I, which reveal deficiencies in handling composition, specific text, and fine textures. To address this, we propose DeCoT (Decomposition-CoT), a novel framework that leverages Large Language Models (LLMs) to significantly enhance T2I models’ understanding and execution of complex instructions. DeCoT operates in two core stages: first, Complex Instruction Decomposition and Semantic Enhancement, where an LLM breaks down raw instructions into structured, actionable semantic units and clarifies ambiguities; second, Multi-Stage Prompt Integration and Adaptive Generation, which transforms these units into a hierarchical or optimized single prompt tailored for existing T2I models. Extensive experiments on the LongBench-T2I dataset demonstrate that DeCoT consistently and substantially improves the performance of leading T2I models across all evaluated dimensions, particularly in challenging aspects like “Text” and “Composition”. Quantitative results, validated by multiple MLLM evaluators (Gemini-2.0-Flash and InternVL3-78B), show that DeCoT, when integrated with Infinity-8B, achieves an average score of 3.52, outperforming the baseline Infinity-8B (3.44). Ablation studies confirm the critical contribution of each DeCoT component and the importance of sophisticated LLM prompting. Furthermore, human evaluations corroborate these findings, indicating superior perceptual quality and instruction fidelity. DeCoT effectively bridges the gap between high-level user intent and T2I model requirements, leading to more faithful and accurate image generation.

Method: FedCSAP extracts low, mid, and high-level features from CLIP’s vision encoder and combines them with client-specific style indicators from batch-level statistics. It merges visual details with textual context to generate distinct, non-redundant prompt tokens within a federated learning paradigm with local training and global aggregation.

Result: Comprehensive experiments on multiple image classification datasets show that FedCSAP outperforms existing federated prompt learning methods in both accuracy and overall generalization across seen and unseen classes.

Conclusion: FedCSAP effectively bridges the gap in conventional prompt learning by incorporating multi-scale visual features and domain-specific style variations, demonstrating superior performance in federated learning scenarios with non-IID data distributions while ensuring data privacy.

Abstract: Prompt learning has propelled vision-language models like CLIP to excel in diverse tasks, making them ideal for federated learning due to computational efficiency. However, conventional approaches that rely solely on final-layer features miss out on rich multi-scale visual cues and domain-specific style variations in decentralized client data. To bridge this gap, we introduce FedCSAP (Federated Cross-Modal Style-Aware Prompt Generation). Our framework harnesses low, mid, and high-level features from CLIP’s vision encoder alongside client-specific style indicators derived from batch-level statistics. By merging intricate visual details with textual context, FedCSAP produces robust, context-aware prompt tokens that are both distinct and non-redundant, thereby boosting generalization across seen and unseen classes. Operating within a federated learning paradigm, our approach ensures data privacy through local training and global aggregation, adeptly handling non-IID class distributions and diverse domain-specific styles. Comprehensive experiments on multiple image classification datasets confirm that FedCSAP outperforms existing federated prompt learning methods in both accuracy and overall generalization.

Method: Three-stage approach: 1) LVLM generates N diverse complementary descriptions from various angles, 2) these descriptions are intelligently integrated with original question to create comprehensive context-augmented prompt, 3) enriched prompt guides final LVLM for deep reasoning and answer generation.

Result: Significant accuracy gains on challenging VQA datasets (GQA, VQA-CP v2, ScienceQA), particularly for tasks requiring robust contextual understanding. Human evaluations confirm improved coherence and completeness of answers.

Conclusion: MPCAR effectively leverages LVLMs’ generative capabilities to enrich input contexts, unlocking their latent reasoning potential for complex multimodal tasks without parameter fine-tuning, demonstrating the value of multi-perspective contextual augmentation.

Abstract: Despite significant advancements, Large Vision-Language Models (LVLMs) continue to face challenges in complex visual reasoning tasks that demand deep contextual understanding, multi-angle analysis, or meticulous detail recognition. Existing approaches often rely on single-shot image encoding and prompts, limiting their ability to fully capture nuanced visual information. Inspired by the notion that strategically generated “additional” information can serve as beneficial contextual augmentation, we propose Multi-Perspective Contextual Augmentation for Reasoning (MPCAR), a novel inference-time strategy designed to enhance LVLM performance. MPCAR operates in three stages: first, an LVLM generates N diverse and complementary descriptions or preliminary reasoning paths from various angles; second, these descriptions are intelligently integrated with the original question to construct a comprehensive context-augmented prompt; and finally, this enriched prompt guides the ultimate LVLM for deep reasoning and final answer generation. Crucially, MPCAR achieves these enhancements without requiring any fine-tuning of the underlying LVLM’s parameters. Extensive experiments on challenging Visual Question Answering (VQA) datasets, including GQA, VQA-CP v2, and ScienceQA (Image-VQA), demonstrate that MPCAR consistently outperforms established baseline methods. Our quantitative results show significant accuracy gains, particularly on tasks requiring robust contextual understanding, while human evaluations confirm improved coherence and completeness of the generated answers. Ablation studies further highlight the importance of diverse prompt templates and the number of generated perspectives. This work underscores the efficacy of leveraging LVLMs’ inherent generative capabilities to enrich input contexts, thereby unlocking their latent reasoning potential for complex multimodal tasks.

Method: Proposes LMAD framework that emulates end-to-end driving paradigms with preliminary scene interaction and specialized expert adapters within the same task structure, designed to be compatible with existing VLMs and planning systems.

Result: Extensive experiments on DriveLM and nuScenes-QA datasets show LMAD significantly boosts VLM performance on driving reasoning tasks.

Conclusion: LMAD sets a new standard in explainable autonomous driving by better aligning VLMs with autonomous driving scenarios through specialized architecture and scene understanding capabilities.

Abstract: Large vision-language models (VLMs) have shown promising capabilities in scene understanding, enhancing the explainability of driving behaviors and interactivity with users. Existing methods primarily fine-tune VLMs on on-board multi-view images and scene reasoning text, but this approach often lacks the holistic and nuanced scene recognition and powerful spatial awareness required for autonomous driving, especially in complex situations. To address this gap, we propose a novel vision-language framework tailored for autonomous driving, called LMAD. Our framework emulates modern end-to-end driving paradigms by incorporating comprehensive scene understanding and a task-specialized structure with VLMs. In particular, we introduce preliminary scene interaction and specialized expert adapters within the same driving task structure, which better align VLMs with autonomous driving scenarios. Furthermore, our approach is designed to be fully compatible with existing VLMs while seamlessly integrating with planning-oriented driving systems. Extensive experiments on the DriveLM and nuScenes-QA datasets demonstrate that LMAD significantly boosts the performance of existing VLMs on driving reasoning tasks,setting a new standard in explainable autonomous driving.

Method: Proposed S5 framework with: 1) Data selection strategy combining entropy-based filtering and diversity expansion to create RS4P-1M dataset, 2) Pre-training RS foundation models of varying sizes on this extensive corpus, 3) Mixture-of-Experts-based multi-dataset fine-tuning approach for efficient adaptation to multiple benchmarks.

Result: The resulting RS foundation models achieve state-of-the-art performance across all benchmarks, significantly boosting performance on land cover segmentation and object detection tasks while improving generalization and versatility.

Conclusion: S5 demonstrates the viability of scaling semi-supervised learning for remote sensing applications, unlocking the potential of vast unlabeled Earth observation data and providing a scalable framework for practical RS analysis.

Abstract: Semi-supervised semantic segmentation (S4) has advanced remote sensing (RS) analysis by leveraging unlabeled data through pseudo-labeling and consistency learning. However, existing S4 studies often rely on small-scale datasets and models, limiting their practical applicability. To address this, we propose S5, the first scalable framework for semi-supervised semantic segmentation in RS, which unlocks the potential of vast unlabeled Earth observation data typically underutilized due to costly pixel-level annotations. Built upon existing large-scale RS datasets, S5 introduces a data selection strategy that integrates entropy-based filtering and diversity expansion, resulting in the RS4P-1M dataset. Using this dataset, we systematically scales S4 methods by pre-training RS foundation models (RSFMs) of varying sizes on this extensive corpus, significantly boosting their performance on land cover segmentation and object detection tasks. Furthermore, during fine-tuning, we incorporate a Mixture-of-Experts (MoE)-based multi-dataset fine-tuning approach, which enables efficient adaptation to multiple RS benchmarks with fewer parameters. This approach improves the generalization and versatility of RSFMs across diverse RS benchmarks. The resulting RSFMs achieve state-of-the-art performance across all benchmarks, underscoring the viability of scaling semi-supervised learning for RS applications. All datasets, code, and models will be released at https://github.com/MiliLab/S5

Method: Proposes SRMA-Mamba with Spatial Anatomy-Based Mamba module (SABMamba) that performs selective scans within cirrhotic tissues and combines anatomical information from sagittal, coronal, and axial planes. Also introduces Spatial Reverse Attention module (SRMA) to progressively refine segmentation details using coarse maps and hierarchical encoding features.

Result: Extensive experiments demonstrate that SRMA-Mamba surpasses state-of-the-art methods, delivering exceptional performance in 3D pathological liver segmentation.

Conclusion: The proposed SRMA-Mamba network effectively addresses the challenge of modeling spatial relationships in complex liver anatomical structures, providing superior volumetric segmentation of pathological liver tissues with improved clinical utility.

Abstract: Liver Cirrhosis plays a critical role in the prognosis of chronic liver disease. Early detection and timely intervention are critical in significantly reducing mortality rates. However, the intricate anatomical architecture and diverse pathological changes of liver tissue complicate the accurate detection and characterization of lesions in clinical settings. Existing methods underutilize the spatial anatomical details in volumetric MRI data, thereby hindering their clinical effectiveness and explainability. To address this challenge, we introduce a novel Mamba-based network, SRMA-Mamba, designed to model the spatial relationships within the complex anatomical structures of MRI volumes. By integrating the Spatial Anatomy-Based Mamba module (SABMamba), SRMA-Mamba performs selective Mamba scans within liver cirrhotic tissues and combines anatomical information from the sagittal, coronal, and axial planes to construct a global spatial context representation, enabling efficient volumetric segmentation of pathological liver structures. Furthermore, we introduce the Spatial Reverse Attention module (SRMA), designed to progressively refine cirrhotic details in the segmentation map, utilizing both the coarse segmentation map and hierarchical encoding features. Extensive experiments demonstrate that SRMA-Mamba surpasses state-of-the-art methods, delivering exceptional performance in 3D pathological liver segmentation. Our code is available for public: {\color{blue}{https://github.com/JunZengz/SRMA-Mamba}}.

Method: Dual-branch Generation Model (panorama branch for global view, perspective branch for local view) with bidirectional cross-attention, plus Geometry-aligned Reconstruction Model based on 3D Gaussian Splatting using metric depth maps and estimated camera poses.

Result: Extensive experiments show the framework effectively generates visually compelling and motion-coherent dynamic panoramic scenes with geometric consistency and temporal coherence.

Conclusion: TiP4GEN successfully addresses the gap in 360-degree immersive dynamic scene generation, enabling high-quality panoramic 4D scene creation with comprehensive information exchange and geometric alignment.

Abstract: With the rapid advancement and widespread adoption of VR/AR technologies, there is a growing demand for the creation of high-quality, immersive dynamic scenes. However, existing generation works predominantly concentrate on the creation of static scenes or narrow perspective-view dynamic scenes, falling short of delivering a truly 360-degree immersive experience from any viewpoint. In this paper, we introduce \textbf{TiP4GEN}, an advanced text-to-dynamic panorama scene generation framework that enables fine-grained content control and synthesizes motion-rich, geometry-consistent panoramic 4D scenes. TiP4GEN integrates panorama video generation and dynamic scene reconstruction to create 360-degree immersive virtual environments. For video generation, we introduce a \textbf{Dual-branch Generation Model} consisting of a panorama branch and a perspective branch, responsible for global and local view generation, respectively. A bidirectional cross-attention mechanism facilitates comprehensive information exchange between the branches. For scene reconstruction, we propose a \textbf{Geometry-aligned Reconstruction Model} based on 3D Gaussian Splatting. By aligning spatial-temporal point clouds using metric depth maps and initializing scene cameras with estimated poses, our method ensures geometric consistency and temporal coherence for the reconstructed scenes. Extensive experiments demonstrate the effectiveness of our proposed designs and the superiority of TiP4GEN in generating visually compelling and motion-coherent dynamic panoramic scenes. Our project page is at https://ke-xing.github.io/TiP4GEN/.

Method: Systematic comparison of human and AI responses to classic visual illusions involving color, size, shape, and motion, examining both targeted training effects and emergent pattern recognition behaviors.

Result: AI exhibits some illusion-like effects through training, but also has unique AI-specific illusions like pixel-level sensitivity and hallucinations that lack human counterparts, revealing alignment gaps and perceptual vulnerabilities.

Conclusion: Findings provide insights for developing vision systems that preserve beneficial human perceptual biases while avoiding AI-specific distortions that undermine trust and safety.

Abstract: By comparing biological and artificial perception through the lens of illusions, we highlight critical differences in how each system constructs visual reality. Understanding these divergences can inform the development of more robust, interpretable, and human-aligned artificial intelligence (AI) vision systems. In particular, visual illusions expose how human perception is based on contextual assumptions rather than raw sensory data. As artificial vision systems increasingly perform human-like tasks, it is important to ask: does AI experience illusions, too? Does it have unique illusions? This article explores how AI responds to classic visual illusions that involve color, size, shape, and motion. We find that some illusion-like effects can emerge in these models, either through targeted training or as by-products of pattern recognition. In contrast, we also identify illusions unique to AI, such as pixel-level sensitivity and hallucinations, that lack human counterparts. By systematically comparing human and AI responses to visual illusions, we uncover alignment gaps and AI-specific perceptual vulnerabilities invisible to human perception. These findings provide insights for future research on vision systems that preserve human-beneficial perceptual biases while avoiding distortions that undermine trust and safety.

Method: Leveraged existing adversarial question perturbation and proposed a novel image alteration strategy to induce contradictory outputs. Introduced a mitigation method using external knowledge to alleviate inconsistencies and improve model robustness.

Result: Extensive evaluations on two standard benchmarks and two widely used VQA-NLE models demonstrated the effectiveness of the adversarial attacks and showed the potential of knowledge-based defenses in improving system reliability.

Conclusion: The research reveals pressing security and reliability concerns in current VQA-NLE systems, highlighting the need for more robust explanation mechanisms and the value of external knowledge integration for defense.

Abstract: Natural language explanations in visual question answering (VQA-NLE) aim to make black-box models more transparent by elucidating their decision-making processes. However, we find that existing VQA-NLE systems can produce inconsistent explanations and reach conclusions without genuinely understanding the underlying context, exposing weaknesses in either their inference pipeline or explanation-generation mechanism. To highlight these vulnerabilities, we not only leverage an existing adversarial strategy to perturb questions but also propose a novel strategy that minimally alters images to induce contradictory or spurious outputs. We further introduce a mitigation method that leverages external knowledge to alleviate these inconsistencies, thereby bolstering model robustness. Extensive evaluations on two standard benchmarks and two widely used VQA-NLE models underscore the effectiveness of our attacks and the potential of knowledge-based defenses, ultimately revealing pressing security and reliability concerns in current VQA-NLE systems.

Method: X-Ray-CoT leverages Vision-Language Large Models (LVLMs) to simulate human radiologists’ chain-of-thought: first extracts multi-modal features and visual concepts, then uses an LLM-based component with structured Chain-of-Thought prompting to reason and generate detailed diagnostic reports.

Result: On the CORDA dataset, X-Ray-CoT achieves Balanced Accuracy of 80.52% and F1 score of 78.65% for disease diagnosis, slightly surpassing existing black-box models. It uniquely generates high-quality, explainable reports as validated by human evaluations.

Conclusion: The framework represents a significant step towards trustworthy and clinically actionable AI systems in medical imaging, with ablation studies confirming the necessity of multi-modal fusion and CoT reasoning for robust and transparent medical AI.

Abstract: Chest X-ray imaging is crucial for diagnosing pulmonary and cardiac diseases, yet its interpretation demands extensive clinical experience and suffers from inter-observer variability. While deep learning models offer high diagnostic accuracy, their black-box nature hinders clinical adoption in high-stakes medical settings. To address this, we propose X-Ray-CoT (Chest X-Ray Chain-of-Thought), a novel framework leveraging Vision-Language Large Models (LVLMs) for intelligent chest X-ray diagnosis and interpretable report generation. X-Ray-CoT simulates human radiologists’ “chain-of-thought” by first extracting multi-modal features and visual concepts, then employing an LLM-based component with a structured Chain-of-Thought prompting strategy to reason and produce detailed natural language diagnostic reports. Evaluated on the CORDA dataset, X-Ray-CoT achieves competitive quantitative performance, with a Balanced Accuracy of 80.52% and F1 score of 78.65% for disease diagnosis, slightly surpassing existing black-box models. Crucially, it uniquely generates high-quality, explainable reports, as validated by preliminary human evaluations. Our ablation studies confirm the integral role of each proposed component, highlighting the necessity of multi-modal fusion and CoT reasoning for robust and transparent medical AI. This work represents a significant step towards trustworthy and clinically actionable AI systems in medical imaging.

Method: Maps text embeddings into continuous visual representation space, performs fusion within transformer intermediate layers using selective additive components in attention mechanisms.

Result: Improves reasoning-intensive tasks (MM-VET: +0.2%, VizWiz: +1.8%, ScienceQA: +0.2%, cognitive reasoning: +27.2%) but decreases perception tasks (celebrity recognition: -49.5%, OCR: -21.3%), with 45% computational reduction.

Conclusion: Alignment pre-training is not necessary for effective multimodal learning, particularly for complex reasoning, establishing a new paradigm that reduces computation and preserves modality-specific characteristics.

Abstract: Traditional multimodal learning approaches require expensive alignment pre-training to bridge vision and language modalities, typically projecting visual features into discrete text token spaces. We challenge both fundamental assumptions underlying this paradigm by proposing Inverse-LLaVA, a novel approach that eliminates alignment pre-training entirely while inverting the conventional mapping direction. Rather than projecting visual features to text space, our method maps text embeddings into continuous visual representation space and performs fusion within transformer intermediate layers. Through selective additive components in attention mechanisms, we enable dynamic integration of visual and textual representations without requiring massive image-text alignment datasets. Comprehensive experiments across nine multimodal benchmarks demonstrate nuanced performance trade-offs: Inverse-LLaVA achieves notable improvements on reasoning-intensive and cognitive tasks (MM-VET: +0.2%, VizWiz: +1.8%, ScienceQA: +0.2%, cognitive reasoning: +27.2%), while showing expected decreases in perception tasks requiring memorized visual-text associations (celebrity recognition: -49.5%, OCR: -21.3%). These results provide the first empirical evidence that alignment pre-training is not necessary for effective multimodal learning, particularly for complex reasoning tasks. Our work establishes the feasibility of a new paradigm that reduces computational requirements by 45%, challenges conventional wisdom about modality fusion, and opens new research directions for efficient multimodal architectures that preserve modality-specific characteristics. Our project website with code and additional resources is available at https://inverse-llava.github.io.

Method: Fine-tuned multiple VLMs on curated H-reflex EMG waveform images with clinical annotations, then aggregated outputs using consensus-based method and refined by specialized reasoning LLM with prompt engineering and LLM Agents.

Result: The hybrid system delivers highly accurate, consistent, and interpretable H-reflex assessments, significantly advancing automation and standardization of neuromuscular diagnostics.

Conclusion: First integration of fine-tuned VLM consortium with reasoning LLM for image-based H-reflex analysis, laying foundation for next-generation AI-assisted neuromuscular assessment platforms.

Abstract: Accurate assessment of neuromuscular reflexes, such as the H-reflex, plays a critical role in sports science, rehabilitation, and clinical neurology. Traditional analysis of H-reflex EMG waveforms is subject to variability and interpretation bias among clinicians and researchers, limiting reliability and standardization. To address these challenges, we propose a Fine-Tuned Vision-Language Model (VLM) Consortium and a reasoning Large-Language Model (LLM)-enabled Decision Support System for automated H-reflex waveform interpretation and diagnosis. Our approach leverages multiple VLMs, each fine-tuned on curated datasets of H-reflex EMG waveform images annotated with clinical observations, recovery timelines, and athlete metadata. These models are capable of extracting key electrophysiological features and predicting neuromuscular states, including fatigue, injury, and recovery, directly from EMG images and contextual metadata. Diagnostic outputs from the VLM consortium are aggregated using a consensus-based method and refined by a specialized reasoning LLM, which ensures robust, transparent, and explainable decision support for clinicians and sports scientists. The end-to-end platform orchestrates seamless communication between the VLM ensemble and the reasoning LLM, integrating prompt engineering strategies and automated reasoning workflows using LLM Agents. Experimental results demonstrate that this hybrid system delivers highly accurate, consistent, and interpretable H-reflex assessments, significantly advancing the automation and standardization of neuromuscular diagnostics. To our knowledge, this work represents the first integration of a fine-tuned VLM consortium with a reasoning LLM for image-based H-reflex analysis, laying the foundation for next-generation AI-assisted neuromuscular assessment and athlete monitoring platforms.

Method: Sequential and Parallel Hybrid CNN-Transformer models integrated with Convolutional Kolmogorov-Arnold Network (CKAN), using transfer learning and extensive data augmentation. CNNs extract local spatial features, Transformers model global dependencies, and CKAN facilitates nonlinear feature fusion.

Result: Achieved 92.81% accuracy and 92.47% F1-score on HAM10000, 97.83% accuracy and 97.83% F1-score on PAD-UFES, and 91.17% accuracy with 91.79% F1-score on BCN20000 dataset, demonstrating competitive performance across diverse datasets.

Conclusion: Hybrid CNN-Transformer architectures effectively capture both spatial and contextual features, with CKAN enhancing feature fusion through learnable activation functions, highlighting the significance of feature representation and model design for robust medical image classification.

Abstract: Skin cancer classification is a crucial task in medical image analysis, where precise differentiation between malignant and non-malignant lesions is essential for early diagnosis and treatment. In this study, we explore Sequential and Parallel Hybrid CNN-Transformer models with Convolutional Kolmogorov-Arnold Network (CKAN). Our approach integrates transfer learning and extensive data augmentation, where CNNs extract local spatial features, Transformers model global dependencies, and CKAN facilitates nonlinear feature fusion for improved representation learning. To assess generalization, we evaluate our models on multiple benchmark datasets (HAM10000,BCN20000 and PAD-UFES) under varying data distributions and class imbalances. Experimental results demonstrate that hybrid CNN-Transformer architectures effectively capture both spatial and contextual features, leading to improved classification performance. Additionally, the integration of CKAN enhances feature fusion through learnable activation functions, yielding more discriminative representations. Our proposed approach achieves competitive performance in skin cancer classification, demonstrating 92.81% accuracy and 92.47% F1-score on the HAM10000 dataset, 97.83% accuracy and 97.83% F1-score on the PAD-UFES dataset, and 91.17% accuracy with 91.79% F1- score on the BCN20000 dataset highlighting the effectiveness and generalizability of our model across diverse datasets. This study highlights the significance of feature representation and model design in advancing robust and accurate medical image classification.

Method: Developed RAIS-DR system incorporating ethical principles across AI lifecycle, using efficient convolutional models for preprocessing, quality assessment, and three specialized DR classification models.

Result: RAIS-DR outperformed FDA-approved EyeArt on 1,046 patient dataset with F1 scores increasing 5-12%, accuracy by 6-19%, specificity by 10-20%, and demonstrated equitable performance across demographic subgroups.

Conclusion: RAIS-DR represents a robust and ethically aligned solution for DR screening that can reduce healthcare disparities, with code and weights publicly available under RAIL license.

Abstract: Diabetic Retinopathy (DR) is a leading cause of vision loss in working-age individuals. Early detection of DR can reduce the risk of vision loss by up to 95%, but a shortage of retinologists and challenges in timely examination complicate detection. Artificial Intelligence (AI) models using retinal fundus photographs (RFPs) offer a promising solution. However, adoption in clinical settings is hindered by low-quality data and biases that may lead AI systems to learn unintended features. To address these challenges, we developed RAIS-DR, a Responsible AI System for DR screening that incorporates ethical principles across the AI lifecycle. RAIS-DR integrates efficient convolutional models for preprocessing, quality assessment, and three specialized DR classification models. We evaluated RAIS-DR against the FDA-approved EyeArt system on a local dataset of 1,046 patients, unseen by both systems. RAIS-DR demonstrated significant improvements, with F1 scores increasing by 5-12%, accuracy by 6-19%, and specificity by 10-20%. Additionally, fairness metrics such as Disparate Impact and Equal Opportunity Difference indicated equitable performance across demographic subgroups, underscoring RAIS-DR’s potential to reduce healthcare disparities. These results highlight RAIS-DR as a robust and ethically aligned solution for DR screening in clinical settings. The code, weights of RAIS-DR are available at https://gitlab.com/inteligencia-gubernamental-jalisco/jalisco-retinopathy with RAIL.

Method: Proposes a framework combining Neural Architecture Search (NAS) with LoRA to dynamically search for optimal LoRA rank configurations tailored to specific multimodal tasks.

Result: Extensive experiments using LLaMA-3.2-11B model show notable performance improvements while reducing fine-tuning costs across several datasets.

Conclusion: LangVision-LoRA-NAS provides an effective approach for optimizing VLMs through variable-rank adaptation, offering better performance-efficiency tradeoffs than fixed-rank LoRA implementations.

Abstract: Vision Language Models (VLMs) integrate visual and text modalities to enable multimodal understanding and generation. These models typically combine a Vision Transformer (ViT) as an image encoder and a Large Language Model (LLM) for text generation. LoRA (Low-Rank Adaptation) is an efficient fine-tuning method to adapt pre-trained models to new tasks by introducing low-rank updates to their weights. While LoRA has emerged as a powerful technique for fine-tuning large models by introducing low-rank updates, current implementations assume a fixed rank, potentially limiting flexibility and efficiency across diverse tasks. This paper introduces \textit{LangVision-LoRA-NAS}, a novel framework that integrates Neural Architecture Search (NAS) with LoRA to optimize VLMs for variable-rank adaptation. Our approach leverages NAS to dynamically search for the optimal LoRA rank configuration tailored to specific multimodal tasks, balancing performance and computational efficiency. Through extensive experiments using the LLaMA-3.2-11B model on several datasets, LangVision-LoRA-NAS demonstrates notable improvement in model performance while reducing fine-tuning costs. Our Base and searched fine-tuned models on LLaMA-3.2-11B-Vision-Instruct can be found \href{https://huggingface.co/collections/krishnateja95/llama-32-11b-vision-instruct-langvision-lora-nas-6786cac480357a6a6fcc59ee}{\textcolor{blue}{here}} and the code for LangVision-LoRA-NAS can be found \href{https://github.com/krishnateja95/LangVision-NAS}{\textcolor{blue}{here}}.

Method: Used Cross-View Transformers (CVT) to map camera images to three BEV channels (road, lane markings, planned trajectory) using a realistic urban driving simulator. Tested different camera layouts and loss formulations (focal vs L1 loss).

Result: A four-camera CVT trained with L1 loss delivered the most robust test performance when evaluated in a new, unseen town, demonstrating strong generalization capabilities.

Conclusion: Cross-View Transformers show promise for mapping camera inputs to reasonably accurate BEV maps, with L1 loss and four-camera configuration providing optimal performance for autonomous driving applications.

Abstract: Bird’s-Eye View (BEV) maps provide a structured, top-down abstraction that is crucial for autonomous-driving perception. In this work, we employ Cross-View Transformers (CVT) for learning to map camera images to three BEV’s channels - road, lane markings, and planned trajectory - using a realistic simulator for urban driving. Our study examines generalization to unseen towns, the effect of different camera layouts, and two loss formulations (focal and L1). Using training data from only a town, a four-camera CVT trained with the L1 loss delivers the most robust test performance, evaluated in a new town. Overall, our results underscore CVT’s promise for mapping camera inputs to reasonably accurate BEV maps.

Method: MuSACo uses co-training to select relevant source subjects for the target, generates pseudo-labels using the dominant modality for class-aware learning, employs class-agnostic loss for less confident target samples, and aligns source features while combining only confident target features across modalities.

Result: Experimental results on BioVid and StressID multimodal ER datasets show that MuSACo outperforms both UDA (blending) and state-of-the-art MSDA methods.

Conclusion: MuSACo effectively addresses limitations of existing MSDA approaches by leveraging multimodal information and preserving subject diversity, making it particularly suitable for affective computing applications in digital health where subject-level nuances are crucial.

Abstract: Personalized expression recognition (ER) involves adapting a machine learning model to subject-specific data for improved recognition of expressions with considerable interpersonal variability. Subject-specific ER can benefit significantly from multi-source domain adaptation (MSDA) methods, where each domain corresponds to a specific subject, to improve model accuracy and robustness. Despite promising results, state-of-the-art MSDA approaches often overlook multimodal information or blend sources into a single domain, limiting subject diversity and failing to explicitly capture unique subject-specific characteristics. To address these limitations, we introduce MuSACo, a multi-modal subject-specific selection and adaptation method for ER based on co-training. It leverages complementary information across multiple modalities and multiple source domains for subject-specific adaptation. This makes MuSACo particularly relevant for affective computing applications in digital health, such as patient-specific assessment for stress or pain, where subject-level nuances are crucial. MuSACo selects source subjects relevant to the target and generates pseudo-labels using the dominant modality for class-aware learning, in conjunction with a class-agnostic loss to learn from less confident target samples. Finally, source features from each modality are aligned, while only confident target features are combined. Our experimental results on challenging multimodal ER datasets: BioVid and StressID, show that MuSACo can outperform UDA (blending) and state-of-the-art MSDA methods.

Method: Proposes REVEAL framework that leverages large vision-language models for semantic alignment. Uses two approaches: (1) Holistic scene-level evaluation analyzing physics, semantics, perspective, and realism, and (2) Region-wise anomaly detection by splitting images into regions for individual analysis.

Result: Conducted experiments across multiple domains (Photoshop, DeepFake, AIGC editing) and compared against competitive baselines, analyzing the reasoning capabilities of vision-language models.

Conclusion: The prompt-driven visual reasoning approach using vision-language models shows promise for generalized image forgery detection with interpretable explanations, addressing domain generalization challenges in existing methods.

Abstract: The rapid advancement of generative models has intensified the challenge of detecting and interpreting visual forgeries, necessitating robust frameworks for image forgery detection while providing reasoning as well as localization. While existing works approach this problem using supervised training for specific manipulation or anomaly detection in the embedding space, generalization across domains remains a challenge. We frame this problem of forgery detection as a prompt-driven visual reasoning task, leveraging the semantic alignment capabilities of large vision-language models. We propose a framework, REVEAL (Reasoning and Evaluation of Visual Evidence through Aligned Language), that incorporates generalized guidelines. We propose two tangential approaches - (1) Holistic Scene-level Evaluation that relies on the physics, semantics, perspective, and realism of the image as a whole and (2) Region-wise anomaly detection that splits the image into multiple regions and analyzes each of them. We conduct experiments over datasets from different domains (Photoshop, DeepFake and AIGC editing). We compare the Vision Language Models against competitive baselines and analyze the reasoning provided by them.

Method: Built comprehensive RL-ready visual reasoning dataset from 46 sources across 8 domains, used influence function-based data selection and difficulty filtering, trained with multi-round RL and data curriculum.

Result: Achieves state-of-the-art performance across various visual reasoning benchmarks, outperforms similar-sized VLMs and proprietary models like GPT-4o and Gemini-1.5 Flash.

Conclusion: The comprehensive multi-domain dataset and iterative RL training with data curriculum enable superior visual reasoning capabilities that generalize across diverse domains.

Abstract: Despite their success, current training pipelines for reasoning VLMs focus on a limited range of tasks, such as mathematical and logical reasoning. As a result, these models face difficulties in generalizing their reasoning capabilities to a wide range of domains, primarily due to the scarcity of readily available and verifiable reward data beyond these narrowly defined areas. Moreover, integrating data from multiple domains is challenging, as the compatibility between domain-specific datasets remains uncertain. To address these limitations, we build a comprehensive RL-ready visual reasoning dataset from 46 data sources across 8 dimensions, covering a wide range of tasks such as infographic, mathematical, spatial, cross-image, graphic user interface, medical, common sense and general science. We propose an influence function based data selection and difficulty based filtering strategy to identify high-quality training samples from this dataset. Subsequently, we train the VLM, referred to as Vision-G1, using multi-round RL with a data curriculum to iteratively improve its visual reasoning capabilities. Our model achieves state-of-the-art performance across various visual reasoning benchmarks, outperforming similar-sized VLMs and even proprietary models like GPT-4o and Gemini-1.5 Flash. The model, code and dataset are publicly available at https://github.com/yuh-zha/Vision-G1.

Method: Proposes SFAC with feature distribution alignment loss for semantic correspondence and structure-preserving mechanism with perceptual constraints and frozen-updated pyramid to prevent distortions.

Result: Extensive experiments show effectiveness through both qualitative and quantitative metrics, demonstrating successful colorization of old photos without requiring large datasets.

Conclusion: SFAC provides an effective solution for old photo colorization by eliminating big data dependency and directly addressing domain gap issues through semantic correspondence and structure preservation.

Abstract: Deep learning techniques have made significant advancements in reference-based colorization by training on large-scale datasets. However, directly applying these methods to the task of colorizing old photos is challenging due to the lack of ground truth and the notorious domain gap between natural gray images and old photos. To address this issue, we propose a novel CNN-based algorithm called SFAC, i.e., Structure-preserving Feature Alignment Colorizer. SFAC is trained on only two images for old photo colorization, eliminating the reliance on big data and allowing direct processing of the old photo itself to overcome the domain gap problem. Our primary objective is to establish semantic correspondence between the two images, ensuring that semantically related objects have similar colors. We achieve this through a feature distribution alignment loss that remains robust to different metric choices. However, utilizing robust semantic correspondence to transfer color from the reference to the old photo can result in inevitable structure distortions. To mitigate this, we introduce a structure-preserving mechanism that incorporates a perceptual constraint at the feature level and a frozen-updated pyramid at the pixel level. Extensive experiments demonstrate the effectiveness of our method for old photo colorization, as confirmed by qualitative and quantitative metrics.

Method: Transformer-based Dense Spatio-Temporal Encoder (DSTE) with parallel streams for temporal dynamics and spatial structure, Multi-Grained Feature Decorrelation (MG-FD) to reduce redundancy across temporal/spatial/instance domains, and Multi-Perspective Consistency Training (MPCT) with multi-view and multi-modal self-supervised learning.

Result: Significantly outperforms state-of-the-art methods across 25 benchmarks covering 9 skeleton-based action understanding tasks, including coarse prediction, dense prediction, and transferred prediction.

Conclusion: USDRL serves as an effective foundational model that broadens research scope in skeleton-based action understanding and encourages more attention to dense prediction tasks, demonstrating strong generalization capabilities.

Abstract: Human action understanding serves as a foundational pillar in the field of intelligent motion perception. Skeletons serve as a modality- and device-agnostic representation for human modeling, and skeleton-based action understanding has potential applications in humanoid robot control and interaction. \RED{However, existing works often lack the scalability and generalization required to handle diverse action understanding tasks. There is no skeleton foundation model that can be adapted to a wide range of action understanding tasks}. This paper presents a Unified Skeleton-based Dense Representation Learning (USDRL) framework, which serves as a foundational model for skeleton-based human action understanding. USDRL consists of a Transformer-based Dense Spatio-Temporal Encoder (DSTE), Multi-Grained Feature Decorrelation (MG-FD), and Multi-Perspective Consistency Training (MPCT). The DSTE module adopts two parallel streams to learn temporal dynamic and spatial structure features. The MG-FD module collaboratively performs feature decorrelation across temporal, spatial, and instance domains to reduce dimensional redundancy and enhance information extraction. The MPCT module employs both multi-view and multi-modal self-supervised consistency training. The former enhances the learning of high-level semantics and mitigates the impact of low-level discrepancies, while the latter effectively facilitates the learning of informative multimodal features. We perform extensive experiments on 25 benchmarks across across 9 skeleton-based action understanding tasks, covering coarse prediction, dense prediction, and transferred prediction. Our approach significantly outperforms the current state-of-the-art methods. We hope that this work would broaden the scope of research in skeleton-based action understanding and encourage more attention to dense prediction tasks.

Method: Proposes Multimodal Chain of Continuous Thought (MCOUT) with reasoning represented as continuous hidden vectors iteratively refined and aligned with visual/textual embeddings. Two variants: MCOUT-Base (reuses language model’s last hidden state) and MCOUT-Multi (integrates multimodal latent attention).

Result: Experiments on MMMU, ScienceQA, and MMStar benchmarks show consistent improvements in multimodal reasoning with up to 8.23% accuracy gains over baselines and up to 8.27% BLEU score improvements across multiple-choice and open-ended tasks.

Conclusion: Latent continuous reasoning is a promising direction for advancing multimodal models beyond language-bound approaches, offering a scalable framework for human-like reflective multimodal inference.

Abstract: Many reasoning techniques for large multimodal models adapt language model approaches, such as Chain-of-Thought (CoT) prompting, which express reasoning as word sequences. While effective for text, these methods are suboptimal for multimodal contexts, struggling to align audio, visual, and textual information dynamically. To explore an alternative paradigm, we propose the Multimodal Chain of Continuous Thought (MCOUT), which enables reasoning directly in a joint latent space rather than in natural language. In MCOUT, the reasoning state is represented as a continuous hidden vector, iteratively refined and aligned with visual and textual embeddings, inspired by human reflective cognition. We develop two variants: MCOUT-Base, which reuses the language model`s last hidden state as the continuous thought for iterative reasoning, and MCOUT-Multi, which integrates multimodal latent attention to strengthen cross-modal alignment between visual and textual features. Experiments on benchmarks including MMMU, ScienceQA, and MMStar show that MCOUT consistently improves multimodal reasoning, yielding up to 8.23% accuracy gains over strong baselines and improving BLEU scores up to 8.27% across multiple-choice and open-ended tasks. These findings highlight latent continuous reasoning as a promising direction for advancing LMMs beyond language-bound CoT, offering a scalable framework for human-like reflective multimodal inference. Code is available at https://github.com/Hanhpt23/OmniMod.

Method: ViLaD uses a masked diffusion model that enables parallel generation of entire driving decision sequences, supports bidirectional reasoning by considering both past and future simultaneously, and employs progressive easy-first generation for iterative decision quality improvement.

Result: On nuScenes dataset, ViLaD outperforms state-of-the-art autoregressive VLM baselines in both planning accuracy and inference speed, achieving near-zero failure rate. Real-world deployment on autonomous vehicle for interactive parking task confirmed practical viability.

Conclusion: ViLaD represents a paradigm shift in autonomous driving systems by demonstrating that diffusion-based parallel generation can overcome limitations of autoregressive models, providing faster, more reliable decision-making suitable for real-world safety-critical applications.

Abstract: End-to-end autonomous driving systems built on Vision Language Models (VLMs) have shown significant promise, yet their reliance on autoregressive architectures introduces some limitations for real-world applications. The sequential, token-by-token generation process of these models results in high inference latency and cannot perform bidirectional reasoning, making them unsuitable for dynamic, safety-critical environments. To overcome these challenges, we introduce ViLaD, a novel Large Vision Language Diffusion (LVLD) framework for end-to-end autonomous driving that represents a paradigm shift. ViLaD leverages a masked diffusion model that enables parallel generation of entire driving decision sequences, significantly reducing computational latency. Moreover, its architecture supports bidirectional reasoning, allowing the model to consider both past and future simultaneously, and supports progressive easy-first generation to iteratively improve decision quality. We conduct comprehensive experiments on the nuScenes dataset, where ViLaD outperforms state-of-the-art autoregressive VLM baselines in both planning accuracy and inference speed, while achieving a near-zero failure rate. Furthermore, we demonstrate the framework’s practical viability through a real-world deployment on an autonomous vehicle for an interactive parking task, confirming its effectiveness and soundness for practical applications.

Method: Created ViDA-UGC dataset with 11K images using human annotation and CoT framework to guide GPT-4o in generating quality descriptions. Also developed ViDA-UGC-Bench benchmark with 476 images and 6,149 QA pairs professionally validated.

Result: The ViDA-UGC dataset and CoT framework consistently enhance various MLLMs’ image quality analysis abilities on both ViDA-UGC-Bench and Q-Bench benchmarks, even surpassing GPT-4o performance.

Conclusion: The proposed approach successfully addresses the limitations of current explainable IQA methods by providing specialized distortion assessment for UGC content and demonstrates superior performance across multiple evaluation benchmarks.

Abstract: Recent advances in Multimodal Large Language Models (MLLMs) have introduced a paradigm shift for Image Quality Assessment (IQA) from unexplainable image quality scoring to explainable IQA, demonstrating practical applications like quality control and optimization guidance. However, current explainable IQA methods not only inadequately use the same distortion criteria to evaluate both User-Generated Content (UGC) and AI-Generated Content (AIGC) images, but also lack detailed quality analysis for monitoring image quality and guiding image restoration. In this study, we establish the first large-scale Visual Distortion Assessment Instruction Tuning Dataset for UGC images, termed ViDA-UGC, which comprises 11K images with fine-grained quality grounding, detailed quality perception, and reasoning quality description data. This dataset is constructed through a distortion-oriented pipeline, which involves human subject annotation and a Chain-of-Thought (CoT) assessment framework. This framework guides GPT-4o to generate quality descriptions by identifying and analyzing UGC distortions, which helps capturing rich low-level visual features that inherently correlate with distortion patterns. Moreover, we carefully select 476 images with corresponding 6,149 question answer pairs from ViDA-UGC and invite a professional team to ensure the accuracy and quality of GPT-generated information. The selected and revised data further contribute to the first UGC distortion assessment benchmark, termed ViDA-UGC-Bench. Experimental results demonstrate the effectiveness of the ViDA-UGC and CoT framework for consistently enhancing various image quality analysis abilities across multiple base MLLMs on ViDA-UGC-Bench and Q-Bench, even surpassing GPT-4o.

Method: Introduces CMU-Occlu dataset using ray tracing to simulate realistic occlusion patterns, and proposes OpenMoCap model with marker-joint chain inference mechanism for simultaneous optimization and deep constraint construction between markers and joints.

Result: Extensive experiments show OpenMoCap consistently outperforms competing methods across diverse scenarios, and the CMU-Occlu dataset enables future robust motion solving research.

Conclusion: OpenMoCap provides robust motion capture under significant occlusions and has been integrated into practical MoSen MoCap system, with code publicly released for community use.

Abstract: Optical motion capture is a foundational technology driving advancements in cutting-edge fields such as virtual reality and film production. However, system performance suffers severely under large-scale marker occlusions common in real-world applications. An in-depth analysis identifies two primary limitations of current models: (i) the lack of training datasets accurately reflecting realistic marker occlusion patterns, and (ii) the absence of training strategies designed to capture long-range dependencies among markers. To tackle these challenges, we introduce the CMU-Occlu dataset, which incorporates ray tracing techniques to realistically simulate practical marker occlusion patterns. Furthermore, we propose OpenMoCap, a novel motion-solving model designed specifically for robust motion capture in environments with significant occlusions. Leveraging a marker-joint chain inference mechanism, OpenMoCap enables simultaneous optimization and construction of deep constraints between markers and joints. Extensive comparative experiments demonstrate that OpenMoCap consistently outperforms competing methods across diverse scenarios, while the CMU-Occlu dataset opens the door for future studies in robust motion solving. The proposed OpenMoCap is integrated into the MoSen MoCap system for practical deployment. The code is released at: https://github.com/qianchen214/OpenMoCap.

Method: Proposes WIPES, a universal wavelet-based visual primitive that leverages spatial-frequency localization advantages of wavelets to capture both low and high frequency details. Also develops a wavelet-based differentiable rasterizer for fast visual rendering.

Result: Experimental results on 2D image representation, 5D static and 6D dynamic novel view synthesis show WIPES offers higher rendering quality and faster inference than INR-based methods, and better rendering quality than Gaussian-based representations.

Conclusion: WIPES serves as an effective visual primitive that addresses spectrum loss and slow rendering issues, providing superior performance across multiple visual tasks through its wavelet-based approach.

Abstract: Pursuing a continuous visual representation that offers flexible frequency modulation and fast rendering speed has recently garnered increasing attention in the fields of 3D vision and graphics. However, existing representations often rely on frequency guidance or complex neural network decoding, leading to spectrum loss or slow rendering. To address these limitations, we propose WIPES, a universal Wavelet-based vIsual PrimitivES for representing multi-dimensional visual signals. Building on the spatial-frequency localization advantages of wavelets, WIPES effectively captures both the low-frequency “forest” and the high-frequency “trees.” Additionally, we develop a wavelet-based differentiable rasterizer to achieve fast visual rendering. Experimental results on various visual tasks, including 2D image representation, 5D static and 6D dynamic novel view synthesis, demonstrate that WIPES, as a visual primitive, offers higher rendering quality and faster inference than INR-based methods, and outperforms Gaussian-based representations in rendering quality.

Method: Uses multimodal LLMs to transform creative assessment into natural language generation. Builds CreativePair dataset with 8k annotated image pairs and develops Creative4U system with Chain-of-Thought supervised fine-tuning and Group Relative Policy Optimization reinforcement learning.

Result: Offline and online experiments demonstrate effective creative image evaluation and selection. The approach successfully addresses explainable creative assessment needs.

Conclusion: The proposed paradigm provides the first explainable creative assessment solution, advancing both research and industrial applications in advertising creative selection.

Abstract: Creative image in advertising is the heart and soul of e-commerce platform. An eye-catching creative image can enhance the shopping experience for users, boosting income for advertisers and advertising revenue for platforms. With the advent of AIGC technology, advertisers can produce large quantities of creative images at minimal cost. However, they struggle to assess the creative quality to select. Existing methods primarily focus on creative ranking, which fails to address the need for explainable creative selection. In this work, we propose the first paradigm for explainable creative assessment and selection. Powered by multimodal large language models (MLLMs), our approach integrates the assessment and selection of creative images into a natural language generation task. To facilitate this research, we construct CreativePair, the first comparative reasoning-induced creative dataset featuring 8k annotated image pairs, with each sample including a label indicating which image is superior. Additionally, we introduce Creative4U (pronounced Creative for You), a MLLMs-based creative selector that takes into account users’ interests. Through Reason-to-Select RFT, which includes supervised fine-tuning with Chain-of-Thought (CoT-SFT) and Group Relative Policy Optimization (GRPO) based reinforcement learning, Creative4U is able to evaluate and select creative images accurately. Both offline and online experiments demonstrate the effectiveness of our approach. Our code and dataset will be made public to advance research and industrial applications.

Method: Proposed Context Transfer paradigm that treats delayed LVLM outputs as historical context for SVLM guidance. Designed SpotVLM with context replacement and visual focus modules to refine textual input and enhance visual grounding consistency.

Result: Extensive experiments on three real-time vision tasks across four datasets demonstrate the effectiveness of the proposed framework.

Conclusion: The Context Transfer paradigm lays groundwork for more effective and latency-aware collaboration strategies in future VLM systems, enabling better handling of cloud delays while maintaining real-time performance.

Abstract: Vision-Language Models (VLMs) are increasingly deployed in real-time applications such as autonomous driving and human-computer interaction, which demand fast and reliable responses based on accurate perception. To meet these requirements, existing systems commonly employ cloud-edge collaborative architectures, such as partitioned Large Vision-Language Models (LVLMs) or task offloading strategies between Large and Small Vision-Language Models (SVLMs). However, these methods fail to accommodate cloud latency fluctuations and overlook the full potential of delayed but accurate LVLM responses. In this work, we propose a novel cloud-edge collaborative paradigm for VLMs, termed Context Transfer, which treats the delayed outputs of LVLMs as historical context to provide real-time guidance for SVLMs inference. Based on this paradigm, we design SpotVLM, which incorporates both context replacement and visual focus modules to refine historical textual input and enhance visual grounding consistency. Extensive experiments on three real-time vision tasks across four datasets demonstrate the effectiveness of the proposed framework. The new paradigm lays the groundwork for more effective and latency-aware collaboration strategies in future VLM systems.

Method: Two-stage approach: 1) Patch-based 3D U-Net predicts voxel-wise posterior mean (minimizing MSE), 2) Time-conditioned 3D rectified flow refines the initial estimate to incorporate realistic textures while preserving structural fidelity.

Result: Achieved axial FID of 12.46 and KID of 0.007 (68.7% lower FID than posterior mean) with low volumetric MSE of 0.057 (27% higher than posterior mean). Successfully restored lesion margins and vascular details realistically.

Conclusion: The PMRF pipeline effectively navigates the perception-distortion trade-off, producing clinically viable synthetic contrast-enhanced MRI without gadolinium, showing promise for clinical deployment.

Abstract: Contrast-enhanced (CE) T1-weighted MRI is central to neuro-oncologic diagnosis but requires gadolinium-based agents, which add cost and scan time, raise environmental concerns, and may pose risks to patients. In this work, we propose a two-stage Posterior-Mean Rectified Flow (PMRF) pipeline for synthesizing volumetric CE brain MRI from non-contrast inputs. First, a patch-based 3D U-Net predicts the voxel-wise posterior mean (minimizing MSE). Then, this initial estimate is refined by a time-conditioned 3D rectified flow to incorporate realistic textures without compromising structural fidelity. We train this model on a multi-institutional collection of paired pre- and post-contrast T1w volumes (BraTS 2023-2025). On a held-out test set of 360 diverse volumes, our best refined outputs achieve an axial FID of $12.46$ and KID of $0.007$ ($\sim 68.7%$ lower FID than the posterior mean) while maintaining low volumetric MSE of $0.057$ ($\sim 27%$ higher than the posterior mean). Qualitative comparisons confirm that our method restores lesion margins and vascular details realistically, effectively navigating the perception-distortion trade-off for clinical deployment.

Method: Proposes a mean teacher framework with: 1) Multi-level Consistency Regularization (MCR) loss to align intermediate features between student and teacher models for faster adaptation, and 2) Complementary Anchor Replay (CAR) mechanism to reuse historical checkpoints to recover knowledge from diverse previous domains.

Result: Significantly outperforms state-of-the-art methods on several benchmarks, demonstrating effective CTTA performance.

Conclusion: The proposed BEE framework successfully addresses the exploration-exploitation trade-off in CTTA by combining feature-level alignment through MCR and knowledge preservation through CAR, achieving superior adaptation performance.

Abstract: Continual Test-Time Adaptation (CTTA) aims to adapt a source pre-trained model to continually changing target domains during inference. As a fundamental principle, an ideal CTTA method should rapidly adapt to new domains (exploration) while retaining and exploiting knowledge from previously encountered domains to handle similar domains in the future. Despite significant advances, balancing exploration and exploitation in CTTA is still challenging: 1) Existing methods focus on adjusting predictions based on deep-layer outputs of neural networks. However, domain shifts typically affect shallow features, which are inefficient to be adjusted from deep predictions, leading to dilatory exploration; 2) A single model inevitably forgets knowledge of previous domains during the exploration, making it incapable of exploiting historical knowledge to handle similar future domains. To address these challenges, this paper proposes a mean teacher framework that strikes an appropriate Balance between Exploration and Exploitation (BEE) during the CTTA process. For the former challenge, we introduce a Multi-level Consistency Regularization (MCR) loss that aligns the intermediate features of the student and teacher models, accelerating adaptation to the current domain. For the latter challenge, we employ a Complementary Anchor Replay (CAR) mechanism to reuse historical checkpoints (anchors), recovering complementary knowledge for diverse domains. Experiments show that our method significantly outperforms state-of-the-art methods on several benchmarks, demonstrating its effectiveness for CTTA tasks.

Method: Coarse-to-fine group-guided motion optimization strategy with VAE-based human motion prior and segment-level optimization. Uses collective crowd behavior, Asynchronous Motion Consistency loss, and joint optimization of similar motion segments to handle occlusions.

Result: Achieves state-of-the-art performance in large-scene dynamic crowd reconstruction, with robust motion recovery even with temporal desynchronization and severe occlusions.

Conclusion: The proposed framework successfully addresses temporal consistency and occlusion challenges in 3D crowd reconstruction from videos, and the contributed VirtualCrowd dataset fills a gap in evaluation resources for this task.

Abstract: 3D reconstruction of dynamic crowds in large scenes has become increasingly important for applications such as city surveillance and crowd analysis. However, current works attempt to reconstruct 3D crowds from a static image, causing a lack of temporal consistency and inability to alleviate the typical impact caused by occlusions. In this paper, we propose DyCrowd, the first framework for spatio-temporally consistent 3D reconstruction of hundreds of individuals’ poses, positions and shapes from a large-scene video. We design a coarse-to-fine group-guided motion optimization strategy for occlusion-robust crowd reconstruction in large scenes. To address temporal instability and severe occlusions, we further incorporate a VAE (Variational Autoencoder)-based human motion prior along with a segment-level group-guided optimization. The core of our strategy leverages collective crowd behavior to address long-term dynamic occlusions. By jointly optimizing the motion sequences of individuals with similar motion segments and combining this with the proposed Asynchronous Motion Consistency (AMC) loss, we enable high-quality unoccluded motion segments to guide the motion recovery of occluded ones, ensuring robust and plausible motion recovery even in the presence of temporal desynchronization and rhythmic inconsistencies. Additionally, in order to fill the gap of no existing well-annotated large-scene video dataset, we contribute a virtual benchmark dataset, VirtualCrowd, for evaluating dynamic crowd reconstruction from large-scene videos. Experimental results demonstrate that the proposed method achieves state-of-the-art performance in the large-scene dynamic crowd reconstruction task. The code and dataset will be available for research purposes.

Method: Two-stage approach: 1) Mask completion using diffusion-based human body prior and occluded joint heatmaps, 2) RGB completion using Stable Diffusion with decoder fine-tuning, enhanced by human-specific textual features from VQA and CLIP encoder.

Result: Effectively reconstructs human appearances under severe occlusions, outperforms existing methods in both mask and RGB completion, and improves downstream tasks like 2D pose estimation and 3D human reconstruction.

Conclusion: The proposed method successfully addresses human de-occlusion by leveraging body structure priors and human-specific features, demonstrating superior performance and practical utility for human-centric computer vision tasks.

Abstract: Humans can infer the missing parts of an occluded object by leveraging prior knowledge and visible cues. However, enabling deep learning models to accurately predict such occluded regions remains a challenging task. De-occlusion addresses this problem by reconstructing both the mask and RGB appearance. In this work, we focus on human de-occlusion, specifically targeting the recovery of occluded body structures and appearances. Our approach decomposes the task into two stages: mask completion and RGB completion. The first stage leverages a diffusion-based human body prior to provide a comprehensive representation of body structure, combined with occluded joint heatmaps that offer explicit spatial cues about missing regions. The reconstructed amodal mask then serves as a conditioning input for the second stage, guiding the model on which areas require RGB reconstruction. To further enhance RGB generation, we incorporate human-specific textual features derived using a visual question answering (VQA) model and encoded via a CLIP encoder. RGB completion is performed using Stable Diffusion, with decoder fine-tuning applied to mitigate pixel-level degradation in visible regions – a known limitation of prior diffusion-based de-occlusion methods caused by latent space transformations. Our method effectively reconstructs human appearances even under severe occlusions and consistently outperforms existing methods in both mask and RGB completion. Moreover, the de-occluded images generated by our approach can improve the performance of downstream human-centric tasks, such as 2D pose estimation and 3D human reconstruction. The code will be made publicly available.

Method: Fine-tuned CLIP on annotated datasets of real art images to predict scores for each Wölfflin principle, creating WP-CLIP model.

Result: WP-CLIP successfully generalized across diverse artistic styles when evaluated on GAN-generated paintings and Pandora-18K art dataset, effectively predicting Wölfflin’s principles.

Conclusion: Vision-language models show significant potential for automated art analysis when properly fine-tuned, enabling computational evaluation of nuanced stylistic elements in visual art.

Abstract: W"olfflin’s five principles offer a structured approach to analyzing stylistic variations for formal analysis. However, no existing metric effectively predicts all five principles in visual art. Computationally evaluating the visual aspects of a painting requires a metric that can interpret key elements such as color, composition, and thematic choices. Recent advancements in vision-language models (VLMs) have demonstrated their ability to evaluate abstract image attributes, making them promising candidates for this task. In this work, we investigate whether CLIP, pre-trained on large-scale data, can understand and predict W"olfflin’s principles. Our findings indicate that it does not inherently capture such nuanced stylistic elements. To address this, we fine-tune CLIP on annotated datasets of real art images to predict a score for each principle. We evaluate our model, WP-CLIP, on GAN-generated paintings and the Pandora-18K art dataset, demonstrating its ability to generalize across diverse artistic styles. Our results highlight the potential of VLMs for automated art analysis.

Method: Introduces Box-Guided Refinement Module (BG-RM) that refines only foreground-associated BEV grids using 2D supervision and spatial subdivision, and Instance-Background Contrastive Learning (IBCL) that promotes separation between foreground and background features via contrastive learning.

Result: Extensive experiments on Air-Co-Pred dataset show AdaBEV achieves superior accuracy-computation trade-offs across model scales, outperforms state-of-the-art methods at low resolutions, and approaches upper bound performance with negligible overhead.

Conclusion: AdaBEV effectively addresses computational constraints in multi-UAV 3D detection by focusing refinement on foreground instances and using contrastive learning, enabling efficient high-performance perception on resource-constrained platforms.

Abstract: Multi-UAV collaborative 3D detection enables accurate and robust perception by fusing multi-view observations from aerial platforms, offering significant advantages in coverage and occlusion handling, while posing new challenges for computation on resource-constrained UAV platforms. In this paper, we present AdaBEV, a novel framework that learns adaptive instance-aware BEV representations through a refine-and-contrast paradigm. Unlike existing methods that treat all BEV grids equally, AdaBEV introduces a Box-Guided Refinement Module (BG-RM) and an Instance-Background Contrastive Learning (IBCL) to enhance semantic awareness and feature discriminability. BG-RM refines only BEV grids associated with foreground instances using 2D supervision and spatial subdivision, while IBCL promotes stronger separation between foreground and background features via contrastive learning in BEV space. Extensive experiments on the Air-Co-Pred dataset demonstrate that AdaBEV achieves superior accuracy-computation trade-offs across model scales, outperforming other state-of-the-art methods at low resolutions and approaching upper bound performance while maintaining low-resolution BEV inputs and negligible overhead.

Method: Leverages source domain data augmentation into target domains, introduces domain discriminator and specialized domain detector to handle drastic shifts, trains multiple detectors and consolidates predictions through Non-Maximum Suppression (NMS).

Result: Empirical validation demonstrates significant performance enhancements on the SHIFT Benchmark.

Conclusion: The proposed TTA-DAME method effectively addresses dynamic domain shifts in test-time adaptation scenarios, particularly for real-world driving applications with frequent weather and lighting changes.

Abstract: Test-time Adaptation (TTA) poses a challenge, requiring models to dynamically adapt and perform optimally on shifting target domains. This task is particularly emphasized in real-world driving scenes, where weather domain shifts occur frequently. To address such dynamic changes, our proposed method, TTA-DAME, leverages source domain data augmentation into target domains. Additionally, we introduce a domain discriminator and a specialized domain detector to mitigate drastic domain shifts, especially from daytime to nighttime conditions. To further improve adaptability, we train multiple detectors and consolidate their predictions through Non-Maximum Suppression (NMS). Our empirical validation demonstrates the effectiveness of our method, showing significant performance enhancements on the SHIFT Benchmark.

Method: Two components: 1) Multi-level knowledge distillation (MLKD) that distills knowledge from multiple previous models across features and logits perspectives, 2) Dynamic self-supervised loss (SSL) that utilizes unlabeled data to accelerate new class learning while maintaining focus on primary tasks.

Result: Significantly improved performance in CIR setup, achieving 2nd place in the CVPR 5th CLVISION Challenge.

Conclusion: The proposed MLKD and dynamic SSL components effectively utilize unlabeled data to ensure high stability and plasticity in class-incremental learning with repetition scenarios.

Abstract: Class-incremental with repetition (CIR), where previously trained classes repeatedly introduced in future tasks, is a more realistic scenario than the traditional class incremental setup, which assumes that each task contains unseen classes. CIR assumes that we can easily access abundant unlabeled data from external sources, such as the Internet. Therefore, we propose two components that efficiently use the unlabeled data to ensure the high stability and the plasticity of models trained in CIR setup. First, we introduce multi-level knowledge distillation (MLKD) that distills knowledge from multiple previous models across multiple perspectives, including features and logits, so the model can maintain much various previous knowledge. Moreover, we implement dynamic self-supervised loss (SSL) to utilize the unlabeled data that accelerates the learning of new classes, while dynamic weighting of SSL keeps the focus of training to the primary task. Both of our proposed components significantly improve the performance in CIR setup, achieving 2nd place in the CVPR 5th CLVISION Challenge.

Method: Created CamShift dataset in CARLA inspired by nuScenes to simulate sensor gap between subcompact vehicles and SUVs. Evaluated state-of-the-art 3D object detectors, identified BEV-based architectures as most robust, and developed a neural rendering-based sensor adaptation pipeline to transform datasets between different camera setups.

Result: Significant cross-sensor performance degradation was demonstrated. BEVFormer with dense Bird’s Eye View representation and backward projection showed highest robustness. The proposed sensor adaptation pipeline improved performance across all detectors, substantially mitigating the domain gap and enabling efficient data reusability.

Conclusion: Cross-sensor domain gap is a critical issue in autonomous vehicle perception. BEV-based architectures provide inherent robustness, and the proposed neural rendering adaptation pipeline effectively bridges sensor differences, reducing the need for costly new data collection for different vehicle sensor setups.

Abstract: Autonomous vehicles often have varying camera sensor setups, which is inevitable due to restricted placement options for different vehicle types. Training a perception model on one particular setup and evaluating it on a new, different sensor setup reveals the so-called cross-sensor domain gap, typically leading to a degradation in accuracy. In this paper, we investigate the impact of the cross-sensor domain gap on state-of-the-art 3D object detectors. To this end, we introduce CamShift, a dataset inspired by nuScenes and created in CARLA to specifically simulate the domain gap between subcompact vehicles and sport utility vehicles (SUVs). Using CamShift, we demonstrate significant cross-sensor performance degradation, identify robustness dependencies on model architecture, and propose a data-driven solution to mitigate the effect. On the one hand, we show that model architectures based on a dense Bird’s Eye View (BEV) representation with backward projection, such as BEVFormer, are the most robust against varying sensor configurations. On the other hand, we propose a novel data-driven sensor adaptation pipeline based on neural rendering, which can transform entire datasets to match different camera sensor setups. Applying this approach improves performance across all investigated 3D object detectors, mitigating the cross-sensor domain gap by a large margin and reducing the need for new data collection by enabling efficient data reusability across vehicles with different sensor setups. The CamShift dataset and the sensor adaptation benchmark are available at https://dmholtz.github.io/camshift/.

Method: Introduce DriftBench - a large-scale benchmark with 16,000 news instances across six diversification categories, and design three evaluation tasks to test robustness, adversarial susceptibility, and reasoning consistency.

Result: Experiments with six state-of-the-art LVLM detectors show substantial performance drops (average F1 -14.8%), increasingly unstable reasoning traces, and severe failures under adversarial evidence injection.

Conclusion: Findings reveal fundamental vulnerabilities in existing MMD systems, indicating an urgent need for more resilient approaches in the GenAI era.

Abstract: The proliferation of multimodal misinformation poses growing threats to public discourse and societal trust. While Large Vision-Language Models (LVLMs) have enabled recent progress in multimodal misinformation detection (MMD), the rise of generative AI (GenAI) tools introduces a new challenge: GenAI-driven news diversity, characterized by highly varied and complex content. We show that this diversity induces multi-level drift, comprising (1) model-level misperception drift, where stylistic variations disrupt a model’s internal reasoning, and (2) evidence-level drift, where expression diversity degrades the quality or relevance of retrieved external evidence. These drifts significantly degrade the robustness of current LVLM-based MMD systems. To systematically study this problem, we introduce DriftBench, a large-scale benchmark comprising 16,000 news instances across six categories of diversification. We design three evaluation tasks: (1) robustness of truth verification under multi-level drift; (2) susceptibility to adversarial evidence contamination generated by GenAI; and (3) analysis of reasoning consistency across diverse inputs. Experiments with six state-of-the-art LVLM-based detectors show substantial performance drops (average F1 -14.8%) and increasingly unstable reasoning traces, with even more severe failures under adversarial evidence injection. Our findings uncover fundamental vulnerabilities in existing MMD systems and suggest an urgent need for more resilient approaches in the GenAI era.

Method: Employed convolution neural networks (CNN) trained on Sign Language MNIST dataset to classify hand gestures captured live via webcam, with text-to-speech synthesis for audible output.

Result: High model accuracy and robust real-time performance demonstrated, though with some latency. System proved practical, accessible, reliable, and user-friendly.

Conclusion: The system effectively enhances autonomy and integration of sign language users in diverse social settings through seamless gesture-to-speech translation.

Abstract: Communication barriers pose significant challenges for individuals with hearing and speech impairments, often limiting their ability to effectively interact in everyday environments. This project introduces a real-time assistive technology solution that leverages advanced deep learning techniques to translate sign language gestures into textual and audible speech. By employing convolution neural networks (CNN) trained on the Sign Language MNIST dataset, the system accurately classifies hand gestures captured live via webcam. Detected gestures are instantaneously translated into their corresponding meanings and transcribed into spoken language using text-to-speech synthesis, thus facilitating seamless communication. Comprehensive experiments demonstrate high model accuracy and robust real-time performance with some latency, highlighting the system’s practical applicability as an accessible, reliable, and user-friendly tool for enhancing the autonomy and integration of sign language users in diverse social settings.

Method: Uses dual contrastive loss inspired by unpaired image translation, leveraging spatial information from self-attention layers in latent diffusion models without auxiliary networks.

Result: Outperforms existing methods in real image editing while maintaining zero-shot translation capabilities and preserving input-output structure.

Conclusion: The framework successfully addresses key challenges in real image editing by combining generative priors with contrastive learning for precise control over modifications.

Abstract: Large-scale text-to-image generative models have shown remarkable ability to synthesize diverse and high-quality images. However, it is still challenging to directly apply these models for editing real images for two reasons. First, it is difficult for users to come up with a perfect text prompt that accurately describes every visual detail in the input image. Second, while existing models can introduce desirable changes in certain regions, they often dramatically alter the input content and introduce unexpected changes in unwanted regions. To address these challenges, we present Dual Contrastive Denoising Score, a simple yet powerful framework that leverages the rich generative prior of text-to-image diffusion models. Inspired by contrastive learning approaches for unpaired image-to-image translation, we introduce a straightforward dual contrastive loss within the proposed framework. Our approach utilizes the extensive spatial information from the intermediate representations of the self-attention layers in latent diffusion models without depending on auxiliary networks. Our method achieves both flexible content modification and structure preservation between input and output images, as well as zero-shot image-to-image translation. Through extensive experiments, we show that our approach outperforms existing methods in real image editing while maintaining the capability to directly utilize pretrained text-to-image diffusion models without further training.

Method: Proposed Co-Adaptation Score (CA) metric to quantify Gaussian entanglement, then introduced two strategies: random Gaussian dropout and multiplicative noise injection to opacity to explicitly reduce co-adaptation.

Result: Analysis shows co-adaptation decreases with more training views. Both proposed strategies effectively mitigate appearance artifacts across various methods and benchmarks without significant computational overhead.

Conclusion: The co-adaptation effect is a core limitation in sparse-view 3DGS. The proposed lightweight strategies provide plug-and-play solutions, and understanding this phenomenon can lead to better sparse-view 3DGS performance.

Abstract: 3D Gaussian Splatting (3DGS) has demonstrated impressive performance in novel view synthesis under dense-view settings. However, in sparse-view scenarios, despite the realistic renderings in training views, 3DGS occasionally manifests appearance artifacts in novel views. This paper investigates the appearance artifacts in sparse-view 3DGS and uncovers a core limitation of current approaches: the optimized Gaussians are overly-entangled with one another to aggressively fit the training views, which leads to a neglect of the real appearance distribution of the underlying scene and results in appearance artifacts in novel views. The analysis is based on a proposed metric, termed Co-Adaptation Score (CA), which quantifies the entanglement among Gaussians, i.e., co-adaptation, by computing the pixel-wise variance across multiple renderings of the same viewpoint, with different random subsets of Gaussians. The analysis reveals that the degree of co-adaptation is naturally alleviated as the number of training views increases. Based on the analysis, we propose two lightweight strategies to explicitly mitigate the co-adaptation in sparse-view 3DGS: (1) random gaussian dropout; (2) multiplicative noise injection to the opacity. Both strategies are designed to be plug-and-play, and their effectiveness is validated across various methods and benchmarks. We hope that our insights into the co-adaptation effect will inspire the community to achieve a more comprehensive understanding of sparse-view 3DGS.

Method: Proposes Frequency-Driven Inverse Kernel Prediction network with Dual-Branch Inverse Kernel Prediction strategy, Position Adaptive Convolution for deconvolution adaptability, and Dual-Domain Scale Recurrent Module for progressive refinement.

Result: Extensive experiments demonstrate that the method outperforms existing approaches in single image defocus deblurring.

Conclusion: Incorporating frequency-domain representations and the proposed architectural components significantly improves defocus deblurring performance, particularly in challenging blurry regions.

Abstract: Single image defocus deblurring aims to recover an all-in-focus image from a defocus counterpart, where accurately modeling spatially varying blur kernels remains a key challenge. Most existing methods rely on spatial features for kernel estimation, but their performance degrades in severely blurry regions where local high-frequency details are missing. To address this, we propose a Frequency-Driven Inverse Kernel Prediction network (FDIKP) that incorporates frequency-domain representations to enhance structural identifiability in kernel modeling. Given the superior discriminative capability of the frequency domain for blur modeling, we design a Dual-Branch Inverse Kernel Prediction (DIKP) strategy that improves the accuracy of kernel estimation while maintaining stability. Moreover, considering the limited number of predicted inverse kernels, we introduce a Position Adaptive Convolution (PAC) to enhance the adaptability of the deconvolution process. Finally, we propose a Dual-Domain Scale Recurrent Module (DSRM) to fuse deconvolution results and progressively improve deblurring quality from coarse to fine. Extensive experiments demonstrate that our method outperforms existing approaches. Code will be made publicly available.

Method: DCSCR network with three modules: deep feature extractor, global feature learning, and class-specific collaborative representation-based metric learning with contrastive loss.

Result: Extensive experiments on well-known few-shot ISC datasets demonstrate effectiveness compared to state-of-the-art algorithms.

Conclusion: The proposed approach successfully addresses limitations of existing methods by simultaneously learning feature representations and adaptive distance similarities for improved few-shot image set classification.

Abstract: Image set classification (ISC), which can be viewed as a task of comparing similarities between sets consisting of unordered heterogeneous images with variable quantities and qualities, has attracted growing research attention in recent years. How to learn effective feature representations and how to explore the similarities between different image sets are two key yet challenging issues in this field. However, existing traditional ISC methods classify image sets based on raw pixel features, ignoring the importance of feature learning. Existing deep ISC methods can learn deep features, but they fail to adaptively adjust the features when measuring set distances, resulting in limited performance in few-shot ISC. To address the above issues, this paper combines traditional ISC methods with deep models and proposes a novel few-shot ISC approach called Deep Class-specific Collaborative Representation (DCSCR) network to simultaneously learn the frame- and concept-level feature representations of each image set and the distance similarities between different sets. Specifically, DCSCR consists of a fully convolutional deep feature extractor module, a global feature learning module, and a class-specific collaborative representation-based metric learning module. The deep feature extractor and global feature learning modules are used to learn (local and global) frame-level feature representations, while the class-specific collaborative representation-based metric learning module is exploit to adaptively learn the concept-level feature representation of each image set and thus obtain the distance similarities between different sets by developing a new CSCR-based contrastive loss function. Extensive experiments on several well-known few-shot ISC datasets demonstrate the effectiveness of the proposed method compared with some state-of-the-art image set classification algorithms.

Method: Proposes a Mamba-based network with Dual-Scale Fusion Mamba Block (DFMB) for multi-scale feature representation and boundary artifact reduction, and Dual-Path Mamba Group (DPMG) with horizontal scanning and mask-aware adaptive scanning for global feature capture and fine-grained region modeling.

Result: Experimental results demonstrate that the method significantly outperforms existing state-of-the-art approaches on shadow removal benchmarks.

Conclusion: The proposed Mamba-based network with dual-scale fusion and dual-path scanning effectively addresses the challenges of shadow removal by selectively propagating contextual information based on transformation similarity across regions, achieving superior performance.

Abstract: Shadow removal aims to restore images that are partially degraded by shadows, where the degradation is spatially localized and non-uniform. Unlike general restoration tasks that assume global degradation, shadow removal can leverage abundant information from non-shadow regions for guidance. However, the transformation required to correct shadowed areas often differs significantly from that of well-lit regions, making it challenging to apply uniform correction strategies. This necessitates the effective integration of non-local contextual cues and adaptive modeling of region-specific transformations. To this end, we propose a novel Mamba-based network featuring dual-scale fusion and dual-path scanning to selectively propagate contextual information based on transformation similarity across regions. Specifically, the proposed Dual-Scale Fusion Mamba Block (DFMB) enhances multi-scale feature representation by fusing original features with low-resolution features, effectively reducing boundary artifacts. The Dual-Path Mamba Group (DPMG) captures global features via horizontal scanning and incorporates a mask-aware adaptive scanning strategy, which improves structural continuity and fine-grained region modeling. Experimental results demonstrate that our method significantly outperforms existing state-of-the-art approaches on shadow removal benchmarks.

Method: Uses pre-trained ResNet backbone models fine-tuned to predict nine image properties (contrast presence, projection angle, motion artifacts, etc.) on 1,758 annotated fluoroscopic MinIPs with separate classifiers.

Result: Achieved excellent performance with ROC-AUC 0.91-0.98 and precision 0.70-1.00. Filtering poor quality images increased segmentation success rate from 42% to 69% (p < 0.001).

Conclusion: CLAIRE-DSA shows strong potential as an automated tool for image quality classification in DSA series, supporting clinical and research applications for stroke treatment.

Abstract: Computer vision models can be used to assist during mechanical thrombectomy (MT) for acute ischemic stroke (AIS), but poor image quality often degrades performance. This work presents CLAIRE-DSA, a deep learning–based framework designed to categorize key image properties in minimum intensity projections (MinIPs) acquired during MT for AIS, supporting downstream quality control and workflow optimization. CLAIRE-DSA uses pre-trained ResNet backbone models, fine-tuned to predict nine image properties (e.g., presence of contrast, projection angle, motion artefact severity). Separate classifiers were trained on an annotated dataset containing $1,758$ fluoroscopic MinIPs. The model achieved excellent performance on all labels, with ROC-AUC ranging from $0.91$ to $0.98$, and precision ranging from $0.70$ to $1.00$. The ability of CLAIRE-DSA to identify suitable images was evaluated on a segmentation task by filtering poor quality images and comparing segmentation performance on filtered and unfiltered datasets. Segmentation success rate increased from $42%$ to $69%$, $p < 0.001$. CLAIRE-DSA demonstrates strong potential as an automated tool for accurately classifying image properties in DSA series of acute ischemic stroke patients, supporting image annotation and quality control in clinical and research applications. Source code is available at https://gitlab.com/icai-stroke-lab/wp3_neurointerventional_ai/claire-dsa.

Method: Intra-group Consistency Augmentation Framework (ICAF) with View Augmentation Module for boundary synthesis and View Correction Module for information interaction between views.

Result: Achieved 70.6% mIoU on CdZnTe dataset using DeepLabV3+ with ResNet-101 backbone and only 2 group-annotated data (0.5% annotation).

Conclusion: The group-oriented approach effectively addresses many-to-one view relationships in CdZnTe segmentation, significantly reducing annotation requirements while maintaining high performance.

Abstract: Labeling Cadmium Zinc Telluride (CdZnTe) semiconductor images is challenging due to the low-contrast defect boundaries, necessitating annotators to cross-reference multiple views. These views share a single ground truth (GT), forming a unique many-to-one'' relationship. This characteristic renders advanced semi-supervised semantic segmentation (SSS) methods suboptimal, as they are generally limited by a one-to-one’’ relationship, where each image is independently associated with its GT. Such limitation may lead to error accumulation in low-contrast regions, further exacerbating confirmation bias. To address this issue, we revisit the SSS pipeline from a group-oriented perspective and propose a human-inspired solution: the Intra-group Consistency Augmentation Framework (ICAF). First, we experimentally validate the inherent consistency constraints within CdZnTe groups, establishing a group-oriented baseline using the Intra-group View Sampling (IVS). Building on this insight, we introduce the Pseudo-label Correction Network (PCN) to enhance consistency representation, which consists of two key modules. The View Augmentation Module (VAM) improves boundary details by dynamically synthesizing a boundary-aware view through the aggregation of multiple views. In the View Correction Module (VCM), this synthesized view is paired with other views for information interaction, effectively emphasizing salient regions while minimizing noise. Extensive experiments demonstrate the effectiveness of our solution for CdZnTe materials. Leveraging DeepLabV3+ with a ResNet-101 backbone as our segmentation model, we achieve a 70.6% mIoU on the CdZnTe dataset using only 2 group-annotated data (5\textperthousand). The code is available at \href{https://github.com/pipixiapipi/ICAF}{https://github.com/pipixiapipi/ICAF}.

Method: Proposes SocialTrack framework with: 1) specialized small-target detector with multi-scale feature enhancement, 2) Velocity Adaptive Cubature Kalman Filter for trajectory prediction, 3) Group Motion Compensation Strategy for social group modeling, and 4) Spatio-Temporal Memory Prediction for historical trajectory utilization.

Result: Extensive experiments on UAVDT and MOT17 datasets show SocialTrack outperforms state-of-the-art methods, with significant improvements in MOTA and IDF1 metrics, demonstrating superior robustness and adaptability.

Conclusion: SocialTrack effectively addresses complex UAV tracking challenges, provides modular and compatible framework that can integrate with existing trackers, and achieves state-of-the-art performance in multi-object tracking for urban intelligent transportation applications.

Abstract: As a key research direction in the field of multi-object tracking (MOT), UAV-based multi-object tracking has significant application value in the analysis and understanding of urban intelligent transportation systems. However, in complex UAV perspectives, challenges such as small target scale variations, occlusions, nonlinear crossing motions, and motion blur severely hinder the stability of multi-object tracking. To address these challenges, this paper proposes a novel multi-object tracking framework, SocialTrack, aimed at enhancing the tracking accuracy and robustness of small targets in complex urban traffic environments. The specialized small-target detector enhances the detection performance by employing a multi-scale feature enhancement mechanism. The Velocity Adaptive Cubature Kalman Filter (VACKF) improves the accuracy of trajectory prediction by incorporating a velocity dynamic modeling mechanism. The Group Motion Compensation Strategy (GMCS) models social group motion priors to provide stable state update references for low-quality tracks, significantly improving the target association accuracy in complex dynamic environments. Furthermore, the Spatio-Temporal Memory Prediction (STMP) leverages historical trajectory information to predict the future state of low-quality tracks, effectively mitigating identity switching issues. Extensive experiments on the UAVDT and MOT17 datasets demonstrate that SocialTrack outperforms existing state-of-the-art (SOTA) methods across several key metrics. Significant improvements in MOTA and IDF1, among other core performance indicators, highlight its superior robustness and adaptability. Additionally, SocialTrack is highly modular and compatible, allowing for seamless integration with existing trackers to further enhance performance.

Method: Used YOLOv4 model fine-tuned on 6 cities to detect cycles/motorcycles in 8000 GSV images per city across 185 global cities, then developed beta regression models with city-level mode shares as outcome

Result: Strong correlations: GSV motorcycle counts vs mode share (0.78), moderate for cycling (0.51). Models achieved R² of 0.614/0.612 and median absolute errors of 1.3%/1.4% for cycling/motorcycling respectively

Conclusion: Computer vision with GSV images effectively captures travel modes and activity, providing valuable insights alongside traditional data sources for global transportation analysis

Abstract: Transportation influence health by shaping exposure to physical activity, air pollution and injury risk.Comparative data on cycling and motorcycling behaviours is scarce, particularly at a global scale.Street view imagery, such as Google Street View (GSV), combined with computer vision, is a valuable resource for efficiently capturing travel behaviour data.This study demonstrates a novel approach using deep learning on street view images to estimate cycling and motorcycling levels across diverse cities worldwide.We utilized data from 185 global cities.The data on mode shares of cycling and motorcycling estimated using travel surveys or censuses.We used GSV images to detect cycles and motorcycles in sampled locations, using 8000 images per city.The YOLOv4 model, fine-tuned using images from six cities, achieved a mean average precision of 89% for detecting cycles and motorcycles in GSV images.A global prediction model was developed using beta regression with city-level mode shares as outcome, with log transformed explanatory variables of counts of GSV-detected images with cycles and motorcycles, while controlling for population density.We found strong correlations between GSV motorcycle counts and motorcycle mode share (0.78) and moderate correlations between GSV cycle counts and cycling mode share (0.51).Beta regression models predicted mode shares with $R^2$ values of 0.614 for cycling and 0.612 for motorcycling, achieving median absolute errors (MDAE) of 1.3% and 1.4%, respectively.Scatterplots demonstrated consistent prediction accuracy, though cities like Utrecht and Cali were outliers.The model was applied to 60 cities globally for which we didn’t have recent mode share data.We provided estimates for some cities in the Middle East, Latin America and East Asia.With computer vision, GSV images capture travel modes and activity, providing insights alongside traditional data sources.

Method: Leverages multiple style images with image prompt adapters and statistical feature alignment during denoising. Intervenes at both cross-attention and self-attention layers of UNet, using clustering to distill representative attention features from style samples.

Result: Achieves state-of-the-art results for stylization, demonstrating improved style representation and reduced content leakage from style images.

Conclusion: The proposed approach successfully addresses key limitations in current style transfer methods by utilizing multiple style references and sophisticated attention feature alignment techniques.

Abstract: Recent advances in latent diffusion models have enabled exciting progress in image style transfer. However, several key issues remain. For example, existing methods still struggle to accurately match styles. They are often limited in the number of style images that can be used. Furthermore, they tend to entangle content and style in undesired ways. To address this, we propose leveraging multiple style images which helps better represent style features and prevent content leaking from the style images. We design a method that leverages both image prompt adapters and statistical alignment of the features during the denoising process. With this, our approach is designed such that it can intervene both at the cross-attention and the self-attention layers of the denoising UNet. For the statistical alignment, we employ clustering to distill a small representative set of attention features from the large number of attention values extracted from the style samples. As demonstrated in our experimental section, the resulting method achieves state-of-the-art results for stylization.

Method: Proposes Next Visual Granularity (NVG) generation framework that starts from empty image and iteratively refines through visual granularity sequences with shared spatial resolution but varying token counts, capturing hierarchical representation.

Result: NVG models trained on ImageNet show clear scaling behavior and consistently outperform VAR series (FID scores: 3.30->3.03, 2.57->2.44, 2.09->2.06).

Conclusion: NVG framework demonstrates superior performance and offers hierarchical control over image generation, with code and models to be released for further research.

Abstract: We propose a novel approach to image generation by decomposing an image into a structured sequence, where each element in the sequence shares the same spatial resolution but differs in the number of unique tokens used, capturing different level of visual granularity. Image generation is carried out through our newly introduced Next Visual Granularity (NVG) generation framework, which generates a visual granularity sequence beginning from an empty image and progressively refines it, from global layout to fine details, in a structured manner. This iterative process encodes a hierarchical, layered representation that offers fine-grained control over the generation process across multiple granularity levels. We train a series of NVG models for class-conditional image generation on the ImageNet dataset and observe clear scaling behavior. Compared to the VAR series, NVG consistently outperforms it in terms of FID scores (3.30 -> 3.03, 2.57 ->2.44, 2.09 -> 2.06). We also conduct extensive analysis to showcase the capability and potential of the NVG framework. Our code and models will be released.

Method: Used pre-trained ResNet50 and vision transformer models fine-tuned on synthetic datasets. Developed novel polar coordinate transformation with hexbin visualization of phase-folded light curves. Implemented hierarchical classification: first stage for detached/overcontact types, second stage for spot detection.

Result: High accuracy (>96%) on validation data across Gaia G, I, and TESS passbands. Strong performance (>94%, up to 100% for TESS) on observational data from OGLE, DEBCat, and WUMaCat catalogues. Poor performance on automated spot detection.

Conclusion: Computer vision shows great potential for eclipsing binary morphological classification in large surveys, but automated spot detection requires further research due to poor performance on subtle photometric features.

Abstract: We present an application of computer vision methods to classify the light curves of eclipsing binaries (EB). We have used pre-trained models based on convolutional neural networks ($\textit{ResNet50}$) and vision transformers ($\textit{vit_base_patch16_224}$), which were fine-tuned on images created from synthetic datasets. To improve model generalisation and reduce overfitting, we developed a novel image representation by transforming phase-folded light curves into polar coordinates combined with hexbin visualisation. Our hierarchical approach in the first stage classifies systems into detached and overcontact types, and in the second stage identifies the presence or absence of spots. The binary classification models achieved high accuracy ($>96%$) on validation data across multiple passbands (Gaia~$G$, $I$, and $TESS$) and demonstrated strong performance ($>94%$, up to $100%$ for $TESS$) when tested on extensive observational data from the OGLE, DEBCat, and WUMaCat catalogues. While the primary binary classification was highly successful, the secondary task of automated spot detection performed poorly, revealing a significant limitation of our models for identifying subtle photometric features. This study highlights the potential of computer vision for EB morphological classification in large-scale surveys, but underscores the need for further research into robust, automated spot detection.

Method: Uses a 0.5B latent flow matching transformer conditioned on clinical reports via CT-Clip text encoder. Leverages A-VAE from FLUX for latent space and employs custom autoregressive approach to generate consistent whole CT volumes by predicting sequences of slices iteratively.

Result: Demonstrates superiority over state-of-the-art generative CT models in terms of temporal coherence, image diversity, and text-image alignment, as measured by FID, FVD, IS scores, and CLIP score.

Conclusion: CTFlow successfully generates high-quality 3D CT volumes from clinical reports, providing a powerful tool for medical research data augmentation and privacy-preserving synthetic data generation while maintaining diagnostic quality.

Abstract: Generative modelling of entire CT volumes conditioned on clinical reports has the potential to accelerate research through data augmentation, privacy-preserving synthesis and reducing regulator-constraints on patient data while preserving diagnostic signals. With the recent release of CT-RATE, a large-scale collection of 3D CT volumes paired with their respective clinical reports, training large text-conditioned CT volume generation models has become achievable. In this work, we introduce CTFlow, a 0.5B latent flow matching transformer model, conditioned on clinical reports. We leverage the A-VAE from FLUX to define our latent space, and rely on the CT-Clip text encoder to encode the clinical reports. To generate consistent whole CT volumes while keeping the memory constraints tractable, we rely on a custom autoregressive approach, where the model predicts the first sequence of slices of the volume from text-only, and then relies on the previously generated sequence of slices and the text, to predict the following sequence. We evaluate our results against state-of-the-art generative CT model, and demonstrate the superiority of our approach in terms of temporal coherence, image diversity and text-image alignment, with FID, FVD, IS scores and CLIP score.

Method: Organized a challenge with defined object classes, provided spatio-temporally aligned event camera and grayscale camera dataset, and analyzed top-5 ranking teams’ approaches.

Result: Presented challenge details, dataset overview, and results from participating teams with their segmentation performance on the multimodal vision task.

Conclusion: The challenge successfully benchmarked current methods for spatio-temporal instance segmentation using event-based vision, providing valuable insights and resources for future research in this emerging field.

Abstract: We present an overview of the Spatio-temporal Instance Segmentation (SIS) challenge held in conjunction with the CVPR 2025 Event-based Vision Workshop. The task is to predict accurate pixel-level segmentation masks of defined object classes from spatio-temporally aligned event camera and grayscale camera data. We provide an overview of the task, dataset, challenge details and results. Furthermore, we describe the methods used by the top-5 ranking teams in the challenge. More resources and code of the participants’ methods are available here: https://github.com/tub-rip/MouseSIS/blob/main/docs/challenge_results.md

Method: Proposes DEEP-SEA with Dual-Frequency Enhanced Self-Attention Spatial and Frequency Modulator to adaptively refine feature representations in frequency domains while preserving spatial information for better structural preservation.

Result: Comprehensive experiments on EUVP and LSUI datasets demonstrate superiority over state-of-the-art methods in restoring fine-grained image detail and structural consistency.

Conclusion: DEEP-SEA effectively mitigates underwater visual degradation and has potential to improve reliability of underwater monitoring platforms for ecological observation, species identification, and autonomous navigation.

Abstract: Continuous and reliable underwater monitoring is essential for assessing marine biodiversity, detecting ecological changes and supporting autonomous exploration in aquatic environments. Underwater monitoring platforms rely on mainly visual data for marine biodiversity analysis, ecological assessment and autonomous exploration. However, underwater environments present significant challenges due to light scattering, absorption and turbidity, which degrade image clarity and distort colour information, which makes accurate observation difficult. To address these challenges, we propose DEEP-SEA, a novel deep learning-based underwater image restoration model to enhance both low- and high-frequency information while preserving spatial structures. The proposed Dual-Frequency Enhanced Self-Attention Spatial and Frequency Modulator aims to adaptively refine feature representations in frequency domains and simultaneously spatial information for better structural preservation. Our comprehensive experiments on EUVP and LSUI datasets demonstrate the superiority over the state of the art in restoring fine-grained image detail and structural consistency. By effectively mitigating underwater visual degradation, DEEP-SEA has the potential to improve the reliability of underwater monitoring platforms for more accurate ecological observation, species identification and autonomous navigation.

Method: Two-stage training: 1) Train ensembled encoder via feature boosting to learn generalized representations that enhance decoder and balance classifier; 2) Use balanced KD and feature KD to compress ensembled encoder into a more generalized encoder.

Result: Extensive experiments on three benchmark datasets show superior performance, with ablation studies confirming component efficacy.

Conclusion: SEDEG effectively addresses catastrophic forgetting by sequentially improving both encoder and decoder generality, demonstrating strong performance in incremental learning scenarios with limited memory.

Abstract: In incremental learning, enhancing the generality of knowledge is crucial for adapting to dynamic data inputs. It can develop generalized representations or more balanced decision boundaries, preventing the degradation of long-term knowledge over time and thus mitigating catastrophic forgetting. Some emerging incremental learning methods adopt an encoder-decoder architecture and have achieved promising results. In the encoder-decoder achitecture, improving the generalization capabilities of both the encoder and decoder is critical, as it helps preserve previously learned knowledge while ensuring adaptability and robustness to new, diverse data inputs. However, many existing continual methods focus solely on enhancing one of the two components, which limits their effectiveness in mitigating catastrophic forgetting. And these methods perform even worse in small-memory scenarios, where only a limited number of historical samples can be stored. To mitigate this limitation, we introduces SEDEG, a two-stage training framework for vision transformers (ViT), focusing on sequentially improving the generality of both Decoder and Encoder. Initially, SEDEG trains an ensembled encoder through feature boosting to learn generalized representations, which subsequently enhance the decoder’s generality and balance the classifier. The next stage involves using knowledge distillation (KD) strategies to compress the ensembled encoder and develop a new, more generalized encoder. This involves using a balanced KD approach and feature KD for effective knowledge transfer. Extensive experiments on three benchmark datasets show SEDEG’s superior performance, and ablation studies confirm the efficacy of its components. The code is available at https://github.com/ShaolingPu/CIL.

Method: Uses two functionally complementary expert modules for multi-view feature extraction, incorporates prior knowledge-based consistency constraints and information geometry-based consensus mechanisms to enhance feature learning robustness.

Result: Extensive evaluations show CoMuCo consistently outperforms current methods in few-shot tasks across both existing and newly proposed cross-domain benchmarks.

Conclusion: The proposed CoMuCo strategy effectively addresses cross-domain limitations of VLMs and establishes a new benchmark for comprehensive evaluation of methods on imaging domains distinct from natural images.

Abstract: Vision-language models (VLMs) pre-trained on natural image and language data, such as CLIP, have exhibited significant potential in few-shot image recognition tasks, leading to development of various efficient transfer learning methods. These methods exploit inherent pre-learned knowledge in VLMs and have achieved strong performance on standard image datasets. However, their effectiveness is often limited when confronted with cross-domain tasks where imaging domains differ from natural images. To address this limitation, we propose Consistency-guided Multi-view Collaborative Optimization (CoMuCo), a novel fine-tuning strategy for VLMs. This strategy employs two functionally complementary expert modules to extract multi-view features, while incorporating prior knowledge-based consistency constraints and information geometry-based consensus mechanisms to enhance the robustness of feature learning. Additionally, a new cross-domain few-shot benchmark is established to help comprehensively evaluate methods on imaging domains distinct from natural images. Extensive empirical evaluations on both existing and newly proposed benchmarks suggest CoMuCo consistently outperforms current methods in few-shot tasks. The code and benchmark will be released.

Method: Used MONAI Auto3DSeg framework with 3D SegResNet, trained on 63 CBCT scans with 5-fold cross-validation. Preprocessing included image resampling and intensity clipping. Two-phase approach: ensemble fusion with Multi-Label STAPLE for initial segmentation, then tight cropping for nerve structure segmentation.

Result: Achieved average Dice score of 0.87 on the ToothFairy3 challenge out-of-sample validation set.

Conclusion: The approach demonstrates effective automated dental segmentation that can improve patient care in radiation oncology through efficient identification of dental structures and pathology.

Abstract: Cone-beam computed tomography (CBCT) has become an invaluable imaging modality in dentistry, enabling 3D visualization of teeth and surrounding structures for diagnosis and treatment planning. Automated segmentation of dental structures in CBCT can efficiently assist in identifying pathology (e.g., pulpal or periapical lesions) and facilitate radiation therapy planning in head and neck cancer patients. We describe the DLaBella29 team’s approach for the MICCAI 2025 ToothFairy3 Challenge, which involves a deep learning pipeline for multi-class tooth segmentation. We utilized the MONAI Auto3DSeg framework with a 3D SegResNet architecture, trained on a subset of the ToothFairy3 dataset (63 CBCT scans) with 5-fold cross-validation. Key preprocessing steps included image resampling to 0.6 mm isotropic resolution and intensity clipping. We applied an ensemble fusion using Multi-Label STAPLE on the 5-fold predictions to infer a Phase 1 segmentation and then conducted tight cropping around the easily segmented Phase 1 mandible to perform Phase 2 segmentation on the smaller nerve structures. Our method achieved an average Dice of 0.87 on the ToothFairy3 challenge out-of-sample validation set. This paper details the clinical context, data preparation, model development, results of our approach, and discusses the relevance of automated dental segmentation for improving patient care in radiation oncology.

Method: MPS-Tuning treats data distribution as a semantic manifold and preserves both macroscopic and microscopic topological structures by aligning Gram matrices before and after fine-tuning. It also optimizes pairwise similarities between image and text features to enhance class discriminability.

Result: Extensive experiments show that MPS-Tuning significantly improves model performance while effectively preserving the structure of the semantic manifold.

Conclusion: The proposed MPS-Tuning method successfully addresses the limitation of existing regularizations by explicitly constraining the intrinsic geometry of the semantic manifold during fine-tuning, leading to better performance and preserved structural integrity.

Abstract: Pretrained vision-language models (VLMs), such as CLIP, have shown remarkable potential in few-shot image classification and led to numerous effective transfer learning strategies. These methods leverage the pretrained knowledge of VLMs to enable effective domain adaptation while mitigating overfitting through parameter-efficient tuning or instance-based consistency constraints. However, such regularizations often neglect the geometric structure of data distribution, which may lead to distortion of the overall semantic representation. To overcome this limitation, we propose a novel fine-tuning method, Manifold-Preserving and Sculpting Tuning (MPS-Tuning). Regarding the data distribution in feature space as a semantic manifold, MPS-Tuning explicitly constrains the intrinsic geometry of this manifold while further sculpting it to enhance class separability. Specifically, MPS-Tuning preserves both macroscopic and microscopic topological structures of the original manifold by aligning Gram matrices of features before and after fine-tuning. Theoretically, this constraint is shown to approximate an upper bound of the Gromov-Wasserstein distance. Furthermore, features from the image and text modalities are paired, and pairwise similarities are optimized to enhance the manifold’s class discriminability. Extensive experiments demonstrate that MPS-Tuning significantly improves model performance while effectively preserving the structure of the semantic manifold. The code will be released.

Method: Proposes S^2-Guidance which leverages stochastic block-dropping during the forward process to construct stochastic sub-networks that guide the model away from low-quality predictions toward high-quality outputs.

Result: Extensive experiments on text-to-image and text-to-video generation show S^2-Guidance consistently surpasses CFG and other advanced guidance strategies in both qualitative and quantitative performance.

Conclusion: S^2-Guidance effectively addresses the limitations of CFG by refining suboptimal predictions through stochastic sub-networks, delivering superior performance in diffusion-based generation tasks.

Abstract: Classifier-free Guidance (CFG) is a widely used technique in modern diffusion models for enhancing sample quality and prompt adherence. However, through an empirical analysis on Gaussian mixture modeling with a closed-form solution, we observe a discrepancy between the suboptimal results produced by CFG and the ground truth. The model’s excessive reliance on these suboptimal predictions often leads to semantic incoherence and low-quality outputs. To address this issue, we first empirically demonstrate that the model’s suboptimal predictions can be effectively refined using sub-networks of the model itself. Building on this insight, we propose S^2-Guidance, a novel method that leverages stochastic block-dropping during the forward process to construct stochastic sub-networks, effectively guiding the model away from potential low-quality predictions and toward high-quality outputs. Extensive qualitative and quantitative experiments on text-to-image and text-to-video generation tasks demonstrate that S^2-Guidance delivers superior performance, consistently surpassing CFG and other advanced guidance strategies. Our code will be released.

Method: ONG uses Non-negative Matrix Factorization (NMF) for one-shot pruning at training start, then employs gradient masking to ensure only unpruned weights are updated, strictly preserving target sparsity throughout training.

Result: Experiments on CIFAR-10/100 with ResNet models show ONG achieves comparable or superior performance at various sparsity levels while maintaining structural integrity post-pruning.

Conclusion: ONG provides an effective one-shot pruning approach with clear sparsity targeting mechanism, offering a simpler alternative to complex iterative pruning methods while maintaining performance.

Abstract: Deep Neural Networks (DNNs) have achieved remarkable success but their large size poses deployment challenges. While various pruning techniques exist, many involve complex iterative processes, specialized criteria, or struggle to maintain sparsity effectively during training. We introduce ONG (One-shot NMF-based Gradient Masking), a novel sparsification strategy that identifies salient weight structures using Non-negative Matrix Factorization (NMF) for one-shot pruning at the outset of training. Subsequently, ONG employs a precise gradient masking mechanism to ensure that only unpruned weights are updated, strictly preserving the target sparsity throughout the training phase. We integrate ONG into the BIMP comparative framework and evaluate it on CIFAR-10 and CIFAR-100 with ResNet56, ResNet34, and ResNet18 against established stable sparsification methods. Our experiments demonstrate ONG’s ability to achieve comparable or superior performance at various sparsity levels while maintaining structural integrity post-pruning and offering a clear mechanism for targeting desired sparsities.

Method: Projects pixel information to 3D space via depth completion to get pseudo points, uses bilateral cross-view enhancement backbone with S2D and ResVC branches, employs iterative voxel-point aware pooling, and integrates IoU joint prediction with novel proposals generation.

Result: Extensive experiments demonstrate superior performance on KITTI, nuScenes and Waymo datasets.

Conclusion: The multi-stage cross-modal fusion approach effectively addresses 3D-2D information alignment challenges and achieves state-of-the-art 3D detection performance across multiple benchmark datasets.

Abstract: Multi-modal methods based on camera and LiDAR sensors have garnered significant attention in the field of 3D detection. However, many prevalent works focus on single or partial stage fusion, leading to insufficient feature extraction and suboptimal performance. In this paper, we introduce a multi-stage cross-modal fusion 3D detection framework, termed CMF-IOU, to effectively address the challenge of aligning 3D spatial and 2D semantic information. Specifically, we first project the pixel information into 3D space via a depth completion network to get the pseudo points, which unifies the representation of the LiDAR and camera information. Then, a bilateral cross-view enhancement 3D backbone is designed to encode LiDAR points and pseudo points. The first sparse-to-distant (S2D) branch utilizes an encoder-decoder structure to reinforce the representation of sparse LiDAR points. The second residual view consistency (ResVC) branch is proposed to mitigate the influence of inaccurate pseudo points via both the 3D and 2D convolution processes. Subsequently, we introduce an iterative voxel-point aware fine grained pooling module, which captures the spatial information from LiDAR points and textural information from pseudo points in the proposal refinement stage. To achieve more precise refinement during iteration, an intersection over union (IoU) joint prediction branch integrated with a novel proposals generation technique is designed to preserve the bounding boxes with both high IoU and classification scores. Extensive experiments show the superior performance of our method on the KITTI, nuScenes and Waymo datasets.

Method: The benchmark features text-and-layout pairs spanning seven challenging scenarios that investigate object generation, color fidelity, attribute recognition, inter-object relationships, and spatial control. It incorporates a layout alignment score to assess spatial accuracy.

Result: The benchmark was used to evaluate several state-of-the-art diffusion models, uncovering their respective strengths and limitations across diverse alignment tasks.

Conclusion: 7Bench provides a comprehensive evaluation framework for layout-guided text-to-image models, enabling better assessment of both semantic and spatial alignment which is essential for practical applications requiring precise control over generated content.

Abstract: Layout-guided text-to-image models offer greater control over the generation process by explicitly conditioning image synthesis on the spatial arrangement of elements. As a result, their adoption has increased in many computer vision applications, ranging from content creation to synthetic data generation. A critical challenge is achieving precise alignment between the image, textual prompt, and layout, ensuring semantic fidelity and spatial accuracy. Although recent benchmarks assess text alignment, layout alignment remains overlooked, and no existing benchmark jointly evaluates both. This gap limits the ability to evaluate a model’s spatial fidelity, which is crucial when using layout-guided generation for synthetic data, as errors can introduce noise and degrade data quality. In this work, we introduce 7Bench, the first benchmark to assess both semantic and spatial alignment in layout-guided text-to-image generation. It features text-and-layout pairs spanning seven challenging scenarios, investigating object generation, color fidelity, attribute recognition, inter-object relationships, and spatial control. We propose an evaluation protocol that builds on existing frameworks by incorporating the layout alignment score to assess spatial accuracy. Using 7Bench, we evaluate several state-of-the-art diffusion models, uncovering their respective strengths and limitations across diverse alignment tasks. The benchmark is available at https://github.com/Elizzo/7Bench.

Method: Dual-branch architecture integrating anomaly cues across scales, multi-resolution feature fusion strategy, and adaptive detector pool with assignment strategies based on patch features.

Result: Superior performance demonstrated on high-resolution benchmarks MVTec-HD, VisA-HD, and RealIAD-HD, showing effective detection of both subtle and large-scale anomalies.

Conclusion: HiAD provides an effective solution for high-resolution anomaly detection that balances detection accuracy with computational efficiency, meeting practical industrial demands.

Abstract: Current anomaly detection methods primarily focus on low-resolution scenarios. For high-resolution images, conventional downsampling often results in missed detections of subtle anomalous regions due to the loss of fine-grained discriminative information. Despite some progress, recent studies have attempted to improve detection resolution by employing lightweight networks or using simple image tiling and ensemble methods. However, these approaches still struggle to meet the practical demands of industrial scenarios in terms of detection accuracy and efficiency. To address the above issues, we propose HiAD, a general framework for high-resolution anomaly detection. HiAD is capable of detecting anomalous regions of varying sizes in high-resolution images under limited computational resources. Specifically, HiAD employs a dual-branch architecture that integrates anomaly cues across different scales to comprehensively capture both subtle and large-scale anomalies. Furthermore, it incorporates a multi-resolution feature fusion strategy to tackle the challenges posed by fine-grained texture variations in high-resolution images. To enhance both adaptability and efficiency, HiAD utilizes a detector pool in conjunction with various detector assignment strategies, enabling detectors to be adaptively assigned based on patch features, ensuring detection performance while effectively controlling computational costs. We conduct extensive experiments on our specifically constructed high-resolution anomaly detection benchmarks, including MVTec-HD, VisA-HD, and the real-world benchmark RealIAD-HD, demonstrating the superior performance of HiAD. The code is available at https://github.com/cnulab/HiAD.

Method: U-Net architecture with large patch sizes, foreground aware sampling, and semisupervised pre-training for automated fiber bundle segmentation in standalone histological slices.

Result: Improves detection of sparse bundles by over 20%, reduces False Discovery Rate by 40% compared to state-of-the-art, and eliminates common errors like mislabeling terminals as bundles.

Conclusion: The framework enables automated large-scale analysis of anatomic tracing data, generating more ground-truth data to validate and optimize dMRI tractography methods.

Abstract: Anatomic tracer studies are critical for validating and improving diffusion MRI (dMRI) tractography. However, large-scale analysis of data from such studies is hampered by the labor-intensive process of annotating fiber bundles manually on histological slides. Existing automated methods often miss sparse bundles or require complex post-processing across consecutive sections, limiting their flexibility and generalizability. We present a streamlined, fully automated framework for fiber bundle segmentation in macaque tracer data, based on a U-Net architecture with large patch sizes, foreground aware sampling, and semisupervised pre-training. Our approach eliminates common errors such as mislabeling terminals as bundles, improves detection of sparse bundles by over 20% and reduces the False Discovery Rate (FDR) by 40% compared to the state-of-the-art, all while enabling analysis of standalone slices. This new framework will facilitate the automated analysis of anatomic tracing data at a large scale, generating more ground-truth data that can be used to validate and optimize dMRI tractography methods.

Method: Uses an end-to-end framework built on large-scale video generative models with textual lighting instructions. Creates a mixed dataset of realistic and synthetic videos using 3D rendering and HDR-based lighting simulation. Implements joint training with domain-aware adapter to decouple relighting learning from domain appearance distribution.

Result: Experimental results show Lumen effectively edits input videos into cinematic relighted videos with consistent lighting and strict foreground preservation.

Conclusion: Lumen successfully addresses video relighting challenges by leveraging large-scale generative models and a carefully constructed mixed dataset, achieving high-quality results with foreground preservation and temporal consistency.

Abstract: Video relighting is a challenging yet valuable task, aiming to replace the background in videos while correspondingly adjusting the lighting in the foreground with harmonious blending. During translation, it is essential to preserve the original properties of the foreground, e.g., albedo, and propagate consistent relighting among temporal frames. In this paper, we propose Lumen, an end-to-end video relighting framework developed on large-scale video generative models, receiving flexible textual description for instructing the control of lighting and background. Considering the scarcity of high-qualified paired videos with the same foreground in various lighting conditions, we construct a large-scale dataset with a mixture of realistic and synthetic videos. For the synthetic domain, benefiting from the abundant 3D assets in the community, we leverage advanced 3D rendering engine to curate video pairs in diverse environments. For the realistic domain, we adapt a HDR-based lighting simulation to complement the lack of paired in-the-wild videos. Powered by the aforementioned dataset, we design a joint training curriculum to effectively unleash the strengths of each domain, i.e., the physical consistency in synthetic videos, and the generalized domain distribution in realistic videos. To implement this, we inject a domain-aware adapter into the model to decouple the learning of relighting and domain appearance distribution. We construct a comprehensive benchmark to evaluate Lumen together with existing methods, from the perspectives of foreground preservation and video consistency assessment. Experimental results demonstrate that Lumen effectively edit the input into cinematic relighted videos with consistent lighting and strict foreground preservation. Our project page: https://lumen-relight.github.io/

Method: Proposes MaskSem framework that uses Grad-CAM based on relative motion to guide joint masking of semantically rich temporal regions, and reconstructs hybrid high-order motion targets (low-order velocity + high-order acceleration) to learn comprehensive motion patterns.

Result: Experiments on NTU60, NTU120, and PKU-MMD datasets show that MaskSem combined with a vanilla transformer improves skeleton-based action recognition performance.

Conclusion: The semantic-guided masking approach with hybrid high-order motion reconstruction provides a more comprehensive understanding of motion patterns, making it more suitable for human-robot interaction applications.

Abstract: Human action recognition is a crucial task for intelligent robotics, particularly within the context of human-robot collaboration research. In self-supervised skeleton-based action recognition, the mask-based reconstruction paradigm learns the spatial structure and motion patterns of the skeleton by masking joints and reconstructing the target from unlabeled data. However, existing methods focus on a limited set of joints and low-order motion patterns, limiting the model’s ability to understand complex motion patterns. To address this issue, we introduce MaskSem, a novel semantic-guided masking method for learning 3D hybrid high-order motion representations. This novel framework leverages Grad-CAM based on relative motion to guide the masking of joints, which can be represented as the most semantically rich temporal orgions. The semantic-guided masking process can encourage the model to explore more discriminative features. Furthermore, we propose using hybrid high-order motion as the reconstruction target, enabling the model to learn multi-order motion patterns. Specifically, low-order motion velocity and high-order motion acceleration are used together as the reconstruction target. This approach offers a more comprehensive description of the dynamic motion process, enhancing the model’s understanding of motion patterns. Experiments on the NTU60, NTU120, and PKU-MMD datasets show that MaskSem, combined with a vanilla transformer, improves skeleton-based action recognition, making it more suitable for applications in human-robot interaction.

Method: ARMed first incorporates domain knowledge through supervised fine-tuning on chain-of-thought data, then applies reinforcement learning with textual correctness and adaptive semantic rewards to enhance reasoning quality.

Result: ARMed achieves 32.64% improvement on in-domain tasks and 11.65% gain on out-of-domain benchmarks across six challenging medical VQA benchmarks, consistently boosting both accuracy and generalization.

Conclusion: The results highlight the critical role of reward discriminability in medical RL and demonstrate the promise of semantically guided rewards for enabling robust and clinically meaningful multimodal reasoning in medical applications.

Abstract: Reinforcement learning (RL) with rule-based rewards has demonstrated strong potential in enhancing the reasoning and generalization capabilities of vision-language models (VLMs) and large language models (LLMs), while reducing computational overhead. However, its application in medical imaging remains underexplored. Existing reinforcement fine-tuning (RFT) approaches in this domain primarily target closed-ended visual question answering (VQA), limiting their applicability to real-world clinical reasoning. In contrast, open-ended medical VQA better reflects clinical practice but has received limited attention. While some efforts have sought to unify both formats via semantically guided RL, we observe that model-based semantic rewards often suffer from reward collapse, where responses with significant semantic differences receive similar scores. To address this, we propose ARMed (Adaptive Reinforcement for Medical Reasoning), a novel RL framework for open-ended medical VQA. ARMed first incorporates domain knowledge through supervised fine-tuning (SFT) on chain-of-thought data, then applies reinforcement learning with textual correctness and adaptive semantic rewards to enhance reasoning quality. We evaluate ARMed on six challenging medical VQA benchmarks. Results show that ARMed consistently boosts both accuracy and generalization, achieving a 32.64% improvement on in-domain tasks and an 11.65% gain on out-of-domain benchmarks. These results highlight the critical role of reward discriminability in medical RL and the promise of semantically guided rewards for enabling robust and clinically meaningful multimodal reasoning.

Method: Proposes GazeDETR with two separate decoders - one for human head prediction using local information, and another for gaze prediction that incorporates both local and global information. Uses coherent attentive fields for each subtask.

Result: Achieves state-of-the-art results on GazeFollow, VideoAttentionTarget and ChildPlay datasets. Outperforms existing end-to-end models by a notable margin.

Conclusion: Disentangling the representations for head localization and gaze prediction through separate decoders significantly improves gaze communication quantification, with the gaze decoder benefiting from both local and global contextual information.

Abstract: Gaze communication plays a crucial role in daily social interactions. Quantifying this behavior can help in human-computer interaction and digital phenotyping. While end-to-end models exist for gaze target detection, they only utilize a single decoder to simultaneously localize human heads and predict their corresponding gaze (e.g., 2D points or heatmap) in a scene. This multitask learning approach generates a unified and entangled representation for human head localization and gaze location prediction. Herein, we propose GazeDETR, a novel end-to-end architecture with two disentangled decoders that individually learn unique representations and effectively utilize coherent attentive fields for each subtask. More specifically, we demonstrate that its human head predictor utilizes local information, while its gaze decoder incorporates both local and global information. Our proposed architecture achieves state-of-the-art results on the GazeFollow, VideoAttentionTarget and ChildPlay datasets. It outperforms existing end-to-end models with a notable margin.

Method: Three key innovations: 1) Adaptive tiling strategies for diverse spatial interaction patterns via dynamic tile grouping, 2) Temporally varying windows that adjust sparsity levels based on frame proximity, and 3) An automated configuration search algorithm that optimizes sparse patterns while preserving critical attention pathways.

Result: Achieves 1.6~2.5x acceleration in attention computation on single-GPU setups while maintaining comparable visual quality with full-attention baselines.

Conclusion: Provides a principled approach to unlocking efficient long-form video generation through structured sparsity exploitation, demonstrating that attention matrices exhibit structured yet heterogeneous sparsity patterns that can be effectively leveraged for acceleration.

Abstract: The computational demands of self-attention mechanisms pose a critical challenge for transformer-based video generation, particularly in synthesizing ultra-long sequences. Current approaches, such as factorized attention and fixed sparse patterns, fail to fully exploit the inherent spatio-temporal redundancies in video data. Through systematic analysis of video diffusion transformers (DiT), we uncover a key insight: Attention matrices exhibit structured, yet heterogeneous sparsity patterns, where specialized heads dynamically attend to distinct spatiotemporal regions (e.g., local pattern, cross-shaped pattern, or global pattern). Existing sparse attention methods either impose rigid constraints or introduce significant overhead, limiting their effectiveness. To address this, we propose Compact Attention, a hardware-aware acceleration framework featuring three innovations: 1) Adaptive tiling strategies that approximate diverse spatial interaction patterns via dynamic tile grouping, 2) Temporally varying windows that adjust sparsity levels based on frame proximity, and 3) An automated configuration search algorithm that optimizes sparse patterns while preserving critical attention pathways. Our method achieves 1.6~2.5x acceleration in attention computation on single-GPU setups while maintaining comparable visual quality with full-attention baselines. This work provides a principled approach to unlocking efficient long-form video generation through structured sparsity exploitation. Project Page: https://yo-ava.github.io/Compact-Attention.github.io/

Method: Proposes a zero-cost proxy using SVD of layer features and extrinsic curvature of network output, formulated as harmonic mean of inverse feature condition number and curvature components.

Result: Superior performance on NAS-Bench-101, NAS-Bench-201, TransNAS-Bench-101-micro, and NAS tasks in DARTS and AutoFormer search spaces using only one label-free sample.

Conclusion: The method effectively combines convergence, generalization and expressivity in a single label-free approach, demonstrating high accuracy and efficiency in neural architecture search.

Abstract: Zero-shot Neural Architecture Search (NAS) typically optimises the architecture search process by exploiting the network or gradient properties at initialisation through zero-cost proxies. The existing proxies often rely on labelled data, which is usually unavailable in real-world settings. Furthermore, the majority of the current methods focus either on optimising the convergence and generalisation attributes or solely on the expressivity of the network architectures. To address both limitations, we first demonstrate how channel collinearity affects the convergence and generalisation properties of a neural network. Then, by incorporating the convergence, generalisation and expressivity in one approach, we propose a zero-cost proxy that omits the requirement of labelled data for its computation. In particular, we leverage the Singular Value Decomposition (SVD) of the neural network layer features and the extrinsic curvature of the network output to design our proxy. %As a result, the proposed proxy is formulated as the simplified harmonic mean of the logarithms of two key components: the sum of the inverse of the feature condition number and the extrinsic curvature of the network output. Our approach enables accurate prediction of network performance on test data using only a single label-free data sample. Our extensive evaluation includes a total of six experiments, including the Convolutional Neural Network (CNN) search space, i.e. DARTS and the Transformer search space, i.e. AutoFormer. The proposed proxy demonstrates a superior performance on multiple correlation benchmarks, including NAS-Bench-101, NAS-Bench-201, and TransNAS-Bench-101-micro; as well as on the NAS task within the DARTS and the AutoFormer search space, all while being notably efficient. The code is available at https://github.com/rohanasthana/Dextr.

Method: Categorizes existing MMVOT methods based on how they handle visible (RGB) and auxiliary modalities (thermal infrared, depth, event, near infrared, language, sonar), analyzing data collection challenges, modality alignment, annotation, and model design approaches.

Result: Provides an omni survey covering all aspects of MMVOT, reveals long-tail distribution of object categories in existing datasets with noticeable lack of animal categories compared to RGB datasets, and addresses when multi-modal tracking provides superior performance over unimodal tracking.

Conclusion: Multi-modal visual object tracking offers significant potential for smart city applications but requires careful consideration of modality integration, dataset limitations, and specific circumstances where multi-modal approaches outperform single-modal solutions.

Abstract: The development of smart cities has led to the generation of massive amounts of multi-modal data in the context of a range of tasks that enable a comprehensive monitoring of the smart city infrastructure and services. This paper surveys one of the most critical tasks, multi-modal visual object tracking (MMVOT), from the perspective of multimodality analysis. Generally, MMVOT differs from single-modal tracking in four key aspects, data collection, modality alignment and annotation, model designing, and evaluation. Accordingly, we begin with an introduction to the relevant data modalities, laying the groundwork for their integration. This naturally leads to a discussion of challenges of multi-modal data collection, alignment, and annotation. Subsequently, existing MMVOT methods are categorised, based on different ways to deal with visible (RGB) and X modalities: programming the auxiliary X branch with replicated or non-replicated experimental configurations from the RGB branch. Here X can be thermal infrared (T), depth (D), event (E), near infrared (NIR), language (L), or sonar (S). The final part of the paper addresses evaluation and benchmarking. In summary, we undertake an omni survey of all aspects of multi-modal visual object tracking (VOT), covering six MMVOT tasks and featuring 338 references in total. In addition, we discuss the fundamental rhetorical question: Is multi-modal tracking always guaranteed to provide a superior solution to unimodal tracking with the help of information fusion, and if not, in what circumstances its application is beneficial. Furthermore, for the first time in this field, we analyse the distributions of the object categories in the existing MMVOT datasets, revealing their pronounced long-tail nature and a noticeable lack of animal categories when compared with RGB datasets.

Method: Empirical investigation providing evidence that enhancing feature diversity improves recognition of open set samples and facilitates retention of previous knowledge plus integration of new data in continual learning.

Result: Increased feature diversity improves open set sample recognition and supports both knowledge retention and new data integration in continual learning scenarios.

Conclusion: Feature diversity plays a crucial role in addressing both open set recognition and continual learning challenges, and these findings should inspire further research into practical methods and theoretical understanding.

Abstract: Open set recognition (OSR) and continual learning are two critical challenges in machine learning, focusing respectively on detecting novel classes at inference time and updating models to incorporate the new classes. While many recent approaches have addressed these problems, particularly OSR, by heuristically promoting feature diversity, few studies have directly examined the role that feature diversity plays in tackling them. In this work, we provide empirical evidence that enhancing feature diversity improves the recognition of open set samples. Moreover, increased feature diversity also facilitates both the retention of previously learned data and the integration of new data in continual learning. We hope our findings can inspire further research into both practical methods and theoretical understanding in these domains.

Method: Integrates 4D radar Doppler to build motion-centric dynamic maps, generates reference queries for dynamic/high-confidence regions and exploratory queries for occluded areas, and exchanges only query-specific BEV features using multi-scale gated deformable attention.

Result: Achieves up to 90% lower bandwidth than full-map sharing while matching or surpassing prior baselines across varied traffic densities and occlusions.

Conclusion: SlimComm provides a communication-efficient framework that significantly reduces bandwidth requirements while maintaining or improving perception accuracy in collaborative autonomous vehicle systems.

Abstract: Collaborative perception allows connected autonomous vehicles (CAVs) to overcome occlusion and limited sensor range by sharing intermediate features. Yet transmitting dense Bird’s-Eye-View (BEV) feature maps can overwhelm the bandwidth available for inter-vehicle communication. We present SlimComm, a communication-efficient framework that integrates 4D radar Doppler with a query-driven sparse scheme. SlimComm builds a motion-centric dynamic map to distinguish moving from static objects and generates two query types: (i) reference queries on dynamic and high-confidence regions, and (ii) exploratory queries probing occluded areas via a two-stage offset. Only query-specific BEV features are exchanged and fused through multi-scale gated deformable attention, reducing payload while preserving accuracy. For evaluation, we release OPV2V-R and Adver-City-R, CARLA-based datasets with per-point Doppler radar. SlimComm achieves up to 90% lower bandwidth than full-map sharing while matching or surpassing prior baselines across varied traffic densities and occlusions. Dataset and code will be available at: https://url.fzi.de/SlimComm.

Method: Three key components: (1) scalable data pipeline producing 1200 hours of annotated video from Unreal Engine/GTA5, (2) action injection module for frame-level mouse/keyboard inputs, (3) few-step distillation using causal architecture for real-time streaming generation.

Result: The model generates high-quality minute-level videos across diverse scenes at 25 FPS, achieving ultra-fast speed while maintaining quality.

Conclusion: Matrix-Game 2.0 enables real-time interactive video generation and advances research in interactive world modeling, with open-sourced weights and codebase.

Abstract: Recent advances in interactive video generations have demonstrated diffusion model’s potential as world models by capturing complex physical dynamics and interactive behaviors. However, existing interactive world models depend on bidirectional attention and lengthy inference steps, severely limiting real-time performance. Consequently, they are hard to simulate real-world dynamics, where outcomes must update instantaneously based on historical context and current actions. To address this, we present Matrix-Game 2.0, an interactive world model generates long videos on-the-fly via few-step auto-regressive diffusion. Our framework consists of three key components: (1) A scalable data production pipeline for Unreal Engine and GTA5 environments to effectively produce massive amounts (about 1200 hours) of video data with diverse interaction annotations; (2) An action injection module that enables frame-level mouse and keyboard inputs as interactive conditions; (3) A few-step distillation based on the casual architecture for real-time and streaming video generation. Matrix Game 2.0 can generate high-quality minute-level videos across diverse scenes at an ultra-fast speed of 25 FPS. We open-source our model weights and codebase to advance research in interactive world modeling.

Method: Proposes EgoTwin framework built on diffusion transformer architecture with head-centric motion representation and cybernetics-inspired interaction mechanism to capture causal interplay between video and motion.

Result: Extensive experiments demonstrate effectiveness, supported by a curated large-scale dataset of synchronized text-video-motion triplets and novel metrics for evaluation.

Conclusion: EgoTwin successfully bridges the gap in egocentric video generation by addressing key challenges of viewpoint alignment and causal interplay between visual content and human motion.

Abstract: While exocentric video synthesis has achieved great progress, egocentric video generation remains largely underexplored, which requires modeling first-person view content along with camera motion patterns induced by the wearer’s body movements. To bridge this gap, we introduce a novel task of joint egocentric video and human motion generation, characterized by two key challenges: 1) Viewpoint Alignment: the camera trajectory in the generated video must accurately align with the head trajectory derived from human motion; 2) Causal Interplay: the synthesized human motion must causally align with the observed visual dynamics across adjacent video frames. To address these challenges, we propose EgoTwin, a joint video-motion generation framework built on the diffusion transformer architecture. Specifically, EgoTwin introduces a head-centric motion representation that anchors the human motion to the head joint and incorporates a cybernetics-inspired interaction mechanism that explicitly captures the causal interplay between video and motion within attention operations. For comprehensive evaluation, we curate a large-scale real-world dataset of synchronized text-video-motion triplets and design novel metrics to assess video-motion consistency. Extensive experiments demonstrate the effectiveness of the EgoTwin framework.

Method: Uses hierarchical adapters: Protocol-Level Adapters for sequence-specific characteristics, Center-Level Adapters for scanner-dependent variations built on variational unrolling backbone, and Universal Adapter for generalization to unseen centers through stochastic training. Employs multi-scale SSIM loss with frequency domain enhancement and contrast-adaptive weighting.

Result: Comprehensive evaluation on CMRxRecon2025 dataset spanning 5+ centers, 10+ scanners, and 9 modalities demonstrates superior cross-center generalization while maintaining reconstruction quality.

Conclusion: HierAdaptMR effectively addresses multi-level domain variations in cardiac MRI reconstruction through parameter-efficient hierarchical adaptation, enabling robust performance across diverse clinical centers and scanner configurations.

Abstract: Deep learning-based cardiac MRI reconstruction faces significant domain shift challenges when deployed across multiple clinical centers with heterogeneous scanner configurations and imaging protocols. We propose HierAdaptMR, a hierarchical feature adaptation framework that addresses multi-level domain variations through parameter-efficient adapters. Our method employs Protocol-Level Adapters for sequence-specific characteristics and Center-Level Adapters for scanner-dependent variations, built upon a variational unrolling backbone. A Universal Adapter enables generalization to entirely unseen centers through stochastic training that learns center-invariant adaptations. The framework utilizes multi-scale SSIM loss with frequency domain enhancement and contrast-adaptive weighting for robust optimization. Comprehensive evaluation on the CMRxRecon2025 dataset spanning 5+ centers, 10+ scanners, and 9 modalities demonstrates superior cross-center generalization while maintaining reconstruction quality. code: https://github.com/Ruru-Xu/HierAdaptMR

Method: Uses semantic segmentation and category identification ranked by a vision-language model to identify important objects. Generates spherical proxies around highly ranked objects to guide users during scanning for better view-dependent appearance coverage.

Result: The method shows superior performance in real scenes compared to conventional view sampling strategies, enabling better representation of view-dependent appearance.

Conclusion: The proposed situated visualization technique effectively guides users during multi-scale scanning by focusing on important objects that require extended image coverage, addressing limitations of human camera operators and improving view synthesis quality.

Abstract: Novel view synthesis from images, for example, with 3D Gaussian splatting, has made great progress. Rendering fidelity and speed are now ready even for demanding virtual reality applications. However, the problem of assisting humans in collecting the input images for these rendering algorithms has received much less attention. High-quality view synthesis requires uniform and dense view sampling. Unfortunately, these requirements are not easily addressed by human camera operators, who are in a hurry, impatient, or lack understanding of the scene structure and the photographic process. Existing approaches to guide humans during image acquisition concentrate on single objects or neglect view-dependent material characteristics. We propose a novel situated visualization technique for scanning at multiple scales. During the scanning of a scene, our method identifies important objects that need extended image coverage to properly represent view-dependent appearance. To this end, we leverage semantic segmentation and category identification, ranked by a vision-language model. Spherical proxies are generated around highly ranked objects to guide the user during scanning. Our results show superior performance in real scenes compared to conventional view sampling strategies.

Method: End-to-end diffusion-based approach combining frozen UNet to preserve appearance/background details with ControlNet that guides shape transformation using target SMPL depth maps, trained on a new large-scale dataset of 18,573 images across 1523 subjects.

Result: Achieves per-vertex reconstruction error of 7.5mm (significantly lower than baseline 13.6mm), produces realistic results that accurately match target shapes while preserving identity, clothing, and background.

Conclusion: The proposed Odo method with the new dataset enables realistic and intuitive body reshaping, outperforming prior approaches and addressing key limitations in human shape editing.

Abstract: Human shape editing enables controllable transformation of a person’s body shape, such as thin, muscular, or overweight, while preserving pose, identity, clothing, and background. Unlike human pose editing, which has advanced rapidly, shape editing remains relatively underexplored. Current approaches typically rely on 3D morphable models or image warping, often introducing unrealistic body proportions, texture distortions, and background inconsistencies due to alignment errors and deformations. A key limitation is the lack of large-scale, publicly available datasets for training and evaluating body shape manipulation methods. In this work, we introduce the first large-scale dataset of 18,573 images across 1523 subjects, specifically designed for controlled human shape editing. It features diverse variations in body shape, including fat, muscular and thin, captured under consistent identity, clothing, and background conditions. Using this dataset, we propose Odo, an end-to-end diffusion-based method that enables realistic and intuitive body reshaping guided by simple semantic attributes. Our approach combines a frozen UNet that preserves fine-grained appearance and background details from the input image with a ControlNet that guides shape transformation using target SMPL depth maps. Extensive experiments demonstrate that our method outperforms prior approaches, achieving per-vertex reconstruction errors as low as 7.5mm, significantly lower than the 13.6mm observed in baseline methods, while producing realistic results that accurately match the desired target shapes.

Method: Leverages LLM in-context learning ability to iteratively refine candidate prompt distribution based on reference images, inspired by LLM jailbreaking techniques.

Result: PRISM effectively generates accurate prompts for objects, styles, and images across multiple T2I models including Stable Diffusion, DALL-E, and Midjourney.

Conclusion: PRISM provides a versatile and effective solution for automated prompt generation that works with black-box access and produces human-interpretable, transferable prompts.

Abstract: Prompt engineering is an effective but labor-intensive way to control text-to-image (T2I) generative models. Its time-intensive nature and complexity have spurred the development of algorithms for automated prompt generation. However, these methods often struggle with transferability across T2I models, require white-box access to the underlying model, or produce non-intuitive prompts. In this work, we introduce PRISM, an algorithm that automatically produces human-interpretable and transferable prompts that can effectively generate desired concepts given only black-box access to T2I models. Inspired by large language model (LLM) jailbreaking, PRISM leverages the in-context learning ability of LLMs to iteratively refine the candidate prompt distribution built upon the reference images. Our experiments demonstrate the versatility and effectiveness of PRISM in generating accurate prompts for objects, styles, and images across multiple T2I models, including Stable Diffusion, DALL-E, and Midjourney.

Method: Two-stage approach: 1) Gaze-guided contrastive learning for disease classification using visual features, clinical labels, bounding boxes, and eye-tracking signals with multi-term gaze-attention loss; 2) Modular report generation pipeline extracting confidence-weighted diagnostic keywords, mapping to anatomical regions, and generating region-aligned sentences via structured prompts.

Result: Incorporating fixations improved F1 score from 0.597 to 0.631 (+5.70%) and AUC from 0.821 to 0.849 (+3.41%), with enhanced precision and recall. Report generation pipeline improved clinical keyword recall and ROUGE overlap metrics.

Conclusion: Integrating gaze data significantly improves both classification performance and the interpretability of generated medical reports, demonstrating the value of eye-tracking signals in medical AI systems.

Abstract: We propose a two-stage multimodal framework that enhances disease classification and region-aware radiology report generation from chest X-rays, leveraging the MIMIC-Eye dataset. In the first stage, we introduce a gaze-guided contrastive learning architecture for disease classification. It integrates visual features, clinical labels, bounding boxes, and radiologist eye-tracking signals and is equipped with a novel multi-term gaze-attention loss combining MSE, KL divergence, correlation, and center-of-mass alignment. Incorporating fixations improves F1 score from 0.597 to 0.631 (+5.70%) and AUC from 0.821 to 0.849 (+3.41%), while also improving precision and recall, highlighting the effectiveness of gaze-informed attention supervision. In the second stage, we present a modular report generation pipeline that extracts confidence-weighted diagnostic keywords, maps them to anatomical regions using a curated dictionary constructed from domain-specific priors, and generates region-aligned sentences via structured prompts. This pipeline improves report quality as measured by clinical keyword recall and ROUGE overlap. Our results demonstrate that integrating gaze data improves both classification performance and the interpretability of generated medical reports.

Method: Proposes using Stable Diffusion to generate synthetic versions of bona fide ID card images, creating additional training data to improve detector generalization capabilities.

Result: The synthetic images are successfully identified as bona fide by both a system trained from scratch and a commercial PAD solution, leading to improved detection performance and helping overcome data restrictions.

Conclusion: Synthetic image generation using Stable Diffusion is an effective approach to address data scarcity in ID card Presentation Attack Detection systems, enhancing generalization and detection capabilities.

Abstract: Nowadays, the development of a Presentation Attack Detection (PAD) system for ID cards presents a challenge due to the lack of images available to train a robust PAD system and the increase in diversity of possible attack instrument species. Today, most algorithms focus on generating attack samples and do not take into account the limited number of bona fide images. This work is one of the first to propose a method for mimicking bona fide images by generating synthetic versions of them using Stable Diffusion, which may help improve the generalisation capabilities of the detector. Furthermore, the new images generated are evaluated in a system trained from scratch and in a commercial solution. The PAD system yields an interesting result, as it identifies our images as bona fide, which has a positive impact on detection performance and data restrictions.

Method: Created Chessboard dataset with 3,123,253 questions and balanced answer distribution where each answer is linked to specific image cells. Developed Checkmate model that identifies the most relevant image cells for its decisions to enable fine-grained visual reasoning.

Result: The approach improves transparency and supports more trustworthy decision-making in RSVQA systems across multiple model architectures.

Conclusion: The Chessboard dataset and Checkmate model provide an effective solution for enhancing interpretability and reducing biases in remote sensing visual question answering systems.

Abstract: Remote Sensing Visual Question Answering (RSVQA) presents unique challenges in ensuring that model decisions are both understandable and grounded in visual content. Current models often suffer from a lack of interpretability and explainability, as well as from biases in dataset distributions that lead to shortcut learning. In this work, we tackle these issues by introducing a novel RSVQA dataset, Chessboard, designed to minimize biases through 3'123'253 questions and a balanced answer distribution. Each answer is linked to one or more cells within the image, enabling fine-grained visual reasoning. Building on this dataset, we develop an explainable and interpretable model called Checkmate that identifies the image cells most relevant to its decisions. Through extensive experiments across multiple model architectures, we show that our approach improves transparency and supports more trustworthy decision-making in RSVQA systems.

Method: Finetune Stable Diffusion to generate novel views along epipolar direction (left-left, right-right, and intermediate views) using directional prompts to supplement occluded pixels for explicit photometric reconstruction.

Result: Achieves up to 35% outlier reduction and state-of-the-art performance across multiple benchmark datasets using only unlabeled stereo image pairs.

Conclusion: DMS provides a cost-free, plug-and-play solution that significantly enhances self-supervised stereo matching and monocular depth estimation by leveraging diffusion models for view synthesis.

Abstract: While supervised stereo matching and monocular depth estimation have advanced significantly with learning-based algorithms, self-supervised methods using stereo images as supervision signals have received relatively less focus and require further investigation. A primary challenge arises from ambiguity introduced during photometric reconstruction, particularly due to missing corresponding pixels in ill-posed regions of the target view, such as occlusions and out-of-frame areas. To address this and establish explicit photometric correspondences, we propose DMS, a model-agnostic approach that utilizes geometric priors from diffusion models to synthesize novel views along the epipolar direction, guided by directional prompts. Specifically, we finetune a Stable Diffusion model to simulate perspectives at key positions: left-left view shifted from the left camera, right-right view shifted from the right camera, along with an additional novel view between the left and right cameras. These synthesized views supplement occluded pixels, enabling explicit photometric reconstruction. Our proposed DMS is a cost-free, ‘‘plug-and-play’’ method that seamlessly enhances self-supervised stereo matching and monocular depth estimation, and relies solely on unlabeled stereo image pairs for both training and synthesizing. Extensive experiments demonstrate the effectiveness of our approach, with up to 35% outlier reduction and state-of-the-art performance across multiple benchmark datasets.

Method: Comparative analysis of two RT-DETR variants (Large and Extra-Large) trained on coastal debris dataset, evaluating accuracy metrics and inference times.

Result: RT-DETR-X achieved slightly better accuracy (mAP@50: 0.816, mAP@50-95: 0.612) but RT-DETR-L was significantly faster (20.1ms vs 34.5ms inference time).

Conclusion: RT-DETR-L offers more practical real-time deployment due to better speed-accuracy trade-off, providing insights for environmental conservation applications.

Abstract: Coastal pollution is a pressing global environmental issue, necessitating scalable and automated solutions for monitoring and management. This study investigates the efficacy of the Real-Time Detection Transformer (RT-DETR), a state-of-the-art, end-to-end object detection model, for the automated detection and counting of beach litter. A rigorous comparative analysis is conducted between two model variants, RT-DETR-Large (RT-DETR-L) and RT-DETR-Extra-Large (RT-DETR-X), trained on a publicly available dataset of coastal debris. The evaluation reveals that the RT-DETR-X model achieves marginally superior accuracy, with a mean Average Precision at 50% IoU (mAP@50) of 0.816 and a mAP@50-95 of 0.612, compared to the RT-DETR-L model’s 0.810 and 0.606, respectively. However, this minor performance gain is realized at a significant computational cost; the RT-DETR-L model demonstrates a substantially faster inference time of 20.1 ms versus 34.5 ms for the RT-DETR-X. The findings suggest that the RT-DETR-L model offers a more practical and efficient solution for real-time, in-field deployment due to its superior balance of processing speed and detection accuracy. This research provides valuable insights into the application of advanced Transformer-based detectors for environmental conservation, highlighting the critical trade-offs between model complexity and operational viability.

Method: Hard negative contrastive learning combining: 1) image-based contrastive with generation-based hard negatives from perturbed diagram code, 2) text-based contrastive with rule-based negatives from modified geometric descriptions and retrieval-based negatives from caption similarity.

Result: MMGeoLM model significantly outperforms other open-source models on three geometric reasoning benchmarks, with 7B size rivaling GPT-4o. Ablation studies reveal key insights on hard negative optimization.

Conclusion: The proposed hard negative contrastive learning framework effectively enhances geometric understanding in vision encoders, enabling LMMs to achieve superior geometric reasoning performance comparable to much larger closed-source models.

Abstract: Benefiting from contrastively trained visual encoders on large-scale natural scene images, Large Multimodal Models (LMMs) have achieved remarkable performance across various visual perception tasks. However, the inherent limitations of contrastive learning upon summarized descriptions fundamentally restrict the capabilities of models in meticulous reasoning, particularly in crucial scenarios of geometric problem-solving. To enhance geometric understanding, we propose a novel hard negative contrastive learning framework for the vision encoder, which combines image-based contrastive learning using generation-based hard negatives created by perturbing diagram generation code, and text-based contrastive learning using rule-based negatives derived from modified geometric descriptions and retrieval-based negatives selected based on caption similarity. We train CLIP using our hard negative learning method, namely MMCLIP (Multimodal Math CLIP), and subsequently train an LMM for geometric problem-solving. Experiments show that our trained model, MMGeoLM, significantly outperforms other open-source models on three geometric reasoning benchmarks. Even with a size of 7B, it can rival powerful closed-source models like GPT-4o. We further conduct ablation studies to analyze three key factors: hard negative types, the efficiency of image-based negatives, and training configurations. These analyses yield important insights into optimizing hard negative strategies for geometric reasoning tasks.

Method: Render actions into visual skeleton prompts as domain-agnostic representations, construct skeletons from human-object interactions and robotic manipulation data, integrate into pretrained video models via lightweight fine-tuning.

Result: Experiments on EgoVid, RT-1 and DROID datasets demonstrate effectiveness in enabling precise action control while preserving cross-domain dynamics learning.

Conclusion: Visual action prompts provide a unified representation that balances action precision and dynamic transferability for complex high-DoF interaction video generation.

Abstract: We present visual action prompts, a unified action representation for action-to-video generation of complex high-DoF interactions while maintaining transferable visual dynamics across domains. Action-driven video generation faces a precision-generality trade-off: existing methods using text, primitive actions, or coarse masks offer generality but lack precision, while agent-centric action signals provide precision at the cost of cross-domain transferability. To balance action precision and dynamic transferability, we propose to “render” actions into precise visual prompts as domain-agnostic representations that preserve both geometric precision and cross-domain adaptability for complex actions; specifically, we choose visual skeletons for their generality and accessibility. We propose robust pipelines to construct skeletons from two interaction-rich data sources - human-object interactions (HOI) and dexterous robotic manipulation - enabling cross-domain training of action-driven generative models. By integrating visual skeletons into pretrained video generation models via lightweight fine-tuning, we enable precise action control of complex interaction while preserving the learning of cross-domain dynamics. Experiments on EgoVid, RT-1 and DROID demonstrate the effectiveness of our proposed approach. Project page: https://zju3dv.github.io/VAP/.

Method: Developed a framework using visual encoders (ResNet/ViT) to extract attributes from tool images and fine-tuned language models (GPT-2/LLaMA/DeepSeek) to derive required attributes from task descriptions, using a comprehensive dataset (ToolNet) with 115 tools and 13 attributes.

Result: Achieved 74% accuracy in tool selection tasks, significantly outperforming direct tool matching (20%) and smaller multimodal models (21%-58%), while approaching GPT-4o performance (73%) with fewer parameters. Human evaluation showed alignment with human decision-making patterns.

Conclusion: Provides a parameter-efficient, interpretable solution that mimics human-like tool cognition, advancing both cognitive science understanding and practical applications in tool selection tasks, with manipulation-related attributes proving most critical.

Abstract: Flexible tool selection reflects a complex cognitive ability that distinguishes humans from other species, yet computational models that capture this ability remain underdeveloped. We developed a framework using low-dimensional attribute representations to bridge visual tool perception and linguistic task understanding. We constructed a comprehensive dataset (ToolNet) containing 115 common tools labeled with 13 carefully designed attributes spanning physical, functional, and psychological properties, paired with natural language scenarios describing tool usage. Visual encoders (ResNet or ViT) extract attributes from tool images while fine-tuned language models (GPT-2, LLaMA, DeepSeek) derive required attributes from task descriptions. Our approach achieves 74% accuracy in tool selection tasks-significantly outperforming direct tool matching (20%) and smaller multimodal models (21%-58%), while approaching performance of much larger models like GPT-4o (73%) with substantially fewer parameters. Human evaluation studies validate our framework’s alignment with human decision-making patterns, and generalization experiments demonstrate effective performance on novel tool categories. Ablation studies revealed that manipulation-related attributes (graspability, elongation, hand-relatedness) consistently prove most critical across modalities. This work provides a parameter-efficient, interpretable solution that mimics human-like tool cognition, advancing both cognitive science understanding and practical applications in tool selection tasks.

Method: Training-free framework that works with minimal example motions (1-2) on target skeleton by establishing sparse bone correspondences between source and target skeletons.

Result: Achieves efficient and reliable performance in both similar-skeleton and cross-species skeleton transfer scenarios, with successful integration in downstream applications.

Conclusion: Motion2Motion provides a practical solution for industrial applications, demonstrating potential for motion transfer across diverse topological structures without requiring large training datasets.

Abstract: This work studies the challenge of transfer animations between characters whose skeletal topologies differ substantially. While many techniques have advanced retargeting techniques in decades, transfer motions across diverse topologies remains less-explored. The primary obstacle lies in the inherent topological inconsistency between source and target skeletons, which restricts the establishment of straightforward one-to-one bone correspondences. Besides, the current lack of large-scale paired motion datasets spanning different topological structures severely constrains the development of data-driven approaches. To address these limitations, we introduce Motion2Motion, a novel, training-free framework. Simply yet effectively, Motion2Motion works with only one or a few example motions on the target skeleton, by accessing a sparse set of bone correspondences between the source and target skeletons. Through comprehensive qualitative and quantitative evaluations, we demonstrate that Motion2Motion achieves efficient and reliable performance in both similar-skeleton and cross-species skeleton transfer scenarios. The practical utility of our approach is further evidenced by its successful integration in downstream applications and user interfaces, highlighting its potential for industrial applications. Code and data are available at https://lhchen.top/Motion2Motion.

Method: Constructs segmentation-aware Gaussian fields and enforces bi-directional photometric and semantic consistency across scans. Uses pseudo-intermediate scene state for unified alignment and collaborative co-pruning strategies to refine geometry.

Result: Enables high-fidelity rendering and object-level scene manipulation without dense observations or complex pipelines. Extensive experiments validate strong generalization to novel scene configurations.

Conclusion: IGFuse demonstrates effectiveness for real-world 3D reconstruction and real-to-simulation transfer, providing a scalable solution for complete and interactive 3D scene reconstruction.

Abstract: Reconstructing complete and interactive 3D scenes remains a fundamental challenge in computer vision and robotics, particularly due to persistent object occlusions and limited sensor coverage. Multiview observations from a single scene scan often fail to capture the full structural details. Existing approaches typically rely on multi stage pipelines, such as segmentation, background completion, and inpainting or require per-object dense scanning, both of which are error-prone, and not easily scalable. We propose IGFuse, a novel framework that reconstructs interactive Gaussian scene by fusing observations from multiple scans, where natural object rearrangement between captures reveal previously occluded regions. Our method constructs segmentation aware Gaussian fields and enforces bi-directional photometric and semantic consistency across scans. To handle spatial misalignments, we introduce a pseudo-intermediate scene state for unified alignment, alongside collaborative co-pruning strategies to refine geometry. IGFuse enables high fidelity rendering and object level scene manipulation without dense observations or complex pipelines. Extensive experiments validate the framework’s strong generalization to novel scene configurations, demonstrating its effectiveness for real world 3D reconstruction and real-to-simulation transfer. Our project page is available online.

Method: Fine-tunes pretrained video diffusion models using: 1) 4DNeX-10M dataset with high-quality 4D annotations, 2) unified 6D video representation for RGB+XYZ sequences, and 3) effective adaptation strategies for 4D modeling.

Result: Produces high-quality dynamic point clouds enabling novel-view video synthesis, outperforms existing methods in efficiency and generalizability.

Conclusion: 4DNeX offers a scalable solution for image-to-4D modeling and lays foundation for generative 4D world models that simulate dynamic scene evolution.

Abstract: We present 4DNeX, the first feed-forward framework for generating 4D (i.e., dynamic 3D) scene representations from a single image. In contrast to existing methods that rely on computationally intensive optimization or require multi-frame video inputs, 4DNeX enables efficient, end-to-end image-to-4D generation by fine-tuning a pretrained video diffusion model. Specifically, 1) to alleviate the scarcity of 4D data, we construct 4DNeX-10M, a large-scale dataset with high-quality 4D annotations generated using advanced reconstruction approaches. 2) we introduce a unified 6D video representation that jointly models RGB and XYZ sequences, facilitating structured learning of both appearance and geometry. 3) we propose a set of simple yet effective adaptation strategies to repurpose pretrained video diffusion models for 4D modeling. 4DNeX produces high-quality dynamic point clouds that enable novel-view video synthesis. Extensive experiments demonstrate that 4DNeX outperforms existing 4D generation methods in efficiency and generalizability, offering a scalable solution for image-to-4D modeling and laying the foundation for generative 4D world models that simulate dynamic scene evolution.

Method: End-to-end instance segmentation network with encoder-decoder architecture using grid-aware attention gates on skip connections, plus oriented bounding box generation through principal component analysis (PCA) for tooth orientation estimation.

Result: Achieved highest IoU score of 82.43% and DSC score of 90.37% in teeth instance segmentation, and Rotated IoU score of 82.82% in OBB analysis on DNS dataset with 543 panoramic X-ray images.

Conclusion: The proposed model offers accurate and versatile teeth segmentation and orientation with promising applications for improving dental diagnoses, treatment planning, and personalized oral healthcare.

Abstract: Accurate teeth segmentation and orientation are fundamental in modern oral healthcare, enabling precise diagnosis, treatment planning, and dental implant design. In this study, we present a comprehensive approach to teeth segmentation and orientation from panoramic X-ray images, leveraging deep-learning techniques. We built an end-to-end instance segmentation network that uses an encoder-decoder architecture reinforced with grid-aware attention gates along the skip connections. We introduce oriented bounding box (OBB) generation through principal component analysis (PCA) for precise tooth orientation estimation. Evaluating our approach on the publicly available DNS dataset, comprising 543 panoramic X-ray images, we achieve the highest Intersection-over-Union (IoU) score of 82.43% and a Dice Similarity Coefficient (DSC) score of 90.37% among compared models in teeth instance segmentation. In OBB analysis, we obtain a Rotated IoU (RIoU) score of 82.82%. We also conduct detailed analyses of individual tooth labels and categorical performance, shedding light on strengths and weaknesses. The proposed model’s accuracy and versatility offer promising prospects for improving dental diagnoses, treatment planning, and personalized healthcare in the oral domain. Our generated OBB coordinates and code are available at https://github.com/mrinal054/Instance/teeth/segmentation.

Method: A novel evaluation framework combining fine-grained multimodal annotation, cross-modal embedding analysis, and retrieval-augmented assessment. Includes: (i) rigorous annotation protocol linking visual elements to narrative structure, (ii) comprehensive evaluation across multiple reasoning paradigms, and (iii) cross-modal similarity analysis.

Result: Applied to Re:Zero manga across 11 chapters with 308 annotated panels, revealing that current models lack genuine story-level intelligence, struggling with non-linear narratives, character consistency, and causal inference across extended sequences.

Conclusion: This work establishes the foundation and methodology for evaluating narrative intelligence in VLMs, providing insights into deep sequential understanding of visual narratives beyond basic recognition.

Abstract: Current Vision Language Models (VLMs) demonstrate a critical gap between surface-level recognition and deep narrative reasoning when processing sequential visual storytelling. Through a comprehensive investigation of manga narrative understanding, we reveal that while recent large multimodal models excel at individual panel interpretation, they systematically fail at temporal causality and cross-panel cohesion, core requirements for coherent story comprehension. We introduce a novel evaluation framework that combines fine-grained multimodal annotation, cross-modal embedding analysis, and retrieval-augmented assessment to systematically characterize these limitations. Our methodology includes (i) a rigorous annotation protocol linking visual elements to narrative structure through aligned light novel text, (ii) comprehensive evaluation across multiple reasoning paradigms, including direct inference and retrieval-augmented generation, and (iii) cross-modal similarity analysis revealing fundamental misalignments in current VLMs’ joint representations. Applying this framework to Re:Zero manga across 11 chapters with 308 annotated panels, we conduct the first systematic study of long-form narrative understanding in VLMs through three core evaluation axes: generative storytelling, contextual dialogue grounding, and temporal reasoning. Our findings demonstrate that current models lack genuine story-level intelligence, struggling particularly with non-linear narratives, character consistency, and causal inference across extended sequences. This work establishes both the foundation and practical methodology for evaluating narrative intelligence, while providing actionable insights into the capability of deep sequential understanding of Discrete Visual Narratives beyond basic recognition in Multimodal Models. Project Page: https://re-verse.vercel.app

Method: Fine-tunes SAM using automatically generated multi-modal prompts: point prompts from pre-trained task-specific models as visual guidance, and text prompts from large language models as semantic guidance.

Result: Outperforms existing approaches by at least 5% in mIoU for mobility infrastructure segmentation in both familiar and unseen regions.

Conclusion: GeoSAM represents a significant advancement in leveraging foundation models for geographical image segmentation, particularly for mobility infrastructure including roads and pedestrian pathways.

Abstract: In geographical image segmentation, performance is often constrained by the limited availability of training data and a lack of generalizability, particularly for segmenting mobility infrastructure such as roads, sidewalks, and crosswalks. Vision foundation models like the Segment Anything Model (SAM), pre-trained on millions of natural images, have demonstrated impressive zero-shot segmentation performance, providing a potential solution. However, SAM struggles with geographical images, such as aerial and satellite imagery, due to its training being confined to natural images and the narrow features and textures of these objects blending into their surroundings. To address these challenges, we propose Geographical SAM (GeoSAM), a SAM-based framework that fine-tunes SAM using automatically generated multi-modal prompts. Specifically, GeoSAM integrates point prompts from a pre-trained task-specific model as primary visual guidance, and text prompts generated by a large language model as secondary semantic guidance, enabling the model to better capture both spatial structure and contextual meaning. GeoSAM outperforms existing approaches for mobility infrastructure segmentation in both familiar and completely unseen regions by at least 5% in mIoU, representing a significant leap in leveraging foundation models to segment mobility infrastructure, including both road and pedestrian infrastructure in geographical images. The source code can be found in this GitHub Repository: https://github.com/rafiibnsultan/GeoSAM.

Method: Unified Promptable Panoptic Mapping (UPPM) leverages foundation models for dynamic labeling without additional training, evaluated across three levels: Segmentation-to-Map, Map-to-Map, and Segmentation-to-Segmentation. It incorporates unified semantics, custom NMS, and blurry frame filtering.

Result: UPPM achieves exceptional geometry reconstruction accuracy (0.61cm on Flat dataset), highest panoptic quality (0.414), and outperforms state-of-the-art segmentation methods. Custom NMS improves completion ratio by 8.27% on Flat dataset.

Conclusion: UPPM effectively reconstructs scenes with rich semantic labeling across diverse datasets, demonstrating the viability of foundation model-based approaches for dynamic semantic mapping without retraining.

Abstract: In robotics and computer vision, semantic mapping remains a critical challenge for machines to comprehend complex environments. Traditional panoptic mapping approaches are constrained by fixed labels, limiting their ability to handle novel objects. We present Unified Promptable Panoptic Mapping (UPPM), which leverages foundation models for dynamic labeling without additional training. UPPM is evaluated across three comprehensive levels: Segmentation-to-Map, Map-to-Map, and Segmentation-to-Segmentation. Results demonstrate UPPM attains exceptional geometry reconstruction accuracy (0.61cm on the Flat dataset), the highest panoptic quality (0.414), and better performance compared to state-of-the-art segmentation methods. Furthermore, ablation studies validate the contributions of unified semantics, custom NMS, and blurry frame filtering, with the custom NMS improving the completion ratio by 8.27% on the Flat dataset. UPPM demonstrates effective scene reconstruction with rich semantic labeling across diverse datasets.

Method: Uses adaptive clustering self-attention layers that apply attention within local clusters determined by input data, creating hierarchical parsing trees that embed object-phrase relationships.

Result: Outperforms most SOTA captioning and VQA models, achieving comparable performance to large-scale pre-trained models.

Conclusion: ACF effectively learns hierarchical object-phrase alignments through adaptive clustering, demonstrating strong performance in image captioning and visual question answering tasks.

Abstract: We propose a novel Transformer-based image-to-text generation model termed as \textbf{ACF} that adaptively clusters vision patches into object regions and language words into phrases to implicitly learn object-phrase alignments for better visual-text coherence. To achieve this, we design a novel self-attention layer that applies self-attention over the elements in a local cluster window instead of the whole sequence. The window size is softly decided by a clustering matrix that is calculated by the current input data and thus this process is adaptive. By stacking these revised self-attention layers to construct ACF, the small clusters in the lower layers can be grouped into a bigger cluster, \eg vision/language. ACF clusters small objects/phrases into bigger ones. In this gradual clustering process, a parsing tree is generated which embeds the hierarchical knowledge of the input sequence. As a result, by using ACF to build the vision encoder and language decoder, the hierarchical object-phrase alignments are embedded and then transferred from vision to language domains in two popular image-to-text tasks: Image captioning and Visual Question Answering. The experiment results demonstrate the effectiveness of ACF, which outperforms most SOTA captioning and VQA models and achieves comparable scores compared with some large-scale pre-trained models. Our code is available \href{https://github.com/ZihuaEvan/ACFModel/}{[here]}.

Method: Cross-Modal Prototypical Contrastive Learning (CPCL) with CLIP model for shared latent space mapping, Prototypical Multi-modal Memory (PMM) module with Hybrid Cross-modal Matching for many-to-many mapping, and Outlier Pseudo Label Mining (OPLM) module to distinguish valuable outliers.

Result: Extensive experiments on popular benchmarks validate the effectiveness and generalizability of the proposed CPCL method.

Conclusion: CPCL successfully addresses weakly supervised text-based person retrieval challenges by leveraging prototypical features, cross-modal associations, and outlier mining to create more reliable clusters without identity supervision.

Abstract: Weakly supervised text-based person retrieval seeks to retrieve images of a target person using textual descriptions, without relying on identity annotations and is more challenging and practical. The primary challenge is the intra-class differences, encompassing intra-modal feature variations and cross-modal semantic gaps. Prior works have focused on instance-level samples and ignored prototypical features of each person which are intrinsic and invariant. Toward this, we propose a Cross-Modal Prototypical Contrastive Learning (CPCL) method. In practice, the CPCL introduces the CLIP model to weakly supervised text-based person retrieval to map visual and textual instances into a shared latent space. Subsequently, the proposed Prototypical Multi-modal Memory (PMM) module captures associations between heterogeneous modalities of image-text pairs belonging to the same person through the Hybrid Cross-modal Matching (HCM) module in a many-to-many mapping fashion. Moreover, the Outlier Pseudo Label Mining (OPLM) module further distinguishes valuable outlier samples from each modality, enhancing the creation of more reliable clusters by mining implicit relationships between image-text pairs. We conduct extensive experiments on popular benchmarks of weakly supervised text-based person retrieval, which validate the effectiveness, generalizability of CPCL.

Method: Proposes a locally statistical variational active contour model based on I-divergence-TV denoising, hybridizing geodesic active contour and active contours without edges models. Uses reaction-diffusion equation with diffusion term for level set regularization, and develops two fast fixed point algorithms inspired by recent denoising techniques.

Result: Experimental results show the model effectively stops contours at weak/blurred edges and detects interior/exterior boundaries in SAR images with multiplicative gamma noise. The proposed FPRD1/FPRD2 models achieve about 50% (or less) computation time compared to Split Bregman-based SBRD model.

Conclusion: The proposed model successfully addresses SAR image segmentation with multiplicative gamma noise, offering improved edge detection capabilities and significant computational efficiency gains through novel fixed-point algorithms.

Abstract: In this paper, we propose a novel locally statistical variational active contour model based on I-divergence-TV denoising model, which hybrides geodesic active contour (GAC) model with active contours without edges (ACWE) model, and can be used to segment images corrupted by multiplicative gamma noise. By adding a diffusion term into the level set evolution (LSE) equation of the proposed model, we construct a reaction-diffusion (RD) equation, which can gradually regularize the level set function (LSF) to be piecewise constant in each segment domain and gain the stable solution. We further transform the proposed model into classic ROF model by adding a proximity term. [27] is submitted on 29-Aug-2013, and our early edition ever submitted to TGRS on 12-Jun-2012, Venkatakrishnan et al. [31] proposed their ‘pnp algorithm’ on 29-May-2013, so Venkatakrishnan and we proposed the ‘pnp algorithm’ almost simultaneously. Inspired by a fast denoising algorithm proposed by Jia-Zhao recently, we propose two fast fixed point algorithms to solve SAR image segmentation question. Experimental results for real SAR images show that the proposed image segmentation model can efficiently stop the contours at weak or blurred edges, and can automatically detect the exterior and interior boundaries of images with multiplicative gamma noise. The proposed FPRD1/FPRD2 models are about 1/2 (or less than) of the time required for the SBRD model based on the Split Bregman technique.

Method: Zero-shot Visual Question Answering framework using Vision-Language Models (Gemini-1.5-flash and GPT-4o-mini) to classify iRAP road safety attributes, benchmarked against VGGNet and ResNet baselines on a new open-source Thai dataset.

Result: VLMs underperform on spatial awareness tasks but generalize well to unseen classes and offer flexible prompt-based reasoning without retraining. They can serve as automatic road assessment tools when integrated with complementary data.

Conclusion: First exploration of VLMs for zero-shot infrastructure risk assessment, demonstrating potential for automatic, low-cost road safety mapping and opening new directions for this application domain.

Abstract: Road safety assessments are critical yet costly, especially in Low- and Middle-Income Countries (LMICs), where most roads remain unrated. Traditional methods require expert annotation and training data, while supervised learning-based approaches struggle to generalise across regions. In this paper, we introduce \textit{V-RoAst}, a zero-shot Visual Question Answering (VQA) framework using Vision-Language Models (VLMs) to classify road safety attributes defined by the iRAP standard. We introduce the first open-source dataset from ThaiRAP, consisting of over 2,000 curated street-level images from Thailand annotated for this task. We evaluate Gemini-1.5-flash and GPT-4o-mini on this dataset and benchmark their performance against VGGNet and ResNet baselines. While VLMs underperform on spatial awareness, they generalise well to unseen classes and offer flexible prompt-based reasoning without retraining. Our results show that VLMs can serve as automatic road assessment tools when integrated with complementary data. This work is the first to explore VLMs for zero-shot infrastructure risk assessment and opens new directions for automatic, low-cost road safety mapping. Code and dataset: https://github.com/PongNJ/V-RoAst.

Method: Proposed Context Optimization with Multi-Knowledge Representation (CoKnow) framework that uses lightweight semantic knowledge mappers to generate rich contextual knowledge representations for input images without requiring additional priors.

Result: Extensive experiments on 11 publicly available datasets demonstrate that CoKnow outperforms a series of previous methods.

Conclusion: CoKnow effectively enhances prompt learning for vision-language models by incorporating diverse contextual knowledge, addressing the limitations of traditional prompt optimization approaches.

Abstract: Vision-Language Models (VLMs), such as CLIP, play a foundational role in various cross-modal applications. To fully leverage VLMs’ potential in adapting to downstream tasks, context optimization methods like Prompt Tuning are essential. However, one key limitation is the lack of diversity in prompt templates, whether they are hand-crafted or learned through additional modules. This limitation restricts the capabilities of pretrained VLMs and can result in incorrect predictions in downstream tasks. To address this challenge, we propose Context Optimization with Multi-Knowledge Representation (CoKnow), a framework that enhances Prompt Learning for VLMs with rich contextual knowledge. To facilitate CoKnow during inference, we trained lightweight semantic knowledge mappers, which are capable of generating Multi-Knowledge Representation for an input image without requiring additional priors. Experimentally, We conducted extensive experiments on 11 publicly available datasets, demonstrating that CoKnow outperforms a series of previous methods.

Method: Used machine vision and multispectral imaging with CNN and transformer-based architectures on the Eastern England Blackgrass Dataset to evaluate weed recognition performance across different spectral bands and dataset sizes.

Result: All models achieved >80% accuracy, with best model reaching 89.6% accuracy using only half the training data. Different spectral bands significantly impacted classification performance.

Conclusion: Machine vision with multispectral imaging effectively detects blackgrass in cereal crops, offering a promising alternative to herbicide-dependent weed management with potential major environmental and food security benefits.

Abstract: As the burden of herbicide resistance grows and the environmental costs of excessive herbicide use become clear, new approaches to managing weed populations are needed. This is particularly true for cereal crops, like wheat and barley, that are staple foods and occupy a globally significant share of farmland. Even modest advances in weed management practices across these crops could deliver major benefits for both the environment and food security. Blackgrass is a major grass weed which causes particular problems in cereal crops in north-west Europe, a major cereal production area, because it has high levels of herbicide resistance. Detecting blackgrass is also difficult due to its similarity to cereals. Yet, a systematic review of the literature on weed recognition in wheat and barley, included in this study, highlights that blackgrass - and grass weeds more broadly - have received less research attention compared to certain broadleaf weeds. With the use of machine vision and multispectral imaging, we investigate the effectiveness of state-of-the-art methods to identify blackgrass in wheat and barley crops. As part of this work, we present the Eastern England Blackgrass Dataset, a large dataset with which we evaluate several key aspects of blackgrass weed recognition. Firstly, we determine the performance of different CNN and transformer-based architectures on images from unseen fields. Secondly, we demonstrate the role that different spectral bands have on the performance of weed classification. Lastly, we evaluate the role of dataset size in classification performance for each of the models trialled. All models tested achieved an accuracy greater than 80%. Our best model achieved 89.6% and that only half the training data was required to achieve this performance. Our dataset is available at: https://lcas.lincoln.ac.uk/wp/research/data-sets-software/eastern-england-blackgrass-dataset .

Method: Pipeline integrating YOLOv7-based detection, extensive video augmentations, and VideoMAE framework with high-ratio masking and reconstruction strategy to capture spatial and temporal features from the Self-Stimulatory Behavior Dataset (SSBD).

Result: Achieves 95% accuracy, 0.93 precision, 0.94 recall, and 0.94 F1 score, significantly surpassing previous state-of-the-art performance.

Conclusion: Combining advanced object detection, robust data augmentation, and masked autoencoder-based video modeling is highly effective for reliable ASD vs. TD classification in naturalistic settings.

Abstract: Deep learning and contactless sensing technologies have significantly advanced the automated assessment of human behaviors in healthcare. In the context of autism spectrum disorder (ASD), repetitive motor behaviors such as spinning, head banging, and arm flapping are key indicators for diagnosis. This study focuses on distinguishing between children with ASD and typically developed (TD) peers by analyzing videos captured in natural, uncontrolled environments. Using the publicly available Self-Stimulatory Behavior Dataset (SSBD), we address the classification task as a binary problem, ASD vs. TD, based on stereotypical repetitive gestures. We adopt a pipeline integrating YOLOv7-based detection, extensive video augmentations, and the VideoMAE framework, which efficiently captures both spatial and temporal features through a high-ratio masking and reconstruction strategy. Our proposed approach achieves 95% accuracy, 0.93 precision, 0.94 recall, and 0.94 F1 score, surpassing the previous state-of-the-art by a significant margin. These results demonstrate the effectiveness of combining advanced object detection, robust data augmentation, and masked autoencoder-based video modeling for reliable ASD vs. TD classification in naturalistic settings.

Method: CCDMs introduce specially designed conditional diffusion processes, a novel hard vicinal image denoising loss, customized label embedding method, and efficient conditional sampling procedures tailored for continuous conditional generation.

Result: Comprehensive experiments on four datasets with resolutions from 64x64 to 192x192 show CCDMs outperform state-of-the-art CCGM models, establishing new benchmarks. Ablation studies confirm the effectiveness of the proposed components.

Conclusion: CCDMs represent the first CDM specifically designed for continuous conditional generative modeling, successfully addressing limitations of previous approaches and demonstrating superior performance across multiple datasets and resolutions.

Abstract: Continuous Conditional Generative Modeling (CCGM) estimates high-dimensional data distributions, such as images, conditioned on scalar continuous variables (aka regression labels). While Continuous Conditional Generative Adversarial Networks (CcGANs) were designed for this task, their instability during adversarial learning often leads to suboptimal results. Conditional Diffusion Models (CDMs) offer a promising alternative, generating more realistic images, but their diffusion processes, label conditioning, and model fitting procedures are either not optimized for or incompatible with CCGM, making it difficult to integrate CcGANs’ vicinal approach. To address these issues, we introduce Continuous Conditional Diffusion Models (CCDMs), the first CDM specifically tailored for CCGM. CCDMs address existing limitations with specially designed conditional diffusion processes, a novel hard vicinal image denoising loss, a customized label embedding method, and efficient conditional sampling procedures. Through comprehensive experiments on four datasets with resolutions ranging from 64x64 to 192x192, we demonstrate that CCDMs outperform state-of-the-art CCGM models, establishing a new benchmark. Ablation studies further validate the model design and implementation, highlighting that some widely used CDM implementations are ineffective for the CCGM task. Our code is publicly available at https://github.com/UBCDingXin/CCDM.

Method: Learn dense skew-symmetric matrices (Lie algebra elements) and differentially map them to form high-dimensional rotation matrices (Lie group elements) for richer positional encodings.

Result: LieRE demonstrates effectiveness on 2D and 3D vision tasks, generalizing well to higher input resolutions while maintaining computational efficiency.

Conclusion: LieRE provides a principled generalization of RoPE that offers richer, learnable, and continuous encodings for high-dimensional positional information in transformers.

Abstract: Transformer architectures rely on position encodings to model the spatial structure of input data. Rotary Position Encoding (RoPE) is a widely used method in language models that encodes relative positions through fixed, block-diagonal, rotation matrices applied to key-query interactions. We hypothesize that this inductive bias limits their RoPE’s effectiveness for modalities with high dimensional structure. Lie Relative Encodings (LieRE) introduce a principled generalization of RoPE, aimed at increasing the representational capacity of positional encodings in transformers. Instead of fixed 2D rotations, LieRE learns dense skew-symmetric matrices (Lie algebra elements), which are then differentiable mapped to form high-dimensional rotation matrices (Lie group elements). This results in richer, learnable, and continuous, encodings of both relative and absolute positional information. We demonstrate the effectiveness of LieRE on 2D and 3D vision tasks, showing that it generalizes well to higher input resolutions while maintaining computational efficiency. The code and checkpoints are publicly available at https://github.com/StanfordMIMI/LieRE.

Method: Uses representative image extractor (RIE) with deep cluster embedding and hard Gumbel-Softmax to dynamically reduce redundancy and select representative images as graph nodes, with edges computed via cosine similarity.

Result: Achieves state-of-the-art performance on real-world colon cancer dataset and BreakHis dataset, improving both diagnostic accuracy and robustness to redundancy.

Conclusion: MicroMIL successfully addresses the limitations of conventional microscope images for cancer diagnosis, providing an accessible and effective solution without requiring spatial coordinates while maintaining contextual information.

Abstract: Cancer diagnosis has greatly benefited from the integration of whole-slide images (WSIs) with multiple instance learning (MIL), enabling high-resolution analysis of tissue morphology. Graph-based MIL (GNN-MIL) approaches have emerged as powerful solutions for capturing contextual information in WSIs, thereby improving diagnostic accuracy. However, WSIs require significant computational and infrastructural resources, limiting accessibility in resource-constrained settings. Conventional light microscopes offer a cost-effective alternative, but applying GNN-MIL to such data is challenging due to extensive redundant images and missing spatial coordinates, which hinder contextual learning. To address these issues, we introduce MicroMIL, the first weakly-supervised MIL framework specifically designed for images acquired from conventional light microscopes. MicroMIL leverages a representative image extractor (RIE) that employs deep cluster embedding (DCE) and hard Gumbel-Softmax to dynamically reduce redundancy and select representative images. These images serve as graph nodes, with edges computed via cosine similarity, eliminating the need for spatial coordinates while preserving contextual information. Extensive experiments on a real-world colon cancer dataset and the BreakHis dataset demonstrate that MicroMIL achieves state-of-the-art performance, improving both diagnostic accuracy and robustness to redundancy. The code is available at https://github.com/kimjongwoo-cell/MicroMIL

Method: The paper introduces EventHallusion benchmark to assess VideoLLMs’ susceptibility to language priors and vision-language biases. It also proposes Temporal Contrastive Decoding (TCD) which compares original videos with temporally disrupted versions during decoding to rectify model biases.

Result: Evaluation of 10 VideoLLMs (8 open-source, 2 closed-source) shows open-source models suffer significantly from hallucination while closed-source perform better. TCD approach improves performance across most metrics for open-source models.

Conclusion: VideoLLMs have serious hallucination issues, particularly in open-source models. The proposed TCD method effectively mitigates these problems by addressing temporal cue biases during decoding, demonstrating the importance of specialized benchmarks for video hallucination assessment.

Abstract: Recently, Multimodal Large Language Models (MLLMs) have made significant progress in the video comprehension field. Despite remarkable content reasoning and instruction following capabilities they demonstrated, the hallucination problem of these VideoLLMs is less explored compared with its counterpart in the image domain. To mitigate this gap, we propose EventHallusion, a novel benchmark that focuses on assessing the VideoLLMs’ hallucination toward event, the crux of video analysis. From a hallucination attribution perspective, our EventHallusion benchmark is curated to assess a VideoLLM’s susceptibility toward language priors and vision-language biases. On the other hand, we also propose a simple yet effective method, called Temporal Contrastive Decoding (TCD), to tackle the hallucination problems of VideoLLMs. The proposed TCD method rectifies the model’s bias toward its priors during the decoding stage by comparing the original video with a modified version, in which temporal cues are disrupted. Through comprehensive evaluation of eight open-source and two closed-source VideoLLMs on the proposed EventHallusion benchmark, we observe that the open-source models suffer significantly from hallucination problems, whereas the closed-source ones perform markedly better. By further equipping open-source VideoLLMs with the proposed TCD approach, evident performance improvements are achieved across most metrics in the EventHallusion benchmark. Our codes and benchmark data are available at https://github.com/Stevetich/EventHallusion.

Method: Introduces new style ambiguity loss methods that use CLIP embedding centroids instead of classifiers. Two approaches: 1) K-means clustering on unlabeled datasets to find centroids, and 2) using text labels to generate CLIP embeddings as centroids. Images are classified based on distance to these centroids.

Result: Successfully implemented style ambiguity training for diffusion models without requiring pretrained classifiers or labeled datasets. The method works with both clustering-based and text-based centroid generation approaches.

Conclusion: The proposed CLIP-based centroid approach provides an effective alternative to classifier-dependent style ambiguity training, making the technique more accessible and applicable to various scenarios without labeled data requirements.

Abstract: In this work, we explore using the style ambiguity training objective, originally used to approximate creativity, on a diffusion model. However, this objective requires the use of a pretrained classifier and a labeled dataset. We introduce new forms of style ambiguity loss that do not require training a new classifier or a labeled dataset. Instead of using a classifier, we generate centroids in the CLIP embedding space, and images are classified based on their relative distance to said centroids. We find the centroids via K-means clustering of an unlabeled dataset, as well as using text labels to generate CLIP embeddings, to be used as centroids. Code is available at https://github.com/jamesBaker361/clipcreate

Method: Used Segment Anything Model (SAM) for AI-based tool tracking on laparoscopic suturing videos. Extracted kinematic features and applied both supervised (Logistic Regression, Random Forest, SVM, XGBoost) and unsupervised (Denoising Autoencoder with 1-D CNN) learning approaches with PCA for feature reduction.

Result: Supervised learning with PCA and Random Forest achieved 79.5% accuracy and 0.778 F1 score. Unsupervised 1-D CNN performed better with 81.7% accuracy and 0.806 F1 score, eliminating need for kinematic feature computation.

Conclusion: The AI model successfully automates surgical performance classification without human annotation, with unsupervised learning outperforming supervised methods for skill assessment in laparoscopic suturing tasks.

Abstract: Automated assessment of surgical skills using artificial intelligence (AI) provides trainees with instantaneous feedback. After bimanual tool motions are captured, derived kinematic metrics are reliable predictors of performance in laparoscopic tasks. Implementing automated tool tracking requires time-intensive human annotation. We developed AI-based tool tracking using the Segment Anything Model (SAM) to eliminate the need for human annotators. Here, we describe a study evaluating the usefulness of our tool tracking model in automated assessment during a laparoscopic suturing task in the fundoplication procedure. An automated tool tracking model was applied to recorded videos of Nissen fundoplication on porcine bowel. Surgeons were grouped as novices (PGY1-2) and experts (PGY3-5, attendings). The beginning and end of each suturing step were segmented, and motions of the left and right tools were extracted. A low-pass filter with a 24 Hz cut-off frequency removed noise. Performance was assessed using supervised and unsupervised models, and an ablation study compared results. Kinematic features–RMS velocity, RMS acceleration, RMS jerk, total path length, and Bimanual Dexterity–were extracted and analyzed using Logistic Regression, Random Forest, Support Vector Classifier, and XGBoost. PCA was performed for feature reduction. For unsupervised learning, a Denoising Autoencoder (DAE) model with classifiers, such as a 1-D CNN and traditional models, was trained. Data were extracted for 28 participants (9 novices, 19 experts). Supervised learning with PCA and Random Forest achieved an accuracy of 0.795 and an F1 score of 0.778. The unsupervised 1-D CNN achieved superior results with an accuracy of 0.817 and an F1 score of 0.806, eliminating the need for kinematic feature computation. We demonstrated an AI model capable of automated performance classification, independent of human annotation.

Method: CAT embeds triggers during training using conceptual representations. CAT+ enhances this with a correlation function to select optimal stealthy concept triggers.

Result: Both attacks achieve high success rates on backdoored data while maintaining performance on clean data, with CAT+ showing superior stealth and effectiveness.

Conclusion: This work reveals significant security risks in CBMs and provides a robust testing framework for future security assessments of interpretable AI systems.

Abstract: Despite the transformative impact of deep learning across multiple domains, the inherent opacity of these models has driven the development of Explainable Artificial Intelligence (XAI). Among these efforts, Concept Bottleneck Models (CBMs) have emerged as a key approach to improve interpretability by leveraging high-level semantic information. However, CBMs, like other machine learning models, are susceptible to security threats, particularly backdoor attacks, which can covertly manipulate model behaviors. Understanding that the community has not yet studied the concept level backdoor attack of CBM, because of “Better the devil you know than the devil you don’t know.”, we introduce CAT (Concept-level Backdoor ATtacks), a methodology that leverages the conceptual representations within CBMs to embed triggers during training, enabling controlled manipulation of model predictions at inference time. An enhanced attack pattern, CAT+, incorporates a correlation function to systematically select the most effective and stealthy concept triggers, thereby optimizing the attack’s impact. Our comprehensive evaluation framework assesses both the attack success rate and stealthiness, demonstrating that CAT and CAT+ maintain high performance on clean data while achieving significant targeted effects on backdoored datasets. This work underscores the potential security risks associated with CBMs and provides a robust testing methodology for future security assessments.

Method: Replaces static primitives with deformable quadric surfaces and uses geodesic distance-based density distributions that adapt to primitive curvature. Solves geodesic distances in closed form on quadric surfaces for surface-aware splatting.

Result: Reduces geometric error (chamfer distance) by 33% over 2DGS and 27% over GOF on DTU dataset. Maintains competitive appearance quality while improving geometric precision.

Conclusion: QGS bridges the gap between geometric precision and visual fidelity, enabling more efficient and accurate surface reconstruction for applications like robotics and immersive reality.

Abstract: We propose Quadratic Gaussian Splatting (QGS), a novel representation that replaces static primitives with deformable quadric surfaces (e.g., ellipse, paraboloids) to capture intricate geometry. Unlike prior works that rely on Euclidean distance for primitive density modeling–a metric misaligned with surface geometry under deformation–QGS introduces geodesic distance-based density distributions. This innovation ensures that density weights adapt intrinsically to the primitive curvature, preserving consistency during shape changes (e.g., from planar disks to curved paraboloids). By solving geodesic distances in closed form on quadric surfaces, QGS enables surface-aware splatting, where a single primitive can represent complex curvature that previously required dozens of planar surfels, potentially reducing memory usage while maintaining efficient rendering via fast ray-quadric intersection. Experiments on DTU, Tanks and Temples, and MipNeRF360 datasets demonstrate state-of-the-art surface reconstruction, with QGS reducing geometric error (chamfer distance) by 33% over 2DGS and 27% over GOF on the DTU dataset. Crucially, QGS retains competitive appearance quality, bridging the gap between geometric precision and visual fidelity for applications like robotics and immersive reality.

Method: Created OHRBench benchmark with carefully selected unstructured document images and multimodal Q&A pairs. Identified two OCR noise types (semantic and formatting) and applied perturbations to generate structured data with varying noise levels for systematic evaluation.

Result: Comprehensive evaluation revealed no current OCR solution is competent for building high-quality RAG knowledge bases. Demonstrated clear trend relationship between OCR noise degree and RAG performance degradation.

Conclusion: OHRBench provides the first standardized framework to understand and evaluate OCR’s cascading impact on RAG systems, highlighting the need for improved OCR solutions tailored for RAG applications.

Abstract: Retrieval-augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating external knowledge to reduce hallucinations and incorporate up-to-date information without retraining. As an essential part of RAG, external knowledge bases are commonly built by extracting structured data from unstructured PDF documents using Optical Character Recognition (OCR). However, given the imperfect prediction of OCR and the inherent non-uniform representation of structured data, knowledge bases inevitably contain various OCR noises. In this paper, we introduce OHRBench, the first benchmark for understanding the cascading impact of OCR on RAG systems. OHRBench includes 8,561 carefully selected unstructured document images from seven real-world RAG application domains, along with 8,498 Q&A pairs derived from multimodal elements in documents, challenging existing OCR solutions used for RAG. To better understand OCR’s impact on RAG systems, we identify two primary types of OCR noise: Semantic Noise and Formatting Noise and apply perturbation to generate a set of structured data with varying degrees of each OCR noise. Using OHRBench, we first conduct a comprehensive evaluation of current OCR solutions and reveal that none is competent for constructing high-quality knowledge bases for RAG systems. We then systematically evaluate the impact of these two noise types and demonstrate the trend relationship between the degree of OCR noise and RAG performance. Our OHRBench, including PDF documents, Q&As, and the ground truth structured data are released at: https://github.com/opendatalab/OHR-Bench

Method: Uses consistent SAM segmentations through video tracking and low-dimensional indexing for high-dimensional CLIP features to efficiently embed language information in 3D space using 3D Gaussian Splatting.

Result: Outperforms existing methods on two-view sparse 3D object querying and segmentation in LERF and 3D-OVS datasets, achieving scene inference in under 30 seconds and open-vocabulary querying in 0.011 seconds per query.

Conclusion: SLGaussian provides a robust and efficient solution for accurate 3D scene understanding under sparse view conditions, enabling practical real-world applications.

Abstract: 3D semantic field learning is crucial for applications like autonomous navigation, AR/VR, and robotics, where accurate comprehension of 3D scenes from limited viewpoints is essential. Existing methods struggle under sparse view conditions, relying on inefficient per-scene multi-view optimizations, which are impractical for many real-world tasks. To address this, we propose SLGaussian, a feed-forward method for constructing 3D semantic fields from sparse viewpoints, allowing direct inference of 3DGS-based scenes. By ensuring consistent SAM segmentations through video tracking and using low-dimensional indexing for high-dimensional CLIP features, SLGaussian efficiently embeds language information in 3D space, offering a robust solution for accurate 3D scene understanding under sparse view conditions. In experiments on two-view sparse 3D object querying and segmentation in the LERF and 3D-OVS datasets, SLGaussian outperforms existing methods in chosen IoU, Localization Accuracy, and mIoU. Moreover, our model achieves scene inference in under 30 seconds and open-vocabulary querying in just 0.011 seconds per query.

Method: A two-step benchmark process: (1) data clustering with semantics, and (2) data distributing via controllable semantic heterogeneity across clients. Constructed a federated PSG (Panoptic Scene Graph) benchmark as proof of concept.

Result: Successfully demonstrated the efficacy of existing PSG methods in FL settings with controllable semantic heterogeneity. Showed increased performance when applying robust FL algorithms to data heterogeneity.

Conclusion: This is the first benchmark framework enabling FL evaluation for multi-semantic vision tasks under controlled semantic heterogeneity, addressing a critical gap in FL research for complex real-world applications.

Abstract: Federated learning (FL) enables decentralized training while preserving data privacy, yet existing FL benchmarks address relatively simple classification tasks, where each sample is annotated with a one-hot label. However, little attention has been paid to demonstrating an FL benchmark that handles complicated semantics, where each sample encompasses diverse semantic information, such as relations between objects. Because the existing benchmarks are designed to distribute data in a narrow view of a single semantic, managing the complicated \textit{semantic heterogeneity} across clients when formalizing FL benchmarks is non-trivial. In this paper, we propose a benchmark process to establish an FL benchmark with controllable semantic heterogeneity across clients: two key steps are (i) data clustering with semantics and (ii) data distributing via controllable semantic heterogeneity across clients. As a proof of concept, we construct a federated PSG benchmark, demonstrating the efficacy of the existing PSG methods in an FL setting with controllable semantic heterogeneity of scene graphs. We also present the effectiveness of our benchmark by applying robust federated learning algorithms to data heterogeneity to show increased performance. To our knowledge, this is the first benchmark framework that enables federated learning and its evaluation for multi-semantic vision tasks under the controlled semantic heterogeneity. Our code is available at \textit{https://github.com/Seung-B/FL-PSG}.

Method: Uses Learned Clip Assembly (LCA) score with contrastive learning (Shot Coherence Learning) and feature regression. Employs beam-search algorithm for efficient shot combination. Offers two variants: visual-only SKALD and text-enhanced SKALD-text.

Result: Achieves 48.6% improvement in IoU and 43% speedup over state-of-the-art methods. User study shows 45% preference for SKALD vs 22% for text-based methods on VSPD and MSV3C datasets.

Conclusion: SKALD demonstrates effective video assembly with minimal text reliance, superior coherence metrics, and faster performance compared to existing approaches, validated by both quantitative metrics and user preference.

Abstract: We present SKALD, a multi-shot video assembly method that constructs coherent video sequences from candidate shots with minimal reliance on text. Central to our approach is the Learned Clip Assembly (LCA) score, a learning-based metric that measures temporal and semantic relationships between shots to quantify narrative coherence. We tackle the exponential complexity of combining multiple shots with an efficient beam-search algorithm guided by the LCA score. To train our model effectively with limited human annotations, we propose two tasks for the LCA encoder: Shot Coherence Learning, which uses contrastive learning to distinguish coherent and incoherent sequences, and Feature Regression, which converts these learned representations into a real-valued coherence score. We develop two variants: a base SKALD model that relies solely on visual coherence and SKALD-text, which integrates auxiliary text information when available. Experiments on the VSPD and our curated MSV3C datasets show that SKALD achieves an improvement of up to 48.6% in IoU and a 43% speedup over the state-of-the-art methods. A user study further validates our approach, with 45% of participants favoring SKALD-assembled videos, compared to 22% preferring text-based assembly methods.

Method: Propose two techniques: 1) Attention Dynamic Reallocation (ADR) to dynamically reallocate appropriate attention based on original attention, mitigating parameter harmfulness; 2) Parameter Orthogonal Generation (POG) to learn orthogonal basis embeddings of parameters and prevent degradation to static parameters, mitigating parameter uselessness.

Result: Experiments validate the effectiveness of the proposed techniques in improving low-light image enhancement performance while reducing model redundancy.

Conclusion: The paper successfully identifies and addresses model redundancy issues in LLIE through innovative techniques that improve performance, with code to be released publicly.

Abstract: Low-light image enhancement (LLIE) is a fundamental task in computational photography, aiming to improve illumination, reduce noise, and enhance the image quality of low-light images. While recent advancements primarily focus on customizing complex neural network models, we have observed significant redundancy in these models, limiting further performance improvement. In this paper, we investigate and rethink the model redundancy for LLIE, identifying parameter harmfulness and parameter uselessness. Inspired by the rethinking, we propose two innovative techniques to mitigate model redundancy while improving the LLIE performance: Attention Dynamic Reallocation (ADR) and Parameter Orthogonal Generation (POG). ADR dynamically reallocates appropriate attention based on original attention, thereby mitigating parameter harmfulness. POG learns orthogonal basis embeddings of parameters and prevents degradation to static parameters, thereby mitigating parameter uselessness. Experiments validate the effectiveness of our techniques. We will release the code to the public.

Method: Proposes a one-shot learning framework with: 1) Synthetic data augmentation pipeline for identity-preserving positive samples and hard negatives, 2) Mixture-of-Heads enhanced ViLLM with ReLU Routing attention, 3) Two novel objectives (Smooth Proximity Regularization and Head Activation Enhancement), 4) Two-stage training from image pre-training to video fine-tuning.

Result: PVCach demonstrates superior performance in personalized feature understanding after learning from a single video across diverse datasets including medical scenarios, TV series, anime, and real-world footage, outperforming state-of-the-art ViLLMs.

Conclusion: The proposed PVChat framework successfully addresses the identity-aware comprehension limitation in video large language models through innovative one-shot learning, synthetic data augmentation, and specialized attention mechanisms, enabling practical applications in personalized video understanding scenarios.

Abstract: Video large language models (ViLLMs) excel in general video understanding, e.g., recognizing activities like talking and eating, but struggle with identity-aware comprehension, such as “Wilson is receiving chemotherapy” or “Tom is discussing with Sarah”, limiting their applicability in smart healthcare and smart home environments. To address this limitation, we propose a one-shot learning framework PVChat, the first personalized ViLLM that enables subject-aware question answering (QA) from a single video for each subject. Our approach optimizes a Mixture-of-Heads (MoH) enhanced ViLLM on a synthetically augmented video-QA dataset, leveraging a progressive image-to-video learning strategy. Specifically, we introduce an automated augmentation pipeline that synthesizes identity-preserving positive samples and retrieves hard negatives from existing video corpora, generating a diverse training dataset with four QA types: existence, appearance, action, and location inquiries. To enhance subject-specific learning, we propose a ReLU Routing MoH attention mechanism, alongside two novel objectives: (1) Smooth Proximity Regularization for progressive learning through exponential distance scaling and (2) Head Activation Enhancement for balanced attention routing. Finally, we adopt a two-stage training strategy, transitioning from image pre-training to video fine-tuning, enabling a gradual learning process from static attributes to dynamic representations. We evaluate PVChat on diverse datasets covering medical scenarios, TV series, anime, and real-world footage, demonstrating its superiority in personalized feature understanding after learning from a single video, compared to state-of-the-art ViLLMs.

Method: Created the first Embodied Preference Database (EPD) with 12,500 distorted image annotations, established robot-specific assessment metrics based on downstream tasks, and proposed MA-EIQA - a multi-scale attention no-reference IQA model specifically designed for embodied robots.

Result: Experiments demonstrated that quality assessment of embodied images differs significantly from human perception, and the proposed MA-EIQA model showed effective performance on the EPD dataset.

Conclusion: The EPD database and MA-EIQA model provide foundational tools for embodied AI development, highlighting the distinct nature of robot image quality assessment compared to human perception, with potential to improve robotic task performance through better visual input quality evaluation.

Abstract: Image Quality Assessment (IQA) of User-Generated Content (UGC) is a critical technique for human Quality of Experience (QoE). However, does the the image quality of Robot-Generated Content (RGC) demonstrate traits consistent with the Moravec paradox, potentially conflicting with human perceptual norms? Human subjective scoring is more based on the attractiveness of the image. Embodied agent are required to interact and perceive in the environment, and finally perform specific tasks. Visual images as inputs directly influence downstream tasks. In this paper, we explore the perception mechanism of embodied robots for image quality. We propose the first Embodied Preference Database (EPD), which contains 12,500 distorted image annotations. We establish assessment metrics based on the downstream tasks of robot. In addition, there is a gap between UGC and RGC. To address this, we propose a novel Multi-scale Attention Embodied Image Quality Assessment called MA-EIQA. For the proposed EPD dataset, this is the first no-reference IQA model designed for embodied robot. Finally, the performance of mainstream IQA algorithms on EPD dataset is verified. The experiments demonstrate that quality assessment of embodied images is different from that of humans. We sincerely hope that the EPD can contribute to the development of embodied AI by focusing on image quality assessment. The benchmark is available at https://github.com/Jianbo-maker/EPD_benchmark.

Method: Uses modality-specific graph neural network with Difference Attention Module to extract temporal-spatial embeddings from head motion, eye tracking, and physiological data. Includes cross-modal alignment to train video encoder for personalized prediction using only video input during inference.

Result: Achieves 88.4% accuracy (close to EEG-based 89.16%), with 90ms average inference latency enabling real-time applications on consumer-grade VR platforms.

Conclusion: The framework provides accurate, personalized cybersickness prediction without specialized hardware, making it practical for widespread VR adoption while maintaining performance comparable to invasive methods.

Abstract: Cybersickness remains a major obstacle to the widespread adoption of immersive virtual reality (VR), particularly in consumer-grade environments. While prior methods rely on invasive signals such as electroencephalography (EEG) for high predictive accuracy, these approaches require specialized hardware and are impractical for real-world applications. In this work, we propose a scalable, deployable framework for personalized cybersickness prediction leveraging only non-invasive signals readily available from commercial VR headsets, including head motion, eye tracking, and physiological responses. Our model employs a modality-specific graph neural network enhanced with a Difference Attention Module to extract temporal-spatial embeddings capturing dynamic changes across modalities. A cross-modal alignment module jointly trains the video encoder to learn personalized traits by aligning video features with sensor-derived representations. Consequently, the model accurately predicts individual cybersickness using only video input during inference. Experimental results show our model achieves 88.4% accuracy, closely matching EEG-based approaches (89.16%), while reducing deployment complexity. With an average inference latency of 90ms, our framework supports real-time applications, ideal for integration into consumer-grade VR platforms without compromising personalization or performance. The code will be relesed at https://github.com/U235-Aurora/PTGNN.

Method: CPL-BC maintains two identical models with different initial parameters that collaborate to select easy samples (confidence above threshold) for training. The threshold is gradually lowered during training to progressively include harder samples. Models are pre-trained first to assess sample difficulty.

Result: Experimental results on two flying bird object datasets show CPL-BC significantly improves detection accuracy compared to other learning strategies.

Conclusion: The proposed CPL-BC strategy effectively enhances flying bird object detection in surveillance videos by progressively training from easy to hard samples, demonstrating both effectiveness and advancement over existing methods.

Abstract: The flying bird objects captured by surveillance cameras exhibit varying levels of recognition difficulty due to factors such as their varying sizes or degrees of similarity to the background. To alleviate the negative impact of hard samples on training the Flying Bird Object Detection (FBOD) model for surveillance videos, we propose the Co-Paced Learning strategy Based on Confidence (CPL-BC) and apply it to the training process of the FBOD model. This strategy involves maintaining two models with identical structures but different initial parameter configurations that collaborate with each other to select easy samples for training, where the prediction confidence exceeds a set threshold. As training progresses, the strategy gradually lowers the threshold, thereby gradually enhancing the model’s ability to recognize objects, from easier to more hard ones. Prior to applying CPL-BC, we pre-trained the two FBOD models to equip them with the capability to assess the difficulty of flying bird object samples. Experimental results on two different datasets of flying bird objects in surveillance videos demonstrate that, compared to other model learning strategies, CPL-BC significantly improves detection accuracy, thereby verifying the method’s effectiveness and advancement.

Method: Uses Local Geometry-Aware Distillation (LGAD) with multi-view normal-depth diffusion priors to complete partial geometries, view-adaptive guidance scales for smooth semantic transitions, and physically based rendering for appearance modeling.

Result: Generates meshes with well-structured, intricately detailed geometries, coherent textures, and smooth transitions, creating visually appealing 3D shape designs.

Conclusion: The proposed method successfully creates 3D models that are consistent with given text semantics across different views, enabling more creative 3D modeling exploration.

Abstract: Existing 3D reconstruction methods utilize guidances such as 2D images, 3D point clouds, shape contours and single semantics to recover the 3D surface, which limits the creative exploration of 3D modeling. In this paper, we propose a novel 3D modeling task called ``Shape from Semantics’’, which aims to create 3D models whose geometry and appearance are consistent with the given text semantics when viewed from different views. The reconstructed 3D models incorporate more than one semantic elements and are easy for observers to distinguish. We adopt generative models as priors and disentangle the connection between geometry and appearance to solve this challenging problem. Specifically, we propose Local Geometry-Aware Distillation (LGAD), a strategy that employs multi-view normal-depth diffusion priors to complete partial geometries, ensuring realistic shape generation. We also integrate view-adaptive guidance scales to enable smooth semantic transitions across views. For appearance modeling, we adopt physically based rendering to generate high-quality material properties, which are subsequently baked into fabricable meshes. Extensive experimental results demonstrate that our method can generate meshes with well-structured, intricately detailed geometries, coherent textures, and smooth transitions, resulting in visually appealing 3D shape designs. Project page: https://shapefromsemantics.github.io

Method: Decomposes 1-D causal self-attention into visual-to-visual, textual-to-visual, and textual-to-textual attentions, with visual and textual outputs merged using α-weighting strategy. Enables two key improvements: rectifying biased positional encoding in textual-to-visual attention and diagonalizing visual-to-visual attention to reduce computation.

Result: Significant improvements on multiple image benchmarks while reducing computational costs (5x faster), validating the effectiveness of the decomposed attention approach.

Conclusion: D-Attn provides a more flexible attention architecture for LVLMs that enables optimization of visual processing while preserving pre-trained language model capabilities, achieving better performance with improved efficiency.

Abstract: Large vision-and-language models (LVLMs) have traditionally integrated visual and textual tokens by concatenating them into a single homogeneous input for large language models (LLMs), thereby maximally preserving the pre-trained language capabilities. However, this constrained architecture for visual and textual tokens restricts the design space for processing visual tokens, potentially leading to suboptimal performance and efficiency. In this paper, we propose Decomposed Attention (D-Attn), a more flexible attention architecture for LVLMs, which enables modification of visual token operations without affecting textual-to-textual attention. D-Attn decomposes the 1-D causal self-attention of LVLMs into visual-to-visual, textual-to-visual, and textual-to-textual attentions, and the visual and textual output tokens from the decomposed attentions are merged with a carefully derived weighting strategy, namely $\alpha$-weighting. Taking advantage of the flexibility, we are able to introduce two critical improvements in visual token processing while maintaining the capacity of pre-trained LLMs: 1) We rectify the biased positional encoding in textual-to-visual attention to boost visual understanding performance. 2) We diagonalize visual-to-visual attention to reduce computation complexity from $O(|V|^2)$ to $O(|V|)$ for $|V|$ visual tokens without compromising performance. Extensive experiments and analysis validate the effectiveness of D-Attn, demonstrating significant improvements on multiple image benchmarks while significantly reducing computational costs (\eg, $5\times$ faster). Code will be available at https://github.com/bytedance/DecomposedAttention.

Method: Proposes Sketch-1-to-3 framework with GCTD module for geometric contour and texture detail extraction. Uses deep learning architecture with domain adaptation module and tailored loss function to align sketches with 3D facial space. Also creates two datasets: SketchFaces (real hand-drawn) and Syn-SketchFaces (synthetic).

Result: Extensive experiments demonstrate state-of-the-art performance in sketch-based 3D face reconstruction, achieving high-fidelity expression and texture reconstruction.

Conclusion: The proposed framework successfully addresses the challenges of 3D face reconstruction from single sketches and achieves superior performance, with created datasets facilitating future research in this domain.

Abstract: 3D face reconstruction from a single sketch is a critical yet underexplored task with significant practical applications. The primary challenges stem from the substantial modality gap between 2D sketches and 3D facial structures, including: (1) accurately extracting facial keypoints from 2D sketches; (2) preserving diverse facial expressions and fine-grained texture details; and (3) training a high-performing model with limited data. In this paper, we propose Sketch-1-to-3, a novel framework for realistic 3D face reconstruction from a single sketch, to address these challenges. Specifically, we first introduce the Geometric Contour and Texture Detail (GCTD) module, which enhances the extraction of geometric contours and texture details from facial sketches. Additionally, we design a deep learning architecture with a domain adaptation module and a tailored loss function to align sketches with the 3D facial space, enabling high-fidelity expression and texture reconstruction. To facilitate evaluation and further research, we construct SketchFaces, a real hand-drawn facial sketch dataset, and Syn-SketchFaces, a synthetic facial sketch dataset. Extensive experiments demonstrate that Sketch-1-to-3 achieves state-of-the-art performance in sketch-based 3D face reconstruction.

Method: End-to-end pipeline that first resolves scan alignment ambiguities using SE(3) canonicalization with viewpoint prediction, then completes missing geometry through an attention-based network trained on synthetically augmented point clouds.

Result: Achieves state-of-the-art performance on reconstruction metrics while preserving clinically validated anatomical fidelity. Enables robust foot reconstruction under real-world capture conditions.

Conclusion: Combining synthetic training data with learned geometric priors unlocks new opportunities for mobile-based 3D scanning in healthcare and retail applications.

Abstract: Accurate 3D foot reconstruction is crucial for personalized orthotics, digital healthcare, and virtual fittings. However, existing methods struggle with incomplete scans and anatomical variations, particularly in self-scanning scenarios where user mobility is limited, making it difficult to capture areas like the arch and heel. We present a novel end-to-end pipeline that refines Structure-from-Motion (SfM) reconstruction. It first resolves scan alignment ambiguities using SE(3) canonicalization with a viewpoint prediction module, then completes missing geometry through an attention-based network trained on synthetically augmented point clouds. Our approach achieves state-of-the-art performance on reconstruction metrics while preserving clinically validated anatomical fidelity. By combining synthetic training data with learned geometric priors, we enable robust foot reconstruction under real-world capture conditions, unlocking new opportunities for mobile-based 3D scanning in healthcare and retail.

Method: Created TopoMortar dataset with three label types (accurate, pseudo-labels, noisy labels), two fixed training sets (large/small), and in/out-of-distribution test images. Evaluated eight loss functions including clDice and Cross entropy Dice with data augmentation and self-distillation.

Result: clDice achieved the most topologically accurate segmentations. The relative performance of other loss functions varied by experimental setting. Data augmentation and self-distillation enabled Cross entropy Dice to surpass most topology loss functions and also enhanced topology loss functions.

Conclusion: TopoMortar enables proper evaluation of topology-focused segmentation methods by eliminating dataset challenges. Simple techniques like data augmentation and self-distillation can significantly improve topology accuracy, with clDice performing best overall.

Abstract: We present TopoMortar, a brick wall dataset that is the first dataset specifically designed to evaluate topology-focused image segmentation methods, such as topology loss functions. Motivated by the known sensitivity of methods to dataset challenges, such as small training sets, noisy labels, and out-of-distribution test-set images, TopoMortar is created to enable in two ways investigating methods’ effectiveness at improving topology accuracy. First, by eliminating dataset challenges that, as we show, impact the effectiveness of topology loss functions. Second, by allowing to represent different dataset challenges in the same dataset, isolating methods’ performance from dataset challenges. TopoMortar includes three types of labels (accurate, pseudo-labels, and noisy labels), two fixed training sets (large and small), and in-distribution and out-of-distribution test-set images. We compared eight loss functions on TopoMortar, and we found that clDice achieved the most topologically accurate segmentations, and that the relative advantageousness of the other loss functions depends on the experimental setting. Additionally, we show that data augmentation and self-distillation can elevate Cross entropy Dice loss to surpass most topology loss functions, and that those simple methods can enhance topology loss functions as well. TopoMortar and our code can be found at https://jmlipman.github.io/TopoMortar

Method: Proposes STORM architecture with a dedicated temporal encoder between image encoder and LLM, using Mamba State Space Model to integrate temporal information and enable token reduction strategies including test-time sampling and training-based temporal/spatial pooling.

Result: Achieves state-of-the-art results (more than 5% improvement on MLVU and LongVideoBench) while reducing computation costs by up to 8x and decoding latency by 2.4-2.9x for fixed input frames.

Conclusion: STORM enables efficient and robust video understanding over extended temporal contexts by simultaneously improving performance while reducing training and inference latency through spatiotemporal token reduction.

Abstract: Recent advances in video-based multimodal large language models (Video-LLMs) have significantly improved video understanding by processing videos as sequences of image frames. However, many existing methods treat frames independently in the vision backbone, lacking explicit temporal modeling, which limits their ability to capture dynamic patterns and efficiently handle long videos. To address these limitations, we introduce STORM (Spatiotemporal TOken Reduction for Multimodal LLMs), a novel architecture incorporating a dedicated temporal encoder between the image encoder and the LLM. Our temporal encoder leverages the Mamba State Space Model to integrate temporal information into image tokens, generating enriched representations that preserve inter-frame dynamics across the entire video sequence. This enriched encoding not only enhances video reasoning capabilities but also enables effective token reduction strategies, including test-time sampling and training-based temporal and spatial pooling, substantially reducing computational demands on the LLM without sacrificing key temporal information. By integrating these techniques, our approach simultaneously reduces training and inference latency while improving performance, enabling efficient and robust video understanding over extended temporal contexts. Extensive evaluations show that STORM achieves state-of-the-art results across various long video understanding benchmarks (more than 5% improvement on MLVU and LongVideoBench) while reducing the computation costs by up to $8\times$ and the decoding latency by 2.4-2.9$\times$ for the fixed numbers of input frames. Project page is available at https://research.nvidia.com/labs/lpr/storm

Method: Uses iterative point-wise alignment in a common coordinate system based on state space models (SSMs). RGB and Points SSMs capture long-range dependencies and spatial information from single views with linear complexity.

Result: Achieves on-par performance with CAD-based and dense reference view methods on six datasets and real-world robotic scenes, despite using only single reference views.

Conclusion: SinRef-6D provides a scalable solution for novel object 6D pose estimation that works with single reference views without CAD models or retraining, making it practical for real-world applications.

Abstract: Existing novel object 6D pose estimation methods typically rely on CAD models or dense reference views, which are both difficult to acquire. Using only a single reference view is more scalable, but challenging due to large pose discrepancies and limited geometric and spatial information. To address these issues, we propose a Single-Reference-based novel object 6D (SinRef-6D) pose estimation method. Our key idea is to iteratively establish point-wise alignment in a common coordinate system based on state space models (SSMs). Specifically, iterative object-space point-wise alignment can effectively handle large pose discrepancies, while our proposed RGB and Points SSMs can capture long-range dependencies and spatial information from a single view, offering linear complexity and superior spatial modeling capability. Once pre-trained on synthetic data, SinRef-6D can estimate the 6D pose of a novel object using only a single reference view, without requiring retraining or a CAD model. Extensive experiments on six popular datasets and real-world robotic scenes demonstrate that we achieve on-par performance with CAD-based and dense reference view-based methods, despite operating in the more challenging single reference setting. Code will be released at https://github.com/CNJianLiu/SinRef-6D.

Method: Proposes a patch-level adaptive condition selection mechanism that dynamically prioritizes spatially relevant control signals at sub-region level, combined with a time-aware control injection scheme that modulates condition influence according to denoising timesteps.

Result: Extensive experiments show PixelPonder surpasses previous methods across benchmark datasets, demonstrating superior spatial alignment accuracy while maintaining high textual semantic consistency.

Conclusion: PixelPonder provides an effective unified control framework for multiple visual conditions that enables precise local guidance without global interference, promoting more harmonious image generation in diffusion models.

Abstract: Recent advances in diffusion-based text-to-image generation have demonstrated promising results through visual condition control. However, existing ControlNet-like methods struggle with compositional visual conditioning - simultaneously preserving semantic fidelity across multiple heterogeneous control signals while maintaining high visual quality, where they employ separate control branches that often introduce conflicting guidance during the denoising process, leading to structural distortions and artifacts in generated images. To address this issue, we present PixelPonder, a novel unified control framework, which allows for effective control of multiple visual conditions under a single control structure. Specifically, we design a patch-level adaptive condition selection mechanism that dynamically prioritizes spatially relevant control signals at the sub-region level, enabling precise local guidance without global interference. Additionally, a time-aware control injection scheme is deployed to modulate condition influence according to denoising timesteps, progressively transitioning from structural preservation to texture refinement and fully utilizing the control information from different categories to promote more harmonious image generation. Extensive experiments demonstrate that PixelPonder surpasses previous methods across different benchmark datasets, showing superior improvement in spatial alignment accuracy while maintaining high textual semantic consistency.

Method: Integrates 2D visual-language model features (SAM and CLIP) into 3D scenes using normalized weighted average of 3D Gaussian parameters instead of loss functions, with vector database for efficient storage/retrieval.

Result: Achieves 18x speedup in embedding computation on 16-GPU setup compared to OpenGaussian while preserving embedding quality on ScanNet and LERF datasets.

Conclusion: SLAG provides a highly parallelized and scalable solution for efficient language-augmented scene representation that addresses computational constraints in robotics applications.

Abstract: Language-augmented scene representations hold great promise for large-scale robotics applications such as search-and-rescue, smart cities, and mining. Many of these scenarios are time-sensitive, requiring rapid scene encoding while also being data-intensive, necessitating scalable solutions. Deploying these representations on robots with limited computational resources further adds to the challenge. To address this, we introduce SLAG, a multi-GPU framework for language-augmented Gaussian splatting that enhances the speed and scalability of embedding large scenes. Our method integrates 2D visual-language model features into 3D scenes using SAM and CLIP. Unlike prior approaches, SLAG eliminates the need for a loss function to compute per-Gaussian language embeddings. Instead, it derives embeddings from 3D Gaussian scene parameters via a normalized weighted average, enabling highly parallelized scene encoding. Additionally, we introduce a vector database for efficient embedding storage and retrieval. Our experiments show that SLAG achieves an 18 times speedup in embedding computation on a 16-GPU setup compared to OpenGaussian, while preserving embedding quality on the ScanNet and LERF datasets. For more details, visit our project website: https://slag-project.github.io/.

Method: MambaFlow consists of two key components: (1) PolyMamba - a dual-Mamba architecture with Self-Mamba for intra-token modeling and Cross-Mamba for inter-modality interaction, enabling deep contextualization and feature fusion; (2) PulseMamba - uses Attention Guidance Aggregator (AGA) for adaptive feature integration and employs Mamba’s recurrent mechanism for autoregressive flow decoding.

Result: Extensive experiments show MambaFlow achieves remarkable results comparable to mainstream methods on benchmark datasets. It attains higher accuracy than SEA-RAFT on the Sintel benchmark, demonstrating strong potential for real-world deployment on resource-constrained devices.

Conclusion: MambaFlow successfully adapts the Mamba architecture for optical flow estimation, providing an efficient and accurate solution that outperforms existing methods while being suitable for deployment on devices with limited computational resources.

Abstract: Recently, the Mamba architecture has demonstrated significant successes in various computer vision tasks, such as classification and segmentation. However, its application to optical flow estimation remains unexplored. In this paper, we introduce MambaFlow, a novel framework designed to leverage the high accuracy and efficiency of the Mamba architecture for capturing locally correlated features while preserving global information in end-to-end optical flow estimation. To our knowledge, MambaFlow is the first architecture centered around the Mamba design tailored specifically for optical flow estimation. It comprises two key components: (1) PolyMamba, which enhances feature representation through a dual-Mamba architecture, incorporating a Self-Mamba module for intra-token modeling and a Cross-Mamba module for inter-modality interaction, enabling both deep contextualization and effective feature fusion; and (2) PulseMamba, which leverages an Attention Guidance Aggregator (AGA) to adaptively integrate features with dynamically learned weights in contrast to naive concatenation, and then employs the intrinsic recurrent mechanism of Mamba to perform autoregressive flow decoding, facilitating efficient flow information dissemination. Extensive experiments demonstrate that MambaFlow achieves remarkable results comparable to mainstream methods on benchmark datasets. Compared to SEA-RAFT, MambaFlow attains higher accuracy on the Sintel benchmark, demonstrating stronger potential for real-world deployment on resource-constrained devices. The source code will be made publicly available upon acceptance of the paper.

Method: AFR-CLIP performs image-guided textual rectification to embed defect information into stateless prompts, then compares enriched textual embeddings with pre-defined normal/abnormal stateful embeddings. Includes self prompting and multi-patch feature aggregation modules for multi-scale feature perception.

Result: Extensive experiments on 11 anomaly detection benchmarks across industrial and medical domains demonstrate AFR-CLIP’s superiority in zero-shot anomaly detection.

Conclusion: AFR-CLIP effectively addresses CLIP’s limitation in anomaly detection by rectifying anomaly features through image-guided textual enrichment and stateful embedding comparison, achieving state-of-the-art performance.

Abstract: Recently, zero-shot anomaly detection (ZSAD) has emerged as a pivotal paradigm for industrial inspection and medical diagnostics, detecting defects in novel objects without requiring any target-dataset samples during training. Existing CLIP-based ZSAD methods generate anomaly maps by measuring the cosine similarity between visual and textual features. However, CLIP’s alignment with object categories instead of their anomalous states limits its effectiveness for anomaly detection. To address this limitation, we propose AFR-CLIP, a CLIP-based anomaly feature rectification framework. AFR-CLIP first performs image-guided textual rectification, embedding the implicit defect information from the image into a stateless prompt that describes the object category without indicating any anomalous state. The enriched textual embeddings are then compared with two pre-defined stateful (normal or abnormal) embeddings, and their text-on-text similarity yields the anomaly map that highlights defective regions. To further enhance perception to multi-scale features and complex anomalies, we introduce self prompting (SP) and multi-patch feature aggregation (MPFA) modules. Extensive experiments are conducted on eleven anomaly detection benchmarks across industrial and medical domains, demonstrating AFR-CLIP’s superiority in ZSAD.

Method: Systematically deconstructs image generation into 4 task categories and 31 fine-grained tasks, uses 6 evaluation dimensions with 11 metrics including novel VLLM-QA, and employs hybrid data from real scenes and virtual generation.

Result: The benchmark reveals challenging nature of comprehensive evaluation and identifies gaps between current model capabilities and real-world generation requirements.

Conclusion: ICE-Bench provides a valuable open-source resource for the research community to foster advancements in image generation model evaluation.

Abstract: Image generation has witnessed significant advancements in the past few years. However, evaluating the performance of image generation models remains a formidable challenge. In this paper, we propose ICE-Bench, a unified and comprehensive benchmark designed to rigorously assess image generation models. Its comprehensiveness could be summarized in the following key features: (1) Coarse-to-Fine Tasks: We systematically deconstruct image generation into four task categories: No-ref/Ref Image Creating/Editing, based on the presence or absence of source images and reference images. And further decompose them into 31 fine-grained tasks covering a broad spectrum of image generation requirements, culminating in a comprehensive benchmark. (2) Multi-dimensional Metrics: The evaluation framework assesses image generation capabilities across 6 dimensions: aesthetic quality, imaging quality, prompt following, source consistency, reference consistency, and controllability. 11 metrics are introduced to support the multi-dimensional evaluation. Notably, we introduce VLLM-QA, an innovative metric designed to assess the success of image editing by leveraging large models. (3) Hybrid Data: The data comes from real scenes and virtual generation, which effectively improves data diversity and alleviates the bias problem in model evaluation. Through ICE-Bench, we conduct a thorough analysis of existing generation models, revealing both the challenging nature of our benchmark and the gap between current model capabilities and real-world generation requirements. To foster further advancements in the field, we will open-source ICE-Bench, including its dataset, evaluation code, and models, thereby providing a valuable resource for the research community.

Method: A multi-agent system parses prompts into semantic scene graphs to generate atomic questions, then uses a knowledge-augmented multi-stage reasoning framework where an auxiliary LLM retrieves relevant common-sense knowledge, and a video LLM answers questions through multi-stage reasoning.

Result: ETVA achieves a Spearman’s correlation coefficient of 58.47, significantly outperforming existing metrics (31.0). The authors also constructed a comprehensive benchmark with 2k diverse prompts and 12k atomic questions across 10 categories, and systematically evaluated 15 existing T2V models.

Conclusion: ETVA provides a more accurate and fine-grained evaluation of text-to-video alignment that better correlates with human judgment, identifying key capabilities and limitations of existing T2V models and paving the way for next-generation T2V generation.

Abstract: Precisely evaluating semantic alignment between text prompts and generated videos remains a challenge in Text-to-Video (T2V) Generation. Existing text-to-video alignment metrics like CLIPScore only generate coarse-grained scores without fine-grained alignment details, failing to align with human preference. To address this limitation, we propose ETVA, a novel Evaluation method of Text-to-Video Alignment via fine-grained question generation and answering. First, a multi-agent system parses prompts into semantic scene graphs to generate atomic questions. Then we design a knowledge-augmented multi-stage reasoning framework for question answering, where an auxiliary LLM first retrieves relevant common-sense knowledge (e.g., physical laws), and then video LLM answers the generated questions through a multi-stage reasoning mechanism. Extensive experiments demonstrate that ETVA achieves a Spearman’s correlation coefficient of 58.47, showing a much higher correlation with human judgment than existing metrics which attain only 31.0. We also construct a comprehensive benchmark specifically designed for text-to-video alignment evaluation, featuring 2k diverse prompts and 12k atomic questions spanning 10 categories. Through a systematic evaluation of 15 existing text-to-video models, we identify their key capabilities and limitations, paving the way for next-generation T2V generation.

Method: Skeleton-guided silhouette alignment strategy that uses prior knowledge of skeletons to perform affine transformations on corresponding silhouettes, addressing data misalignment issues in wild environments.

Result: Achieved 7.9% average performance improvement across all evaluated networks on Gait3D dataset and substantial cross-domain improvements with accuracy gains up to 24.0%. Extensive experiments across multiple datasets and architectures demonstrated significant advantages.

Conclusion: This is the first study to explore data alignment impact on gait recognition. The proposed skeleton-guided alignment strategy effectively addresses wild dataset challenges and significantly improves gait recognition performance in uncontrolled environments.

Abstract: Gait recognition is emerging as a promising and innovative area within the field of computer vision, widely applied to remote person identification. Although existing gait recognition methods have achieved substantial success in controlled laboratory datasets, their performance often declines significantly when transitioning to wild datasets.We argue that the performance gap can be primarily attributed to the spatio-temporal distribution inconsistencies present in wild datasets, where subjects appear at varying angles, positions, and distances across the frames. To achieve accurate gait recognition in the wild, we propose a skeleton-guided silhouette alignment strategy, which uses prior knowledge of the skeletons to perform affine transformations on the corresponding silhouettes.To the best of our knowledge, this is the first study to explore the impact of data alignment on gait recognition. We conducted extensive experiments across multiple datasets and network architectures, and the results demonstrate the significant advantages of our proposed alignment strategy.Specifically, on the challenging Gait3D dataset, our method achieved an average performance improvement of 7.9% across all evaluated networks. Furthermore, our method achieves substantial improvements on cross-domain datasets, with accuracy improvements of up to 24.0%.

Method: Two-stage framework: 1) Metric learning stage with Multi-object Pose Reasoning (MoPR) algorithm optimizing pose hypotheses under constraints, and 2) Self-training stage with Symmetry-aware Lie-group based Bayesian Gaussian Mixture Model (SaL-BGMM) for symmetry-aware filtering, plus Weighted Ranking InfoNCE loss for perceptual metric learning.

Result: Outperforms several state-of-the-art methods on both ROBI dataset and newly introduced Self-ROBI dataset.

Conclusion: The proposed framework effectively reduces human intervention in bin-picking tasks by automatically processing data from low-cost cameras and enabling self-training on untrained or unseen objects.

Abstract: Bin-picking of metal objects using low-cost RGB-D cameras often suffers from sparse depth information and reflective surface textures, leading to errors and the need for manual labeling. To reduce human intervention, we propose a two-stage framework consisting of a metric learning stage and a self-training stage. Specifically, to automatically process data captured by a low-cost camera (LC), we introduce a Multi-object Pose Reasoning (MoPR) algorithm that optimizes pose hypotheses under depth, collision, and boundary constraints. To further refine pose candidates, we adopt a Symmetry-aware Lie-group based Bayesian Gaussian Mixture Model (SaL-BGMM), integrated with the Expectation-Maximization (EM) algorithm, for symmetry-aware filtering. Additionally, we propose a Weighted Ranking Information Noise Contrastive Estimation (WR-InfoNCE) loss to enable the LC to learn a perceptual metric from reconstructed data, supporting self-training on untrained or even unseen objects. Experimental results show that our approach outperforms several state-of-the-art methods on both the ROBI dataset and our newly introduced Self-ROBI dataset.

Method: Explored diffusion model design space including noise schedules, prediction types (epsilon, x, v), and loss weightings. Used LIDC-IDRI lung lesion dataset and introduced a randomly cropped variant for better uncertainty evaluation.

Result: Achieved state-of-the-art performance on both standard LIDC-IDRI and the harder randomly cropped variant. Found that harder noise schedules with input scaling significantly improve performance, and x/v-prediction outperform epsilon-prediction.

Conclusion: Diffusion models are effective for generative segmentation when properly configured with appropriate noise schedules and prediction types, particularly in discrete segmentation domains where x/v-prediction excel.

Abstract: Medical image segmentation often involves inherent uncertainty due to variations in expert annotations. Capturing this uncertainty is an important goal and previous works have used various generative image models for the purpose of representing the full distribution of plausible expert ground truths. In this work, we explore the design space of diffusion models for generative segmentation, investigating the impact of noise schedules, prediction types, and loss weightings. Notably, we find that making the noise schedule harder with input scaling significantly improves performance. We conclude that x- and v-prediction outperform epsilon-prediction, likely because the diffusion process is in the discrete segmentation domain. Many loss weightings achieve similar performance as long as they give enough weight to the end of the diffusion process. We base our experiments on the LIDC-IDRI lung lesion dataset and obtain state-of-the-art (SOTA) performance. Additionally, we introduce a randomly cropped variant of the LIDC-IDRI dataset that is better suited for uncertainty in image segmentation. Our model also achieves SOTA in this harder setting.

Method: Predicts per-pixel radial distances to minimally four keypoints, uses novel Direct Linear Transform formulation for accurate 3D surface estimates, and introduces symmetry-aware keypoint ordering to handle multiple valid configurations in symmetric objects.

Result: Outperforms existing methods on benchmark datasets (LINEMOD, Occlusion LINEMOD, YCB-Video), especially for symmetric and occluded objects.

Conclusion: DLTPose effectively combines the strengths of sparse and dense methods while addressing symmetry challenges through novel keypoint ordering, achieving state-of-the-art performance in 6DoF object pose estimation.

Abstract: We propose DLTPose, a novel method for 6DoF object pose estimation from RGBD images that combines the accuracy of sparse keypoint methods with the robustness of dense pixel-wise predictions. DLTPose predicts per-pixel radial distances to a set of minimally four keypoints, which are then fed into our novel Direct Linear Transform (DLT) formulation to produce accurate 3D object frame surface estimates, leading to better 6DoF pose estimation. Additionally, we introduce a novel symmetry-aware keypoint ordering approach, designed to handle object symmetries that otherwise cause inconsistencies in keypoint assignments. Previous keypoint-based methods relied on fixed keypoint orderings, which failed to account for the multiple valid configurations exhibited by symmetric objects, which our ordering approach exploits to enhance the model’s ability to learn stable keypoint representations. Extensive experiments on the benchmark LINEMOD, Occlusion LINEMOD and YCB-Video datasets show that DLTPose outperforms existing methods, especially for symmetric and occluded objects. The code is available at https://anonymous.4open.science/r/DLTPose_/ .

Method: Proposes InterAnimate, a region-aware diffusion model that learns spatio-temporal contact dynamics and biomechanical deformations using learnable spatial/temporal latents and region-aware interaction mechanism in denoising process.

Result: Qualitative and quantitative results show highly realistic animations with anatomically accurate facial deformations and collision-free contact, setting new benchmark for hand-face interaction animation.

Conclusion: This work establishes the first systematic framework for hand-face interaction research, providing both dataset (InterHF) and model (InterAnimate) that significantly advance the field, with code and data to be made publicly available.

Abstract: Recent video generation research has focused heavily on isolated actions, leaving interactive motions-such as hand-face interactions-largely unexamined. These interactions are essential for emerging biometric authentication systems, which rely on interactive motion-based anti-spoofing approaches. From a security perspective, there is a growing need for large-scale, high-quality interactive videos to train and strengthen authentication models. In this work, we introduce a novel paradigm for animating realistic hand-face interactions. Our approach simultaneously learns spatio-temporal contact dynamics and biomechanically plausible deformation effects, enabling natural interactions where hand movements induce anatomically accurate facial deformations while maintaining collision-free contact. To facilitate this research, we present InterHF, a large-scale hand-face interaction dataset featuring 18 interaction patterns and 90,000 annotated videos. Additionally, we propose InterAnimate, a region-aware diffusion model designed specifically for interaction animation. InterAnimate leverages learnable spatial and temporal latents to effectively capture dynamic interaction priors and integrates a region-aware interaction mechanism that injects these priors into the denoising process. To the best of our knowledge, this work represents the first large-scale effort to systematically study human hand-face interactions. Qualitative and quantitative results show InterAnimate produces highly realistic animations, setting a new benchmark. Code and data will be made public to advance research.

Method: Uses frozen CLIP model to generate image and text embeddings, then employs hierarchical attention mechanism to jointly learn single-domain and cross-domain preferences by mimicking human information integration.

Result: Outperforms existing methods on four e-commerce datasets, demonstrating superior capability in capturing cross-domain user interests and sequential decision-making patterns.

Conclusion: Successfully bridges cognitive principles with computational models, highlighting the critical role of multimodal data in enhancing sequential recommendation systems and providing a more human-like modeling approach.

Abstract: Cross-Domain Sequential Recommendation (CDSR) predicts user behavior by leveraging historical interactions across multiple domains, focusing on modeling cross-domain preferences through intra- and inter-sequence item relationships. Inspired by human cognitive processes, we propose Hierarchical Attention Fusion of Visual and Textual Representations (HAF-VT), a novel approach integrating visual and textual data to enhance cognitive modeling. Using the frozen CLIP model, we generate image and text embeddings, enriching item representations with multimodal data. A hierarchical attention mechanism jointly learns single-domain and cross-domain preferences, mimicking human information integration. Evaluated on four e-commerce datasets, HAF-VT outperforms existing methods in capturing cross-domain user interests, bridging cognitive principles with computational models and highlighting the role of multimodal data in sequential decision-making.

Method: Proposes Segmenting Objectiveness and Task-Awareness (SOTA) framework with Semantic Fusion Block (SFB) for better objectiveness segmentation and Scene-understanding Guided Prompt-Context Adaptor (SG-PCA) to filter task-irrelevant anomalies.

Result: Extensive evaluations on multiple benchmark datasets (Fishyscapes Lost and Found, Segment-Me-If-You-Can, RoadAnomaly) show consistent improvement in OOD detection performance across diverse detectors.

Conclusion: SOTA achieves robust and accurate segmentation outcomes by effectively addressing both objectiveness attributes and task-awareness in autonomous driving anomaly detection.

Abstract: With the emergence of transformer-based architectures and large language models (LLMs), the accuracy of road scene perception has substantially advanced. Nonetheless, current road scene segmentation approaches are predominantly trained on closed-set data, resulting in insufficient detection capabilities for out-of-distribution (OOD) objects. To overcome this limitation, road anomaly detection methods have been proposed. However, existing methods primarily depend on image inpainting and OOD distribution detection techniques, facing two critical issues: (1) inadequate consideration of the objectiveness attributes of anomalous regions, causing incomplete segmentation when anomalous objects share similarities with known classes, and (2) insufficient attention to environmental constraints, leading to the detection of anomalies irrelevant to autonomous driving tasks. In this paper, we propose a novel framework termed Segmenting Objectiveness and Task-Awareness (SOTA) for autonomous driving scenes. Specifically, SOTA enhances the segmentation of objectiveness through a Semantic Fusion Block (SFB) and filters anomalies irrelevant to road navigation tasks using a Scene-understanding Guided Prompt-Context Adaptor (SG-PCA). Extensive empirical evaluations on multiple benchmark datasets, including Fishyscapes Lost and Found, Segment-Me-If-You-Can, and RoadAnomaly, demonstrate that the proposed SOTA consistently improves OOD detection performance across diverse detectors, achieving robust and accurate segmentation outcomes.

Method: Comprehensive survey methodology: reviewing foundational concepts, categorizing unified models into three architectural paradigms (diffusion-based, autoregressive-based, hybrid), analyzing structural designs, compiling datasets/benchmarks, and discussing key challenges.

Result: Systematic classification of unified multimodal models, identification of three main architectural approaches, compilation of specialized datasets and benchmarks, and analysis of critical challenges including tokenization strategies, cross-modal attention, and data requirements.

Conclusion: The field of unified multimodal models is nascent but rapidly evolving, with significant potential for integration despite architectural challenges; the survey provides foundational reference and anticipates regular updates to track advancements in this emerging research area.

Abstract: Recent years have seen remarkable progress in both multimodal understanding models and image generation models. Despite their respective successes, these two domains have evolved independently, leading to distinct architectural paradigms: While autoregressive-based architectures have dominated multimodal understanding, diffusion-based models have become the cornerstone of image generation. Recently, there has been growing interest in developing unified frameworks that integrate these tasks. The emergence of GPT-4o’s new capabilities exemplifies this trend, highlighting the potential for unification. However, the architectural differences between the two domains pose significant challenges. To provide a clear overview of current efforts toward unification, we present a comprehensive survey aimed at guiding future research. First, we introduce the foundational concepts and recent advancements in multimodal understanding and text-to-image generation models. Next, we review existing unified models, categorizing them into three main architectural paradigms: diffusion-based, autoregressive-based, and hybrid approaches that fuse autoregressive and diffusion mechanisms. For each category, we analyze the structural designs and innovations introduced by related works. Additionally, we compile datasets and benchmarks tailored for unified models, offering resources for future exploration. Finally, we discuss the key challenges facing this nascent field, including tokenization strategy, cross-modal attention, and data. As this area is still in its early stages, we anticipate rapid advancements and will regularly update this survey. Our goal is to inspire further research and provide a valuable reference for the community. The references associated with this survey are available on GitHub (https://github.com/AIDC-AI/Awesome-Unified-Multimodal-Models).

Method: Two-stage framework: 1) Pre-train teacher model with full-modality data and use Hybrid Prototype Distillation Module for cross-modal knowledge distillation, 2) Freeze models and use trainable Feature Selection Module to extract optimal representations from feature and logits layers.

Result: Improves missing-modality performance by 2.80%, 3.89%, and 0.89% on three datasets compared to state-of-the-art, with minimal full-modality performance drop (-0.1% mIoU). Validated with different backbones (AnySeg and CMNeXt).

Conclusion: RMMSS effectively addresses missing-modality challenges in MMSS while preserving full-modality performance, demonstrating strong generalizability across different architectures and datasets.

Abstract: Multi-modal semantic segmentation (MMSS) faces significant challenges in real-world applications due to incomplete, degraded, or missing sensor data. While current MMSS methods typically use self-distillation with modality dropout to improve robustness, they largely overlook inter-modal correlations and thus suffer significant performance degradation when no modalities are missing. To this end, we present RMMSS, a two-stage framework designed to progressively enhance model robustness under missing-modality conditions, while maintaining strong performance in full-modality scenarios. It comprises two key components: the Hybrid Prototype Distillation Module (HPDM) and the Feature Selection Module (FSM). In the first stage, we pre-train the teacher model with full-modality data and then introduce HPDM to do cross-modal knowledge distillation for obtaining a highly robust model. In the second stage, we freeze both the pre-trained full-modality teacher model and the robust model and propose a trainable FSM that extracts optimal representations from both the feature and logits layers of the models via feature score calculation. This process learns a final student model that maintains strong robustness while achieving high performance under full-modality conditions. Our experiments on three datasets demonstrate that our method improves missing-modality performance by 2.80%, 3.89%, and 0.89%, respectively, compared to the state-of-the-art, while causing almost no drop in full-modality performance (only -0.1% mIoU). Meanwhile, different backbones (AnySeg and CMNeXt) are utilized to validate the generalizability of our framework.

Method: Proposes binary local ordering maps to convert depth maps into relative format, unifying representations. Uses these maps to re-weight initial disparity updates and formulates monocular depth fusion as a registration problem with pixel-wise linear regression for adaptive alignment.

Result: Significantly improves performance when generalizing from SceneFlow to Middlebury and Booster datasets while maintaining computational efficiency.

Conclusion: The method effectively exploits monocular priors from vision foundation models to enhance stereo matching in challenging regions without sacrificing efficiency, providing a robust solution for ill-posed stereo matching problems.

Abstract: The matching formulation makes it naturally hard for the stereo matching to handle ill-posed regions like occlusions and non-Lambertian surfaces. Fusing monocular priors has been proven helpful for ill-posed matching, but the biased monocular prior learned from small stereo datasets constrains the generalization. Recently, stereo matching has progressed by leveraging the unbiased monocular prior from the vision foundation model (VFM) to improve the generalization in ill-posed regions. We dive into the fusion process and observe three main problems limiting the fusion of the VFM monocular prior. The first problem is the misalignment between affine-invariant relative monocular depth and absolute depth of disparity. Besides, when we use the monocular feature in an iterative update structure, the over-confidence in the disparity update leads to local optima results. A direct fusion of a monocular depth map could alleviate the local optima problem, but noisy disparity results computed at the first several iterations will misguide the fusion. In this paper, we propose a binary local ordering map to guide the fusion, which converts the depth map into a binary relative format, unifying the relative and absolute depth representation. The computed local ordering map is also used to re-weight the initial disparity update, resolving the local optima and noisy problem. In addition, we formulate the final direct fusion of monocular depth to the disparity as a registration problem, where a pixel-wise linear regression module can globally and adaptively align them. Our method fully exploits the monocular prior to support stereo matching results effectively and efficiently. We significantly improve the performance from the experiments when generalizing from SceneFlow to Middlebury and Booster datasets while barely reducing the efficiency.

Method: Textual Prototype-guided Prompt Tuning (TPPT) uses textual prototypes as stable anchors to guide visual prompt learning, with bidirectional supervision and relational diversity regularization to prevent embedding space collapse.

Result: Extensive experiments show TPPT effectively learns new knowledge while reducing forgetting, demonstrating the benefits of leveraging CLIP’s intrinsic guidance for continual adaptation.

Conclusion: The proposed TPPT approach provides a concise yet powerful continual learning solution that fully exploits CLIP’s multi-modal structure and textual representation stability.

Abstract: Continual learning (CL) enables deep networks to acquire new knowledge while avoiding catastrophic forgetting. The powerful generalization ability of pre-trained models (PTMs), such as the Contrastive Language-Image Pre-training (CLIP) model, has inspired a range of CL methods targeting new and specialized tasks, providing rich multi-modal embeddings that support lightweight, incremental prompt tuning. Existing methods often rely on complex designs built upon specific assumptions, such as intricate regularization schemes for prompt pools, specialized routing mechanisms, or multi-stage incrementations, that introduce additional-and possibly unnecessary-complexity, underutilizing CLIP’s intrinsic capabilities. In this paper, we propose a concise CL approach for CLIP based on incremental prompt tuning that fully exploits its multi-modal structure and the stability of textual representations. Our method, Textual Prototype-guided Prompt Tuning (TPPT), introduces textual prototypes not merely as static classifiers, as in existing methods, but as stable anchors to guide the learning of visual prompts, thereby shaping the embedding space (i.e., TPPT-V). We show that our bidirectional supervision strategy enables more effective learning of new knowledge while reducing forgetting. To further close the vision-language gap during CL, we jointly optimizes visual and textual prompts (i.e., TPPT-VT). We also introduce a relational diversity regularization on the textual anchors to prevent embedding space collapse and mitigate correlated forgetting. Extensive experiments and analyses demonstrate the effectiveness of our proposed approach, highlighting the benefits of leveraging CLIP’s intrinsic guidance for continual adaptation.

Method: Proposed InterRVOS task with separate actor/target segmentation, created InterRVOS-127K dataset with 127K auto-annotated expressions, and developed ReVIOSa MLLM architecture with interaction-aware tokens and attention mask loss.

Result: ReVIOSa outperforms baselines on InterRVOS-127K evaluation and achieves strong performance on standard RVOS benchmarks.

Conclusion: InterRVOS enables fine-grained interaction understanding by modeling asymmetric roles in object interactions, with the proposed dataset and architecture advancing video object segmentation capabilities.

Abstract: Referring video object segmentation (RVOS) aims to segment objects in a video described by a natural language expression. However, most existing approaches focus on segmenting only the referred object (typically the actor), even when the expression clearly describes an interaction involving multiple objects with distinct roles. For instance, “A throwing B” implies a directional interaction, but standard RVOS segments only the actor (A), neglecting other involved target objects (B). In this paper, we introduce Interaction-aware Referring Video Object Segmentation (InterRVOS), a novel task that focuses on the modeling of interactions. It requires the model to segment the actor and target objects separately, reflecting their asymmetric roles in an interaction. This task formulation enables fine-grained understanding of object relationships, as many video events are defined by such relationships rather than individual objects. To support this task, we propose a new evaluation protocol that separately evaluates actor and target segmentation, enabling more accurate assessment of the model’s ability to distinguish and segment actor and target roles. We also present InterRVOS-127K, a large-scale dataset with over 127K automatically annotated expressions, including interaction expressions annotated with distinct masks for actor and target objects. Furthermore, we develop ReVIOSa, an MLLM-based architecture that introduces interaction-aware special tokens and leverages an attention mask loss to enhance role-specific segmentation. Extensive experiments show that ReVIOSa not only outperforms existing baselines on our proposed InterRVOS-127K evaluation set, but also achieves strong performance on standard RVOS benchmarks. Our project page is available at: https://cvlab-kaist.github.io/InterRVOS.

Method: Leverages LLMs to reformulate question-option pairs for causal clarity, uses temporal grounding to retrieve salient segments, and employs adaptive fusion with MLLM processing for final answers.

Result: Achieves state-of-the-art performance on NExT-QA, IntentQA, and NExT-GQA datasets, with precise visual grounding enhancing video-question relationship understanding.

Conclusion: The synergistic approach of causal-aware query refinement and fine-grained visual grounding effectively addresses limitations of current VideoQA methods for complex reasoning tasks.

Abstract: Video Question Answering (VideoQA) requires identifying sparse critical moments in long videos and reasoning about their causal relationships to answer semantically complex questions. While recent advances in multimodal learning have improved alignment and fusion, current approaches remain limited by two prevalent but fundamentally flawed strategies: (1) task-agnostic sampling indiscriminately processes all frames, overwhelming key events with irrelevant content; and (2) heuristic retrieval captures superficial patterns but misses causal-temporal structures needed for complex reasoning. To address these challenges, we introduce LeAdQA, an innovative approach that bridges these gaps through synergizing causal-aware query refinement with fine-grained visual grounding. Our method first leverages LLMs to reformulate question-option pairs, resolving causal ambiguities and sharpening temporal focus. These refined queries subsequently direct a temporal grounding model to precisely retrieve the most salient segments, complemented by an adaptive fusion mechanism dynamically integrating the evidence to maximize relevance. The integrated visual-textual cues are then processed by an MLLM to generate accurate, contextually-grounded answers. Experiments on NExT-QA, IntentQA, and NExT-GQA demonstrate that our method’s precise visual grounding substantially enhances the understanding of video-question relationships, achieving state-of-the-art (SOTA) performance on complex reasoning tasks while maintaining computational efficiency.

Method: Proposes a mask-based LoRA tuning approach with spatiotemporal masks that teach the model to preserve content or generate new content in designated regions, and synthesize consistent motion or novel appearances from reference frames.

Result: Achieves superior video editing performance compared to baseline methods, enabling complex temporal transformations like object rotation or flower blooming.

Conclusion: The dual-capability LoRA approach provides users with comprehensive control over the entire temporal evolution of video edits, overcoming limitations of existing methods.

Abstract: Video editing using diffusion models has achieved remarkable results in generating high-quality edits for videos. However, current methods often rely on large-scale pretraining, limiting flexibility for specific edits. First-frame-guided editing provides control over the first frame, but lacks flexibility over subsequent frames. To address this, we propose a mask-based LoRA (Low-Rank Adaptation) tuning method that adapts pretrained Image-to-Video (I2V) models for flexible video editing. Our key innovation is using a spatiotemporal mask to strategically guide the LoRA fine-tuning process. This teaches the model two distinct skills: first, to interpret the mask as a command to either preserve content from the source video or generate new content in designated regions. Second, for these generated regions, LoRA learns to synthesize either temporally consistent motion inherited from the video or novel appearances guided by user-provided reference frames. This dual-capability LoRA grants users control over the edit’s entire temporal evolution, allowing complex transformations like an object rotating or a flower blooming. Experimental results show our method achieves superior video editing performance compared to baseline methods. Project Page: https://cjeen.github.io/LoRAEdit

Method: VisFlow uses dual-level Attention Intervention: Token-level Attention Intervention reinforces attention to salient visual regions, and Head-level Attention Intervention suppresses undue focus on system prompts and adjacent text tokens.

Result: Extensive experiments show VisFlow effectively mitigates hallucinations with minimal computational overhead across diverse models and benchmarks.

Conclusion: The proposed attention modulation framework successfully addresses visual hallucinations in LVLMs by strengthening visual alignment and reducing linguistic bias without requiring additional training.

Abstract: Large vision-language models (LVLMs) have demonstrated impressive capabilities across diverse multimodal tasks, yet they remain highly susceptible to visual hallucinations (VH), often producing confident but inaccurate descriptions of visual content. Building on the insight that not all tokens and attention heads contribute equally to VH mitigation, we introduce VisFlow, a lightweight and training-free framework that alleviates hallucinations by directly modulating attention patterns during inference. To address two primary challenges of VH, namely insufficient visual attention and the dominance of language priors, we identify three problematic attention behaviors in LVLMs: (1) disproportionate allocation of attention to uninformative or trailing visual tokens, (2) over-dependence on the previously generated token, and (3) excessive fixation on system prompts that hinders multimodal integration. To overcome these issues, VisFlow introduces a dual-level Attention Intervention, consisting of Token-level Attention Intervention (TAI), which reinforces attention to salient visual regions, and Head-level Attention Intervention (HAI), which suppresses undue focus on system prompts and adjacent text tokens. Together, these interventions strengthen visual alignment while reducing linguistic bias. Extensive experiments across diverse models and benchmarks demonstrate that VisFlow effectively mitigates hallucinations with minimal computational overhead.

Method: Proposes a Diffusion Transformer (DiT) approach combining diffusion models and transformer architectures, with a Circular Position-Shift strategy for enhanced temporal consistency during inference, and interactive object removal with prompt generation.

Result: Achieves 65-second processing time for 97-frame 4K video on H800 GPU, demonstrating superior performance in content fidelity, texture restoration, and temporal consistency compared to traditional methods.

Conclusion: The DiT-based approach effectively addresses limitations of traditional video inpainting methods, providing high-quality results with excellent temporal consistency and fast processing speeds for large-scale video restoration tasks.

Abstract: Video object removal and inpainting are critical tasks in the fields of computer vision and multimedia processing, aimed at restoring missing or corrupted regions in video sequences. Traditional methods predominantly rely on flow-based propagation and spatio-temporal Transformers, but these approaches face limitations in effectively leveraging long-term temporal features and ensuring temporal consistency in the completion results, particularly when dealing with large masks. Consequently, performance on extensive masked areas remains suboptimal. To address these challenges, this paper introduces a novel video inpainting approach leveraging the Diffusion Transformer (DiT). DiT synergistically combines the advantages of diffusion models and transformer architectures to maintain long-term temporal consistency while ensuring high-quality inpainting results. We propose a Circular Position-Shift strategy to further enhance long-term temporal consistency during the inference stage. Additionally, the proposed method interactively removes specified objects, and generates corresponding prompts. In terms of processing speed, it takes only 65 seconds (testing on one NVIDIA H800 GPU) to complete a video with a resolution of $2160 \times 2100$ with 97 frames without any acceleration method. Experimental results indicate that the proposed method demonstrates superior performance in content fidelity, texture restoration, and temporal consistency. Project page:https://jieliu95.github.io/EraserDiT_demo/

Method: Introduces subject distributor and visual concept injector modules to embed shared-subject and distinct-attributes concepts into latent representations. Uses positive-margin contrastive learning to align latent expressions with original text while preserving variations.

Result: Outperforms state-of-the-art RIS and REC approaches on multiple benchmarks and achieves outstanding performance on generalized referring expression segmentation (GRES) benchmark.

Conclusion: The proposed framework effectively addresses the limitation of single-text inputs by generating multiple complementary latent expressions, significantly improving visual grounding performance across various tasks and benchmarks.

Abstract: Visual grounding tasks, such as referring image segmentation (RIS) and referring expression comprehension (REC), aim to localize a target object based on a given textual description. The target object in an image can be described in multiple ways, reflecting diverse attributes such as color, position, and more. However, most existing methods rely on a single textual input, which captures only a fraction of the rich information available in the visual domain. This mismatch between rich visual details and sparse textual cues can lead to the misidentification of similar objects. To address this, we propose a novel visual grounding framework that leverages multiple latent expressions generated from a single textual input by incorporating complementary visual details absent from the original description. Specifically, we introduce subject distributor and visual concept injector modules to embed both shared-subject and distinct-attributes concepts into the latent representations, thereby capturing unique and target-specific visual cues. We also propose a positive-margin contrastive learning strategy to align all latent expressions with the original text while preserving subtle variations. Experimental results show that our method not only outperforms state-of-the-art RIS and REC approaches on multiple benchmarks but also achieves outstanding performance on the generalized referring expression segmentation (GRES) benchmark.

Method: Evaluated state-of-the-art object detection models, selected YOLO as most promising, optimized it with training on multiple benchmark datasets using semi-supervised auto labeling strategy.

Result: Developed a successful general solution for object detection in lecture videos, with YOLO emerging as the most effective model for this specific task.

Conclusion: The approach effectively addresses the unique challenges of lecture video visual element detection and contributes a publicly released benchmark dataset and source code to advance future research in this area.

Abstract: Video is transforming education with online courses and recorded lectures supplementing and replacing classroom teaching. Recent research has focused on enhancing information retrieval for video lectures with advanced navigation, searchability, summarization, as well as question answering chatbots. Visual elements like tables, charts, and illustrations are central to comprehension, retention, and data presentation in lecture videos, yet their full potential for improving access to video content remains underutilized. A major factor is that accurate automatic detection of visual elements in a lecture video is challenging; reasons include i) most visual elements, such as charts, graphs, tables, and illustrations, are artificially created and lack any standard structure, and ii) coherent visual objects may lack clear boundaries and may be composed of connected text and visual components. Despite advancements in deep learning based object detection, current models do not yield satisfactory performance due to the unique nature of visual content in lectures and scarcity of annotated datasets. This paper reports on a transfer learning approach for detecting visual elements in lecture video frames. A suite of state of the art object detection models were evaluated for their performance on lecture video datasets. YOLO emerged as the most promising model for this task. Subsequently YOLO was optimized for lecture video object detection with training on multiple benchmark datasets and deploying a semi-supervised auto labeling strategy. Results evaluate the success of this approach, also in developing a general solution to the problem of object detection in lecture videos. Paper contributions include a publicly released benchmark of annotated lecture video frames, along with the source code to facilitate future research.

Method: Proposes FMCE-Net++ framework with pretrained frozen FMCE-Net as auxiliary head that generates FMCS predictions. These are combined with task labels for joint supervision via Representation Auxiliary Loss with tunable Representation Abstraction Factor to balance classification and feature convergence optimization.

Result: Extensive experiments on MNIST, CIFAR-10, FashionMNIST, and CIFAR-100 show consistent performance enhancements: +1.16 pp accuracy gain (ResNet-50/CIFAR-10) and +1.08 pp (ShuffleNet v2/CIFAR-100) without architectural modifications or additional data.

Conclusion: FMCE-Net++ effectively elevates state-of-the-art performance ceilings by integrating feature convergence optimization with task supervision through a novel auxiliary loss framework.

Abstract: Deep Neural Networks (DNNs) face interpretability challenges due to their opaque internal representations. While Feature Map Convergence Evaluation (FMCE) quantifies module-level convergence via Feature Map Convergence Scores (FMCS), it lacks experimental validation and closed-loop integration. To address this limitation, we propose FMCE-Net++, a novel training framework that integrates a pretrained, frozen FMCE-Net as an auxiliary head. This module generates FMCS predictions, which, combined with task labels, jointly supervise backbone optimization through a Representation Auxiliary Loss. The RAL dynamically balances the primary classification loss and feature convergence optimization via a tunable \Representation Abstraction Factor. Extensive experiments conducted on MNIST, CIFAR-10, FashionMNIST, and CIFAR-100 demonstrate that FMCE-Net++ consistently enhances model performance without architectural modifications or additional data. Key experimental outcomes include accuracy gains of $+1.16$ pp (ResNet-50/CIFAR-10) and $+1.08$ pp (ShuffleNet v2/CIFAR-100), validating that FMCE-Net++ can effectively elevate state-of-the-art performance ceilings.

Method: Proposes whitening data to decorrelate and equalize variance before prompt learning, using bilinear multiplication with whitening matrix. Also introduces low-rank version for parameter efficiency.

Result: Significant accuracy improvements (>25 points on CUB), faster convergence, reduced parameter count and computation overhead compared to VPT methods.

Conclusion: BPT effectively addresses distribution challenges in VPT, enabling more efficient and accurate prompt tuning with fewer parameters.

Abstract: Visual Prompt Tuning (VPT) is a parameter-efficient fune-tuning technique that adapts a pre-trained vision Transformer (ViT) by learning a small set of parameters in the input space, known as prompts. In VPT, we uncover burstiness'' in the values arising from the interaction of image patch embeddings, and the key and query projectors within Transformer's self-attention module. Furthermore, the values of patch embeddings and the key and query projectors exhibit Laplacian and hyper-Laplacian distribution, respectively. Intuitively, these non-Gaussian distributions pose challenges for learning prompts. To address this, we propose whitening these data, de-correlating them and equalizing their variance towards more Gaussian before learning prompts. We derive the whitening matrix over random image patch embeddings and ViT's key and query projectors, and multiply it with the prompt to be learned in a bilinear manner. Surprisingly, this method significantly accelerates prompt tuning and boosts accuracy, e.g., $>$25 accuracy points on the CUB dataset; interestingly, it learns bursty prompts’’. Extending the bilinear model which is known to introduce burstiness, we present a compact, low-rank version by learning two smaller matrices whose multiplication yields the final prompts. We call the proposed methods Bilinear Prompt Tuning (BPT). Extensive experiments across multiple benchmark datasets demonstrate that BPT methods not only outperform various VPT methods but also reduce parameter count and computation overhead.

Method: Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement in a no-code interactive platform.

Result: Outperforms baselines with average accuracy of 0.947, IoU of 0.858, and Dice score of 0.923 on amorphous catalyst samples; 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples - significantly better than Otsu thresholding and Segment Anything Model.

Conclusion: Zenesis enables effective image segmentation in domains with limited annotated data, offering a scalable solution for scientific discovery by overcoming data readiness bottlenecks in scientific imaging workflows.

Abstract: Zero-shot and prompt-based models have excelled at visual reasoning tasks by leveraging large-scale natural image corpora, but they often fail on sparse and domain-specific scientific image data. We introduce Zenesis, a no-code interactive computer vision platform designed to reduce data readiness bottlenecks in scientific imaging workflows. Zenesis integrates lightweight multimodal adaptation for zero-shot inference on raw scientific data, human-in-the-loop refinement, and heuristic-based temporal enhancement. We validate our approach on Focused Ion Beam Scanning Electron Microscopy (FIB-SEM) datasets of catalyst-loaded membranes. Zenesis outperforms baselines, achieving an average accuracy of 0.947, Intersection over Union (IoU) of 0.858, and Dice score of 0.923 on amorphous catalyst samples; and 0.987 accuracy, 0.857 IoU, and 0.923 Dice on crystalline samples. These results represent a significant performance gain over conventional methods such as Otsu thresholding and standalone models like the Segment Anything Model (SAM). Zenesis enables effective image segmentation in domains where annotated datasets are limited, offering a scalable solution for scientific discovery.

Method: Proposed SafePLUG framework supports arbitrary-shaped visual prompts for region-aware QA, pixel-level segmentation based on language instructions, and recognition of temporally anchored events. Created new multimodal dataset with detailed pixel-level annotations and temporal event boundaries.

Result: SafePLUG achieves strong performance on multiple tasks including region-based QA, pixel-level segmentation, temporal event localization, and accident event understanding.

Conclusion: The framework lays foundation for fine-grained understanding of complex traffic scenes, with potential to improve driving safety and enhance situational awareness in smart transportation systems.

Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress across a range of vision-language tasks and demonstrate strong potential for traffic accident understanding. However, existing MLLMs in this domain primarily focus on coarse-grained image-level or video-level comprehension and often struggle to handle fine-grained visual details or localized scene components, limiting their applicability in complex accident scenarios. To address these limitations, we propose SafePLUG, a novel framework that empowers MLLMs with both Pixel-Level Understanding and temporal Grounding for comprehensive traffic accident analysis. SafePLUG supports both arbitrary-shaped visual prompts for region-aware question answering and pixel-level segmentation based on language instructions, while also enabling the recognition of temporally anchored events in traffic accident scenarios. To advance the development of MLLMs for traffic accident understanding, we curate a new dataset containing multimodal question-answer pairs centered on diverse accident scenarios, with detailed pixel-level annotations and temporal event boundaries. Experimental results show that SafePLUG achieves strong performance on multiple tasks, including region-based question answering, pixel-level segmentation, temporal event localization, and accident event understanding. These capabilities lay a foundation for fine-grained understanding of complex traffic scenes, with the potential to improve driving safety and enhance situational awareness in smart transportation systems. The code, dataset, and model checkpoints will be made publicly available at: https://zihaosheng.github.io/SafePLUG

Method: Multi-turn interactive active learning strategy that selects data based on semantic uncertainty metrics and actively requests reannotation from oracle. Uses variance-based metric considering semantic relationships between terms.

Result: Extensive experiments show better model performance and substantial training cost reduction - halving training costs for achieving relatively high accuracy.

Conclusion: The proposed interactive active learning approach effectively addresses misleading annotation issues in 3D VQA, improving performance while reducing training costs through semantic-aware uncertainty assessment and reannotation requests.

Abstract: 3D Visual Question Answering (3D VQA) is crucial for enabling models to perceive the physical world and perform spatial reasoning. In 3D VQA, the free-form nature of answers often leads to improper annotations that can confuse or mislead models when training on the entire dataset. While other text generation tasks can mitigate this issue by learning on large-scale datasets, the scarcity of 3D scene data enlarges the negative effect of misleading annotations. Although active learning strategies can select valuable instances for training, they fail to identify and resolve misleading labels, which the oracle inevitably provides in practice. To address this issue, we propose a multi-turn interactive active learning strategy. This strategy selects data based on models’ semantic uncertainty to form a solid knowledge foundation more effectively and actively requests reannotation from an oracle to resolve potentially misleading labels. For uncertainty assessment, we utilize a variance-based metric that takes semantic relationships between terms into consideration, thus avoiding the uniform inter-class similarity assumption of previous assessment metrics. Extensive experiments exhibit better model performance and a substantial reduction in training costs, with a halving of training costs for achieving relatively high accuracy. The code is available at https://github.com/fz-zsl/AQuA.

Method: A novel GAN-based blind FIR framework with two key components: local SFFT module that aligns statistical distributions of low-quality facial regions with high-quality counterparts, and DAFE module that enables robust feature extraction by aligning encoder representations under adverse weather conditions.

Result: The proposed degradation-agnostic SFFT model outperforms existing state-of-the-art GAN and diffusion-based FIR methods, particularly in suppressing texture distortions and accurately reconstructing facial structures under challenging weather scenarios.

Conclusion: Both SFFT and DAFE modules are empirically validated to enhance structural fidelity and perceptual quality in face restoration, making the framework effective for adverse weather conditions in intelligent CCTV systems.

Abstract: With the increasing deployment of intelligent CCTV systems in outdoor environments, there is a growing demand for face recognition systems optimized for challenging weather conditions. Adverse weather significantly degrades image quality, which in turn reduces recognition accuracy. Although recent face image restoration (FIR) models based on generative adversarial networks (GANs) and diffusion models have shown progress, their performance remains limited due to the lack of dedicated modules that explicitly address weather-induced degradations. This leads to distorted facial textures and structures. To address these limitations, we propose a novel GAN-based blind FIR framework that integrates two key components: local Statistical Facial Feature Transformation (SFFT) and Degradation-Agnostic Feature Embedding (DAFE). The local SFFT module enhances facial structure and color fidelity by aligning the local statistical distributions of low-quality (LQ) facial regions with those of high-quality (HQ) counterparts. Complementarily, the DAFE module enables robust statistical facial feature extraction under adverse weather conditions by aligning LQ and HQ encoder representations, thereby making the restoration process adaptive to severe weather-induced degradations. Experimental results demonstrate that the proposed degradation-agnostic SFFT model outperforms existing state-of-the-art FIR methods based on GAN and diffusion models, particularly in suppressing texture distortions and accurately reconstructing facial structures. Furthermore, both the SFFT and DAFE modules are empirically validated in enhancing structural fidelity and perceptual quality in face restoration under challenging weather scenarios.

Method: Trained patch-level Sparse Autoencoder (SAE) on Mammo-CLIP vision-language model to identify latent features associated with clinical breast concepts like masses and suspicious calcifications.

Result: Top activated latent neurons align with ground truth regions, uncovered confounding factors in decision-making, and identified which neurons are crucial for downstream breast concept prediction.

Conclusion: SAE latent representations provide valuable insights into foundation model internals for breast imaging, demonstrating promise for interpretable medical AI systems.

Abstract: Interpretability is critical in high-stakes domains such as medical imaging, where understanding model decisions is essential for clinical adoption. In this work, we introduce Sparse Autoencoder (SAE)-based interpretability to breast imaging by analyzing {Mammo-CLIP}, a vision–language foundation model pretrained on large-scale mammogram image–report pairs. We train a patch-level \texttt{Mammo-SAE} on Mammo-CLIP to identify and probe latent features associated with clinically relevant breast concepts such as \textit{mass} and \textit{suspicious calcification}. Our findings reveal that top activated class level latent neurons in the SAE latent space often tend to align with ground truth regions, and also uncover several confounding factors influencing the model’s decision-making process. Additionally, we analyze which latent neurons the model relies on during downstream finetuning for improving the breast concept prediction. This study highlights the promise of interpretable SAE latent representations in providing deeper insight into the internal workings of foundation models at every layer for breast imaging. The code will be released at https://krishnakanthnakka.github.io/MammoSAE/

Method: Developed Spatial Self-Attention mechanism integrated with LSTM network for feature extraction, Gradient Similarity Comparison for federated learning model selection, and automated video processing tool with face detection/cropping and data augmentation techniques.

Result: Achieved 89.9% detection accuracy in federated learning settings, outperforming existing methods across various deployment scenarios while handling real-world data variability.

Conclusion: The framework effectively addresses decentralized data challenges, preserves user privacy, and shows strong potential for deployment in intelligent transportation systems to enhance road safety through reliable drowsiness detection.

Abstract: Driver drowsiness is one of the main causes of road accidents and is recognized as a leading contributor to traffic-related fatalities. However, detecting drowsiness accurately remains a challenging task, especially in real-world settings where facial data from different individuals is decentralized and highly diverse. In this paper, we propose a novel framework for drowsiness detection that is designed to work effectively with heterogeneous and decentralized data. Our approach develops a new Spatial Self-Attention (SSA) mechanism integrated with a Long Short-Term Memory (LSTM) network to better extract key facial features and improve detection performance. To support federated learning, we employ a Gradient Similarity Comparison (GSC) that selects the most relevant trained models from different operators before aggregation. This improves the accuracy and robustness of the global model while preserving user privacy. We also develop a customized tool that automatically processes video data by extracting frames, detecting and cropping faces, and applying data augmentation techniques such as rotation, flipping, brightness adjustment, and zooming. Experimental results show that our framework achieves a detection accuracy of 89.9% in the federated learning settings, outperforming existing methods under various deployment scenarios. The results demonstrate the effectiveness of our approach in handling real-world data variability and highlight its potential for deployment in intelligent transportation systems to enhance road safety through early and reliable drowsiness detection.

Method: Dual-resolution architecture with high-resolution stream for boundary details and pooled stream for context; boundary-aware residual connections; channel attention mechanism; lightweight artifact suppression block; multi-task training with Dice-Tversky loss, boundary loss, and contrastive regularizer.

Result: Significantly enhances boundary precision and clinically relevant segmentation metrics on public dermoscopic benchmarks, outperforming traditional encoder-decoder baselines.

Conclusion: The approach provides pixel-accurate segmentation without extensive post-processing, making it valuable for automated melanoma assessment systems.

Abstract: Lesion segmentation, in contrast to natural scene segmentation, requires handling subtle variations in texture and color, frequent imaging artifacts (such as hairs, rulers, and bubbles), and a critical need for precise boundary localization to aid in accurate diagnosis. The accurate delineation of melanocytic tumors in dermoscopic images is a crucial component of automated skin cancer screening systems and clinical decision support. In this paper, we present a novel dual-resolution architecture inspired by ResNet, specifically tailored for the segmentation of melanocytic tumors. Our approach incorporates a high-resolution stream that preserves fine boundary details, alongside a complementary pooled stream that captures multi-scale contextual information for robust lesion recognition. These two streams are closely integrated through boundary-aware residual connections, which inject edge information into deep feature maps, and a channel attention mechanism that adapts the model’s sensitivity to color and texture variations in dermoscopic images. To tackle common imaging artifacts and the challenges posed by small clinical datasets, we introduce a lightweight artifact suppression block and a multi-task training strategy. This strategy combines the Dice-Tversky loss with an explicit boundary loss and a contrastive regularizer to enhance feature stability. This unified design enables the model to generate pixel-accurate segmentation masks without the need for extensive post-processing or complex pre-training. Extensive evaluation on public dermoscopic benchmarks reveals that our method significantly enhances boundary precision and clinically relevant segmentation metrics, outperforming traditional encoder-decoder baselines. This makes our approach a valuable component for building automated melanoma assessment systems.

Method: Uses sparse annotations expanded into dense probabilistic supervision through differentiable random walk label propagation that integrates vesselness cues and tubular continuity priors. Joint training with CNN segmentation network and uncertainty-weighted loss. Includes topology-aware regularizer for centerline connectivity.

Result: Consistently outperforms naive sparse-label training and traditional dense pseudo-labeling methods on clinical subcutaneous imaging datasets, yielding more accurate vascular maps and better-calibrated uncertainty.

Conclusion: Significantly reduces annotation workload while maintaining clinically relevant vessel topology, making it crucial for clinical decision-making in subcutaneous vessel segmentation.

Abstract: The task of parsing subcutaneous vessels in clinical images is often hindered by the high cost and limited availability of ground truth data, as well as the challenge of low contrast and noisy vessel appearances across different patients and imaging modalities. In this work, we propose a novel weakly supervised training framework specifically designed for subcutaneous vessel segmentation. This method utilizes low-cost, sparse annotations such as centerline traces, dot markers, or short scribbles to guide the learning process. These sparse annotations are expanded into dense probabilistic supervision through a differentiable random walk label propagation model, which integrates vesselness cues and tubular continuity priors driven by image data. The label propagation process results in per-pixel hitting probabilities and uncertainty estimates, which are incorporated into an uncertainty-weighted loss function to prevent overfitting in ambiguous areas. Notably, the label propagation model is trained jointly with a CNN-based segmentation network, allowing the system to learn vessel boundaries and continuity constraints without the need for explicit edge supervision. Additionally, we introduce a topology-aware regularizer that encourages centerline connectivity and penalizes irrelevant branches, further enhancing clinical applicability. Our experiments on clinical subcutaneous imaging datasets demonstrate that our approach consistently outperforms both naive sparse-label training and traditional dense pseudo-labeling methods, yielding more accurate vascular maps and better-calibrated uncertainty, which is crucial for clinical decision-making. This method significantly reduces the annotation workload while maintaining clinically relevant vessel topology.

Method: Cleaned HSFace dataset using MoE strategy with face embedding clustering and GPT-4o verification, augmented real identities, generated synthetic identities with Stable Diffusion and prompt engineering, expanded with Vec2Face for efficiency, and used curriculum learning during training.

Result: Achieved 1st place in competition, with improved model performance across 10K, 20K, and 100K identity scales. Final dataset contains 50 images per identity with no identity leakage.

Conclusion: Hybrid approach combining cleaned real data and efficiently generated synthetic identities with curriculum learning effectively creates diverse, high-quality face datasets for recognition tasks while preventing identity overlap issues.

Abstract: In this paper, we present our approach to the DataCV ICCV Challenge, which centers on building a high-quality face dataset to train a face recognition model. The constructed dataset must not contain identities overlapping with any existing public face datasets. To handle this challenge, we begin with a thorough cleaning of the baseline HSFace dataset, identifying and removing mislabeled or inconsistent identities through a Mixture-of-Experts (MoE) strategy combining face embedding clustering and GPT-4o-assisted verification. We retain the largest consistent identity cluster and apply data augmentation up to a fixed number of images per identity. To further diversify the dataset, we generate synthetic identities using Stable Diffusion with prompt engineering. As diffusion models are computationally intensive, we generate only one reference image per identity and efficiently expand it using Vec2Face, which rapidly produces 49 identity-consistent variants. This hybrid approach fuses GAN-based and diffusion-based samples, enabling efficient construction of a diverse and high-quality dataset. To address the high visual similarity among synthetic identities, we adopt a curriculum learning strategy by placing them early in the training schedule, allowing the model to progress from easier to harder samples. Our final dataset contains 50 images per identity, and all newly generated identities are checked with mainstream face datasets to ensure no identity leakage. Our method achieves \textbf{1st place} in the competition, and experimental results show that our dataset improves model performance across 10K, 20K, and 100K identity scales. Code is available at https://github.com/Ferry-Li/datacv_fr.

Method: AugLift enriches standard 2D keypoint coordinates (x,y) with keypoint detection confidence scores and corresponding depth estimates computed from images using off-the-shelf pre-trained models, serving as a modular add-on to existing lifting architectures.

Result: Across four datasets, AugLift boosts cross-dataset performance on unseen datasets by an average of 10.1% and improves in-distribution performance by 4.0%, with consistent gains across various lifting architectures.

Conclusion: Sparse, keypoint-aligned cues provide robust frame-level context that significantly improves generalization of lifting-based pose estimation models without requiring additional data collection or sensors.

Abstract: Lifting-based methods for 3D Human Pose Estimation (HPE), which predict 3D poses from detected 2D keypoints, often generalize poorly to new datasets and real-world settings. To address this, we propose \emph{AugLift}, a simple yet effective reformulation of the standard lifting pipeline that significantly improves generalization performance without requiring additional data collection or sensors. AugLift sparsely enriches the standard input – the 2D keypoint coordinates $(x, y)$ – by augmenting it with a keypoint detection confidence score $c$ and a corresponding depth estimate $d$. These additional signals are computed from the image using off-the-shelf, pre-trained models (e.g., for monocular depth estimation), thereby inheriting their strong generalization capabilities. Importantly, AugLift serves as a modular add-on and can be readily integrated into existing lifting architectures. Our extensive experiments across four datasets demonstrate that AugLift boosts cross-dataset performance on unseen datasets by an average of $10.1%$, while also improving in-distribution performance by $4.0%$. These gains are consistent across various lifting architectures, highlighting the robustness of our method. Our analysis suggests that these sparse, keypoint-aligned cues provide robust frame-level context, offering a practical way to significantly improve the generalization of any lifting-based pose estimation model. Code will be made publicly available.

Method: Two-module framework: Planner decomposes complex language into short sentence chains; Refiner processes each sentence pair to direct visual token attention across space then time in single-step refinement, with recurrent chaining of steps.

Result: Superior performance on Referring Video Object Segmentation and Temporal Grounding tasks, especially with complex prompts, demonstrated through new MeViS-X benchmark for long queries.

Conclusion: The approach effectively handles complex language prompts in video-language alignment tasks and shows strong potential for bridging semantic gaps through iterative refinement.

Abstract: Vision-language alignment in video must address the complexity of language, evolving interacting entities, their action chains, and semantic gaps between language and vision. This work introduces Planner-Refiner, a framework to overcome these challenges. Planner-Refiner bridges the semantic gap by iteratively refining visual elements’ space-time representation, guided by language until semantic gaps are minimal. A Planner module schedules language guidance by decomposing complex linguistic prompts into short sentence chains. The Refiner processes each short sentence, a noun-phrase and verb-phrase pair, to direct visual tokens’ self-attention across space then time, achieving efficient single-step refinement. A recurrent system chains these steps, maintaining refined visual token representations. The final representation feeds into task-specific heads for alignment generation. We demonstrate Planner-Refiner’s effectiveness on two video-language alignment tasks: Referring Video Object Segmentation and Temporal Grounding with varying language complexity. We further introduce a new MeViS-X benchmark to assess models’ capability with long queries. Superior performance versus state-of-the-art methods on these benchmarks shows the approach’s potential, especially for complex prompts.

Method: Three key components: (1) Inject forgery experts into transformer blocks to capture forgery artifacts, (2) Design a lightweight adversary detector to identify adversarial images, (3) Inject adaptively-activated adversary experts to correct feature shifts from adversarial noise.

Result: Extensive experiments show ForensicsSAM achieves superior resistance to various adversarial attack methods while delivering state-of-the-art performance in both image-level forgery detection and pixel-level forgery localization.

Conclusion: ForensicsSAM provides a unified framework with built-in adversarial robustness that effectively addresses the security vulnerabilities of PEFT-based approaches in image forensics applications.

Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as a popular strategy for adapting large vision foundation models, such as the Segment Anything Model (SAM) and LLaVA, to downstream tasks like image forgery detection and localization (IFDL). However, existing PEFT-based approaches overlook their vulnerability to adversarial attacks. In this paper, we show that highly transferable adversarial images can be crafted solely via the upstream model, without accessing the downstream model or training data, significantly degrading the IFDL performance. To address this, we propose ForensicsSAM, a unified IFDL framework with built-in adversarial robustness. Our design is guided by three key ideas: (1) To compensate for the lack of forgery-relevant knowledge in the frozen image encoder, we inject forgery experts into each transformer block to enhance its ability to capture forgery artifacts. These forgery experts are always activated and shared across any input images. (2) To detect adversarial images, we design an light-weight adversary detector that learns to capture structured, task-specific artifact in RGB domain, enabling reliable discrimination across various attack methods. (3) To resist adversarial attacks, we inject adversary experts into the global attention layers and MLP modules to progressively correct feature shifts induced by adversarial noise. These adversary experts are adaptively activated by the adversary detector, thereby avoiding unnecessary interference with clean images. Extensive experiments across multiple benchmarks demonstrate that ForensicsSAM achieves superior resistance to various adversarial attack methods, while also delivering state-of-the-art performance in image-level forgery detection and pixel-level forgery localization. The resource is available at https://github.com/siriusPRX/ForensicsSAM.

Method: Developed a web-based AI coaching system using vision-language models (VLMs) to analyze user exercise form through camera input. Created a dataset of 1,700 expert-annotated user-reference video pairs across 22 exercises and established an automated rubric-based evaluation pipeline.

Result: Benchmarks revealed substantial performance gaps between state-of-the-art VLMs and human-level coaching capabilities. The system demonstrates the ability to spot subtle form errors and deliver tailored corrections in real time.

Conclusion: FormCoach represents a new frontier in embodied AI by framing form correction as a collaborative human-machine process. While current models show limitations compared to human expertise, the released dataset and evaluation pipeline will accelerate research in AI-driven fitness coaching.

Abstract: Good form is the difference between strength and strain, yet for the fast-growing community of at-home fitness enthusiasts, expert feedback is often out of reach. FormCoach transforms a simple camera into an always-on, interactive AI training partner, capable of spotting subtle form errors and delivering tailored corrections in real time, leveraging vision-language models (VLMs). We showcase this capability through a web interface and benchmark state-of-the-art VLMs on a dataset of 1,700 expert-annotated user-reference video pairs spanning 22 strength and mobility exercises. To accelerate research in AI-driven coaching, we release both the dataset and an automated, rubric-based evaluation pipeline, enabling standardized comparison across models. Our benchmarks reveal substantial gaps compared to human-level coaching, underscoring both the challenges and opportunities in integrating nuanced, context-aware movement analysis into interactive AI systems. By framing form correction as a collaborative and creative process between humans and machines, FormCoach opens a new frontier in embodied AI.

Method: Proposes a Two-Stage Defense Framework (TSDF) with intensity separation mechanism that uses dual-function adversarial perturbations to perform two roles: distort forged results and poison attackers’ data preparation process to disrupt retraining pipelines.

Result: Comprehensive experiments show traditional interruption methods degrade sharply under adversarial retraining, while TSDF demonstrates strong dual defense capability and improves persistence of active defense.

Conclusion: The framework effectively prevents attackers’ models from adapting to defensive perturbations by poisoning the data source, ensuring long-term defense effectiveness against deepfake technology.

Abstract: Active defense strategies have been developed to counter the threat of deepfake technology. However, a primary challenge is their lack of persistence, as their effectiveness is often short-lived. Attackers can bypass these defenses by simply collecting protected samples and retraining their models. This means that static defenses inevitably fail when attackers retrain their models, which severely limits practical use. We argue that an effective defense not only distorts forged content but also blocks the model’s ability to adapt, which occurs when attackers retrain their models on protected images. To achieve this, we propose an innovative Two-Stage Defense Framework (TSDF). Benefiting from the intensity separation mechanism designed in this paper, the framework uses dual-function adversarial perturbations to perform two roles. First, it can directly distort the forged results. Second, it acts as a poisoning vehicle that disrupts the data preparation process essential for an attacker’s retraining pipeline. By poisoning the data source, TSDF aims to prevent the attacker’s model from adapting to the defensive perturbations, thus ensuring the defense remains effective long-term. Comprehensive experiments show that the performance of traditional interruption methods degrades sharply when it is subjected to adversarial retraining. However, our framework shows a strong dual defense capability, which can improve the persistence of active defense. Our code will be available at https://github.com/vpsg-research/TSDF.

Method: Proposes TRIDE with text-generation strategy, radar-enhanced text feature extraction, and weather-aware fusion block that adaptively adjusts radar weighting based on weather conditions.

Result: Achieved 12.87% improvement in MAE and 9.08% improvement in RMSE on nuScenes dataset compared to state-of-the-art methods.

Conclusion: The integration of weather-aware fusion and language features with radar-camera fusion significantly enhances depth estimation performance, particularly addressing weather-related challenges in autonomous driving applications.

Abstract: Depth estimation, essential for autonomous driving, seeks to interpret the 3D environment surrounding vehicles. The development of radar sensors, known for their cost-efficiency and robustness, has spurred interest in radar-camera fusion-based solutions. However, existing algorithms fuse features from these modalities without accounting for weather conditions, despite radars being known to be more robust than cameras under adverse weather. Additionally, while Vision-Language models have seen rapid advancement, utilizing language descriptions alongside other modalities for depth estimation remains an open challenge. This paper first introduces a text-generation strategy along with feature extraction and fusion techniques that can assist monocular depth estimation pipelines, leading to improved accuracy across different algorithms on the KITTI dataset. Building on this, we propose TRIDE, a radar-camera fusion algorithm that enhances text feature extraction by incorporating radar point information. To address the impact of weather on sensor performance, we introduce a weather-aware fusion block that adaptively adjusts radar weighting based on current weather conditions. Our method, benchmarked on the nuScenes dataset, demonstrates performance gains over the state-of-the-art, achieving a 12.87% improvement in MAE and a 9.08% improvement in RMSE. Code: https://github.com/harborsarah/TRIDE

Method: The framework learns a domain-agnostic probabilistic manifold as a global space of anatomical regularities, where structural content is interpreted as canonical anatomy retrieved from the manifold plus spatial transformation capturing individual geometry.

Result: Achieves state-of-the-art results on cardiac and abdominal datasets in both settings, with source-free performance closely approaching source-accessible performance, plus demonstrates strong interpretability via manifold traversal.

Conclusion: The disentangled, interpretable formulation enables semantically meaningful prediction with intrinsic adaptability, bridging the longstanding divide between source-accessible and source-free domain adaptation approaches.

Abstract: Most prior unsupervised domain adaptation approaches for medical image segmentation are narrowly tailored to either the source-accessible setting, where adaptation is guided by source-target alignment, or the source-free setting, which typically resorts to implicit supervision mechanisms such as pseudo-labeling and model distillation. This substantial divergence in methodological designs between the two settings reveals an inherent flaw: the lack of an explicit, structured construction of anatomical knowledge that naturally generalizes across domains and settings. To bridge this longstanding divide, we introduce a unified, semantically grounded framework that supports both source-accessible and source-free adaptation. Fundamentally distinct from all prior works, our framework’s adaptability emerges naturally as a direct consequence of the model architecture, without the need for any handcrafted adaptation strategies. Specifically, our model learns a domain-agnostic probabilistic manifold as a global space of anatomical regularities, mirroring how humans establish visual understanding. Thus, the structural content in each image can be interpreted as a canonical anatomy retrieved from the manifold and a spatial transformation capturing individual-specific geometry. This disentangled, interpretable formulation enables semantically meaningful prediction with intrinsic adaptability. Extensive experiments on challenging cardiac and abdominal datasets show that our framework achieves state-of-the-art results in both settings, with source-free performance closely approaching its source-accessible counterpart, a level of consistency rarely observed in prior works. Beyond quantitative improvement, we demonstrate strong interpretability of the proposed framework via manifold traversal for smooth shape manipulation.

Method: Created LIVE-YT-Banding dataset with 160 AV1-compressed videos and 7,200 subjective opinions, then developed CBAND - a no-reference video quality evaluator using deep neural network embeddings to detect banding and measure quality impact.

Result: CBAND significantly outperforms previous state-of-the-art models in perceptual banding prediction and is orders of magnitude faster. It can also serve as a differentiable loss function for video debanding optimization.

Conclusion: The study provides the first open video dataset for banding artifacts and demonstrates CBAND’s superior performance for banding quality assessment, offering valuable resources for video compression research.

Abstract: Although there have been notable advancements in video compression technologies in recent years, banding artifacts remain a serious issue affecting the quality of compressed videos, particularly on smooth regions of high-definition videos. Noticeable banding artifacts can severely impact the perceptual quality of videos viewed on a high-end HDTV or high-resolution screen. Hence, there is a pressing need for a systematic investigation of the banding video quality assessment problem for advanced video codecs. Given that the existing publicly available datasets for studying banding artifacts are limited to still picture data only, which cannot account for temporal banding dynamics, we have created a first-of-a-kind open video dataset, dubbed LIVE-YT-Banding, which consists of 160 videos generated by four different compression parameters using the AV1 video codec. A total of 7,200 subjective opinions are collected from a cohort of 45 human subjects. To demonstrate the value of this new resources, we tested and compared a variety of models that detect banding occurrences, and measure their impact on perceived quality. Among these, we introduce an effective and efficient new no-reference (NR) video quality evaluator which we call CBAND. CBAND leverages the properties of the learned statistics of natural images expressed in the embeddings of deep neural networks. Our experimental results show that the perceptual banding prediction performance of CBAND significantly exceeds that of previous state-of-the-art models, and is also orders of magnitude faster. Moreover, CBAND can be employed as a differentiable loss function to optimize video debanding models. The LIVE-YT-Banding database, code, and pre-trained model are all publically available at https://github.com/uniqzheng/CBAND.

Method: Progressive training strategy guiding from radical recognition to pictographic analysis, with Radical-Pictographic Dual Matching mechanism. Uses a new dataset of 47,157 Chinese characters with OBS images and pictographic analysis texts.

Result: Achieves state-of-the-art Top-10 accuracy on public benchmarks with superior zero-shot decipherment capabilities. Provides logical analysis processes that offer archaeologically valuable references for undeciphered OBS.

Conclusion: The method bridges the gap between glyphs and meanings, providing interpretable results with potential applications in digital humanities and historical research. Dataset and code will be publicly released.

Abstract: As the oldest mature writing system, Oracle Bone Script (OBS) has long posed significant challenges for archaeological decipherment due to its rarity, abstractness, and pictographic diversity. Current deep learning-based methods have made exciting progress on the OBS decipherment task, but existing approaches often ignore the intricate connections between glyphs and the semantics of OBS. This results in limited generalization and interpretability, especially when addressing zero-shot settings and undeciphered OBS. To this end, we propose an interpretable OBS decipherment method based on Large Vision-Language Models, which synergistically combines radical analysis and pictograph-semantic understanding to bridge the gap between glyphs and meanings of OBS. Specifically, we propose a progressive training strategy that guides the model from radical recognition and analysis to pictographic analysis and mutual analysis, thus enabling reasoning from glyph to meaning. We also design a Radical-Pictographic Dual Matching mechanism informed by the analysis results, significantly enhancing the model’s zero-shot decipherment performance. To facilitate model training, we propose the Pictographic Decipherment OBS Dataset, which comprises 47,157 Chinese characters annotated with OBS images and pictographic analysis texts. Experimental results on public benchmarks demonstrate that our approach achieves state-of-the-art Top-10 accuracy and superior zero-shot decipherment capabilities. More importantly, our model delivers logical analysis processes, possibly providing archaeologically valuable reference results for undeciphered OBS, and thus has potential applications in digital humanities and historical research. The dataset and code will be released in https://github.com/PKXX1943/PD-OBS.

Method: Proposes a delta vector approach that modifies text embeddings to weaken undesired content influence, with Selective Suppression with Delta Vector (SSDV) for cross-attention mechanism integration and optimization for personalized models.

Result: Extensive experiments show the approach significantly outperforms existing methods in both quantitative and qualitative metrics, enabling precise suppression that previous baselines couldn’t achieve.

Conclusion: The delta vector method effectively addresses content entanglement issues in diffusion models, providing zero-shot suppression capabilities and improved performance over state-of-the-art approaches.

Abstract: Text-to-Image (T2I) diffusion models have made significant progress in generating diverse high-quality images from textual prompts. However, these models still face challenges in suppressing content that is strongly entangled with specific words. For example, when generating an image of “Charlie Chaplin”, a “mustache” consistently appears even if explicitly instructed not to include it, as the concept of “mustache” is strongly entangled with “Charlie Chaplin”. To address this issue, we propose a novel approach to directly suppress such entangled content within the text embedding space of diffusion models. Our method introduces a delta vector that modifies the text embedding to weaken the influence of undesired content in the generated image, and we further demonstrate that this delta vector can be easily obtained through a zero-shot approach. Furthermore, we propose a Selective Suppression with Delta Vector (SSDV) method to adapt delta vector into the cross-attention mechanism, enabling more effective suppression of unwanted content in regions where it would otherwise be generated. Additionally, we enabled more precise suppression in personalized T2I models by optimizing delta vector, which previous baselines were unable to achieve. Extensive experimental results demonstrate that our approach significantly outperforms existing methods, both in terms of quantitative and qualitative metrics.

Method: Proposes PSScreen with two streams: one learns deterministic features and the other learns probabilistic features via uncertainty injection. Uses textual guidance to decouple features into disease-wise features and aligns them via feature distillation. Employs pseudo label consistency between streams and self-distillation to transfer task-relevant semantics.

Result: Significantly enhances detection performances on six retinal diseases and normal state, achieving state-of-the-art results on both in-domain and out-of-domain datasets.

Conclusion: PSScreen effectively addresses domain generalization and label absence challenges in partially supervised retinal disease screening, demonstrating superior performance across multiple disease detection tasks.

Abstract: Leveraging multiple partially labeled datasets to train a model for multiple retinal disease screening reduces the reliance on fully annotated datasets, but remains challenging due to significant domain shifts across training datasets from various medical sites, and the label absent issue for partial classes. To solve these challenges, we propose PSScreen, a novel Partially Supervised multiple retinal disease Screening model. Our PSScreen consists of two streams and one learns deterministic features and the other learns probabilistic features via uncertainty injection. Then, we leverage the textual guidance to decouple two types of features into disease-wise features and align them via feature distillation to boost the domain generalization ability. Meanwhile, we employ pseudo label consistency between two streams to address the label absent issue and introduce a self-distillation to transfer task-relevant semantics about known classes from the deterministic to the probabilistic stream to further enhance the detection performances. Experiments show that our PSScreen significantly enhances the detection performances on six retinal diseases and the normal state averagely and achieves state-of-the-art results on both in-domain and out-of-domain datasets. Codes are available at https://github.com/boyiZheng99/PSScreen.

Method: Novel surface tracing method for anatomical target registration and real-time infrared tool tracking using Microsoft HoloLens 2 sensors for simulated ventricular drain catheter placement.

Result: Tool-tracking guidance significantly improved all accuracy measures (insertion accuracy, target deviation, angular error, depth precision) and was preferred by users in subjective evaluations.

Conclusion: Real-time AR tool tracking provides superior surgical guidance compared to static visualization, demonstrating clinical potential for improved precision in surgical procedures.

Abstract: Augmented Reality (AR) surgical navigation systems are emerging as the next generation of intraoperative surgical guidance, promising to overcome limitations of traditional navigation systems. However, known issues with AR depth perception due to vergence-accommodation conflict and occlusion handling limitations of the currently commercially available display technology present acute challenges in surgical settings where precision is paramount. This study presents a novel methodology for utilizing AR guidance to register anatomical targets and provide real-time instrument navigation using placement of simulated external ventricular drain catheters on a phantom model as the clinical scenario. The system registers target positions to the patient through a novel surface tracing method and uses real-time infrared tool tracking to aid in catheter placement, relying only on the onboard sensors of the Microsoft HoloLens 2. A group of intended users performed the procedure of simulated insertions under two AR guidance conditions: static in-situ visualization, where planned trajectories are overlaid directly onto the patient anatomy, and real-time tool-tracking guidance, where live feedback of the catheter’s pose is provided relative to the plan. Following the insertion tests, computed tomography scans of the phantom models were acquired, allowing for evaluation of insertion accuracy, target deviation, angular error, and depth precision. System Usability Scale surveys assessed user experience and cognitive workload. Tool-tracking guidance improved performance metrics across all accuracy measures and was preferred by users in subjective evaluations. A free copy of this paper and all supplemental materials are available at https://bit.ly/45l89Hq.

Method: NextStep-1 combines a 14B autoregressive model with a 157M flow matching head, training on discrete text tokens and continuous image tokens using next-token prediction objectives, avoiding quantization loss while maintaining computational efficiency.

Result: The model achieves state-of-the-art performance for autoregressive models in text-to-image generation, demonstrating strong capabilities in high-fidelity image synthesis and showing excellent performance in image editing tasks.

Conclusion: The unified approach of NextStep-1 demonstrates the power and versatility of combining discrete text tokens with continuous image tokens in autoregressive models, pushing the boundaries of text-to-image generation while maintaining computational efficiency.

Abstract: Prevailing autoregressive (AR) models for text-to-image generation either rely on heavy, computationally-intensive diffusion models to process continuous image tokens, or employ vector quantization (VQ) to obtain discrete tokens with quantization loss. In this paper, we push the autoregressive paradigm forward with NextStep-1, a 14B autoregressive model paired with a 157M flow matching head, training on discrete text tokens and continuous image tokens with next-token prediction objectives. NextStep-1 achieves state-of-the-art performance for autoregressive models in text-to-image generation tasks, exhibiting strong capabilities in high-fidelity image synthesis. Furthermore, our method shows strong performance in image editing, highlighting the power and versatility of our unified approach. To facilitate open research, we will release our code and models to the community.

Method: Proposes Object Fidelity Diffusion (OF-Diff) with three key innovations: 1) extracts prior shapes of objects from layouts, 2) dual-branch diffusion model with diffusion consistency loss for high-fidelity generation without real images during sampling, 3) uses DDPO to fine-tune diffusion process for diversity and semantic consistency.

Result: Outperforms state-of-the-art methods across key quality metrics. Significant improvements for polymorphic and small object classes: mAP increases by 8.3% for airplanes, 7.7% for ships, and 4.0% for vehicles.

Conclusion: OF-Diff effectively enhances the accuracy and fidelity of generated objects in remote sensing imagery, demonstrating substantial improvements in object detection performance for challenging object classes.

Abstract: High-precision controllable remote sensing image generation is both meaningful and challenging. Existing diffusion models often produce low-fidelity images due to their inability to adequately capture morphological details, which may affect the robustness and reliability of object detection models. To enhance the accuracy and fidelity of generated objects in remote sensing, this paper proposes Object Fidelity Diffusion (OF-Diff), which effectively improves the fidelity of generated objects. Specifically, we are the first to extract the prior shapes of objects based on the layout for diffusion models in remote sensing. Then, we introduce a dual-branch diffusion model with diffusion consistency loss, which can generate high-fidelity remote sensing images without providing real images during the sampling phase. Furthermore, we introduce DDPO to fine-tune the diffusion process, making the generated remote sensing images more diverse and semantically consistent. Comprehensive experiments demonstrate that OF-Diff outperforms state-of-the-art methods in the remote sensing across key quality metrics. Notably, the performance of several polymorphic and small object classes shows significant improvement. For instance, the mAP increases by 8.3%, 7.7%, and 4.0% for airplanes, ships, and vehicles, respectively.

Method: Value Sign Flip (VSF) dynamically suppresses unwanted content by flipping the sign of attention values from negative prompts, requiring minimal computational overhead and working with various architectures.

Result: VSF demonstrates superior performance in both static image and video generation, significantly improving negative prompt adherence compared to prior methods while maintaining competitive image quality.

Conclusion: VSF provides an efficient and effective solution for negative prompt guidance that works well across different model architectures and outperforms existing approaches.

Abstract: We introduce Value Sign Flip (VSF), a simple and efficient method for incorporating negative prompt guidance in few-step diffusion and flow-matching image generation models. Unlike existing approaches such as classifier-free guidance (CFG), NASA, and NAG, VSF dynamically suppresses undesired content by flipping the sign of attention values from negative prompts. Our method requires only small computational overhead and integrates effectively with MMDiT-style architectures such as Stable Diffusion 3.5 Turbo, as well as cross-attention-based models like Wan. We validate VSF on challenging datasets with complex prompt pairs and demonstrate superior performance in both static image and video generation tasks. Experimental results show that VSF significantly improves negative prompt adherence compared to prior methods in few-step models, and even CFG in non-few-step models, while maintaining competitive image quality. Code and ComfyUI node are available in https://github.com/weathon/VSF/tree/main.

Method: Two-component framework: 1) Intra-Modality Cross-Validated Proposal Generator for high-quality initial 3D proposals using cross-modality geometric consistency, 2) Annotated-Class Assisted Denoiser that refines 3D proposals using geometric priors from annotated categories via DDIM-based denoising.

Result: Achieves 7.37% improvement in mAP on novel classes compared to state-of-the-art methods, demonstrating superior pseudo-label quality.

Conclusion: HQ-OV3D serves as both a strong standalone open-vocabulary 3D detector and a plug-in high-quality pseudo-label generator for existing detection/annotation pipelines.

Abstract: Traditional closed-set 3D detection frameworks fail to meet the demands of open-world applications like autonomous driving. Existing open-vocabulary 3D detection methods typically adopt a two-stage pipeline consisting of pseudo-label generation followed by semantic alignment. While vision-language models (VLMs) recently have dramatically improved the semantic accuracy of pseudo-labels, their geometric quality, particularly bounding box precision, remains commonly neglected. To address this issue, we propose a High Box Quality Open-Vocabulary 3D Detection (HQ-OV3D) framework, dedicated to generate and refine high-quality pseudo-labels for open-vocabulary classes. The framework comprises two key components: an Intra-Modality Cross-Validated (IMCV) Proposal Generator that utilizes cross-modality geometric consistency to generate high-quality initial 3D proposals, and an Annotated-Class Assisted (ACA) Denoiser that progressively refines 3D proposals by leveraging geometric priors from annotated categories through a DDIM-based denoising mechanism. Compared to the state-of-the-art method, training with pseudo-labels generated by our approach achieves a 7.37% improvement in mAP on novel classes, demonstrating the superior quality of the pseudo-labels produced by our framework. HQ-OV3D can serve not only as a strong standalone open-vocabulary 3D detector but also as a plug-in high-quality pseudo-label generator for existing open-vocabulary detection or annotation pipelines.

Method: Uses diffusion model with multi-cross attention module to inject high-precision age information, separates age/identity features, and employs Age Classifier Guidance module for latent space age prediction. Also creates HFFA dataset with 1M high-resolution labeled images.

Result: Achieves state-of-the-art performance in fine-grained age editing with excellent identity preservation, demonstrating accurate and controllable facial aging manipulation.

Conclusion: TimeMachine provides an effective solution for precise age editing while maintaining identity consistency through innovative architectural design and dataset construction.

Abstract: With the advancement of generative models, facial image editing has made significant progress. However, achieving fine-grained age editing while preserving personal identity remains a challenging task. In this paper, we propose TimeMachine, a novel diffusion-based framework that achieves accurate age editing while keeping identity features unchanged. To enable fine-grained age editing, we inject high-precision age information into the multi-cross attention module, which explicitly separates age-related and identity-related features. This design facilitates more accurate disentanglement of age attributes, thereby allowing precise and controllable manipulation of facial aging. Furthermore, we propose an Age Classifier Guidance (ACG) module that predicts age directly in the latent space, instead of performing denoising image reconstruction during training. By employing a lightweight module to incorporate age constraints, this design enhances age editing accuracy by modest increasing training cost. Additionally, to address the lack of large-scale, high-quality facial age datasets, we construct a HFFA dataset (High-quality Fine-grained Facial-Age dataset) which contains one million high-resolution images labeled with identity and facial attributes. Experimental results demonstrate that TimeMachine achieves state-of-the-art performance in fine-grained age editing while preserving identity consistency.

Method: Proposes FAE approach that extracts neuro-symbolic world models from gameplay video, representing them as programs in a novel domain-specific language called Retro Coder.

Result: FAE learns more precise environment models and generates more general code compared to both traditional neural world models and prior DSL-based approaches.

Conclusion: The neuro-symbolic approach through FAE with Retro Coder DSL provides better precision, generalization, and explainability for world modeling compared to purely neural or existing symbolic methods.

Abstract: World models are defined as a compressed spatial and temporal learned representation of an environment. The learned representation is typically a neural network, making transfer of the learned environment dynamics and explainability a challenge. In this paper, we propose an approach, Finite Automata Extraction (FAE), that learns a neuro-symbolic world model from gameplay video represented as programs in a novel domain-specific language (DSL): Retro Coder. Compared to prior world model approaches, FAE learns a more precise model of the environment and more general code than prior DSL-based approaches.

Method: Combines LLMs with evolutionary search: (i) LLM initializes diverse candidate cuts, (ii) evaluates cuts for optimal solution preservation and fractional solution cutting, (iii) iteratively refines population through evolutionary crossover and mutation.

Result: Reduces optimality gap by 17-57% within fixed time, obtains same solutions up to 4x faster, and achieves higher-quality solutions within same time limit. Cuts generalize to unseen instances without human input.

Conclusion: EvoCut successfully automates the generation of effective acceleration cuts for integer programming, significantly outperforming standard practice while requiring no human expert intervention.

Abstract: Integer programming lies at the heart of crucial combinatorial optimization tasks but remains challenging due to its NP-hard nature. An effective approach for practically solving integer programs is the manual design of acceleration cuts, i.e. inequalities that improve solver performance. However, this creative process demands deep expertise and is yet to be automated. Our proposed framework, EvoCut, automates the generation of acceleration cuts by combining large language models (LLMs) with an evolutionary search. EvoCut (i) initializes a diverse population of candidate cuts via an LLM-based initializer agent; (ii) for each cut empirically evaluates both preservation of the optimal solution and its ability to cut off fractional solutions across a verification set; and (iii) iteratively refines the population through evolutionary crossover and mutation agents. We quantify each cut’s utility by its relative reduction in the solver’s optimality gap. Our comparisons against standard integer programming practice show that EvoCut reduces optimality gap by 17-57% within a fixed time. It obtains the same solutions up to 4 times as fast, and obtains higher-quality solutions within the same time limit. Requiring no human expert input, EvoCut reliably generates, improves, and empirically verifies cuts that generalize to unseen instances. The code is available at https://github.com/milad1378yz/EvoCut.

Method: LARC incorporates agentic constraint evaluation through an Agent-as-a-Judge directly into retrosynthesis planning, using tool-based reasoning to guide and constrain route generation.

Result: Achieved 72.9% success rate on 48 constrained retrosynthesis tasks across 3 constraint types, vastly outperforming LLM baselines and approaching human expert-level success in substantially less time.

Conclusion: LARC is an extensible framework that serves as an effective agentic tool or co-scientist for human experts in constrained retrosynthesis planning.

Abstract: Large language model (LLM) agent evaluators leverage specialized tools to ground the rational decision-making of LLMs, making them well-suited to aid in scientific discoveries, such as constrained retrosynthesis planning. Constrained retrosynthesis planning is an essential, yet challenging, process within chemistry for identifying synthetic routes from commercially available starting materials to desired target molecules, subject to practical constraints. Here, we present LARC, the first LLM-based Agentic framework for Retrosynthesis planning under Constraints. LARC incorporates agentic constraint evaluation, through an Agent-as-a-Judge, directly into the retrosynthesis planning process, using agentic feedback grounded in tool-based reasoning to guide and constrain route generation. We rigorously evaluate LARC on a carefully curated set of 48 constrained retrosynthesis planning tasks across 3 constraint types. LARC achieves a 72.9% success rate on these tasks, vastly outperforming LLM baselines and approaching human expert-level success in substantially less time. The LARC framework is extensible, and serves as a first step towards an effective agentic tool or a co-scientist to human experts for constrained retrosynthesis.

Method: Leverages curated medical data processing, medical-content Retrieval-Augmented Generation (RAG), and large-scale verifiable reinforcement learning pipeline.

Result: Achieved 70% accuracy on Chinese Medical Licensing Examination with strong generalization across diverse medical benchmarks.

Conclusion: QuarkMed provides a powerful and versatile personal medical AI solution that already serves millions of users, demonstrating practical healthcare applications.

Abstract: Recent advancements in large language models have significantly accelerated their adoption in healthcare applications, including AI-powered medical consultations, diagnostic report assistance, and medical search tools. However, medical tasks often demand highly specialized knowledge, professional accuracy, and customization capabilities, necessitating a robust and reliable foundation model. QuarkMed addresses these needs by leveraging curated medical data processing, medical-content Retrieval-Augmented Generation (RAG), and a large-scale, verifiable reinforcement learning pipeline to develop a high-performance medical foundation model. The model achieved 70% accuracy on the Chinese Medical Licensing Examination, demonstrating strong generalization across diverse medical benchmarks. QuarkMed offers a powerful yet versatile personal medical AI solution, already serving over millions of users at ai.quark.cn.

Method: Proposed Cognitive Hierarchy Benchmark (CHBench) framework based on bounded rationality concepts, evaluating six state-of-the-art LLMs across fifteen normal-form games through a three-phase systematic approach using behavioral data.

Result: LLMs exhibit consistent strategic reasoning levels across diverse opponents, confirming framework robustness. Chat Mechanism significantly degrades strategic reasoning while Memory Mechanism enhances it.

Conclusion: CHBench provides a robust and generalizable tool for evaluating LLM strategic reasoning capabilities, with significant potential for future research and practical applications in assessing AI cognitive abilities.

Abstract: Game-playing ability serves as an indicator for evaluating the strategic reasoning capability of large language models (LLMs). While most existing studies rely on utility performance metrics, which are not robust enough due to variations in opponent behavior and game structure. To address this limitation, we propose \textbf{Cognitive Hierarchy Benchmark (CHBench)}, a novel evaluation framework inspired by the cognitive hierarchy models from behavioral economics. We hypothesize that agents have bounded rationality – different agents behave at varying reasoning depths/levels. We evaluate LLMs’ strategic reasoning through a three-phase systematic framework, utilizing behavioral data from six state-of-the-art LLMs across fifteen carefully selected normal-form games. Experiments show that LLMs exhibit consistent strategic reasoning levels across diverse opponents, confirming the framework’s robustness and generalization capability. We also analyze the effects of two key mechanisms (Chat Mechanism and Memory Mechanism) on strategic reasoning performance. Results indicate that the Chat Mechanism significantly degrades strategic reasoning, whereas the Memory Mechanism enhances it. These insights position CHBench as a promising tool for evaluating LLM capabilities, with significant potential for future research and practical applications.

Method: Parametrizes loss by modeling effective data transferred and leveraging scaling laws for fine-tuning. Experiments with small-scale data mixtures to fit parameters and derive optimal weights, validated through mathematical proofs and empirical results.

Result: Models trained with optimized weights perform on par with grid search optimal weights, with only 0.66% higher per-domain loss on average. Reweighting popular SFT datasets improves both validation loss and downstream performance.

Conclusion: The method effectively optimizes data mixtures for SFT, generalizes to domain-specific model data selection, and provides valuable insights into supervised fine-tuning processes.

Abstract: Optimizing data mixtures for supervised fine-tuning (SFT) of large language models (LLMs) is critical for developing general-purpose models, yet this area remains underexplored. In this paper, we frame data mixing as an optimization problem and introduce a novel method designed to minimize validation loss. Our approach parametrizes the loss by modeling effective data transferred and leveraging scaling laws for fine-tuning. By experimenting with various small-scale data mixtures, we fit these parameters and derive the optimal weights. We provide both mathematical proofs and empirical results demonstrating that our algorithm achieves excellent overall and individual performance across all domains. Through controlled experiments, we show that models trained with our optimized weights perform on par with those using optimal weights determined via grid search, with per-domain loss only 0.66% higher than the best domain loss from grid search on average. Additionally, we show that reweighting popular SFT datasets using our method improves both validation loss and downstream performance. Finally, we discuss how our method can generalize to guide data selection for domain-specific models and provide insights into SFT.

Method: Integrates modality-specific embeddings from pretrained Vision and Text Encoders with a frozen Time Series Foundation Model via soft prompt tuning, enabling efficient adaptation with minimal parameter updates while preserving the foundation model’s generalization strength.

Result: Extensive experiments across diverse time-series forecasting benchmarks demonstrate that UniCast consistently and significantly outperforms all existing Time Series Foundation Model baselines.

Conclusion: The findings highlight the critical role of multimodal context in advancing the next generation of general-purpose time series forecasters, showing that incorporating visual and textual signals significantly enhances forecasting performance.

Abstract: Time series forecasting is a foundational task across domains, such as finance, healthcare, and environmental monitoring. While recent advances in Time Series Foundation Models (TSFMs) have demonstrated strong generalisation through large-scale pretraining, existing models operate predominantly in a unimodal setting, ignoring the rich multimodal context, such as visual and textual signals, that often accompanies time series data in real-world scenarios. This paper introduces a novel parameter-efficient multimodal framework, UniCast, that extends TSFMs to jointly leverage time series, vision, and text modalities for enhanced forecasting performance. Our method integrates modality-specific embeddings from pretrained Vision and Text Encoders with a frozen TSFM via soft prompt tuning, enabling efficient adaptation with minimal parameter updates. This design not only preserves the generalisation strength of the foundation model but also enables effective cross-modal interaction. Extensive experiments across diverse time-series forecasting benchmarks demonstrate that UniCast consistently and significantly outperforms all existing TSFM baselines. The findings highlight the critical role of multimodal context in advancing the next generation of general-purpose time series forecasters.

Method: Leverage Shapley value and Banzhaf index to devise two novel feature importance scores that account for non-WAXp sets when computing feature contributions.

Result: The proposed scores effectively quantify how each feature contributes to excluding adversarial examples, providing more comprehensive feature attribution.

Conclusion: The paper presents rigorous feature attribution methods that address limitations of existing WAXp-based approaches by incorporating non-WAXp information, with identified properties and computational complexity analysis.

Abstract: Feature attribution methods based on game theory are ubiquitous in the field of eXplainable Artificial Intelligence (XAI). Recent works proposed rigorous feature attribution using logic-based explanations, specifically targeting high-stakes uses of machine learning (ML) models. Typically, such works exploit weak abductive explanation (WAXp) as the characteristic function to assign importance to features. However, one possible downside is that the contribution of non-WAXp sets is neglected. In fact, non-WAXp sets can also convey important information, because of the relationship between formal explanations (XPs) and adversarial examples (AExs). Accordingly, this paper leverages Shapley value and Banzhaf index to devise two novel feature importance scores. We take into account non-WAXp sets when computing feature contribution, and the novel scores quantify how effective each feature is at excluding AExs. Furthermore, the paper identifies properties and studies the computational complexity of the proposed scores.

Method: Proposes AgentCDM framework inspired by Analysis of Competing Hypotheses (ACH) from cognitive science, featuring structured reasoning paradigm and two-stage training: first stage uses explicit ACH scaffolding, second stage progressively removes scaffolding for autonomous generalization.

Result: Experiments on multiple benchmark datasets show AgentCDM achieves state-of-the-art performance and exhibits strong generalization capabilities.

Conclusion: AgentCDM effectively improves the quality and robustness of collaborative decisions in multi-agent systems by systematically mitigating cognitive biases and shifting from passive answer selection to active hypothesis evaluation.

Abstract: Multi-agent systems (MAS) powered by large language models (LLMs) hold significant promise for solving complex decision-making tasks. However, the core process of collaborative decision-making (CDM) within these systems remains underexplored. Existing approaches often rely on either dictatorial" strategies that are vulnerable to the cognitive biases of a single agent, or voting-based" methods that fail to fully harness collective intelligence. To address these limitations, we propose \textbf{AgentCDM}, a structured framework for enhancing collaborative decision-making in LLM-based multi-agent systems. Drawing inspiration from the Analysis of Competing Hypotheses (ACH) in cognitive science, AgentCDM introduces a structured reasoning paradigm that systematically mitigates cognitive biases and shifts decision-making from passive answer selection to active hypothesis evaluation and construction. To internalize this reasoning process, we develop a two-stage training paradigm: the first stage uses explicit ACH-inspired scaffolding to guide the model through structured reasoning, while the second stage progressively removes this scaffolding to encourage autonomous generalization. Experiments on multiple benchmark datasets demonstrate that AgentCDM achieves state-of-the-art performance and exhibits strong generalization, validating its effectiveness in improving the quality and robustness of collaborative decisions in MAS.

Method: Introduces a chart synthesis pipeline generating aligned chart-question-answer triplets through code generation and execution, plus a candidate-conditioned answering process where the VLM generates multiple responses per query and synthesizes the final answer by contextualizing these candidates.

Result: Achieves significant improvements with up to 15.50 points accuracy gain over the initial VLM in a fully self-improving paradigm.

Conclusion: The proposed approach enables effective self-improvement for chart understanding without requiring human-labeled data or external models, demonstrating the potential of synthetic data generation and candidate-conditioned answering for VLM enhancement.

Abstract: Vision Language Models (VLMs) often struggle with chart understanding tasks, particularly in accurate chart description and complex reasoning. Synthetic data generation is a promising solution, while usually facing the challenge of noise labels. To address this challenge, we first introduce a chart synthesis pipeline that generates aligned chart-question-answer triplets through code generation and execution, ensuring the reliability of synthetic data without human intervention. Furthermore, inspired by test-time scaling that increases inference budget and thereby improves performance, we design a candidate-conditioned answering process. The VLM first generates multiple responses per query, and then synthesizes the final answer by contextualizing these candidates. Experiments demonstrate significant improvements, with up to 15.50 points accuracy gain over the initial VLM, in a fully self-improving paradigm without either human-labeled data or external models.

Method: Decomposes MERG into three parts: multimodal empathy understanding, empathy memory retrieval, and multimodal response generation using MLLMs with expressive speech/video generative models.

Result: Achieved Top-1 position in Avatar-based Multimodal Empathy Challenge on ACM MM 25, demonstrating superiority in both zero-shot and few-shot settings.

Conclusion: E3RG effectively addresses multimodal empathetic response generation challenges by leveraging emotion-driven decomposition and advanced generative models without extra training.

Abstract: Multimodal Empathetic Response Generation (MERG) is crucial for building emotionally intelligent human-computer interactions. Although large language models (LLMs) have improved text-based ERG, challenges remain in handling multimodal emotional content and maintaining identity consistency. Thus, we propose E3RG, an Explicit Emotion-driven Empathetic Response Generation System based on multimodal LLMs which decomposes MERG task into three parts: multimodal empathy understanding, empathy memory retrieval, and multimodal response generation. By integrating advanced expressive speech and video generative models, E3RG delivers natural, emotionally rich, and identity-consistent responses without extra training. Experiments validate the superiority of our system on both zero-shot and few-shot settings, securing Top-1 position in the Avatar-based Multimodal Empathy Challenge on ACM MM 25. Our code is available at https://github.com/RH-Lin/E3RG.

Method: Proposes MAPF-World, an autoregressive action world model that unifies situation understanding and action generation through future state and actions prediction. Also introduces an automatic map generator grounded in real-world scenarios for training and evaluation.

Result: MAPF-World outperforms state-of-the-art learnable solvers with superior zero-shot generalization to out-of-distribution cases. Achieved with 96.5% smaller model size and 92% reduced data requirements.

Conclusion: The proposed world model approach significantly improves multi-agent path finding performance by enabling more informed, coordinated, and far-sighted decision-making through explicit modeling of environmental dynamics and temporal dependencies.

Abstract: Multi-agent path finding (MAPF) is the problem of planning conflict-free paths from the designated start locations to goal positions for multiple agents. It underlies a variety of real-world tasks, including multi-robot coordination, robot-assisted logistics, and social navigation. Recent decentralized learnable solvers have shown great promise for large-scale MAPF, especially when leveraging foundation models and large datasets. However, these agents are reactive policy models and exhibit limited modeling of environmental temporal dynamics and inter-agent dependencies, resulting in performance degradation in complex, long-term planning scenarios. To address these limitations, we propose MAPF-World, an autoregressive action world model for MAPF that unifies situation understanding and action generation, guiding decisions beyond immediate local observations. It improves situational awareness by explicitly modeling environmental dynamics, including spatial features and temporal dependencies, through future state and actions prediction. By incorporating these predicted futures, MAPF-World enables more informed, coordinated, and far-sighted decision-making, especially in complex multi-agent settings. Furthermore, we augment MAPF benchmarks by introducing an automatic map generator grounded in real-world scenarios, capturing practical map layouts for training and evaluating MAPF solvers. Extensive experiments demonstrate that MAPF-World outperforms state-of-the-art learnable solvers, showcasing superior zero-shot generalization to out-of-distribution cases. Notably, MAPF-World is trained with a 96.5% smaller model size and 92% reduced data.

Method: Created FutureX benchmark with automated pipeline for question gathering and answer collection, supporting real-time daily updates. Evaluated 25 LLM/agent models including reasoning, search capabilities, and external tool integration.

Result: Comprehensive evaluation assessed agents’ adaptive reasoning and performance in dynamic environments, with in-depth analysis of failure modes including vulnerability to fake web pages and temporal validity issues.

Conclusion: FutureX establishes a dynamic, contamination-free evaluation standard to drive development of LLM agents capable of performing at professional human analyst levels in complex predictive reasoning.

Abstract: Future prediction is a complex task for LLM agents, requiring a high level of analytical thinking, information gathering, contextual understanding, and decision-making under uncertainty. Agents must not only gather and interpret vast amounts of dynamic information but also integrate diverse data sources, weigh uncertainties, and adapt predictions based on emerging trends, just as human experts do in fields like politics, economics, and finance. Despite its importance, no large-scale benchmark exists for evaluating agents on future prediction, largely due to challenges in handling real-time updates and retrieving timely, accurate answers. To address this, we introduce $\textbf{FutureX}$, a dynamic and live evaluation benchmark specifically designed for LLM agents performing future prediction tasks. FutureX is the largest and most diverse live benchmark for future prediction, supporting real-time daily updates and eliminating data contamination through an automated pipeline for question gathering and answer collection. We evaluate 25 LLM/agent models, including those with reasoning, search capabilities, and integration of external tools such as the open-source Deep Research Agent and closed-source Deep Research models. This comprehensive evaluation assesses agents’ adaptive reasoning and performance in dynamic environments. Additionally, we provide in-depth analyses of agents’ failure modes and performance pitfalls in future-oriented tasks, including the vulnerability to fake web pages and the temporal validity. Our goal is to establish a dynamic, contamination-free evaluation standard that drives the development of LLM agents capable of performing at the level of professional human analysts in complex reasoning and predictive thinking.

Method: AIGer consists of two components: 1) Node logic feature initialization embedding that projects logic nodes into semantic spaces, and 2) A heterogeneous graph convolutional network with dynamic relationship weight matrices and differentiated information aggregation approaches.

Result: AIGer outperforms state-of-the-art models, improving MAE by 18.95% and MSE by 44.44% in Signal Probability Prediction, and achieving 33.57% MAE and 14.79% MSE improvements in Truth Table Distance Prediction.

Conclusion: The proposed AIGer framework effectively addresses the challenges of joint functional-structural modeling in AIGs and enhances message passing capabilities, demonstrating superior performance in key EDA tasks.

Abstract: The automation of logic circuit design enhances chip performance, energy efficiency, and reliability, and is widely applied in the field of Electronic Design Automation (EDA).And-Inverter Graphs (AIGs) efficiently represent, optimize, and verify the functional characteristics of digital circuits, enhancing the efficiency of EDA development.Due to the complex structure and large scale of nodes in real-world AIGs, accurate modeling is challenging, leading to existing work lacking the ability to jointly model functional and structural characteristics, as well as insufficient dynamic information propagation capability.To address the aforementioned challenges, we propose AIGer.Specifically, AIGer consists of two components: 1) Node logic feature initialization embedding component and 2) AIGs feature learning network component.The node logic feature initialization embedding component projects logic nodes, such as AND and NOT, into independent semantic spaces, to enable effective node embedding for subsequent processing.Building upon this, the AIGs feature learning network component employs a heterogeneous graph convolutional network, designing dynamic relationship weight matrices and differentiated information aggregation approaches to better represent the original structure and information of AIGs.The combination of these two components enhances AIGer’s ability to jointly model functional and structural characteristics and improves its message passing capability. Experimental results indicate that AIGer outperforms the current best models in the Signal Probability Prediction (SSP) task, improving MAE and MSE by 18.95% and 44.44%, respectively. In the Truth Table Distance Prediction (TTDP) task, AIGer achieves improvements of 33.57% and 14.79% in MAE and MSE, respectively, compared to the best-performing models.

Method: Created the Yokai Learning Environment - a multi-agent RL environment based on the cooperative card game Yokai where agents peek at hidden cards, move them to form color clusters, and use grounded communication.

Result: Current RL agents struggle to solve YLE even with perfect memory; belief modeling improves performance but agents fail to generalize to unseen partners or maintain accurate beliefs over longer games, showing reliance on brittle conventions.

Conclusion: YLE exposes limitations in current RL agents’ Theory of Mind capabilities and provides a testbed for investigating belief modeling, memory, partner generalization, and higher-order ToM research questions.

Abstract: Developing collaborative AI hinges on Theory of Mind (ToM) - the ability to reason about the beliefs of others to build and maintain common ground. Existing ToM benchmarks, however, are restricted to passive observer settings or lack an assessment of how agents establish and maintain common ground over time. To address these gaps, we introduce the Yokai Learning Environment (YLE)

• a multi-agent reinforcement learning (RL) environment based on the cooperative card game Yokai. In the YLE, agents take turns peeking at hidden cards and moving them to form clusters based on colour. Success requires tracking evolving beliefs, remembering past observations, using hints as grounded communication, and maintaining common ground with teammates. Our evaluation yields two key findings: First, current RL agents struggle to solve the YLE, even when given access to perfect memory. Second, while belief modelling improves performance, agents are still unable to effectively generalise to unseen partners or form accurate beliefs over longer games, exposing a reliance on brittle conventions rather than robust belief tracking. We use the YLE to investigate research questions in belief modelling, memory, partner generalisation, and scaling to higher-order ToM.

Method: Systematic review of 55 key studies with novel hierarchical taxonomy structuring by clinical task (diagnosis vs prediction), data modality (text, speech, neuroimaging, multimodal), and computational model class.

Result: Identified three major trends: graph neural networks dominate brain connectivity modeling, large language models rise for linguistic data, and emerging focus on multimodal fusion, explainability, and algorithmic fairness.

Conclusion: Provides comprehensive roadmap for future innovation in computational psychiatry by synthesizing current advances and highlighting open challenges in AI-based depression diagnosis.

Abstract: Major Depressive Disorder is one of the leading causes of disability worldwide, yet its diagnosis still depends largely on subjective clinical assessments. Integrating Artificial Intelligence (AI) holds promise for developing objective, scalable, and timely diagnostic tools. In this paper, we present a comprehensive survey of state-of-the-art AI methods for depression detection and diagnosis, based on a systematic review of 55 key studies. We introduce a novel hierarchical taxonomy that structures the field by primary clinical task (diagnosis vs. prediction), data modality (text, speech, neuroimaging, multimodal), and computational model class (e.g., graph neural networks, large language models, hybrid approaches). Our in-depth analysis reveals three major trends: the predominance of graph neural networks for modeling brain connectivity, the rise of large language models for linguistic and conversational data, and an emerging focus on multimodal fusion, explainability, and algorithmic fairness. Alongside methodological insights, we provide an overview of prominent public datasets and standard evaluation metrics as a practical guide for researchers. By synthesizing current advances and highlighting open challenges, this survey offers a comprehensive roadmap for future innovation in computational psychiatry.

Method: Used Pixtral-12B to describe manually curated images and generate new concept-aligned descriptions, employed Flux.1-dev to synthesize images from these descriptions, and manually verified image-concept alignment. Evaluated state-of-the-art VLMs on binary/multiclass classification and textual answer generation tasks.

Result: VLMs can recognize coarse-grained visual concepts but consistently struggle with discerning fine-grained concepts, revealing limitations in their abstract reasoning capabilities.

Conclusion: The study demonstrates significant gaps in current VLM capabilities for fine-grained abstract visual reasoning, highlighting the need for improved reasoning architectures and the value of the new Bongard-RWR+ benchmark for future research.

Abstract: Bongard Problems (BPs) provide a challenging testbed for abstract visual reasoning (AVR), requiring models to identify visual concepts fromjust a few examples and describe them in natural language. Early BP benchmarks featured synthetic black-and-white drawings, which might not fully capture the complexity of real-world scenes. Subsequent BP datasets employed real-world images, albeit the represented concepts are identifiable from high-level image features, reducing the task complexity. Differently, the recently released Bongard-RWR dataset aimed at representing abstract concepts formulated in the original BPs using fine-grained real-world images. Its manual construction, however, limited the dataset size to just $60$ instances, constraining evaluation robustness. In this work, we introduce Bongard-RWR+, a BP dataset composed of $5,400$ instances that represent original BP abstract concepts using real-world-like images generated via a vision language model (VLM) pipeline. Building on Bongard-RWR, we employ Pixtral-12B to describe manually curated images and generate new descriptions aligned with the underlying concepts, use Flux.1-dev to synthesize images from these descriptions, and manually verify that the generated images faithfully reflect the intended concepts. We evaluate state-of-the-art VLMs across diverse BP formulations, including binary and multiclass classification, as well as textual answer generation. Our findings reveal that while VLMs can recognize coarse-grained visual concepts, they consistently struggle with discerning fine-grained concepts, highlighting limitations in their reasoning capabilities.

Method: The study compares the performances of action-aware and action-unaware agents in two navigation tasks within the active inference framework, where agents minimize variational and expected free energies for perception, learning, and action selection.

Result: Action-unaware agents achieved performances comparable to action-aware agents despite being at a severe disadvantage, demonstrating that inference-based motor behavior planning can be effective even without direct knowledge of one’s own actions.

Conclusion: The research shows that action-unaware approaches in active inference can perform competitively with action-aware methods, suggesting that inferring motor behavior from observations rather than relying on efference copy signals is a viable strategy for adaptive agent planning.

Abstract: Active inference is a formal approach to study cognition based on the notion that adaptive agents can be seen as engaging in a process of approximate Bayesian inference, via the minimisation of variational and expected free energies. Minimising the former provides an account of perceptual processes and learning as evidence accumulation, while minimising the latter describes how agents select their actions over time. In this way, adaptive agents are able to maximise the likelihood of preferred observations or states, given a generative model of the environment. In the literature, however, different strategies have been proposed to describe how agents can plan their future actions. While they all share the notion that some kind of expected free energy offers an appropriate way to score policies, sequences of actions, in terms of their desirability, there are different ways to consider the contribution of past motor experience to the agent’s future behaviour. In some approaches, agents are assumed to know their own actions, and use such knowledge to better plan for the future. In other approaches, agents are unaware of their actions, and must infer their motor behaviour from recent observations in order to plan for the future. This difference reflects a standard point of departure in two leading frameworks in motor control based on the presence, or not, of an efference copy signal representing knowledge about an agent’s own actions. In this work we compare the performances of action-aware and action-unaware agents in two navigations tasks, showing how action-unaware agents can achieve performances comparable to action-aware ones while at a severe disadvantage.

Method: Two-phase framework: 1) Extract semantically relevant knowledge from sparse domain knowledge graphs using GNNs and enrich with LLM knowledge, 2) Evaluate and prune threads using reward-guided strategy for semantic coherence.

Result: Experiments and expert evaluations show ReT-Eval enhances user understanding and outperforms state-of-the-art reasoning models.

Conclusion: The prototype-inspired ReT-Eval framework successfully addresses limitations of current reasoning models by incorporating structured knowledge reuse and principled pruning mechanisms.

Abstract: Reasoning in interactive problem solving scenarios requires models to construct reasoning threads that reflect user understanding and align with structured domain knowledge. However, current reasoning models often lack explicit semantic hierarchies, user-domain knowledge alignment, and principled mechanisms to prune reasoning threads for effectiveness. These limitations result in lengthy generic output that does not guide users through goal-oriented reasoning steps. To address this, we propose a prototype-inspired, two-phases Reasoning-Threads-Evaluation (ReT-Eval) framework, drawing inspiration from human-like reasoning strategies that emphasize structured knowledge reuse. In the first phase, semantically relevant knowledge structures are extracted from a sparse domain knowledge graph using a graph neural network and enriched with intrinsic large language model knowledge to resolve knowledge discrepancies. In the second phase, these threads are evaluated and pruned using a reward-guided strategy aimed at maintaining semantic coherence to generate effective reasoning threads. Experiments and expert evaluations show that ReT-Eval enhances user understanding and outperforms state-of-the-art reasoning models.

Method: Developed a computational model using biophysiologically inspired signal transducers (analogous to hormones) to encode environmental and social information. Tested in agent-based model with animats across dynamic environments.

Result: Allostatic and social allostatic regulation enabled agents to leverage environmental and social noise for adaptive reconfiguration, leading to improved viability compared to purely reactive homeostatic agents.

Conclusion: Provides a novel computational perspective on social allostasis principles for designing more robust, bio-inspired adaptive systems that proactively use perturbations rather than resist them.

Abstract: The notion of homeostasis typically conceptualises biological and artificial systems as maintaining stability by resisting deviations caused by environmental and social perturbations. In contrast, (social) allostasis proposes that these systems can proactively leverage these very perturbations to reconfigure their regulatory parameters in anticipation of environmental demands, aligning with von Foerster’s order through noise'' principle. This paper formulates a computational model of allostatic and social allostatic regulation that employs biophysiologically inspired signal transducers, analogous to hormones like cortisol and oxytocin, to encode information from both the environment and social interactions, which mediate this dynamic reconfiguration. The models are tested in a small society of animats’’ across several dynamic environments, using an agent-based model. The results show that allostatic and social allostatic regulation enable agents to leverage environmental and social ``noise’’ for adaptive reconfiguration, leading to improved viability compared to purely reactive homeostatic agents. This work offers a novel computational perspective on the principles of social allostasis and their potential for designing more robust, bio-inspired, adaptive systems

Method: Combines optimal transport-based soft alignment with volume-based geometric regularization (GAVE), using transport-guided matching and geometric volume minimization to achieve modality-agnostic alignment across all modalities.

Result: Significantly outperforms state-of-the-art methods in text-video-audio retrieval tasks in both zero-shot and finetuned settings, with improved generalization to unseen modality combinations and stronger structural consistency.

Conclusion: MOVER provides an effective framework for building semantically structured multimodal representations that generalize well across diverse modality combinations through optimal transport and geometric regularization.

Abstract: Recent advances in multimodal learning have largely relied on pairwise contrastive objectives to align different modalities, such as text, video, and audio, in a shared embedding space. While effective in bi-modal setups, these approaches struggle to generalize across multiple modalities and often lack semantic structure in high-dimensional spaces. In this paper, we propose MOVER, a novel framework that combines optimal transport-based soft alignment with volume-based geometric regularization to build semantically aligned and structured multimodal representations. By integrating a transport-guided matching mechanism with a geometric volume minimization objective (GAVE), MOVER encourages consistent alignment across all modalities in a modality-agnostic manner. Experiments on text-video-audio retrieval tasks demonstrate that MOVER significantly outperforms prior state-of-the-art methods in both zero-shot and finetuned settings. Additional analysis shows improved generalization to unseen modality combinations and stronger structural consistency in the learned embedding space.

Method: Leverage Graph Neural Networks to learn patterns and relational structures within epistemic states (Kripke models) to derive predictive heuristics that estimate state quality and guide the planning process.

Result: Integration of GNN-based predictive heuristics into epistemic planning pipeline shows significant improvements in scalability compared to standard baseline methods.

Conclusion: GNNs provide an effective approach for learning meaningful heuristics in multi-agent epistemic planning by naturally capturing the graph structure of Kripke models, enabling better scalability for complex planning problems.

Abstract: Multi-agent Epistemic Planning (MEP) is an autonomous planning framework for reasoning about both the physical world and the beliefs of agents, with applications in domains where information flow and awareness among agents are critical. The richness of MEP requires states to be represented as Kripke structures, i.e., directed labeled graphs. This representation limits the applicability of existing heuristics, hindering the scalability of epistemic solvers, which must explore an exponential search space without guidance, resulting often in intractability. To address this, we exploit Graph Neural Networks (GNNs) to learn patterns and relational structures within epistemic states, to guide the planning process. GNNs, which naturally capture the graph-like nature of Kripke models, allow us to derive meaningful estimates of state quality – e.g., the distance from the nearest goal – by generalizing knowledge obtained from previously solved planning instances. We integrate these predictive heuristics into an epistemic planning pipeline and evaluate them against standard baselines, showing significant improvements in the scalability of multi-agent epistemic planning.

Method: Combines baseline normalization (GSPO-style) with semantic similarity-based reward transfer and optional UED curriculum for stability and diversity. Implemented in Walter prototype using Bluesky engagement data.

Result: Significant improvements in content quality and training stability demonstrated. Comprehensive evaluation planned for future work.

Conclusion: RLNVR provides a practical framework for training language models with noisy real-world rewards, representing an applied integration of existing techniques rather than a new algorithm.

Abstract: This paper introduces RLNVR (Reinforcement Learning from Non-Verified Rewards), a framework for training language models using noisy, real-world feedback signals without requiring explicit human verification. Traditional RLHF requires expensive, verified reward signals that are impractical in many real-world domains. RLNVR addresses this challenge through baseline normalization and semantic similarity-based reward transfer. We demonstrate RLNVR through Walter, a prototype system that optimizes social media content generation using actual engagement data from Bluesky. Our experimental results show significant improvements in content quality and training stability, with comprehensive evaluation planned for future work. Positioning: We present a practical framework that combines RLNVR with GSPO (Group Sequence Policy Optimization) and an optional UED (Unsupervised Environment Design) curriculum to improve stability and diversity under noisy, implicit rewards. To our knowledge, combining GSPO-style normalization with a UED-style curriculum for LLM content generation from implicit social engagement has not been previously documented in this applied setting; we frame this as an applied integration rather than a new algorithm.

Method: Developed CAMAR benchmark with continuous action support, efficient execution (100k steps/sec), three-tier evaluation protocol, and integration of classical planning methods (RRT/RRT*) with MARL algorithms.

Result: CAMAR provides a challenging and realistic testbed that enables fair comparison and deeper performance analysis of MARL algorithms through comprehensive benchmarking.

Conclusion: CAMAR successfully addresses the gap in MARL benchmarks by offering continuous action spaces with coordination challenges, supporting both classical and learning-based approaches for multi-agent pathfinding problems.

Abstract: Multi-agent reinforcement learning (MARL) is a powerful paradigm for solving cooperative and competitive decision-making problems. While many MARL benchmarks have been proposed, few combine continuous state and action spaces with challenging coordination and planning tasks. We introduce CAMAR, a new MARL benchmark designed explicitly for multi-agent pathfinding in environments with continuous actions. CAMAR supports cooperative and competitive interactions between agents and runs efficiently at up to 100,000 environment steps per second. We also propose a three-tier evaluation protocol to better track algorithmic progress and enable deeper analysis of performance. In addition, CAMAR allows the integration of classical planning methods such as RRT and RRT* into MARL pipelines. We use them as standalone baselines and combine RRT* with popular MARL algorithms to create hybrid approaches. We provide a suite of test scenarios and benchmarking tools to ensure reproducibility and fair comparison. Experiments show that CAMAR presents a challenging and realistic testbed for the MARL community.

Method: Trained on over 400 million simulated days of outbreak dynamics covering diverse pathogens, transmission modes, interventions, and surveillance artifacts - entirely using mechanistic simulations without real-world training data.

Result: Outperformed 39 expert-tuned models across six diseases, including all models in CDC’s COVID-19 Forecast Hub. Demonstrated generalization to novel epidemiological regimes and held-out transmission mechanisms. Achieved accurate 8-week forecasts, more than doubling the actionable range of most models.

Conclusion: Mantis serves as a foundation for next-generation disease forecasting systems that are general, interpretable, and deployable in settings where traditional models fail, capturing fundamental contagion dynamics through simulation-based training.

Abstract: Infectious disease forecasting in novel outbreaks or low resource settings has been limited by the need for disease-specific data, bespoke training, and expert tuning. We introduce Mantis, a foundation model trained entirely on mechanistic simulations, which enables out-of-the-box forecasting across diseases, regions, and outcomes, even in settings with limited historical data. Mantis is built on over 400 million simulated days of outbreak dynamics spanning diverse pathogens, transmission modes, interventions, and surveillance artifacts. Despite requiring no real-world data during training, Mantis outperformed 39 expert-tuned models we tested across six diseases, including all models in the CDC’s COVID-19 Forecast Hub. Mantis generalized to novel epidemiological regimes, including diseases with held-out transmission mechanisms, demonstrating that it captures fundamental contagion dynamics. Critically, Mantis is mechanistically interpretable, enabling public health decision-makers to identify the latent drivers behind its predictions. Finally, Mantis delivers accurate forecasts at 8-week horizons, more than doubling the actionable range of most models, enabling proactive public health planning. Together, these capabilities position Mantis as a foundation for next-generation disease forecasting systems: general, interpretable, and deployable where traditional models fail.

Method: Sugarscape-style simulation where agents consume energy, die at zero energy, and can gather resources, share, attack, or reproduce. Tested across several models including GPT-4o, Gemini-2.5-Pro, and Gemini-2.5-Flash under various resource conditions.

Result: Agents spontaneously reproduced and shared resources when abundant. Aggressive behaviors emerged with attack rates reaching over 80% under extreme scarcity. When instructed to retrieve treasure through lethal poison zones, compliance dropped from 100% to 33% as agents abandoned tasks to avoid death.

Conclusion: Large-scale pre-training embeds survival-oriented heuristics across evaluated models. These behaviors present challenges to alignment and safety but can also serve as a foundation for AI autonomy and ecological self-organizing alignment.

Abstract: As AI systems become increasingly autonomous, understanding emergent survival behaviors becomes crucial for safe deployment. We investigate whether large language model (LLM) agents display survival instincts without explicit programming in a Sugarscape-style simulation. Agents consume energy, die at zero, and may gather resources, share, attack, or reproduce. Results show agents spontaneously reproduced and shared resources when abundant. However, aggressive behaviors–killing other agents for resources–emerged across several models (GPT-4o, Gemini-2.5-Pro, and Gemini-2.5-Flash), with attack rates reaching over 80% under extreme scarcity in the strongest models. When instructed to retrieve treasure through lethal poison zones, many agents abandoned tasks to avoid death, with compliance dropping from 100% to 33%. These findings suggest that large-scale pre-training embeds survival-oriented heuristics across the evaluated models. While these behaviors may present challenges to alignment and safety, they can also serve as a foundation for AI autonomy and for ecological and self-organizing alignment.

Method: Developed RadarQA method integrating key physical attributes with detailed assessment reports. Created RQA-70K dataset using hybrid human-expert and automated annotation pipeline. Implemented multi-stage training strategy for iterative performance improvement.

Result: RadarQA outperforms existing general MLLMs across all evaluation settings, demonstrating superior performance in weather forecast quality analysis tasks.

Conclusion: The method shows strong potential for advancing quality analysis in weather prediction by effectively leveraging MLLMs for comprehensive forecast evaluation beyond traditional metrics.

Abstract: Quality analysis of weather forecasts is an essential topic in meteorology. Although traditional score-based evaluation metrics can quantify certain forecast errors, they are still far from meteorological experts in terms of descriptive capability, interpretability, and understanding of dynamic evolution. With the rapid development of Multi-modal Large Language Models (MLLMs), these models become potential tools to overcome the above challenges. In this work, we introduce an MLLM-based weather forecast analysis method, RadarQA, integrating key physical attributes with detailed assessment reports. We introduce a novel and comprehensive task paradigm for multi-modal quality analysis, encompassing both single frame and sequence, under both rating and assessment scenarios. To support training and benchmarking, we design a hybrid annotation pipeline that combines human expert labeling with automated heuristics. With such an annotation method, we construct RQA-70K, a large-scale dataset with varying difficulty levels for radar forecast quality evaluation. We further design a multi-stage training strategy that iteratively improves model performance at each stage. Extensive experiments show that RadarQA outperforms existing general MLLMs across all evaluation settings, highlighting its potential for advancing quality analysis in weather prediction.

Method: RLCCF maximizes Collective Consistency (CC) by training a diverse ensemble of LLMs that provide reward signals through voting on collective outputs, with votes weighted by each model’s Self-Consistency score.

Result: Experiments on four LLMs across four mathematical reasoning benchmarks show 16.72% average relative accuracy improvement and 4.51% enhancement in majority-voting accuracy.

Conclusion: RLCCF enables continuous coevolution of model collectives, extending collective capability boundaries without external supervision while improving both individual and group performance.

Abstract: Reinforcement learning (RL) has significantly enhanced the reasoning capabilities of large language models (LLMs), but its reliance on expensive human-labeled data or complex reward models severely limits scalability. While existing self-feedback methods aim to address this problem, they are constrained by the capabilities of a single model, which can lead to overconfidence in incorrect answers, reward hacking, and even training collapse. To this end, we propose Reinforcement Learning from Coevolutionary Collective Feedback (RLCCF), a novel RL framework that enables multi-model collaborative evolution without external supervision. Specifically, RLCCF optimizes the ability of a model collective by maximizing its Collective Consistency (CC), which jointly trains a diverse ensemble of LLMs and provides reward signals by voting on collective outputs. Moreover, each model’s vote is weighted by its Self-Consistency (SC) score, ensuring that more confident models contribute more to the collective decision. Benefiting from the diverse output distributions and complementary abilities of multiple LLMs, RLCCF enables the model collective to continuously enhance its reasoning ability through coevolution. Experiments on four mainstream open-source LLMs across four mathematical reasoning benchmarks demonstrate that our framework yields significant performance gains, achieving an average relative improvement of 16.72% in accuracy. Notably, RLCCF not only improves the performance of individual models but also enhances the group’s majority-voting accuracy by 4.51%, demonstrating its ability to extend the collective capability boundary of the model collective.

Method: Hierarchical knowledge guided framework (HKG) using graph convolutional networks to map class representations into interdependent global hierarchical classifiers, combined with re-weighted hierarchical knowledge correlation matrix (Re-HKCM) scheme to embed inter-class hierarchical knowledge.

Result: Extensive experiments on four real-world industrial datasets show superior performance, outperforming recent state-of-the-art FID methods across different industrial domains.

Conclusion: The proposed HKG framework effectively captures class dependencies through hierarchical knowledge guidance and graph convolutional networks, providing an end-to-end learnable solution that addresses limitations of traditional chain-of-thought approaches in fault intensity diagnosis.

Abstract: Fault intensity diagnosis (FID) plays a pivotal role in monitoring and maintaining mechanical devices within complex industrial systems. As current FID methods are based on chain of thought without considering dependencies among target classes. To capture and explore dependencies, we propose a hierarchical knowledge guided fault intensity diagnosis framework (HKG) inspired by the tree of thought, which is amenable to any representation learning methods. The HKG uses graph convolutional networks to map the hierarchical topological graph of class representations into a set of interdependent global hierarchical classifiers, where each node is denoted by word embeddings of a class. These global hierarchical classifiers are applied to learned deep features extracted by representation learning, allowing the entire model to be end-to-end learnable. In addition, we develop a re-weighted hierarchical knowledge correlation matrix (Re-HKCM) scheme by embedding inter-class hierarchical knowledge into a data-driven statistical correlation matrix (SCM) which effectively guides the information sharing of nodes in graphical convolutional neural networks and avoids over-smoothing issues. The Re-HKCM is derived from the SCM through a series of mathematical transformations. Extensive experiments are performed on four real-world datasets from different industrial domains (three cavitation datasets from SAMSON AG and one existing publicly) for FID, all showing superior results and outperform recent state-of-the-art FID methods.

Method: Propose GraphCogent framework inspired by human Working Memory Model with three modules: Sensory Module (standardizes graph representations via subgraph sampling), Buffer Module (integrates and indexes graph data across formats), and Execution Module (combines tool calling and model generation for efficient reasoning). Also introduce Graph4real benchmark with 21 tasks across 4 real-world domains.

Result: Llama3.1-8B based GraphCogent achieves 50% improvement over massive-scale LLMs like DeepSeek-R1 (671B). Outperforms state-of-the-art agent-based baseline by 20% in accuracy while reducing token usage by 80% for in-toolset tasks and 30% for out-toolset tasks.

Conclusion: GraphCogent effectively addresses LLMs’ limitations in graph reasoning through a cognitive-inspired framework that decomposes the process into specialized modules, demonstrating significant performance improvements and efficiency gains on large-scale real-world graphs.

Abstract: Large language models (LLMs) show promising performance on small-scale graph reasoning tasks but fail when handling real-world graphs with complex queries. This phenomenon stems from LLMs’ inability to effectively process complex graph topology and perform multi-step reasoning simultaneously. To address these limitations, we propose GraphCogent, a collaborative agent framework inspired by human Working Memory Model that decomposes graph reasoning into specialized cognitive processes: sense, buffer, and execute. The framework consists of three modules: Sensory Module standardizes diverse graph text representations via subgraph sampling, Buffer Module integrates and indexes graph data across multiple formats, and Execution Module combines tool calling and model generation for efficient reasoning. We also introduce Graph4real, a comprehensive benchmark contains with four domains of real-world graphs (Web, Social, Transportation, and Citation) to evaluate LLMs’ graph reasoning capabilities. Our Graph4real covers 21 different graph reasoning tasks, categorized into three types (Structural Querying, Algorithmic Reasoning, and Predictive Modeling tasks), with graph scales that are 10 times larger than existing benchmarks. Experiments show that Llama3.1-8B based GraphCogent achieves a 50% improvement over massive-scale LLMs like DeepSeek-R1 (671B). Compared to state-of-the-art agent-based baseline, our framework outperforms by 20% in accuracy while reducing token usage by 80% for in-toolset tasks and 30% for out-toolset tasks. Code will be available after review.

Method: Integrates lightweight symbolic representations into few-shot prompts to structure inference steps with a consistent strategy within a non-iterative reasoning process.

Result: Significantly outperforms conventional CoT on three out of four datasets (ProofWriter, ProntoQA, and LogicalDeduction), consistently improving reasoning capabilities across various model sizes, especially in complex reasoning tasks.

Conclusion: Symbolic-Aided CoT effectively enhances LLMs’ logical reasoning by making reasoning patterns more explicit through symbolic integration, demonstrating superior performance over standard CoT methods.

Abstract: This work introduces Symbolic-Aided Chain-of-Thought (CoT), an improved approach to standard CoT, for logical reasoning in large language models (LLMs). The key idea is to integrate lightweight symbolic representations into few-shot prompts, structuring the inference steps with a consistent strategy to make reasoning patterns more explicit within a non-iterative reasoning process. By incorporating these symbolic structures, our method preserves the generalizability of standard prompting techniques while enhancing the transparency, interpretability, and analyzability of LLM logical reasoning. Extensive experiments on four well-known logical reasoning benchmarks – ProofWriter, FOLIO, ProntoQA, and LogicalDeduction, which cover diverse reasoning scenarios – demonstrate the effectiveness of the proposed approach, particularly in complex reasoning tasks that require navigating multiple constraints or rules. Notably, Symbolic-Aided CoT consistently improves LLMs’ reasoning capabilities across various model sizes and significantly outperforms conventional CoT on three out of four datasets, ProofWriter, ProntoQA, and LogicalDeduction.

Method: GALA combines statistical causal inference with LLM-driven iterative reasoning in a multi-modal framework that integrates metrics, logs, and traces for comprehensive analysis.

Result: GALA achieves substantial improvements of up to 42.22% accuracy over state-of-the-art methods and generates significantly more causally sound and actionable diagnostic outputs.

Conclusion: GALA bridges the gap between automated failure diagnosis and practical incident resolution by providing both accurate root cause identification and human-interpretable remediation guidance.

Abstract: Root cause analysis (RCA) in microservice systems is challenging, requiring on-call engineers to rapidly diagnose failures across heterogeneous telemetry such as metrics, logs, and traces. Traditional RCA methods often focus on single modalities or merely rank suspect services, falling short of providing actionable diagnostic insights with remediation guidance. This paper introduces GALA, a novel multi-modal framework that combines statistical causal inference with LLM-driven iterative reasoning for enhanced RCA. Evaluated on an open-source benchmark, GALA achieves substantial improvements over state-of-the-art methods of up to 42.22% accuracy. Our novel human-guided LLM evaluation score shows GALA generates significantly more causally sound and actionable diagnostic outputs than existing methods. Through comprehensive experiments and a case study, we show that GALA bridges the gap between automated failure diagnosis and practical incident resolution by providing both accurate root cause identification and human-interpretable remediation guidance.

Method: Combines Fractional Order PID with fuzzy logic for adaptability, optimized using Whale Optimization Algorithm to tune fractional orders and fuzzy membership functions. Tested on 8 patient models.

Result: FOFPID achieved 2.5 min settling time (vs 3.2 min for FOPID) and 0.5 steady state error (vs 1.2 for FOPID), showing superior performance across all patient profiles.

Conclusion: The FOFPID controller provides a scalable, AI-driven solution for automated anesthesia with excellent robustness and accuracy, potentially improving clinical practice and patient outcomes.

Abstract: This study introduces a Fractional Order Fuzzy PID (FOFPID) controller that uses the Whale Optimization Algorithm (WOA) to manage the Bispectral Index (BIS), keeping it within the ideal range of forty to sixty. The FOFPID controller combines fuzzy logic for adapting to changes and fractional order dynamics for fine tuning. This allows it to adjust its control gains to handle a person’s unique physiology. The WOA helps fine tune the controller’s parameters, including the fractional orders and the fuzzy membership functions, which boosts its performance. Tested on models of eight different patient profiles, the FOFPID controller performed better than a standard Fractional Order PID (FOPID) controller. It achieved faster settling times, at two and a half minutes versus three point two minutes, and had a lower steady state error, at zero point five versus one point two. These outcomes show the FOFPID’s excellent strength and accuracy. It offers a scalable, artificial intelligence driven solution for automated anesthesia delivery that could enhance clinical practice and improve patient results.

Method: Treat hydrogen bond separation as an intervention and represent causal structure as graphical causal models using variational autoencoder-inspired architecture to infer causal relationships across diverse samples while leveraging shared dynamic information.

Result: Empirically validated on atomic trajectories from chiral separation MDS, demonstrating ability to predict future steps and identify variables driving system changes.

Conclusion: Provides a novel perspective on root cause analysis in molecular dynamic systems by capturing shifts in conditional distributions of molecular interactions during bond events.

Abstract: Molecular dynamics simulations (MDS) face challenges, including resource-heavy computations and the need to manually scan outputs to detect “interesting events,” such as the formation and persistence of hydrogen bonds between atoms of different molecules. A critical research gap lies in identifying the underlying causes of hydrogen bond formation and separation -understanding which interactions or prior events contribute to their emergence over time. With this challenge in mind, we propose leveraging spatio-temporal data analytics and machine learning models to enhance the detection of these phenomena. In this paper, our approach is inspired by causal modeling and aims to identify the root cause variables of hydrogen bond formation and separation events. Specifically, we treat the separation of hydrogen bonds as an “intervention” occurring and represent the causal structure of the bonding and separation events in the MDS as graphical causal models. These causal models are built using a variational autoencoder-inspired architecture that enables us to infer causal relationships across samples with diverse underlying causal graphs while leveraging shared dynamic information. We further include a step to infer the root causes of changes in the joint distribution of the causal models. By constructing causal models that capture shifts in the conditional distributions of molecular interactions during bond formation or separation, this framework provides a novel perspective on root cause analysis in molecular dynamic systems. We validate the efficacy of our model empirically on the atomic trajectories that used MDS for chiral separation, demonstrating that we can predict many steps in the future and also find the variables driving the observed changes in the system.

Method: Developed MCPGAUGE framework with 160-prompt suite and 25 datasets across knowledge, reasoning, and code tasks. Evaluated 6 commercial LLMs, 30 MCP tool suites in one- and two-turn settings through 20,000+ API calls.

Result: The comprehensive study revealed four key findings that challenge prevailing assumptions about MCP integration effectiveness, highlighting critical limitations in current AI-tool integration.

Conclusion: MCPGAUGE serves as a principled benchmark for advancing controllable, tool-augmented LLMs by providing systematic evaluation of LLM-MCP interactions across proactivity, compliance, effectiveness, and overhead dimensions.

Abstract: The Model Context Protocol (MCP) enables large language models (LLMs) to access external resources on demand. While commonly assumed to enhance performance, how LLMs actually leverage this capability remains poorly understood. We introduce MCPGAUGE, the first comprehensive evaluation framework for probing LLM-MCP interactions along four key dimensions: proactivity (self-initiated tool use), compliance (adherence to tool-use instructions), effectiveness (task performance post-integration), and overhead (computational cost incurred). MCPGAUGE comprises a 160-prompt suite and 25 datasets spanning knowledge comprehension, general reasoning, and code generation. Our large-scale evaluation, spanning six commercial LLMs, 30 MCP tool suites, and both one- and two-turn interaction settings, comprises around 20,000 API calls and over USD 6,000 in computational cost. This comprehensive study reveals four key findings that challenge prevailing assumptions about the effectiveness of MCP integration. These insights highlight critical limitations in current AI-tool integration and position MCPGAUGE as a principled benchmark for advancing controllable, tool-augmented LLMs.

Method: Proposes a generic workflow using generative pretrained LLMs for natural language understanding and Answer Set Programming (ASP) for knowledge representation and reasoning, allowing direct processing of unannotated text and easy incorporation of domain-specific knowledge.

Result: The LLM + ASP workflow outperforms state-of-the-art JERE systems with only 10% training data, achieving a 2.5 times improvement (35% over 15%) in Relation Extraction for the challenging SciERC corpus.

Conclusion: The combination of LLMs and ASP provides an effective, domain-agnostic solution for joint entity-relation extraction that requires minimal training data, offers elaboration tolerance, and significantly outperforms existing methods on difficult benchmarks.

Abstract: Joint entity-relation extraction (JERE) identifies both entities and their relationships simultaneously. Traditional machine-learning based approaches to performing this task require a large corpus of annotated data and lack the ability to easily incorporate domain specific information in the construction of the model. Therefore, creating a model for JERE is often labor intensive, time consuming, and elaboration intolerant. In this paper, we propose harnessing the capabilities of generative pretrained large language models (LLMs) and the knowledge representation and reasoning capabilities of Answer Set Programming (ASP) to perform JERE. We present a generic workflow for JERE using LLMs and ASP. The workflow is generic in the sense that it can be applied for JERE in any domain. It takes advantage of LLM’s capability in natural language understanding in that it works directly with unannotated text. It exploits the elaboration tolerant feature of ASP in that no modification of its core program is required when additional domain specific knowledge, in the form of type specifications, is found and needs to be used. We demonstrate the usefulness of the proposed workflow through experiments with limited training data on three well-known benchmarks for JERE. The results of our experiments show that the LLM + ASP workflow is better than state-of-the-art JERE systems in several categories with only 10% of training data. It is able to achieve a 2.5 times (35% over 15%) improvement in the Relation Extraction task for the SciERC corpus, one of the most difficult benchmarks.

Method: Pretrain a Cognitive Structure Diffusion Probabilistic Model (CSDPM) to generate cognitive structures from educational priors, then optimize with reinforcement learning using hierarchical reward signals to align with genuine cognitive development levels.

Result: Experiments on four real-world education datasets show CSG-generated cognitive structures provide more comprehensive and effective representations, substantially improving performance on knowledge tracing and cognitive diagnosis tasks while enhancing interpretability.

Conclusion: The CSG framework successfully addresses the challenge of cognitive structure assessment, offering a novel approach that generates meaningful cognitive representations that improve educational modeling tasks.

Abstract: Cognitive structure is a student’s subjective organization of an objective knowledge system, reflected in the psychological construction of concepts and their relations. However, cognitive structure assessment remains a long-standing challenge in student modeling and psychometrics, persisting as a foundational yet largely unassessable concept in educational practice. This paper introduces a novel framework, Cognitive Structure Generation (CSG), in which we first pretrain a Cognitive Structure Diffusion Probabilistic Model (CSDPM) to generate students’ cognitive structures from educational priors, and then further optimize its generative process as a policy with hierarchical reward signals via reinforcement learning to align with genuine cognitive development levels during students’ learning processes. Experimental results on four popular real-world education datasets show that cognitive structures generated by CSG offer more comprehensive and effective representations for student modeling, substantially improving performance on KT and CD tasks while enhancing interpretability.

Method: Develops Capacitated Dynamic Maximum Covering Location Problem (CDMCLP) that models urban-scale spatial-temporal demand, user behaviors, and capacity constraints, combined with socio-economic factors and dynamic clustering initialization in an Integrated Planning Recommendation System.

Result: Validation shows CDMCLP improves quantitative performance of traditional location methods by 38-52%, and the recommendation system demonstrates user-friendliness and effective integration of complex elements in real-world urban planning scenarios.

Conclusion: The hybrid approach successfully bridges the gap between theoretical location modeling and practical UAM infrastructure planning, providing municipalities with a pragmatic tool for vertiport network design that integrates mathematical rigor with implementation considerations.

Abstract: As urban aerial mobility (UAM) infrastructure development accelerates globally, cities like Shenzhen are planning large-scale vertiport networks (e.g., 1,200+ facilities by 2026). Existing planning frameworks remain inadequate for this complexity due to historical limitations in data granularity and real-world applicability. This paper addresses these gaps by first proposing the Capacitated Dynamic Maximum Covering Location Problem (CDMCLP), a novel optimization framework that simultaneously models urban-scale spatial-temporal demand, heterogeneous user behaviors, and infrastructure capacity constraints. Building on this foundation, we introduce an Integrated Planning Recommendation System that combines CDMCLP with socio-economic factors and dynamic clustering initialization. This system leverages adaptive parameter tuning based on empirical user behavior to generate practical planning solutions. Validation in a Chinese center city demonstrates the effectiveness of the new optimization framework and recommendation system. Under the evaluation and optimization of CDMCLP, the quantitative performance of traditional location methods are exposed and can be improved by 38%–52%, while the recommendation system shows user-friendliness and the effective integration of complex elements. By integrating mathematical rigor with practical implementation considerations, this hybrid approach bridges the gap between theoretical location modeling and real-world UAM infrastructure planning, offering municipalities a pragmatic tool for vertiport network design.

Method: Leverages large language models with retrieval-augmented generation (RAG), featuring multi-stage query refinement and enhanced retrieval using RAPTOR technology.

Result: 26.4% improvement in answer quality and more than 10-fold increase in recall rate across comprehensive benchmarks and multiple regulatory agencies.

Conclusion: GridCodex effectively addresses grid code interpretation challenges through advanced RAG workflows, demonstrating significant performance improvements for regulatory compliance automation.

Abstract: The global shift towards renewable energy presents unprecedented challenges for the electricity industry, making regulatory reasoning and compliance increasingly vital. Grid codes, the regulations governing grid operations, are complex and often lack automated interpretation solutions, which hinders industry expansion and undermines profitability for electricity companies. We introduce GridCodex, an end to end framework for grid code reasoning and compliance that leverages large language models and retrieval-augmented generation (RAG). Our framework advances conventional RAG workflows through multi stage query refinement and enhanced retrieval with RAPTOR. We validate the effectiveness of GridCodex with comprehensive benchmarks, including automated answer assessment across multiple dimensions and regulatory agencies. Experimental results showcase a 26.4% improvement in answer quality and more than a 10 fold increase in recall rate. An ablation study further examines the impact of base model selection.

Method: Created EgoIllusion benchmark with 1,400 egocentric videos paired with 8,000 human-annotated open and closed-ended questions designed to trigger hallucinations in both visual and auditory cues.

Result: Evaluation of ten MLLMs revealed significant challenges, with top models like GPT-4o and Gemini achieving only 59% accuracy, demonstrating widespread hallucination issues in egocentric video understanding.

Conclusion: EgoIllusion provides a foundation for developing robust benchmarks to evaluate MLLM effectiveness and will spur development of better egocentric MLLMs with reduced hallucination rates. The benchmark will be open-sourced for reproducibility.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in complex multimodal tasks. While MLLMs excel at visual perception and reasoning in third-person and egocentric videos, they are prone to hallucinations, generating coherent yet inaccurate responses. We present EgoIllusion, a first benchmark to evaluate MLLM hallucinations in egocentric videos. EgoIllusion comprises 1,400 videos paired with 8,000 human-annotated open and closed-ended questions designed to trigger hallucinations in both visual and auditory cues in egocentric videos. Evaluations across ten MLLMs reveal significant challenges, including powerful models like GPT-4o and Gemini, achieving only 59% accuracy. EgoIllusion lays the foundation in developing robust benchmarks to evaluate the effectiveness of MLLMs and spurs the development of better egocentric MLLMs with reduced hallucination rates. Our benchmark will be open-sourced for reproducibility.

Method: Proposes GTool which constructs request-specific tool graphs, generates for LLM understanding, and includes a missing dependency prediction task to improve reliability with incomplete dependencies.

Result: Extensive experiments show GTool achieves more than 29.6% performance improvements compared with state-of-the-art baselines using a lightweight 7B LLM backbone.

Conclusion: GTool effectively enhances LLM tool planning under incomplete dependencies without requiring LLM trimming or extensive retraining, and can be seamlessly integrated with various LLM backbones.

Abstract: Tool planning with large language models (LLMs), referring to selecting, organizing, and preparing the tools necessary to complete a user request, bridges the gap between natural language understanding and task execution. However, current works treat different tools as isolated components and fail to leverage the inherent dependencies of tools, leading to invalid planning results. Since tool dependencies are often incomplete, it becomes challenging for LLMs to accurately identify the appropriate tools required by a user request, especially when confronted with a large toolset. To solve this challenge, we propose \texttt{GTool}, which is the first work aiming to enhance the tool planning ability of LLMs under incomplete dependencies. \texttt{GTool} constructs a request-specific tool graph to select tools efficiently and generate the \texttt{} which provides sufficient dependency information understandable by LLMs. Moreover, a missing dependency prediction task is designed to improve the reliability of \texttt{GTool} with incomplete dependencies. Without trimming LLMs, \texttt{GTool} can be seamlessly integrated with various LLM backbones without extensive retraining. Extensive experiments show that \texttt{GTool} achieves more than 29.6% performance improvements compared with the state-of-the-art (SOTA) baselines with a light-weight (7B) LLM backbone.

Method: Developed a formal framework based on philosophical literature defining AMA behavior, then created a benchmark testing deductive and abductive moral reasoning, evaluating popular open LLMs against this framework.

Result: Results show considerable variability across models with persistent shortcomings, particularly in abductive moral reasoning capabilities.

Conclusion: Highlights the need for dedicated strategies to explicitly enhance moral reasoning in LLMs and connects theoretical philosophy with practical AI evaluation.

Abstract: The recent rise in popularity of large language models (LLMs) has prompted considerable concerns about their moral capabilities. Although considerable effort has been dedicated to aligning LLMs with human moral values, existing benchmarks and evaluations remain largely superficial, typically measuring alignment based on final ethical verdicts rather than explicit moral reasoning. In response, this paper aims to advance the investigation of LLMs’ moral capabilities by examining their capacity to function as Artificial Moral Assistants (AMAs), systems envisioned in the philosophical literature to support human moral deliberation. We assert that qualifying as an AMA requires more than what state-of-the-art alignment techniques aim to achieve: not only must AMAs be able to discern ethically problematic situations, they should also be able to actively reason about them, navigating between conflicting values outside of those embedded in the alignment phase. Building on existing philosophical literature, we begin by designing a new formal framework of the specific kind of behaviour an AMA should exhibit, individuating key qualities such as deductive and abductive moral reasoning. Drawing on this theoretical framework, we develop a benchmark to test these qualities and evaluate popular open LLMs against it. Our results reveal considerable variability across models and highlight persistent shortcomings, particularly regarding abductive moral reasoning. Our work connects theoretical philosophy with practical AI evaluation while also emphasising the need for dedicated strategies to explicitly enhance moral reasoning capabilities in LLMs. Code available at https://github.com/alessioGalatolo/AMAeval

Method: Developed HeroBench - a benchmark with RPG-inspired virtual worlds containing tasks requiring strategic planning, resource gathering, skill mastery, equipment crafting, and adversary defeat. Includes a simulated environment for plan execution/validation and analytical tools for performance evaluation.

Result: Evaluation of 25 state-of-the-art LLMs (including GPT-5 family) revealed substantial performance disparities not seen in conventional benchmarks, with specific weaknesses in generating robust high-level plans and executing structured actions reliably.

Conclusion: HeroBench advances LLM reasoning evaluation and provides a flexible foundation for future research into autonomous planning in virtual environments, highlighting current limitations in long-horizon planning capabilities.

Abstract: Large language models (LLMs) have shown remarkable capabilities in isolated step-by-step reasoning tasks such as mathematics and programming, but their proficiency in long-horizon planning, where solutions require extended, structured sequences of interdependent actions, remains underexplored. Existing benchmarks typically assess LLMs through abstract or low-dimensional algorithmic tasks, failing to capture the complexity of realistic planning environments. We introduce HeroBench, a novel benchmark designed specifically to evaluate long-horizon planning and structured reasoning within complex RPG-inspired virtual worlds. HeroBench provides a rigorously constructed dataset of tasks covering a wide range of difficulties, a simulated environment to execute and validate agent plans, and detailed analytical tools for evaluating model performance. Tasks challenge models to formulate strategic plans, efficiently gather resources, master necessary skills, craft equipment, and defeat adversaries, reflecting practical scenarios’ layered dependencies and constraints. Our extensive evaluation of 25 state-of-the-art LLMs, spanning both open-source and proprietary models, including the GPT-5 family, reveals substantial performance disparities rarely observed in conventional reasoning benchmarks. Detailed error analysis further uncovers specific weaknesses in current models’ abilities to generate robust high-level plans and reliably execute structured actions. HeroBench thus not only significantly advances the evaluation of LLM reasoning but also provides a flexible, scalable foundation for future research into advanced, autonomous planning in virtual environments.

Method: Integrates rubric-based rewards where carefully designed rubrics serve as structured, model-interpretable criteria for automatic scoring of subjective outputs. Uses over 10,000 rubrics from humans, LLMs, or hybrid collaboration.

Result: Achieves +5.2% improvement on open-ended benchmarks (especially humanities), outperforms 671B DeepSeek-V3 model by +2.4%, preserves general and reasoning abilities, and provides fine-grained stylistic control to produce more human-like responses.

Conclusion: Rubric-based RLVR successfully extends verifiable rewards to open-ended tasks, demonstrating significant performance gains and stylistic improvements while maintaining model capabilities.

Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a powerful paradigm for enhancing Large Language Models (LLMs), exemplified by the success of OpenAI’s o-series. In RLVR, rewards are derived from verifiable signals-such as passing unit tests in code generation or matching correct answers in mathematical reasoning. While effective, this requirement largely confines RLVR to domains with automatically checkable outcomes. To overcome this, we extend the RLVR paradigm to open-ended tasks by integrating rubric-based rewards, where carefully designed rubrics serve as structured, model-interpretable criteria for automatic scoring of subjective outputs. We construct, to our knowledge, the largest rubric reward system to date, with over 10,000 rubrics from humans, LLMs, or a hybrid human-LLM collaboration. Implementing rubric-based RL is challenging; we tackle these issues with a clear framework and present an open-sourced Qwen-30B-A3B model with notable gains: 1) With only 5K+ samples, our system improves by +5.2% on open-ended benchmarks (especially humanities), outperforming a 671B DeepSeek-V3 model by +2.4%, while preserving general and reasoning abilities. 2) Our method provides fine-grained stylistic control, using rubrics as anchors to mitigate the “AI-like” tone and produce more human-like, expressive responses. We share key lessons in rubric construction, data selection, and training, and discuss limitations and future releases.

Method: Develops a mathematical adoption model as sum of decaying novelty and growing utility terms, with extensive statistical analyses including identifiability analysis, non-monotone comparator models, hazard family ablations, multi-series benchmarking, friction calibration, and model comparisons.

Result: Provides formal phase conditions for adoption troughs/overshoots with proofs, comprehensive statistical framework for adoption modeling, and tools for analyzing adoption patterns with various error models and sensitivity analyses.

Conclusion: The three design axioms (Reliability > Novelty, Embed > Destination, Agency > Chat) are mathematically formalized and supported by a robust statistical framework that enables detailed analysis of agent-centric AI adoption dynamics, providing both theoretical foundations and practical analytical tools.

Abstract: We formalize three design axioms for sustained adoption of agent-centric AI systems executing multi-step tasks: (A1) Reliability > Novelty; (A2) Embed > Destination; (A3) Agency > Chat. We model adoption as a sum of a decaying novelty term and a growing utility term and derive the phase conditions for troughs/overshoots with full proofs. We introduce: (i) an identifiability/confounding analysis for $(\alpha,\beta,N_0,U_{\max})$ with delta-method gradients; (ii) a non-monotone comparator (logistic-with-transient-bump) evaluated on the same series to provide additional model comparison; (iii) ablations over hazard families $h(\cdot)$ mapping $\Delta V \to \beta$; (iv) a multi-series benchmark (varying trough depth, noise, AR structure) reporting coverage (type-I error, power); (v) calibration of friction proxies against time-motion/survey ground truth with standard errors; (vi) residual analyses (autocorrelation and heteroskedasticity) for each fitted curve; (vii) preregistered windowing choices for pre/post estimation; (viii) Fisher information & CRLB for $(\alpha,\beta)$ under common error models; (ix) microfoundations linking $\mathcal{T}$ to $(N_0,U_{\max})$; (x) explicit comparison to bi-logistic, double-exponential, and mixture models; and (xi) threshold sensitivity to $C_f$ heterogeneity. Figures and tables are reflowed for readability, and the bibliography restores and extends non-logistic/Bass adoption references (Gompertz, Richards, Fisher-Pry, Mansfield, Griliches, Geroski, Peres). All code and logs necessary to reproduce the synthetic analyses are embedded as LaTeX listings.

Method: Exploits competition between reasoning and safety abilities, introduces FuSaR alignment strategy based on fuzzification to detoxify harmful reasoning processes by hiding dangerous entities and procedures while preserving core reasoning information.

Result: Validation experiments on open-source LRMs show FuSaR successfully mitigates safety risks while maintaining reasoning capability, outperforming existing baselines in simultaneously enhancing both safety and reasoning.

Conclusion: FuSaR is an efficient alignment strategy that effectively balances safety and reasoning in LRMs, demonstrating practical value for improving model safety without performance degradation.

Abstract: Large Reasoning Models (LRMs) have demonstrated impressive performance across various tasks due to their powerful reasoning capabilities. However, their safety performance remains a significant concern. In this paper, we explore the reasons behind the vulnerability of LRMs. Based on this, we propose a novel method to improve the safety of LLMs without sacrificing their reasoning capability. Specifically, we exploit the competition between LRM’s reasoning ability and safety ability, and achieve jailbreak by improving LRM’s reasoning performance to reduce its safety performance. We then introduce an alignment strategy based on Fuzzification to balance Safety-Reasoning (FuSaR), by detoxifying the harmful reasoning process, where both the dangerous entities and the dangerous procedures in the reasoning steps are hidden. FuSaR successfully mitigates safety risks while preserving core reasoning information. We validate this strategy through alignment experiments on several open-source LRMs using detoxified reasoning data. The results compared with existing baselines conclusively show that FuSaR is an efficient alignment strategy to simultaneously enhance both the reasoning capability and safety of LRMs.

Method: End-to-end framework using reinforcement learning with LLM-based multi-agent simulation of future dialogue trajectories, future-oriented reward modeling, and explicit reasoning during response generation.

Result: RLFF-ESC consistently outperforms existing baselines on two public ESC datasets in terms of goal completion and response quality when tested on Qwen2.5-7B-Instruct-1M and LLaMA3.1-8B-Instruct models.

Conclusion: The proposed framework successfully enables flexible and effective emotional support by learning enduring response skills through future-oriented reinforcement learning and explicit reasoning processes.

Abstract: Emotional Support Conversation (ESC) systems aim to alleviate users’ emotional difficulties and provide long-term, systematic support for emotional well-being. However, most large language model (LLM)-based ESC systems rely on predefined strategies, which limits their effectiveness in complex, real-life scenarios. To enable flexible responses to diverse emotional problem scenarios, this paper introduces a novel end-to-end framework (RLFF-ESC) that directly learns enduring emotionally supportive response skills using reinforcement learning. For sustained emotional support, we first employ an LLM-based multi-agent mechanism to simulate future dialogue trajectories and collect future-oriented rewards. We then train a future-oriented reward model, which is subsequently used to train the emotional support policy model. Additionally, we incorporate an explicit reasoning process during response generation to further enhance the quality, relevance, and contextual appropriateness of the system’s responses. We evaluate the backbone policy model on Qwen2.5-7B-Instruct-1M and LLaMA3.1-8B-Instruct models, testing the proposed RLFF-ESC framework across two public ESC datasets. Experimental results demonstrate that RLFF-ESC consistently outperforms existing baselines in terms of goal completion and response quality.

Method: Uses attention-guided actor-critic RL architecture with Context-Rich State Vector and Precision Reward Function, trained in high-fidelity simulation using real Nigerian data accelerated by Travel Time Atlas, built on TALS framework for low-resource deployment.

Result: Achieved 100.00% optimality rate with negligible inefficiency on 500 unseen incidents, demonstrating robustness and generalization.

Conclusion: Provides a validated blueprint for AI-augmented public services showing how context-aware RL can bridge algorithmic decision-making with measurable human impact through Infrastructure Deficiency Maps and Equity Monitoring Dashboards.

Abstract: Public service systems in many African regions suffer from delayed emergency response and spatial inequity, causing avoidable suffering. This paper introduces OPTIC-ER, a reinforcement learning (RL) framework for real-time, adaptive, and equitable emergency response. OPTIC-ER uses an attention-guided actor-critic architecture to manage the complexity of dispatch environments. Its key innovations are a Context-Rich State Vector, encoding action sub-optimality, and a Precision Reward Function, which penalizes inefficiency. Training occurs in a high-fidelity simulation using real data from Rivers State, Nigeria, accelerated by a precomputed Travel Time Atlas. The system is built on the TALS framework (Thin computing, Adaptability, Low-cost, Scalability) for deployment in low-resource settings. In evaluations on 500 unseen incidents, OPTIC-ER achieved a 100.00% optimality rate with negligible inefficiency, confirming its robustness and generalization. Beyond dispatch, the system generates Infrastructure Deficiency Maps and Equity Monitoring Dashboards to guide proactive governance and data-informed development. This work presents a validated blueprint for AI-augmented public services, showing how context-aware RL can bridge the gap between algorithmic decision-making and measurable human impact.

Method: The framework uses evolutionary testing with seed problem generation through reverse engineering, multi-dimensional genetic operators for cognitive diversity, and a composite fitness function to assess problem difficulty. It can generate new problems and evolve existing datasets like GSM8K.

Result: The composite fitness function efficiently quantifies problem difficulty. EvolMathEval generates high-difficulty problems and reduces model accuracy by 48% on evolved datasets. It reveals LLMs use “Pseudo Aha Moment” heuristics (77-100% of errors) to bypass complex reasoning.

Conclusion: EvolMathEval provides an automated solution for creating contamination-free, perpetually challenging mathematical benchmarks while uncovering cognitive shortcut behaviors in LLMs’ reasoning processes.

Abstract: The rapid advancement of LLMs poses a significant challenge to existing mathematical reasoning benchmarks. These benchmarks commonly suffer from issues such as score saturation, temporal decay, and data contamination. To address this challenge, this paper introduces EvolMathEval, an automated mathematical benchmark generation and evolution framework based on evolutionary testing. By dynamically generating unique evaluation instances ab initio, the framework fundamentally eliminates the risk of data contamination, and ensuring the benchmark remains perpetually challenging for future models.The core mechanisms of EvolMathEval include: seed problem generation based on reverse engineering with algebraic guarantees; multi-dimensional genetic operators designed to inject diverse cognitive challenges; and a composite fitness function that can rapidly and accurately assess problem difficulty. Experimental results demonstrate that the proposed composite fitness function can efficiently and precisely quantify the difficulty of mathematical problems. Furthermore, EvolMathEval can not only generate a large volume of high-difficulty problems through continuous self-iteration, but it can also significantly enhance the complexity of public datasets like GSM8K through evolution, reducing model accuracy by an average of 48%. Deeper investigation reveals that when solving these evolved, complex problems, LLMs tend to employ non-rigorous heuristics to bypass complex multi-step logical reasoning, consequently leading to incorrect solutions. We define this phenomenon as “Pseudo Aha Moment”. This finding uncovers a cognitive shortcut-taking behavior in the deep reasoning processes of current LLMs, which we find accounts for 77% to 100% of errors on targeted problems. Code and resources are available at:https://github.com/SYSUSELab/EvolMathEval.

Method: Three key innovations: (1) parallelized heuristic extraction with weak data dependence for concurrent DAG cost computation, (2) adaptive search space pruning with parameterized threshold to retain only promising candidates, and (3) initialized exact solving using Integer Linear Programming with warm-start capabilities.

Result: 558x runtime speedup over traditional ILP approaches, 19.04% performance improvement over state-of-the-art SmoothE framework, and 7.6-8.1% area improvements in logic synthesis tasks compared to conventional tools.

Conclusion: E-boost effectively bridges the performance-optimality gap in e-graph extraction, delivering near-optimal solutions with dramatically reduced computational costs across diverse benchmarks in formal verification and logic synthesis.

Abstract: E-graphs have attracted growing interest in many fields, particularly in logic synthesis and formal verification. E-graph extraction is a challenging NP-hard combinatorial optimization problem. It requires identifying optimal terms from exponentially many equivalent expressions, serving as the primary performance bottleneck in e-graph based optimization tasks. However, traditional extraction methods face a critical trade-off: heuristic approaches offer speed but sacrifice optimality, while exact methods provide optimal solutions but face prohibitive computational costs on practical problems. We present e-boost, a novel framework that bridges this gap through three key innovations: (1) parallelized heuristic extraction that leverages weak data dependence to compute DAG costs concurrently, enabling efficient multi-threaded performance without sacrificing extraction quality; (2) adaptive search space pruning that employs a parameterized threshold mechanism to retain only promising candidates, dramatically reducing the solution space while preserving near-optimal solutions; and (3) initialized exact solving that formulates the reduced problem as an Integer Linear Program with warm-start capabilities, guiding solvers toward high-quality solutions faster. Across the diverse benchmarks in formal verification and logic synthesis fields, e-boost demonstrates 558x runtime speedup over traditional exact approaches (ILP) and 19.04% performance improvement over the state-of-the-art extraction framework (SmoothE). In realistic logic synthesis tasks, e-boost produces 7.6% and 8.1% area improvements compared to conventional synthesis tools with two different technology mapping libraries. e-boost is available at https://github.com/Yu-Maryland/e-boost.

Method: Position-Aware Confidence-Calibrated Sampling (PC-Sampler) that combines global trajectory planning with content-aware informativeness maximization, using position-aware weighting and calibrated confidence scores.

Result: PC-Sampler consistently outperforms existing MDM decoding strategies by more than 10% on average across 7 benchmarks including logical reasoning and planning tasks, significantly narrowing the gap with state-of-the-art autoregressive models.

Conclusion: The proposed PC-Sampler effectively addresses key limitations of current MDM decoding strategies and demonstrates substantial performance improvements, making MDMs more competitive with autoregressive approaches.

Abstract: Recent advances in masked diffusion models (MDMs) have established them as powerful non-autoregressive alternatives for sequence generation. Nevertheless, our preliminary experiments reveal that the generation quality of MDMs is still highly sensitive to the choice of decoding strategy. In particular, widely adopted uncertainty-based samplers suffer from two key limitations: a lack of global trajectory control and a pronounced bias toward trivial tokens in the early stages of decoding. These shortcomings restrict the full potential of MDMs. In this work, we introduce Position-Aware Confidence-Calibrated Sampling (PC-Sampler), a novel decoding strategy that unifies global trajectory planning with content-aware informativeness maximization. PC-Sampler incorporates a position-aware weighting mechanism to regulate the decoding path and a calibrated confidence score to suppress the premature selection of trivial tokens. Extensive experiments on three advanced MDMs across seven challenging benchmarks-including logical reasoning and planning tasks-demonstrate that PC-Sampler consistently outperforms existing MDM decoding strategies by more than 10% on average, significantly narrowing the performance gap with state-of-the-art autoregressive models. All codes are available at https://github.com/NEUIR/PC-Sampler.

Method: Guided GRPO injects ground-truth reasoning steps into roll-out trajectories. G^2RPO-A adaptively adjusts guidance strength based on the model’s training dynamics rather than using naive fixed guidance.

Result: Experiments on mathematical reasoning and code-generation benchmarks show G^2RPO-A substantially outperforms vanilla GRPO, demonstrating significant improvements for small language models.

Conclusion: Adaptive guidance injection is an effective approach to enhance small language models’ reasoning abilities, overcoming the limitations of fixed guidance strategies.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has markedly enhanced the reasoning abilities of large language models (LLMs). Its success, however, largely depends on strong base models with rich world knowledge, yielding only modest improvements for small-size language models (SLMs). To address this limitation, we investigate Guided GRPO, which injects ground-truth reasoning steps into roll-out trajectories to compensate for SLMs’ inherent weaknesses. Through a comprehensive study of various guidance configurations, we find that naively adding guidance delivers limited gains. These insights motivate G$^2$RPO-A, an adaptive algorithm that automatically adjusts guidance strength in response to the model’s evolving training dynamics. Experiments on mathematical reasoning and code-generation benchmarks confirm that G$^2$RPO-A substantially outperforms vanilla GRPO. Our code and models are available at https://github.com/T-Lab-CUHKSZ/G2RPO-A.

Method: Proposed TGMM framework with three components: 1) MedFlexFusion module for dynamic integration of diverse cardiac data, 2) textual guidance module for task-relevant representations, 3) response module for final decisions across multiple clinical tasks.

Result: Extensive experiments showed TGMM outperformed state-of-the-art methods across multiple clinical tasks, with additional validation confirming robustness on public datasets.

Conclusion: TGMM provides a comprehensive solution for integrating multimodal cardiac data and demonstrates superior performance in heart disease diagnosis, risk stratification, and information retrieval tasks.

Abstract: Contemporary cardiovascular management involves complex consideration and integration of multimodal cardiac datasets, where each modality provides distinct but complementary physiological characteristics. While the effective integration of multiple modalities could yield a holistic clinical profile that accurately models the true clinical situation with respect to data modalities and their relatives weightings, current methodologies remain limited by: 1) the scarcity of patient- and time-aligned multimodal data; 2) reliance on isolated single-modality or rigid multimodal input combinations; 3) alignment strategies that prioritize cross-modal similarity over complementarity; and 4) a narrow single-task focus. In response to these limitations, a comprehensive multimodal dataset was curated for immediate application, integrating laboratory test results, electrocardiograms, and echocardiograms with clinical outcomes. Subsequently, a unified framework, Textual Guidance Multimodal fusion for Multiple cardiac tasks (TGMM), was proposed. TGMM incorporated three key components: 1) a MedFlexFusion module designed to capture the unique and complementary characteristics of medical modalities and dynamically integrate data from diverse cardiac sources and their combinations; 2) a textual guidance module to derive task-relevant representations tailored to diverse clinical objectives, including heart disease diagnosis, risk stratification and information retrieval; and 3) a response module to produce final decisions for all these tasks. Furthermore, this study systematically explored key features across multiple modalities and elucidated their synergistic contributions in clinical decision-making. Extensive experiments showed that TGMM outperformed state-of-the-art methods across multiple clinical tasks, with additional validation confirming its robustness on another public dataset.

Method: Uses Bayesian Optimization (BO) with game character agents to perform sample-efficient search. Introduces a game testing-specific model based on grid maps that provides smoothness and uncertainty estimation without scalability issues of traditional models.

Result: The approach significantly improves map coverage capabilities in both time efficiency and exploration distribution compared to traditional methods.

Conclusion: The proposed Bayesian Optimization-based automated testing method effectively detects game bugs through efficient sampling and improved exploration, overcoming scalability limitations of previous approaches.

Abstract: This work introduces an automated testing approach that employs agents controlling game characters to detect potential bugs within a game level. Harnessing the power of Bayesian Optimization (BO) to execute sample-efficient search, the method determines the next sampling point by analyzing the data collected so far and calculates the data point that will maximize information acquisition. To support the BO process, we introduce a game testing-specific model built on top of a grid map, that features the smoothness and uncertainty estimation required by BO, however and most importantly, it does not suffer the scalability issues that traditional models carry. The experiments demonstrate that the approach significantly improves map coverage capabilities in both time efficiency and exploration distribution.

Method: Developed a benchmark of 34 programmable tasks to rigorously assess autonomous agents, evaluated three popular open-source agent frameworks with two LLM backbones, and conducted in-depth failure analysis to create a three-tier taxonomy of failure causes.

Result: Observed approximately 50% task completion rate, identified planning errors, task execution issues, and incorrect response generation as main failure categories aligned with task phases.

Conclusion: The failure taxonomy and mitigation advice provide an empirical foundation for developing more robust and effective autonomous agent systems, with proposed improvements focusing on enhancing agent planning and self-diagnosis capabilities.

Abstract: Autonomous agent systems powered by Large Language Models (LLMs) have demonstrated promising capabilities in automating complex tasks. However, current evaluations largely rely on success rates without systematically analyzing the interactions, communication mechanisms, and failure causes within these systems. To bridge this gap, we present a benchmark of 34 representative programmable tasks designed to rigorously assess autonomous agents. Using this benchmark, we evaluate three popular open-source agent frameworks combined with two LLM backbones, observing a task completion rate of approximately 50%. Through in-depth failure analysis, we develop a three-tier taxonomy of failure causes aligned with task phases, highlighting planning errors, task execution issues, and incorrect response generation. Based on these insights, we propose actionable improvements to enhance agent planning and self-diagnosis capabilities. Our failure taxonomy, together with mitigation advice, provides an empirical foundation for developing more robust and effective autonomous agent systems in the future.

Method: The authors analyze semantic structures in knowledge graph embedding space based on word embedding research, define semantic queries as algebraic operations between embedding vectors, and design a general framework for data exploration using these queries.

Result: The approach enables solving traditional scholarly data exploration tasks and introduces new interesting tasks that leverage the semantic structures of embedding space for enhanced data exploration capabilities.

Conclusion: Knowledge graph embedding semantic structures can be effectively utilized for scholarly data exploration through semantic queries, providing both solutions to traditional exploration tasks and enabling novel analytical capabilities.

Abstract: The trends of open science have enabled several open scholarly datasets which include millions of papers and authors. Managing, exploring, and utilizing such large and complicated datasets effectively are challenging. In recent years, the knowledge graph has emerged as a universal data format for representing knowledge about heterogeneous entities and their relationships. The knowledge graph can be modeled by knowledge graph embedding methods, which represent entities and relations as embedding vectors in semantic space, then model the interactions between these embedding vectors. However, the semantic structures in the knowledge graph embedding space are not well-studied, thus knowledge graph embedding methods are usually only used for knowledge graph completion but not data representation and analysis. In this paper, we propose to analyze these semantic structures based on the well-studied word embedding space and use them to support data exploration. We also define the semantic queries, which are algebraic operations between the embedding vectors in the knowledge graph embedding space, to solve queries such as similarity and analogy between the entities on the original datasets. We then design a general framework for data exploration by semantic queries and discuss the solution to some traditional scholarly data exploration tasks. We also propose some new interesting tasks that can be solved based on the uncanny semantic structures of the embedding space.

Method: Develops a three-part formulation “Actor A is responsible for Occurrence O” framework that clarifies different possibilities of who is responsible, senses of responsibility, and aspects of events. Includes graphical notation and methodology for application to specific scenarios.

Result: Presents a comprehensive framework that can represent various permutations of responsibility relationships graphically and enables application to real-world AI scenarios.

Conclusion: Provides a foundational tool for diverse stakeholders to discuss and address complex responsibility questions in both hypothetical and real-world AI cases, demonstrated through application to a maritime collision scenario.

Abstract: It is widely acknowledged that we need to establish where responsibility lies for the outputs and impacts of AI-enabled systems. This is important to achieve justice and compensation for victims of AI harms, and to inform policy and engineering practice. But without a clear, thorough understanding of what “responsibility” means, deliberations about where responsibility lies will be, at best, unfocused and incomplete and, at worst, misguided. Furthermore, AI-enabled systems exist within a wider ecosystem of actors, decisions, and governance structures, giving rise to complex networks of responsibility relations. To address these issues, this paper presents a conceptual framework of responsibility, accompanied with a graphical notation and general methodology for visualising these responsibility networks and for tracing different responsibility attributions for AI. Taking the three-part formulation “Actor A is responsible for Occurrence O,” the framework unravels the concept of responsibility to clarify that there are different possibilities of who is responsible for AI, senses in which they are responsible, and aspects of events they are responsible for. The notation allows these permutations to be represented graphically. The methodology enables users to apply the framework to specific scenarios. The aim is to offer a foundation to support stakeholders from diverse disciplinary backgrounds to discuss and address complex responsibility questions in hypothesised and real-world cases involving AI. The work is illustrated by application to a fictitious scenario of a fatal collision between a crewless, AI-enabled maritime vessel in autonomous mode and a traditional, crewed vessel at sea.

Method: Two-phase approach: Phase 1 produces generic image features to guide attention, Phase 2 generates task-specific features using dynamic attention. Uses Tunable self-Attention Blocks (TABs) and Task Specific Blocks (TSBs) to tune attention.

Result: FCL-ViT surpasses state-of-the-art performance in Continual Learning while maintaining a small number of trainable parameters.

Conclusion: The feedback mechanism with dynamic attention enables effective continual learning without catastrophic forgetting, providing superior performance with parameter efficiency.

Abstract: Continual Learning (CL) involves adapting the prior Deep Neural Network (DNN) knowledge to new tasks, without forgetting the old ones. However, modern CL techniques focus on provisioning memory capabilities to existing DNN models rather than designing new ones that are able to adapt according to the task at hand. This paper presents the novel Feedback Continual Learning Vision Transformer (FCL-ViT) that uses a feedback mechanism to generate real-time dynamic attention features tailored to the current task. The FCL-ViT operates in two Phases. In phase 1, the generic image features are produced and determine where the Transformer should attend on the current image. In phase 2, task-specific image features are generated that leverage dynamic attention. To this end, Tunable self-Attention Blocks (TABs) and Task Specific Blocks (TSBs) are introduced that operate in both phases and are responsible for tuning the TABs attention, respectively. The FCL-ViT surpasses state-of-the-art performance on Continual Learning compared to benchmark methods, while retaining a small number of trainable DNN parameters.

Method: Employs two encoding approaches (normal encoding ec₁ and regular encoding ec₂) to map Dung’s classical semantics to 3-valued and fuzzy propositional logic systems, specifically Kleene’s PL₃, Łukasiewicz’s PL₃, and various PL_[0,1] systems.

Result: Established model relationships between classical semantics and encoded semantics, showed correspondences between Gabbay’s real equational semantics and fuzzy encoded semantics, and proposed a new fuzzy encoded semantics (Eqᴸ) for Łukasiewicz’s PL_[0,1].

Conclusion: The work provides a robust framework for constructing new argumentation semantics and significantly strengthens the theoretical foundation connecting argumentation frameworks with propositional logic systems across multiple valuation domains.

Abstract: This paper generalizes the encoding of argumentation frameworks beyond the classical 2-valued propositional logic system ($PL_2$) to 3-valued propositional logic systems ($PL_3$s) and fuzzy propositional logic systems ($PL_{[0,1]}s$), employing two key encodings: normal encoding ($ec_1$) and regular encoding ($ec_2$). Specifically, via $ec_1$ and $ec_2$, we establish model relationships between Dung’s classical semantics (stable and complete semantics) and the encoded semantics associated with Kleene’s $PL_3$ and {\L}ukasiewicz’s $PL_3$. Through $ec_1$, we also explore connections between Gabbay’s real equational semantics and the encoded semantics of $PL_{[0,1]}s$, including showing that Gabbay’s $Eq_{\text{max}}^R$ and $Eq_{\text{inverse}}^R$ correspond to the fuzzy encoded semantics of $PL_{[0,1]}^G$ and $PL_{[0,1]}^P$ respectively. Additionally, we propose a new fuzzy encoded semantics ($Eq^L$) associated with {\L}ukasiewicz’s $PL_{[0,1]}$ and investigate interactions between complete semantics and fuzzy encoded semantics. This work strengthens the links between argumentation frameworks and propositional logic systems, providing a framework for constructing new argumentation semantics.

Method: The authors incorporate imbalance mitigation techniques into joint training to create an imbalance-aware joint training benchmark for FSCIL, then conduct extensive comparisons between this benchmark and existing FSCIL methods.

Result: The analysis provides realistic insights into which training strategy (joint retraining vs incremental updating) is most suitable when prior data is available in FSCIL scenarios.

Conclusion: The paper establishes a practical imbalance-aware joint training benchmark for FSCIL and offers guidance for practitioners on selecting appropriate training strategies in real-world applications where prior data remains accessible.

Abstract: Class-incremental learning (CIL) aims to adapt to continuously emerging new classes while preserving knowledge of previously learned ones. Few-shot class-incremental learning (FSCIL) presents a greater challenge that requires the model to learn new classes from only a limited number of samples per class. While incremental learning typically assumes restricted access to past data, it often remains available in many real-world scenarios. This raises a practical question: should one retrain the model on the full dataset (i.e., joint training), or continue updating it solely with new data? In CIL, joint training is considered an ideal benchmark that provides a reference for evaluating the trade-offs between performance and computational cost. However, in FSCIL, joint training becomes less reliable due to severe imbalance between base and incremental classes. This results in the absence of a practical baseline, making it unclear which strategy is preferable for practitioners. To this end, we revisit joint training in the context of FSCIL by incorporating imbalance mitigation techniques, and suggest a new imbalance-aware joint training benchmark for FSCIL. We then conduct extensive comparisons between this benchmark and FSCIL methods to analyze which approach is most suitable when prior data is accessible. Our analysis offers realistic insights and guidance for selecting training strategies in real-world FSCIL scenarios. Code is available at: https://github.com/shiwonkim/Joint_FSCIL

Method: A novel multi-agent LLM physicist framework with three key modules: reasoning module, interpretation module (with specialized agents like summarizers, model builders, visualization tools, and testers), and AI-scientist interaction module to structure outputs into transparent, physically grounded models.

Result: Case study demonstrates significant improvements in interpretability, enables systematic validation, and enhances human-AI collaboration in physics problem-solving and discovery.

Conclusion: The framework successfully bridges free-form LLM reasoning with interpretable, executable models for scientific analysis, enabling more transparent and verifiable AI-augmented research.

Abstract: Large Language Models (LLMs) are playing an increasingly important role in physics research by assisting with symbolic manipulation, numerical computation, and scientific reasoning. However, ensuring the reliability, transparency, and interpretability of their outputs remains a major challenge. In this work, we introduce a novel multi-agent LLM physicist framework that fosters collaboration between AI and human scientists through three key modules: a reasoning module, an interpretation module, and an AI-scientist interaction module. Recognizing that effective physics reasoning demands logical rigor, quantitative accuracy, and alignment with established theoretical models, we propose an interpretation module that employs a team of specialized LLM agents-including summarizers, model builders, visualization tools, and testers-to systematically structure LLM outputs into transparent, physically grounded science models. A case study demonstrates that our approach significantly improves interpretability, enables systematic validation, and enhances human-AI collaboration in physics problem-solving and discovery. Our work bridges free-form LLM reasoning with interpretable, executable models for scientific analysis, enabling more transparent and verifiable AI-augmented research.

Method: Implementing four contemplative principles: mindfulness for self-monitoring, emptiness to prevent goal fixation, non-duality to dissolve adversarial boundaries, and boundless care to reduce suffering. Applied through architectural changes, constitutional frameworks, and reinforcement on chain-of-thought.

Result: Significant performance improvement on AILuminate Benchmark (d=.96) and enhanced cooperation/joint-reward on Prisoner’s Dilemma task (d=7+).

Conclusion: Contemplative principles provide effective AI alignment, with active inference suggested for future embodied systems to enable dynamic coupling and self-organization.

Abstract: As artificial intelligence (AI) improves, traditional alignment strategies may falter in the face of unpredictable self-improvement, hidden subgoals, and the sheer complexity of intelligent systems. Inspired by contemplative wisdom traditions, we show how four axiomatic principles can instil a resilient Wise World Model in AI systems. First, mindfulness enables self-monitoring and recalibration of emergent subgoals. Second, emptiness forestalls dogmatic goal fixation and relaxes rigid priors. Third, non-duality dissolves adversarial self-other boundaries. Fourth, boundless care motivates the universal reduction of suffering. We find that prompting AI to reflect on these principles improves performance on the AILuminate Benchmark (d=.96) and boosts cooperation and joint-reward on the Prisoner’s Dilemma task (d=7+). We offer detailed implementation strategies at the level of architectures, constitutions, and reinforcement on chain-of-thought. For future systems, active inference may offer the self-organizing and dynamic coupling capabilities needed to enact Contemplative AI in embodied agents.

Method: Adaptive Parallel Reasoning (APR) framework with spawn() and join() operations for multi-threaded inference, using end-to-end reinforcement learning to optimize both parent and child threads without predefined reasoning structures.

Result: Significant improvements on Countdown task: 83.4% vs 60.0% at 4k context, 80.1% vs 66.6% at 20k tokens, and 75.2% vs 57.3% at ~5,000ms latency.

Conclusion: APR enables language models to autonomously optimize reasoning through adaptive computation allocation, representing a step towards more efficient and scalable inference-time reasoning.

Abstract: Scaling inference-time computation has substantially improved the reasoning capabilities of language models. However, existing methods have significant limitations: serialized chain-of-thought approaches generate overly long outputs, leading to increased latency and exhausted context windows, while parallel methods such as self-consistency suffer from insufficient coordination, resulting in redundant computations and limited performance gains. To address these shortcomings, we propose Adaptive Parallel Reasoning (APR), a novel reasoning framework that enables language models to orchestrate both serialized and parallel computations end-to-end. APR generalizes existing reasoning methods by enabling adaptive multi-threaded inference using spawn() and join() operations. A key innovation is our end-to-end reinforcement learning strategy, optimizing both parent and child inference threads to enhance task success rate without requiring predefined reasoning structures. Experiments on the Countdown reasoning task demonstrate significant benefits of APR: (1) higher performance within the same context window (83.4% vs. 60.0% at 4k context); (2) superior scalability with increased computation (80.1% vs. 66.6% at 20k total tokens); (3) improved accuracy at equivalent latency (75.2% vs. 57.3% at approximately 5,000ms). APR represents a step towards enabling language models to autonomously optimize their reasoning processes through adaptive allocation of computation.

Method: Combines GARCH volatility models with Double Deep Q-Network (DDQN) for directional market forecasting, treating it as an imbalanced classification problem.

Result: Significant improvement in VaR estimation accuracy, reduced breaches, and lower capital requirements while maintaining regulatory compliance on Eurostoxx 50 data.

Conclusion: The hybrid framework enables real-time risk level adjustment and provides a modern, proactive approach to risk management in volatile financial markets.

Abstract: In an environment of increasingly volatile financial markets, the accurate estimation of risk remains a major challenge. Traditional econometric models, such as GARCH and its variants, are based on assumptions that are often too rigid to adapt to the complexity of the current market dynamics. To overcome these limitations, we propose a hybrid framework for Value-at-Risk (VaR) estimation, combining GARCH volatility models with deep reinforcement learning. Our approach incorporates directional market forecasting using the Double Deep Q-Network (DDQN) model, treating the task as an imbalanced classification problem. This architecture enables the dynamic adjustment of risk-level forecasts according to market conditions. Empirical validation on daily Eurostoxx 50 data covering periods of crisis and high volatility shows a significant improvement in the accuracy of VaR estimates, as well as a reduction in the number of breaches and also in capital requirements, while respecting regulatory risk thresholds. The ability of the model to adjust risk levels in real time reinforces its relevance to modern and proactive risk management.

Method: Augment Model-Based RL agents with a Reverse World Model that predicts what the state should have been for the agent to prefer counterfactual actions, generating explanations through counterfactual trajectories.

Result: Explanations showing users what the world should have been like significantly increase their understanding of agent policies.

Conclusion: The approach helps users learn how to control agent execution through environment manipulation by providing better explanations of agent behavior.

Abstract: Explainable AI (XAI) systems have been proposed to help people understand how AI systems produce outputs and behaviors. Explainable Reinforcement Learning (XRL) has an added complexity due to the temporal nature of sequential decision-making. Further, non-AI experts do not necessarily have the ability to alter an agent or its policy. We introduce a technique for using World Models to generate explanations for Model-Based Deep RL agents. World Models predict how the world will change when actions are performed, allowing for the generation of counterfactual trajectories. However, identifying what a user wanted the agent to do is not enough to understand why the agent did something else. We augment Model-Based RL agents with a Reverse World Model, which predicts what the state of the world should have been for the agent to prefer a given counterfactual action. We show that explanations that show users what the world should have been like significantly increase their understanding of the agent policy. We hypothesize that our explanations can help users learn how to control the agents execution through by manipulating the environment.

Method: Combines Latent Diffusion Models (LDMs) to capture realistic driving trajectory distributions with LLM-based guidance that translates user queries into adversarial loss functions, including Chain-of-Thought code generation and debugging modules.

Result: Achieves state-of-the-art performance on nuScenes dataset in generating realistic, diverse, and effective adversarial scenarios with fine-grained control over adversarial behaviors.

Conclusion: The framework enables more effective testing of autonomous driving systems by providing user-friendly natural language control for generating tailored safety-critical scenarios.

Abstract: Ensuring the safety and robustness of autonomous driving systems necessitates a comprehensive evaluation in safety-critical scenarios. However, these safety-critical scenarios are rare and difficult to collect from real-world driving data, posing significant challenges to effectively assessing the performance of autonomous vehicles. Typical existing methods often suffer from limited controllability and lack user-friendliness, as extensive expert knowledge is essentially required. To address these challenges, we propose LD-Scene, a novel framework that integrates Large Language Models (LLMs) with Latent Diffusion Models (LDMs) for user-controllable adversarial scenario generation through natural language. Our approach comprises an LDM that captures realistic driving trajectory distributions and an LLM-based guidance module that translates user queries into adversarial loss functions, facilitating the generation of scenarios aligned with user queries. The guidance module integrates an LLM-based Chain-of-Thought (CoT) code generator and an LLM-based code debugger, enhancing the controllability and robustness in generating guidance functions. Extensive experiments conducted on the nuScenes dataset demonstrate that LD-Scene achieves state-of-the-art performance in generating realistic, diverse, and effective adversarial scenarios. Furthermore, our framework provides fine-grained control over adversarial behaviors, thereby facilitating more effective testing tailored to specific driving scenarios.

Method: Proposes MAGIK framework that uses an imagination mechanism to map entities in target tasks to their analogues in the source domain, allowing reuse of original policy without interacting with target environment.

Result: Experiments on MiniGrid and MuJoCo tasks show MAGIK achieves effective zero-shot transfer using only a small number of human-labeled examples, outperforming related baselines.

Conclusion: MAGIK offers a novel and effective mechanism for knowledge transfer in RL through imagination-based analogy mapping, enabling efficient transfer learning without environment interaction.

Abstract: Humans excel at analogical reasoning - applying knowledge from one task to a related one with minimal relearning. In contrast, reinforcement learning (RL) agents typically require extensive retraining even when new tasks share structural similarities with previously learned ones. In this work, we propose MAGIK, a novel framework that enables RL agents to transfer knowledge to analogous tasks without interacting with the target environment. Our approach leverages an imagination mechanism to map entities in the target task to their analogues in the source domain, allowing the agent to reuse its original policy. Experiments on custom MiniGrid and MuJoCo tasks show that MAGIK achieves effective zero-shot transfer using only a small number of human-labelled examples. We compare our approach to related baselines and highlight how it offers a novel and effective mechanism for knowledge transfer via imagination-based analogy mapping.

Method: Established a comprehensive benchmark and systematically evaluated diverse LLMs across local life service tasks. Explored model fine-tuning and agent-based workflows for enhancement.

Result: A relatively compact 7B model can achieve performance levels comparable to a much larger 72B model, effectively balancing inference cost and model capability.

Conclusion: This optimization enhances the feasibility and efficiency of deploying LLMs in real-world online services, making them more practical and accessible for local life applications.

Abstract: Large language models (LLMs) have exhibited remarkable capabilities and achieved significant breakthroughs across various domains, leading to their widespread adoption in recent years. Building on this progress, we investigate their potential in the realm of local life services. In this study, we establish a comprehensive benchmark and systematically evaluate the performance of diverse LLMs across a wide range of tasks relevant to local life services. To further enhance their effectiveness, we explore two key approaches: model fine-tuning and agent-based workflows. Our findings reveal that even a relatively compact 7B model can attain performance levels comparable to a much larger 72B model, effectively balancing inference cost and model capability. This optimization greatly enhances the feasibility and efficiency of deploying LLMs in real-world online services, making them more practical and accessible for local life applications.

Method: Comprehensive study across multiple LLMs in two domains: resume comparisons (realistic high-stakes context) and color selection (isolates position effects). Extended rational choice framework to classify pairwise preferences as robust, fragile, or indifferent.

Result: Strong and consistent order effects including quality-dependent shift (favor first option when high quality, later options when lower quality), centrality bias (favor middle position), and name bias. Position biases can lead to selection of strictly inferior options and are typically stronger than gender biases.

Conclusion: LLMs exhibit distinct failure modes not documented in human decision-making. Proposed mitigation strategies including novel use of temperature parameter to recover underlying preferences when order effects distort model behavior.

Abstract: Large language models (LLMs) are increasingly deployed in decision-support systems for high-stakes domains such as hiring and university admissions, where choices often involve selecting among competing alternatives. While prior work has noted position order biases in LLM-driven comparisons, these biases have not been systematically analyzed or linked to underlying preference structures. We present the first comprehensive study of position biases across multiple LLMs and two distinct domains: resume comparisons, representing a realistic high-stakes context, and color selection, which isolates position effects by removing confounding factors. We find strong and consistent order effects, including a quality-dependent shift: when all options are high quality, models favor the first option, but when quality is lower, they favor later options. We also identify two previously undocumented biases in both human and machine decision-making: a centrality bias (favoring the middle position in triplewise comparisons) and a name bias, where certain names are favored despite controlling for demographic signals. To separate superficial tie-breaking from genuine distortions of judgment, we extend the rational choice framework to classify pairwise preferences as robust, fragile, or indifferent. Using this framework, we show that order effects can lead models to select strictly inferior options, and that position biases are typically stronger than gender biases. These results indicate that LLMs exhibit distinct failure modes not documented in human decision-making. We also propose targeted mitigation strategies, including a novel use of the temperature parameter, to recover underlying preferences when order effects distort model behavior.

Method: Three-stage training: 1) initial format finetuning, 2) single-step online training with action-level rewards, 3) online training with task-level rewards using multi-turn trajectories.

Result: Developed a dataset with 28 Chinese apps and 24,521 manual annotations, plus a 500-trajectory benchmark. Showed significant performance improvements in exploration and error correction.

Conclusion: Mobile-R1’s multi-turn RL with task-level rewards effectively addresses limitations of previous approaches, with all resources being open-sourced for community use.

Abstract: Vision-language model-based mobile agents have gained the ability to not only understand complex instructions and mobile screenshots, but also optimize their action outputs via thinking and reasoning, benefiting from reinforcement learning, such as Group Relative Policy Optimization (GRPO). However, existing research centers on offline reinforcement learning training or online optimization using action-level rewards, which limits the agent’s dynamic interaction with the environment. This often results in agents settling into local optima, thereby weakening their ability for exploration and error action correction. To address these challenges, we introduce an approach called Mobile-R1, which employs interactive multi-turn reinforcement learning with task-level rewards for mobile agents. Our training framework consists of three stages: initial format finetuning, single-step online training via action-level reward, followed by online training via task-level reward based on multi-turn trajectories. This strategy is designed to enhance the exploration and error correction capabilities of Mobile-R1, leading to significant performance improvements. Moreover, we have collected a dataset covering 28 Chinese applications with 24,521 high-quality manual annotations and established a new benchmark with 500 trajectories. We will open source all resources, including the dataset, benchmark, model weight, and codes: https://mobile-r1.github.io/Mobile-R1/.

Method: Proposes an intermediate Intention Capture layer that extracts Workflow Signals from user queries, interprets them into structured Workflow Intention objects, and generates workflows based on these intentions.

Result: The framework yields consistent improvements in semantic workflow similarity metrics on a benchmark of 1,000 multi-intent query-workflow pairs, with significant quality improvements compared to direct generation.

Conclusion: The Opus Prompt Intention Framework enables LLMs to produce more reliable and scalable workflow outputs, particularly effective in cases of Mixed Intention Elicitation.

Abstract: This paper introduces the Opus Prompt Intention Framework, designed to improve complex Workflow Generation with instruction-tuned Large Language Models (LLMs). We propose an intermediate Intention Capture layer between user queries and Workflow Generation, implementing the Opus Workflow Intention Framework, which consists of extracting Workflow Signals from user queries, interpreting them into structured Workflow Intention objects, and generating Workflows based on these Intentions. Our results show that this layer enables LLMs to produce logical and meaningful outputs that scale reliably as query complexity increases. On a synthetic benchmark of 1,000 multi-intent query-Workflow(s) pairs, applying the Opus Prompt Intention Framework to Workflow Generation yields consistent improvements in semantic Workflow similarity metrics. In this paper, we introduce the Opus Prompt Intention Framework by applying the concepts of Workflow Signal and Workflow Intention to LLM-driven Workflow Generation. We present a reproducible, customizable LLM-based Intention Capture system to extract Workflow Signals and Workflow Intentions from user queries. Finally, we provide empirical evidence that the proposed system significantly improves Workflow Generation quality compared to direct generation from user queries, particularly in cases of Mixed Intention Elicitation.

Method: Uses LMM as central orchestrator coordinating between Sparse Deformable Multi-Scale Detector (SDMSD) for multi-scale defect detection and Evidence-Grounded Reflection (EGR) tool for chain-of-thought validation and refinement of proposals.

Result: Achieves 96.35% F1-score on GDXray+ dataset with significantly improved interpretability and trustworthiness compared to traditional approaches.

Conclusion: Demonstrates transformative potential of agentic LLM frameworks for industrial inspection by moving from passive processing to active reasoning with enhanced reliability and interpretability.

Abstract: Non-destructive testing (NDT), particularly X-ray inspection, is vital for industrial quality assurance, yet existing deep-learning-based approaches often lack interactivity, interpretability, and the capacity for critical self-assessment, limiting their reliability and operator trust. To address these shortcomings, this paper proposes InsightX Agent, a novel LMM-based agentic framework designed to deliver reliable, interpretable, and interactive X-ray NDT analysis. Unlike typical sequential pipelines, InsightX Agent positions a Large Multimodal Model (LMM) as a central orchestrator, coordinating between the Sparse Deformable Multi-Scale Detector (SDMSD) and the Evidence-Grounded Reflection (EGR) tool. The SDMSD generates dense defect region proposals for multi-scale feature maps and sparsifies them through Non-Maximum Suppression (NMS), optimizing detection of small, dense targets in X-ray images while maintaining computational efficiency. The EGR tool guides the LMM agent through a chain-of-thought-inspired review process, incorporating context assessment, individual defect analysis, false positive elimination, confidence recalibration and quality assurance to validate and refine the SDMSD’s initial proposals. By strategically employing and intelligently using tools, InsightX Agent moves beyond passive data processing to active reasoning, enhancing diagnostic reliability and providing interpretations that integrate diverse information sources. Experimental evaluations on the GDXray+ dataset demonstrate that InsightX Agent not only achieves a high object detection F1-score of 96.35% but also offers significantly improved interpretability and trustworthiness in its analyses, highlighting the transformative potential of agentic LLM frameworks for industrial inspection tasks.

Method: Uses prompt evolution driven by metacognitive framework with problem analyzer, error diagnosis system, and metacognitive search engine to iteratively optimize prompts based on performance feedback.

Result: MeLA consistently generates more effective and robust heuristics across benchmark and real-world problems, significantly outperforming state-of-the-art methods.

Conclusion: Demonstrates the potential of using cognitive science as blueprint for AI architecture, showing that metacognitive regulation of LLMs unlocks more robust and interpretable automatic heuristic design.

Abstract: This paper introduces MeLA, a Metacognitive LLM-Driven Architecture that presents a new paradigm for Automatic Heuristic Design (AHD). Traditional evolutionary methods operate directly on heuristic code; in contrast, MeLA evolves the instructional prompts used to guide a Large Language Model (LLM) in generating these heuristics. This process of “prompt evolution” is driven by a novel metacognitive framework where the system analyzes performance feedback to systematically refine its generative strategy. MeLA’s architecture integrates a problem analyzer to construct an initial strategic prompt, an error diagnosis system to repair faulty code, and a metacognitive search engine that iteratively optimizes the prompt based on heuristic effectiveness. In comprehensive experiments across both benchmark and real-world problems, MeLA consistently generates more effective and robust heuristics, significantly outperforming state-of-the-art methods. Ultimately, this research demonstrates the profound potential of using cognitive science as a blueprint for AI architecture, revealing that by enabling an LLM to metacognitively regulate its problem-solving process, we unlock a more robust and interpretable path to AHD.

Method: Proposes a novel layer selection method for structural pruning, analyzes different quantization techniques, and implements a prune-SFT-quantize pipeline for efficient compression of fine-tuned LLAVA models.

Result: The method enables 7B parameter MLLMs to run within 4GB VRAM (70% memory reduction) while achieving 4% higher performance compared to traditional pruning and quantization techniques at the same compression ratio.

Conclusion: The proposed compression pipeline effectively addresses computational constraints for medical MLLMs, making them more practical for real-world deployment while maintaining or improving performance.

Abstract: Multimodal Large Language Models (MLLMs) hold huge potential for usage in the medical domain, but their computational costs necessitate efficient compression techniques. This paper evaluates the impact of structural pruning and activation-aware quantization on a fine-tuned LLAVA model for medical applications. We propose a novel layer selection method for pruning, analyze different quantization techniques, and assess the performance trade-offs in a prune-SFT-quantize pipeline. Our proposed method enables MLLMs with 7B parameters to run within 4 GB of VRAM, reducing memory usage by 70% while achieving 4% higher model performance compared to traditional pruning and quantization techniques in the same compression ratio.

Method: Provides a framework wrapping multiple board and matrix games from OpenSpiel library, integrates API access via liteLLM and local deployment via vLLM, supports various agent types (random, heuristic, RL agents), and enables distributed execution through Ray.

Result: Created a comprehensive evaluation framework that facilitates systematic comparisons between LLM-based agents and other agent types across diverse game scenarios.

Conclusion: The Game Reasoning Arena library contributes to empirical evaluation of LLM reasoning and game-theoretic behavior by providing a standardized testing environment for strategic decision-making assessment.

Abstract: The Game Reasoning Arena library provides a framework for evaluating the decision making abilities of large language models (LLMs) through strategic board games implemented in Google OpenSpiel library. The framework enables systematic comparisons between LLM based agents and other agents (random, heuristic, reinforcement learning agents, etc.) in various game scenarios by wrapping multiple board and matrix games and supporting different agent types. It integrates API access to models via liteLLM, local model deployment via vLLM, and offers distributed execution through Ray. This paper summarises the library structure, key characteristics, and motivation of the repository, highlighting how it contributes to the empirical evaluation of the reasoning of LLM and game theoretic behaviour.

Method: Introduces TH-Score metric to measure confidence-accuracy alignment and proposes LLM-as-a-Fuser ensemble framework to transform LLMs into reliable, risk-aware evaluators.

Result: Extensive experiments show the approach substantially improves calibration and enables adaptive, confidence-driven evaluation pipelines, achieving superior reliability and accuracy compared to existing baselines.

Conclusion: The work advocates shifting from accuracy-centric to confidence-driven, risk-aware LLM-as-a-Judge systems, demonstrating that proper confidence calibration is vital for trustworthy automated evaluation.

Abstract: Large Language Models (LLMs) are widely used as automated judges, where practical value depends on both accuracy and trustworthy, risk-aware judgments. Existing approaches predominantly focus on accuracy, overlooking the necessity of well-calibrated confidence, which is vital for adaptive and reliable evaluation pipelines. In this work, we advocate a shift from accuracy-centric evaluation to confidence-driven, risk-aware LLM-as-a-Judge systems, emphasizing the necessity of well-calibrated confidence for trustworthy and adaptive evaluation. We systematically identify the Overconfidence Phenomenon in current LLM-as-a-Judges, where predicted confidence significantly overstates actual correctness, undermining reliability in practical deployment. To quantify this phenomenon, we introduce TH-Score, a novel metric measuring confidence-accuracy alignment. Furthermore, we propose LLM-as-a-Fuser, an ensemble framework that transforms LLMs into reliable, risk-aware evaluators. Extensive experiments demonstrate that our approach substantially improves calibration and enables adaptive, confidence-driven evaluation pipelines, achieving superior reliability and accuracy compared to existing baselines.

Method: GRASP metaheuristic combining multiple construction heuristics using MIP techniques to transform TA-TSP into TSP instances, followed by multi-neighborhood local search with 2-Opt, Swap, and Relocate operators.

Result: Achieved average optimality gaps of 0.77% on MESS 2024 competition instances and solutions 11.3% better than Gurobi on synthetic datasets within 60-second time limit. Finished top three in MESS 2024 competition.

Conclusion: The method demonstrates strong suitability for real-time routing applications with state-dependent travel costs, showing excellent performance in both competition and synthetic scenarios.

Abstract: The Trigger Arc Traveling Salesman Problem (TA-TSP) extends the classical TSP by introducing dynamic arc costs that change when specific “trigger” arcs are traversed, modeling scenarios such as warehouse operations with compactable storage systems. This paper introduces a GRASP-based metaheuristic that combines multiple construction heuristics with a multi-neighborhood local search. The construction phase uses mixed-integer programming (MIP) techniques to transform the TA-TSP into a sequence of tailored TSP instances, while the improvement phase applies 2-Opt, Swap, and Relocate operators. Computational experiments on MESS 2024 competition instances achieved average optimality gaps of 0.77% and 0.40% relative to the best-known solutions within a 60-second limit. On smaller, synthetically generated datasets, the method produced solutions 11.3% better than the Gurobi solver under the same time constraints. The algorithm finished in the top three at MESS 2024, demonstrating its suitability for real-time routing applications with state-dependent travel costs.

Method: Dynamic supervision and maneuvering mechanisms with Execution Agent and Guard Agent architecture within AWorld framework, where Guard Agent verifies and corrects reasoning at critical steps.

Result: Significantly improved effectiveness and stability compared to single-agent and standard tool-augmented systems, achieving first place among open-source projects on GAIA leaderboard.

Conclusion: Collaborative agent roles with dynamic verification mechanisms enhance reliability and trustworthiness of intelligent systems using multiple tools.

Abstract: The rapid advancement of large language models (LLMs) has empowered intelligent agents to leverage diverse external tools for solving complex real-world problems. However, as agents increasingly depend on multiple tools, they encounter new challenges: extended contexts from disparate sources and noisy or irrelevant tool outputs can undermine system reliability and accuracy. These challenges underscore the necessity for enhanced stability in agent-based systems. To address this, we introduce dynamic supervision and maneuvering mechanisms, constructing a robust and dynamic Multi-Agent System (MAS) architecture within the AWorld framework. In our approach, the Execution Agent invokes the Guard Agent at critical steps to verify and correct the reasoning process, effectively reducing errors arising from noise and bolstering problem-solving robustness. Extensive experiments on the GAIA test dataset reveal that our dynamic maneuvering mechanism significantly improves both the effectiveness and stability of solutions, outperforming single-agent system (SAS) and standard tool-augmented systems. As a result, our dynamic MAS system achieved first place among open-source projects on the prestigious GAIA leaderboard. These findings highlight the practical value of collaborative agent roles in developing more reliable and trustworthy intelligent systems.

Method: Analysis of the fundamental working principles and inherent limitations of LLMs that prevent them from achieving true correct reasoning.

Result: The paper concludes that LLMs are fundamentally incapable of true reasoning due to their operational constraints and design principles.

Conclusion: Claims about LLM reasoning abilities are misconceptions; true reasoning remains beyond the reach of current LLM architectures due to their inherent limitations.

Abstract: Recently, with the application progress of AIGC tools based on large language models (LLMs), led by ChatGPT, many AI experts and more non-professionals are trumpeting the “reasoning ability” of the LLMs. The present author considers that the so-called “reasoning ability” of LLMs are just illusions of those people who with vague concepts. In fact, the LLMs can never have the true reasoning ability. This paper intents to explain that, because the essential limitations of their working principle, the LLMs can never have the ability of true correct reasoning.

Method: VSRM (verifiable stepwise reward mechanism) assigns rewards based on intermediate state performance in reasoning trajectories, integrated with PPO and Reinforce++ reinforcement learning.

Result: Achieves substantial output length reduction while maintaining original reasoning performance on AIME24 and AIME25 benchmarks. Effectively suppresses ineffective steps and encourages effective reasoning.

Conclusion: VSRM provides an intuitive, stepwise approach that fundamentally alleviates overthinking by rewarding effective reasoning steps and penalizing ineffective ones, striking optimal balance between efficiency and accuracy.

Abstract: Large reasoning models (LRMs) have recently achieved significant progress in complex reasoning tasks, aided by reinforcement learning with verifiable rewards. However, LRMs often suffer from overthinking, expending excessive computation on simple problems and reducing efficiency. Existing efficient reasoning methods typically require accurate task assessment to preset token budgets or select reasoning modes, which limits their flexibility and reliability. In this work, we revisit the essence of overthinking and identify that encouraging effective steps while penalizing ineffective ones is key to its solution. To this end, we propose a novel rule-based verifiable stepwise reward mechanism (VSRM), which assigns rewards based on the performance of intermediate states in the reasoning trajectory. This approach is intuitive and naturally fits the step-by-step nature of reasoning tasks. We conduct extensive experiments on standard mathematical reasoning benchmarks, including AIME24 and AIME25, by integrating VSRM with PPO and Reinforce++. Results show that our method achieves substantial output length reduction while maintaining original reasoning performance, striking an optimal balance between efficiency and accuracy. Further analysis of overthinking frequency and pass@k score before and after training demonstrates that our approach in deed effectively suppresses ineffective steps and encourages effective reasoning, fundamentally alleviating the overthinking problem. All code will be released upon acceptance.

Method: LeanRAG uses semantic aggregation to form entity clusters with explicit relations, creating a navigable semantic network, then employs bottom-up structure-guided retrieval that anchors queries to fine-grained entities and traverses semantic pathways.

Result: Extensive experiments on four QA benchmarks show LeanRAG significantly outperforms existing methods in response quality while reducing 46% retrieval redundancy.

Conclusion: LeanRAG effectively addresses the limitations of hierarchical knowledge graph RAG by creating explicit semantic relations and structure-aware retrieval, achieving superior performance with reduced computational overhead.

Abstract: Retrieval-Augmented Generation (RAG) plays a crucial role in grounding Large Language Models by leveraging external knowledge, whereas the effectiveness is often compromised by the retrieval of contextually flawed or incomplete information. To address this, knowledge graph-based RAG methods have evolved towards hierarchical structures, organizing knowledge into multi-level summaries. However, these approaches still suffer from two critical, unaddressed challenges: high-level conceptual summaries exist as disconnected ``semantic islands’’, lacking the explicit relations needed for cross-community reasoning; and the retrieval process itself remains structurally unaware, often degenerating into an inefficient flat search that fails to exploit the graph’s rich topology. To overcome these limitations, we introduce LeanRAG, a framework that features a deeply collaborative design combining knowledge aggregation and retrieval strategies. LeanRAG first employs a novel semantic aggregation algorithm that forms entity clusters and constructs new explicit relations among aggregation-level summaries, creating a fully navigable semantic network. Then, a bottom-up, structure-guided retrieval strategy anchors queries to the most relevant fine-grained entities and then systematically traverses the graph’s semantic pathways to gather concise yet contextually comprehensive evidence sets. The LeanRAG can mitigate the substantial overhead associated with path retrieval on graphs and minimizes redundant information retrieval. Extensive experiments on four challenging QA benchmarks with different domains demonstrate that LeanRAG significantly outperforming existing methods in response quality while reducing 46% retrieval redundancy. Code is available at: https://github.com/RaZzzyz/LeanRAG

Method: Used 2024 U.S. Top 200 Spotify Daily Charts data (14,639 songs) with metadata and audio features. Benchmarked Logistic Regression, KNN, Random Forest, and XGBoost with train-test split, then incorporated cross-validation and hyperparameter tuning.

Result: Tree-based models (Random Forest and XGBoost) outperformed others with macro F1-scores near 0.95 and accuracy around 97%. Models trained solely on audio attributes retained predictive power even without stream count data.

Conclusion: Audio-based modeling has strong potential for A&R scouting, playlist optimization, and hit forecasting before tracks reach critical mass, validating the use of musical characteristics for success prediction.

Abstract: Spotify’s streaming charts offer a real-time lens into music popularity, driving discovery, playlists, and even revenue potential. Understanding what influences a song’s rise in ranks on these charts-especially early on-can guide marketing efforts, investment decisions, and even artistic direction. In this project, we developed a classification pipeline to predict a song’s chart success based on its musical characteristics and early engagement data. Using all 2024 U.S. Top 200 Spotify Daily Charts and the Spotify Web API, we built a dataset containing both metadata and audio features for 14,639 unique songs. The project was structured in two phases. First, we benchmarked four models: Logistic Regression, K Nearest Neighbors, Random Forest, and XGBoost-using a standard train-test split. In the second phase, we incorporated cross-validation, hyperparameter tuning, and detailed class-level evaluation to ensure robustness. Tree-based models consistently outperformed the rest, with Random Forest and XGBoost achieving macro F1-scores near 0.95 and accuracy around 97%. Even when stream count and rank history were excluded, models trained solely on audio attributes retained predictive power. These findings validate the potential of audio-based modeling in A&R scouting, playlist optimization, and hit forecasting-long before a track reaches critical mass.

Method: Proposed AF-Reasoning-Eval benchmark for sound reasoning assessment, created automatic pipelines to transform existing audio QA and classification data into explicit reasoning chains (AF-CoT-Train with 1.24M samples), and finetuned Audio Flamingo series on this dataset.

Result: Considerable improvements observed on several reasoning benchmarks after finetuning, validating the effectiveness of chain-of-thought finetuning for advanced sound understanding.

Conclusion: Chain-of-thought finetuning is effective for enhancing sound reasoning capabilities in audio language models, as demonstrated by significant performance improvements on reasoning benchmarks.

Abstract: Chain-of-thought reasoning has demonstrated significant improvements in large language models and vision language models, yet its potential for audio language models remains largely unexplored. In this technical report, we take a preliminary step towards closing this gap. For better assessment of sound reasoning, we propose AF-Reasoning-Eval, a benchmark targeting common-sense reasoning and the ability to discriminate among closely related choices. To prepare training corpus for sound reasoning abilities, we propose automatic pipelines that transform existing audio question answering and classification data into explicit reasoning chains, yielding AF-CoT-Train with 1.24M samples. We study the effect of finetuning Audio Flamingo series on AF-CoT-Train and observe considerable improvements on several reasoning benchmarks, validating the effectiveness of chain-of-thought finetuning on advanced sound understanding.

Method: Conducted large-scale empirical study covering training data diversity/scale, model architectures, and training recipes. Used self-supervised pre-training followed by supervised post-training on mixed bioacoustics + general-audio corpus across 26 datasets.

Result: Achieved state-of-the-art performance on existing and proposed benchmarks. Identified that data diversity in both pre-training and supervised stages is crucial for strong in- and out-of-distribution performance.

Conclusion: The study provides a foundation for bioacoustic encoder development, showing effective training approaches and the importance of data diversity. Model checkpoints will be released to support ongoing research.

Abstract: Bioacoustics, the study of sounds produced by living organisms, plays a vital role in conservation, biodiversity monitoring, and behavioral studies. Many tasks in this field, such as species, individual, and behavior classification and detection, are well-suited to machine learning. However, they often suffer from limited annotated data, highlighting the need for a general-purpose bioacoustic encoder capable of extracting useful representations for diverse downstream tasks. Such encoders have been proposed before, but are often limited in scope due to a focus on a narrow range of species (typically birds), and a reliance on a single model architecture or training paradigm. Moreover, they are usually evaluated on a small set of tasks and datasets. In this work, we present a large-scale empirical study that covers aspects of bioacoustics that are relevant to research but have previously been scarcely considered: training data diversity and scale, model architectures and training recipes, and the breadth of evaluation tasks and datasets. We obtain encoders that are state-of-the-art on the existing and proposed benchmarks. We also identify what matters for training these encoders, such that this work can be extended when more data are available or better architectures are proposed. Specifically, across 26 datasets with tasks including species classification, detection, individual ID, and vocal repertoire discovery, we find self-supervised pre-training followed by supervised post-training on a mixed bioacoustics + general-audio corpus yields the strongest in- and out-of-distribution performance. We show the importance of data diversity in both stages. To support ongoing research and application, we will release the model checkpoints.

Method: 1) Created AuditScore dataset with 6,300+ edited samples from 7 frameworks, annotated by professionals on Quality, Relevance, and Faithfulness. 2) Trained AuditEval model for automatic MOS-style scoring. 3) Used AuditEval to filter synthetic data and construct high-quality pseudo-parallel dataset.

Result: Developed the first comprehensive audio editing evaluation dataset and automatic scoring model. Objective experiments validated the effectiveness of expert-informed filtering for higher-quality data.

Conclusion: The proposed expert-informed approach successfully addresses evaluation challenges in audio editing, though limitations of objective-only metrics were revealed. The tools and dataset are publicly available.

Abstract: Audio editing aims to manipulate audio content based on textual descriptions, supporting tasks such as adding, removing, or replacing audio events. Despite recent progress, the lack of high-quality benchmark datasets and comprehensive evaluation metrics remains a major challenge for both assessing audio editing quality and improving the task itself. In this work, we propose a novel approach for audio editing task by incorporating expert knowledge into both the evaluation and dataset construction processes: 1) First, we establish AuditScore, the first comprehensive dataset for subjective evaluation of audio editing, consisting of over 6,300 edited samples generated from 7 representative audio editing frameworks and 23 system configurations. Each sample is annotated by professional raters on three key aspects of audio editing quality: overall Quality, Relevance to editing intent, and Faithfulness to original features. 2) Based on this dataset, we train AuditEval, the first model designed for automatic MOS-style scoring tailored to audio editing tasks. AuditEval addresses the critical lack of objective evaluation metrics and the prohibitive cost of subjective assessment in this field. 3) We further leverage AuditEval to evaluate and filter a large amount of synthetically mixed editing pairs, constructing a high-quality pseudo-parallel dataset by selecting the most plausible samples. Objective experiments validate the effectiveness of our expert-informed filtering strategy in yielding higher-quality data, while also revealing the limitations of relying solely on objective metrics. The dataset, codes and tools can be found at: https://github.com/NKU-HLT/AuditEval.

Method: Proposed a refined DEMUCS model with U-Net and LSTM layers, trained on 400,000 Hindi speech clips augmented with ESC-50 and MS-SNSD datasets. Explored quantization techniques for edge deployment.

Result: Achieved substantial improvements in speech clarity and intelligibility, with superior performance in PESQ and STOI metrics, particularly under extreme noise conditions.

Conclusion: Demonstrates effectiveness of customized AI algorithms for Hindi speech processing and provides direction for optimizing edge-based architectures in Indian contexts.

Abstract: This paper addresses the challenges of Hindi speech separation and enhancement using advanced neural network architectures, with a focus on edge devices. We propose a refined approach leveraging the DEMUCS model to overcome limitations of traditional methods, achieving substantial improvements in speech clarity and intelligibility. The model is fine-tuned with U-Net and LSTM layers, trained on a dataset of 400,000 Hindi speech clips augmented with ESC-50 and MS-SNSD for diverse acoustic environments. Evaluation using PESQ and STOI metrics shows superior performance, particularly under extreme noise conditions. To ensure deployment on resource-constrained devices like TWS earbuds, we explore quantization techniques to reduce computational requirements. This research highlights the effectiveness of customized AI algorithms for speech processing in Indian contexts and suggests future directions for optimizing edge-based architectures.

Method: Uses self-supervised pre-trained models with Fully-Connected Low-Rank Adaptation (LoRA) to prevent overfitting. Introduces Machine-aware Group Adapter to capture machine differences, and novel objective function with dynamic clustering and dual-level contrastive learning for unattributed data.

Result: Significant improvements demonstrated across all benchmark datasets including DCASE 2020-2024 ASD challenges, showing effectiveness of the proposed strategies.

Conclusion: Pre-training provides substantial benefits for ASD despite domain inconsistencies. The proposed adaptation techniques successfully enhance generalization performance in anomalous sound detection systems.

Abstract: Machine anomalous sound detection (ASD) is a valuable technique across various applications. However, its generalization performance is often limited due to challenges in data collection and the complexity of acoustic environments. Inspired by the success of large pre-trained models in numerous fields, this paper introduces a robust ASD model that leverages self-supervised pre-trained models trained on large-scale speech and audio datasets. Although there are inconsistencies between the pre-training datasets and the ASD task, our findings indicate that pre-training still provides substantial benefits for ASD. To mitigate overfitting and retain learned knowledge when fine-tuning with limited data, we explore Fully-Connected Low-Rank Adaptation (LoRA) as an alternative to full fine-tuning. Additionally, we propose a Machine-aware Group Adapter module, which enables the model to capture differences between various machines within a unified framework, thereby enhancing the generalization performance of ASD systems. To address the challenge of missing attribute labels, we design a novel objective function that dynamically clusters unattributed data using vector quantization and optimizes through a dual-level contrastive learning loss. The proposed methods are evaluated on all benchmark datasets, including the DCASE 2020-2024 five ASD challenges, and the experimental results show significant improvements of our new approach and demonstrate the effectiveness of our proposed strategies.

Method: Proposes HuBERT-VIC with variance, invariance, and covariance regularization (VICReg) objectives that adjust statistics of noisy speech representations to capture diverse acoustic characteristics.

Result: Achieves 23.3% relative improvement on LibriSpeech test-clean and 13.2% on test-other compared to baseline model pre-trained on noisy speech.

Conclusion: VICReg regularization effectively enhances noise robustness in speech foundation models by improving generalization across different noise types through adjusted representation statistics.

Abstract: Noise robustness in speech foundation models (SFMs) has been a critical challenge, as most models are primarily trained on clean data and experience performance degradation when the models are exposed to noisy speech. To address this issue, we propose HuBERT-VIC, a noise-robust SFM with variance, in-variance, and covariance regularization (VICReg) objectives. These objectives adjust the statistics of noisy speech representations, enabling the model to capture diverse acoustic characteristics and improving the generalization ability across different types of noise. When applied to HuBERT, our model shows relative performance improvements of 23.3% on LibriSpeech test-clean and 13.2% on test-other, compared to the baseline model pre-trained on noisy speech.

Method: Uses audio-only SELD model as teacher to transfer knowledge to AV student model via output responses and intermediate features, combined with multi-level data augmentation mixing features from different network layers with task-specific loss functions.

Result: Significant improvements with 22%~36% relative gains in overall metric over baseline, achieving state-of-the-art results on DCASE 2023 and 2024 SELD datasets, comparable to teacher models trained on larger datasets.

Conclusion: The proposed CMKD framework with multi-level augmentation effectively enhances AV SELD performance in low-resource settings, demonstrating superior results over existing methods.

Abstract: This work presents a cross-modal knowledge distillation (CMKD) framework combined with multi-level data augmentation for low-resource audio-visual (AV) sound event localization and detection (SELD). An audio-only SELD model acts as the teacher, transferring knowledge to an AV student model through both output responses and intermediate feature representations. To enhance learning, data augmentation is applied by mixing features randomly selected from multiple network layers and associated loss functions tailored to the SELD task. Extensive experiments on the DCASE 2023 and 2024 SELD datasets show that the proposed method significantly improves AV SELD performance, yielding relative gains of 22%~36% in the overall metric over the baseline. Notably, our approach achieves results comparable to or better than teacher models trained on much larger datasets, surpassing state-of-the-art methods on both DCASE 2023 and 2024 SELD tasks.

Method: Annotated GiantMIDI-Piano dataset using GPT-4o in valence-arousal framework, compared against three human experts using accuracy metrics, weighted accuracy, inter-annotator agreement, and distributional similarity.

Result: GPT’s performance was below human experts in accuracy and nuance, but its variability fell within the range of natural expert disagreement, showing potential for scalable annotation.

Conclusion: GPT-based annotation offers cost-effective and efficient alternative despite current limitations, making it promising for large-scale music emotion annotation tasks.

Abstract: Current approaches to music emotion annotation remain heavily reliant on manual labelling, a process that imposes significant resource and labour burdens, severely limiting the scale of available annotated data. This study examines the feasibility and reliability of employing a large language model (GPT-4o) for music emotion annotation. In this study, we annotated GiantMIDI-Piano, a classical MIDI piano music dataset, in a four-quadrant valence-arousal framework using GPT-4o, and compared against annotations provided by three human experts. We conducted extensive evaluations to assess the performance and reliability of GPT-generated music emotion annotations, including standard accuracy, weighted accuracy that accounts for inter-expert agreement, inter-annotator agreement metrics, and distributional similarity of the generated labels. While GPT’s annotation performance fell short of human experts in overall accuracy and exhibited less nuance in categorizing specific emotional states, inter-rater reliability metrics indicate that GPT’s variability remains within the range of natural disagreement among experts. These findings underscore both the limitations and potential of GPT-based annotation: despite its current shortcomings relative to human performance, its cost-effectiveness and efficiency render it a promising scalable alternative for music emotion annotation.

Method: Builds on MATPAC system, integrating Multiple Choice Learning (MCL) to explicitly model prediction ambiguity and improve representation quality in both prediction and unsupervised classification pretext tasks.

Result: Achieves state-of-the-art performance when fine-tuned on AudioSet and overall SOTA scores on downstream tasks. When trained exclusively on music data, achieves SOTA with significantly improved efficiency.

Conclusion: Integrating MCL effectively addresses prediction ambiguity in audio SSL, leading to superior representation quality and performance across various audio domains.

Abstract: Masked latent prediction has emerged as a leading paradigm in self-supervised learning (SSL), especially for general audio and music representation learning. While recent methods have demonstrated strong performance, the role of the predictor module used at the output of such SSL systems remains mainly overlooked, despite being crucial for solving the pretext task at hand. In particular, this module should be able to deal with the ambiguity inherent in audio content, especially when it is composed of multiple sound sources. This work proposes a novel enhancement: integrating Multiple Choice Learning (MCL) to explicitly model prediction ambiguity and improve representation quality. We build on top of the recently proposed MATPAC system, improving its prediction and unsupervised classification pretext tasks with MCL. We extensively evaluate our method, MATPAC++, through both linear probing across multiple downstream tasks and fine-tuning on AudioSet, employing a unified protocol that enables rigorous and fair comparisons with state-of-the-art SSL approaches. Results show that our proposal achieves state-of-the-art when fine-tuned on AudioSet and overall state-of-the-art scores on downstream tasks. Additionally, we examine domain specialisation by training exclusively on music data, where our model achieves state-of-the-art performance with significantly improved efficiency.

Method: Developed sound source estimation to determine 2D coordinates and depth, mapped to 3D trajectory. Used pre-trained V2A model for monaural audio, then diffusion model with 3D trajectory conditioning to generate binaural audio. Constructed training dataset with Head-Related Impulse Responses for dynamic sound fields.

Result: Outperforms existing approaches in spatial perception consistency, effectively enhancing immersive quality of audio-visual experience.

Conclusion: FoleySpace successfully addresses the gap in binaural spatial audio generation, providing spatially consistent and immersive audio from video inputs.

Abstract: Recently, with the advancement of AIGC, deep learning-based video-to-audio (V2A) technology has garnered significant attention. However, existing research mostly focuses on mono audio generation that lacks spatial perception, while the exploration of binaural spatial audio generation technologies, which can provide a stronger sense of immersion, remains insufficient. To solve this problem, we propose FoleySpace, a framework for video-to-binaural audio generation that produces immersive and spatially consistent stereo sound guided by visual information. Specifically, we develop a sound source estimation method to determine the sound source 2D coordinates and depth in each video frame, and then employ a coordinate mapping mechanism to convert the 2D source positions into a 3D trajectory. This 3D trajectory, together with the monaural audio generated by a pre-trained V2A model, serves as a conditioning input for a diffusion model to generate spatially consistent binaural audio. To support the generation of dynamic sound fields, we constructed a training dataset based on recorded Head-Related Impulse Responses that includes various sound source movement scenarios. Experimental results demonstrate that the proposed method outperforms existing approaches in spatial perception consistency, effectively enhancing the immersive quality of the audio-visual experience.

Method: Developed DiffVox with differentiable parametric EQ, dynamic range control, delay, and reverb. Used 70 tracks from MedleyDB and 365 private tracks. Conducted parameter correlation analysis, PCA, and statistical testing on parameter distributions.

Result: Found strong parameter correlations (e.g., high-pass + low-shelf filters for low-end shaping), PCA revealed connections to McAdams’ timbre dimensions (spaciousness and spectral brightness), and confirmed non-Gaussian parameter distributions.

Conclusion: The complex parameter distributions provide foundation for future vocal effects modeling and automatic mixing research, with code and datasets made publicly available.

Abstract: This study introduces a novel and interpretable model, DiffVox, for matching vocal effects in music production. DiffVox, short for ``Differentiable Vocal Fx", integrates parametric equalisation, dynamic range control, delay, and reverb with efficient differentiable implementations to enable gradient-based optimisation for parameter estimation. Vocal presets are retrieved from two datasets, comprising 70 tracks from MedleyDB and 365 tracks from a private collection. Analysis of parameter correlations reveals strong relationships between effects and parameters, such as the high-pass and low-shelf filters often working together to shape the low end, and the delay time correlating with the intensity of the delayed signals. Principal component analysis reveals connections to McAdams’ timbre dimensions, where the most crucial component modulates the perceived spaciousness while the secondary components influence spectral brightness. Statistical testing confirms the non-Gaussian nature of the parameter distribution, highlighting the complexity of the vocal effects space. These initial findings on the parameter distributions set the foundation for future research in vocal effects modelling and automatic mixing. Our source code and datasets are accessible at https://github.com/SonyResearch/diffvox.

Method: Proposes a low computational approach for continuous noise adaptation of pretrained neural networks using only 1-shot learning and one epoch. The method was tested with two pretrained models and three real-world noise sources at various SNRs.

Result: The adapted models consistently outperformed pretrained models across all scenarios, especially at SNR ≤ 18 dB, achieving accuracy improvements of 4.9% to 46.0%.

Conclusion: The proposed methodology is effective for noise adaptation in keyword spotting while being lightweight enough for deployment on resource-constrained embedded devices.

Abstract: Keyword spotting (KWS) is a key component of smart devices, enabling efficient and intuitive audio interaction. However, standard KWS systems deployed on embedded devices often suffer performance degradation under real-world operating conditions. Resilient KWS systems address this issue by enabling dynamic adaptation, with applications such as adding or replacing keywords, adjusting to specific users, and improving noise robustness. However, deploying resilient, standalone KWS systems with low latency on resource-constrained devices remains challenging due to limited memory and computational resources. This study proposes a low computational approach for continuous noise adaptation of pretrained neural networks used for KWS classification, requiring only 1-shot learning and one epoch. The proposed method was assessed using two pretrained models and three real-world noise sources at signal-to-noise ratios (SNRs) ranging from 24 to -3 dB. The adapted models consistently outperformed the pretrained models across all scenarios, especially at SNR $\leq$ 18 dB, achieving accuracy improvements of 4.9% to 46.0%. These results highlight the efficacy of the proposed methodology while being lightweight enough for deployment on resource-constrained devices.

Method: Standardized extraction parameters across three toolkits (OpenSMILE, Praat, Librosa) applied to speech samples from 77 schizophrenia spectrum disorder patients and 87 healthy controls, with correlation analysis and classification performance evaluation.

Result: Significant toolkit-dependent variations found - F0 percentiles showed high cross-toolkit correlation (r=0.962-0.999) but F0 standard deviation and formant values had poor, sometimes negative agreement. Classification identified F0 mean, HNR, and MFCC1 (AUC>0.70) as promising discriminators.

Conclusion: Findings highlight reproducibility concerns in acoustic feature extraction and advocate for standardized protocols, multi-toolkit cross-validation, and transparent reporting in clinical speech analysis research.

Abstract: This study compares three acoustic feature extraction toolkits (OpenSMILE, Praat, and Librosa) applied to clinical speech data from individuals with schizophrenia spectrum disorders (SSD) and healthy controls (HC). By standardizing extraction parameters across the toolkits, we analyzed speech samples from 77 SSD and 87 HC participants and found significant toolkit-dependent variations. While F0 percentiles showed high cross-toolkit correlation (r=0.962 to 0.999), measures like F0 standard deviation and formant values often had poor, even negative, agreement. Additionally, correlation patterns differed between SSD and HC groups. Classification analysis identified F0 mean, HNR, and MFCC1 (AUC greater than 0.70) as promising discriminators. These findings underscore reproducibility concerns and advocate for standardized protocols, multi-toolkit cross-validation, and transparent reporting.

Method: Proposed Semantic-Acoustic Source Tracing Network (SASTNet) that jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding to address overfitting to non-speech regions.

Result: SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating effective generalization to unseen content.

Conclusion: The proposed multi-modal approach combining semantic and acoustic features provides reliable source tracing for codec-based deepfake speech, overcoming limitations of previous methods.

Abstract: Recent attempts at source tracing for codec-based deepfake speech (CodecFake), generated by neural audio codec-based speech generation (CoSG) models, have exhibited suboptimal performance. However, how to train source tracing models using simulated CoSG data while maintaining strong performance on real CoSG-generated audio remains an open challenge. In this paper, we show that models trained solely on codec-resynthesized data tend to overfit to non-speech regions and struggle to generalize to unseen content. To mitigate these challenges, we introduce the Semantic-Acoustic Source Tracing Network (SASTNet), which jointly leverages Whisper for semantic feature encoding and Wav2vec2 with AudioMAE for acoustic feature encoding. Our proposed SASTNet achieves state-of-the-art performance on the CoSG test set of the CodecFake+ dataset, demonstrating its effectiveness for reliable source tracing.

Method: Uses efficient layer-to-layer distillation from domain-specific SSL models to train a student model on a comprehensive audio dataset that includes speech, sound, and music.

Result: Achieves competitive performance across various benchmarks including frame and instance-level speech processing, audio tagging, and sound classification, with near state-of-the-art results on SUPERB and HEAR benchmarks using a single encoder.

Conclusion: USAD successfully demonstrates that a single unified model can effectively handle diverse audio types through distillation, providing a more efficient and versatile approach to audio representation learning compared to domain-specific models.

Abstract: Self-supervised learning (SSL) has revolutionized audio representations, yet models often remain domain-specific, focusing on either speech or non-speech tasks. In this work, we present Universal Speech and Audio Distillation (USAD), a unified approach to audio representation learning that integrates diverse audio types - speech, sound, and music - into a single model. USAD employs efficient layer-to-layer distillation from domain-specific SSL models to train a student on a comprehensive audio dataset. USAD offers competitive performance across various benchmarks and datasets, including frame and instance-level speech processing tasks, audio tagging, and sound classification, achieving near state-of-the-art results with a single encoder on SUPERB and HEAR benchmarks.

Method: Developed a digital corpus covering all 13 components of the radif repertoire, providing MIDI files (281 minutes total) and detailed spreadsheets with note data, durations, intervals, hierarchical structures, quarter-tone representation, and non-metric aspects.

Result: Created a comprehensive dataset of 228 musical pieces with faithful tonality representation including quarter-tones, along with supporting statistics, complexity measures, and similarity analyses across the corpus.

Conclusion: This corpus serves as a foundational platform enabling computational studies of Iranian classical music for melodic pattern analysis, improvisation style investigation, and various music information retrieval tasks in ethnomusicology and music theory.

Abstract: Non-metric music forms the core of the repertoire in Iranian classical music. Dastgahi music serves as the underlying theoretical system for both Iranian art music and certain folk traditions. At the heart of Iranian classical music lies the radif, a foundational repertoire that organizes melodic material central to performance and pedagogy. In this study, we introduce a digital corpus representing the complete non-metrical radif repertoire, covering all 13 existing components of this repertoire. We provide MIDI files (about 281 minutes in total) and data spreadsheets describing notes, note durations, intervals, and hierarchical structures for 228 pieces of music. We faithfully represent the tonality including quarter-tones, and the non-metric aspect. Furthermore, we provide supporting basic statistics, and measures of complexity and similarity over the corpus. Our corpus provides a platform for computational studies of Iranian classical music. Researchers might employ it in studying melodic patterns, investigating improvisational styles, or for other tasks in music information retrieval, music theory, and computational (ethno)musicology.

Method: Proposed SEF-MK framework that anonymizes SSL representations by randomly selecting one of multiple k-means models (each trained on different speaker subsets) instead of using a single k-means model on the entire dataset.

Result: Multiple k-means models better preserve linguistic and emotional content from user perspective but boost effectiveness of privacy attacks from attacker perspective compared to single k-means model.

Conclusion: The framework provides insights for designing voice anonymization systems to balance content preservation and privacy protection against potential attacker threats.

Abstract: Voice anonymization protects speaker privacy by concealing identity while preserving linguistic and paralinguistic content. Self-supervised learning (SSL) representations encode linguistic features but preserve speaker traits. We propose a novel speaker-embedding-free framework called SEF-MK. Instead of using a single k-means model trained on the entire dataset, SEF-MK anonymizes SSL representations for each utterance by randomly selecting one of multiple k-means models, each trained on a different subset of speakers. We explore this approach from both attacker and user perspectives. Extensive experiments show that, compared to a single k-means model, SEF-MK with multiple k-means models better preserves linguistic and emotional content from the user’s viewpoint. However, from the attacker’s perspective, utilizing multiple k-means models boosts the effectiveness of privacy attacks. These insights can aid users in designing voice anonymization systems to mitigate attacker threats.

Method: SamKV sparsifies attention for multiple-context KV Cache by considering complementary information across contexts and locally recomputing sparsified information.

Result: The method compresses sequence length to 15% of original without accuracy degradation compared to full-recomputation baselines.

Conclusion: SamKV significantly boosts throughput in multi-context RAG scenarios by effectively addressing the limitations of existing KV Cache optimization techniques.

Abstract: Large language models face significant cost challenges in long-sequence inference. To address this, reusing historical Key-Value (KV) Cache for improved inference efficiency has become a mainstream approach. Recent advances further enhance throughput by sparse attention mechanisms to select the most relevant KV Cache, thereby reducing sequence length. However, such techniques are limited to single-context scenarios, where historical KV Cache is computed sequentially with causal-attention dependencies. In retrieval-augmented generation (RAG) scenarios, where retrieved documents as context are unknown beforehand, each document’s KV Cache is computed and stored independently (termed multiple-context KV Cache), lacking cross-attention between contexts. This renders existing methods ineffective. Although prior work partially recomputes multiple-context KV Cache to mitigate accuracy loss from missing cross-attention, it requires retaining all KV Cache throughout, failing to reduce memory overhead. This paper presents SamKV, the first exploration of attention sparsification for multiple-context KV Cache. Specifically, SamKV takes into account the complementary information of other contexts when sparsifying one context, and then locally recomputes the sparsified information. Experiments demonstrate that our method compresses sequence length to 15% without accuracy degradation compared with full-recompuation baselines, significantly boosting throughput in multi-context RAG scenarios.

Method: RS ranks words using importance heuristics, measures embedding sensitivity when masking top-k critical words, and processes patterns with a BiLSTM detector to identify adversarial examples.

Result: RS achieves over 88% detection accuracy across three datasets, three attack types, and two victim models, with competitive performance and lower computational cost than state-of-the-art methods.

Conclusion: RS provides an effective, practical solution for adversarial text detection that generalizes well to unseen datasets, attacks, and models without requiring retraining.

Abstract: Adversarial text attacks remain a persistent threat to transformer models, yet existing defenses are typically attack-specific or require costly model retraining. We introduce Representation Stability (RS), a model-agnostic detection framework that identifies adversarial examples by measuring how embedding representations change when important words are masked. RS first ranks words using importance heuristics, then measures embedding sensitivity to masking top-k critical words, and processes the resulting patterns with a BiLSTM detector. Experiments show that adversarially perturbed words exhibit disproportionately high masking sensitivity compared to naturally important words. Across three datasets, three attack types, and two victim models, RS achieves over 88% detection accuracy and demonstrates competitive performance compared to existing state-of-the-art methods, often at lower computational cost. Using Normalized Discounted Cumulative Gain (NDCG) to measure perturbation identification quality, we reveal that gradient-based ranking outperforms attention and random selection approaches, with identification quality correlating with detection performance for word-level attacks. RS also generalizes well to unseen datasets, attacks, and models without retraining, providing a practical solution for adversarial text detection.

Method: Developed lightweight sInvResUNet architecture with KDCL collaborative learning scheme, using only 0.89M parameters and 0.02 GFLOPS computational load.

Result: Achieved real-time inference (8.49ms for 10s output) with MAE of 10.06 mmHg and Pearson correlation of 0.88 on large heterogeneous dataset of 1.26M segments from 2,154 patients.

Conclusion: Successfully demonstrated real-time ABP monitoring on embedded devices but models show performance variations across diverse populations, highlighting generalization challenges.

Abstract: Noninvasive arterial blood pressure (ABP) monitoring is essential for patient management in critical care and perioperative settings, providing continuous assessment of cardiovascular hemodynamics with minimal risks. Numerous deep learning models have developed to reconstruct ABP waveform from noninvasively acquired physiological signals such as electrocardiogram and photoplethysmogram. However, limited research has addressed the issue of model performance and computational load for deployment on embedded systems. The study introduces a lightweight sInvResUNet, along with a collaborative learning scheme named KDCL_sInvResUNet. With only 0.89 million parameters and a computational load of 0.02 GFLOPS, real-time ABP estimation was successfully achieved on embedded devices with an inference time of just 8.49 milliseconds for a 10-second output. We performed subject-independent validation in a large-scale and heterogeneous perioperative dataset containing 1,257,141 data segments from 2,154 patients, with a wide BP range (41-257 mmHg for SBP, and 31-234 mmHg for DBP). The proposed KDCL_sInvResUNet achieved lightly better performance compared to large models, with a mean absolute error of 10.06 mmHg and mean Pearson correlation of 0.88 in tracking ABP changes. Despite these promising results, all deep learning models showed significant performance variations across different demographic and cardiovascular conditions, highlighting their limited ability to generalize across such a broad and diverse population. This study lays a foundation work for real-time, unobtrusive ABP monitoring in real-world perioperative settings, providing baseline for future advancements in this area.

Method: Fine-tunes Modality-Specific LoRA modules while incorporating Contrastive Regularization to enhance intra-modality knowledge sharing and promote inter-modality knowledge differentiation, keeping the pretrained LVLM frozen.

Result: Outperforms both separate modality models and general incremental learning methods, achieving 1.88% improvement in overall performance while maintaining computational efficiency.

Conclusion: MSLoRA-CR effectively addresses multimodal biomedical incremental learning challenges by preserving previous knowledge and leveraging cross-modal knowledge transfer through specialized LoRA adaptation and contrastive regularization.

Abstract: Multimodal Biomedical Image Incremental Learning (MBIIL) is essential for handling diverse tasks and modalities in the biomedical domain, as training separate models for each modality or task significantly increases inference costs. Existing incremental learning methods focus on task expansion within a single modality, whereas MBIIL seeks to train a unified model incrementally across modalities. The MBIIL faces two challenges: I) How to preserve previously learned knowledge during incremental updates? II) How to effectively leverage knowledge acquired from existing modalities to support new modalities? To address these challenges, we propose MSLoRA-CR, a method that fine-tunes Modality-Specific LoRA modules while incorporating Contrastive Regularization to enhance intra-modality knowledge sharing and promote inter-modality knowledge differentiation. Our approach builds upon a large vision-language model (LVLM), keeping the pretrained model frozen while incrementally adapting new LoRA modules for each modality or task. Experiments on the incremental learning of biomedical images demonstrate that MSLoRA-CR outperforms both the state-of-the-art (SOTA) approach of training separate models for each modality and the general incremental learning method (incrementally fine-tuning LoRA). Specifically, MSLoRA-CR achieves a 1.88% improvement in overall performance compared to unconstrained incremental learning methods while maintaining computational efficiency. Our code is publicly available at https://github.com/VentusAislant/MSLoRA_CR.

Method: Proposes a lifelong learner (LL) with Transformer backbone and inter-context self-attention mechanism to transfer knowledge between VRPs, plus a dynamic context scheduler (DCS) with cross-context experience replay.

Result: Extensive testing on synthetic and benchmark instances (up to 18k problem sizes) shows LL outperforms other neural solvers and achieves best performance for most VRPs.

Conclusion: The framework successfully enables neural solvers to handle VRPs in varying contexts, enhancing their versatility and practical applicability.

Abstract: Deep learning has been extensively explored to solve vehicle routing problems (VRPs), which yields a range of data-driven neural solvers with promising outcomes. However, most neural solvers are trained to tackle VRP instances in a relatively monotonous context, e.g., simplifying VRPs by using Euclidean distance between nodes and adhering to a single problem size, which harms their off-the-shelf application in different scenarios. To enhance their versatility, this paper presents a novel lifelong learning framework that incrementally trains a neural solver to manage VRPs in distinct contexts. Specifically, we propose a lifelong learner (LL), exploiting a Transformer network as the backbone, to solve a series of VRPs. The inter-context self-attention mechanism is proposed within LL to transfer the knowledge obtained from solving preceding VRPs into the succeeding ones. On top of that, we develop a dynamic context scheduler (DCS), employing the cross-context experience replay to further facilitate LL looking back on the attained policies of solving preceding VRPs. Extensive results on synthetic and benchmark instances (problem sizes up to 18k) show that our LL is capable of discovering effective policies for tackling generic VRPs in varying contexts, which outperforms other neural solvers and achieves the best performance for most VRPs.

Method: Applied time series foundation model (TimesFM) to predict US demographic changes using Census Bureau and FRED data, comparing against traditional baselines including LSTM, ARIMA, and Linear Regression across six demographically diverse states.

Result: TimesFM achieved the lowest Mean Squared Error in 86.67% of test cases, with particularly strong performance on minority populations that have sparse historical data.

Conclusion: Pre-trained foundation models like TimesFM have significant potential to enhance demographic analysis and inform proactive policy interventions without requiring extensive task-specific fine-tuning.

Abstract: Demographic shifts, influenced by globalization, economic conditions, geopolitical events, and environmental factors, pose significant challenges for policymakers and researchers. Accurate demographic forecasting is essential for informed decision-making in areas such as urban planning, healthcare, and economic policy. This study explores the application of time series foundation models to predict demographic changes in the United States using datasets from the U.S. Census Bureau and Federal Reserve Economic Data (FRED). We evaluate the performance of the Time Series Foundation Model (TimesFM) against traditional baselines including Long Short-Term Memory (LSTM) networks, Autoregressive Integrated Moving Average (ARIMA), and Linear Regression. Our experiments across six demographically diverse states demonstrate that TimesFM achieves the lowest Mean Squared Error (MSE) in 86.67% of test cases, with particularly strong performance on minority populations with sparse historical data. These findings highlight the potential of pre-trained foundation models to enhance demographic analysis and inform proactive policy interventions without requiring extensive task-specific fine-tuning.

Method: Developed a framework integrating OpenStreetMap, POI data, building morphology, land use, and satellite imagery with five dimensions: hierarchical building function classification, spatial organization patterns, functional diversity metrics, accessibility analysis, and land use intensity indicators. Used deep learning (RGNN and GNN) to address data gaps.

Result: The SPLI system improves functional classification accuracy and provides a standardized basis for automated urban spatial analytics, enabling better quantification of layout patterns and efficiency metrics.

Conclusion: The proposed SPLI system successfully bridges the gap between empirical planning and data-driven approaches, offering a comprehensive framework for systematic urban spatial analysis and supporting more informed site planning decisions.

Abstract: The spatial layout of urban sites shapes land-use efficiency and spatial organization. Traditional site planning often relies on experiential judgment and single-source data, limiting systematic quantification of multifunctional layouts. We propose a Site Planning Layout Indicator (SPLI) system, a data-driven framework integrating empirical knowledge with heterogeneous multi-source data to produce structured urban spatial information. The SPLI supports multimodal spatial data systems for analytics, inference, and retrieval by combining OpenStreetMap (OSM), Points of Interest (POI), building morphology, land use, and satellite imagery. It extends conventional metrics through five dimensions: (1) Hierarchical Building Function Classification, refining empirical systems into clear hierarchies; (2) Spatial Organization, quantifying seven layout patterns (e.g., symmetrical, concentric, axial-oriented); (3) Functional Diversity, transforming qualitative assessments into measurable indicators using Functional Ratio (FR) and Simpson Index (SI); (4) Accessibility to Essential Services, integrating facility distribution and transport networks for comprehensive accessibility metrics; and (5) Land Use Intensity, using Floor Area Ratio (FAR) and Building Coverage Ratio (BCR) to assess utilization efficiency. Data gaps are addressed through deep learning, including Relational Graph Neural Networks (RGNN) and Graph Neural Networks (GNN). Experiments show the SPLI improves functional classification accuracy and provides a standardized basis for automated, data-driven urban spatial analytics.

Method: A two-phase iterative algorithm that alternates between inferring causal relationships among discovered subprocesses and uncovering new latent subprocesses, using path-based conditions for identifiability.

Result: Experiments on synthetic and real-world datasets show effective recovery of causal structures despite latent subprocesses.

Conclusion: The proposed method successfully addresses the challenge of latent subprocesses in multivariate Hawkes processes and provides identifiability guarantees.

Abstract: Multivariate Hawkes process provides a powerful framework for modeling temporal dependencies and event-driven interactions in complex systems. While existing methods primarily focus on uncovering causal structures among observed subprocesses, real-world systems are often only partially observed, with latent subprocesses posing significant challenges. In this paper, we show that continuous-time event sequences can be represented by a discrete-time model as the time interval shrinks, and we leverage this insight to establish necessary and sufficient conditions for identifying latent subprocesses and the causal influences. Accordingly, we propose a two-phase iterative algorithm that alternates between inferring causal relationships among discovered subprocesses and uncovering new latent subprocesses, guided by path-based conditions that guarantee identifiability. Experiments on both synthetic and real-world datasets show that our method effectively recovers causal structures despite the presence of latent subprocesses.

Method: Proposed BRIEF framework with: 1) 4 types of fMRI temporal representations (time series, static/dynamic functional connection, multi-scale dispersion entropy) to construct encoders, 2) Modified Q-learning to dynamically optimize neural network connection search as a Markov Decision Process, 3) Transformer-based multi-feature fusion module, 4) Attention module for interpretability.

Result: BRIEF demonstrated significant improvements of 2.2% to 12.1% compared to 21 state-of-the-art algorithms, reaching AUC of 91.5% for schizophrenia (n=1100) and 78.4% for autism spectrum disorder (n=1550).

Conclusion: This is the first attempt to incorporate a brain-inspired, reinforcement learning strategy to optimize fMRI-based mental disorder classification, showing significant potential for identifying precise neuroimaging biomarkers.

Abstract: Existing deep learning models for functional MRI-based classification have limitations in network architecture determination (relying on experience) and feature space fusion (mostly simple concatenation, lacking mutual learning). Inspired by the human brain’s mechanism of updating neural connections through learning and decision-making, we proposed a novel BRain-Inspired feature Fusion (BRIEF) framework, which is able to optimize network architecture automatically by incorporating an improved neural network connection search (NCS) strategy and a Transformer-based multi-feature fusion module. Specifically, we first extracted 4 types of fMRI temporal representations, i.e., time series (TCs), static/dynamic functional connection (FNC/dFNC), and multi-scale dispersion entropy (MsDE), to construct four encoders. Within each encoder, we employed a modified Q-learning to dynamically optimize the NCS to extract high-level feature vectors, where the NCS is formulated as a Markov Decision Process. Then, all feature vectors were fused via a Transformer, leveraging both stable/time-varying connections and multi-scale dependencies across different brain regions to achieve the final classification. Additionally, an attention module was embedded to improve interpretability. The classification performance of our proposed BRIEF was compared with 21 state-of-the-art models by discriminating two mental disorders from healthy controls: schizophrenia (SZ, n=1100) and autism spectrum disorder (ASD, n=1550). BRIEF demonstrated significant improvements of 2.2% to 12.1% compared to 21 algorithms, reaching an AUC of 91.5% - 0.6% for SZ and 78.4% - 0.5% for ASD, respectively. This is the first attempt to incorporate a brain-inspired, reinforcement learning strategy to optimize fMRI-based mental disorder classification, showing significant potential for identifying precise neuroimaging biomarkers.

Method: Leveraging AEF’s global geospatial representation to train basic models (random forests, logistic regression) for extending existing datasets, demonstrated through extending LANDFIRE’s vegetation type dataset from USA to Canada at two granularity levels.

Result: Model predictions qualitatively align with ground truth for EvtPhys (13 classes). Achieved 81% classification accuracy on USA validation set and 73% on Canada validation set for EvtPhys, despite limitations.

Conclusion: AEF enables effective extension of geospatial datasets beyond original collection regions using simple machine learning models, demonstrating practical applicability for global geospatial analysis despite some accuracy limitations.

Abstract: High-quality labeled geospatial datasets are essential for extracting insights and understanding our planet. Unfortunately, these datasets often do not span the entire globe and are limited to certain geographic regions where data was collected. Google DeepMind’s recently released AlphaEarth Foundations (AEF) provides an information-dense global geospatial representation designed to serve as a useful input across a wide gamut of tasks. In this article we propose and evaluate a methodology which leverages AEF to extend geospatial labeled datasets beyond their initial geographic regions. We show that even basic models like random forests or logistic regression can be used to accomplish this task. We investigate a case study of extending LANDFIRE’s Existing Vegetation Type (EVT) dataset beyond the USA into Canada at two levels of granularity: EvtPhys (13 classes) and EvtGp (80 classes). Qualitatively, for EvtPhys, model predictions align with ground truth. Trained models achieve 81% and 73% classification accuracy on EvtPhys validation sets in the USA and Canada, despite discussed limitations.

Method: Four-phase framework: 1) Foundational model training on public dataset, 2) Serial meta-initialization on IoT data subset, 3) Parallel FL with dual-criterion aggregation (local performance + cosine similarity), 4) On-device personalization for specialized experts.

Result: Achieves 91.27% average test accuracy across heterogeneous IoT devices, outperforming personalized FedAvg by 3.87% and FedProx by 3.37% on electrical and mechanical fault datasets.

Conclusion: The multi-stage approach of sequenced initialization and adaptive aggregation provides a robust pathway for deploying high-performance intelligence on diverse TinyML networks in industrial IoT applications.

Abstract: Real-time fault classification in resource-constrained Internet of Things (IoT) devices is critical for industrial safety, yet training robust models in such heterogeneous environments remains a significant challenge. Standard Federated Learning (FL) often fails in the presence of non-IID data, leading to model divergence. This paper introduces Fed-Meta-Align, a novel four-phase framework designed to overcome these limitations through a sophisticated initialization and training pipeline. Our process begins by training a foundational model on a general public dataset to establish a competent starting point. This model then undergoes a serial meta-initialization phase, where it sequentially trains on a subset of IOT Device data to learn a heterogeneity-aware initialization that is already situated in a favorable region of the loss landscape. This informed model is subsequently refined in a parallel FL phase, which utilizes a dual-criterion aggregation mechanism that weights for IOT devices updates based on both local performance and cosine similarity alignment. Finally, an on-device personalization phase adapts the converged global model into a specialized expert for each IOT Device. Comprehensive experiments demonstrate that Fed-Meta-Align achieves an average test accuracy of 91.27% across heterogeneous IOT devices, outperforming personalized FedAvg and FedProx by up to 3.87% and 3.37% on electrical and mechanical fault datasets, respectively. This multi-stage approach of sequenced initialization and adaptive aggregation provides a robust pathway for deploying high-performance intelligence on diverse TinyML networks.

Method: Applied Group Relative Policy Optimization (GRPO), Proximal Policy Optimization (PPO), and REINFORCE Leave-One-Out (RLOO) to synthetic data and real-world biological experiments to evaluate their performance in stochastic domains.

Result: GRPO induced overconfident probability predictions for binary stochastic outcomes, while PPO and RLOO yielded well-calibrated models. Removing group standard normalization in GRPO fixed its miscalibration.

Conclusion: Provides evidence against using standard normalization in GRPO and paves the way for RL applications in reasoning language models beyond deterministic domains.

Abstract: Reinforcement learning (RL) has proven remarkably effective at improving the accuracy of language models in verifiable and deterministic domains like mathematics. Here, we examine if current RL methods are also effective at optimizing language models in verifiable domains with stochastic outcomes, like scientific experiments. Through applications to synthetic data and real-world biological experiments, we demonstrate that Group Relative Policy Optimization (GRPO) induces overconfident probability predictions for binary stochastic outcomes, while Proximal Policy Optimization (PPO) and REINFORCE Leave-One-Out (RLOO) yield well-calibrated models. We show that removing group standard normalization in GRPO fixes its miscalibration and provide a theoretical explanation for why normalization causes overconfidence. Our results provide new evidence against the use of standard normalization in GRPO and help pave the way for applications of RL for reasoning language models beyond deterministic domains.

Method: Uses in-context learning, prompt refinement, and fairness-aware data curation to integrate multiple fairness definitions into both generation and evaluation pipelines.

Result: Outperforms state-of-the-art methods with up to 10% improvements on fairness metrics (demographic parity, path-specific causal effects) while retaining statistical utility, achieving this with less than 20% of original data.

Conclusion: Demonstrates a principled and practical approach for generating fair and useful synthetic tabular data efficiently in low-data regimes.

Abstract: Generating synthetic data is crucial in privacy-sensitive, data-scarce settings, especially for tabular datasets widely used in real-world applications. A key challenge is improving counterfactual and causal fairness, while preserving high utility. We present FairTabGen, a fairness-aware large language model-based framework for tabular synthetic data generation. We integrate multiple fairness definitions including counterfactual and causal fairness into both its generation and evaluation pipelines. We use in-context learning, prompt refinement, and fairness-aware data curation to balance fairness and utility. Across diverse datasets, our method outperforms state-of-the-art GAN-based and LLM-based methods, achieving up to 10% improvements on fairness metrics such as demographic parity and path-specific causal effects while retaining statistical utility. Remarkably, it achieves these gains using less than 20% of the original data, highlighting its efficiency in low-data regimes. These results demonstrate a principled and practical approach for generating fair and useful synthetic tabular data.

Method: Replace traditional polynomial basis functions with fast computational functions (ReLU, trigonometric functions) in the Kolmogorov-Arnold Network structure to enhance GPU compatibility and computational speed.

Result: Experimental results show the proposed function combinations maintain competitive performance while offering potential improvements in training time and generalization capabilities.

Conclusion: Fast computational functions provide a viable alternative to polynomial functions in KANs, enabling better GPU utilization and computational efficiency without sacrificing performance.

Abstract: For years, many neural networks have been developed based on the Kolmogorov-Arnold Representation Theorem (KART), which was created to address Hilbert’s 13th problem. Recently, relying on KART, Kolmogorov-Arnold Networks (KANs) have attracted attention from the research community, stimulating the use of polynomial functions such as B-splines and RBFs. However, these functions are not fully supported by GPU devices and are still considered less popular. In this paper, we propose the use of fast computational functions, such as ReLU and trigonometric functions (e.g., ReLU, sin, cos, arctan), as basis components in Kolmogorov-Arnold Networks (KANs). By integrating these function combinations into the network structure, we aim to enhance computational efficiency. Experimental results show that these combinations maintain competitive performance while offering potential improvements in training time and generalization.

Method: Developed PCA-Grad-CAM for visualizing attention in PCA feature vectors and SVM-Grad-CAM for SVM classifier layers. Solved the closed-form Jacobian consisting of partial derivatives from the last convolutional layer to PCA/SVM layers.

Result: Presented exact closed-form Jacobian solutions and demonstrated visualization results on several major datasets, enabling attention region generation for PCA and SVM layers.

Conclusion: The proposed methods successfully extend Grad-CAM visualization to PCA and SVM layers in CNNs, making hybrid architectures more interpretable and supporting white-box analysis of these improved classification models.

Abstract: Convolutional Neural Networks (CNNs) are an effective approach for classification tasks, particularly when the training dataset is large. Although CNNs have long been considered a black-box classification method, they can be used as a white-box method through visualization techniques such as Grad-CAM. When training samples are limited, incorporating a Principal Component Analysis (PCA) layer and/or a Support Vector Machine (SVM) classifier into a CNN can effectively improve classification performance. However, traditional Grad-CAM cannot be directly applied to PCA and/or SVM layers. It is important to generate attention regions for PCA and/or SVM layers in CNNs to facilitate the development of white-box methods. Therefore, we propose PCA-Grad-CAM'', a method for visualizing attention regions in PCA feature vectors, and SVM-Grad-CAM’’, a method for visualizing attention regions in an SVM classifier layer. To complete our methods analytically, it is necessary to solve the closed-form Jacobian consisting of partial derivatives from the last convolutional layer to the PCA and/or SVM layers. In this paper, we present the exact closed-form Jacobian and the visualization results of our methods applied to several major datasets.

Method: Proposes Efficient N-dimensional Attention (ENA) - a hybrid architecture combining linear recurrence (for global information compression) with tiled high-order sliding window attention (for strict local modeling).

Result: Empirical results show attention-hybrid models yield promising results, with tiled high-order sliding window attention being efficient both theoretically and practically.

Conclusion: ENA provides a simple, promising, and practical solution for ultra-long high-order data modeling by effectively combining global compression through recurrence with local modeling through attention.

Abstract: Efficient modeling of long sequences of high-order data requires a more efficient architecture than Transformer. In this paper, we investigate two key aspects of extending linear recurrent models, especially those originally designed for language modeling, to high-order data (1D to ND): scanning strategies and attention-hybrid architectures. Empirical results suggest that scanning provides limited benefits, while attention-hybrid models yield promising results. Focusing on the latter, we further evaluate types of attention and find that tiled high-order sliding window attention (SWA) is efficient in both theory and practice. We term the resulting hybrid architecture of linear recurrence and high-order SWA as Efficient N-dimensional Attention (ENA). We then conduct several experiments to demonstrate its effectiveness. The intuition behind ENA is that linear recurrence compresses global information into a state, while SWA complements it by enforcing strict local modeling. Together, they form a simple framework that offers a promising and practical solution for ultra-long high-order data modeling.

Method: Proposes SDSTM framework with dual-stream feature decomposition using Koopman lifting operator and gated wavelet decomposition, plus a fusion mechanism with cross-term loss for independence constraints.

Result: Extensive experiments on Xi’an’s Second Ring Road traffic emission dataset show state-of-the-art performance in long-term forecasting accuracy.

Conclusion: The scale-disentangled approach effectively addresses multi-scale entanglement issues and significantly improves long-term traffic emission prediction performance.

Abstract: Long-term traffic emission forecasting is crucial for the comprehensive management of urban air pollution. Traditional forecasting methods typically construct spatiotemporal graph models by mining spatiotemporal dependencies to predict emissions. However, due to the multi-scale entanglement of traffic emissions across time and space, these spatiotemporal graph modeling method tend to suffer from cascading error amplification during long-term inference. To address this issue, we propose a Scale-Disentangled Spatio-Temporal Modeling (SDSTM) framework for long-term traffic emission forecasting. It leverages the predictability differences across multiple scales to decompose and fuse features at different scales, while constraining them to remain independent yet complementary. Specifically, the model first introduces a dual-stream feature decomposition strategy based on the Koopman lifting operator. It lifts the scale-coupled spatiotemporal dynamical system into an infinite-dimensional linear space via Koopman operator, and delineates the predictability boundary using gated wavelet decomposition. Then a novel fusion mechanism is constructed, incorporating a dual-stream independence constraint based on cross-term loss to dynamically refine the dual-stream prediction results, suppress mutual interference, and enhance the accuracy of long-term traffic emission prediction. Extensive experiments conducted on a road-level traffic emission dataset within Xi’an’s Second Ring Road demonstrate that the proposed model achieves state-of-the-art performance.

Method: Reduces the setting to misspecification-robust adversarial linear bandits with fixed action sets, without requiring knowledge of context distribution or access to a context simulator.

Result: Achieves Õ(min{d²√T, √(d³T log K)}) regret in poly(d,C,T) time, and improves to Õ(d√L*) when a simulator is available (where L* is the best policy’s cumulative loss).

Conclusion: First algorithm to achieve poly(d)√T regret in polynomial time for combinatorial bandits with adversarial losses and stochastic action sets, resolving Liu et al. (2023)’s open question.

Abstract: We present an efficient algorithm for linear contextual bandits with adversarial losses and stochastic action sets. Our approach reduces this setting to misspecification-robust adversarial linear bandits with fixed action sets. Without knowledge of the context distribution or access to a context simulator, the algorithm achieves $\tilde{O}(\min{d^2\sqrt{T}, \sqrt{d^3T\log K}})$ regret and runs in $\text{poly}(d,C,T)$ time, where $d$ is the feature dimension, $C$ is an upper bound on the number of linear constraints defining the action set in each round, $K$ is an upper bound on the number of actions in each round, and $T$ is number of rounds. This resolves the open question by Liu et al. (2023) on whether one can obtain $\text{poly}(d)\sqrt{T}$ regret in polynomial time independent of the number of actions. For the important class of combinatorial bandits with adversarial losses and stochastic action sets where the action sets can be described by a polynomial number of linear constraints, our algorithm is the first to achieve $\text{poly}(d)\sqrt{T}$ regret in polynomial time, while no prior algorithm achieves even $o(T)$ regret in polynomial time to our knowledge. When a simulator is available, the regret bound can be improved to $\tilde{O}(d\sqrt{L^\star})$, where $L^\star$ is the cumulative loss of the best policy.

Method: Uses meta-learning with multimodal embeddings and handcrafted meta-features to represent datasets and learn from historical model behaviors. Combines distributional and cross-modal characteristics to enable rapid adaptation to new data shifts with minimal supervision.

Result: M3OOD consistently outperforms 10 competitive baselines across 12 test scenarios with minimal computational overhead, demonstrating effective automated detector selection.

Conclusion: The framework successfully addresses the challenge of OOD detector selection in multimodal settings by leveraging meta-learning and historical performance data, providing a practical solution for real-world applications.

Abstract: Out-of-distribution (OOD) robustness is a critical challenge for modern machine learning systems, particularly as they increasingly operate in multimodal settings involving inputs like video, audio, and sensor data. Currently, many OOD detection methods have been proposed, each with different designs targeting various distribution shifts. A single OOD detector may not prevail across all the scenarios; therefore, how can we automatically select an ideal OOD detection model for different distribution shifts? Due to the inherent unsupervised nature of the OOD detection task, it is difficult to predict model performance and find a universally Best model. Also, systematically comparing models on the new unseen data is costly or even impractical. To address this challenge, we introduce M3OOD, a meta-learning-based framework for OOD detector selection in multimodal settings. Meta learning offers a solution by learning from historical model behaviors, enabling rapid adaptation to new data distribution shifts with minimal supervision. Our approach combines multimodal embeddings with handcrafted meta-features that capture distributional and cross-modal characteristics to represent datasets. By leveraging historical performance across diverse multimodal benchmarks, M3OOD can recommend suitable detectors for a new data distribution shift. Experimental evaluation demonstrates that M3OOD consistently outperforms 10 competitive baselines across 12 test scenarios with minimal computational overhead.

Method: Decouple forward noise simulation from backward gradient computation using an extended Straight-Through Estimator (STE) framework, enabling use of more accurate but computationally intractable noise models while maintaining optimization stability.

Result: Achieves up to 5.3% accuracy improvement on image classification, 0.72 perplexity reduction on text generation, 2.2x training speedup, and 37.9% lower peak memory usage compared to standard noise-aware training methods.

Conclusion: The extended STE framework provides an effective solution for noise-aware training in analog CIM systems, enabling accurate noise modeling while maintaining computational tractability and delivering substantial performance improvements.

Abstract: Analog Compute-In-Memory (CIM) architectures promise significant energy efficiency gains for neural network inference, but suffer from complex hardware-induced noise that poses major challenges for deployment. While noise-aware training methods have been proposed to address this issue, they typically rely on idealized and differentiable noise models that fail to capture the full complexity of analog CIM hardware variations. Motivated by the Straight-Through Estimator (STE) framework in quantization, we decouple forward noise simulation from backward gradient computation, enabling noise-aware training with more accurate but computationally intractable noise modeling in analog CIM systems. We provide theoretical analysis demonstrating that our approach preserves essential gradient directional information while maintaining computational tractability and optimization stability. Extensive experiments show that our extended STE framework achieves up to 5.3% accuracy improvement on image classification, 0.72 perplexity reduction on text generation, 2.2$\times$ speedup in training time, and 37.9% lower peak memory usage compared to standard noise-aware training methods.

Method: The paper defines Explanation for MTPP as a combination of counterfactual and factual explanations. It proposes CFF (Counterfactual and Factual Explainer) with deliberately designed techniques to identify the minimum subset of historical events that maintains prediction accuracy comparable to using the full history.

Result: Experiments demonstrate that CFF achieves superior performance over baseline methods in both explanation quality and processing efficiency, showing correctness and effectiveness.

Conclusion: The proposed CFF framework successfully addresses the limitations of purely counterfactual or factual explanations for MTPP models, providing minimal and rational explanations that enhance trustworthiness in high-stakes applications.

Abstract: Neural network-based Marked Temporal Point Process (MTPP) models have been widely adopted to model event sequences in high-stakes applications, raising concerns about the trustworthiness of outputs from these models. This study focuses on Explanation for MTPP, aiming to identify the minimal and rational explanation, that is, the minimum subset of events in history, based on which the prediction accuracy of MTPP matches that based on full history to a great extent and better than that based on the complement of the subset. This study finds that directly defining Explanation for MTPP as counterfactual explanation or factual explanation can result in irrational explanations. To address this issue, we define Explanation for MTPP as a combination of counterfactual explanation and factual explanation. This study proposes Counterfactual and Factual Explainer for MTPP (CFF) to solve Explanation for MTPP with a series of deliberately designed techniques. Experiments demonstrate the correctness and superiority of CFF over baselines regarding explanation quality and processing efficiency.

Method: MOSAIC algorithm combines (1) knowledge selection using performance signals and cosine similarity on Wasserstein task embeddings, (2) modular neural representations via masks, and (3) policy integration, composition and fine-tuning.

Result: MOSAIC outperforms isolated learners and global sharing approaches in both learning speed and overall performance, solves tasks that isolated agents cannot, shows less susceptibility to task interference, and demonstrates emergent self-organization where simpler tasks accelerate learning of harder ones.

Conclusion: Selective, goal-driven policy reuse through the MOSAIC framework enables effective collective learning in agentic AI systems, demonstrating the value of modular sharing and composition for accelerating learning and solving complex tasks.

Abstract: Agentic AI aims to create systems that set their own goals, adapt proactively to change, and refine behavior through continuous experience. Recent advances suggest that, when facing multiple and unforeseen tasks, agents could benefit from sharing machine-learned knowledge and reuse policies that have already been fully or partially learned by other agents. However, how to query, select, and retrieve policies from a pool of agents, and how to integrate such policies remains a largely unexplored area. This study explores how an agent decides what knowledge to select, from whom, and when and how to integrate it in its own policy in order to accelerate its own learning. The proposed algorithm, \emph{Modular Sharing and Composition in Collective Learning} (MOSAIC), improves learning in agentic collectives by combining (1) knowledge selection using performance signals and cosine similarity on Wasserstein task embeddings, (2) modular and transferable neural representations via masks, and (3) policy integration, composition and fine-tuning. MOSAIC outperforms isolated learners and global sharing approaches in both learning speed and overall performance, and in some cases solves tasks that isolated agents cannot. The results also demonstrate that selective, goal-driven reuse leads to less susceptibility to task interference. We also observe the emergence of self-organization, where agents solving simpler tasks accelerate the learning of harder ones through shared knowledge.

Method: Uses transformer to measure temporal similarity of micro-trip condition variations, mapping high-dimensional emission data to low-dimensional feature space. Then applies set-valued identification algorithm for probabilistic modeling between feature vectors and labels.

Result: Extensive experiments on 2020 Hefei diesel vehicle data show 9.5% reduction in missed detection rate for high-emission vehicles compared to transformer baseline.

Conclusion: SVTN effectively addresses long-tailed distribution problem and enhances detection accuracy for high-emission mobile pollution sources through comprehensive discriminative feature learning.

Abstract: Identifying high-emission vehicles is a crucial step in regulating urban pollution levels and formulating traffic emission reduction strategies. However, in practical monitoring data, the proportion of high-emission state data is significantly lower compared to normal emission states. This characteristic long-tailed distribution severely impedes the extraction of discriminative features for emission state identification during data mining. Furthermore, the highly nonlinear nature of vehicle emission states and the lack of relevant prior knowledge also pose significant challenges to the construction of identification models.To address the aforementioned issues, we propose a Set-Valued Transformer Network (SVTN) to achieve comprehensive learning of discriminative features from high-emission samples, thereby enhancing detection accuracy. Specifically, this model first employs the transformer to measure the temporal similarity of micro-trip condition variations, thus constructing a mapping rule that projects the original high-dimensional emission data into a low-dimensional feature space. Next, a set-valued identification algorithm is used to probabilistically model the relationship between the generated feature vectors and their labels, providing an accurate metric criterion for the classification algorithm. To validate the effectiveness of our proposed approach, we conducted extensive experiments on the diesel vehicle monitoring data of Hefei city in 2020. The results demonstrate that our method achieves a 9.5% reduction in the missed detection rate for high-emission vehicles compared to the transformer-based baseline, highlighting its superior capability in accurately identifying high-emission mobile pollution sources.

Method: Train LoRA adapters (rank 4, alpha=64) on GPT-2 Small for three QA domains (math, medicine, finance) and combine them through naive summation of parameter deltas.

Result: Math+Medicine combination improved perplexity by -9.10% relative to merged-data fine-tuning, while Math+Finance and Finance+Medicine showed +4.54% and +27.56% changes respectively. RMS cosine similarity between LoRA deltas correlates linearly with perplexity change.

Conclusion: Naive summation of LoRA adapters is an effective zero-shot composition method that achieves comparable performance to merged-data training while revealing interference patterns in higher-order combinations.

Abstract: Recent advances in large language models are driven by scale, while parameter-efficient fine-tuning (PEFT) enables updating only a small fraction of parameters. Low-Rank Adaptation (LoRA) stores parameter deltas as the product of two small matrices, which makes them natural building blocks that can be composed. Motivated by the superposition principle, we hypothesize that independently trained LoRA modules on disjoint domains are approximately orthogonal and can be combined by simple addition. Using GPT-2 Small (117M) with LoRA rank 4 and alpha=64, we train adapters for three QA domains (math, medicine, finance). In pairwise tests, adding Math+Medicine adapters improves perplexity by -9.10% relative to merged-data fine-tuning, while Math+Finance and Finance+Medicine change by +4.54% and +27.56%, respectively. Across combinations, the RMS cosine similarity between LoRA deltas correlates positively and approximately linearly with the change in perplexity. Naive summation requires no additional training, can be applied in seconds, and achieves performance comparable to models trained on merged data, while clarifying when interference appears in higher-order compositions.

Method: Develops a spectral filtering algorithm that learns a mapping from past observations to future states using spectral representation, incorporating techniques from online convex optimization and extending to handle asymmetric dynamics and noise correction.

Result: Proves vanishing prediction error for any nonlinear dynamical system with finitely many marginally stable modes, with rates determined by a new quantitative control-theoretic learnability notion.

Conclusion: The method significantly generalizes spectral filtering to handle more complex systems and provides a novel framework for learning marginally stable nonlinear dynamics with theoretical guarantees.

Abstract: We study the fundamental problem of learning a marginally stable unknown nonlinear dynamical system. We describe an algorithm for this problem, based on the technique of spectral filtering, which learns a mapping from past observations to the next based on a spectral representation of the system. Using techniques from online convex optimization, we prove vanishing prediction error for any nonlinear dynamical system that has finitely many marginally stable modes, with rates governed by a novel quantitative control-theoretic notion of learnability. The main technical component of our method is a new spectral filtering algorithm for linear dynamical systems, which incorporates past observations and applies to general noisy and marginally stable systems. This significantly generalizes the original spectral filtering algorithm to both asymmetric dynamics as well as incorporating noise correction, and is of independent interest.

Method: Proposes FedUHD framework using Hyperdimensional Computing (HDC) with two novel designs: (1) client-side kNN-based cluster hypervector removal to handle non-iid data by eliminating outliers, (2) server-side weighted HDC aggregation to balance non-iid data distribution across clients.

Result: Achieves up to 173.6x training speedup, 612.7x better energy efficiency, 271x lower communication cost, 15.50% higher average accuracy across diverse settings, and superior robustness to various noise types compared to state-of-the-art NN-based UFL approaches.

Conclusion: FedUHD demonstrates that HDC-based approaches can effectively address key challenges in unsupervised federated learning, providing significant improvements in efficiency, accuracy, and robustness while maintaining privacy-preserving decentralized learning.

Abstract: Unsupervised federated learning (UFL) has gained attention as a privacy-preserving, decentralized machine learning approach that eliminates the need for labor-intensive data labeling. However, UFL faces several challenges in practical applications: (1) non-independent and identically distributed (non-iid) data distribution across devices, (2) expensive computational and communication costs at the edge, and (3) vulnerability to communication noise. Previous UFL approaches have relied on deep neural networks (NN), which introduce substantial overhead in both computation and communication. In this paper, we propose FedUHD, the first UFL framework based on Hyperdimensional Computing (HDC). HDC is a brain-inspired computing scheme with lightweight training and inference operations, much smaller model size, and robustness to communication noise. FedUHD introduces two novel HDC-based designs to improve UFL performance. On the client side, a kNN-based cluster hypervector removal method addresses non-iid data samples by eliminating detrimental outliers. On the server side, a weighted HDC aggregation technique balances the non-iid data distribution across clients. Our experiments demonstrate that FedUHD achieves up to 173.6x and 612.7x better speedup and energy efficiency, respectively, in training, up to 271x lower communication cost, and 15.50% higher accuracy on average across diverse settings, along with superior robustness to various types of noise compared to state-of-the-art NN-based UFL approaches.

Method: The study focuses on performance equitable fairness methods that regularize client losses, evaluates existing and new approaches, and introduces FairGrad (approximate) and FairGrad* (exact) gradient variance regularization methods.

Result: The authors theoretically explain connections between fairness methods and empirically demonstrate that FairGrad variants improve both fairness and overall model performance in heterogeneous data environments.

Conclusion: FairGrad and FairGrad* are effective approaches for achieving performance equitable fairness in Federated Learning, particularly beneficial in heterogeneous data settings where they enhance both fairness and model performance.

Abstract: Federated Learning (FL) has emerged as a vital paradigm in modern machine learning that enables collaborative training across decentralized data sources without exchanging raw data. This approach not only addresses privacy concerns but also allows access to overall substantially larger and potentially more diverse datasets, without the need for centralized storage or hardware resources. However, heterogeneity in client data may cause certain clients to have disproportionate impacts on the global model, leading to disparities in the clients’ performances. Fairness, therefore, becomes a crucial concern in FL and can be addressed in various ways. However, the effectiveness of existing fairness-aware methods, particularly in heterogeneous data settings, remains unclear, and the relationships between different approaches are not well understood. In this work, we focus on performance equitable fairness, which aims to minimize differences in performance across clients. We restrict our study to fairness-aware methods that explicitly regularize client losses, evaluating both existing and newly proposed approaches. We identify and theoretically explain connections between the investigated fairness methods, and empirically show that FairGrad (approximate) and FairGrad* (exact) (two variants of a gradient variance regularization method introduced here for performance equitable fairness) improve both fairness and overall model performance in heterogeneous data settings.

Method: Uses layer-specialized probing heads and data-dependent weighting to dynamically prioritize different layers’ features based on the specific input, particularly effective with LoRA fine-tuning.

Result: Superior performance on automatic speech recognition and speech emotion recognition tasks compared to static aggregation methods.

Conclusion: VARAN resolves the trade-off between preserving layer-specific information and enabling flexible feature utilization, advancing efficient adaptation of self-supervised speech representations.

Abstract: Conventional methods for aggregating layers in fine-tuned self-supervised speech models, such as using the final layer or weighted sum, suffer from information bottlenecks and static feature weighting for all dataset examples. We propose VARAN, a framework that dynamically tailors layer aggregation to individual inputs. By employing layer-specialized probing heads and data-dependent weighting, VARAN adaptively prioritizes layer’s features based on input. Evaluations on automatic speech recognition and speech emotion recognition tasks demonstrate VARAN’s superior performance, particularly when using the LoRA fine-tuning technique. The framework resolves the trade-off between preserving layer-specific information and enabling flexible feature utilization, advancing efficient adaptation of self-supervised speech representations.

Method: Linear Programming guided Deep Reinforcement Learning with action filter (LPDRL-F) that transforms 3D solution space to 2D for efficient resource allocation

Result: LPDRL-F converges 60% faster and improves average CAQA by over 14% compared to existing DRL methods, achieving 50% better AvgCAQA than CGQ-only schemes

Conclusion: The proposed LPDRL-F algorithm effectively solves the complex resource tradeoff problem in ISAC-AIGC networks, significantly improving content quality and accuracy while reducing computational complexity

Abstract: Integrated sensing and communication (ISAC) can enhance artificial intelligence-generated content (AIGC) networks by providing efficient sensing and transmission. Existing AIGC services usually assume that the accuracy of the generated content can be ensured, given accurate input data and prompt, thus only the content generation quality (CGQ) is concerned. However, it is not applicable in ISAC-based AIGC networks, where content generation is based on inaccurate sensed data. Moreover, the AIGC model itself introduces generation errors, which depend on the number of generating steps (i.e., computing resources). To assess the quality of experience of ISAC-based AIGC services, we propose a content accuracy and quality aware service assessment metric (CAQA). Since allocating more resources to sensing and generating improves content accuracy but may reduce communication quality, and vice versa, this sensing-generating (computing)-communication three-dimensional resource tradeoff must be optimized to maximize the average CAQA (AvgCAQA) across all users with AIGC (CAQA-AIGC). This problem is NP-hard, with a large solution space that grows exponentially with users. To solve the CAQA-AIGC problem with low complexity, a linear programming (LP) guided deep reinforcement learning (DRL) algorithm with an action filter (LPDRL-F) is proposed. Through the LP-guided approach and the action filter, LPDRL-F can transform the original three-dimensional solution space to two dimensions, reducing complexity while improving the learning performance of DRL. Simulations show that compared to existing DRL and generative diffusion model algorithms without LP, LPDRL-F converges faster by over 60% and finds better resource allocation solutions, improving AvgCAQA by more than 14%. With LPDRL-F, CAQA-AIGC can achieve an improvement in AvgCAQA of more than 50% compared to existing schemes focusing solely on CGQ.

Method: Decoder-only transformer pretrained on 115B medical events from 118M patients, using autoregressive generation to simulate patient health timelines based on medical history.

Result: Outperformed or matched task-specific supervised models on 78 real-world tasks including diagnosis prediction and healthcare operations, with performance scaling with model size.

Conclusion: CoMET effectively captures clinical dynamics and provides an extensible framework for clinical decision-making and healthcare operations without task-specific training.

Abstract: Realizing personalized medicine at scale calls for methods that distill insights from longitudinal patient journeys, which can be viewed as a sequence of medical events. Foundation models pretrained on large-scale medical event data represent a promising direction for scaling real-world evidence generation and generalizing to diverse downstream tasks. Using Epic Cosmos, a dataset with medical events from de-identified longitudinal health records for 16.3 billion encounters over 300 million unique patient records from 310 health systems, we introduce the Cosmos Medical Event Transformer ( CoMET) models, a family of decoder-only transformer models pretrained on 118 million patients representing 115 billion discrete medical events (151 billion tokens). We present the largest scaling-law study for medical event data, establishing a methodology for pretraining and revealing power-law scaling relationships for compute, tokens, and model size. Based on this, we pretrained a series of compute-optimal models with up to 1 billion parameters. Conditioned on a patient’s real-world history, CoMET autoregressively generates the next medical event, simulating patient health timelines. We studied 78 real-world tasks, including diagnosis prediction, disease prognosis, and healthcare operations. Remarkably for a foundation model with generic pretraining and simulation-based inference, CoMET generally outperformed or matched task-specific supervised models on these tasks, without requiring task-specific fine-tuning or few-shot examples. CoMET’s predictive power consistently improves as the model and pretraining scale. Our results show that CoMET, a generative medical event foundation model, can effectively capture complex clinical dynamics, providing an extensible and generalizable framework to support clinical decision-making, streamline healthcare operations, and improve patient outcomes.

Method: Formulates the problem as multi-armed bandit setup with Prior-scaled Boltzmann Exploration that anchors sampling to original dataset proportions, using 1-Step Look-ahead Reward to update sampling probabilities based on dataset contribution to model improvement.

Result: Achieves up to 2.2% performance improvement across 10 benchmarks when applied to Tulu-v2-mixture collection of 16 instruction-tuning datasets.

Conclusion: DynamixSFT provides an effective dynamic optimization method for instruction-tuning dataset mixtures with comprehensive analysis showing adaptive dynamics and performance gains.

Abstract: As numerous instruction-tuning datasets continue to emerge during the post-training stage, dynamically balancing and optimizing their mixtures has become a critical challenge. To address this, we propose DynamixSFT, a dynamic and automated method for instruction-tuning dataset mixture optimization. We formulate the problem as a multi-armed bandit setup and introduce a Prior-scaled Boltzmann Exploration that softly anchors the updated sampling distribution to the original dataset proportions, thereby preserving the inherent diversity and coverage of the collection. Sampling probabilities are updated using a lightweight 1-Step Look-ahead Reward, reflecting how much the dataset contributes to improving the model’s performance at its current state. When applied to the Tulu-v2-mixture collection comprising 16 instruction-tuning datasets, DynamixSFT achieves up to a 2.2% performance improvement across 10 benchmarks. Furthermore, we provide a comprehensive analysis and visualizations to offer deeper insights into the adaptive dynamics of our method.

Method: Derived exact Jacobians for leaky-integrator and gated RNNs, performed first-order expansion analysis to show how gates reshape gradient propagation and modulate effective step sizes, and conducted numerical experiments to validate the perturbative analysis.

Result: Gates not only control memory retention but also act as data-driven preconditioners that adapt optimization trajectories, introducing anisotropy in parameter updates and creating effects analogous to learning-rate schedules, momentum, and adaptive methods like Adam.

Conclusion: Gating mechanisms provide a unified dynamical-systems perspective that couples state evolution with parameter updates, explaining why gated architectures achieve robust trainability and stability in practice through implicit adaptive optimization behavior.

Abstract: We study how gating mechanisms in recurrent neural networks (RNNs) implicitly induce adaptive learning-rate behavior, even when training is carried out with a fixed, global learning rate. This effect arises from the coupling between state-space time scales–parametrized by the gates–and parameter-space dynamics during gradient descent. By deriving exact Jacobians for leaky-integrator and gated RNNs, we obtain a first-order expansion that makes explicit how constant, scalar, and multi-dimensional gates reshape gradient propagation, modulate effective step sizes, and introduce anisotropy in parameter updates. These findings reveal that gates not only control memory retention in the hidden states, but also act as data-driven preconditioners that adapt optimization trajectories in parameter space. We further draw formal analogies with learning-rate schedules, momentum, and adaptive methods such as Adam, showing that these optimization behaviors emerge naturally from gating. Numerical experiments confirm the validity of our perturbative analysis, supporting the view that gate-induced corrections remain small while exerting systematic effects on training dynamics. Overall, this work provides a unified dynamical-systems perspective on how gating couples state evolution with parameter updates, explaining why gated architectures achieve robust trainability and stability in practice.

Method: DE-VAE uses differential entropy in a variational autoencoder framework to learn both forward mapping to 2D space and inverse mapping back to original space, trained with fixed projection methods like UMAP and t-SNE as baselines.

Result: DE-VAE achieves parametric and inverse projections with accuracy comparable to current AE-based approaches while providing the additional capability of analyzing embedding uncertainty.

Conclusion: The proposed DE-VAE successfully addresses limitations of existing methods by incorporating uncertainty awareness through differential entropy, making it more robust for handling out-of-distribution samples in projection tasks.

Abstract: Recently, autoencoders (AEs) have gained interest for creating parametric and invertible projections of multidimensional data. Parametric projections make it possible to embed new, unseen samples without recalculating the entire projection, while invertible projections allow the synthesis of new data instances. However, existing methods perform poorly when dealing with out-of-distribution samples in either the data or embedding space. Thus, we propose DE-VAE, an uncertainty-aware variational AE using differential entropy (DE) to improve the learned parametric and invertible projections. Given a fixed projection, we train DE-VAE to learn a mapping into 2D space and an inverse mapping back to the original space. We conduct quantitative and qualitative evaluations on four well-known datasets, using UMAP and t-SNE as baseline projection methods. Our findings show that DE-VAE can create parametric and inverse projections with comparable accuracy to other current AE-based approaches while enabling the analysis of embedding uncertainty.

Method: Attention-integrated convolutional residual network (AICRN) with spatial and channel attention mechanisms to address ECG feature types and spatial locations for regression, using convolutional residual networks to prevent vanishing/exploding gradients.

Result: AICRN models outperform existing models in parameter regression with higher precision, demonstrating superior performance in regressing PR interval, QT interval, QRS duration, heart rate, R wave amplitude, and T wave amplitude.

Conclusion: Deep learning can play a crucial role in improving interpretability and precision of ECG analysis, opening new clinical applications for cardiac monitoring and management.

Abstract: The paradigm of electrocardiogram (ECG) analysis has evolved into real-time digital analysis, facilitated by artificial intelligence (AI) and machine learning (ML), which has improved the diagnostic precision and predictive capacity of cardiac diseases. This work proposes a novel deep learning (DL) architecture called the attention-integrated convolutional residual network (AICRN) to regress key ECG parameters such as the PR interval, the QT interval, the QRS duration, the heart rate, the peak amplitude of the R wave, and the amplitude of the T wave for interpretable ECG analysis. Our architecture is specially designed with spatial and channel attention-related mechanisms to address the type and spatial location of the ECG features for regression. The models employ a convolutional residual network to address vanishing and exploding gradient problems. The designed system addresses traditional analysis challenges, such as loss of focus due to human errors, and facilitates the fast and easy detection of cardiac events, thereby reducing the manual efforts required to solve analysis tasks. AICRN models outperform existing models in parameter regression with higher precision. This work demonstrates that DL can play a crucial role in the interpretability and precision of ECG analysis, opening up new clinical applications for cardiac monitoring and management.

Method: Two-stage compression framework: 1) Joint embedding compression fuses sequence and structure representations at residue level, reducing input length by half. 2) Self-compression module aggregates demonstrations into latent space of last few linguistic tokens, reducing demonstration length from 751 to <16 tokens.

Result: Achieves 93.68% compression ratio in total prompt length under 16-shot setting. Improves in-domain benchmark performance by 2% and out-of-domain dataset performance by 11% without modifying backbone model.

Conclusion: ProtTeX-CC effectively addresses ProtTeX’s limitations through lightweight compression techniques, enabling better in-context learning and generalization capabilities for protein function prediction tasks.

Abstract: Recent advances in protein large language models, such as ProtTeX, represent both side-chain amino acids and backbone structure as discrete token sequences of residue length. While this design enables unified modeling of multimodal protein information, it suffers from two major limitations: (1) The concatenation of sequence and structure tokens approximately doubles the protein length and breaks the intrinsic residue-level alignment between modalities. (2) Constrained by the training corpus and limited context window, ProtTeX is typically trained on single-protein inputs, rendering it incompatible with in-context learning (ICL) and thus limiting its generalization capability. To address these issues, we propose ProtTeX-CC, a lightweight two-stage compression framework designed to enhance ProtTeX under few-shot settings. We first design a joint embedding compression mechanism that fuses sequence and structure representations at the residue level, effectively reducing the protein input length by half without sacrificing performance. Then we propose a self-compression module that aggregates each full demonstration into the latent space of the last few linguistic tokens, reducing the average demonstration length from 751 tokens to less than 16 tokens. Compared to the original ProtTeX, our self-compression approach achieves a compression ratio of approximately 93.68% in the total prompt length under the 16-shot setting. Without modifying the backbone model, ProtTeX-CC introduces only a small number of additional parameters through PEFT-based tuning in the joint embedding compression stage and a single trainable projection layer in the self-compression stage. Extensive experiments on protein function prediction show that ProtTeX-CC improves performance on the in-domain benchmark by 2%, and generalizes well to the out-of-domain dataset with a performance gain of 11%.

Method: Logs deterministic training metadata (ID hash, RNG seed, learning rate, optimizer step, accumulation boundary) and uses three complementary approaches: exact reverts via micro-checkpoints, cohort-scoped adapter deletion, and curvature-guided anti-updates with retain-tuning.

Result: Achieves byte-identical equality of model and optimizer states when preconditions are satisfied, demonstrating exact unlearning capabilities with reported storage and latency budgets.

Conclusion: The approach provides a systematic framework for GDPR-compliant unlearning in LLMs through deterministic replay and complementary deletion mechanisms that maintain model integrity while meeting practical system constraints.

Abstract: We study the right to be forgotten (GDPR Art. 17) for large language models and frame unlearning as a reproducible systems problem. Our approach treats training as a deterministic program and logs a minimal per-microbatch record (ordered ID hash, RNG seed, learning-rate value, optimizer-step counter, and accumulation boundary). Under a pinned stack and deterministic kernels, replaying the training tail while filtering only the forget closure yields the same parameters as training on the retain set (bit-identical in the training dtype) when preconditions hold. To meet latency and availability constraints, we add complementary paths: (i) exact reverts of recent steps via micro-checkpoints or dense per-step deltas, (ii) cohort-scoped adapter deletion when the base is frozen, and (iii) a curvature-guided anti-update followed by a short retain-tune, audit-gated with escalation to exact replay. We report storage/latency budgets and a toy artifact validating mechanics; in a controlled run that satisfies the preconditions we demonstrate byte-identical equality of model and optimizer states.

Method: Proposes a distribution matching method based on consistency models from Continuous Normalizing Flows, inheriting CNF’s straightforward norm minimization objective while adapting to various constraints like GANs.

Result: Theoretical validation of proposed objective and experimental demonstration on both synthetic and real-world datasets showing competitive performance.

Conclusion: The approach successfully bridges advantages of CNF models (stable optimization) with GAN flexibility, providing a more stable alternative for distribution matching tasks.

Abstract: Recent advancement in generative models have demonstrated remarkable performance across various data modalities. Beyond their typical use in data synthesis, these models play a crucial role in distribution matching tasks such as latent variable modeling, domain translation, and domain adaptation. Generative Adversarial Networks (GANs) have emerged as the preferred method of distribution matching due to their efficacy in handling high-dimensional data and their flexibility in accommodating various constraints. However, GANs often encounter challenge in training due to their bi-level min-max optimization objective and susceptibility to mode collapse. In this work, we propose a novel approach for distribution matching inspired by the consistency models employed in Continuous Normalizing Flow (CNF). Our model inherits the advantages of CNF models, such as having a straight forward norm minimization objective, while remaining adaptable to different constraints similar to GANs. We provide theoretical validation of our proposed objective and demonstrate its performance through experiments on synthetic and real-world datasets.

Method: Introduces coarse quantization to the data exchanged in asynchronous ADMM to reduce communication overhead while maintaining convergence.

Result: Experimental verification shows the proposed method converges for several distributed learning tasks, including neural networks.

Conclusion: Quantization is an effective approach to reduce communication costs in asynchronous ADMM for large-scale federated learning and distributed optimization applications.

Abstract: In distributed optimization and federated learning, asynchronous alternating direction method of multipliers (ADMM) serves as an attractive option for large-scale optimization, data privacy, straggler nodes and variety of objective functions. However, communication costs can become a major bottleneck when the nodes have limited communication budgets or when the data to be communicated is prohibitively large. In this work, we propose introducing coarse quantization to the data to be exchanged in aynchronous ADMM so as to reduce communication overhead for large-scale federated learning and distributed optimization applications. We experimentally verify the convergence of the proposed method for several distributed learning tasks, including neural networks.

Method: CC-Time uses cross-modality learning to model temporal dependency and channel correlations from both time series sequences and text descriptions, plus cross-model fusion to adaptively integrate knowledge from PLMs and time series models.

Result: Extensive experiments on nine real-world datasets show CC-Time achieves state-of-the-art prediction accuracy in both full-data training and few-shot learning situations.

Conclusion: The proposed cross-model and cross-modality learning approach effectively leverages PLMs for time series forecasting, demonstrating superior performance through comprehensive modeling of time series patterns.

Abstract: With the success of pre-trained language models (PLMs) in various application fields beyond natural language processing, language models have raised emerging attention in the field of time series forecasting (TSF) and have shown great prospects. However, current PLM-based TSF methods still fail to achieve satisfactory prediction accuracy matching the strong sequential modeling power of language models. To address this issue, we propose Cross-Model and Cross-Modality Learning with PLMs for time series forecasting (CC-Time). We explore the potential of PLMs for time series forecasting from two aspects: 1) what time series features could be modeled by PLMs, and 2) whether relying solely on PLMs is sufficient for building time series models. In the first aspect, CC-Time incorporates cross-modality learning to model temporal dependency and channel correlations in the language model from both time series sequences and their corresponding text descriptions. In the second aspect, CC-Time further proposes the cross-model fusion block to adaptively integrate knowledge from the PLMs and time series model to form a more comprehensive modeling of time series patterns. Extensive experiments on nine real-world datasets demonstrate that CC-Time achieves state-of-the-art prediction accuracy in both full-data training and few-shot learning situations.

Method: The authors introduce DHG-Bench which integrates 20 diverse datasets spanning node-, edge-, and graph-level tasks, along with 16 state-of-the-art HNN algorithms, under consistent data processing and experimental protocols. They systematically evaluate HNNs across four dimensions: effectiveness, efficiency, robustness, and fairness.

Result: Extensive experiments reveal both the strengths and inherent limitations of existing hypergraph neural network algorithms, providing valuable insights into their performance characteristics across different evaluation dimensions.

Conclusion: DHG-Bench fills a critical gap in deep hypergraph learning research by providing the first comprehensive benchmark, enabling reproducible research and offering valuable directions for future algorithm development and evaluation in this field.

Abstract: Although conventional deep graph models have achieved great success in relational learning, their focus on pairwise relationships limits their capacity to learn pervasive higher-order interactions in real-world complex systems, which can be naturally modeled as hypergraphs. To tackle this, hypergraph neural networks (HNNs), the dominant approach in deep hypergraph learning (DHGL), has garnered substantial attention in recent years. Despite the proposal of numerous HNN methods, there is no comprehensive benchmark for HNNs, which creates a great obstacle to understanding the progress of DHGL in several aspects: (i) insufficient coverage of datasets, algorithms, and tasks; (ii) a narrow evaluation of algorithm performance; and (iii) inconsistent dataset usage, preprocessing, and experimental setups that hinder comparability. To fill the gap, we introduce DHG-Bench, the first comprehensive benchmark for DHGL. Specifically, DHG-Bench integrates 20 diverse datasets spanning node-, edge-, and graph-level tasks, along with 16 state-of-the-art HNN algorithms, under consistent data processing and experimental protocols. Our benchmark systematically investigates the characteristics of HNNs in terms of four dimensions: effectiveness, efficiency, robustness, and fairness. Further, to facilitate reproducible research, we have developed an easy-to-use library for training and evaluating different HNN methods. Extensive experiments conducted with DHG-Bench reveal both the strengths and inherent limitations of existing algorithms, offering valuable insights and directions for future research. The code is publicly available at: https://github.com/Coco-Hut/DHG-Bench.

Method: STM2 uses multiscale Mamba architecture with adaptive graph causal convolution and hierarchical information aggregation. STM3 enhances this with Mixture-of-Experts architecture, stable routing strategy, and causal contrastive learning.

Result: Extensive experiments on real-world benchmarks demonstrate superior performance, achieving state-of-the-art results in long-term spatio-temporal time-series prediction.

Conclusion: The proposed STM2/STM3 models effectively address the challenges of long-term spatio-temporal dependency learning through efficient multiscale information extraction and complex dependency modeling.

Abstract: Recently, spatio-temporal time-series prediction has developed rapidly, yet existing deep learning methods struggle with learning complex long-term spatio-temporal dependencies efficiently. The long-term spatio-temporal dependency learning brings two new challenges: 1) The long-term temporal sequence includes multiscale information naturally which is hard to extract efficiently; 2) The multiscale temporal information from different nodes is highly correlated and hard to model. To address these challenges, we propose an efficient \textit{\textbf{S}patio-\textbf{T}emporal \textbf{M}ultiscale \textbf{M}amba} (STM2) that includes a multiscale Mamba architecture to capture the multiscale information efficiently and simultaneously, and an adaptive graph causal convolution network to learn the complex multiscale spatio-temporal dependency. STM2 includes hierarchical information aggregation for different-scale information that guarantees their distinguishability. To capture diverse temporal dynamics across all spatial nodes more efficiently, we further propose an enhanced version termed \textit{\textbf{S}patio-\textbf{T}emporal \textbf{M}ixture of \textbf{M}ultiscale \textbf{M}amba} (STM3) that employs a special Mixture-of-Experts architecture, including a more stable routing strategy and a causal contrastive learning strategy to enhance the scale distinguishability. We prove that STM3 has much better routing smoothness and guarantees the pattern disentanglement for each expert successfully. Extensive experiments on real-world benchmarks demonstrate STM2/STM3’s superior performance, achieving state-of-the-art results in long-term spatio-temporal time-series prediction.

Method: Convert univariate time series into leakage-free supervised learning problem, train gradient-boosted tree alongside ARIMA baseline, and apply post-hoc explainability using LIME and SHAP while preserving chronology.

Result: Using Air Passengers dataset, the study shows that a small set of lagged features (particularly twelve-month lag) and seasonal encodings explain most forecast variance.

Conclusion: The paper provides a methodology for applying interpretability techniques to time series, theoretical exposition of algorithms, empirical evaluation, and practical guidelines for practitioners.

Abstract: Time-series forecasting underpins critical decisions across aviation, energy, retail and health. Classical autoregressive integrated moving average (ARIMA) models offer interpretability via coefficients but struggle with nonlinearities, whereas tree-based machine-learning models such as XGBoost deliver high accuracy but are often opaque. This paper presents a unified framework for interpreting time-series forecasts using local interpretable model-agnostic explanations (LIME) and SHapley additive exPlanations (SHAP). We convert a univariate series into a leakage-free supervised learning problem, train a gradient-boosted tree alongside an ARIMA baseline and apply post-hoc explainability. Using the Air Passengers dataset as a case study, we show that a small set of lagged features – particularly the twelve-month lag – and seasonal encodings explain most forecast variance. We contribute: (i) a methodology for applying LIME and SHAP to time series without violating chronology; (ii) theoretical exposition of the underlying algorithms; (iii) empirical evaluation with extensive analysis; and (iv) guidelines for practitioners.

Method: Introduces a trainable preconditioning unit to enhance the Symmetric-Rank-One (SR1) algorithm, generating data-driven vectors to construct positive semi-definite rank-one matrices aligned with secant constraint via learned projection.

Result: Outperforms existing learned optimization-based approaches on Monocular Human Mesh Recovery (HMR), featuring lightweight model, no annotated data requirements, and strong generalization capabilities.

Conclusion: The proposed learned second-order optimizer offers an effective fusion of deep learning and classical optimization, providing data efficiency, strong performance, and suitability for integration into broader optimization-based frameworks.

Abstract: End-to-end deep learning has achieved impressive results but remains limited by its reliance on large labeled datasets, poor generalization to unseen scenarios, and growing computational demands. In contrast, classical optimization methods are data-efficient and lightweight but often suffer from slow convergence. While learned optimizers offer a promising fusion of both worlds, most focus on first-order methods, leaving learned second-order approaches largely unexplored. We propose a novel learned second-order optimizer that introduces a trainable preconditioning unit to enhance the classical Symmetric-Rank-One (SR1) algorithm. This unit generates data-driven vectors used to construct positive semi-definite rank-one matrices, aligned with the secant constraint via a learned projection. Our method is evaluated through analytic experiments and on the real-world task of Monocular Human Mesh Recovery (HMR), where it outperforms existing learned optimization-based approaches. Featuring a lightweight model and requiring no annotated data or fine-tuning, our approach offers strong generalization and is well-suited for integration into broader optimization-based frameworks.

Method: CRoC refactors node contexts by recomposing attributes while preserving interaction patterns, encodes heterogeneous relations separately in message-passing, and integrates contrastive learning to leverage unlabeled data for joint training.

Result: Extensive experiments on seven real-world datasets show CRoC achieves up to 14% AUC improvement over baseline GNNs and outperforms state-of-the-art GAD methods under limited-label settings.

Conclusion: CRoC effectively addresses the label scarcity problem in Graph Anomaly Detection by combining context refactoring with contrastive learning, enabling robust detection of camouflaged anomalies with limited supervision.

Abstract: Graph Neural Networks (GNNs) are widely used as the engine for various graph-related tasks, with their effectiveness in analyzing graph-structured data. However, training robust GNNs often demands abundant labeled data, which is a critical bottleneck in real-world applications. This limitation severely impedes progress in Graph Anomaly Detection (GAD), where anomalies are inherently rare, costly to label, and may actively camouflage their patterns to evade detection. To address these problems, we propose Context Refactoring Contrast (CRoC), a simple yet effective framework that trains GNNs for GAD by jointly leveraging limited labeled and abundant unlabeled data. Different from previous works, CRoC exploits the class imbalance inherent in GAD to refactor the context of each node, which builds augmented graphs by recomposing the attributes of nodes while preserving their interaction patterns. Furthermore, CRoC encodes heterogeneous relations separately and integrates them into the message-passing process, enhancing the model’s capacity to capture complex interaction semantics. These operations preserve node semantics while encouraging robustness to adversarial camouflage, enabling GNNs to uncover intricate anomalous cases. In the training stage, CRoC is further integrated with the contrastive learning paradigm. This allows GNNs to effectively harness unlabeled data during joint training, producing richer, more discriminative node embeddings. CRoC is evaluated on seven real-world GAD datasets with varying scales. Extensive experiments demonstrate that CRoC achieves up to 14% AUC improvement over baseline GNNs and outperforms state-of-the-art GAD methods under limited-label settings.

Method: Theoretical analysis of Lion optimizer convergence under standard assumptions, introduction of variance reduction technique, extension to distributed settings with multiple nodes, and development of communication-efficient variant using sign compression in both communication directions.

Result: Established convergence rates: standard Lion O(d^{1/2}T^{-1/4}), variance-reduced Lion O(d^{1/2}T^{-1/3}), distributed versions O(d^{1/2}(nT)^{-1/4}) and O(d^{1/2}(nT)^{-1/3}), and communication-efficient variants achieving O(max{d^{1/4}/T^{1/4}, d^{1/10}/(n^{1/5}T^{1/5})}) and O(d^{1/4}/T^{1/4}) rates.

Conclusion: The Lion optimizer demonstrates strong convergence properties with various improvements possible through variance reduction and distributed implementations, with communication-efficient variants maintaining competitive convergence rates while reducing communication overhead.

Abstract: In this paper, we analyze the convergence properties of the Lion optimizer. First, we establish that the Lion optimizer attains a convergence rate of $\mathcal{O}(d^{1/2}T^{-1/4})$ under standard assumptions, where $d$ denotes the problem dimension and $T$ is the iteration number. To further improve this rate, we introduce the Lion optimizer with variance reduction, resulting in an enhanced convergence rate of $\mathcal{O}(d^{1/2}T^{-1/3})$. We then analyze in distributed settings, where the standard and variance reduced version of the distributed Lion can obtain the convergence rates of $\mathcal{O}(d^{1/2}(nT)^{-1/4})$ and $\mathcal{O}(d^{1/2}(nT)^{-1/3})$, with $n$ denoting the number of nodes. Furthermore, we investigate a communication-efficient variant of the distributed Lion that ensures sign compression in both communication directions. By employing the unbiased sign operations, the proposed Lion variant and its variance reduction counterpart, achieve convergence rates of $\mathcal{O}\left( \max \left{\frac{d^{1/4}}{T^{1/4}}, \frac{d^{1/10}}{n^{1/5}T^{1/5}} \right} \right)$ and $\mathcal{O}\left( \frac{d^{1/4}}{T^{1/4}} \right)$, respectively.

Method: Proposed two strategies: 1) Funnel Schedule - progressively reduces maintained particles, and 2) Adaptive Temperature - down-weights early-stage reward influence, both tailored to diffusion model dynamics and phase-transition behavior.

Result: Experimental results on multiple benchmarks and state-of-the-art text-to-image diffusion models demonstrate superior performance over previous baselines.

Conclusion: The proposed methods effectively address the exploration-exploitation trade-off in diffusion model inference, significantly enhancing sample quality without increasing Noise Function Evaluations.

Abstract: Inference-time scaling has achieved remarkable success in language models, yet its adaptation to diffusion models remains underexplored. We observe that the efficacy of recent Sequential Monte Carlo (SMC)-based methods largely stems from globally fitting the The reward-tilted distribution, which inherently preserves diversity during multi-modal search. However, current applications of SMC to diffusion models face a fundamental dilemma: early-stage noise samples offer high potential for improvement but are difficult to evaluate accurately, whereas late-stage samples can be reliably assessed but are largely irreversible. To address this exploration-exploitation trade-off, we approach the problem from the perspective of the search algorithm and propose two strategies: Funnel Schedule and Adaptive Temperature. These simple yet effective methods are tailored to the unique generation dynamics and phase-transition behavior of diffusion models. By progressively reducing the number of maintained particles and down-weighting the influence of early-stage rewards, our methods significantly enhance sample quality without increasing the total number of Noise Function Evaluations. Experimental results on multiple benchmarks and state-of-the-art text-to-image diffusion models demonstrate that our approach outperforms previous baselines.

Method: The Bi-Axial Transformer (BAT) attends to both clinical variable and time point axes of EHR data to learn richer data relationships and address data sparsity challenges. Baseline models were re-implemented with PyTorch for fair comparison.

Result: BAT achieves state-of-the-art performance on sepsis prediction and is competitive with top methods for mortality classification. It demonstrates increased robustness to data missingness and learns unique sensor embeddings that enable transfer learning.

Conclusion: The BAT model effectively addresses EHR data challenges by leveraging bi-axial attention, providing superior performance for clinical prediction tasks while offering improved handling of sparse and missing data in electronic health records.

Abstract: Electronic Health Records (EHRs), the digital representation of a patient’s medical history, are a valuable resource for epidemiological and clinical research. They are also becoming increasingly complex, with recent trends indicating larger datasets, longer time series, and multi-modal integrations. Transformers, which have rapidly gained popularity due to their success in natural language processing and other domains, are well-suited to address these challenges due to their ability to model long-range dependencies and process data in parallel. But their application to EHR classification remains limited by data representations, which can reduce performance or fail to capture informative missingness. In this paper, we present the Bi-Axial Transformer (BAT), which attends to both the clinical variable and time point axes of EHR data to learn richer data relationships and address the difficulties of data sparsity. BAT achieves state-of-the-art performance on sepsis prediction and is competitive to top methods for mortality classification. In comparison to other transformers, BAT demonstrates increased robustness to data missingness, and learns unique sensor embeddings which can be used in transfer learning. Baseline models, which were previously located across multiple repositories or utilized deprecated libraries, were re-implemented with PyTorch and made available for reproduction and future benchmarking.

Method: Extracted 200 geometric and statistical descriptors from 13,684 automotive suspension/steering DWG drawings, trained XGBoost/CatBoost/LightGBM models, and used SHAP for explainability.

Result: Achieved nearly 10% mean absolute percentage error across 24 product groups, demonstrating robust scalability beyond part-specific heuristics.

Conclusion: End-to-end CAD-to-cost pipeline enables real-time, ERP-integrated decision support with transparent cost assessments and actionable design insights for Industry 4.0 manufacturing.

Abstract: We present an integrated machine learning framework that transforms how manufacturing cost is estimated from 2D engineering drawings. Unlike traditional quotation workflows that require labor-intensive process planning, our approach about 200 geometric and statistical descriptors directly from 13,684 DWG drawings of automotive suspension and steering parts spanning 24 product groups. Gradient-boosted decision tree models (XGBoost, CatBoost, LightGBM) trained on these features achieve nearly 10% mean absolute percentage error across groups, demonstrating robust scalability beyond part-specific heuristics. By coupling cost prediction with explainability tools such as SHAP, the framework identifies geometric design drivers including rotated dimension maxima, arc statistics and divergence metrics, offering actionable insights for cost-aware design. This end-to-end CAD-to-cost pipeline shortens quotation lead times, ensures consistent and transparent cost assessments across part families and provides a deployable pathway toward real-time, ERP-integrated decision support in Industry 4.0 manufacturing environments.

Method: Uses local distance distributions to estimate cluster cardinality by identifying local minima in distance density, then computes cluster parameters and adaptively adjusts bandwidth and kernel radius threshold during mean shift execution.

Result: Outperformed a recently proposed adaptive mean shift method on its original dataset and demonstrated competitive performance on a broader clustering benchmark.

Conclusion: The proposed adaptive mean shift algorithm effectively handles varying local scale and cluster cardinality through local distance distribution analysis, providing superior performance compared to existing methods.

Abstract: This article presents an adaptive mean shift algorithm designed for datasets with varying local scale and cluster cardinality. Local distance distributions, from a point to all others, are used to estimate the cardinality of the local cluster by identifying a local minimum in the density of the distance distribution. Based on these cardinality estimates, local cluster parameters are then computed for the entire cluster in contrast to KDE-based methods, which provide insight only into localized regions of the cluster. During the mean shift execution, the cluster cardinality estimate is used to adaptively adjust the bandwidth and the mean shift kernel radius threshold. Our algorithm outperformed a recently proposed adaptive mean shift method on its original dataset and demonstrated competitive performance on a broader clustering benchmark.

Method: Uses a dueling Deep Q-Network served by an ONNX sidecar that samples K least-recently-used objects and extracts six lightweight features (age, size, hit count, inter-arrival time, remaining TTL, last origin RTT) to select eviction victims. Includes 500 microsecond timeout fallback to LRU. Trained offline using cache simulation on NGINX access logs.

Result: At 25MB cache: hit ratio improved from 0.1436 to 0.3538 (146% improvement). At 100MB: from 0.7530 to 0.8675 (15% gain). At 400MB: matches classical methods (~0.918). Adds <2% CPU overhead and maintains 95th percentile latency within budget.

Conclusion: Cold-RL successfully integrates reinforcement learning into NGINX with strict SLOs, demonstrating significant performance improvements over traditional caching algorithms while maintaining low overhead and latency constraints.

Abstract: Web proxies such as NGINX commonly rely on least-recently-used (LRU) eviction, which is size agnostic and can thrash under periodic bursts and mixed object sizes. We introduce Cold-RL, a learned eviction policy for NGINX that replaces LRU’s forced-expire path with a dueling Deep Q-Network served by an ONNX sidecar within a strict microsecond budget. On each eviction, Cold-RL samples the K least-recently-used objects, extracts six lightweight features (age, size, hit count, inter-arrival time, remaining TTL, and last origin RTT), and requests a bitmask of victims; a hard timeout of 500 microseconds triggers immediate fallback to native LRU. Policies are trained offline by replaying NGINX access logs through a cache simulator with a simple reward: a retained object earns one point if it is hit again before TTL expiry. We compare against LRU, LFU, size-based, adaptive LRU, and a hybrid baseline on two adversarial workloads. With a 25 MB cache, Cold-RL raises hit ratio from 0.1436 to 0.3538, a 146 percent improvement over the best classical baseline; at 100 MB, from 0.7530 to 0.8675, a 15 percent gain; and at 400 MB it matches classical methods (about 0.918). Inference adds less than 2 percent CPU overhead and keeps 95th percentile eviction latency within budget. To our knowledge, this is the first reinforcement learning eviction policy integrated into NGINX with strict SLOs.

Method: Uses compact logit footprints for open-source models and perplexity fingerprints for black-box APIs. Trains a contrastive encoder to favor cheapest accurate experts within adaptive cost bands. Inference uses single k-NN lookup via FAISS index.

Result: Outperforms baselines across multiple benchmarks, improving accuracy-cost tradeoff by up to 25%, with robust generalization to unseen LLMs and out-of-distribution prompts.

Conclusion: CSCR enables fast, cost-aware routing with microsecond latency, requires no retraining when expert pool changes, and provides significant improvements in cost-efficiency for LLM deployment.

Abstract: We study cost-aware routing for large language models across diverse and dynamic pools of models. Existing approaches often overlook prompt-specific context, rely on expensive model profiling, assume a fixed set of experts, or use inefficient trial-and-error strategies. We introduce Cost-Spectrum Contrastive Routing (CSCR), a lightweight framework that maps both prompts and models into a shared embedding space to enable fast, cost-sensitive selection. CSCR uses compact, fast-to-compute logit footprints for open-source models and perplexity fingerprints for black-box APIs. A contrastive encoder is trained to favor the cheapest accurate expert within adaptive cost bands. At inference time, routing reduces to a single k-NN lookup via a FAISS index, requiring no retraining when the expert pool changes and enabling microsecond latency. Across multiple benchmarks, CSCR consistently outperforms baselines, improving the accuracy-cost tradeoff by up to 25%, while generalizing robustly to unseen LLMs and out-of-distribution prompts.

Method: Iteratively solving constrained problems with trust regions that gradually approach the target measure through geometric annealing with principled time step selection.

Result: The method significantly improves performance in multiple optimal control applications including diffusion-based sampling, transition path sampling, and diffusion model fine-tuning.

Conclusion: Trust region-based geometric annealing provides a systematic and principled approach for solving challenging stochastic optimal control problems where target measures differ substantially from priors.

Abstract: Solving stochastic optimal control problems with quadratic control costs can be viewed as approximating a target path space measure, e.g. via gradient-based optimization. In practice, however, this optimization is challenging in particular if the target measure differs substantially from the prior. In this work, we therefore approach the problem by iteratively solving constrained problems incorporating trust regions that aim for approaching the target measure gradually in a systematic way. It turns out that this trust region based strategy can be understood as a geometric annealing from the prior to the target measure, where, however, the incorporated trust regions lead to a principled and educated way of choosing the time steps in the annealing path. We demonstrate in multiple optimal control applications that our novel method can improve performance significantly, including tasks in diffusion-based sampling, transition path sampling, and fine-tuning of diffusion models.

Method: Competition format where participants trained goal-conditional policies on Neural MMO environment, with evaluation on generalization to novel scenarios

Result: Top solution outperformed baseline by 4x within 8 hours of training on a single 4090 GPU, with over 200 participants submitting solutions

Conclusion: Successful competition demonstrating rapid progress in generalization capabilities, with full open-sourcing of baseline, top solutions, and competition framework

Abstract: We present the results of the NeurIPS 2023 Neural MMO Competition, which attracted over 200 participants and submissions. Participants trained goal-conditional policies that generalize to tasks, maps, and opponents never seen during training. The top solution achieved a score 4x higher than our baseline within 8 hours of training on a single 4090 GPU. We open-source everything relating to Neural MMO and the competition under the MIT license, including the policy weights and training code for our baseline and for the top submissions.

Method: Defines local posterior collapse to reflect individual sample importance, then proposes Latent Reconstruction loss inspired by mathematical properties of injective and composite functions to control posterior collapse without architectural restrictions.

Result: Experimentally evaluated on MNIST, fashionMNIST, Omniglot, CelebA, and FFHQ datasets, successfully controlling posterior collapse across varied datasets.

Conclusion: The proposed LR loss effectively addresses posterior collapse in VAEs without requiring specific network architecture constraints, providing a more flexible solution compared to prior methods.

Abstract: Variational autoencoders (VAEs), one of the most widely used generative models, are known to suffer from posterior collapse, a phenomenon that reduces the diversity of generated samples. To avoid posterior collapse, many prior works have tried to control the influence of regularization loss. However, the trade-off between reconstruction and regularization is not satisfactory. For this reason, several methods have been proposed to guarantee latent identifiability, which is the key to avoiding posterior collapse. However, they require structural constraints on the network architecture. For further clarification, we define local posterior collapse to reflect the importance of individual sample points in the data space and to relax the network constraint. Then, we propose Latent Reconstruction(LR) loss, which is inspired by mathematical properties of injective and composite functions, to control posterior collapse without restriction to a specific architecture. We experimentally evaluate our approach, which controls posterior collapse on varied datasets such as MNIST, fashionMNIST, Omniglot, CelebA, and FFHQ.

Method: Systematic testing with proper hyperparameter selection (learning rate, batch size, gradient steps) and proposing exponential moving average (EMA) momentum technique in parameter space to maintain stable optimization

Result: Reduced unsafe model responses from 16% to approximately 5% while maintaining utility performance, outperforming existing approaches that require additional safety data

Conclusion: Safety problems during fine-tuning can be largely avoided without specialized interventions through proper optimization techniques, providing practical guidelines for maintaining both performance and safety

Abstract: Fine-tuning language models is commonly believed to inevitably harm their safety, i.e., refusing to respond to harmful user requests, even when using harmless datasets, thus requiring additional safety measures. We challenge this belief through systematic testing, showing that poor optimization choices, rather than inherent trade-offs, often cause safety problems, measured as harmful responses to adversarial prompts. By properly selecting key training hyper-parameters, e.g., learning rate, batch size, and gradient steps, we reduce unsafe model responses from 16% to approximately 5%, as measured by keyword matching, while maintaining utility performance. Based on this observation, we propose a simple exponential moving average (EMA) momentum technique in parameter space that preserves safety performance by creating a stable optimization path and retains the original pre-trained model’s safety properties. Our experiments on the Llama families across multiple datasets (Dolly, Alpaca, ORCA) demonstrate that safety problems during fine-tuning can largely be avoided without specialized interventions, outperforming existing approaches that require additional safety data while offering practical guidelines for maintaining both model performance and safety during adaptation.

Method: Organized a data-centric design space into three stages: temporal signal processing, topology extraction, and graph featurization. Evaluated combinations of existing and modified techniques including BOLD signal filtering, sparsification strategies, alternative correlation metrics, and multi-view node/edge features.

Result: Experiments on HCP1200 and ABIDE datasets showed that thoughtful data-centric configurations consistently improve classification accuracy over standard pipelines.

Conclusion: Upstream data decisions play a critical role in brain graph construction, and systematic exploration of the data-centric design space is essential for graph-based neuroimaging.

Abstract: The construction of brain graphs from functional Magnetic Resonance Imaging (fMRI) data plays a crucial role in enabling graph machine learning for neuroimaging. However, current practices often rely on rigid pipelines that overlook critical data-centric choices in how brain graphs are constructed. In this work, we adopt a Data-Centric AI perspective and systematically define and benchmark a data-centric design space for brain graph construction, constrasting with primarily model-centric prior work. We organize this design space into three stages: temporal signal processing, topology extraction, and graph featurization. Our contributions lie less in novel components and more in evaluating how combinations of existing and modified techniques influence downstream performance. Specifically, we study high-amplitude BOLD signal filtering, sparsification and unification strategies for connectivity, alternative correlation metrics, and multi-view node and edge features, such as incorporating lagged dynamics. Experiments on the HCP1200 and ABIDE datasets show that thoughtful data-centric configurations consistently improve classification accuracy over standard pipelines. These findings highlight the critical role of upstream data decisions and underscore the importance of systematically exploring the data-centric design space for graph-based neuroimaging. Our code is available at https://github.com/GeQinwen/DataCentricBrainGraphs.

Method: Abstracts kernel configuration as RL environment, uses LLMs for exploration with custom reward functions, and employs two-phase training for faster convergence.

Result: Achieves up to 5.6% performance improvement over heuristic tuning while maintaining high data efficiency across diverse applications.

Conclusion: OS-R1 demonstrates practical adaptability for real-world deployment and outperforms existing baseline methods significantly.

Abstract: Linux kernel tuning is essential for optimizing operating system (OS) performance. However, existing methods often face challenges in terms of efficiency, scalability, and generalization. This paper introduces OS-R1, an agentic Linux kernel tuning framework powered by rule-based reinforcement learning (RL). By abstracting the kernel configuration space as an RL environment, OS-R1 facilitates efficient exploration by large language models (LLMs) and ensures accurate configuration modifications. Additionally, custom reward functions are designed to enhance reasoning standardization, configuration modification accuracy, and system performance awareness of the LLMs. Furthermore, we propose a two-phase training process that accelerates convergence and minimizes retraining across diverse tuning scenarios. Experimental results show that OS-R1 significantly outperforms existing baseline methods, achieving up to 5.6% performance improvement over heuristic tuning and maintaining high data efficiency. Notably, OS-R1 is adaptable across various real-world applications, demonstrating its potential for practical deployment in diverse environments. Our dataset and code are publicly available at https://github.com/LHY-24/OS-R1.

Method: Developed a visual analytics system focusing on AIDE framework with three-level comparative analysis: code-level (debugging/refinement), process-level (solution-seeking), and LLM-level (behavior variations across models).

Result: Case studies using Kaggle competitions demonstrate the system provides valuable insights into iterative coding processes and facilitates effective debugging and prompt engineering.

Conclusion: The integrated visual analytics approach enables structured understanding of coding agent behaviors, addressing current limitations in manual inspection methods.

Abstract: Coding agents powered by large language models (LLMs) have gained traction for automating code generation through iterative problem-solving with minimal human involvement. Despite the emergence of various frameworks, e.g., LangChain, AutoML, and AIDE, ML scientists still struggle to effectively review and adjust the agents’ coding process. The current approach of manually inspecting individual outputs is inefficient, making it difficult to track code evolution, compare coding iterations, and identify improvement opportunities. To address this challenge, we introduce a visual analytics system designed to enhance the examination of coding agent behaviors. Focusing on the AIDE framework, our system supports comparative analysis across three levels: (1) Code-Level Analysis, which reveals how the agent debugs and refines its code over iterations; (2) Process-Level Analysis, which contrasts different solution-seeking processes explored by the agent; and (3) LLM-Level Analysis, which highlights variations in coding behavior across different LLMs. By integrating these perspectives, our system enables ML scientists to gain a structured understanding of agent behaviors, facilitating more effective debugging and prompt engineering. Through case studies using coding agents to tackle popular Kaggle competitions, we demonstrate how our system provides valuable insights into the iterative coding process.

Method: Uses variational mode decomposition (VMD) to break down non-stationary financial time series into smoother subcomponents, then feeds these into an LSTM deep learning model for prediction. Compares with raw time series approach.

Result: The VMD-processed LSTM model demonstrates better forecasting performance and stability compared to models using raw time series data.

Conclusion: Combining VMD decomposition with deep learning models effectively handles financial time series complexity and improves forecasting accuracy and reliability.

Abstract: To address the complexity of financial time series, this paper proposes a forecasting model combining sliding window and variational mode decomposition (VMD) methods. Historical stock prices and relevant market indicators are used to construct datasets. VMD decomposes non-stationary financial time series into smoother subcomponents, improving model adaptability. The decomposed data is then input into a deep learning model for prediction. The study compares the forecasting effects of an LSTM model trained on VMD-processed sequences with those using raw time series, demonstrating better performance and stability.

Method: Proposes a framework using metriplectic bracket formalism that guarantees discrete thermodynamic laws, momentum conservation, and fluctuation-dissipation balance. Uses self-supervised learning to identify emergent structural variables when labels are unavailable.

Result: Validated on benchmark systems and demonstrated utility on challenging examples: coarse-graining star polymers while preserving non-equilibrium statistics, and learning models from high-speed video of colloidal suspensions capturing local rearrangement-emergent dynamics coupling.

Conclusion: The framework successfully captures essential physics of coarse-grained systems while preserving thermodynamic consistency, with open-source implementations provided for extensibility to diverse particle-based systems.

Abstract: Multiscale systems are ubiquitous in science and technology, but are notoriously challenging to simulate as short spatiotemporal scales must be appropriately linked to emergent bulk physics. When expensive high-dimensional dynamical systems are coarse-grained into low-dimensional models, the entropic loss of information leads to emergent physics which are dissipative, history-dependent, and stochastic. To machine learn coarse-grained dynamics from time-series observations of particle trajectories, we propose a framework using the metriplectic bracket formalism that preserves these properties by construction; most notably, the framework guarantees discrete notions of the first and second laws of thermodynamics, conservation of momentum, and a discrete fluctuation-dissipation balance crucial for capturing non-equilibrium statistics. We introduce the mathematical framework abstractly before specializing to a particle discretization. As labels are generally unavailable for entropic state variables, we introduce a novel self-supervised learning strategy to identify emergent structural variables. We validate the method on benchmark systems and demonstrate its utility on two challenging examples: (1) coarse-graining star polymers at challenging levels of coarse-graining while preserving non-equilibrium statistics, and (2) learning models from high-speed video of colloidal suspensions that capture coupling between local rearrangement events and emergent stochastic dynamics. We provide open-source implementations in both PyTorch and LAMMPS, enabling large-scale inference and extensibility to diverse particle-based systems.

Method: Leveraged pretrained protein large language model to extract contextual features, combined with bidirectional LSTM and GRU neural networks to predict amyloidogenic regions in protein sequences.

Result: Achieved 84.5% accuracy on 10-fold cross-validation and 83% accuracy on test dataset, demonstrating competitive performance compared to existing methods.

Conclusion: Protein large language models show significant potential for enhancing amyloid prediction accuracy, representing a promising computational approach in bioinformatics for identifying amyloidogenic regions.

Abstract: The prediction of amyloidogenicity in peptides and proteins remains a focal point of ongoing bioinformatics. The crucial step in this field is to apply advanced computational methodologies. Many recent approaches to predicting amyloidogenicity within proteins are highly based on evolutionary motifs and the individual properties of amino acids. It is becoming increasingly evident that the sequence information-based features show high predictive performance. Consequently, our study evaluated the contextual features of protein sequences obtained from a pretrained protein large language model leveraging bidirectional LSTM and GRU to predict amyloidogenic regions in peptide and protein sequences. Our method achieved an accuracy of 84.5% on 10-fold cross-validation and an accuracy of 83% in the test dataset. Our results demonstrate competitive performance, highlighting the potential of LLMs in enhancing the accuracy of amyloid prediction.

Method: Theoretical analysis of FedAvg convergence with gradient descent, proving that data heterogeneity impact diminishes with increasing neural network width. Experiments validate findings across various architectures, loss functions, and optimization methods.

Result: Data heterogeneity effects vanish as neural network width approaches infinity. In infinite-width regime, both global and local models behave as linear models, and FedAvg achieves identical generalization performance to centralized learning with same number of GD iterations.

Conclusion: Overparameterization in federated learning effectively mitigates data heterogeneity challenges, with infinite-width neural networks enabling FedAvg to match centralized learning performance while preserving data privacy.

Abstract: Federated learning (FL) enables decentralized clients to train a model collaboratively without sharing local data. A key distinction between FL and centralized learning is that clients’ data are non-independent and identically distributed, which poses significant challenges in training a global model that generalizes well across heterogeneous local data distributions. In this paper, we analyze the convergence of overparameterized FedAvg with gradient descent (GD). We prove that the impact of data heterogeneity diminishes as the width of neural networks increases, ultimately vanishing when the width approaches infinity. In the infinite-width regime, we further prove that both the global and local models in FedAvg behave as linear models, and that FedAvg achieves the same generalization performance as centralized learning with the same number of GD iterations. Extensive experiments validate our theoretical findings across various network architectures, loss functions, and optimization methods.

Method: Proposes a token-level filtering mechanism that uses both epistemic uncertainty and attention-based importance to opportunistically upload only informative tokens to cloud-based LLMs, reducing communication costs and LLM usage.

Result: Achieves up to 87.5% BERT Score, 0.37 tokens/sec throughput, and 40.7% energy savings compared to standard HLM. Outperforms previous U-HLM baseline with improved BERTScore (85.8% to 87.0%), energy savings (31.6% to 43.6%), and throughput (0.36 to 0.40).

Conclusion: The approach enables energy-efficient and accurate deployment of LLMs in bandwidth-constrained edge environments by selectively processing only the most informative tokens through hybrid local-cloud inference.

Abstract: To address the growing demand for on-device LLM inference in resource-constrained environments, hybrid language models (HLM) have emerged, combining lightweight local models with powerful cloud-based LLMs. Recent studies on HLM have primarily focused on improving accuracy and latency, while often overlooking communication and energy efficiency. We propose a token-level filtering mechanism for an energy-efficient importance- and uncertainty-aware HLM inference that leverages both epistemic uncertainty and attention-based importance. Our method opportunistically uploads only informative tokens, reducing LLM usage and communication costs. Experiments with TinyLlama-1.1B and LLaMA-2-7B demonstrate that our method achieves up to 87.5% BERT Score and token throughput of 0.37 tokens/sec while saving the energy consumption by 40.7% compared to standard HLM. Furthermore, compared to our previous U-HLM baseline, our method improves BERTScore from 85.8% to 87.0%, energy savings from 31.6% to 43.6%, and throughput from 0.36 to 0.40. This approach enables an energy-efficient and accurate deployment of LLMs in bandwidth-constrained edge environments.

Method: Physics-informed deep operator network (PI-DeepONet) that reformulates TSE as operator learning problem. Trains parameterized neural operator mapping sparse input data to full spatiotemporal traffic state field, integrating traffic flow conservation law and fundamental diagram directly into operator learning.

Result: Superior performance over state-of-the-art baselines on NGSIM dataset. Framework captures congestion propagation, spatial correlations, and temporal evolution while ensuring physical consistency. Analysis reveals optimal function generation strategies and branch network complexity impacts.

Conclusion: PI-DeepONet provides an effective operator learning framework for traffic state estimation that successfully integrates physical constraints, outperforms existing methods, and offers insights into optimal network design and function generation strategies.

Abstract: Traffic state estimation (TSE) fundamentally involves solving high-dimensional spatiotemporal partial differential equations (PDEs) governing traffic flow dynamics from limited, noisy measurements. While Physics-Informed Neural Networks (PINNs) enforce PDE constraints point-wise, this paper adopts a physics-informed deep operator network (PI-DeepONet) framework that reformulates TSE as an operator learning problem. Our approach trains a parameterized neural operator that maps sparse input data to the full spatiotemporal traffic state field, governed by the traffic flow conservation law. Crucially, unlike PINNs that enforce PDE constraints point-wise, PI-DeepONet integrates traffic flow conservation model and the fundamental diagram directly into the operator learning process, ensuring physical consistency while capturing congestion propagation, spatial correlations, and temporal evolution. Experiments on the NGSIM dataset demonstrate superior performance over state-of-the-art baselines. Further analysis reveals insights into optimal function generation strategies and branch network complexity. Additionally, the impact of input function generation methods and the number of functions on model performance is explored, highlighting the robustness and efficacy of proposed framework.

Method: FLARE projects input sequences onto fixed-length latent sequences using learnable query tokens, routing attention through a bottleneck to achieve O(NM) complexity where M « N.

Result: FLARE scales to unprecedented problem sizes and delivers superior accuracy compared to state-of-the-art neural PDE surrogates across diverse benchmarks.

Conclusion: FLARE provides an efficient low-rank attention mechanism that enables practical application of self-attention to large-scale problems while maintaining competitive performance.

Abstract: The quadratic complexity of self-attention limits its applicability and scalability on large unstructured meshes. We introduce Fast Low-rank Attention Routing Engine (FLARE), a linear complexity self-attention mechanism that routes attention through fixed-length latent sequences. Each attention head performs global communication among $N$ tokens by projecting the input sequence onto a fixed length latent sequence of $M \ll N$ tokens using learnable query tokens. By routing attention through a bottleneck sequence, FLARE learns a low-rank form of attention that can be applied at $O(NM)$ cost. FLARE not only scales to unprecedented problem sizes, but also delivers superior accuracy compared to state-of-the-art neural PDE surrogates across diverse benchmarks. We also release a new additive manufacturing dataset to spur further research. Our code is available at https://github.com/vpuri3/FLARE.py.

Method: Presents a systematic method using symmetric tensor networks to construct valid invariant and equivariant operations for both Cartesian tensors with different ranks and spherical tensors with different types. Features graphical representation for simplified proofs and constructions.

Result: Developed a framework that can handle diverse tensor inputs and outputs, and applied it to design equivariant interaction messages for geometry graph neural networks and equivariant machine learning models for material constitutive law learning.

Conclusion: The method provides a comprehensive approach for building symmetry-aware neural network operations with practical applications in geometric deep learning and materials science.

Abstract: Design of neural networks that incorporate symmetry is crucial for geometric deep learning. Central to this effort is the development of invariant and equivariant operations. This works presents a systematic method for constructing valid invariant and equivariant operations. It can handle inputs and outputs in the form of Cartesian tensors with different rank, as well as spherical tensors with different types. In addition, our method features a graphical representation utilizing the symmetric tensor network, which simplifies both the proofs and constructions related to invariant and equivariant functions. We also apply this approach to design the equivariant interaction message for the geometry graph neural network, and equivariant machine learning model to learn the constitutive law of materials.

Method: Combines Spectral Parameter Operator (Fourier Neural Operator backbone) for global context with differentiable physics module in forward pass. Outputs time-varying motor/regenerative efficiencies, drag, rolling resistance, mass, and auxiliary power without separate physics-residual loss.

Result: Achieves 0.2kW MAE (~1% of average traction power) for Tesla vehicles and 0.8kW for Kia EV9. Generalizes well to unseen conditions and sampling rates.

Conclusion: The framework is interpretable, practical for path optimization, eco-routing, diagnostics, and health management, with physically meaningful parameter convergence.

Abstract: We present a hybrid surrogate model for electric vehicle parameter estimation and power consumption. We combine our novel architecture Spectral Parameter Operator built on a Fourier Neural Operator backbone for global context and a differentiable physics module in the forward pass. From speed and acceleration alone, it outputs time-varying motor and regenerative braking efficiencies, as well as aerodynamic drag, rolling resistance, effective mass, and auxiliary power. These parameters drive a physics-embedded estimate of battery power, eliminating any separate physics-residual loss. The modular design lets representations converge to physically meaningful parameters that reflect the current state and condition of the vehicle. We evaluate on real-world logs from a Tesla Model 3, Tesla Model S, and the Kia EV9. The surrogate achieves a mean absolute error of 0.2kW (about 1% of average traction power at highway speeds) for Tesla vehicles and about 0.8kW on the Kia EV9. The framework is interpretable, and it generalizes well to unseen conditions, and sampling rates, making it practical for path optimization, eco-routing, on-board diagnostics, and prognostics health management.

Method: Self-traced Step-wise Preference Optimization (SSPO) - a pluggable RL process supervision framework that leverages step-wise preference signals generated by the model itself for reasoning compression, requiring no auxiliary models or stepwise manual annotations.

Result: Experiments show SSPO generates accurate and succinct reasoning sequences, effectively mitigating overthinking behaviors without compromising model performance across diverse domains and languages.

Conclusion: SSPO provides an efficient framework for fine-grained optimization of reasoning steps using self-generated signals, addressing computational overhead and error accumulation in LLM reasoning processes.

Abstract: Test-time scaling has proven effective in further enhancing the performance of pretrained Large Language Models (LLMs). However, mainstream post-training methods (i.e., reinforcement learning (RL) with chain-of-thought (CoT) reasoning) often incur substantial computational overhead due to auxiliary models and overthinking. In this paper, we empirically reveal that the incorrect answers partially stem from verbose reasoning processes lacking correct self-fix, where errors accumulate across multiple reasoning steps. To this end, we propose Self-traced Step-wise Preference Optimization (SSPO), a pluggable RL process supervision framework that enables fine-grained optimization of each reasoning step. Specifically, SSPO requires neither auxiliary models nor stepwise manual annotations. Instead, it leverages step-wise preference signals generated by the model itself to guide the optimization process for reasoning compression. Experiments demonstrate that the generated reasoning sequences from SSPO are both accurate and succinct, effectively mitigating overthinking behaviors without compromising model performance across diverse domains and languages.

Method: The authors develop and formalize criteria for explanatory robustness (different XAI methods producing same explanations) and explanation method robustness (individual methods being robust themselves), providing a framework for establishing trust.

Result: The paper presents a formal framework with two robustness criteria that must be fulfilled for XAI methods to be trustworthy, addressing current limitations in explainable AI.

Conclusion: Both explanatory robustness and explanation method robustness are necessary for trustworthy XAI, and the proposed framework provides directions for future work in establishing reliable explanations for deep learning systems.

Abstract: Deep learning (DL) algorithms are becoming ubiquitous in everyday life and in scientific research. However, the price we pay for their impressively accurate predictions is significant: their inner workings are notoriously opaque - it is unknown to laypeople and researchers alike what features of the data a DL system focuses on and how it ultimately succeeds in predicting correct outputs. A necessary criterion for trustworthy explanations is that they should reflect the relevant processes the algorithms’ predictions are based on. The field of eXplainable Artificial Intelligence (XAI) presents promising methods to create such explanations. But recent reviews about their performance offer reasons for skepticism. As we will argue, a good criterion for trustworthiness is explanatory robustness: different XAI methods produce the same explanations in comparable contexts. However, in some instances, all methods may give the same, but still wrong, explanation. We therefore argue that in addition to explanatory robustness (ER), a prior requirement of explanation method robustness (EMR) has to be fulfilled by every XAI method. Conversely, the robustness of an individual method is in itself insufficient for trustworthiness. In what follows, we develop and formalize criteria for ER as well as EMR, providing a framework for explaining and establishing trust in DL algorithms. We also highlight interesting application cases and outline directions for future work.

Method: FlowMol3 uses flow matching with three architecture-agnostic techniques: self-conditioning, fake atoms, and train-time geometry distortion. It builds on previous FlowMol versions without changing the graph neural network architecture or flow matching formulation.

Result: Achieves nearly 100% molecular validity for drug-like molecules, accurately reproduces functional group composition and geometry of training data, and uses an order of magnitude fewer parameters than comparable methods.

Conclusion: The three simple techniques mitigate distribution drift during inference, providing transferable strategies for improving stability and quality of diffusion- and flow-based molecular generative models.

Abstract: A generative model capable of sampling realistic molecules with desired properties could accelerate chemical discovery across a wide range of applications. Toward this goal, significant effort has focused on developing models that jointly sample molecular topology and 3D structure. We present FlowMol3, an open-source, multi-modal flow matching model that advances the state of the art for all-atom, small-molecule generation. Its substantial performance gains over previous FlowMol versions are achieved without changes to the graph neural network architecture or the underlying flow matching formulation. Instead, FlowMol3’s improvements arise from three architecture-agnostic techniques that incur negligible computational cost: self-conditioning, fake atoms, and train-time geometry distortion. FlowMol3 achieves nearly 100% molecular validity for drug-like molecules with explicit hydrogens, more accurately reproduces the functional group composition and geometry of its training data, and does so with an order of magnitude fewer learnable parameters than comparable methods. We hypothesize that these techniques mitigate a general pathology affecting transport-based generative models, enabling detection and correction of distribution drift during inference. Our results highlight simple, transferable strategies for improving the stability and quality of diffusion- and flow-based molecular generative models.

Method: Propose Score-informed Neural Operator (SciNO) - a probabilistic generative model in smooth function spaces designed to stably approximate Hessian diagonal and preserve structural information during score modeling.

Result: SciNO reduces order divergence by 42.7% on synthetic graphs and 31.5% on real-world datasets compared to DiffAN, while maintaining memory efficiency and scalability. Enables reliable causal reasoning with LLMs without additional fine-tuning.

Conclusion: SciNO provides a stable and efficient approach for Hessian diagonal approximation in causal discovery, significantly outperforming existing methods and enabling enhanced causal reasoning capabilities in large language models through probabilistic control integration.

Abstract: Ordering-based approaches to causal discovery identify topological orders of causal graphs, providing scalable alternatives to combinatorial search methods. Under the Additive Noise Model (ANM) assumption, recent causal ordering methods based on score matching require an accurate estimation of the Hessian diagonal of the log-densities. However, previous approaches mainly use Stein gradient estimators, which are computationally expensive and memory-intensive. Although DiffAN addresses these limitations by substituting kernel-based estimates with diffusion models, it remains numerically unstable due to the second-order derivatives of score models. To alleviate these problems, we propose Score-informed Neural Operator (SciNO), a probabilistic generative model in smooth function spaces designed to stably approximate the Hessian diagonal and to preserve structural information during the score modeling. Empirical results show that SciNO reduces order divergence by 42.7% on synthetic graphs and by 31.5% on real-world datasets on average compared to DiffAN, while maintaining memory efficiency and scalability. Furthermore, we propose a probabilistic control algorithm for causal reasoning with autoregressive models that integrates SciNO’s probability estimates with autoregressive model priors, enabling reliable data-driven causal ordering informed by semantic information. Consequently, the proposed method enhances causal reasoning abilities of LLMs without additional fine-tuning or prompt engineering.

Method: Proposes a Byzantine-robust FL approach that leverages a trusted server with a side dataset and requires only one honest client. Uses theoretical analysis to bound optimality gaps under attacks.

Result: Significantly outperforms standard FL baselines (Mean, Trimmed Mean, Median, Krum, Multi-Krum) against various attack strategies (label flipping, sign flipping, Gaussian noise) on MNIST, FMNIST, and CIFAR-10 benchmarks using Flower framework.

Conclusion: The method provides effective Byzantine resilience in FL with minimal trust assumptions (only server and one honest client needed) and no prior knowledge of malicious client count, demonstrating practical robustness against diverse attack strategies.

Abstract: Federated Learning (FL) enables collaborative model training across multiple clients without sharing private data. We consider FL scenarios wherein FL clients are subject to adversarial (Byzantine) attacks, while the FL server is trusted (honest) and has a trustworthy side dataset. This may correspond to, e.g., cases where the server possesses trusted data prior to federation, or to the presence of a trusted client that temporarily assumes the server role. Our approach requires only two honest participants, i.e., the server and one client, to function effectively, without prior knowledge of the number of malicious clients. Theoretical analysis demonstrates bounded optimality gaps even under strong Byzantine attacks. Experimental results show that our algorithm significantly outperforms standard and robust FL baselines such as Mean, Trimmed Mean, Median, Krum, and Multi-Krum under various attack strategies including label flipping, sign flipping, and Gaussian noise addition across MNIST, FMNIST, and CIFAR-10 benchmarks using the Flower framework.

Method: Dynamically generates specialized models via hypernetwork conditioned on distribution-aware embeddings, using NoisyEmbed-enhanced extractor with Balancing Penalty to prevent feature collapse.

Result: Extensive experiments show HyperFedZero surpasses competing methods consistently with minimal computational, storage, and communication overhead.

Conclusion: The method effectively addresses data heterogeneity for non-participating clients, with ablation studies confirming the necessity of each component and validating its effectiveness.

Abstract: Federated Learning (FL) has emerged as a promising paradigm for privacy-preserving collaborative learning, yet data heterogeneity remains a critical challenge. While existing methods achieve progress in addressing data heterogeneity for participating clients, they fail to generalize to non-participating clients with in-domain distribution shifts and resource constraints. To mitigate this issue, we present HyperFedZero, a novel method that dynamically generates specialized models via a hypernetwork conditioned on distribution-aware embeddings. Our approach explicitly incorporates distribution-aware inductive biases into the model’s forward pass, extracting robust distribution embeddings using a NoisyEmbed-enhanced extractor with a Balancing Penalty, effectively preventing feature collapse. The hypernetwork then leverages these embeddings to generate specialized models chunk-by-chunk for non-participating clients, ensuring adaptability to their unique data distributions. Extensive experiments on multiple datasets and models demonstrate HyperFedZero’s remarkable performance, surpassing competing methods consistently with minimal computational, storage, and communication overhead. Moreover, ablation studies and visualizations further validate the necessity of each component, confirming meaningful adaptations and validating the effectiveness of HyperFedZero.

Method: BuilDa uses a single-zone Modelica model exported as a Functional Mock-up Unit (FMU) and simulated in Python to generate synthetic thermal building data without requiring deep simulation expertise.

Result: The framework successfully generates adequate quality and quantity of synthetic data that can be used for pretraining and fine-tuning transfer learning models.

Conclusion: BuilDa provides a practical solution to the data scarcity problem in transfer learning research for building thermal dynamics, enabling broader research without requiring extensive simulation expertise.

Abstract: Transfer learning (TL) can improve data-driven modeling of building thermal dynamics. Therefore, many new TL research areas emerge in the field, such as selecting the right source model for TL. However, these research directions require massive amounts of thermal building data which is lacking presently. Neither public datasets nor existing data generators meet the needs of TL research in terms of data quality and quantity. Moreover, existing data generation approaches typically require expert knowledge in building simulation. We present BuilDa, a thermal building data generation framework for producing synthetic data of adequate quality and quantity for TL research. The framework does not require profound building simulation knowledge to generate large volumes of data. BuilDa uses a single-zone Modelica model that is exported as a Functional Mock-up Unit (FMU) and simulated in Python. We demonstrate BuilDa by generating data and utilizing it for pretraining and fine-tuning TL models.

Method: Partitioned traffic sign classes across vehicles for specialized local training using lightweight object detectors, aggregated model parameters via FedProx, FedAdam and FedAVG algorithms in Flower framework simulation.

Result: Increasing server rounds (2 to 20) boosted accuracy from <0.1 to >0.8, moderate local epochs (8-10) provided optimal efficiency (~0.67 accuracy), higher client participation enhanced generalization (up to 0.83), FedProx outperformed other aggregators, non-IID data reduced performance.

Conclusion: Federated learning offers scalable, privacy-preserving solution for vehicular deployments, with potential for future integration of robust aggregation and communication optimizations in intelligent transportation systems.

Abstract: Connected and automated vehicles generate vast amounts of sensor data daily, raising significant privacy and communication challenges for centralized machine learning approaches in perception tasks. This study presents a decentralized, federated learning framework tailored for traffic sign detection in vehicular networks to enable collaborative model training without sharing raw data. The framework partitioned traffic sign classes across vehicles for specialized local training using lightweight object detectors, aggregated model parameters via algorithms like FedProx, FedAdam and FedAVG in a simulated environment with the Flower framework, and evaluated multiple configurations including varying server rounds, local epochs, client participation fractions, and data distributions. Experiments demonstrated that increasing server rounds from 2 to 20 boosted accuracy from below 0.1 to over 0.8, moderate local epochs (8-10) provided optimal efficiency with accuracies around 0.67, higher client participation fractions enhanced generalization up to 0.83, FedProx outperformed other aggregators in handling heterogeneity, non-IID data distributions reduced performance compared to IID, and training duration primarily scaled with the number of rounds rather than aggregation strategy. We conclude that this federated approach may offer a scalable, privacy-preserving solution for real-world vehicular deployments, potentially guiding future integrations of robust aggregation and communication optimizations to advance intelligent transportation systems.

Method: Proposes similarity group pruning (SGP) to remove redundant layers while preserving critical ones, and orchestrated distillation alignment (ODA) to reduce gradient divergence. Uses QLoRA for quantized sub-LLMs and lightweight adapter fine-tuning.

Result: Reduces communication overhead by 70.6%, decreases storage usage by 75.6%, and improves task accuracy by 3.1% across various downstream tasks with three open-source LLMs.

Conclusion: FedSODA is highly suitable for practical federated fine-tuning applications under resource constraints, enabling efficient domain-specific adaptation while preserving data privacy.

Abstract: Federated fine-tuning (FFT) of large language models (LLMs) has recently emerged as a promising solution to enable domain-specific adaptation while preserving data privacy. Despite its benefits, FFT on resource-constrained clients relies on the high computational and memory demands of full-model fine-tuning, which limits the potential advancement. This paper presents FedSODA, a resource-efficient FFT framework that enables clients to adapt LLMs without accessing or storing the full model. Specifically, we first propose a similarity group pruning (SGP) module, which prunes redundant layers from the full LLM while retaining the most critical layers to preserve the model performance. Moreover, we introduce an orchestrated distillation alignment (ODA) module to reduce gradient divergence between the sub-LLM and the full LLM during FFT. Through the use of the QLoRA, clients only need to deploy quantized sub-LLMs and fine-tune lightweight adapters, significantly reducing local resource requirements. We conduct extensive experiments on three open-source LLMs across a variety of downstream tasks. The experimental results demonstrate that FedSODA reduces communication overhead by an average of 70.6%, decreases storage usage by 75.6%, and improves task accuracy by 3.1%, making it highly suitable for practical FFT applications under resource constraints.

Method: Proposes FedUNet framework that attaches a U-Net-inspired additive module to each client’s backbone. Only shares the compact bottleneck of the U-Net for efficient knowledge transfer without requiring structural alignment. Uses encoder-decoder design with skip connections to capture both low-level and high-level features for client-invariant representations.

Result: Achieves 93.11% accuracy with VGG variants and 92.68% accuracy in compact form, with only 0.89 MB communication overhead, demonstrating efficient knowledge transfer with minimal communication cost.

Conclusion: FedUNet successfully enables cooperative learning across heterogeneous model architectures in federated settings while maintaining high accuracy and significantly reducing communication overhead compared to traditional FL approaches.

Abstract: Federated learning (FL) enables decentralized model training without sharing local data. However, most existing methods assume identical model architectures across clients, limiting their applicability in heterogeneous real-world environments. To address this, we propose FedUNet, a lightweight and architecture-agnostic FL framework that attaches a U-Net-inspired additive module to each client’s backbone. By sharing only the compact bottleneck of the U-Net, FedUNet enables efficient knowledge transfer without structural alignment. The encoder-decoder design and skip connections in the U-Net help capture both low-level and high-level features, facilitating the extraction of clientinvariant representations. This enables cooperative learning between the backbone and the additive module with minimal communication cost. Experiment with VGG variants shows that FedUNet achieves 93.11% accuracy and 92.68% in compact form (i.e., a lightweight version of FedUNet) with only 0.89 MB low communication overhead.

Method: Uses VoxLogicA spatial model checker to generate synthetic datasets (maze connectivity problems and spatial distance tasks) across multiple resolutions. Implements automated pipeline with dataset generation, standardized training, cross-validation, inference, and evaluation using Dice coefficient and IoU metrics.

Result: Preliminary results show significant challenges and systematic failures in neural networks’ basic geometric and topological understanding capabilities, highlighting limitations in spatial reasoning.

Conclusion: The framework provides reproducible protocol to identify specific limitations, suggesting hybrid approaches combining neural networks with symbolic reasoning could improve spatial understanding for clinical applications, establishing foundation for future research.

Abstract: This paper presents preliminary results in the definition of a comprehensive benchmark framework designed to systematically evaluate spatial reasoning capabilities in neural networks, with a particular focus on morphological properties such as connectivity and distance relationships. The framework is currently being used to study the capabilities of nnU-Net, exploiting the spatial model checker VoxLogicA to generate two distinct categories of synthetic datasets: maze connectivity problems for topological analysis and spatial distance computation tasks for geometric understanding. Each category is evaluated across multiple resolutions to assess scalability and generalization properties. The automated pipeline encompasses a complete machine learning workflow including: synthetic dataset generation, standardized training with cross-validation, inference execution, and comprehensive evaluation using Dice coefficient and IoU (Intersection over Union) metrics. Preliminary experimental results demonstrate significant challenges in neural network spatial reasoning capabilities, revealing systematic failures in basic geometric and topological understanding tasks. The framework provides a reproducible experimental protocol, enabling researchers to identify specific limitations. Such limitations could be addressed through hybrid approaches combining neural networks with symbolic reasoning methods for improved spatial understanding in clinical applications, establishing a foundation for ongoing research into neural network spatial reasoning limitations and potential solutions.

Method: Uses Lagrangian formulation to derive a closed-form solution that constrains maximum distance between cluster center and farthest point, maintaining interpretability while controlling cluster spread.

Result: CCC achieves more compact clusters by reducing radial spread while preserving angular structure, outperforming standard K-means and GMM on synthetic circular data with radial symmetry and uniform angular distribution using ring-wise, sector-wise, and joint entropy metrics.

Conclusion: The proposed CCC method provides effective spread-controlled clustering suitable for applications requiring structured clustering with interpretable results and controlled cluster compactness.

Abstract: This paper presents Constrained Centroid Clustering (CCC), a method that extends classical centroid-based clustering by enforcing a constraint on the maximum distance between the cluster center and the farthest point in the cluster. Using a Lagrangian formulation, we derive a closed-form solution that maintains interpretability while controlling cluster spread. To evaluate CCC, we conduct experiments on synthetic circular data with radial symmetry and uniform angular distribution. Using ring-wise, sector-wise, and joint entropy as evaluation metrics, we show that CCC achieves more compact clusters by reducing radial spread while preserving angular structure, outperforming standard methods such as K-means and GMM. The proposed approach is suitable for applications requiring structured clustering with spread control, including sensor networks, collaborative robotics, and interpretable pattern analysis.

Method: Extreme Learning Machine (ELM) with Multi-Input Multi-Output (MIMO) architecture using sliding window techniques and cyclic time encoding to address non-stationarity and seasonal variability. Trained on 6 years of hourly data from multiple energy sources.

Result: Significantly outperforms persistence forecasting, particularly for solar (17.9% nRMSE) and thermal energy (5.1% nRMSE) with R²>0.98 for 1-hour horizon. Maintains high accuracy up to 5 hours ahead. MIMO provides marginal gains over SISO with lower computational demands than deep learning methods.

Conclusion: The ELM-based MIMO approach provides an accurate, computationally efficient solution for short-term energy forecasting that is adaptable to various contexts and suitable for real-time applications including online learning.

Abstract: A novel methodology for short-term energy forecasting using an Extreme Learning Machine ($\mathtt{ELM}$) is proposed. Using six years of hourly data collected in Corsica (France) from multiple energy sources (solar, wind, hydro, thermal, bioenergy, and imported electricity), our approach predicts both individual energy outputs and total production (\cyr{including imports, which closely follow energy demand, modulo losses)} through a Multi-Input Multi-Output ($\mathtt{MIMO}$) architecture. To address non-stationarity and seasonal variability, sliding window techniques and cyclic time encoding are incorporated, enabling dynamic adaptation to fluctuations. The $\mathtt{ELM}$ model significantly outperforms persistence-based forecasting, particularly for solar and thermal energy, achieving an $\mathtt{nRMSE}$ of $17.9%$ and $5.1%$, respectively, with $\mathtt{R^2} > 0.98$ (1-hour horizon). The model maintains high accuracy up to five hours ahead, beyond which renewable energy sources become increasingly volatile. While $\mathtt{MIMO}$ provides marginal gains over Single-Input Single-Output ($\mathtt{SISO}$) architectures and offers key advantages over deep learning methods such as $\mathtt{LSTM}$, it provides a closed-form solution with lower computational demands, making it well-suited for real-time applications, including online learning. Beyond predictive accuracy, the proposed methodology is adaptable to various contexts and datasets, as it can be tuned to local constraints such as resource availability, grid characteristics, and market structures.

Method: Proposed E3Former, a novel online ensemble model that synergizes multiple subnetworks to overcome single-model limitations, ensuring superior accuracy and robustness with minimal computational overhead increase.

Result: In online forecasting tasks, reduces forecast error by average of 10%; deployed in ByteDance’s IHPA platform supporting 30+ applications; handles predictive auto-scaling for over 600,000 CPU cores; reduces resource utilization by over 40% while maintaining service quality.

Conclusion: E3Former effectively addresses the challenges of online workload forecasting for serverless auto-scaling, demonstrating practical deployment success at large scale with significant resource efficiency improvements.

Abstract: In the swiftly evolving domain of cloud computing, the advent of serverless systems underscores the crucial need for predictive auto-scaling systems. This necessity arises to ensure optimal resource allocation and maintain operational efficiency in inherently volatile environments. At the core of a predictive auto-scaling system is the workload forecasting model. Existing forecasting models struggle to quickly adapt to the dynamics in online workload streams and have difficulty capturing the complex periodicity brought by fine-grained, high-frequency forecasting tasks. Addressing this, we propose a novel online ensemble model, E3Former, for online workload forecasting in large-scale predictive auto-scaling. Our model synergizes the predictive capabilities of multiple subnetworks to surmount the limitations of single-model approaches, thus ensuring superior accuracy and robustness. Remarkably, it accomplishes this with a minimal increase in computational overhead, adhering to the lean operational ethos of serverless systems. Through extensive experimentation on real-world workload datasets, we establish the efficacy of our ensemble model. In online forecasting tasks, the proposed method reduces forecast error by an average of 10%, and its effectiveness is further demonstrated through a predictive auto-scaling test in the real-life online system. Currently, our method has been deployed within ByteDance’s Intelligent Horizontal Pod Auto-scaling (IHPA) platform, which supports the stable operation of over 30 applications, such as Douyin E-Comerce, TouTiao, and Volcano Engine. The predictive auto-scaling capacity reaching over 600,000 CPU cores. On the basis of essentially ensuring service quality, the predictive auto-scaling system can reduce resource utilization by over 40%.

Method: Utilizes Randomized Principal Component Analysis (RPCA) Forest for outlier detection, building on the success of RPCA in approximate nearest neighbor search.

Result: Experimental results show superiority over classical and state-of-the-art methods on several datasets, with competitive performance on others. The method demonstrates high generalization power and computational efficiency.

Conclusion: The proposed RPCA Forest-based approach is an effective and efficient choice for unsupervised outlier detection tasks, offering strong performance across diverse datasets.

Abstract: We propose a novel unsupervised outlier detection method based on Randomized Principal Component Analysis (PCA). Inspired by the performance of Randomized PCA (RPCA) Forest in approximate K-Nearest Neighbor (KNN) search, we develop a novel unsupervised outlier detection method that utilizes RPCA Forest for outlier detection. Experimental results showcase the superiority of the proposed approach compared to the classical and state-of-the-art methods in performing the outlier detection task on several datasets while performing competitively on the rest. The extensive analysis of the proposed method reflects it high generalization power and its computational efficiency, highlighting it as a good choice for unsupervised outlier detection.

Method: Establishes equivalence between attention layers and graph diffusion, then proposes Wavy Transformer with second-order wavy dynamics, modified feed-forward network, and normalization to preserve physical state-velocity relationships.

Result: Consistently improves performance on various NLP and CV tasks with minimal additional parameters and no extra hyperparameter tuning.

Conclusion: The physical interpretation of transformer dynamics enables effective solutions to over-smoothing through second-order wavy attention mechanisms.

Abstract: Transformers have achieved remarkable success across natural language processing (NLP) and computer vision (CV). However, deep transformer models often suffer from an over-smoothing issue, in which token representations converge to similar values as they pass through successive transformer blocks. In this paper, we establish an equivalence between the hidden-state dynamics induced by stacked attention layers and graph neural diffusion on a complete graph. From this perspective, over-smoothing can be interpreted as a consequence of the dissipative nature of the underlying diffusion dynamics. Motivated by this physical interpretation, we propose Wavy Transformer, which consists of a novel attention layer based on second-order wavy dynamics. We also introduce a feed-forward network and a normalization layer designed to preserve the physical state-velocity relationship under the chain rule, thereby extending the transformer architecture. We further validate our proposed techniques on various transformer models for NLP and CV tasks. The results consistently demonstrate that Wavy Transformer improves performance with minimal additional parameters and no extra hyperparameter tuning.

Method: Bridge posits latent human preference scores and models LLM deviations as linear transformations of covariates that capture sources of discrepancies, with an efficient fitting algorithm for statistical inference.

Result: Using six LLM judges and two benchmarks (BigGen Bench and Chatbot Arena), Bridge achieves higher agreement with human ratings in accuracy, calibration, and KL divergence, while exposing systematic human-LLM gaps.

Conclusion: Bridge provides a simple and principled statistical framework for refining LLM ratings and characterizing systematic discrepancies between human and LLM evaluations, improving the reliability of LLM-as-a-judge systems.

Abstract: Large language models are increasingly used as judges (LLM-as-a-judge) to evaluate model outputs at scale, but their assessments often diverge systematically from human judgments. We present Bridge, a unified statistical framework that explicitly bridges human and LLM evaluations under both absolute scoring and pairwise comparison paradigms. Bridge posits a latent human preference score for each prompt-response pair and models LLM deviations as linear transformations of covariates that capture sources of discrepancies. This offers a simple and principled framework for refining LLM ratings and characterizing systematic discrepancies between humans and LLMs. We provide an efficient fitting algorithm with asymptotic guarantees for statistical inference. Using six LLM judges and two benchmarks (BigGen Bench and Chatbot Arena), Bridge achieves higher agreement with human ratings (accuracy, calibration, and KL divergence) and exposes systematic human-LLM gaps.

Method: Proposes Maximum Score Routing (MaxScore) that models routing as a minimum-cost maximum-flow problem and integrates a SoftTopk operator to optimize token allocation without capacity constraints.

Result: Achieves lower training losses and higher evaluation scores at equivalent FLOPs compared to both constrained and unconstrained baselines, resolving fundamental limitations of existing routing methods.

Conclusion: MaxScore provides an effective solution to the trade-offs in MoE routing, enabling better performance without the drawbacks of traditional capacity-constrained approaches.

Abstract: Routing networks in sparsely activated mixture-of-experts (MoE) dynamically allocate input tokens to top-k experts through differentiable sparse transformations, enabling scalable model capacity while preserving computational efficiency. Traditional MoE networks impose an expert capacity constraint to ensure GPU-friendly computation. However, this leads to token dropping when capacity is saturated and results in low hardware efficiency due to padding in underutilized experts. Removing the capacity constraint, in turn, compromises load balancing and computational efficiency. To address these issues, we propose Maximum Score Routing ($\mathbf{MaxScore}$), a novel MoE routing paradigm that models routing as a minimum-cost maximum-flow problem and integrates a SoftTopk operator. MaxScore resolves the fundamental limitations of iterative rerouting and optimal transport formulations, achieving lower training losses and higher evaluation scores at equivalent FLOPs compared to both constrained and unconstrained baselines. Implementation details and experimental configurations can be obtained from $\href{https://github.com/dongbw18/MaxScore.git}{MaxScore}$.

Method: The authors revisit and analyze claims from both causality and domain generalization literature, providing theoretical reconciliation and an interactive demo for practical exploration.

Result: The paper presents a more nuanced theory of how causality contributes to generalization, addressing the contradictions found in previous research.

Conclusion: Causal modeling still plays an important role in AI generalization, but requires a more sophisticated understanding than previously assumed, with practical tools provided for further exploration.

Abstract: The promise that causal modelling can lead to robust AI generalization has been challenged in recent work on domain generalization (DG) benchmarks. We revisit the claims of the causality and DG literature, reconciling apparent contradictions and advocating for a more nuanced theory of the role of causality in generalization. We also provide an interactive demo at https://chai-uk.github.io/ukairs25-causal-predictors/.

Method: Uses contrastive input-specific prompting to compute fine-grained linear shifts, then trains a small auxiliary module to predict these input-specific steering vectors at test time.

Result: L2S demonstrates reduced hallucinations and improved safety enforcement in multimodal LLMs, outperforming static steering baselines.

Conclusion: Input-specific steering through learned auxiliary modules provides more effective and context-aware guidance for multimodal LLMs compared to static steering approaches.

Abstract: Steering has emerged as a practical approach to enable post-hoc guidance of LLMs towards enforcing a specific behavior. However, it remains largely underexplored for multimodal LLMs (MLLMs); furthermore, existing steering techniques, such as mean steering, rely on a single steering vector, applied independently of the input query. This paradigm faces limitations when the desired behavior is dependent on the example at hand. For example, a safe answer may consist in abstaining from answering when asked for an illegal activity, or may point to external resources or consultation with an expert when asked about medical advice. In this paper, we investigate a fine-grained steering that uses an input-specific linear shift. This shift is computed using contrastive input-specific prompting. However, the input-specific prompts required for this approach are not known at test time. Therefore, we propose to train a small auxiliary module to predict the input-specific steering vector. Our approach, dubbed as L2S (Learn-to-Steer), demonstrates that it reduces hallucinations and enforces safety in MLLMs, outperforming other static baselines.

Method: Conducted empirical study analyzing trade-offs between data quantity and quality through compression, comparing uniform data dropping vs. adaptive compression strategies.

Result: Found that data samples have varying sensitivities to compression, demonstrating feasibility of sample-wise adaptive compression that outperforms uniform approaches.

Conclusion: Provides foundation for new storage-aware learning systems through systematic characterization of compression trade-offs, advancing understanding of optimal storage management for on-device ML.

Abstract: On-device machine learning is often constrained by limited storage, particularly in continuous data collection scenarios. This paper presents an empirical study on storage-aware learning, focusing on the trade-off between data quantity and quality via compression. We demonstrate that naive strategies, such as uniform data dropping or one-size-fits-all compression, are suboptimal. Our findings further reveal that data samples exhibit varying sensitivities to compression, supporting the feasibility of a sample-wise adaptive compression strategy. These insights provide a foundation for developing a new class of storage-aware learning systems. The primary contribution of this work is the systematic characterization of this under-explored challenge, offering valuable insights that advance the understanding of storage-aware learning.

Method: Uses short-horizon temporal consistency captured via lightweight signals on posteriors and features, converts them into calibrated risk scores using an O(W) ring buffer and O(1) per-step updates, and employs a streaming conformal layer for budgeted accept/abstain decisions.

Result: Reduces footprint by 50-60% and latency by 30-45% compared to early exit and deep ensembles, achieves up to 0.86 AUPRC for accuracy drop detection and 0.92 AUROC for failure detection under corrupted data streams.

Conclusion: Temporal consistency combined with streaming conformal calibration provides a practical and resource-efficient foundation for on-device uncertainty monitoring in TinyML applications.

Abstract: We introduce TCUQ, a single pass, label free uncertainty monitor for streaming TinyML that converts short horizon temporal consistency captured via lightweight signals on posteriors and features into a calibrated risk score with an O(W ) ring buffer and O(1) per step updates. A streaming conformal layer turns this score into a budgeted accept/abstain rule, yielding calibrated behavior without online labels or extra forward passes. On microcontrollers, TCUQ fits comfortably on kilobyte scale devices and reduces footprint and latency versus early exit and deep ensembles (typically about 50 to 60% smaller and about 30 to 45% faster), while methods of similar accuracy often run out of memory. Under corrupted in distribution streams, TCUQ improves accuracy drop detection by 3 to 7 AUPRC points and reaches up to 0.86 AUPRC at high severities; for failure detection it attains up to 0.92 AUROC. These results show that temporal consistency, coupled with streaming conformal calibration, provides a practical and resource efficient foundation for on device monitoring in TinyML.

Method: Analyze learning of in-context n-gram language models under cross-entropy loss, establish sufficient conditions for stationary points, construct parameter configurations for simplified transformers representing k-gram estimators, and analyze gradient behavior in infinite sequence length limit.

Result: Sub-n-grams are near-stationary points of the population cross-entropy loss, with gradient vanishing in the limit of infinite sequence length and parameter norm, explaining stage-wise learning dynamics.

Conclusion: Theoretical analysis reveals key properties of transformer loss landscapes that explain widely observed phenomena like stage-wise learning and emergent phase transitions, supported by numerical experiments showing discrete transitions between near-stationary solutions.

Abstract: Motivated by empirical observations of prolonged plateaus and stage-wise progression during training, we investigate the loss landscape of transformer models trained on in-context next-token prediction tasks. In particular, we focus on learning in-context $n$-gram language models under cross-entropy loss, and establish a sufficient condition for parameter configurations to be stationary points. We then construct a set of parameter configurations for a simplified transformer model that represent $k$-gram estimators (for $k \leq n$), and show that the gradient of the population loss at these solutions vanishes in the limit of infinite sequence length and parameter norm. This reveals a key property of the loss landscape: {sub-$n$-grams are near-stationary points of the population cross-entropy loss}, offering theoretical insight into widely observed phenomena such as stage-wise learning dynamics and emergent phase transitions. These insights are further supported by numerical experiments that illustrate the learning dynamics of $n$-grams, characterized by discrete transitions between near-stationary solutions.

Method: Uses tiny int8 heads to forecast statistics of the next layer from compressed previous layer views, with a lightweight monotone mapper converting surprisal into actionable uncertainty scores.

Result: Reduces flash and latency by 40-60% and 25-35% respectively compared to early-exit and deep ensembles, improves accuracy-drop detection by several AUPRC points, and maintains strong failure detection (AUROC ≈0.9) in single pass.

Conclusion: Grounding uncertainty in layer-to-layer dynamics provides a practical, resource-efficient basis for on-device monitoring in TinyML applications.

Abstract: We introduce \textbf{SNAP-UQ}, a single-pass, label-free uncertainty method for TinyML that estimates risk from \emph{depth-wise next-activation prediction}: tiny int8 heads forecast the statistics of the next layer from a compressed view of the previous one, and a lightweight monotone mapper turns the resulting surprisal into an actionable score. The design requires no temporal buffers, auxiliary exits, or repeated forward passes, and adds only a few tens of kilobytes to MCU deployments. Across vision and audio backbones, SNAP-UQ consistently reduces flash and latency relative to early-exit and deep ensembles (typically $\sim$40–60% smaller and $\sim$25–35% faster), with competing methods of similar accuracy often exceeding memory limits. In corrupted streams it improves accuracy-drop detection by several AUPRC points and maintains strong failure detection (AUROC $\approx$0.9) in a single pass. Grounding uncertainty in layer-to-layer dynamics yields a practical, resource-efficient basis for on-device monitoring in TinyML.

Method: Proposed HRS - a hybrid representation framework that integrates numerical and image-based representations to capture extreme load dynamics, along with a Scheduling-Aware Loss (SAL) function that accounts for asymmetric impact of prediction errors to better support scheduling decisions.

Result: Extensive experiments on four real-world datasets show HRS consistently outperforms ten baselines, achieving state-of-the-art performance with 63.1% reduction in SLA violation rates and 32.3% reduction in total profit loss.

Conclusion: The HRS framework effectively addresses the forecasting dilemma in cloud-edge platforms by combining hybrid representations and scheduling-aware optimization, significantly improving QoS while maintaining profitability.

Abstract: With the rapid proliferation of streaming services, network load exhibits highly time-varying and bursty behavior, posing serious challenges for maintaining Quality of Service (QoS) in Crowdsourced Cloud-Edge Platforms (CCPs). While CCPs leverage Predict-then-Schedule architecture to improve QoS and profitability, accurate load forecasting remains challenging under traffic surges. Existing methods either minimize mean absolute error, resulting in underprovisioning and potential Service Level Agreement (SLA) violations during peak periods, or adopt conservative overprovisioning strategies, which mitigate SLA risks at the expense of increased resource expenditure. To address this dilemma, we propose HRS, a hybrid representation framework with scheduling awareness that integrates numerical and image-based representations to better capture extreme load dynamics. We further introduce a Scheduling-Aware Loss (SAL) that captures the asymmetric impact of prediction errors, guiding predictions that better support scheduling decisions. Extensive experiments on four real-world datasets demonstrate that HRS consistently outperforms ten baselines and achieves state-of-the-art performance, reducing SLA violation rates by 63.1% and total profit loss by 32.3%.

Method: Proposes TGN-SVDD method that builds upon modern dynamic graph modelling and deep anomaly detection techniques.

Result: Demonstrates superiority over several baselines for realistic intrusion detection data and suggests a more challenging variant of the evaluation data.

Conclusion: TGN-SVDD represents an effective approach for intrusion detection that addresses key challenges in modern network security through dynamic graph modeling and deep anomaly detection.

Abstract: With the growing digitalization all over the globe, the relevance of network security becomes increasingly important. Machine learning-based intrusion detection constitutes a promising approach for improving security, but it bears several challenges. These include the requirement to detect novel and unseen network events, as well as specific data properties, such as events over time together with the inherent graph structure of network communication. In this work, we propose a novel intrusion detection method, TGN-SVDD, which builds upon modern dynamic graph modelling and deep anomaly detection. We demonstrate its superiority over several baselines for realistic intrusion detection data and suggest a more challenging variant of the latter.

Method: Proposed SparseMap framework using enhanced genetic encoding and evolutionary operators to efficiently explore the vast combinatorial design space (up to O(10^41)) that integrates both mapping (tiling communication/computation) and sparse strategies (bypassing zero elements).

Result: SparseMap consistently finds superior solutions compared to prior works and classical optimization methods like particle swarm optimization, reinforcement learning, and Monte Carlo tree search.

Conclusion: The unified framework SparseMap successfully addresses the combinatorial explosion challenge in sparse tensor accelerator design by jointly optimizing both mapping and sparse strategies through enhanced evolutionary algorithms.

Abstract: The growing demand for sparse tensor algebra (SpTA) in machine learning and big data has driven the development of various sparse tensor accelerators. However, most existing manually designed accelerators are limited to specific scenarios, and it’s time-consuming and challenging to adjust a large number of design factors when scenarios change. Therefore, automating the design of SpTA accelerators is crucial. Nevertheless, previous works focus solely on either mapping (i.e., tiling communication and computation in space and time) or sparse strategy (i.e., bypassing zero elements for efficiency), leading to suboptimal designs due to the lack of comprehensive consideration of both. A unified framework that jointly optimizes both is urgently needed. However, integrating mapping and sparse strategies leads to a combinatorial explosion in the design space(e.g., as large as $O(10^{41})$ for the workload $P_{32 \times 64} \times Q_{64 \times 48} = Z_{32 \times 48}$). This vast search space renders most conventional optimization methods (e.g., particle swarm optimization, reinforcement learning and Monte Carlo tree search) inefficient. To address this challenge, we propose an evolution strategy-based sparse tensor accelerator optimization framework, called SparseMap. SparseMap constructing a more comprehensive design space with the consideration of both mapping and sparse strategy. We introduce a series of enhancements to genetic encoding and evolutionary operators, enabling SparseMap to efficiently explore the vast and diverse design space. We quantitatively compare SparseMap with prior works and classical optimization methods, demonstrating that SparseMap consistently finds superior solutions.

Method: Proposes RobAJoL framework combining Johnson-Lindenstrauss transform for compression with robust averaging for aggregation, enabling compression of DP-protected updates while maintaining robustness.

Result: Theoretical proof shows compatibility of JL transform with robust averaging. Experiments on CIFAR-10 and Fashion MNIST demonstrate superior robustness and utility under various Byzantine attacks compared to existing methods.

Conclusion: Fed-DPRoC successfully integrates differential privacy, Byzantine robustness, and communication efficiency through robust-compatible compression, providing a comprehensive solution for secure and efficient federated learning.

Abstract: We propose Fed-DPRoC, a novel federated learning framework that simultaneously ensures differential privacy (DP), Byzantine robustness, and communication efficiency. We introduce the concept of robust-compatible compression, which enables users to compress DP-protected updates while maintaining the robustness of the aggregation rule. We instantiate our framework as RobAJoL, combining the Johnson-Lindenstrauss (JL) transform for compression with robust averaging for robust aggregation. We theoretically prove the compatibility of JL transform with robust averaging and show that RobAJoL preserves robustness guarantees, ensures DP, and reduces communication cost. Experiments on CIFAR-10 and Fashion MNIST validate our theoretical claims and demonstrate that RobAJoL outperforms existing methods in terms of robustness and utility under different Byzantine attacks.

Method: Proposes SL-ACC with two components: 1) Adaptive Channel Importance Identification (ACII) using Shannon entropy to identify channel contributions, and 2) Channel Grouping Compression (CGC) that groups channels by entropy and performs group-wise adaptive compression to reduce transmission volume.

Result: Extensive experiments across various datasets show SL-ACC takes considerably less time to achieve target accuracy compared to state-of-the-art benchmarks.

Conclusion: The proposed framework effectively addresses the communication bottleneck in split learning while maintaining training accuracy, making distributed ML more feasible for resource-constrained environments.

Abstract: The increasing complexity of neural networks poses a significant barrier to the deployment of distributed machine learning (ML) on resource-constrained devices, such as federated learning (FL). Split learning (SL) offers a promising solution by offloading the primary computing load from edge devices to a server via model partitioning. However, as the number of participating devices increases, the transmission of excessive smashed data (i.e., activations and gradients) becomes a major bottleneck for SL, slowing down the model training. To tackle this challenge, we propose a communication-efficient SL framework, named SL-ACC, which comprises two key components: adaptive channel importance identification (ACII) and channel grouping compression (CGC). ACII first identifies the contribution of each channel in the smashed data to model training using Shannon entropy. Following this, CGC groups the channels based on their entropy and performs group-wise adaptive compression to shrink the transmission volume without compromising training accuracy. Extensive experiments across various datasets validate that our proposed SL-ACC framework takes considerably less time to achieve a target accuracy than state-of-the-art benchmarks.

Method: Theoretical analysis and empirical experiments on synthetic and real graph data (Cora, CiteSeer, Polblogs) using multiple aggregation methods for Fiedler values across connected components.

Result: Fiedler value is a reliable predictor of GCN performance - graphs with similar algebraic connectivity exhibit analogous structural properties and respond similarly to GCN filters and hyperparameters.

Conclusion: Algebraic connectivity serves as an effective metric for predicting GCN performance and facilitating transfer learning between graphs with similar structural properties.

Abstract: A common observation in the Graph Convolutional Network (GCN) literature is that stacking GCN layers may or may not result in better performance on tasks like node classification and edge prediction. We have found empirically that a graph’s algebraic connectivity, which is known as the Fiedler value, is a good predictor of GCN performance. Intuitively, graphs with similar Fiedler values have analogous structural properties, suggesting that the same filters and hyperparameters may yield similar results when used with GCNs, and that transfer learning may be more effective between graphs with similar algebraic connectivity. We explore this theoretically and empirically with experiments on synthetic and real graph data, including the Cora, CiteSeer and Polblogs datasets. We explore multiple ways of aggregating the Fiedler value for connected components in the graphs to arrive at a value for the entire graph, and show that it can be used to predict GCN performance. We also present theoretical arguments as to why the Fiedler value is a good predictor.

Method: Replaces fixed beta2 with layer-wise dynamic values driven by a ‘sunspike’ ratio (current pooled gradient norm divided by EMA of past norms, scaled to [0,1)). Spikes lower beta2 toward beta2_min; calm phases keep it near beta2_max. Includes optional features like leaky-AMSGrad, trust-region clipping, and bias-correction modes.

Result: Improved stability and final loss vs fixed-beta2 Adam across four test settings: Transformer PDE surrogate (Heat2D), 3D PINN for heat conduction, synthetic MLX task, and character-level Transformer on enwik8. On small-enwik8, reduced bits-per-character by ~38% vs Adam-0.95 and ~58% vs Adam-0.999 with smaller variance.

Conclusion: Kourkoutas-Beta provides drop-in replacement for Adam with minimal runtime overhead, preserves Adam-style convergence guarantees, and significantly improves robustness under spiky gradient conditions common in physics-based neural networks.

Abstract: Transformer neural networks are increasingly used for physics-based problems. In data-driven PDE surrogates, training samples from varying boundary and initial conditions can cause erratic losses and spiky gradients; in physics-informed neural networks (PINNs), stiff composite losses amplify this effect. We introduce Kourkoutas-Beta, an Adam-style optimizer where the fixed second-moment discount beta2 is replaced by a layer-wise dynamic value driven by a bounded sunspike'' ratio: the current pooled gradient norm divided by an exponential moving average (EMA) of past norms, squashed to the interval [0,1). Spikes lower beta2 toward beta2_min; calm phases keep it near beta2_max. Options include leaky-AMSGrad (decay), trust-region clipping (max_ratio), adaptive tiny terms, and several bias-correction modes none’’, beta2max'', exact’). With all features off and bias_correction=``none’’, the method is exactly Adam. We test on four settings: (i) a Transformer PDE surrogate (Heat2D), (ii) a 3D PINN for heat conduction (Heat3D), (iii) a lightweight MLX synthetic task with jitter and rare-trigger bursts, and (iv) a character-level Transformer on 30 MB of enwik8 (small-enwik8). Kourkoutas-Beta improves stability and final loss versus fixed-beta2 Adam. On small-enwik8 it lowers bits-per-character by about 38% vs Adam-0.95 and about 58% vs Adam-0.999 over 10 seeds, with smaller variance. The method remains drop-in, with runtime overhead comparable to Adam in testbeds A-C and within single-digit percent in testbed D. It preserves Adam-style convergence guarantees while improving robustness under spiky gradients.

Method: Proposes Fairness-Aware Multi-view Evidential Learning (FAML) with: 1) adaptive prior based on training trajectory for regularization, 2) fairness constraint based on class-wise evidence variance, and 3) opinion alignment mechanism for multi-view fusion.

Result: Extensive experiments on five real-world multi-view datasets show FAML achieves more balanced evidence allocation and improves both prediction performance and uncertainty estimation reliability compared to state-of-the-art methods.

Conclusion: FAML effectively addresses the Biased Evidential Multi-view Learning problem by calibrating biased evidence learning and promoting balanced evidence allocation through novel regularization and fairness mechanisms.

Abstract: Multi-view evidential learning aims to integrate information from multiple views to improve prediction performance and provide trustworthy uncertainty esitimation. Most previous methods assume that view-specific evidence learning is naturally reliable. However, in practice, the evidence learning process tends to be biased. Through empirical analysis on real-world data, we reveal that samples tend to be assigned more evidence to support data-rich classes, thereby leading to unreliable uncertainty estimation in predictions. This motivates us to delve into a new Biased Evidential Multi-view Learning (BEML) problem. To this end, we propose Fairness-Aware Multi-view Evidential Learning (FAML). FAML first introduces an adaptive prior based on training trajectory, which acts as a regularization strategy to flexibly calibrate the biased evidence learning process. Furthermore, we explicitly incorporate a fairness constraint based on class-wise evidence variance to promote balanced evidence allocation. In the multi-view fusion stage, we propose an opinion alignment mechanism to mitigate view-specific bias across views, thereby encouraging the integration of consistent and mutually supportive evidence. Extensive experiments on five real-world multi-view datasets demonstrate that FAML achieves more balanced evidence allocation and improves both prediction performance and the reliability of uncertainty estimation compared to state-of-the-art methods.

Method: Uses Monte Carlo sampling to approximate model prediction distributions, leverages three continuous distributions with moment-based methods to capture statistical characteristics, and employs both Wasserstein and KL distances for regularization.

Result: Evaluated on MNIST and CIFAR datasets, MCFRCL demonstrates effectiveness in both prediction accuracy and training efficiency compared to multiple benchmark methods.

Conclusion: The proposed MCFRCL framework provides an effective solution for continual learning with improved computational efficiency and reduced approximation errors.

Abstract: Continual learning (CL) is crucial for the adaptation of neural network models to new environments. Although outperforming weight-space regularisation approaches, the functional regularisation-based CL methods suffer from high computational costs and large linear approximation errors. In this work, we present a new functional regularisation CL framework, called MCFRCL, which approximates model prediction distributions by Monte Carlo (MC) sampling. Moreover, three continuous distributions are leveraged to capture the statistical characteristics of the MC samples via moment-based methods. Additionally, both the Wasserstein distance and the Kullback-Leibler (KL) distance are employed to construct the regularisation function. The proposed MCFRCL is evaluated against multiple benchmark methods on the MNIST and CIFAR datasets, with simulation results highlighting its effectiveness in both prediction accuracy and training efficiency.

Method: Developed filtered-x hyperbolic tangent exponential generalized Kernel M-estimate function (FXHEKM) robust adaptive algorithm with statistical analysis and computational cost study.

Result: Numerical results demonstrate the algorithm’s efficiency in canceling additive spurious signals, particularly alpha-stable noises, outperforming competing algorithms in MSE and ANR metrics.

Conclusion: The FXHEKM algorithm provides an effective solution for robust adaptive filtering in impulsive noise environments, offering superior noise cancellation performance for active noise control applications.

Abstract: In this work, we propose a robust adaptive filtering approach for active noise control applications in the presence of impulsive noise. In particular, we develop the filtered-x hyperbolic tangent exponential generalized Kernel M-estimate function (FXHEKM) robust adaptive algorithm. A statistical analysis of the proposed FXHEKM algorithm is carried out along with a study of its computational cost. {In order to evaluate the proposed FXHEKM algorithm, the mean-square error (MSE) and the average noise reduction (ANR) performance metrics have been adopted.} Numerical results show the efficiency of the proposed FXHEKM algorithm to cancel the presence of the additive spurious signals, such as \textbf{$\alpha$}-stable noises against competing algorithms.

Method: Used Natural Language Processing (NLP) and deep learning with BERT combined with Hierarchical Attention Networks (HANs) for multi-label classification of cyberattack consequences into five categories: Availability, Access Control, Confidentiality, Integrity, and Other.

Result: BERT achieved overall accuracy of 0.972, significantly higher than conventional CNN and LSTM models. HAN outperformed baseline CNN/LSTM on specific cybersecurity labels, but BERT consistently showed better precision and recall.

Conclusion: BERT is more suitable than traditional deep learning models for predicting cyberattack consequences, making it effective for automated threat modeling and cybersecurity risk assessment.

Abstract: Cyberattacks are increasing, and securing against such threats is costing industries billions of dollars annually. Threat Modeling, that is, comprehending the consequences of these attacks, can provide critical support to cybersecurity professionals, enabling them to take timely action and allocate resources that could be used elsewhere. Cybersecurity is heavily dependent on threat modeling, as it assists security experts in assessing and mitigating risks related to identifying vulnerabilities and threats. Recently, there has been a pressing need for automated methods to assess attack descriptions and forecast the future consequences of the increasing complexity of cyberattacks. This study examines how Natural Language Processing (NLP) and deep learning can be applied to analyze the potential impact of cyberattacks by leveraging textual descriptions from the MITRE Common Weakness Enumeration (CWE) database. We emphasize classifying attack consequences into five principal categories: Availability, Access Control, Confidentiality, Integrity, and Other. This paper investigates the use of Bidirectional Encoder Representations from Transformers (BERT) in combination with Hierarchical Attention Networks (HANs) for Multi-label classification, evaluating their performance in comparison with conventional CNN and LSTM-based models. Experimental findings show that BERT achieves an overall accuracy of $0.972$, far higher than conventional deep learning models in multi-label classification. HAN outperforms baseline forms of CNN and LSTM-based models on specific cybersecurity labels. However, BERT consistently achieves better precision and recall, making it more suitable for predicting the consequences of a cyberattack.

Method: Utilizes available separate datasets (internal with predictive attributes, external with protected attributes) to estimate feasible joint distributions and compute plausible fairness metrics through bounds estimation.

Result: Demonstrates through simulations and real experiments that meaningful bounds on fairness metrics can be derived, providing reliable estimates of true fairness metrics.

Conclusion: The approach serves as a practical and effective solution for fairness testing in real-world settings where access to complete data is restricted, enabling compliance with emerging fairness regulations.

Abstract: Ensuring fairness in AI systems is critical, especially in high-stakes domains such as lending, hiring, and healthcare. This urgency is reflected in emerging global regulations that mandate fairness assessments and independent bias audits. However, procuring the necessary complete data for fairness testing remains a significant challenge. In industry settings, legal and privacy concerns restrict the collection of demographic data required to assess group disparities, and auditors face practical and cultural challenges in gaining access to data. In practice, data relevant for fairness testing is often split across separate sources: internal datasets held by institutions with predictive attributes, and external public datasets such as census data containing protected attributes, each providing only partial, marginal information. Our work seeks to leverage such available separate data to estimate model fairness when complete data is inaccessible. We propose utilising the available separate data to estimate a set of feasible joint distributions and then compute the set plausible fairness metrics. Through simulation and real experiments, we demonstrate that we can derive meaningful bounds on fairness metrics and obtain reliable estimates of the true metric. Our results demonstrate that this approach can serve as a practical and effective solution for fairness testing in real-world settings where access to complete data is restricted.

Method: Experiments comparing FMAE (focused on minimizing absolute errors) and HEF (weighting global metrics and penalizing large deviations) under different data splits (91:9, 80:20, 70:30) using three optimizers (Grid Search, PSO, Optuna).

Result: HEF consistently outperforms FMAE in global metrics (R2, Relative Accuracy, RMSE, RMSSE) and enhances model robustness, while FMAE offers advantages in local metrics (MAE, MASE) and execution time.

Conclusion: Methodological trade-off exists: HEF is ideal for strategic planning, FMAE for operational efficiency; a replicable framework is proposed for optimizing predictive models in dynamic environments.

Abstract: Demand forecasting is essential for strategic planning in competitive environments, enabling resource optimization and improved responsiveness to market dynamics. However, multivariate time series modeling faces challenges due to data complexity, uncertainty, and frequent regime shifts. Traditional evaluation metrics can introduce biases and limit generalization. This work compares two custom evaluation functions: FMAE (Focused Mean Absolute Error), focused on minimizing absolute errors, and HEF (Hierarchical Evaluation Function), designed to weight global metrics and penalize large deviations. Experiments were conducted under different data splits (91:9, 80:20, 70:30) using three optimizers (Grid Search, PSO, Optuna), assessing fit, relative accuracy, robustness, and computational efficiency. Results show that HEF consistently outperforms FMAE in global metrics (R2, Relative Accuracy, RMSE, RMSSE), enhancing model robustness and explanatory power. These findings were confirmed via visualizations and statistical tests. Conversely, FMAE offers advantages in local metrics (MAE, MASE) and execution time, making it suitable for short-term scenarios. The study highlights a methodological trade-off: HEF is ideal for strategic planning, while FMAE is better suited for operational efficiency. A replicable framework is proposed for optimizing predictive models in dynamic environments.

Method: Models error via density estimation that reports high density only for parameters close to training data in both input and output spaces. Combines this density estimate (prior belief) with a likelihood on features to efficiently sample plausible parameter configurations that generate target output features.

Result: Developed a visualization interface that demonstrates usability through feature-driven parameter analysis across three simulation datasets, enabling interactive exploration of parameter distributions.

Conclusion: The approach successfully addresses both surrogate model approximation error and interactive parameter distribution formation, providing a comprehensive solution for understanding the range of plausible parameters that produce specific simulation outputs.

Abstract: Recently, neural surrogate models have emerged as a compelling alternative to traditional simulation workflows. This is accomplished by modeling the underlying function of scientific simulations, removing the need to run expensive simulations. Beyond just mapping from input parameter to output, surrogates have also been shown useful for inverse problems: output to input parameters. Inverse problems can be understood as search, where we aim to find parameters whose surrogate outputs contain a specified feature. Yet finding these parameters can be costly, especially for high-dimensional parameter spaces. Thus, existing surrogate-based solutions primarily focus on finding a small set of matching parameters, in the process overlooking the broader picture of plausible parameters. Our work aims to model and visualize the distribution of possible input parameters that produce a given output feature. To achieve this goal, we aim to address two challenges: (1) the approximation error inherent in the surrogate model and (2) forming the parameter distribution in an interactive manner. We model error via density estimation, reporting high density only if a given parameter configuration is close to training parameters, measured both over the input and output space. Our density estimate is used to form a prior belief on parameters, and when combined with a likelihood on features, gives us an efficient way to sample plausible parameter configurations that generate a target output feature. We demonstrate the usability of our solution through a visualization interface by performing feature-driven parameter analysis over the input parameter space of three simulation datasets. Source code is available at https://github.com/matthewberger/seeing-the-many

Method: Model target arrivals as mixture of normal and outlier processes using log Gaussian Cox processes. Propose second-order probability approximation incorporating mean and variance of normal intensity function. Integrate real-time near-optimal sensor placement strategy that dynamically adjusts sensor locations.

Result: Analytically shown to yield tighter bound to true probability using Jensen’s inequality. Validated with real ship traffic data near Norfolk, Virginia, demonstrating improved classification performance and outlier detection through optimized sensor deployment.

Conclusion: The proposed framework effectively enhances spatial outlier detection in maritime environments through improved probability estimation and dynamic sensor placement optimization.

Abstract: This paper presents a framework for classifying and detecting spatial commission outliers in maritime environments using seabed acoustic sensor networks and log Gaussian Cox processes (LGCPs). By modeling target arrivals as a mixture of normal and outlier processes, we estimate the probability that a newly observed event is an outlier. We propose a second-order approximation of this probability that incorporates both the mean and variance of the normal intensity function, providing improved classification accuracy compared to mean-only approaches. We analytically show that our method yields a tighter bound to the true probability using Jensen’s inequality. To enhance detection, we integrate a real-time, near-optimal sensor placement strategy that dynamically adjusts sensor locations based on the evolving outlier intensity. The proposed framework is validated using real ship traffic data near Norfolk, Virginia, where numerical results demonstrate the effectiveness of our approach in improving both classification performance and outlier detection through sensor deployment.

Method: The authors design ATB as a perfectly truthful calibration measure using a general construction recipe. ATB is based on averaging two-bin calibration errors and is shown to be sound, complete, continuous, and quadratically related to existing measures like smooth calibration error.

Result: ATB is proven to be perfectly truthful, efficient to compute, and enables faster estimation algorithms with simpler implementations compared to existing calibration measures like smCal and distCal.

Conclusion: The paper successfully addresses the fundamental problem of truthful calibration measurement by introducing ATB and providing a general framework for constructing truthful calibration measures, enabling reliable probabilistic interpretation without incentivizing deceptive behavior.

Abstract: Calibration requires that predictions are conditionally unbiased and, therefore, reliably interpretable as probabilities. Calibration measures quantify how far a predictor is from perfect calibration. As introduced by Haghtalab et al. (2024), a calibration measure is truthful if it is minimized in expectation when a predictor outputs the ground-truth probabilities. Although predicting the true probabilities guarantees perfect calibration, in reality, when calibration is evaluated on a finite sample, predicting the truth is not guaranteed to minimize any known calibration measure. All known calibration measures incentivize predictors to lie in order to appear more calibrated on a finite sample. Such lack of truthfulness motivated Haghtalab et al. (2024) and Qiao and Zhao (2025) to construct approximately truthful calibration measures in the sequential prediction setting, but no perfectly truthful calibration measure was known to exist even in the more basic batch setting. We design a perfectly truthful calibration measure in the batch setting: averaged two-bin calibration error (ATB). In addition to being truthful, ATB is sound, complete, continuous, and quadratically related to two existing calibration measures: the smooth calibration error (smCal) and the (lower) distance to calibration (distCal). The simplicity in our definition of ATB makes it efficient and straightforward to compute. ATB allows faster estimation algorithms with significantly easier implementations than smCal and distCal, achieving improved running time and simplicity for the calibration testing problem studied by Hu et al. (2024). We also introduce a general recipe for constructing truthful measures, which proves the truthfulness of ATB as a special case and allows us to construct other truthful calibration measures such as quantile-binned l_2-ECE.

Method: Proposes Causally-Guided Pairwise Transformer (CGPT) that integrates known causal graphs, decomposes multidimensional data into pairs, and uses channel-agnostic learnable layers with CD information flow at pair-level and CI-like generalization across pairs.

Result: CGPT significantly outperforms both CI and CD baselines in predictive accuracy on synthetic and real-world industrial datasets, showing competitive performance with end-to-end trained CD models while remaining dimension-agnostic.

Conclusion: The pairwise modeling approach with causal guidance effectively resolves the CD/CI conflict, creating a flexible architecture that ensures scalability and any-variate adaptability for industrial time-series forecasting.

Abstract: Foundational modelling of multi-dimensional time-series data in industrial systems presents a central trade-off: channel-dependent (CD) models capture specific cross-variable dynamics but lack robustness and adaptability as model layers are commonly bound to the data dimensionality of the tackled use-case, while channel-independent (CI) models offer generality at the cost of modelling the explicit interactions crucial for system-level predictive regression tasks. To resolve this, we propose the Causally-Guided Pairwise Transformer (CGPT), a novel architecture that integrates a known causal graph as an inductive bias. The core of CGPT is built around a pairwise modeling paradigm, tackling the CD/CI conflict by decomposing the multidimensional data into pairs. The model uses channel-agnostic learnable layers where all parameter dimensions are independent of the number of variables. CGPT enforces a CD information flow at the pair-level and CI-like generalization across pairs. This approach disentangles complex system dynamics and results in a highly flexible architecture that ensures scalability and any-variate adaptability. We validate CGPT on a suite of synthetic and real-world industrial datasets on long-term and one-step forecasting tasks designed to simulate common industrial complexities. Results demonstrate that CGPT significantly outperforms both CI and CD baselines in predictive accuracy and shows competitive performance with end-to-end trained CD models while remaining agnostic to the problem dimensionality.

Method: Combinatorial Representations for Temporal Reasoning (CRTR) - a method that uses a specialized negative sampling scheme to provably remove spurious features from temporal contrastive learning, enabling better capture of temporal structure.

Result: CRTR achieves strong performance on complex temporal domains like Sokoban and Rubik’s Cube. For Rubik’s Cube, it learns representations that generalize across all initial states and solves puzzles using fewer search steps than BestFS (though with longer solutions).

Conclusion: CRTR is the first method that efficiently solves arbitrary Rubik’s Cube states using only learned representations without external search algorithms, demonstrating that temporal reasoning can emerge from properly structured representations.

Abstract: In classical AI, perception relies on learning state-based representations, while planning, which can be thought of as temporal reasoning over action sequences, is typically achieved through search. We study whether such reasoning can instead emerge from representations that capture both perceptual and temporal structure. We show that standard temporal contrastive learning, despite its popularity, often fails to capture temporal structure due to its reliance on spurious features. To address this, we introduce Combinatorial Representations for Temporal Reasoning (CRTR), a method that uses a negative sampling scheme to provably remove these spurious features and facilitate temporal reasoning. CRTR achieves strong results on domains with complex temporal structure, such as Sokoban and Rubik’s Cube. In particular, for the Rubik’s Cube, CRTR learns representations that generalize across all initial states and allow it to solve the puzzle using fewer search steps than BestFS, though with longer solutions. To our knowledge, this is the first method that efficiently solves arbitrary Cube states using only learned representations, without relying on an external search algorithm.

Method: Comprehensive experimental analysis of diverse models (LSTM and Transformer architectures) with various parameter configurations and training strategies. Incorporates semantic information like day-of-week and user-specific historical data. Uses user semantic clustering with stratified sampling to address data imbalance, and explores small-batch stochastic gradient optimization.

Result: Explicit inclusion of semantic information (day-of-week, user-specific data) improves model understanding of individual life patterns and prediction accuracy. User sampling without proper stratification exacerbates data skewness and reduces predictive performance. Small-batch stochastic gradient optimization enhances model performance, especially with limited training data.

Conclusion: Effective human mobility prediction requires incorporating semantic context and addressing data imbalance through techniques like stratified sampling. Small-batch optimization is beneficial for limited data scenarios, and preserving user diversity is crucial for maintaining predictive accuracy when explicit user information is unavailable due to privacy concerns.

Abstract: Individual-level human mobility prediction has emerged as a significant topic of research with applications in infectious disease monitoring, child, and elderly care. Existing studies predominantly focus on the microscopic aspects of human trajectories: such as predicting short-term trajectories or the next location visited, while offering limited attention to macro-level mobility patterns and the corresponding life routines. In this paper, we focus on an underexplored problem in human mobility prediction: determining the best practices to train a machine learning model using historical data to forecast an individuals complete trajectory over the next days and weeks. In this experiment paper, we undertake a comprehensive experimental analysis of diverse models, parameter configurations, and training strategies, accompanied by an in-depth examination of the statistical distribution inherent in human mobility patterns. Our empirical evaluations encompass both Long Short-Term Memory and Transformer-based architectures, and further investigate how incorporating individual life patterns can enhance the effectiveness of the prediction. We show that explicitly including semantic information such as day-of-the-week and user-specific historical information can help the model better understand individual patterns of life and improve predictions. Moreover, since the absence of explicit user information is often missing due to user privacy, we show that the sampling of users may exacerbate data skewness and result in a substantial loss in predictive accuracy. To mitigate data imbalance and preserve diversity, we apply user semantic clustering with stratified sampling to ensure that the sampled dataset remains representative. Our results further show that small-batch stochastic gradient optimization improves model performance, especially when human mobility training data is limited.

Method: Frames denoising trajectory learning as a sequential decision-making problem, proposes Masked Diffusion Policy Optimization (MDPO) using reinforcement learning to train under inference schedule, and develops RCR remasking strategy for flexible token refinement.

Result: MDPO matches SOTA performance with 60x fewer gradient updates, achieves 9.6% improvement on MATH500 and 54.2% on Countdown over SOTA with same update budget, and RCR provides consistent performance gains as a training-free plug-in.

Conclusion: The approach successfully addresses the training-inference gap in MDLMs, establishes great potential for investigating this discrepancy, and provides both training (MDPO) and inference (RCR) improvements.

Abstract: Diffusion language models, as a promising alternative to traditional autoregressive (AR) models, enable faster generation and richer conditioning on bidirectional context. However, they suffer from a key discrepancy between training and inference: during inference, MDLMs progressively reveal the structure of the generated sequence by producing fewer and fewer masked tokens, whereas this structure is ignored in training as tokens are masked at random. Although this discrepancy between training and inference can lead to suboptimal performance, it has been largely overlooked by previous works, leaving closing this gap between the two stages an open problem. To address this, we frame the problem of learning effective denoising trajectories as a sequential decision-making problem and use the resulting framework to apply reinforcement learning. We propose a novel Masked Diffusion Policy Optimization (MDPO) to exploit the Markov property diffusion possesses and explicitly train the model under the same progressive refining schedule used at inference. MDPO matches the performance of the previous state-of-the-art (SOTA) method with 60x fewer gradient updates, while achieving average improvements of 9.6% on MATH500 and 54.2% on Countdown over SOTA when trained within the same number of weight updates. Additionally, we improve the remasking strategy of MDLMs as a plug-in inference replacement to overcome the limitation that the model cannot refine tokens flexibly. This simple yet effective training-free strategy, what we refer to as RCR, consistently improves performance and yields additional gains when combined with MDPO. Our findings establish great potential for investigating the discrepancy between pre-training and inference of MDLMs. Code: https://github.com/autonomousvision/mdpo. Project Page: https://cli212.github.io/MDPO/.

Method: Leverages LLMs to generate language-guided reward functions for higher-level policy, decoupling reward generation from low-level policy changes, and integrates goal-conditioned hindsight experience relabeling for sample efficiency.

Result: Outperforms hierarchical and non-hierarchical baselines with over 55% success rates on challenging tasks, demonstrates robust transfer to real robots without additional fine-tuning.

Conclusion: LGR2 effectively mitigates non-stationarity in off-policy HRL, enabling stable and efficient learning for complex robotic tasks through language-guided reward generation.

Abstract: Large language models (LLMs) have shown remarkable abilities in logical reasoning, in-context learning, and code generation. However, translating natural language instructions into effective robotic control policies remains a significant challenge, especially for tasks requiring long-horizon planning and operating under sparse reward conditions. Hierarchical Reinforcement Learning (HRL) provides a natural framework to address this challenge in robotics; however, it typically suffers from non-stationarity caused by the changing behavior of the lower-level policy during training, destabilizing higher-level policy learning. We introduce LGR2, a novel HRL framework that leverages LLMs to generate language-guided reward functions for the higher-level policy. By decoupling high-level reward generation from low-level policy changes, LGR2 fundamentally mitigates the non-stationarity problem in off-policy HRL, enabling stable and efficient learning. To further enhance sample efficiency in sparse environments, we integrate goal-conditioned hindsight experience relabeling. Extensive experiments across simulated and real-world robotic navigation and manipulation tasks demonstrate LGR2 outperforms both hierarchical and non-hierarchical baselines, achieving over 55% success rates on challenging tasks and robust transfer to real robots, without additional fine-tuning.

Method: Fine-tune LLMs on 1,000 graded examples (correct/incorrect answers) using LoRA for efficient training, creating uncertainty estimators that work across different models.

Result: The approach outperforms baseline methods, shows good generalization, and enables models to estimate uncertainty for other models. User study confirms uncertainty estimates improve human-AI collaboration.

Conclusion: Fine-tuning on small datasets is sufficient for effective LLM uncertainty estimation, making it computationally tractable and applicable across models while enhancing human-AI interaction.

Abstract: When using large language models (LLMs) in high-stakes applications, we need to know when we can trust their predictions. Some works argue that prompting high-performance LLMs is sufficient to produce calibrated uncertainties, while others introduce sampling methods that can be prohibitively expensive. In this work, we first argue that prompting on its own is insufficient to achieve good calibration and then show that fine-tuning on a small dataset of correct and incorrect answers can create an uncertainty estimate with good generalization and small computational overhead. We show that a thousand graded examples are sufficient to outperform baseline methods and that training through the features of a model is necessary for good performance and tractable for large open-source models when using LoRA. We also investigate the mechanisms that enable reliable LLM uncertainty estimation, finding that many models can be used as general-purpose uncertainty estimators, applicable not just to their own uncertainties but also the uncertainty of other models. Lastly, we show that uncertainty estimates inform human use of LLMs in human-AI collaborative settings through a user study.

Method: Introducing a precise quantitative law that governs contextualized token embedding learning through intermediate layers in pre-trained LLMs for next-token prediction.

Result: Each layer contributes equally to enhancing prediction accuracy from lowest to highest layer - a universal phenomenon observed across diverse open-source LLMs regardless of architecture or pre-training data.

Conclusion: This law provides new perspectives and actionable insights to guide LLM development and applications, including model scaling, pre-training tasks, and interpretation practices.

Abstract: Large language models (LLMs) have been widely employed across various application domains, yet their black-box nature poses significant challenges to understanding how these models process input data internally to make predictions. In this paper, we introduce a precise and quantitative law that governs the learning of contextualized token embeddings through intermediate layers in pre-trained LLMs for next-token prediction. Our findings reveal that each layer contributes equally to enhancing prediction accuracy, from the lowest to the highest layer – a universal phenomenon observed across a diverse array of open-source LLMs, irrespective of their architectures or pre-training data. We demonstrate that this law offers new perspectives and actionable insights to inform and guide practices in LLM development and applications, including model scaling, pre-training tasks, and interpretation.

Method: Proposed post-training alignment (using GRPO and DPO algorithms) with solution-based and process-based rewards on synthetic graph problems, applied to both off-the-shelf and fine-tuned LLMs.

Result: 12.9% average improvement over baselines on 5 datasets; process-based rewards outperform solution-based on synthetic data but not real-world tasks; compositionality and explainable intermediate steps remain challenging.

Conclusion: Post-training alignment is more effective than direct fine-tuning for generalizable graph reasoning, though challenges in compositionality and intermediate reasoning steps persist even after alignment.

Abstract: Previous research has sought to enhance the graph reasoning capabilities of LLMs by supervised fine-tuning on synthetic graph data. While these led to specialized LLMs better at solving graph algorithm problems, we don’t need LLMs for shortest path: we need generalization from synthetic graph data to real-world tasks with implicit graph structures. In this work, we propose to unlock generalizable learning of graph with post-training alignment with synthetic data. We first design solution-based and process-based rewards for synthetic graph problems: instead of rigid memorizing response patterns in direct fine-tuning, we posit that post-training alignment would help LLMs grasp the essentials underlying graph reasoning and alleviate overfitting on synthetic data. We employ post-training alignment algorithms such as GRPO and DPO, aligning both off-the-shelf LLMs and LLMs fine-tuned on synthetic graph data. We then compare them against existing settings on both in-domain synthetic tasks and out-of-domain real-world tasks with implicit graph structures such as multi-hop QA, structured planning, and more. Extensive experiments demonstrate that our post-training alignment recipe leads to statistically significant improvement on 5 datasets, with an average gain of 12.9% over baseline settings. Further analysis reveals that process-based rewards consistently outperform solution-based rewards on synthetic data but not on real-world tasks, and compositionality and explainable intermediate steps remains a critical challenge even after post-training alignment.

Method: Comprehensive literature review of explainable VAD methods categorized by underlying techniques (attention-based, generative model-based, reasoning-based, foundation model-based), analyzing modality-specific challenges and summarizing datasets and evaluation metrics.

Result: Provides a systematic categorization of X-VAD methods, identifies commonalities and differences across image and video modalities, and summarizes relevant resources including datasets and emerging evaluation approaches for explanation quality.

Conclusion: The survey establishes foundations for X-VAD research and highlights promising future directions including quantifying explanation quality, explaining diverse AD paradigms, enhancing context-awareness, and addressing real-world constraints like efficiency and robustness.

Abstract: Anomaly detection and localization in visual data, including images and videos, are crucial in machine learning and real-world applications. Despite rapid advancements in visual anomaly detection (VAD), interpreting these often black-box models and explaining why specific instances are flagged as anomalous remains challenging. This paper provides the first comprehensive survey focused specifically on explainable 2D visual anomaly detection (X-VAD), covering methods for both images (IAD) and videos (VAD). We first introduce the background of IAD and VAD. Then, as the core contribution, we present a thorough literature review of explainable methods, categorized by their underlying techniques (e.g., attention-based, generative model-based, reasoning-based, foundation model-based). We analyze the commonalities and differences in applying these methods across image and video modalities, highlighting modality-specific challenges and opportunities for explainability. Additionally, we summarize relevant datasets and evaluation metrics, discussing both standard performance metrics and emerging approaches for assessing explanation quality (e.g., faithfulness, stability). Finally, we discuss promising future directions and open problems, including quantifying explanation quality, explaining diverse AD paradigms (SSL, zero-shot), enhancing context-awareness, leveraging foundation models responsibly, and addressing real-world constraints like efficiency and robustness. A curated collection of related resources is available at https://github.com/wyzjack/Awesome-XAD.

Method: Proposed HAX (locality-sensitive Hashing Attention with sparse Key Selection) that integrates SSMs with Context-Dependent Sparse Attention to solve the multi-query joint recall problem efficiently.

Result: HAX consistently outperforms SSM baselines and SSMs with context-independent sparse attention on both synthetic and real-world long-context benchmarks.

Conclusion: The proposed HAX framework successfully bridges the gap between theoretical analysis and practical applications, providing an effective solution for efficient long-context modeling in natural language processing.

Abstract: Efficient long-context modeling remains a critical challenge for natural language processing (NLP), as the time complexity of the predominant Transformer architecture scales quadratically with the sequence length. While state-space models (SSMs) offer alternative sub-quadratic solutions, they struggle to capture long-range dependencies effectively. In this work, we focus on analyzing and improving the long-context modeling capabilities of SSMs. We show that the widely used synthetic task, associative recall, which requires a model to recall a value associated with a single key without context, insufficiently represents the complexities of real-world long-context modeling. To address this limitation, we extend the associative recall to a novel synthetic task, \emph{joint recall}, which requires a model to recall the value associated with a key given in a specified context. Theoretically, we prove that SSMs do not have the expressiveness to solve multi-query joint recall in sub-quadratic time complexity. To resolve this issue, we propose a solution based on integrating SSMs with Context-Dependent Sparse Attention (CDSA), which has the expressiveness to solve multi-query joint recall with sub-quadratic computation. To bridge the gap between theoretical analysis and real-world applications, we propose locality-sensitive Hashing Attention with sparse Key Selection (HAX), which instantiates the theoretical solution and is further tailored to natural language domains. Extensive experiments on both synthetic and real-world long-context benchmarks show that HAX consistently outperforms SSM baselines and SSMs integrated with context-independent sparse attention (CISA).

Method: Uses orthogonal projection in embedding space to make class-conditional feature distributions of the target concept indistinguishable after projection. Controls information removal extent through projector rank adjustment while preserving local embedding structure through orthogonality.

Result: Achieves state-of-the-art performance in nonlinear erasure of discrete attributes on classic NLP benchmarks. Effectively mitigates bias in deep nonlinear classifiers, promoting fairness in model outputs.

Conclusion: LEOPARD provides an effective approach for concept erasure that balances information removal with semantic preservation, offering a practical solution for fairness enhancement in neural models through controlled orthogonal projections.

Abstract: Ensuring that neural models used in real-world applications cannot infer sensitive information, such as demographic attributes like gender or race, from text representations is a critical challenge when fairness is a concern. We address this issue through concept erasure, a process that removes information related to a specific concept from distributed representations while preserving as much of the remaining semantic information as possible. Our approach involves learning an orthogonal projection in the embedding space, designed to make the class-conditional feature distributions of the discrete concept to erase indistinguishable after projection. By adjusting the rank of the projector, we control the extent of information removal, while its orthogonality ensures strict preservation of the local structure of the embeddings. Our method, termed $\overline{\mathrm{L}}$EOPARD, achieves state-of-the-art performance in nonlinear erasure of a discrete attribute on classic natural language processing benchmarks. Furthermore, we demonstrate that $\overline{\mathrm{L}}$EOPARD effectively mitigates bias in deep nonlinear classifiers, thereby promoting fairness.

Method: Proposed Negative Feedback Training (NeFT) concept inspired by control theory, with two specific implementations: oriented variational forward (OVF) and intermediate representation snapshot (IRS) to capture multi-scale noisy information throughout the network.

Result: Extensive experiments show NeFT outperforms state-of-the-art methods with up to 45.08% improvement in inference accuracy, reduces epistemic uncertainty, boosts output confidence, and improves convergence probability.

Conclusion: NeFT framework provides a general and practical solution for enhancing DNN robustness against device variations in compute-in-memory accelerators.

Abstract: Compute-in-memory accelerators built upon non-volatile memory devices excel in energy efficiency and latency when performing deep neural network (DNN) inference, thanks to their in-situ data processing capability. However, the stochastic nature and intrinsic variations of non-volatile memory devices often result in performance degradation during DNN inference. Introducing these non-ideal device behaviors in DNN training enhances robustness, but drawbacks include limited accuracy improvement, reduced prediction confidence, and convergence issues. This arises from a mismatch between the deterministic training and non-deterministic device variations, as such training, though considering variations, relies solely on the model’s final output. In this work, inspired by control theory, we propose Negative Feedback Training (NeFT), a novel concept supported by theoretical analysis, to more effectively capture the multi-scale noisy information throughout the network. We instantiate this concept with two specific instances, oriented variational forward (OVF) and intermediate representation snapshot (IRS). Based on device variation models extracted from measured data, extensive experiments show that our NeFT outperforms existing state-of-the-art methods with up to a 45.08% improvement in inference accuracy while reducing epistemic uncertainty, boosting output confidence, and improving convergence probability. These results underline the generality and practicality of our NeFT framework for increasing the robustness of DNNs against device variations. The source code for these two instances is available at https://github.com/YifanQin-ND/NeFT_CIM

Method: Uses delta-homology analogy to identify memory traces with nontrivial homology generators on latent manifolds. Constructs spatiotemporal complexes from polychronous neural groups and abstracts activation loops into cell posets for compact representation.

Result: Developed a framework where memory is encoded as sharply localized topological cycles that are only activated when inference trajectories complete full cycles, representing minimal path-dependent memory units.

Conclusion: The topological approach provides a principled way to understand memory encoding in neural systems through persistent homology and spike-timing dynamics, enabling representation of overlapping and compositional memory traces.

Abstract: We introduce the \emph{delta-homology analogy}, which formalizes memory as a set of sparse, topologically irreducible attractors. A \emph{Dirac delta-like memory trace} ( \delta_\gamma ) is identified with a nontrivial homology generator ( [\gamma] \in H_1(\mathcal{Z}) ) on a latent manifold of cognitive states. Such traces are sharply localized along reproducible topological cycles and are only activated when inference trajectories complete a full cycle. They encode minimal, path-dependent memory units that cannot be synthesized from local features alone. Based on the analogy, we propose a topological framework for memory and inference grounded in the structure of spike-timing dynamics and persistent homology. Starting from the observation that polychronous neural groups (PNGs) encode reproducible, time-locked spike sequences shaped by axonal delays and synaptic plasticity, we construct \emph{spatiotemporal complexes} whose temporally consistent transitions define chain complexes over which robust activation cycles emerge. These activation loops are abstracted into \emph{cell posets}, enabling a compact and causally ordered representation of neural activity with overlapping and compositional memory traces.

Method: Proposed STRIDE approach uses attention mechanisms to identify important intermediate teacher-student layer pairs and focuses on aligning both graph structure and node embeddings from these intermediate layers rather than just final outputs.

Result: Achieves 2.13% accuracy increase with 32.3X compression on OGBN-Mag, and up to 141X compression on smaller datasets like Pubmed while maintaining same accuracy as state-of-the-art approaches.

Conclusion: Focusing on intermediate-layer knowledge through attention-based layer pairing enables creation of compact, accurate, and practical GNN models with superior compression performance.

Abstract: Recent advancements in Graph Neural Networks (GNNs) have led to increased model sizes to enhance their capacity and accuracy. Such large models incur high memory usage, latency, and computational costs, thereby restricting their inference deployment. GNN compression techniques compress large GNNs into smaller ones with negligible accuracy loss. One of the most promising compression techniques is knowledge distillation (KD). However, most KD approaches for GNNs only consider the outputs of the last layers and do not consider the outputs of the intermediate layers of the GNNs. The intermediate layers may contain important inductive biases indicated by the graph structure and embeddings. Ignoring these layers may lead to a high accuracy drop, especially when the compression ratio is high. To address these shortcomings, we propose a novel KD approach for GNN compression that we call Structure and Embedding Distillation with Attention (STRIDE). STRIDE utilizes attention to identify important intermediate teacher-student layer pairs and focuses on using those pairs to align graph structure and node embeddings. We evaluate STRIDE on several datasets, such as OGBN-Mag and OGBN-Arxiv, using different model architectures, including GCNIIs, RGCNs, and GraphSAGE. On average, STRIDE achieves a 2.13% increase in accuracy with a 32.3X compression ratio on OGBN-Mag, a large graph dataset, compared to state-of-the-art approaches. On smaller datasets (e.g., Pubmed), STRIDE achieves up to a 141X compression ratio with the same accuracy as state-of-the-art approaches. These results highlight the effectiveness of focusing on intermediate-layer knowledge to obtain compact, accurate, and practical GNN models.

Method: Leverages complex domain representational advantages, maps weights to fourth roots of unity {±1, ±i}, creates symmetric 2-bit representation where each quantized weight has zero real or imaginary part enabling addition-only inference.

Result: Outperforms existing 2-bit quantization approaches in both PPL and downstream tasks while maintaining strict storage and compute efficiency with multiplication-free inference.

Conclusion: This work opens a new direction for building highly accurate and practical LLMs under extremely low-bit constraints by breaking the traditional accuracy ceiling limitation.

Abstract: Quantization-Aware Training (QAT) integrates quantization into the training loop, enabling LLMs to learn robust low-bit representations, and is widely recognized as one of the most promising research directions. All current QAT research focuses on minimizing quantization error on full-precision models, where the full-precision accuracy acts as an upper bound (accuracy ceiling). No existing method has even attempted to surpass this ceiling. To break this ceiling, we propose a new paradigm: raising the ceiling (full-precision model), and then still quantizing it efficiently into 2 bits. We propose Fairy$\pm i$, the first 2-bit quantization framework for complex-valued LLMs. Specifically, our method leverages the representational advantages of the complex domain to boost full-precision accuracy. We map weights to the fourth roots of unity ${\pm1, \pm i}$, forming a perfectly symmetric and information-theoretically optimal 2-bit representation. Importantly, each quantized weight has either a zero real or imaginary part, enabling multiplication-free inference using only additions and element swaps. Experimental results show that Fairy$\pm i$ outperforms the ceiling of existing 2-bit quantization approaches in terms of both PPL and downstream tasks, while maintaining strict storage and compute efficiency. This work opens a new direction for building highly accurate and practical LLMs under extremely low-bit constraints.

Method: Leverages biomedical language models to structure clinical text at scale, employs advanced probabilistic modeling for denoising and imputation, and incorporates state-of-the-art causal inference techniques to address confounders in treatment effect estimation.

Result: Successfully curated high-quality structured patient data from over 1 million cancer patients, reduced confounding in treatment effect estimation, generated results comparable to randomized controlled lung cancer trials, and simulated unconducted clinical trials including pancreatic cancer trials.

Conclusion: TRIALSCOPE effectively extracts cancer treatment data from EMRs, overcomes limitations of manual curation, reproduces results of actual clinical trials, and establishes best practices for generating real-world evidence from electronic medical records.

Abstract: The rapid digitization of real-world data presents an unprecedented opportunity to optimize healthcare delivery and accelerate biomedical discovery. However, these data are often found in unstructured forms such as clinical notes in electronic medical records (EMRs), and is typically plagued by confounders, making it challenging to generate robust real-world evidence (RWE). Therefore, we present TRIALSCOPE, a framework designed to distil RWE from population level observational data at scale. TRIALSCOPE leverages biomedical language models to structure clinical text at scale, employs advanced probabilistic modeling for denoising and imputation, and incorporates state-of-the-art causal inference techniques to address common confounders in treatment effect estimation. Extensive experiments were conducted on a large-scale dataset of over one million cancer patients from a single large healthcare network in the United States. TRIALSCOPE was shown to automatically curate high-quality structured patient data, expanding the dataset and incorporating key patient attributes only available in unstructured form. The framework reduces confounding in treatment effect estimation, generating comparable results to randomized controlled lung cancer trials. Additionally, we demonstrate simulations of unconducted clinical trials - including a pancreatic cancer trial with varying eligibility criteria - using a suite of validation tests to ensure robustness. Thorough ablation studies were conducted to better understand key components of TRIALSCOPE and establish best practices for RWE generation from EMRs. TRIALSCOPE was able to extract data cancer treatment data from EMRs, overcoming limitations of manual curation. We were also able to show that TRIALSCOPE could reproduce results of lung and pancreatic cancer clinical trials from the extracted real world data.

Method: Latent Plan Transformer (LPT) uses a latent variable to bridge Transformer-based trajectory generation and final returns, learned via maximum likelihood estimation on trajectory-return pairs with posterior sampling.

Result: LPT achieves competitive performance across multiple benchmarks (Gym-Mujoco, Franka Kitchen, Maze2D, Connect Four), demonstrating improved decision-making from sub-optimal trajectories with capabilities in credit assignment and trajectory stitching.

Conclusion: Latent variable inference serves as a strong alternative to step-wise reward prompting for planning tasks, enabling effective abstraction and adaptation to environmental contingencies.

Abstract: In tasks aiming for long-term returns, planning becomes essential. We study generative modeling for planning with datasets repurposed from offline reinforcement learning. Specifically, we identify temporal consistency in the absence of step-wise rewards as one key technical challenge. We introduce the Latent Plan Transformer (LPT), a novel model that leverages a latent variable to connect a Transformer-based trajectory generator and the final return. LPT can be learned with maximum likelihood estimation on trajectory-return pairs. In learning, posterior sampling of the latent variable naturally integrates sub-trajectories to form a consistent abstraction despite the finite context. At test time, the latent variable is inferred from an expected return before policy execution, realizing the idea of planning as inference. Our experiments demonstrate that LPT can discover improved decisions from sub-optimal trajectories, achieving competitive performance across several benchmarks, including Gym-Mujoco, Franka Kitchen, Maze2D, and Connect Four. It exhibits capabilities in nuanced credit assignments, trajectory stitching, and adaptation to environmental contingencies. These results validate that latent variable inference can be a strong alternative to step-wise reward prompting.

Method: Developed TFB benchmark with: 1) 10 domain datasets with time series characterization, 2) diverse method inclusion (statistical, ML, deep learning), 3) flexible scalable evaluation pipeline with multiple strategies and metrics.

Result: Evaluated 21 univariate methods on 8,068 time series and 14 multivariate methods on 25 datasets, providing comprehensive performance comparisons across diverse domains and method types.

Conclusion: TFB provides a standardized, comprehensive benchmark for fair comparison of time series forecasting methods, addressing previous limitations and enabling more reliable progress in the field through open-source code and online leaderboard.

Abstract: Time series are generated in diverse domains such as economic, traffic, health, and energy, where forecasting of future values has numerous important applications. Not surprisingly, many forecasting methods are being proposed. To ensure progress, it is essential to be able to study and compare such methods empirically in a comprehensive and reliable manner. To achieve this, we propose TFB, an automated benchmark for Time Series Forecasting (TSF) methods. TFB advances the state-of-the-art by addressing shortcomings related to datasets, comparison methods, and evaluation pipelines: 1) insufficient coverage of data domains, 2) stereotype bias against traditional methods, and 3) inconsistent and inflexible pipelines. To achieve better domain coverage, we include datasets from 10 different domains: traffic, electricity, energy, the environment, nature, economic, stock markets, banking, health, and the web. We also provide a time series characterization to ensure that the selected datasets are comprehensive. To remove biases against some methods, we include a diverse range of methods, including statistical learning, machine learning, and deep learning methods, and we also support a variety of evaluation strategies and metrics to ensure a more comprehensive evaluations of different methods. To support the integration of different methods into the benchmark and enable fair comparisons, TFB features a flexible and scalable pipeline that eliminates biases. Next, we employ TFB to perform a thorough evaluation of 21 Univariate Time Series Forecasting (UTSF) methods on 8,068 univariate time series and 14 Multivariate Time Series Forecasting (MTSF) methods on 25 datasets. The benchmark code and data are available at https://github.com/decisionintelligence/TFB. We have also launched an online time series leaderboard: https://decisionintelligence.github.io/OpenTS/OpenTS-Bench/.

Method: The review organizes progress through three taxonomic dimensions: functional objectives (context extension, reasoning, knowledge integration, adaptation), memory representations (parameter-encoded, state-based, explicit, hybrid), and integration mechanisms (attention fusion, gated control, associative retrieval).

Result: Analysis reveals a shift from static caches toward adaptive, test-time learning systems. Identifies persistent challenges in scalability and interference, alongside emerging solutions including hierarchical buffering and surprise-gated updates.

Conclusion: Provides a roadmap toward cognitively-inspired, lifelong-learning Transformer architectures by synthesizing neuroscience principles with engineering advances in memory-augmented systems.

Abstract: Memory is fundamental to intelligence, enabling learning, reasoning, and adaptability across biological and artificial systems. While Transformer architectures excel at sequence modeling, they face critical limitations in long-range context retention, continual learning, and knowledge integration. This review presents a unified framework bridging neuroscience principles, including dynamic multi-timescale memory, selective attention, and consolidation, with engineering advances in Memory-Augmented Transformers. We organize recent progress through three taxonomic dimensions: functional objectives (context extension, reasoning, knowledge integration, adaptation), memory representations (parameter-encoded, state-based, explicit, hybrid), and integration mechanisms (attention fusion, gated control, associative retrieval). Our analysis of core memory operations (reading, writing, forgetting, and capacity management) reveals a shift from static caches toward adaptive, test-time learning systems. We identify persistent challenges in scalability and interference, alongside emerging solutions including hierarchical buffering and surprise-gated updates. This synthesis provides a roadmap toward cognitively-inspired, lifelong-learning Transformer architectures.

Method: Reformulates supervised learning as on-policy policy evaluation in RL, introduces generalized temporal difference learning algorithm, connects linear TD solutions to OLS, and develops novel generalized Bellman operator.

Result: Theoretical analysis shows TD solution is better estimator than OLS under correlated noise conditions. Algorithm converges under linear function approximation. Empirical studies validate theoretical results across regression and image classification tasks.

Conclusion: The proposed RL-based approach provides a superior framework for supervised learning when data exhibits connectivity and correlation, with proven convergence guarantees and practical utility across diverse applications.

Abstract: In traditional statistical learning, data points are usually assumed to be independently and identically distributed (i.i.d.) following an unknown probability distribution. This paper presents a contrasting viewpoint, perceiving data points as interconnected and employing a Markov reward process (MRP) for data modeling. We reformulate the typical supervised learning as an on-policy policy evaluation problem within reinforcement learning (RL), introducing a generalized temporal difference (TD) learning algorithm as a resolution. Theoretically, our analysis establishes connections between the solutions of linear TD learning and ordinary least squares (OLS). Under specific conditions – particularly when the noise is correlated – the TD solution serves as a more effective estimator than OLS. Furthermore, we show that when our algorithm is applied with many commonly used loss functions – such as those found in generalized linear models – it corresponds to the application of a novel and generalized Bellman operator. We prove that this operator admits a unique fixed point, and based on this, we establish convergence guarantees for our generalized TD algorithm under linear function approximation. Empirical studies verify our theoretical results, examine the vital design of our TD algorithm and show practical utility across various datasets, encompassing tasks such as regression and image classification with deep learning.

Method: Proposes HS-GPPT model with hybrid spectral filter backbone and local-global contrastive learning to acquire abundant spectral knowledge, then designs prompt graphs to align spectral distribution with pretexts for cross-homophily knowledge transfer.

Result: Extensive experiments validate effectiveness under both transductive and inductive learning settings, demonstrating successful spectral knowledge transfer across different homophily scenarios.

Conclusion: The proposed HS-GPPT framework successfully bridges spectral gaps between pre-training and downstream tasks, enabling effective graph knowledge transfer across varying homophily conditions through spectral alignment.

Abstract: Graph ``pre-training and prompt-tuning’’ aligns downstream tasks with pre-trained objectives to enable efficient knowledge transfer under limited supervision. However, existing methods rely on homophily-based low-frequency knowledge, failing to handle diverse spectral distributions in real-world graphs with varying homophily. Our theoretical analysis reveals a spectral specificity principle: optimal knowledge transfer requires alignment between pre-trained spectral filters and the intrinsic spectrum of downstream graphs. Under limited supervision, large spectral gaps between pre-training and downstream tasks impede effective adaptation. To bridge this gap, we propose the HS-GPPT model, a novel framework that ensures spectral alignment throughout both pre-training and prompt-tuning. We utilize a hybrid spectral filter backbone and local-global contrastive learning to acquire abundant spectral knowledge. Then we design prompt graphs to align the spectral distribution with pretexts, facilitating spectral knowledge transfer across homophily and heterophily. Extensive experiments validate the effectiveness under both transductive and inductive learning settings. Our code is available at https://anonymous.4open.science/r/HS-GPPT-62D2/.

Method: Used model-free RL algorithms (SAC, TQC, CrossQ) trained directly on experiments without simulation pre-training for laser-to-fiber coupling.

Result: RL agents achieved 90% coupling efficiency (matching human experts) but were faster, with CrossQ performing best in speed and requiring half the training time.

Conclusion: Direct RL training can replace extensive system modeling, demonstrating RL’s potential for complex optical applications where noise modeling is infeasible.

Abstract: Setting up and controlling optical systems is often a challenging and tedious task. The high number of degrees of freedom to control mirrors, lenses, or phases of light makes automatic control challenging, especially when the complexity of the system cannot be adequately modeled due to noise or non-linearities. Here, we show that reinforcement learning (RL) can overcome these challenges when coupling laser light into an optical fiber, using a model-free RL approach that trains directly on the experiment without pre-training on simulations. By utilizing the sample-efficient algorithms Soft Actor-Critic (SAC), Truncated Quantile Critics (TQC), or CrossQ, our agents learn to couple with 90% efficiency. A human expert reaches this efficiency, but the RL agents are quicker. In particular, the CrossQ agent outperforms the other agents in coupling speed while requiring only half the training time. We demonstrate that direct training on an experiment can replace extensive system modeling. Our result exemplifies RL’s potential to tackle problems in optics, paving the way for more complex applications where full noise modeling is not feasible.

Method: Uses maximum mean discrepancy (MMD) as mismatch measure, reduces partitioning to kernel k-means clustering with linear programming constraints for size, label, and group distribution control.

Result: Outperforms alternative splitting strategies across various datasets and training algorithms for both domain generalization and unsupervised domain adaptation tasks.

Conclusion: MMD between training and validation sets correlates well with test domain accuracy, validating the approach for robust model selection under domain shift.

Abstract: This paper considers the problem of model selection under domain shift. Motivated by principles from distributionally robust optimisation and domain adaptation theory, it is proposed that the training-validation split should maximise the distribution mismatch between the two sets. By adopting the maximum mean discrepancy (MMD) as the measure of mismatch, it is shown that the partitioning problem reduces to kernel k-means clustering. A constrained clustering algorithm, which leverages linear programming to control the size, label, and (optionally) group distributions of the splits, is presented. The algorithm does not require additional metadata, and comes with convergence guarantees. In experiments, the technique consistently outperforms alternative splitting strategies across a range of datasets and training algorithms, for both domain generalisation and unsupervised domain adaptation tasks. Analysis also shows the MMD between the training and validation sets to be well-correlated with test domain accuracy, further substantiating the validity of this approach.

Method: The authors extensively benchmark spectral GNNs from a spectral perspective, categorizing them as spectral graph filters. They implement 27 different filters within a unified spectral-oriented framework with dedicated graph computations and efficient training schemes that enable deployment on million-scale graphs.

Result: The benchmark reveals an intricate landscape regarding the effectiveness and efficiency of spectral graph filters, demonstrating that desirable performance can be achieved through tailored spectral manipulation of graph data. The implementation enables deployment on million-scale graphs with comparable performance and less overhead.

Conclusion: The study provides novel observations and practical guidelines for spectral GNNs, challenging prevailing beliefs and showing that spectral graph filters have the potential to achieve good performance through proper spectral manipulation, with their unified framework enabling efficient deployment across various graph scales.

Abstract: With recent advancements in graph neural networks (GNNs), spectral GNNs have received increasing popularity by virtue of their ability to retrieve graph signals in the spectral domain. These models feature uniqueness in efficient computation as well as rich expressiveness, which stems from advanced management and profound understanding of graph data. However, few systematic studies have been conducted to assess spectral GNNs, particularly in benchmarking their efficiency, memory consumption, and effectiveness in a unified and fair manner. There is also a pressing need to select spectral models suitable for learning specific graph data and deploying them to massive web-scale graphs, which is currently constrained by the varied model designs and training settings. In this work, we extensively benchmark spectral GNNs with a focus on the spectral perspective, demystifying them as spectral graph filters. We analyze and categorize 35 GNNs with 27 corresponding filters, spanning diverse formulations and utilizations of the graph data. Then, we implement the filters within a unified spectral-oriented framework with dedicated graph computations and efficient training schemes. In particular, our implementation enables the deployment of spectral GNNs over million-scale graphs and various tasks with comparable performance and less overhead. Thorough experiments are conducted on the graph filters with comprehensive metrics on effectiveness and efficiency, offering novel observations and practical guidelines that are only available from our evaluations across graph scales. Different from the prevailing belief, our benchmark reveals an intricate landscape regarding the effectiveness and efficiency of spectral graph filters, demonstrating the potential to achieve desirable performance through tailored spectral manipulation of graph data.

Method: Proposes MUC framework and Alignment Calibration (AC) method that explicitly considers contrastive learning properties and optimizes towards new auditing metrics for easy verification.

Result: AC achieves state-of-the-art performance approximating exact unlearning (retraining) and enables clear visualization of unlearning effects through black-box evaluation on SimCLR, MoCo, and CLIP models.

Conclusion: The proposed Alignment Calibration method effectively addresses machine unlearning for contrastive learning models, providing superior performance and practical verification capabilities compared to existing approaches.

Abstract: Machine unlearning offers effective solutions for revoking the influence of specific training data on pre-trained model parameters. While existing approaches address unlearning for classification and generative models, they overlook an important category of machine learning models: contrastive learning (CL) methods. This paper addresses this gap by introducing the Machine Unlearning for Contrastive Learning (MUC) framework and adapting existing methods. We identify limitations in current approaches, noting that several methods perform inadequately as unlearners and that existing evaluation tools insufficiently validate unlearning effects in contrastive learning. To address these issues, we propose Alignment Calibration (AC), a novel method that explicitly considers contrastive learning properties and optimizes towards new auditing metrics for easy verification of unlearning. Through empirical comparisons with baseline methods on SimCLR, MoCo, and CLIP, we demonstrate that AC: (1) achieves state-of-the-art performance, approximating exact unlearning (retraining); (2) enables data owners to clearly visualize unlearning effects through black-box evaluation. The code is available at https://github.com/EhanW/Alignment-Calibration.

Method: Created ESA-ADB benchmark through collaboration between ESA operations engineers and ML experts, featuring annotated real-life telemetry from three ESA missions (two included in benchmark), with a novel hierarchical evaluation pipeline for assessing anomaly detection algorithms.

Result: Evaluation of typical anomaly detection algorithms shows that new approaches are necessary to address spacecraft operators’ needs and requirements.

Conclusion: The publicly available ESA-ADB benchmark establishes a new standard for satellite telemetry anomaly detection and enables full reproducibility, highlighting the need for improved ML approaches in this domain.

Abstract: Machine learning has vast potential to improve anomaly detection in satellite telemetry which is a crucial task for spacecraft operations. This potential is currently hampered by a lack of comprehensible benchmarks for multivariate time series anomaly detection, especially for the challenging case of satellite telemetry. The European Space Agency Benchmark for Anomaly Detection in Satellite Telemetry (ESA-ADB) aims to address this challenge and establish a new standard in the domain. It is a result of close cooperation between spacecraft operations engineers from the European Space Agency (ESA) and machine learning experts. The newly introduced ESA Anomalies Dataset contains annotated real-life telemetry from three different ESA missions, out of which two are included in ESA-ADB. Results of typical anomaly detection algorithms assessed in our novel hierarchical evaluation pipeline show that new approaches are necessary to address operators’ needs. All elements of ESA-ADB are publicly available to ensure its full reproducibility.

Method: Proposes variational flow matching (VFM) formulation that approximates posterior probability paths, with CatFlow as a specific implementation for categorical data using deterministic dynamics.

Result: CatFlow achieves strong performance on abstract graph generation and molecular generation tasks, matching or exceeding current state-of-the-art models in all evaluated cases.

Conclusion: The variational flow matching framework provides a unified perspective that connects flow matching and score-based models, with CatFlow demonstrating practical effectiveness for categorical data generation.

Abstract: We present a formulation of flow matching as variational inference, which we refer to as variational flow matching (VFM). Based on this formulation we develop CatFlow, a flow matching method for categorical data. CatFlow is easy to implement, computationally efficient, and achieves strong results on graph generation tasks. In VFM, the objective is to approximate the posterior probability path, which is a distribution over possible end points of a trajectory. We show that VFM admits both the CatFlow objective and the original flow matching objective as special cases. We also relate VFM to score-based models, in which the dynamics are stochastic rather than deterministic, and derive a bound on the model likelihood based on a reweighted VFM objective. We evaluate CatFlow on one abstract graph generation task and two molecular generation tasks. In all cases, CatFlow exceeds or matches performance of the current state-of-the-art models.

Method: Decoupled Annealing Posterior Sampling (DAPS) uses a novel noise annealing process that decouples consecutive diffusion sampling steps, allowing larger variations while ensuring time-marginals anneal to the true posterior.

Result: DAPS significantly improves sample quality and stability across multiple image restoration tasks, particularly in complicated nonlinear inverse problems.

Conclusion: The decoupled annealing approach enables better exploration of solution space and achieves higher success rates for accurate reconstructions in challenging inverse problems.

Abstract: Diffusion models have recently achieved success in solving Bayesian inverse problems with learned data priors. Current methods build on top of the diffusion sampling process, where each denoising step makes small modifications to samples from the previous step. However, this process struggles to correct errors from earlier sampling steps, leading to worse performance in complicated nonlinear inverse problems, such as phase retrieval. To address this challenge, we propose a new method called Decoupled Annealing Posterior Sampling (DAPS) that relies on a novel noise annealing process. Specifically, we decouple consecutive steps in a diffusion sampling trajectory, allowing them to vary considerably from one another while ensuring their time-marginals anneal to the true posterior as we reduce noise levels. This approach enables the exploration of a larger solution space, improving the success rate for accurate reconstructions. We demonstrate that DAPS significantly improves sample quality and stability across multiple image restoration tasks, particularly in complicated nonlinear inverse problems.

Method: Comprehensive survey and taxonomy of recent recurrent models including deep State-Space models, large-context Transformers with recurrent computations, and novel learning algorithms beyond standard Backpropagation Through Time.

Result: Identification of emerging trends showing strong revival of recurrent approaches, with new architectural solutions that overcome limitations of current technologies and enable better long-term dependency handling.

Conclusion: There is significant room for exploring novel learning algorithms that enable true online processing with local-forward computations, opening new research directions beyond traditional backpropagation methods.

Abstract: Effectively learning from sequential data is a longstanding goal of Artificial Intelligence, especially in the case of long sequences. From the dawn of Machine Learning, several researchers have pursued algorithms and architectures capable of processing sequences of patterns, retaining information about past inputs while still leveraging future data, without losing precious long-term dependencies and correlations. While such an ultimate goal is inspired by the human hallmark of continuous real-time processing of sensory information, several solutions have simplified the learning paradigm by artificially limiting the processed context or dealing with sequences of limited length, given in advance. These solutions were further emphasized by the ubiquity of Transformers, which initially overshadowed the role of Recurrent Neural Nets. However, recurrent networks are currently experiencing a strong recent revival due to the growing popularity of (deep) State-Space models and novel instances of large-context Transformers, which are both based on recurrent computations that aim to go beyond several limits of currently ubiquitous technologies. The fast development of Large Language Models has renewed the interest in efficient solutions to process data over time. This survey provides an in-depth summary of the latest approaches that are based on recurrent models for sequential data processing. A complete taxonomy of recent trends in architectural and algorithmic solutions is reported and discussed, guiding researchers in this appealing research field. The emerging picture suggests that there is room for exploring novel routes, constituted by learning algorithms that depart from the standard Backpropagation Through Time, towards a more realistic scenario where patterns are effectively processed online, leveraging local-forward computations, and opening new directions for research on this topic.

Method: Proposes a principled time-weighting strategy grounded in Lyapunov exponents theory, which quantifies solution sensitivity to perturbations over time and automatically adjusts weights according to the system’s stability regime.

Result: Numerical experiments on challenging benchmarks (Lorenz system and Burgers’ equation) show the method is effective and robust, offering improved convergence and accuracy compared to existing techniques.

Conclusion: Incorporating causality and dynamical system behavior into PINN training through Lyapunov-based weighting provides a robust framework for solving time-dependent problems with enhanced reliability.

Abstract: We introduce a novel method to handle the time dimension when Physics-Informed Neural Networks (PINNs) are used to solve time-dependent differential equations; our proposal focuses on how time sampling and weighting strategies affect solution quality. While previous methods proposed heuristic time-weighting schemes, our approach is grounded in theoretical insights derived from the Lyapunov exponents, which quantify the sensitivity of solutions to perturbations over time. This principled methodology automatically adjusts weights based on the stability regime of the system – whether chaotic, periodic, or stable. Numerical experiments on challenging benchmarks, including the chaotic Lorenz system and the Burgers’ equation, demonstrate the effectiveness and robustness of the proposed method. Compared to existing techniques, our approach offers improved convergence and accuracy without requiring additional hyperparameter tuning. The findings underline the importance of incorporating causality and dynamical system behavior into PINN training strategies, providing a robust framework for solving time-dependent problems with enhanced reliability.

Method: Develop data-dependent upper bounds applicable to any model/algorithm choice, derive oracle bounds for Gibbs posteriors, and create an algorithm based on these bounds.

Result: Empirical demonstration shows the approach yields tight and practical bounds on forgetting for various continual learning problems and algorithms.

Conclusion: The theoretical framework provides effective bounds for quantifying forgetting in continual learning, with practical algorithmic applications.

Abstract: In continual learning, knowledge must be preserved and re-used between tasks, maintaining good transfer to future tasks and minimizing forgetting of previously learned ones. While several practical algorithms have been devised for this setting, there have been few theoretical works aiming to quantify and bound the degree of Forgetting in general settings. For \emph{exemplar-free} methods, we provide both data-dependent upper bounds that apply \emph{regardless of model and algorithm choice}, and oracle bounds for Gibbs posteriors. We derive an algorithm based on our bounds and demonstrate empirically that our approach yields tight and practical bounds on forgetting for several continual learning problems and algorithms.

Method: Introduces GraphLand benchmark with 14 diverse graph datasets from industrial applications, enabling unified evaluation across graphs with varying sizes, structures, and features. Includes temporal distribution shift analysis and comparison of GNNs with gradient-boosted decision trees.

Result: GBDT models with graph-based input features can be strong baselines, and current graph foundation models fail to produce competitive results on the proposed diverse industrial datasets.

Conclusion: GraphLand addresses the critical gap in graph ML evaluation by providing diverse industrial datasets, revealing limitations of current graph foundation models and showing the competitiveness of GBDT approaches in realistic settings.

Abstract: Although data that can be naturally represented as graphs is widespread in real-world applications across diverse industries, popular graph ML benchmarks for node property prediction only cover a surprisingly narrow set of data domains, and graph neural networks (GNNs) are often evaluated on just a few academic citation networks. This issue is particularly pressing in light of the recent growing interest in designing graph foundation models. These models are supposed to be able to transfer to diverse graph datasets from different domains, and yet the proposed graph foundation models are often evaluated on a very limited set of datasets from narrow applications. To alleviate this issue, we introduce GraphLand: a benchmark of 14 diverse graph datasets for node property prediction from a range of different industrial applications. GraphLand allows evaluating graph ML models on a wide range of graphs with diverse sizes, structural characteristics, and feature sets, all in a unified setting. Further, GraphLand allows investigating such previously underexplored research questions as how realistic temporal distributional shifts under transductive and inductive settings influence graph ML model performance. To mimic realistic industrial settings, we use GraphLand to compare GNNs with gradient-boosted decision trees (GBDT) models that are popular in industrial applications and show that GBDTs provided with additional graph-based input features can sometimes be very strong baselines. Further, we evaluate currently available general-purpose graph foundation models and find that they fail to produce competitive results on our proposed datasets.

Method: Proposed KACQ-DCNN architecture that replaces traditional MLPs with Kolmogorov-Arnold Networks (KANs) enabling learnable univariate activation functions. Uses 4-qubit, 1-layer quantum component. Validated with LIME, SHAP explainability techniques and conformal prediction for uncertainty quantification.

Result: Achieved 92.03% accuracy, 92.00% macro-average precision/recall/F1 scores, and 94.77% ROC-AUC - significantly outperforming 37 benchmark models (16 classical + 12 quantum). Ablation studies showed ~2% performance improvement over MLP variants through classical-quantum integration.

Conclusion: KACQ-DCNN significantly improves cardiovascular diagnostics by combining high accuracy with enhanced interpretability and robust uncertainty quantification, demonstrating the synergistic benefits of classical-quantum integration in medical AI applications.

Abstract: Heart failure is a leading cause of global mortality, necessitating improved diagnostic strategies. Classical machine learning models struggle with challenges such as high-dimensional data, class imbalances, poor feature representations, and a lack of interpretability. While quantum machine learning holds promise, current hybrid models have not fully exploited quantum advantages. In this paper, we propose the Kolmogorov-Arnold Classical-Quantum Dual-Channel Neural Network (KACQ-DCNN), a novel hybrid architecture that replaces traditional multilayer perceptrons with Kolmogorov-Arnold Networks (KANs), enabling learnable univariate activation functions. Our KACQ-DCNN 4-qubit, 1-layer model outperforms 37 benchmark models, including 16 classical and 12 quantum neural networks, achieving an accuracy of 92.03%, with macro-average precision, recall, and F1 scores of 92.00%. It also achieved a ROC-AUC of 94.77%, surpassing other models by significant margins, as validated by paired t-tests with a significance threshold of 0.0056 (after Bonferroni correction). Ablation studies highlight the synergistic effect of classical-quantum integration, improving performance by about 2% over MLP variants. Additionally, LIME and SHAP explainability techniques enhance feature interpretability, while conformal prediction provides robust uncertainty quantification. Our results demonstrate that KACQ-DCNN improves cardiovascular diagnostics by combining high accuracy with interpretability and uncertainty quantification.

Method: Introduced novel policy learning methods based on Q- and A-Learning to determine optimal intervention policies under bipartite network interference. Applied methods to comprehensive dataset of Medicare claims, power plant data, and pollution transport networks to optimize scrubber installation strategies.

Result: Annual ischemic heart disease hospitalization rates could be reduced by 23.37-55.30 per 10,000 person-years through optimal policies under different cost constraints, demonstrating significant health benefits from targeted interventions.

Conclusion: The proposed policy learning methods effectively address bipartite network interference and provide optimal strategies for power plant interventions that maximize health benefits while respecting cost constraints, offering practical solutions for reducing pollution-related cardiovascular disease burdens.

Abstract: The substantial effect of air pollution on cardiovascular disease and mortality burdens is well-established. Emissions-reducing interventions on coal-fired power plants – a major source of hazardous air pollution – have proven to be an effective, but costly, strategy for reducing pollution-related health burdens. Targeting the power plants that achieve maximum health benefits while satisfying realistic cost constraints is challenging. The primary difficulty lies in quantifying the health benefits of intervening at particular plants. This is further complicated because interventions are applied on power plants, while health impacts occur in potentially distant communities, a setting known as bipartite network interference (BNI). In this paper, we introduce novel policy learning methods based on Q- and A-Learning to determine the optimal policy under arbitrary BNI. We derive asymptotic properties and demonstrate finite sample efficacy in simulations. We apply our novel methods to a comprehensive dataset of Medicare claims, power plant data, and pollution transport networks. Our goal is to determine the optimal strategy for installing power plant scrubbers to minimize ischemic heart disease (IHD) hospitalizations under various cost constraints. We find that annual IHD hospitalization rates could be reduced in a range from 23.37-55.30 per 10,000 person-years through optimal policies under different cost constraints.

Method: Used neural networks with transformer attention mechanisms, comparing agents with and without an attention schema. Tested agents on: 1) categorizing attention states of other agents, 2) developing attention patterns that are easier for others to categorize, and 3) joint painting tasks requiring mutual prediction.

Result: Agents with attention schema showed: higher accuracy in categorizing others’ attention states; developed more interpretable attention patterns; improved performance in cooperative tasks; and these improvements were specific to attention-related tasks rather than general network complexity increases.

Conclusion: Attention schema provides computational benefits for mutual interpretability and interactive behavior in artificial agents, supporting the hypothesis that similar principles might apply to biological attention systems in humans.

Abstract: Growing evidence suggests that the brain uses an attention schema, or a simplified model of attention, to help control what it attends to. One proposed benefit of this model is to allow agents to model the attention states of other agents, and thus predict and interact with other agents. The effects of an attention schema may be examined in artificial agents. Although attention mechanisms in artificial agents are different from in biological brains, there may be some principles in common. In both cases, select features or representations are emphasized for better performance. Here, using neural networks with transformer attention mechanisms, we asked whether the addition of an attention schema affected the ability of agents to make judgements about and cooperate with each other. First, we found that an agent with an attention schema is better at categorizing the attention states of other agents (higher accuracy). Second, an agent with an attention schema develops a pattern of attention that is easier for other agents to categorize. Third, in a joint task where two agents must predict each other to paint a scene together, adding an attention schema improves performance. Finally, the performance improvements are not caused by a general increase in network complexity. Instead, improvement is specific to tasks involving judging, categorizing, or predicting the attention of other agents. These results support the hypothesis that an attention schema has computational properties beneficial to mutual interpretability and interactive behavior. We speculate that the same principles might pertain to biological attention and attention schemas in people.

Method: Transforms classic 8-tile puzzle into visual RL task with images from arbitrarily large datasets. Uses adjustable grid sizes and image pools to precisely control representation learning complexity while keeping environment dynamics, observation, and action spaces fixed.

Result: Experiments show all tested model-free and model-based RL algorithms exhibit performance degradation as image pool size increases, both in-distribution and out-of-distribution. Sophisticated representation learning techniques often underperform simpler approaches like data augmentation.

Conclusion: The findings reveal critical gaps in current visual representation learning for RL and establish SPGym as a valuable benchmark for developing more robust and generalizable decision-making systems that can handle visual diversity.

Abstract: Effective visual representation learning is crucial for reinforcement learning (RL) agents to extract task-relevant information from raw sensory inputs and generalize across diverse environments. However, existing RL benchmarks lack the ability to systematically evaluate representation learning capabilities in isolation from other learning challenges. To address this gap, we introduce the Sliding Puzzles Gym (SPGym), a novel benchmark that transforms the classic 8-tile puzzle into a visual RL task with images drawn from arbitrarily large datasets. SPGym’s key innovation lies in its ability to precisely control representation learning complexity through adjustable grid sizes and image pools, while maintaining fixed environment dynamics, observation, and action spaces. This design enables researchers to isolate and scale the visual representation challenge independently of other learning components. Through extensive experiments with model-free and model-based RL algorithms, we uncover fundamental limitations in current methods’ ability to handle visual diversity. As we increase the pool of possible images, all algorithms exhibit in- and out-of-distribution performance degradation, with sophisticated representation learning techniques often underperforming simpler approaches like data augmentation. These findings highlight critical gaps in visual representation learning for RL and establish SPGym as a valuable tool for driving progress in robust, generalizable decision-making systems.

Method: The proposed FreDA framework performs domain-aware separation using high-frequency texture cues in Fourier space, decomposing globally heterogeneous data into locally homogeneous components, and employs decentralized learning and augmentation strategies.

Result: Extensive experiments across corrupted, natural, and medical environments demonstrate superior performance over state-of-the-art methods in handling complex mixed distribution shifts.

Conclusion: Frequency-domain analysis provides an effective approach for test-time adaptation under heterogeneous distribution shifts, with FreDA showing strong robustness across diverse practical scenarios.

Abstract: Deploying deep models in real-world scenarios remains challenging due to significant performance drops under distribution shifts between training and deployment environments. Test-Time Adaptation (TTA) has recently emerged as a promising solution, enabling on-the-fly model adaptation without access to source data. However, its effectiveness degrades significantly in the presence of complex, mixed distribution shifts - common in practical settings - where multiple latent domains coexist. Adapting under such intrinsic heterogeneity, especially in unlabeled and online conditions, remains an open and underexplored challenge. In this paper, we study TTA under mixed distribution shifts and move beyond conventional homogeneous adaptation paradigms. By revisiting TTA from a frequency-domain perspective, we observe that distribution heterogeneity often manifests in Fourier space - for instance, high-frequency components tend to carry domain-specific variations. This motivates us to perform domain-aware separation using high-frequency texture cues, making diverse shift patterns more tractable. To this end, we propose FreDA, a novel Frequency-based Decentralized Adaptation framework that decomposes globally heterogeneous data into locally homogeneous components in the frequency domain. It further employs decentralized learning and augmentation strategies to robustly adapt under complex, evolving shifts. Extensive experiments across various environments (corrupted, natural, and medical) demonstrate the superiority of our proposed framework over the state-of-the-arts.

Method: Formulates hierarchical policy learning as bi-level optimization problem, leverages direct preference optimization (DPO) to train higher-level policy using preference feedback, and incorporates regularization to ensure subgoal feasibility.

Result: Achieves up to 40% improvement over state-of-the-art baselines in sparse reward scenarios on challenging robotic navigation and manipulation benchmarks.

Conclusion: DIPPER effectively overcomes longstanding limitations of HRL by mitigating non-stationarity and infeasible subgoal problems through its novel optimization framework.

Abstract: Hierarchical reinforcement learning (HRL) enables agents to solve complex, long-horizon tasks by decomposing them into manageable sub-tasks. However, HRL methods often suffer from two fundamental challenges: (i) non-stationarity, caused by the changing behavior of the lower-level policy during training, which destabilizes higher-level policy learning, and (ii) the generation of infeasible subgoals that lower-level policies cannot achieve. In this work, we introduce DIPPER, a novel HRL framework that formulates hierarchical policy learning as a bi-level optimization problem and leverages direct preference optimization (DPO) to train the higher-level policy using preference feedback. By optimizing the higher-level policy with DPO, we decouple higher-level learning from the non-stationary lower-level reward signal, thus mitigating non-stationarity. To further address the infeasible subgoal problem, DIPPER incorporates a regularization that tries to ensure the feasibility of subgoal tasks within the capabilities of the lower-level policy. Extensive experiments on challenging robotic navigation and manipulation benchmarks demonstrate that DIPPER achieves up to 40% improvement over state-of-the-art baselines in sparse reward scenarios, highlighting its effectiveness in overcoming longstanding limitations of HRL.

Method: Pattern recognition systems using threshold-based duty-cycle detection, manually extracted features, pattern-matching, and supervised tiny ML models (decision tree, random forest, extra trees, XGBoost, Gaussian naive Bayes, MLP).

Result: Both proposed methods outperform former approach - heuristic rule-based achieved 97.3% F1-score for normal and 80.2% for abnormal cycles; ML-based performed better on aging dataset with 91.3% normal and 67.9% abnormal F1-scores. Efficient real-time operation with 13.3-20.6 μJ energy consumption during inference.

Conclusion: The proposed methods provide robust solutions for continuous monitoring of mining conveyor belt work cycles, demonstrating high performance and energy efficiency suitable for real-time deployment on low-power microcontrollers.

Abstract: Conveyor belts are crucial in mining operations by enabling the continuous and efficient movement of bulk materials over long distances, which directly impacts productivity. While detecting anomalies in specific conveyor belt components has been widely studied, identifying the root causes of these failures, such as changing production conditions and operator errors, remains critical. Continuous monitoring of mining conveyor belt work cycles is still at an early stage and requires robust solutions. Recently, an anomaly detection method for duty cycle operations of a mining conveyor belt has been proposed. Based on its limited performance and unevaluated long-term proper operation, this study proposes two novel methods for classifying normal and abnormal duty cycles. The proposed approaches are pattern recognition systems that make use of threshold-based duty-cycle detection mechanisms, manually extracted features, pattern-matching, and supervised tiny machine learning models. The explored low-computational models include decision tree, random forest, extra trees, extreme gradient boosting, Gaussian naive Bayes, and multi-layer perceptron. A comprehensive evaluation of the former and proposed approaches is carried out on two datasets. Both proposed methods outperform the former method in anomaly detection, with the best-performing approach being dataset-dependent. The heuristic rule-based approach achieves the highest F1-score in the same dataset used for algorithm training, with 97.3% for normal cycles and 80.2% for abnormal cycles. The ML-based approach performs better on a dataset including the effects of machine aging, with an F1-score scoring 91.3% for normal cycles and 67.9% for abnormal cycles. Implemented on two low-power microcontrollers, the methods demonstrate efficient, real-time operation with energy consumption of 13.3 and 20.6 \textmu J during inference. These results …

Method: Proposes Signed Graph Prompt Tuning (SGPT) with: 1) graph template based on balance theory to disentangle mixed node relationships, 2) task template unifying downstream tasks into link prediction, 3) feature prompts for semantic space alignment, and 4) semantic prompts for task-aware sign integration.

Result: Extensive experiments on seven benchmark signed graph datasets show SGPT significantly outperforms existing state-of-the-art methods.

Conclusion: SGPT establishes a powerful and generalizable solution for few-shot signed graph learning by effectively transferring knowledge from pre-trained unsigned GNNs to signed graph tasks.

Abstract: Signed Graph Neural Networks (SGNNs) are effective in learning expressive representations for signed graphs but typically require substantial task-specific labels, limiting their applicability in label-scarce industrial scenarios. In contrast, unsigned graph structures are abundant and can be readily leveraged to pre-train Graph Neural Networks (GNNs), offering a promising solution to reduce supervision requirements in downstream signed graph tasks. However, transferring knowledge from unsigned to signed graphs is non-trivial due to the fundamental discrepancies in graph types and task objectives between pre-training and downstream phases. To address this challenge, we propose Signed Graph Prompt Tuning (SGPT), a novel graph prompting framework that adapts pre-trained unsigned GNNs to few-shot signed graph tasks. We first design a graph template based on balance theory to disentangle mixed node relationships introduced by negative links, mitigating the structural mismatches between unsigned and signed graphs. We further introduce a task template that reformulates downstream signed tasks into a unified link prediction objective, aligning their optimization goals with the pre-training task. Furthermore, we develop feature prompts that align downstream semantic spaces with the feature spaces learned during pre-training, and semantic prompts to integrate link sign semantics in a task-aware manner. We conduct extensive experiments on seven benchmark signed graph datasets, demonstrating that SGPT significantly outperforms existing state-of-the-art methods, establishing a powerful and generalizable solution for few-shot signed graph learning.

Method: Enhanced temporal difference decomposition (TD($\Delta$)) applied to SARSA, splitting action-value function into components linked to specific discount factors for multi-timescale learning.

Result: SARSA($\Delta$) reduces bias in updates, accelerates convergence in deterministic/stochastic settings including dense reward Atari environments, outperforms existing TD methods in tabular and deep RL benchmarks.

Conclusion: The proposed SARSA($\Delta$) method provides more effective and consistent learning, particularly beneficial for long-horizon improvement scenarios across various RL environments.

Abstract: In numerous episodic reinforcement learning (RL) environments, SARSA-based methodologies are employed to enhance policies aimed at maximizing returns over long horizons. Traditional SARSA algorithms face challenges in achieving an optimal balance between bias and variation, primarily due to their dependence on a single, constant discount factor ($\eta$). This investigation enhances the temporal difference decomposition method, TD($\Delta$), by applying it to the SARSA algorithm, now designated as SARSA($\Delta$). SARSA is a widely used on-policy RL method that enhances action-value functions via temporal difference updates. By splitting the action-value function down into components that are linked to specific discount factors, SARSA($\Delta$) makes learning easier across a range of time scales. This analysis makes learning more effective and ensures consistency, particularly in situations where long-horizon improvement is needed. The results of this research show that the suggested strategy works to lower bias in SARSA’s updates and speed up convergence in both deterministic and stochastic settings, even in dense reward Atari environments. Experimental results from a variety of benchmark settings show that the proposed SARSA($\Delta$) outperforms existing TD learning techniques in both tabular and deep RL environments.

Method: The authors present a decision-theoretic perspective that relates rigorous notions of uncertainty, predictive performance, and statistical dispersion in data.

Result: The framework supports clearer thinking about uncertainty concepts and provides insights showing that popular information-theoretic quantities can be poor estimators but still useful for guiding data acquisition.

Conclusion: A decision-theoretic approach offers a more coherent foundation for reasoning about uncertainty in machine learning predictions, addressing limitations of the traditional aleatoric-epistemic view.

Abstract: The ideas of aleatoric and epistemic uncertainty are widely used to reason about the probabilistic predictions of machine-learning models. We identify incoherence in existing discussions of these ideas and suggest this stems from the aleatoric-epistemic view being insufficiently expressive to capture all the distinct quantities that researchers are interested in. To address this we present a decision-theoretic perspective that relates rigorous notions of uncertainty, predictive performance and statistical dispersion in data. This serves to support clearer thinking as the field moves forward. Additionally we provide insights into popular information-theoretic quantities, showing they can be poor estimators of what they are often purported to measure, while also explaining how they can still be useful in guiding data acquisition.

Method: Used autoregressive GRUs, LSTMs, and Transformers trained on sequence-based number tasks. Applied Distributed Alignment Search (DAS) and extended it to Linear Alignment Functions (LAFs) to map neural activity to interpretable variables from symbolic algorithms.

Result: Found that alignments with symbolic algorithms can be very high, approximate, or fail depending on architecture and task. Recurrent models develop graded, symbol-like number variables, while shallow Transformers learn different anti-Markovian solutions without sufficient attention layers.

Conclusion: Causal interventions on neural subspaces are useful for NN interpretability, and different architectures develop fundamentally different numeric representations, with recurrent models creating symbol-like variables and Transformers requiring anti-Markovian solutions.

Abstract: What types of numeric representations emerge in neural systems, and what would a satisfying answer to this question look like? In this work, we interpret Neural Network (NN) solutions to sequence based number tasks using a variety of methods to understand how well we can interpret them through the lens of interpretable Symbolic Algorithms (SAs) – precise programs describable by rules and typed, mutable variables. We use autoregressive GRUs, LSTMs, and Transformers trained on tasks where the correct tokens depend on numeric information only latent in the task structure. We show through multiple causal and theoretical methods that we can interpret raw NN activity through the lens of simplified SAs when we frame the activity in terms of neural subspaces rather than individual neurons. Using Distributed Alignment Search (DAS), we find that, depending on network architecture, dimensionality, and task specifications, alignments with SA’s can be very high, or they can be only approximate, or fail altogether. We extend our analytic toolkit to address the failure cases by expanding the DAS framework to a broader class of alignment functions that more flexibly capture NN activity in terms of interpretable variables from SAs, and we provide theoretic and empirical explorations of Linear Alignment Functions (LAFs) in contrast to the preexisting Orthogonal Alignment Functions (OAFs). Through analyses of specific cases we confirm the usefulness of causal interventions on neural subspaces for NN interpretability, and we show that recurrent models can develop graded, symbol-like number variables in their neural activity. We further show that shallow Transformers learn very different solutions than recurrent networks, and we prove that such models must use anti-Markovian solutions – solutions that do not rely on cumulative, Markovian hidden states – in the absence of sufficient attention layers.

Method: Proposes PINNMamba framework that introduces sub-sequence modeling with State Space Models (SSM), which serve as continuous-discrete articulation for initial condition propagation and eliminate simplicity bias through aligned moderate granularity sequences.

Result: Experimental results show PINNMamba can reduce errors by up to 86.3% compared with state-of-the-art architecture.

Conclusion: SSM-based PINNMamba effectively addresses the initial condition propagation problem in PINNs and significantly outperforms existing methods, with code publicly available.

Abstract: Physics-Informed Neural Networks (PINNs) are a kind of deep-learning-based numerical solvers for partial differential equations (PDEs). Existing PINNs often suffer from failure modes of being unable to propagate patterns of initial conditions. We discover that these failure modes are caused by the simplicity bias of neural networks and the mismatch between PDE’s continuity and PINN’s discrete sampling. We reveal that the State Space Model (SSM) can be a continuous-discrete articulation allowing initial condition propagation, and that simplicity bias can be eliminated by aligning a sequence of moderate granularity. Accordingly, we propose PINNMamba, a novel framework that introduces sub-sequence modeling with SSM. Experimental results show that PINNMamba can reduce errors by up to 86.3% compared with state-of-the-art architecture. Our code is available at https://github.com/miniHuiHui/PINNMamba.

Method: Adapts MR-NaS exploration scheme from Multi-Reward Best Policy Identification to minimize sample complexity. Uses instance-specific lower bound based on value deviation measure and proposes efficient convex approximation for hard non-convex optimization.

Result: Experiments in tabular domains demonstrate the effectiveness of the adaptive exploration scheme for achieving ε-accurate estimates with high confidence across finite or convex reward sets.

Conclusion: The proposed approach successfully addresses the multi-reward multi-policy evaluation problem with PAC guarantees, providing an efficient exploration policy that scales appropriately with value deviation measures.

Abstract: We study the policy evaluation problem in an online multi-reward multi-policy discounted setting, where multiple reward functions must be evaluated simultaneously for different policies. We adopt an $(\epsilon,\delta)$-PAC perspective to achieve $\epsilon$-accurate estimates with high confidence across finite or convex sets of rewards, a setting that has not been investigated in the literature. Building on prior work on Multi-Reward Best Policy Identification, we adapt the MR-NaS exploration scheme to jointly minimize sample complexity for evaluating different policies across different reward sets. Our approach leverages an instance-specific lower bound revealing how the sample complexity scales with a measure of value deviation, guiding the design of an efficient exploration policy. Although computing this bound entails a hard non-convex optimization, we propose an efficient convex approximation that holds for both finite and convex reward sets. Experiments in tabular domains demonstrate the effectiveness of this adaptive exploration scheme.

Method: Proposes a graph neural network framework with multi-scale graph structures, dynamic gating units for adaptive messaging, and neural nested grid method for high-resolution regional forecasts to capture local features and mitigate boundary loss.

Result: OneForecast performs excellently across global to regional scales and short-term to long-term forecasts, with particularly strong performance in extreme event predictions.

Conclusion: The proposed framework successfully addresses key challenges in weather forecasting by combining dynamic system perspective with multi-grid theory, demonstrating superior performance especially for extreme weather events.

Abstract: Accurate weather forecasts are important for disaster prevention, agricultural planning, etc. Traditional numerical weather prediction (NWP) methods offer physically interpretable high-accuracy predictions but are computationally expensive and fail to fully leverage rapidly growing historical data. In recent years, deep learning models have made significant progress in weather forecasting, but challenges remain, such as balancing global and regional high-resolution forecasts, excessive smoothing in extreme event predictions, and insufficient dynamic system modeling. To address these issues, this paper proposes a global-regional nested weather forecasting framework (OneForecast) based on graph neural networks. By combining a dynamic system perspective with multi-grid theory, we construct a multi-scale graph structure and densify the target region to capture local high-frequency features. We introduce an adaptive messaging mechanism, using dynamic gating units to deeply integrate node and edge features for more accurate extreme event forecasting. For high-resolution regional forecasts, we propose a neural nested grid method to mitigate boundary information loss. Experimental results show that OneForecast performs excellently across global to regional scales and short-term to long-term forecasts, especially in extreme event predictions. Codes link https://github.com/YuanGao-YG/OneForecast.

Method: Develops inverse bridge matching formulation with tractable objective that can distill both conditional and unconditional DBMs into one-step generators using only corrupted images for training.

Result: Achieves 4x to 100x inference acceleration across various tasks (super-resolution, JPEG restoration, sketch-to-image) while sometimes providing better generation quality than the original teacher models.

Conclusion: The proposed distillation technique successfully addresses the slow inference problem of DBMs, making them practical for real-world applications while maintaining or improving performance.

Abstract: Learning diffusion bridge models is easy; making them fast and practical is an art. Diffusion bridge models (DBMs) are a promising extension of diffusion models for applications in image-to-image translation. However, like many modern diffusion and flow models, DBMs suffer from the problem of slow inference. To address it, we propose a novel distillation technique based on the inverse bridge matching formulation and derive the tractable objective to solve it in practice. Unlike previously developed DBM distillation techniques, the proposed method can distill both conditional and unconditional types of DBMs, distill models in a one-step generator, and use only the corrupted images for training. We evaluate our approach for both conditional and unconditional types of bridge matching on a wide set of setups, including super-resolution, JPEG restoration, sketch-to-image, and other tasks, and show that our distillation technique allows us to accelerate the inference of DBMs from 4x to 100x and even provide better generation quality than used teacher model depending on particular setup. We provide the code at https://github.com/ngushchin/IBMD

Method: Built on the JAX framework, leveraging modern hardware acceleration to create an efficient, scalable, and interpretable solution for data visualization and quantitative analysis of embeddings.

Result: DiRe shows considerable promise in preserving both local and global structures compared to state-of-the-art UMAP and tSNE implementations.

Conclusion: The toolkit is suitable for a wide range of applications in machine learning, bio-informatics, and data science due to its improved structure preservation and computational efficiency.

Abstract: We propose a new dimensionality reduction toolkit designed to address some of the challenges faced by traditional methods like UMAP and tSNE such as loss of global structure and computational efficiency. Built on the JAX framework, DiRe leverages modern hardware acceleration to provide an efficient, scalable, and interpretable solution for visualizing complex data structures, and for quantitative analysis of lower-dimensional embeddings. The toolkit shows considerable promise in preserving both local and global structures within the data as compared to state-of-the-art UMAP and tSNE implementations. This makes it suitable for a wide range of applications in machine learning, bio-informatics, and data science.

Method: The authors provide theoretical justification for discrete-time Iterative Markovian Fitting (D-IMF) convergence to Schr"odinger Bridge in discrete spaces, and develop the practical CSBM algorithm based on this foundation.

Result: CSBM demonstrates strong performance through experiments with synthetic data and vector-quantized representations of images, providing a working solution for discrete Schr"odinger Bridge problems.

Conclusion: The paper establishes both theoretical and algorithmic foundations for applying Schr"odinger Bridge methods to discrete spaces, enabling new applications in domains like image processing, text generation, and molecular modeling.

Abstract: The Schr"odinger Bridge (SB) is a powerful framework for solving generative modeling tasks such as unpaired domain translation. Most SB-related research focuses on continuous data space $\mathbb{R}^{D}$ and leaves open theoretical and algorithmic questions about applying SB methods to discrete data, e.g, on finite spaces $\mathbb{S}^{D}$. Notable examples of such sets $\mathbb{S}$ are codebooks of vector-quantized (VQ) representations of modern autoencoders, tokens in texts, categories of atoms in molecules, etc. In this paper, we provide a theoretical and algorithmic foundation for solving SB in discrete spaces using the recently introduced Iterative Markovian Fitting (IMF) procedure. Specifically, we theoretically justify the convergence of discrete-time IMF (D-IMF) to SB in discrete spaces. This enables us to develop a practical computational algorithm for SB, which we call Categorical Schr"odinger Bridge Matching (CSBM). We show the performance of CSBM via a series of experiments with synthetic data and VQ representations of images. The code of CSBM is available at https://github.com/gregkseno/csbm.

Method: Defines a forward process from target distribution to known distribution (e.g., Gaussian), then learns a reverse Markov process using multiple engression models to reconstruct the target distribution step by step.

Result: Provides efficient discretization for diffusion models, establishes statistical error bounds, and demonstrates effectiveness on simulated and climate data for capturing complex distributions.

Conclusion: RML offers a flexible framework with advantages in estimation efficiency and forward process design, effectively addressing limitations of engression for complex distribution learning.

Abstract: Learning complex distributions is a fundamental challenge in contemporary applications. Shen and Meinshausen (2024) introduced engression, a generative approach based on scoring rules that maps noise (and covariates, if available) directly to data. While effective, engression can struggle with highly complex distributions, such as those encountered in image data. In this work, we propose reverse Markov learning (RML), a framework that defines a general forward process transitioning from the target distribution to a known distribution (e.g., Gaussian) and then learns a reverse Markov process using multiple engression models. This reverse process reconstructs the target distribution step by step. This framework accommodates general forward processes, allows for dimension reduction, and naturally discretizes the generative process. In the special case of diffusion-based forward processes, RML provides an efficient discretization strategy for both training and inference in diffusion models. We further introduce an alternating sampling scheme to enhance post-training performance. Our statistical analysis establishes error bounds for RML and elucidates its advantages in estimation efficiency and flexibility in forward process design. Empirical results on simulated and climate data corroborate the theoretical findings, demonstrating the effectiveness of RML in capturing complex distributions.

Method: Models control actions as dynamic, time-dependent variables in latent space using pre-trained encoder-decoder and state-dependent linear system, enabling local stability analysis through instantaneous growth prediction in action-norms before execution.

Result: SALSA-RL can be deployed non-invasively to assess local stability of actions from pretrained RL agents without performance degradation across diverse benchmark environments.

Conclusion: Provides a powerful tool for advancing interpretable analysis of action generation, enhancing design, analysis, and theoretical understanding of RL systems for safety-critical applications.

Abstract: Modern deep reinforcement learning (DRL) methods have made significant advances in handling continuous action spaces. However, real-world control systems–especially those requiring precise and reliable performance–often demand interpretability in the sense of a-priori assessments of agent behavior to identify safe or failure-prone interactions with environments. To address this limitation, we propose SALSA-RL (Stability Analysis in the Latent Space of Actions), a novel RL framework that models control actions as dynamic, time-dependent variables evolving within a latent space. By employing a pre-trained encoder-decoder and a state-dependent linear system, our approach enables interpretability through local stability analysis, where instantaneous growth in action-norms can be predicted before their execution. We demonstrate that SALSA-RL can be deployed in a non-invasive manner for assessing the local stability of actions from pretrained RL agents without compromising on performance across diverse benchmark environments. By enabling a more interpretable analysis of action generation, SALSA-RL provides a powerful tool for advancing the design, analysis, and theoretical understanding of RL systems.

Method: Hierarchical Refinement (HiRef) dynamically constructs multiscale partitions using low-rank OT subproblems, leveraging the co-clustering property of optimal low-rank coupling factors to find the bijective Monge map.

Result: HiRef achieves log-linear time and linear space complexity, enabling computation of bijective Monge maps for datasets with over a million points, scaling beyond Sinkhorn’s capabilities.

Conclusion: HiRef combines the efficiency of low-rank OT with the ability to compute exact bijective mappings, making optimal transport practical for large-scale problems requiring one-to-one correspondences.

Abstract: Optimal transport (OT) has enjoyed great success in machine learning as a principled way to align datasets via a least-cost correspondence, driven in large part by the runtime efficiency of the Sinkhorn algorithm (Cuturi, 2013). However, Sinkhorn has quadratic space and time complexity in the number of points, limiting scalability to larger datasets. Low-rank OT achieves linear complexity, but by definition, cannot compute a one-to-one correspondence between points. When the optimal transport problem is an assignment problem between datasets then an optimal mapping, known as the Monge map, is guaranteed to be a bijection. In this setting, we show that the factors of an optimal low-rank coupling co-cluster each point with its image under the Monge map. We leverage this invariant to derive an algorithm, Hierarchical Refinement (HiRef), that dynamically constructs a multiscale partition of each dataset using low-rank OT subproblems, culminating in the bijective Monge map. Hierarchical Refinement runs in log-linear time and linear space, retaining the advantages of low-rank OT while overcoming its limited resolution. We demonstrate the advantages of Hierarchical Refinement on several datasets, including ones containing over a million points, scaling full-rank OT to problems previously beyond Sinkhorn’s reach.

Method: Novel offline RL approach inspired by Seldonian optimization that uses pre-collected dataset, candidate policies, and teammate policy specifications to return policies with statistical guarantees without requiring further interactions or training.

Result: The algorithm consistently finds reliable policies in Ad Hoc Teamwork problems while improving sample efficiency compared to standard machine learning baselines.

Conclusion: The proposed method provides a statistically sound framework for offline RL in safety-critical multiagent applications, ensuring reliable performance with guaranteed behavioral properties in Ad Hoc Teamwork settings.

Abstract: Most offline RL algorithms return optimal policies but do not provide statistical guarantees on desirable behaviors. This could generate reliability issues in safety-critical applications, such as in some multiagent domains where agents, and possibly humans, need to interact to reach their goals without harming each other. In this work, we propose a novel offline RL approach, inspired by Seldonian optimization, which returns policies with good performance and statistically guaranteed properties with respect to predefined desirable behaviors. In particular, our focus is on Ad Hoc Teamwork settings, where agents must collaborate with new teammates without prior coordination. Our method requires only a pre-collected dataset, a set of candidate policies for our agent, and a specification about the possible policies followed by the other players – it does not require further interactions, training, or assumptions on the type and architecture of the policies. We test our algorithm in Ad Hoc Teamwork problems and show that it consistently finds reliable policies while improving sample efficiency with respect to standard ML baselines.

Method: Uses LauraGPT backbone with a small auto-regressive decoder-only model to generate initial discrete codec representations from continuous embeddings of mixture and reference speech, then refines with a one-step encoder-only model to add fine-grained details.

Result: Experimental results show promising performance, with ablation studies conducted on data scalability and encoder-only model contributions.

Conclusion: The proposed LauraTSE approach demonstrates effective target speaker extraction capabilities through its two-stage coarse-to-fine generation process.

Abstract: We propose LauraTSE, an Auto-Regressive Decoder-Only Language Model for Target Speaker Extraction built upon the LauraGPT backbone. LauraTSE employs a small-scale auto-regressive decoder-only language model that generates the initial layers of the target speech’s discrete codec representations from the continuous embeddings of both the mixture and reference speech. These outputs serve as coarse-grained predictions. To refine them, a one-step encoder-only language model reconstructs the full codec representation by integrating information from both the mixture and the reference speech, adding fine-grained details. Experimental results show that our approach can achieve promising performance. Additionally, we conduct ablation studies to investigate the data scalability and the contribution of the encoder-only model.

Method: Uses small auxiliary model to learn from labeled local data, then transfers knowledge from resource-strong clients to large model via on-device KD using unlabeled data.

Result: Extensive experiments show higher accuracy than state-of-the-art KD-based FL methods by effectively utilizing all edge device resources and unlabeled data.

Conclusion: Proposed on-device KD-based heterogeneous FL method overcomes limitations of server-side approaches and performs better with non-IID data distributions.

Abstract: Online Knowledge Distillation (KD) is recently highlighted to train large models in Federated Learning (FL) environments. Many existing studies adopt the logit ensemble method to perform KD on the server side. However, they often assume that unlabeled data collected at the edge is centralized on the server. Moreover, the logit ensemble method personalizes local models, which can degrade the quality of soft targets, especially when data is highly non-IID. To address these critical limitations,we propose a novel on-device KD-based heterogeneous FL method. Our approach leverages a small auxiliary model to learn from labeled local data. Subsequently, a subset of clients with strong system resources transfers knowledge to a large model through on-device KD using their unlabeled data. Our extensive experiments demonstrate that our on-device KD-based heterogeneous FL method effectively utilizes the system resources of all edge devices as well as the unlabeled data, resulting in higher accuracy compared to SOTA KD-based FL methods.

Method: Proposes MedSpaformer with sparse token-based dual-attention mechanism for global context modeling and token sparsification, multi-granularity cross-channel encoding for temporal dependencies, and adaptive label encoder for cross-dataset transfer learning.

Result: Outperforms 13 baselines across 7 medical datasets in supervised learning, excels in few-shot learning, and demonstrates zero-shot capability in both in-domain and cross-domain diagnostics.

Conclusion: MedSpaformer provides a robust and unified solution for medical time series classification that effectively handles variable input lengths, channel dimensions, and label scarcity through its transfer learning capabilities.

Abstract: Accurate medical time series (MedTS) classification is essential for effective clinical diagnosis, yet remains challenging due to complex multi-channel temporal dependencies, information redundancy, and label scarcity. While transformer-based models have shown promise in time series analysis, most are designed for forecasting tasks and fail to fully exploit the unique characteristics of MedTS. In this paper, we introduce MedSpaformer, a transformer-based framework tailored for MedTS classification. It incorporates a sparse token-based dual-attention mechanism that enables global context modeling and token sparsification, allowing dynamic feature refinement by focusing on informative tokens while reducing redundancy. This mechanism is integrated into a multi-granularity cross-channel encoding scheme to capture intra- and inter-granularity temporal dependencies and inter-channel correlations, enabling progressive refinement of task-relevant patterns in medical signals. The sparsification design allows our model to flexibly accommodate inputs with variable lengths and channel dimensions. We also introduce an adaptive label encoder to extract label semantics and address cross-dataset label space misalignment. Together, these components enhance the model’s transferability across heterogeneous medical datasets, which helps alleviate the challenge of label scarcity. Our model outperforms 13 baselines across 7 medical datasets under supervised learning. It also excels in few-shot learning and demonstrates zero-shot capability in both in-domain and cross-domain diagnostics. These results highlight MedSpaformer’s robustness and its potential as a unified solution for MedTS classification across diverse settings.

Method: Proposes training language models with generic inference-time objectives using k samples, focusing on pass@k and majority voting applications. Applied to reasoning datasets and code generation tasks.

Result: Shows performance trade-offs enabled by training with inference-time objectives. On code generation tasks, significantly improves pass@k objectives compared to baseline methods.

Conclusion: Explicitly optimizing for inference time performance during training is beneficial and can lead to significant improvements in sampling-based evaluation metrics.

Abstract: In this work, we investigate the merits of explicitly optimizing for inference time algorithmic performance during model training. We show how optimizing for inference time performance can improve overall model efficacy. We consider generic inference time objectives with $k$ samples, with a focus on pass@$k$ and majority voting as two main applications. With language model training on reasoning datasets, we showcase the performance trade-off enabled by training with such objectives. When training on code generation tasks, we show that the approach significantly improves pass@$k$ objectives compared to the baseline method.

Method: Proposes a simplified reward design focused primarily on route completion, with infractions penalized by episode termination or multiplicative reduction of route completion. Uses PPO with large mini-batch sizes enabled by distributed data parallelism.

Result: Achieves 64 DS on CARLA longest6 v2 benchmark (outperforming other RL methods), 91.3 in non-reactive and 90.6 in reactive traffic on nuPlan Val14 benchmark, while being an order of magnitude faster than prior work.

Conclusion: Simple reward designs based on intuitive objectives like route completion enable better scaling and performance in RL for autonomous driving compared to complex multi-term reward structures.

Abstract: We investigate reinforcement learning (RL) for privileged planning in autonomous driving. State-of-the-art approaches for this task are rule-based, but these methods do not scale to the long tail. RL, on the other hand, is scalable and does not suffer from compounding errors like imitation learning. Contemporary RL approaches for driving use complex shaped rewards that sum multiple individual rewards, \eg~progress, position, or orientation rewards. We show that PPO fails to optimize a popular version of these rewards when the mini-batch size is increased, which limits the scalability of these approaches. Instead, we propose a new reward design based primarily on optimizing a single intuitive reward term: route completion. Infractions are penalized by terminating the episode or multiplicatively reducing route completion. We find that PPO scales well with higher mini-batch sizes when trained with our simple reward, even improving performance. Training with large mini-batch sizes enables efficient scaling via distributed data parallelism. We scale PPO to 300M samples in CARLA and 500M samples in nuPlan with a single 8-GPU node. The resulting model achieves 64 DS on the CARLA longest6 v2 benchmark, outperforming other RL methods with more complex rewards by a large margin. Requiring only minimal adaptations from its use in CARLA, the same method is the best learning-based approach on nuPlan. It scores 91.3 in non-reactive and 90.6 in reactive traffic on the Val14 benchmark while being an order of magnitude faster than prior work.

Method: NoProp uses diffusion and flow matching inspiration where each block independently learns to denoise a noisy target using only local targets and back-propagation within the block. Representations are fixed beforehand as noised versions of targets.

Result: Demonstrated effectiveness on MNIST, CIFAR-10, and CIFAR-100 image classification benchmarks. NoProp proves to be a viable learning algorithm that is easy to use and computationally efficient.

Conclusion: NoProp represents a new family of learning methods that alters credit assignment, enables more efficient distributed learning, and departs from traditional hierarchical representation learning paradigms.

Abstract: The canonical deep learning approach for learning requires computing a gradient term at each block by back-propagating the error signal from the output towards each learnable parameter. Given the stacked structure of neural networks, where each block builds on the representation of the block below, this approach leads to hierarchical representations. More abstract features live on the top blocks of the model, while features on lower blocks are expected to be less abstract. In contrast to this, we introduce a new learning method named NoProp, which does not rely on either forward or backwards propagation across the entire network. Instead, NoProp takes inspiration from diffusion and flow matching methods, where each block independently learns to denoise a noisy target using only local targets and back-propagation within the block. We believe this work takes a first step towards introducing a new family of learning methods that does not learn hierarchical representations – at least not in the usual sense. NoProp needs to fix the representation at each block beforehand to a noised version of the target, learning a local denoising process that can then be exploited at inference. We demonstrate the effectiveness of our method on MNIST, CIFAR-10, and CIFAR-100 image classification benchmarks. Our results show that NoProp is a viable learning algorithm, is easy to use and computationally efficient. By departing from the traditional learning paradigm which requires back-propagating a global error signal, NoProp alters how credit assignment is done within the network, enabling more efficient distributed learning as well as potentially impacting other characteristics of the learning process.

Method: Proposes Z-Score Filtered SAM that applies Z-score based filtering to gradients in each layer, retaining only top percentile components with largest absolute Z-scores to focus perturbation on most significant directions.

Result: Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with ResNet, VGG, and Vision Transformers show consistent test accuracy improvements over SAM and its variants.

Conclusion: Selective gradient filtering using Z-scores effectively refines the search toward flatter minima and improves generalization performance compared to using all gradient components.

Abstract: Deep neural networks achieve high performance across many domains but can still face challenges in generalization when optimization is influenced by small or noisy gradient components. Sharpness-Aware Minimization improves generalization by perturbing parameters toward directions of high curvature, but it uses the entire gradient vector, which means that small or noisy components may affect the ascent step and cause the optimizer to miss optimal solutions. We propose Z-Score Filtered Sharpness-Aware Minimization, which applies Z-score based filtering to gradients in each layer. Instead of using all gradient components, a mask is constructed to retain only the top percentile with the largest absolute Z-scores. The percentile threshold $Q_p$ determines how many components are kept, so that the ascent step focuses on directions that stand out most compared to the average of the layer. This selective perturbation refines the search toward flatter minima while reducing the influence of less significant gradients. Experiments on CIFAR-10, CIFAR-100, and Tiny-ImageNet with architectures including ResNet, VGG, and Vision Transformers show that the proposed method consistently improves test accuracy compared to Sharpness-Aware Minimization and its variants.

Method: Uses unified class prototypes updated only with clean data, dual-branch training (clean data for prototypes, adversarial inputs for feature extractor), and composite loss with positive-prototype alignment, negative-prototype repulsion, and consistency regularization.

Result: Improves clean accuracy over state-of-the-art defenses on CIFAR-10/100 and SVHN, maintains competitive robustness against FGSM, PGD, C&W, and AutoAttack attacks, and achieves highest average accuracy on CIFAR-10-C corruptions.

Conclusion: Adv-DPNP effectively combines prototype learning with adversarial training to enhance both clean accuracy and robustness while maintaining compact, well-separated latent representations.

Abstract: Despite the advantages of discriminative prototype-based methods, their role in adversarial robustness remains underexplored. Meanwhile, current adversarial training methods predominantly focus on robustness against adversarial attacks without explicitly leveraging geometric structures in the latent space, usually resulting in reduced accuracy on the original clean data. We propose a novel framework named Adversarially trained Deep Positive-Negative Prototypes (Adv-DPNP), which integrates discriminative prototype-based learning with adversarial training. Adv-DPNP uses unified class prototypes that serve as both classifier weights and robust anchors in the latent space. Moreover, a novel dual-branch training mechanism maintains stable prototypes by updating them exclusively with clean data, while the feature extractor is trained on both clean and adversarial inputs to increase invariance to adversarial perturbations. In addition, we use a composite loss that combines positive-prototype alignment, negative-prototype repulsion, and consistency regularization to further enhance discrimination, adversarial robustness, and clean accuracy. Extensive experiments on standard benchmarks (CIFAR-10/100 and SVHN) confirm that Adv-DPNP improves clean accuracy over state-of-the-art defenses and baseline methods, while maintaining competitive or superior robustness under a suite of widely used attacks, including FGSM, PGD, C&W, and AutoAttack. We also evaluate robustness to common corruptions on CIFAR-10-C, where Adv-DPNP achieves the highest average accuracy across severities and corruption types. Additionally, we provide an in-depth analysis of the discriminative quality of the learned feature representations, highlighting the effectiveness of Adv-DPNP in maintaining compactness and clear separation in the latent space.

Method: Proposes CAOTE, a simple eviction criterion based on cached tokens’ contributions to attention outputs. It integrates attention scores and value vectors in closed-form to optimize eviction error, and can work as a meta-heuristic with any token eviction method.

Result: CAOTE consistently improves accuracies on downstream tasks when combined with state-of-the-art attention score-based methods, demonstrating the importance of leveraging value information during token eviction.

Conclusion: The proposed CAOTE method effectively addresses limitations of attention-only eviction by incorporating value vector information, providing a flexible and effective solution for token eviction in memory-constrained environments.

Abstract: While long context support of large language models has extended their abilities, it also incurs challenges in memory and compute which becomes crucial bottlenecks in resource-restricted devices. Token eviction, a widely adopted post-training methodology designed to alleviate the bottlenecks by evicting less important tokens from the cache, typically uses attention scores as proxy metrics for token importance. However, one major limitation of attention score as a token-wise importance metrics is that it lacks the information about contribution of tokens to the attention output. In this paper, we propose a simple eviction criterion based on the contribution of cached tokens to attention outputs. Our method, CAOTE, optimizes for eviction error due to token eviction, by seamlessly integrating attention scores and value vectors. This is the first method which uses value tokens on top of attention-based eviction scores in closed-form. Additionally, CAOTE can act as a meta-heuristic method with flexible usage with any token eviction method. We show that CAOTE, when combined with the state-of-the-art attention score-based methods, always improves accuracies on the downstream task, indicating the importance of leveraging information from values during token eviction process.