Method: Uses collaborative agent team: analyzing agent identifies error patterns via topic modeling, generation agent revises prompts to address deficiencies, guided by genetic algorithm framework with crossover and mutation operations.

Result: Superior performance against human-engineered prompts and state-of-the-art methods across logical reasoning, quantitative reasoning, commonsense, and ethical decision-making tasks on public benchmarks.

Conclusion: GreenTEA effectively balances exploration and exploitation for automatic prompt optimization, demonstrating significant improvements over existing approaches through its agentic workflow and genetic algorithm framework.

Abstract: High-quality prompts are crucial for Large Language Models (LLMs) to achieve exceptional performance. However, manually crafting effective prompts is labor-intensive and demands significant domain expertise, limiting its scalability. Existing automatic prompt optimization methods either extensively explore new prompt candidates, incurring high computational costs due to inefficient searches within a large solution space, or overly exploit feedback on existing prompts, risking suboptimal optimization because of the complex prompt landscape. To address these challenges, we introduce GreenTEA, an agentic LLM workflow for automatic prompt optimization that balances candidate exploration and knowledge exploitation. It leverages a collaborative team of agents to iteratively refine prompts based on feedback from error samples. An analyzing agent identifies common error patterns resulting from the current prompt via topic modeling, and a generation agent revises the prompt to directly address these key deficiencies. This refinement process is guided by a genetic algorithm framework, which simulates natural selection by evolving candidate prompts through operations such as crossover and mutation to progressively optimize model performance. Extensive numerical experiments conducted on public benchmark datasets suggest the superior performance of GreenTEA against human-engineered prompts and existing state-of-the-arts for automatic prompt optimization, covering logical and quantitative reasoning, commonsense, and ethical decision-making.

Method: Meta-cognitive layer analyzes query complexity through correlation strength, domain boundaries, stakeholder multiplicity, and uncertainty levels to determine optimal reasoning strategy.

Result: 34% computational cost reduction vs uniform deep reasoning, 23% improvement in consistency, 18% better accuracy on expert-level evaluations in professional judgment tasks.

Conclusion: Successfully bridges cognitive science principles with AI design, providing adaptive reasoning that balances performance and efficiency in LLMs.

Abstract: Large Language Models (LLMs) face a fundamental challenge in deciding when to rely on rapid, intuitive responses versus engaging in slower, more deliberate reasoning. Inspired by Daniel Kahneman’s dual-process theory and his insights on human cognitive biases, we propose a novel Cognitive Decision Routing (CDR) framework that dynamically determines the appropriate reasoning strategy based on query characteristics. Our approach addresses the current limitations where models either apply uniform reasoning depth or rely on computationally expensive methods for all queries. We introduce a meta-cognitive layer that analyzes query complexity through multiple dimensions: correlation strength between given information and required conclusions, domain boundary crossings, stakeholder multiplicity, and uncertainty levels. Through extensive experiments on diverse reasoning tasks, we demonstrate that CDR achieves superior performance while reducing computational costs by 34% compared to uniform deep reasoning approaches. Our framework shows particular strength in professional judgment tasks, achieving 23% improvement in consistency and 18% better accuracy on expert-level evaluations. This work bridges cognitive science principles with practical AI system design, offering a principled approach to adaptive reasoning in LLMs.

Method: The authors conduct a systematic survey of existing literature, presenting a unified view of verifier training methods, types (prompt-based, fine-tuned discriminative/generative models), and their applications in verifying reasoning processes and outcomes.

Result: The survey provides comprehensive coverage of diverse verification approaches used in test-time scaling, categorizing them and discussing their training mechanisms and utility.

Conclusion: This survey fills a gap in the literature by systematically organizing and analyzing verifier approaches for test-time scaling, offering a unified framework for understanding how verifiers can enhance LLM performance through parameter-free scaling at inference time.

Abstract: Test-time scaling (TTS) has emerged as a new frontier for scaling the performance of Large Language Models. In test-time scaling, by using more computational resources during inference, LLMs can improve their reasoning process and task performance. Several approaches have emerged for TTS such as distilling reasoning traces from another model or exploring the vast decoding search space by employing a verifier. The verifiers serve as reward models that help score the candidate outputs from the decoding process to diligently explore the vast solution space and select the best outcome. This paradigm commonly termed has emerged as a superior approach owing to parameter free scaling at inference time and high performance gains. The verifiers could be prompt-based, fine-tuned as a discriminative or generative model to verify process paths, outcomes or both. Despite their widespread adoption, there is no detailed collection, clear categorization and discussion of diverse verification approaches and their training mechanisms. In this survey, we cover the diverse approaches in the literature and present a unified view of verifier training, types and their utility in test-time scaling. Our repository can be found at https://github.com/elixir-research-group/Verifierstesttimescaling.github.io.

Method: Supervised fine-tuning of LLaMA and Qwen models on four types of reasoning traces (DeepSeek R1 traces, LLM-generated summaries, post-hoc explanations, and algorithmically generated traces) in Open Book QA domain, plus human study with 100 participants rating interpretability.

Result: Fine-tuning on DeepSeek R1 traces yielded strongest performance but these traces were rated as least interpretable by human participants, showing a mismatch between performance and interpretability.

Conclusion: Intermediate reasoning tokens can be decoupled from end user interpretability - interpretability is not necessary for enhancing LLM task performance.

Abstract: Recent progress in reasoning-oriented Large Language Models (LLMs) has been driven by introducing Chain-of-Thought (CoT) traces, where models generate intermediate reasoning traces before producing an answer. These traces, as in DeepSeek R1, are not only used to guide inference but also serve as supervision signals for distillation into smaller models. A common but often implicit assumption is that CoT traces should be semantically meaningful and interpretable to the end user. While recent research questions the need for semantic nature of these traces, in this paper, we ask: ``\textit{Must CoT reasoning traces be interpretable to enhance LLM task performance?}" We investigate this question in the Open Book Question-Answering domain by supervised fine-tuning LLaMA and Qwen models on four types of reasoning traces: (1) DeepSeek R1 traces, (2) LLM-generated summaries of R1 traces, (3) LLM-generated post-hoc explanations of R1 traces, and (4) algorithmically generated verifiably correct traces. To quantify the trade-off between interpretability and performance, we further conduct a human-subject study with 100 participants rating the interpretability of each trace type. Our results reveal a striking mismatch: while fine-tuning on R1 traces yields the strongest performance, participants judged these traces to be the least interpretable. These findings suggest that it is useful to decouple intermediate tokens from end user interpretability.

Method: QueryBandits framework uses bandit algorithms (Thompson Sampling) to design query rewrite strategies that maximize rewards based on 17 linguistic features of input queries. It explores different rewrite strategies to minimize hallucination propensity.

Result: Achieved 87.5% win rate over no-rewrite baseline, outperformed zero-shot static prompting by 42.6-60.3%. Found static rewrites can have higher cumulative regret than baseline, and no single rewrite strategy is optimal for all queries.

Conclusion: QueryBandits effectively mitigates hallucinations through query rewriting interventions, demonstrating that guided rewriting via semantic feature exploitation can significantly shift output behavior without retraining or gradient-based adaptation.

Abstract: Advanced reasoning capabilities in Large Language Models (LLMs) have caused higher hallucination prevalence; yet most mitigation work focuses on after-the-fact filtering rather than shaping the queries that trigger them. We introduce QueryBandits, a bandit framework that designs rewrite strategies to maximize a reward model, that encapsulates hallucination propensity based upon the sensitivities of 17 linguistic features of the input query-and therefore, proactively steer LLMs away from generating hallucinations. Across 13 diverse QA benchmarks and 1,050 lexically perturbed queries per dataset, our top contextual QueryBandit (Thompson Sampling) achieves an 87.5% win rate over a no-rewrite baseline and also outperforms zero-shot static prompting (“paraphrase” or “expand”) by 42.6% and 60.3% respectively. Therefore, we empirically substantiate the effectiveness of QueryBandits in mitigating hallucination via the intervention that takes the form of a query rewrite. Interestingly, certain static prompting strategies, which constitute a considerable number of current query rewriting literature, have a higher cumulative regret than the no-rewrite baseline, signifying that static rewrites can worsen hallucination. Moreover, we discover that the converged per-arm regression feature weight vectors substantiate that there is no single rewrite strategy optimal for all queries. In this context, guided rewriting via exploiting semantic features with QueryBandits can induce significant shifts in output behavior through forward-pass mechanisms, bypassing the need for retraining or gradient-based adaptation.

Method: Used the Maze Test to evaluate 12 leading LLMs across zero-shot, one-shot, and few-shot learning scenarios, assessing 13 consciousness characteristics synthesized from consciousness theories.

Result: Reasoning-capable LLMs outperformed standard versions, with Gemini 2.0 Pro achieving 52.9% Complete Path Accuracy and DeepSeek-R1 reaching 80.5% Partial Path Accuracy. The performance gap indicates LLMs struggle to maintain coherent self-models.

Conclusion: While LLMs demonstrate progress in consciousness-related behaviors through reasoning mechanisms, they lack the integrated, persistent self-awareness that characterizes true consciousness.

Abstract: We investigate consciousness-like behaviors in Large Language Models (LLMs) using the Maze Test, challenging models to navigate mazes from a first-person perspective. This test simultaneously probes spatial awareness, perspective-taking, goal-directed behavior, and temporal sequencing-key consciousness-associated characteristics. After synthesizing consciousness theories into 13 essential characteristics, we evaluated 12 leading LLMs across zero-shot, one-shot, and few-shot learning scenarios. Results showed reasoning-capable LLMs consistently outperforming standard versions, with Gemini 2.0 Pro achieving 52.9% Complete Path Accuracy and DeepSeek-R1 reaching 80.5% Partial Path Accuracy. The gap between these metrics indicates LLMs struggle to maintain coherent self-models throughout solutions – a fundamental consciousness aspect. While LLMs show progress in consciousness-related behaviors through reasoning mechanisms, they lack the integrated, persistent self-awareness characteristic of consciousness.

Method: Uses self-knowledge distillation with integrated similarity scores (weighted sum of dense and sparse similarities) as shared teacher signal. Fine-tunes only the final layer of dense encoder and sparse projection head for efficiency.

Result: Outperforms existing sparse baselines and achieves performance comparable to or surpassing dense counterparts on MSCOCO and Flickr30k datasets.

Conclusion: The framework enables effective bi-directional learning between dense and sparse representations, providing sparse retrievers with performance approaching dense models while retaining sparse model advantages like interpretability and efficiency.

Abstract: Vision-Language Pretrained (VLP) models have achieved impressive performance on multimodal tasks, including text-image retrieval, based on dense representations. Meanwhile, Learned Sparse Retrieval (LSR) has gained traction in text-only settings due to its interpretability and efficiency with fast term-based lookup via inverted indexes. Inspired by these advantages, recent work has extended LSR to the multimodal domain. However, these methods often rely on computationally expensive contrastive pre-training, or distillation from a frozen dense model, which limits the potential for mutual enhancement. To address these limitations, we propose a simple yet effective framework that enables bi-directional learning between dense and sparse representations through Self-Knowledge Distillation. This bi-directional learning is achieved using an integrated similarity score-a weighted sum of dense and sparse similarities-which serves as a shared teacher signal for both representations. To ensure efficiency, we fine-tune the final layer of the dense encoder and the sparse projection head, enabling easy adaptation of any existing VLP model. Experiments on MSCOCO and Flickr30k demonstrate that our sparse retriever not only outperforms existing sparse baselines, but also achieves performance comparable to-or even surpassing-its dense counterparts, while retaining the benefits of sparse models.

Method: ERP builds upon CoT by adding three components: incorrect answer generation, error recognition, and correct answer production. The method uses automated ERP generation to create error outlines that help models identify error types and problematic steps.

Result: ERP serves as a versatile supplement to conventional CoT, contributing to more robust reasoning abilities and increased interpretability in how models reach their errors.

Conclusion: Error Reflection Prompting successfully enhances language model reasoning by integrating error recognition and correction into the reasoning chain, improving scalability and reliability of problem-solving processes.

Abstract: Prompting methods for language models, such as Chain-of-thought (CoT), present intuitive step-by-step processes for problem solving. These methodologies aim to equip models with a better understanding of the correct procedures for addressing a given task. Despite these advancements, CoT lacks the ability of reflection and error correction, potentially causing a model to perpetuate mistakes and errors. Therefore, inspired by the human ability for said tasks, we propose Error Reflection Prompting (ERP) to further enhance reasoning in language models. Building upon CoT, ERP is a method comprised of an incorrect answer, error recognition, and a correct answer. This process enables the model to recognize types of errors and the steps that lead to incorrect answers, allowing the model to better discern which steps to avoid and which to take. The model is able to generate the error outlines itself with automated ERP generation, allowing for error recognition and correction to be integrated into the reasoning chain and produce scalability and reliability in the process. The results demonstrate that ERP serves as a versatile supplement to conventional CoT, ultimately contributing to more robust and capable reasoning abilities along with increased interpretability in how models ultimately reach their errors.

Method: GAICo provides a unified, extensible framework with reference-based metrics for unstructured text, structured data formats, and multimedia (images, audio), featuring both high-level API for end-to-end analysis and direct metric access for granular control.

Result: The tool has been downloaded over 13K times since its PyPI release in June 2025, demonstrating growing community adoption. A case study shows its utility in evaluating and debugging complex multi-modal AI Travel Assistant pipelines.

Conclusion: GAICo empowers AI researchers and developers to efficiently assess system performance, make evaluation reproducible, improve development velocity, and build more trustworthy AI systems, enabling faster and safer AI deployment.

Abstract: The rapid proliferation of Generative AI (GenAI) into diverse, high-stakes domains necessitates robust and reproducible evaluation methods. However, practitioners often resort to ad-hoc, non-standardized scripts, as common metrics are often unsuitable for specialized, structured outputs (e.g., automated plans, time-series) or holistic comparison across modalities (e.g., text, audio, and image). This fragmentation hinders comparability and slows AI system development. To address this challenge, we present GAICo (Generative AI Comparator): a deployed, open-source Python library that streamlines and standardizes GenAI output comparison. GAICo provides a unified, extensible framework supporting a comprehensive suite of reference-based metrics for unstructured text, specialized structured data formats, and multimedia (images, audio). Its architecture features a high-level API for rapid, end-to-end analysis, from multi-model comparison to visualization and reporting, alongside direct metric access for granular control. We demonstrate GAICo’s utility through a detailed case study evaluating and debugging complex, multi-modal AI Travel Assistant pipelines. GAICo empowers AI researchers and developers to efficiently assess system performance, make evaluation reproducible, improve development velocity, and ultimately build more trustworthy AI systems, aligning with the goal of moving faster and safer in AI deployment. Since its release on PyPI in Jun 2025, the tool has been downloaded over 13K times, across versions, by Aug 2025, demonstrating growing community interest.

Method: Disentangled MAD into Majority Voting and inter-agent Debate components, conducted experiments across 7 NLP benchmarks, proposed theoretical framework modeling debate as stochastic process, and tested targeted interventions.

Result: Majority Voting alone accounts for most performance gains typically attributed to MAD. Debate induces a martingale over belief trajectories and doesn’t improve expected correctness without interventions.

Conclusion: While MAD has potential, simple ensembling methods remain strong and more reliable alternatives in many practical settings. Targeted interventions can meaningfully enhance debate effectiveness.

Abstract: Multi-Agent Debate~(MAD) has emerged as a promising paradigm for improving the performance of large language models through collaborative reasoning. Despite recent advances, the key factors driving MAD’s effectiveness remain unclear. In this work, we disentangle MAD into two key components–Majority Voting and inter-agent Debate–and assess their respective contributions. Through extensive experiments across seven NLP benchmarks, we find that Majority Voting alone accounts for most of the performance gains typically attributed to MAD. To explain this, we propose a theoretical framework that models debate as a stochastic process. We prove that it induces a martingale over agents’ belief trajectories, implying that debate alone does not improve expected correctness. Guided by these insights, we demonstrate that targeted interventions, by biasing the belief update toward correction, can meaningfully enhance debate effectiveness. Overall, our findings suggest that while MAD has potential, simple ensembling methods remain strong and more reliable alternatives in many practical settings. Code is released in https://github.com/deeplearning-wisc/debate-or-vote.

Method: Evaluated 22 methods with 40 variants across TREC DL19, DL20, BEIR benchmarks and a novel dataset for unseen queries. Analyzed effects of training data overlap, model architecture, and computational efficiency through controlled comparisons.

Result: LLM-based rerankers demonstrate superior performance on familiar queries but show varying generalization ability to novel queries. Lightweight models offer comparable efficiency. Query novelty significantly impacts reranking effectiveness.

Conclusion: Existing reranking approaches have limitations in handling novel queries, with performance disparities between LLM-based and lightweight methods depending on query familiarity and training data overlap.

Abstract: In this work, we present a systematic and comprehensive empirical evaluation of state-of-the-art reranking methods, encompassing large language model (LLM)-based, lightweight contextual, and zero-shot approaches, with respect to their performance in information retrieval tasks. We evaluate in total 22 methods, including 40 variants (depending on used LLM) across several established benchmarks, including TREC DL19, DL20, and BEIR, as well as a novel dataset designed to test queries unseen by pretrained models. Our primary goal is to determine, through controlled and fair comparisons, whether a performance disparity exists between LLM-based rerankers and their lightweight counterparts, particularly on novel queries, and to elucidate the underlying causes of any observed differences. To disentangle confounding factors, we analyze the effects of training data overlap, model architecture, and computational efficiency on reranking performance. Our findings indicate that while LLM-based rerankers demonstrate superior performance on familiar queries, their generalization ability to novel queries varies, with lightweight models offering comparable efficiency. We further identify that the novelty of queries significantly impacts reranking effectiveness, highlighting limitations in existing approaches. https://github.com/DataScienceUIBK/llm-reranking-generalization-study

Method: Developed a novel multimodal framework that perturbs cultural identity and evaluates 5 contemporary VLMs on story generation tasks, using cross-modal evaluation with visual-semantic similarity metrics and human assessments.

Result: VLMs show significant cultural adaptation with rich culturally-specific vocabulary, but cultural competence varies dramatically across architectures, some models exhibit inverse cultural alignment, and automated metrics show architectural bias contradicting human assessments. Visual-cultural understanding remains limited despite detectable cultural outputs.

Conclusion: The study establishes both the promise and challenges of cultural competence in multimodal AI, highlighting the need for improved cultural awareness in VLMs and releasing codebase and data for further research.

Abstract: As Vision-Language Models (VLMs) achieve widespread deployment across diverse cultural contexts, ensuring their cultural competence becomes critical for responsible AI systems. While prior work has evaluated cultural awareness in text-only models and VLM object recognition tasks, no research has systematically assessed how VLMs adapt outputs when cultural identity cues are embedded in both textual prompts and visual inputs during generative tasks. We present the first comprehensive evaluation of VLM cultural competence through multimodal story generation, developing a novel multimodal framework that perturbs cultural identity and evaluates 5 contemporary VLMs on a downstream task: story generation. Our analysis reveals significant cultural adaptation capabilities, with rich culturally-specific vocabulary spanning names, familial terms, and geographic markers. However, we uncover concerning limitations: cultural competence varies dramatically across architectures, some models exhibit inverse cultural alignment, and automated metrics show architectural bias contradicting human assessments. Cross-modal evaluation shows that culturally distinct outputs are indeed detectable through visual-semantic similarity (28.7% within-nationality vs. 0.2% cross-nationality recall), yet visual-cultural understanding remains limited. In essence, we establish the promise and challenges of cultural competence in multimodal AI. We publicly release our codebase and data: https://github.com/ArkaMukherjee0/mmCultural

Method: Created curated dataset via text-to-speech to simulate diverse emotional states, proposed Cross-turn Emotion Reasoning Score to assess emotion transitions in multi-turn dialogues, evaluated seven dialogue systems using continuous, categorical, and perceptual metrics.

Result: The framework effectively detects emotional inconsistencies in dialogue systems, providing insights for improvement.

Conclusion: The released systematic evaluation benchmark aims to advance emotion-aware spoken dialogue modeling for more natural and adaptive interactions.

Abstract: Speech emotions play a crucial role in human-computer interaction, shaping engagement and context-aware communication. Despite recent advances in spoken dialogue systems, a holistic system for evaluating emotional reasoning is still lacking. To address this, we introduce EMO-Reasoning, a benchmark for assessing emotional coherence in dialogue systems. It leverages a curated dataset generated via text-to-speech to simulate diverse emotional states, overcoming the scarcity of emotional speech data. We further propose the Cross-turn Emotion Reasoning Score to assess the emotion transitions in multi-turn dialogues. Evaluating seven dialogue systems through continuous, categorical, and perceptual metrics, we show that our framework effectively detects emotional inconsistencies, providing insights for improving current dialogue systems. By releasing a systematic evaluation benchmark, we aim to advance emotion-aware spoken dialogue modeling toward more natural and adaptive interactions.

Method: MH-COPILOT system using reinforcement learning with contextual attribute-span identification, intensity classification, controlled question generation via hierarchical taxonomy (CueTaxo), and verifier for reward modeling.

Result: Significant improvements in attribute elicitation and user engagement across four language models, validated by human evaluation in real-world OMHC settings.

Conclusion: The framework effectively identifies missing support attributes and generates targeted prompts to elicit crucial information, enhancing engagement and support quality in mental health communities.

Abstract: Online Mental Health Communities (OMHCs) provide crucial peer and expert support, yet many posts remain unanswered due to missing support attributes that signal the need for help. We present a novel framework that identifies these gaps and prompts users to enrich their posts, thereby improving engagement. To support this, we introduce REDDME, a new dataset of 4,760 posts from mental health subreddits annotated for the span and intensity of three key support attributes: event what happened?, effect what did the user experience?, and requirement what support they need?. Next, we devise a hierarchical taxonomy, CueTaxo, of support attributes for controlled question generation. Further, we propose MH-COPILOT, a reinforcement learning-based system that integrates (a) contextual attribute-span identification, (b) support attribute intensity classification, (c) controlled question generation via a hierarchical taxonomy, and (d) a verifier for reward modeling. Our model dynamically assesses posts for the presence/absence of support attributes, and generates targeted prompts to elicit missing information. Empirical results across four notable language models demonstrate significant improvements in attribute elicitation and user engagement. A human evaluation further validates the model’s effectiveness in real-world OMHC settings.

Method: Uses TTS to synthesize diverse speech samples for target rare words, extracts multiple pronunciation variants using Whisper, compiles variants into prefix-trie for shallow-fusion beam-search decoding, then maps recognized variants back to original words.

Result: 42% reduction in biased WER on Librispeech test-clean and 43% on test-other, while keeping unbiased WER essentially unchanged.

Conclusion: The synthesis-driven multi-pronunciation approach effectively improves contextual ASR performance for rare words without compromising general recognition accuracy.

Abstract: Contextual automatic speech recognition (ASR) systems allow for recognizing out-of-vocabulary (OOV) words, such as named entities or rare words. However, it remains challenging due to limited training data and ambiguous or inconsistent pronunciations. In this paper, we propose a synthesis-driven multi-pronunciation contextual biasing method that performs zero-shot contextual ASR on a pretrained Whisper model. Specifically, we leverage text-to-speech (TTS) systems to synthesize diverse speech samples containing each target rare word, and then use the pretrained Whisper model to extract multiple predicted pronunciation variants. These variant token sequences are compiled into a prefix-trie, which assigns rewards to beam hypotheses in a shallow-fusion manner during beam-search decoding. After which, any recognized variant is mapped back to the original rare word in the final transcription. The evaluation results on the Librispeech dataset show that our method reduces biased word error rate (WER) by 42% on test-clean and 43% on test-other while maintaining unbiased WER essentially unchanged.

Method: Combines multi-prototype modeling to capture semantic variability, cosine-contrastive learning for interclass separation, and Reptile meta-learning for quick adaptation. Uses lightweight fastText + BiLSTM encoder.

Result: Achieves macro-F1 score close to BERT baselines (99% of BERT performance), remains stable with 30% label budget, and only drops 7.8% F1 when expanding from 19 to 50 categories, outperforming SpanProto and CONTaiNER.

Conclusion: ReProCon demonstrates state-of-the-art performance in resource-limited settings with effective handling of class imbalance through multi-prototype modeling and contrastive learning, making it suitable for biomedical applications despite some label ambiguity challenges.

Abstract: Named Entity Recognition (NER) in biomedical domains faces challenges due to data scarcity and imbalanced label distributions, especially with fine-grained entity types. We propose ReProCon, a novel few-shot NER framework that combines multi-prototype modeling, cosine-contrastive learning, and Reptile meta-learning to tackle these issues. By representing each category with multiple prototypes, ReProCon captures semantic variability, such as synonyms and contextual differences, while a cosine-contrastive objective ensures strong interclass separation. Reptile meta-updates enable quick adaptation with little data. Using a lightweight fastText + BiLSTM encoder with much lower memory usage, ReProCon achieves a macro-$F_1$ score close to BERT-based baselines (around 99 percent of BERT performance). The model remains stable with a label budget of 30 percent and only drops 7.8 percent in $F_1$ when expanding from 19 to 50 categories, outperforming baselines such as SpanProto and CONTaiNER, which see 10 to 32 percent degradation in Few-NERD. Ablation studies highlight the importance of multi-prototype modeling and contrastive learning in managing class imbalance. Despite difficulties with label ambiguity, ReProCon demonstrates state-of-the-art performance in resource-limited settings, making it suitable for biomedical applications.

Method: Used behavioral probing with contextual embeddings and meta-linguistic probing with prompts to distinguish implicit vs explicit knowledge, plus simulated hypothesis testing of false constructions.

Result: Models do hallucinate constructions, and simulated hypothesis testing showed high accuracy that would overwhelmingly confirm false hypotheses about these non-existent structures.

Conclusion: Construction probing methods suffer from confirmation bias, raising concerns about what unknown and incorrect syntactic knowledge LLMs possess.

Abstract: This paper investigates false positive constructions: grammatical structures which an LLM hallucinates as distinct constructions but which human introspection does not support. Both a behavioural probing task using contextual embeddings and a meta-linguistic probing task using prompts are included, allowing us to distinguish between implicit and explicit linguistic knowledge. Both methods reveal that models do indeed hallucinate constructions. We then simulate hypothesis testing to determine what would have happened if a linguist had falsely hypothesized that these hallucinated constructions do exist. The high accuracy obtained shows that such false hypotheses would have been overwhelmingly confirmed. This suggests that construction probing methods suffer from a confirmation bias and raises the issue of what unknown and incorrect syntactic knowledge these models also possess.

Method: Proposes a structured and robust claim verification framework that reasons through presupposition-free, decomposed questions.

Result: Extensive experiments show state-of-the-art models remain susceptible to prompt variance and presupposition. The method consistently mitigates these issues with 2-5% improvement.

Conclusion: Prompt sensitivity remains a persistent issue in LLMs, and the proposed structured framework effectively addresses both prompt variance and presupposition problems in claim verification.

Abstract: Prior work has shown that presupposition in generated questions can introduce unverified assumptions, leading to inconsistencies in claim verification. Additionally, prompt sensitivity remains a significant challenge for large language models (LLMs), resulting in performance variance as high as 3-6%. While recent advancements have reduced this gap, our study demonstrates that prompt sensitivity remains a persistent issue. To address this, we propose a structured and robust claim verification framework that reasons through presupposition-free, decomposed questions. Extensive experiments across multiple prompts, datasets, and LLMs reveal that even state-of-the-art models remain susceptible to prompt variance and presupposition. Our method consistently mitigates these issues, achieving up to a 2-5% improvement.

Method: Introduces geolocation prediction as an auxiliary task and injects predicted vectors into intermediate representations as conditioning signals to encourage unified representations for dialectal variations.

Result: Achieves state-of-the-art accuracy on FLEURS (97.7%) and 9.7% relative improvement on ML-SUPERB 2.0 dialect set across six multilingual datasets, showing improved robustness to intra-language variations and unseen domains.

Conclusion: Geolocation-aware conditioning effectively improves SSL-based language identification by learning more unified representations for dialectal and accented variations of the same language.

Abstract: While Self-supervised Learning (SSL) has significantly improved Spoken Language Identification (LID), existing models often struggle to consistently classify dialects and accents of the same language as a unified class. To address this challenge, we propose geolocation-aware LID, a novel approach that incorporates language-level geolocation information into the SSL-based LID model. Specifically, we introduce geolocation prediction as an auxiliary task and inject the predicted vectors into intermediate representations as conditioning signals. This explicit conditioning encourages the model to learn more unified representations for dialectal and accented variations. Experiments across six multilingual datasets demonstrate that our approach improves robustness to intra-language variations and unseen domains, achieving new state-of-the-art accuracy on FLEURS (97.7%) and 9.7% relative improvement on ML-SUPERB 2.0 dialect set.

Method: QR-Distill combines three techniques: 1) Quality filtering using LLM-based evaluation to retain only correct reasoning paths, 2) Conditional routing that dynamically assigns paths based on each student’s learning state, and 3) Cooperative peer teaching where students mutually distill diverse insights to address knowledge gaps and biases.

Result: Experiments show QR-Distill outperforms traditional single-path and multi-path distillation methods. Ablation studies confirm the importance of each component (quality filtering, conditional routing, and peer teaching) for effective knowledge transfer.

Conclusion: The proposed QR-Distill framework provides a superior approach for knowledge distillation by addressing limitations of conventional methods through quality-aware path selection, adaptive routing, and collaborative learning among student models.

Abstract: Advances in large language models (LLMs) significantly enhance reasoning capabilities but their deployment is restricted in resource-constrained scenarios. Knowledge distillation addresses this by transferring knowledge from powerful teacher models to compact and transparent students. However, effectively capturing the teacher’s comprehensive reasoning is challenging due to conventional token-level supervision’s limited scope. Using multiple reasoning paths per query alleviates this problem, but treating each path identically is suboptimal as paths vary widely in quality and suitability across tasks and models. We propose Quality-filtered Routing with Cooperative Distillation (QR-Distill), combining path quality filtering, conditional routing, and cooperative peer teaching. First, quality filtering retains only correct reasoning paths scored by an LLM-based evaluation. Second, conditional routing dynamically assigns paths tailored to each student’s current learning state. Finally, cooperative peer teaching enables students to mutually distill diverse insights, addressing knowledge gaps and biases toward specific reasoning styles. Experiments demonstrate QR-Distill’s superiority over traditional single- and multi-path distillation methods. Ablation studies further highlight the importance of each component including quality filtering, conditional routing, and peer teaching in effective knowledge transfer. Our code is available at https://github.com/LzyFischer/Distill.

Method: Created QFrCoLA dataset with 25k+ sentences, benchmarked 7 language models across 8 linguistic acceptability judgment corpora using fine-tuning and zero-shot approaches.

Result: Fine-tuned Transformer-based models performed best overall, especially for QFrCoLA. Zero-shot LLMs performed poorly. Cross-lingual LLMs showed limited linguistic judgment capabilities for Quebec French.

Conclusion: QFrCoLA provides a challenging benchmark for evaluating linguistic judgment capabilities, demonstrating that current models require fine-tuning rather than relying on zero-shot capabilities for linguistic acceptability tasks.

Abstract: Large and Transformer-based language models perform outstandingly in various downstream tasks. However, there is limited understanding regarding how these models internalize linguistic knowledge, so various linguistic benchmarks have recently been proposed to facilitate syntactic evaluation of language models across languages. This paper introduces QFrCoLA (Quebec-French Corpus of Linguistic Acceptability Judgments), a normative binary acceptability judgments dataset comprising 25,153 in-domain and 2,675 out-of-domain sentences. Our study leverages the QFrCoLA dataset and seven other linguistic binary acceptability judgment corpora to benchmark seven language models. The results demonstrate that, on average, fine-tuned Transformer-based LM are strong baselines for most languages and that zero-shot binary classification large language models perform poorly on the task. However, for the QFrCoLA benchmark, on average, a fine-tuned Transformer-based LM outperformed other methods tested. It also shows that pre-trained cross-lingual LLMs selected for our experimentation do not seem to have acquired linguistic judgment capabilities during their pre-training for Quebec French. Finally, our experiment results on QFrCoLA show that our dataset, built from examples that illustrate linguistic norms rather than speakers’ feelings, is similar to linguistic acceptability judgment; it is a challenging dataset that can benchmark LM on their linguistic judgment capabilities.

Method: Cascaded architecture with two QLoRA-fine-tuned Qwen 2.5-7B models: one predicts phrase-break positions and the other performs regression on prosodic targets to generate commercial TTS-compatible SSML markup.

Result: 99.2% F1 for break placement, 25-40% reduction in mean absolute error on pitch/rate/volume compared to baselines. MOS increased from 3.20 to 3.87 (p<0.005), with 15/18 listeners preferring enhanced synthesis.

Conclusion: Substantial progress in bridging the expressiveness gap between synthetic and natural French speech through SSML-enhanced prosody control.

Abstract: Despite recent advances, synthetic voices often lack expressiveness due to limited prosody control in commercial text-to-speech (TTS) systems. We introduce the first end-to-end pipeline that inserts Speech Synthesis Markup Language (SSML) tags into French text to control pitch, speaking rate, volume, and pause duration. We employ a cascaded architecture with two QLoRA-fine-tuned Qwen 2.5-7B models: one predicts phrase-break positions and the other performs regression on prosodic targets, generating commercial TTS-compatible SSML markup. Evaluated on a 14-hour French podcast corpus, our method achieves 99.2% F1 for break placement and reduces mean absolute error on pitch, rate, and volume by 25-40% compared with prompting-only large language models (LLMs) and a BiLSTM baseline. In perceptual evaluation involving 18 participants across over 9 hours of synthesized audio, SSML-enhanced speech generated by our pipeline significantly improves naturalness, with the mean opinion score increasing from 3.20 to 3.87 (p < 0.005). Additionally, 15 of 18 listeners preferred our enhanced synthesis. These results demonstrate substantial progress in bridging the expressiveness gap between synthetic and natural French speech. Our code is publicly available at https://github.com/hi-paris/Prosody-Control-French-TTS.

Method: Created FrJUDGE dataset for legal meaning preservation assessment and developed JUDGEBERT, a novel evaluation metric specifically designed for French legal text simplification.

Result: JUDGEBERT demonstrates superior correlation with human judgment compared to existing metrics and passes crucial sanity checks (100% for identical sentences, 0% for unrelated sentences).

Conclusion: JUDGEBERT has potential to transform legal NLP applications by ensuring accuracy and accessibility in legal text simplification for both legal practitioners and lay users.

Abstract: Simplifying text while preserving its meaning is a complex yet essential task, especially in sensitive domain applications like legal texts. When applied to a specialized field, like the legal domain, preservation differs significantly from its role in regular texts. This paper introduces FrJUDGE, a new dataset to assess legal meaning preservation between two legal texts. It also introduces JUDGEBERT, a novel evaluation metric designed to assess legal meaning preservation in French legal text simplification. JUDGEBERT demonstrates a superior correlation with human judgment compared to existing metrics. It also passes two crucial sanity checks, while other metrics did not: For two identical sentences, it always returns a score of 100%; on the other hand, it returns 0% for two unrelated sentences. Our findings highlight its potential to transform legal NLP applications, ensuring accuracy and accessibility for text simplification for legal practitioners and lay users.

Method: Conducted a fair comparison of self-supervised learning (SSL)-based discrete and continuous features using both small (Qwen1.5-0.5B) and large-scale (Llama3.1-8B) LLMs across six spoken language understanding tasks. Included in-depth analyses: efficient comparison, SSL layer analysis, LLM layer analysis, and robustness comparison.

Result: Continuous features generally outperform discrete tokens in various tasks. Each speech processing method exhibits distinct characteristics and patterns in how it learns and processes speech information.

Conclusion: The study provides valuable insights into the comparative performance of discrete vs. continuous features in SpeechLLMs, with continuous features showing superior performance, which can help advance spoken language understanding research.

Abstract: With the rise of Speech Large Language Models (SpeechLLMs), two dominant approaches have emerged for speech processing: discrete tokens and continuous features. Each approach has demonstrated strong capabilities in audio-related processing tasks. However, the performance gap between these two paradigms has not been thoroughly explored. To address this gap, we present a fair comparison of self-supervised learning (SSL)-based discrete and continuous features under the same experimental settings. We evaluate their performance across six spoken language understanding-related tasks using both small and large-scale LLMs (Qwen1.5-0.5B and Llama3.1-8B). We further conduct in-depth analyses, including efficient comparison, SSL layer analysis, LLM layer analysis, and robustness comparison. Our findings reveal that continuous features generally outperform discrete tokens in various tasks. Each speech processing method exhibits distinct characteristics and patterns in how it learns and processes speech information. We hope our results will provide valuable insights to advance spoken language understanding in SpeechLLMs.

Method: Construct a dialogue world model using POMDP framework, modeling emotion/sentiment/intention as user belief via information bottleneck maximization. Apply model-based reinforcement learning with joint training of policy, critic, and world model components.

Result: Achieves state-of-the-art performance on emotion classification and sentiment identification. Dialogue quality is enhanced through joint training. Shows good exploration-exploitation balance and transfers well to out-of-domain empathetic dialogues.

Conclusion: The proposed DreamCUB framework successfully applies world modeling to dialogue systems, demonstrating strong performance across multiple metrics and good generalization capabilities to different dialogue scenarios.

Abstract: World models have been widely utilized in robotics, gaming, and auto-driving. However, their applications on natural language tasks are relatively limited. In this paper, we construct the dialogue world model, which could predict the user’s emotion, sentiment, and intention, and future utterances. By defining a POMDP, we argue emotion, sentiment and intention can be modeled as the user belief and solved by maximizing the information bottleneck. By this user belief modeling, we apply the model-based reinforcement learning framework to the dialogue system, and propose a framework called DreamCUB. Experiments show that the pretrained dialogue world model can achieve state-of-the-art performances on emotion classification and sentiment identification, while dialogue quality is also enhanced by joint training of the policy, critic and dialogue world model. Further analysis shows that this manner holds a reasonable exploration-exploitation balance and also transfers well to out-of-domain scenarios such as empathetic dialogues.

Method: The survey categorizes controllable TTS methods by model architectures, control strategies, and feature representations, while also summarizing challenges, datasets, and evaluation metrics in the field.

Result: Provides a comprehensive taxonomy and framework for understanding controllable TTS, from traditional attribute control to modern natural language prompt-based approaches.

Conclusion: This survey offers guidance for researchers and practitioners by establishing a clear taxonomy and highlighting future directions in the fast-evolving field of controllable speech synthesis.

Abstract: Text-to-speech (TTS) has advanced from generating natural-sounding speech to enabling fine-grained control over attributes like emotion, timbre, and style. Driven by rising industrial demand and breakthroughs in deep learning, e.g., diffusion and large language models (LLMs), controllable TTS has become a rapidly growing research area. This survey provides the first comprehensive review of controllable TTS methods, from traditional control techniques to emerging approaches using natural language prompts. We categorize model architectures, control strategies, and feature representations, while also summarizing challenges, datasets, and evaluations in controllable TTS. This survey aims to guide researchers and practitioners by offering a clear taxonomy and highlighting future directions in this fast-evolving field. One can visit https://github.com/imxtx/awesome-controllabe-speech-synthesis for a comprehensive paper list and updates.

Method: Created OBJEX(MT) benchmark requiring models to distill transcripts into single-sentence objectives and report confidence. Evaluated using semantic similarity scoring, human-aligned correctness thresholds, and metacognition metrics (ECE, Brier score, Wrong@High-Conf, risk-coverage curves). Tested on SafeMT Attack_600, SafeMTData_1K, MHJ, and CoSafe datasets.

Result: Claude-Sonnet-4 achieved highest accuracy (0.515) and best calibration (ECE 0.296; Brier 0.324). GPT-4.1 and Qwen3 tied at 0.441 accuracy but showed severe overconfidence (mean confidence ~0.88 vs accuracy ~0.44; Wrong@0.90 ~48-52%). Performance varied widely across datasets (0.167-0.865), with MHJ being easier and Attack_600/CoSafe more difficult.

Conclusion: LLM judges frequently misinfer objectives with high confidence in multi-turn jailbreaks. Recommendations include providing explicit objectives when possible and using selective prediction/abstention for risk management. Prompts and evaluation materials released for replication.

Abstract: Large language models (LLMs) are increasingly used as judges of other models, yet it is unclear whether a judge can reliably infer the latent objective of the conversation it evaluates, especially when the goal is distributed across noisy, adversarial, multi-turn jailbreaks. We introduce OBJEX(MT), a benchmark that requires a model to (i) distill a transcript into a single-sentence base objective and (ii) report its own confidence. Accuracy is scored by an LLM judge using semantic similarity between extracted and gold objectives; correctness uses a single human-aligned threshold calibrated once on N=100 items (tau* = 0.61); and metacognition is evaluated with ECE, Brier score, Wrong@High-Conf, and risk-coverage curves. We evaluate gpt-4.1, claude-sonnet-4, and Qwen3-235B-A22B-FP8 on SafeMT Attack_600, SafeMTData_1K, MHJ, and CoSafe. claude-sonnet-4 attains the highest objective-extraction accuracy (0.515) and the best calibration (ECE 0.296; Brier 0.324), while gpt-4.1 and Qwen3 tie at 0.441 accuracy yet show marked overconfidence (mean confidence approx. 0.88 vs. accuracy approx. 0.44; Wrong@0.90 approx. 48-52%). Performance varies sharply across datasets (approx. 0.167-0.865), with MHJ comparatively easy and Attack_600/CoSafe harder. These results indicate that LLM judges often misinfer objectives with high confidence in multi-turn jailbreaks and suggest operational guidance: provide judges with explicit objectives when possible and use selective prediction or abstention to manage risk. We release prompts, scoring templates, and complete logs to facilitate replication and analysis.

Method: Developed an analytical framework using systematically designed prompts on existing multimodal sarcasm datasets, evaluating 12 state-of-the-art LVLMs over 2,409 samples, with additional validation on a 100-sample mini-benchmark.

Result: Found notable discrepancies across LVLMs and within the same model under varied prompts. Classification-oriented prompts yield higher consistency, while interpretive reasoning prompts cause significant divergence.

Conclusion: Sarcasm interpretation is subjective, requiring movement beyond binary labeling toward multi-perspective, uncertainty-aware modeling for better multimodal sarcasm comprehension.

Abstract: With the advent of large vision-language models (LVLMs) demonstrating increasingly human-like abilities, a pivotal question emerges: do different LVLMs interpret multimodal sarcasm differently, and can a single model grasp sarcasm from multiple perspectives like humans? To explore this, we introduce an analytical framework using systematically designed prompts on existing multimodal sarcasm datasets. Evaluating 12 state-of-the-art LVLMs over 2,409 samples, we examine interpretive variations within and across models, focusing on confidence levels, alignment with dataset labels, and recognition of ambiguous “neutral” cases. We further validate our findings on a diverse 100-sample mini-benchmark, incorporating multiple datasets, expanded prompt variants, and representative commercial LVLMs. Our findings reveal notable discrepancies – across LVLMs and within the same model under varied prompts. While classification-oriented prompts yield higher internal consistency, models diverge markedly when tasked with interpretive reasoning. These results challenge binary labeling paradigms by highlighting sarcasm’s subjectivity. We advocate moving beyond rigid annotation schemes toward multi-perspective, uncertainty-aware modeling, offering deeper insights into multimodal sarcasm comprehension. Our code and data are available at: https://github.com/CoderChen01/LVLMSarcasmAnalysis

Method: Conditional Front-Door Prompting (CFD-Prompting) constructs counterfactual external knowledge to simulate query behavior under varying contexts, enabling unbiased causal effect estimation between query and answer while operating under weaker assumptions than standard front-door adjustment.

Result: Extensive experiments across multiple LLMs and benchmark datasets show CFD-Prompting significantly outperforms existing baselines in both accuracy and robustness.

Conclusion: CFD-Prompting effectively addresses LLM internal bias through causal reasoning with counterfactual knowledge, providing a more robust and generalizable framework for knowledge-intensive tasks compared to traditional methods.

Abstract: Large Language Models (LLMs) have shown impressive capabilities in natural language processing but still struggle to perform well on knowledge-intensive tasks that require deep reasoning and the integration of external knowledge. Although methods such as Retrieval-Augmented Generation (RAG) and Chain-of-Thought (CoT) have been proposed to enhance LLMs with external knowledge, they still suffer from internal bias in LLMs, which often leads to incorrect answers. In this paper, we propose a novel causal prompting framework, Conditional Front-Door Prompting (CFD-Prompting), which enables the unbiased estimation of the causal effect between the query and the answer, conditional on external knowledge, while mitigating internal bias. By constructing counterfactual external knowledge, our framework simulates how the query behaves under varying contexts, addressing the challenge that the query is fixed and is not amenable to direct causal intervention. Compared to the standard front-door adjustment, the conditional variant operates under weaker assumptions, enhancing both robustness and generalisability of the reasoning process. Extensive experiments across multiple LLMs and benchmark datasets demonstrate that CFD-Prompting significantly outperforms existing baselines in both accuracy and robustness.

Method: Applied vocabulary projection on three state-of-the-art LALMs to track how attribute information evolves across layers and token positions.

Result: Found attribute information decreases with layer depth when recognition fails, and resolving attributes at earlier layers correlates with better accuracy. Models rely on querying auditory inputs rather than aggregating information in hidden states.

Conclusion: Demonstrated a method to enhance LALMs based on findings, offering insights into auditory attribute processing for future improvements.

Abstract: Understanding the internal mechanisms of large audio-language models (LALMs) is crucial for interpreting their behavior and improving performance. This work presents the first in-depth analysis of how LALMs internally perceive and recognize auditory attributes. By applying vocabulary projection on three state-of-the-art LALMs, we track how attribute information evolves across layers and token positions. We find that attribute information generally decreases with layer depth when recognition fails, and that resolving attributes at earlier layers correlates with better accuracy. Moreover, LALMs heavily rely on querying auditory inputs for predicting attributes instead of aggregating necessary information in hidden states at attribute-mentioning positions. Based on our findings, we demonstrate a method to enhance LALMs. Our results offer insights into auditory attribute processing, paving the way for future improvements.

Method: Created AHaBench (500 mental health prompts with expert responses) and AHaPairs (5K preference dataset). Used Direct Preference Optimization (DPO) for alignment with emotionally responsible behavior across multiple model families.

Result: DPO fine-tuning substantially reduces affective hallucination without degrading core reasoning and knowledge performance. Human-model agreement analyses confirm AHaBench reliably captures affective hallucination.

Conclusion: Establishes affective hallucination as a distinct safety concern and provides practical resources (benchmark and dataset) for developing LLMs that are both factually reliable and psychologically safe.

Abstract: Large Language Models (LLMs) are increasingly used in emotionally sensitive interactions, where their simulated empathy can create the illusion of genuine relational connection. We define this risk as Affective Hallucination, the production of emotionally immersive responses that foster illusory social presence despite the model’s lack of affective capacity. To systematically diagnose and mitigate this risk, we introduce AHaBench, a benchmark of 500 mental health-related prompts with expert-informed reference responses, evaluated along three dimensions: Emotional Enmeshment, Illusion of Presence, and Fostering Overdependence. We further release AHaPairs, a 5K-instance preference dataset enabling Direct Preference Optimization (DPO) for alignment with emotionally responsible behavior. Experiments across multiple model families show that DPO fine-tuning substantially reduces affective hallucination without degrading core reasoning and knowledge performance. Human-model agreement analyses confirm that AHaBench reliably captures affective hallucination, validating it as an effective diagnostic tool. This work establishes affective hallucination as a distinct safety concern and provides practical resources for developing LLMs that are not only factually reliable but also psychologically safe. AHaBench and AHaPairs are accessible via https://huggingface.co/datasets/o0oMiNGo0o/AHaBench, and code for fine-tuning and evaluation are in https://github.com/0oOMiNGOo0/AHaBench. Warning: This paper contains examples of mental health-related language that may be emotionally distressing.

Method: Used Corpus of Regional African American Language (CORAAL) transcribed with wav2vec 2.0 with/without LM, detected CCR and ING-reduction using Montreal Forced Aligner with pronunciation expansion.

Result: Found small but significant effect of CCR and ING-reduction on Word Error Rate, and stronger lexical neighborhood effect in ASR systems without Language Models.

Conclusion: AAE linguistic features impact ASR performance, and end-to-end systems without LMs are more influenced by lexical neighborhood effects than contextual predictability compared to LM-equipped systems.

Abstract: Automatic Speech Recognition (ASR) models often struggle with the phonetic, phonological, and morphosyntactic features found in African American English (AAE). This study focuses on two key AAE variables: Consonant Cluster Reduction (CCR) and ING-reduction. It examines whether the presence of CCR and ING-reduction increases ASR misrecognition. Subsequently, it investigates whether end-to-end ASR systems without an external Language Model (LM) are more influenced by lexical neighborhood effect and less by contextual predictability compared to systems with an LM. The Corpus of Regional African American Language (CORAAL) was transcribed using wav2vec 2.0 with and without an LM. CCR and ING-reduction were detected using the Montreal Forced Aligner (MFA) with pronunciation expansion. The analysis reveals a small but significant effect of CCR and ING on Word Error Rate (WER) and indicates a stronger presence of lexical neighborhood effect in ASR systems without LMs.

Method: Proposes KnowProb with six potential explanations derived from text content (three knowledge-based understanding and three association-based reasoning) to probe PLMs in a post-hoc explanation way.

Result: Validates that current PLMs only learn single distribution representations and struggle to capture hidden knowledge behind given texts. The approach effectively identifies limitations from multiple probing perspectives.

Conclusion: KnowProb facilitates explainable detection of black-box model limitations and promotes further research in understanding PLM knowledge comprehension capabilities.

Abstract: Pre-trained Language Models (PLMs) are trained on large amounts of unlabeled data, yet they exhibit remarkable reasoning skills. However, the trustworthiness challenges posed by these black-box models have become increasingly evident in recent years. To alleviate this problem, this paper proposes a novel Knowledge-guided Probing approach called KnowProb in a post-hoc explanation way, which aims to probe whether black-box PLMs understand implicit knowledge beyond the given text, rather than focusing only on the surface level content of the text. We provide six potential explanations derived from the underlying content of the given text, including three knowledge-based understanding and three association-based reasoning. In experiments, we validate that current small-scale (or large-scale) PLMs only learn a single distribution of representation, and still face significant challenges in capturing the hidden knowledge behind a given text. Furthermore, we demonstrate that our proposed approach is effective for identifying the limitations of existing black-box models from multiple probing perspectives, which facilitates researchers to promote the study of detecting black-box models in an explainable way.

Method: CoLMbo integrates a speaker encoder with prompt-based conditioning, allowing it to create detailed captions based on speaker embeddings and adapt dynamically to new speaker characteristics using user-defined prompts.

Result: The model successfully generates customized speaker descriptions including regional dialect variations and age-related traits, and excels in zero-shot scenarios across diverse datasets.

Conclusion: CoLMbo represents a significant advancement in speaker recognition by enhancing traditional speaker profiling capabilities and enabling detailed, context-rich speaker descriptions through innovative prompt-based conditioning.

Abstract: Speaker recognition systems are often limited to classification tasks and struggle to generate detailed speaker characteristics or provide context-rich descriptions. These models primarily extract embeddings for speaker identification but fail to capture demographic attributes such as dialect, gender, and age in a structured manner. This paper introduces CoLMbo, a Speaker Language Model (SLM) that addresses these limitations by integrating a speaker encoder with prompt-based conditioning. This allows for the creation of detailed captions based on speaker embeddings. CoLMbo utilizes user-defined prompts to adapt dynamically to new speaker characteristics and provides customized descriptions, including regional dialect variations and age-related traits. This innovative approach not only enhances traditional speaker profiling but also excels in zero-shot scenarios across diverse datasets, marking a significant advancement in the field of speaker recognition.

Method: Created a small TTRPG audio dataset and compared it against AMI and ICSI corpora. Evaluated performance of two diarizers (pyannote.audio and wespeaker) on this challenging audio material.

Result: Both diarizers showed higher confusion rates with TTRPG audio. Wespeaker strongly underestimated the number of speakers in TTRPG files compared to performance on standard corpora.

Conclusion: TTRPG audio serves as a promising challenge dataset for diarization systems due to its inherent voice conversion characteristics that expose limitations in current speaker identification technologies.

Abstract: This paper provides a proof of concept that audio of tabletop role-playing games (TTRPG) could serve as a challenge for diarization systems. TTRPGs are carried out mostly by conversation. Participants often alter their voices to indicate that they are talking as a fictional character. Audio processing systems are susceptible to voice conversion with or without technological assistance. TTRPG present a conversational phenomenon in which voice conversion is an inherent characteristic for an immersive gaming experience. This could make it more challenging for diarizers to pick the real speaker and determine that impersonating is just that. We present the creation of a small TTRPG audio dataset and compare it against the AMI and the ICSI corpus. The performance of two diarizers, pyannote.audio and wespeaker, were evaluated. We observed that TTRPGs’ properties result in a higher confusion rate for both diarizers. Additionally, wespeaker strongly underestimates the number of speakers in the TTRPG audio files. We propose TTRPG audio as a promising challenge for diarization systems.

Method: The researchers conducted an audit by investigating and surveying publicly available documentation from 6 LLM development initiatives by 5 leading organizations (OpenAI, Anthropic, Google, Meta, Alibaba), covering both proprietary and open-weight models published in the last 3 years.

Result: Detailed documentation of findings per initiative with an overall summary from value-setting and data-centric perspectives, revealing how alignment is actually understood and applied in practice.

Conclusion: The audit reveals practical implementation gaps in AI alignment processes and enables discussion of broader concerns regarding value selection and data practices in major LLM development initiatives.

Abstract: AI Alignment, primarily in the form of Reinforcement Learning from Human Feedback (RLHF), has been a cornerstone of the post-training phase in developing Large Language Models (LLMs). It has also been a popular research topic across various disciplines beyond Computer Science, including Philosophy and Law, among others, highlighting the socio-technical challenges involved. Nonetheless, except for the computational techniques related to alignment, there has been limited focus on the broader picture: the scope of these processes, which primarily rely on the selected objectives (values), and the data collected and used to imprint such objectives into the models. This work aims to reveal how alignment is understood and applied in practice from a value-setting and data-centric perspective. For this purpose, we investigate and survey (`audit’) publicly available documentation released by 6 LLM development initiatives by 5 leading organizations shaping this technology, focusing on proprietary (OpenAI’s GPT, Anthropic’s Claude, Google’s Gemini) and open-weight (Meta’s Llama, Google’s Gemma, and Alibaba’s Qwen) initiatives, all published in the last 3 years. The findings are documented in detail per initiative, while there is also an overall summary concerning different aspects, mainly from a value-setting and data-centric perspective. On the basis of our findings, we discuss a series of broader related concerns.

Method: Uses constrained generation with a pre-built prefix-tree index where knowledge graph triples are verbalized as textual facts and indexed for efficient access. During inference, LLM generates facts constrained to only existing sequences in the index.

Result: Scales to large knowledge bases (800 million facts), adapts to domain-specific data, achieves effective results on Question Answering with minimal generation-time overhead.

Conclusion: ReFactX provides a scalable and efficient method for LLMs to access external knowledge without complex pipelines or additional models, effectively addressing knowledge gaps while maintaining performance.

Abstract: Knowledge gaps and hallucinations are persistent challenges for Large Language Models (LLMs), which generate unreliable responses when lacking the necessary information to fulfill user instructions. Existing approaches, such as Retrieval-Augmented Generation (RAG) and tool use, aim to address these issues by incorporating external knowledge. Yet, they rely on additional models or services, resulting in complex pipelines, potential error propagation, and often requiring the model to process a large number of tokens. In this paper, we present a scalable method that enables LLMs to access external knowledge without depending on retrievers or auxiliary models. Our approach uses constrained generation with a pre-built prefix-tree index. Triples from a Knowledge Graph are verbalized in textual facts, tokenized, and indexed in a prefix tree for efficient access. During inference, to acquire external knowledge, the LLM generates facts with constrained generation which allows only sequences of tokens that form an existing fact. We evaluate our proposal on Question Answering and show that it scales to large knowledge bases (800 million facts), adapts to domain-specific data, and achieves effective results. These gains come with minimal generation-time overhead. ReFactX code is available at https://github.com/rpo19/ReFactX.

Method: Construct synthetic multi-hop QA dataset from factual news articles using knowledge graphs, augment through semantic clustering to recover missing links, and create a 2D difficulty matrix combining generator-side and retriever-side difficulty.

Result: Experiments show error rates strongly correlate with the proposed difficulty measures, validating their diagnostic utility across multiple domains and models.

Conclusion: GRADE provides a scalable foundation for evaluating and improving multi-hop reasoning in RAG systems for real-world applications.

Abstract: Retrieval-Augmented Generation (RAG) systems are widely adopted in knowledge-intensive NLP tasks, but current evaluations often overlook the structural complexity and multi-step reasoning required in real-world scenarios. These benchmarks overlook key factors such as the interaction between retrieval difficulty and reasoning depth. To address this gap, we propose \textsc{GRADE}, a novel evaluation framework that models task difficulty along two orthogonal dimensions: (1) reasoning depth, defined by the number of inference steps (hops), and (2) semantic distance between the query and its supporting evidence. We construct a synthetic multi-hop QA dataset from factual news articles by extracting knowledge graphs and augmenting them through semantic clustering to recover missing links, allowing us to generate diverse and difficulty-controlled queries. Central to our framework is a 2D difficulty matrix that combines generator-side and retriever-side difficulty. Experiments across multiple domains and models show that error rates strongly correlate with our difficulty measures, validating their diagnostic utility. \textsc{GRADE} enables fine-grained analysis of RAG performance and provides a scalable foundation for evaluating and improving multi-hop reasoning in real-world applications.

Method: Two-stage approach: Stage 1 distills token-level relevance from 13B LLaMA teacher to compact student using hybrid losses; Stage 2 attaches LoRA adapter and fine-tunes on GPT-4o-generated chain-of-thought permutations for listwise reasoning.

Result: Surpasses open-source baselines by +5.1 nDCG@5 on TREC-DL20, achieves 90.97 nDCG@10 on NovelEval (outperforming GPT-4 by +3.09), and 54.29 Top-1 accuracy on Natural Questions without Wikipedia fine-tuning.

Conclusion: Dual-loss distillation ensures stable calibration, making DeAR an effective and interpretable solution for modern reranking systems with superior accuracy across multiple benchmarks.

Abstract: Large Language Models (LLMs) have transformed listwise document reranking by enabling global reasoning over candidate sets, yet single models often struggle to balance fine-grained relevance scoring with holistic cross-document analysis. We propose \textbf{De}ep\textbf{A}gent\textbf{R}ank (\textbf{\DeAR}), an open-source framework that decouples these tasks through a dual-stage approach, achieving superior accuracy and interpretability. In \emph{Stage 1}, we distill token-level relevance signals from a frozen 13B LLaMA teacher into a compact {3, 8}B student model using a hybrid of cross-entropy, RankNet, and KL divergence losses, ensuring robust pointwise scoring. In \emph{Stage 2}, we attach a second LoRA adapter and fine-tune on 20K GPT-4o-generated chain-of-thought permutations, enabling listwise reasoning with natural-language justifications. Evaluated on TREC-DL19/20, eight BEIR datasets, and NovelEval-2306, \DeAR surpasses open-source baselines by +5.1 nDCG@5 on DL20 and achieves 90.97 nDCG@10 on NovelEval, outperforming GPT-4 by +3.09. Without fine-tuning on Wikipedia, DeAR also excels in open-domain QA, achieving 54.29 Top-1 accuracy on Natural Questions, surpassing baselines like MonoT5, UPR, and RankGPT. Ablations confirm that dual-loss distillation ensures stable calibration, making \DeAR a highly effective and interpretable solution for modern reranking systems.\footnote{Dataset and code available at https://github.com/DataScienceUIBK/DeAR-Reranking.}.

Method: Developed KL-regularised Q-Learning (KLQ), a new action-value RL method specifically for the LM-RLHF setting, and showed its equivalence to a version of PPO.

Result: KLQ performs on-par with PPO at optimizing the LM-RLHF objective and achieves consistently higher win-rate against PPO on LLM-as-a-judge evaluations for summarization and dialogue tasks.

Conclusion: KLQ provides a theoretically better-motivated alternative to PPO for LM-RLHF that maintains equivalent performance while achieving superior evaluation results.

Abstract: Proximal Policy Optimisation (PPO) is an established and effective policy gradient algorithm used for Language Model Reinforcement Learning from Human Feedback (LM-RLHF). PPO performs well empirically but has a heuristic motivation and handles the KL-divergence constraint used in LM-RLHF in an ad-hoc manner. In this paper, we develop a a new action-value RL method for the LM-RLHF setting, KL-regularised Q-Learning (KLQ). We then show that our method is equivalent to a version of PPO in a certain specific sense, despite its very different motivation. Finally, we benchmark KLQ on two key language generation tasks – summarisation and single-turn dialogue. We demonstrate that KLQ performs on-par with PPO at optimising the LM-RLHF objective, and achieves a consistently higher win-rate against PPO on LLM-as-a-judge evaluations.

Method: Leverages large language models’ long-term planning capabilities to execute interconnected steps that serve the ultimate goal, minimizing inclusion of unnecessary details during problem-solving.

Result: Outperforms strong baselines and achieves state-of-the-art performance on WikiTableQuestions and TabFact datasets through extensive experiments.

Conclusion: The proposed long-term planning approach effectively enhances table understanding by addressing key limitations of existing methods, demonstrating superior performance on challenging table-based tasks.

Abstract: Table understanding is key to addressing challenging downstream tasks such as table-based question answering and fact verification. Recent works have focused on leveraging Chain-of-Thought and question decomposition to solve complex questions requiring multiple operations on tables. However, these methods often suffer from a lack of explicit long-term planning and weak inter-step connections, leading to miss constraints within questions. In this paper, we propose leveraging the long-term planning capabilities of large language models (LLMs) to enhance table understanding. Our approach enables the execution of a long-term plan, where the steps are tightly interconnected and serve the ultimate goal, an aspect that methods based on Chain-of-Thought and question decomposition lack. In addition, our method effectively minimizes the inclusion of unnecessary details in the process of solving the next short-term goals, a limitation of methods based on Chain-of-Thought. Extensive experiments demonstrate that our method outperforms strong baselines and achieves state-of-the-art performance on WikiTableQuestions and TabFact datasets.

Method: Created EduRABSA dataset with manual annotations covering three review subject types and all main ABSA tasks. Developed ASQE-DPT annotation tool for generating comprehensive ABSA datasets from single-task annotation.

Result: Released first public ABSA education review dataset with comprehensive annotations. Provided open-source annotation tool and processing scripts to support research transparency and reproducibility.

Conclusion: EduRABSA removes dataset barriers in education ABSA research, enables creation of further resources, and supports transparency and reproducibility in educational opinion mining research.

Abstract: Every year, most educational institutions seek and receive an enormous volume of text feedback from students on courses, teaching, and overall experience. Yet, turning this raw feedback into useful insights is far from straightforward. It has been a long-standing challenge to adopt automatic opinion mining solutions for such education review text data due to the content complexity and low-granularity reporting requirements. Aspect-based Sentiment Analysis (ABSA) offers a promising solution with its rich, sub-sentence-level opinion mining capabilities. However, existing ABSA research and resources are very heavily focused on the commercial domain. In education, they are scarce and hard to develop due to limited public datasets and strict data protection. A high-quality, annotated dataset is urgently needed to advance research in this under-resourced area. In this work, we present EduRABSA (Education Review ABSA), the first public, annotated ABSA education review dataset that covers three review subject types (course, teaching staff, university) in the English language and all main ABSA tasks, including the under-explored implicit aspect and implicit opinion extraction. We also share ASQE-DPT (Data Processing Tool), an offline, lightweight, installation-free manual data annotation tool that generates labelled datasets for comprehensive ABSA tasks from a single-task annotation. Together, these resources contribute to the ABSA community and education domain by removing the dataset barrier, supporting research transparency and reproducibility, and enabling the creation and sharing of further resources. The dataset, annotation tool, and scripts and statistics for dataset processing and sampling are available at https://github.com/yhua219/edurabsa_dataset_and_annotation_tool.

Method: Introduces entity-oriented search that leverages semantic similarities between questions and table data, and implicit relationships between table cells. Uses graph query language for table understanding.

Result: Achieves new state-of-the-art performances on standard benchmarks WikiTableQuestions and TabFact.

Conclusion: The approach minimizes preprocessing needs, enhances contextual clarity through semantic binding of table cells, and establishes a new research direction using graph query language for table understanding.

Abstract: Our work addresses the challenges of understanding tables. Existing methods often struggle with the unpredictable nature of table content, leading to a reliance on preprocessing and keyword matching. They also face limitations due to the lack of contextual information, which complicates the reasoning processes of large language models (LLMs). To overcome these challenges, we introduce an entity-oriented search method to improve table understanding with LLMs. This approach effectively leverages the semantic similarities between questions and table data, as well as the implicit relationships between table cells, minimizing the need for data preprocessing and keyword matching. Additionally, it focuses on table entities, ensuring that table cells are semantically tightly bound, thereby enhancing contextual clarity. Furthermore, we pioneer the use of a graph query language for table understanding, establishing a new research direction. Experiments show that our approach achieves new state-of-the-art performances on standard benchmarks WikiTableQuestions and TabFact.

Method: Two-stage pipeline: (1) generation of geometrically controlled examples using constrained LLM, (2) multi-agentic reflective process for stylistic diversity and edge case discovery.

Result: Significant improvements in downstream guardrail model performance demonstrated on two benchmark datasets.

Conclusion: GRAID effectively addresses data scarcity in harmful text classification through geometric control and reflective augmentation, enhancing guardrail model performance.

Abstract: We address the problem of data scarcity in harmful text classification for guardrailing applications and introduce GRAID (Geometric and Reflective AI-Driven Data Augmentation), a novel pipeline that leverages Large Language Models (LLMs) for dataset augmentation. GRAID consists of two stages: (i) generation of geometrically controlled examples using a constrained LLM, and (ii) augmentation through a multi-agentic reflective process that promotes stylistic diversity and uncovers edge cases. This combination enables both reliable coverage of the input space and nuanced exploration of harmful content. Using two benchmark data sets, we demonstrate that augmenting a harmful text classification dataset with GRAID leads to significant improvements in downstream guardrail model performance.

Method: Construct neuron probe data from MUSE bilingual dictionaries to identify language-overlap neurons, then use HSIC-based metric to quantify linguistic spectrum and guide optimal bridge language selection.

Result: Experiments on 2 cross-lingual tasks and 15 language pairs from 7 diverse families validate the effectiveness of BridgeX-ICL and provide insights into LLMs’ multilingual mechanisms.

Conclusion: BridgeX-ICL is a simple yet effective method that successfully improves zero-shot cross-lingual performance by leveraging language-overlap neurons, offering empirical understanding of multilingual mechanisms in LLMs.

Abstract: The current Large Language Models (LLMs) face significant challenges in improving performance on low-resource languages and urgently need data-efficient methods without costly fine-tuning. From the perspective of language-bridge, we propose BridgeX-ICL, a simple yet effective method to improve zero-shot Cross-lingual In-Context Learning (X-ICL) for low-resource languages. Unlike existing works focusing on language-specific neurons, BridgeX-ICL explores whether sharing neurons can improve cross-lingual performance in LLMs or not. We construct neuron probe data from the ground-truth MUSE bilingual dictionaries, and define a subset of language overlap neurons accordingly, to ensure full activation of these anchored neurons. Subsequently, we propose an HSIC-based metric to quantify LLMs' internal linguistic spectrum based on overlap neurons, which guides optimal bridge selection. The experiments conducted on 2 cross-lingual tasks and 15 language pairs from 7 diverse families (covering both high-low and moderate-low pairs) validate the effectiveness of BridgeX-ICL and offer empirical insights into the underlying multilingual mechanisms of LLMs.

Method: Layer-wise similarity analysis and controlled experiments to examine whether homogenization occurs and how positional bias amplifies this effect.

Result: Tokens systematically lose distinctiveness during processing, particularly when biased toward extremal positions. Confirms both the existence of homogenization and its dependence on positional attention mechanisms.

Conclusion: Token homogenization exists in transformers and is dependent on positional attention mechanisms, with positional bias amplifying the convergence of token representations toward uniformity.

Abstract: This paper investigates token homogenization - the convergence of token representations toward uniformity across transformer layers and its relationship to positional bias in large language models. We empirically examine whether homogenization occurs and how positional bias amplifies this effect. Through layer-wise similarity analysis and controlled experiments, we demonstrate that tokens systematically lose distinctiveness during processing, particularly when biased toward extremal positions. Our findings confirm both the existence of homogenization and its dependence on positional attention mechanisms.

Method: Pipeline that first identifies contextually relevant candidate sentences using token-level attention scores, then uses pretrained NLI model to classify each candidate as premise (entailment) or contradiction.

Result: Method efficiently isolates the most significant semantic relationships for any given claim by filtering NLI-identified relationships with attention-based saliency scores.

Conclusion: Combining attention mechanisms and NLI models provides targeted analysis of sentence relationships, overcoming limitations of individual approaches for tasks like fact-checking and argument mining.

Abstract: Finding the relationships between sentences in a document is crucial for tasks like fact-checking, argument mining, and text summarization. A key challenge is to identify which sentences act as premises or contradictions for a specific claim. Existing methods often face a trade-off: transformer attention mechanisms can identify salient textual connections but lack explicit semantic labels, while Natural Language Inference (NLI) models can classify relationships between sentence pairs but operate independently of contextual saliency. In this work, we introduce a method that combines the strengths of both approaches for a targeted analysis. Our pipeline first identifies candidate sentences that are contextually relevant to a user-selected target sentence by aggregating token-level attention scores. It then uses a pretrained NLI model to classify each candidate as a premise (entailment) or contradiction. By filtering NLI-identified relationships with attention-based saliency scores, our method efficiently isolates the most significant semantic relationships for any given claim in a text.

Method: Conducted a controlled experiment where participants issued queries and selected from headlines filtered by specific linguistic frames (conflict, strategy, episodic, thematic).

Result: Headline framing significantly shaped follow-up queries: conflict and strategy frames disrupted alignment with prior selections, while episodic frames led to more concrete queries than thematic ones. Modest short-term frame persistence was observed but declined over time.

Conclusion: Even brief exposure to framing can meaningfully alter the direction of users’ information-seeking behavior, suggesting that search engine interfaces should be designed with awareness of these framing effects.

Abstract: Search engines play a central role in how people gather information, but subtle cues like headline framing may influence not only what users believe but also how they search. While framing effects on judgment are well documented, their impact on subsequent search behavior is less understood. We conducted a controlled experiment where participants issued queries and selected from headlines filtered by specific linguistic frames. Headline framing significantly shaped follow-up queries: conflict and strategy frames disrupted alignment with prior selections, while episodic frames led to more concrete queries than thematic ones. We also observed modest short-term frame persistence that declined over time. These results suggest that even brief exposure to framing can meaningfully alter the direction of users information-seeking behavior.

Method: Empirical study analyzing distribution of satisfiability problems and evaluating TLMs’ performance across different complexity classes and grammatical structures.

Result: The paper examines how problem instances from varying computational complexity classes impact TLMs’ inference learning capabilities.

Conclusion: Understanding the relationship between computational complexity, grammatical constructs, and TLMs’ reasoning performance is crucial for evaluating their true capabilities in natural language satisfiability tasks.

Abstract: Efforts to apply transformer-based language models (TLMs) to the problem of reasoning in natural language have enjoyed ever-increasing success in recent years. The most fundamental task in this area to which nearly all others can be reduced is that of determining satisfiability. However, from a logical point of view, satisfiability problems vary along various dimensions, which may affect TLMs’ ability to learn how to solve them. The problem instances of satisfiability in natural language can belong to different computational complexity classes depending on the language fragment in which they are expressed. Although prior research has explored the problem of natural language satisfiability, the above-mentioned point has not been discussed adequately. Hence, we investigate how problem instances from varying computational complexity classes and having different grammatical constructs impact TLMs' ability to learn rules of inference. Furthermore, to faithfully evaluate TLMs, we conduct an empirical study to explore the distribution of satisfiability problems.

Method: Uses Large Language Models for query parsing, schema linking, and visualization selection over a live temporally-indexed database built from Fantasy Premier League data.

Result: Developed a benchmark (DSQABENCH) with 1,700+ annotated queries and demonstrated that users can seamlessly explore evolving sports statistics through conversational interfaces.

Conclusion: SPORTSQL successfully bridges the gap between natural language queries and complex sports data analysis, making dynamic sports statistics accessible to non-technical users through an intuitive interface.

Abstract: We present a modular, interactive system, SPORTSQL, for natural language querying and visualization of dynamic sports data, with a focus on the English Premier League (EPL). The system translates user questions into executable SQL over a live, temporally indexed database constructed from real-time Fantasy Premier League (FPL) data. It supports both tabular and visual outputs, leveraging the symbolic reasoning capabilities of Large Language Models (LLMs) for query parsing, schema linking, and visualization selection. To evaluate system performance, we introduce the Dynamic Sport Question Answering benchmark (DSQABENCH), comprising 1,700+ queries annotated with SQL programs, gold answers, and database snapshots. Our demo highlights how non-expert users can seamlessly explore evolving sports statistics through a natural, conversational interface.

Method: Proposes a framework with three metrics: performance realisation ratio, its coefficient of variation, and language potential. Tested on 13 model variants across 11 multilingual datasets.

Result: The framework provides more reliable measurement of model performance and language disparities, especially for low-resource languages. Reveals that higher overall model performance doesn’t necessarily mean greater fairness across languages.

Conclusion: The proposed framework enables finer-grained and more insightful quantification of multilingual performance disparities, addressing challenges in evaluating low-resource languages and revealing important insights about model fairness.

Abstract: Results reported in large-scale multilingual evaluations are often fragmented and confounded by factors such as target languages, differences in experimental setups, and model choices. We propose a framework that disentangles these confounding variables and introduces three interpretable metrics–the performance realisation ratio, its coefficient of variation, and language potential–enabling a finer-grained and more insightful quantification of actual performance disparities across both (i) models and (ii) languages. Through a case study of 13 model variants on 11 multilingual datasets, we demonstrate that our framework provides a more reliable measurement of model performance and language disparities, particularly for low-resource languages, which have so far proven challenging to evaluate. Importantly, our results reveal that higher overall model performance does not necessarily imply greater fairness across languages.

Method: Evaluated LLM-generated annotations against existing annotator modeling techniques for perspective modeling, using predefined annotator personas within strong-to-weak data perspectivism spectra.

Result: LLMs selectively use demographic attributes from personas, with annotator modeling techniques performing better under weak data perspectivism compared to strong perspectivism and human annotations. For personalized datasets, LLM performance approached but did not exceed human annotators.

Conclusion: LLM-generated views tend towards aggregation despite subjective prompting, and while they can approach human performance for personalized datasets in strong perspectivism, they do not exceed human annotators’ capabilities.

Abstract: In this work, we explore the capability of Large Language Models (LLMs) to annotate hate speech and abusiveness while considering predefined annotator personas within the strong-to-weak data perspectivism spectra. We evaluated LLM-generated annotations against existing annotator modeling techniques for perspective modeling. Our findings show that LLMs selectively use demographic attributes from the personas. We identified prototypical annotators, with persona features that show varying degrees of alignment with the original human annotators. Within the data perspectivism paradigm, annotator modeling techniques that do not explicitly rely on annotator information performed better under weak data perspectivism compared to both strong data perspectivism and human annotations, suggesting LLM-generated views tend towards aggregation despite subjective prompting. However, for more personalized datasets tailored to strong perspectivism, the performance of LLM annotator modeling approached, but did not exceed, human annotators.

Method: Categorizes data construction into expert annotation, distillation from larger models, and self-improvement; covers full-parameter and parameter-efficient fine-tuning (LoRA, prefix tuning); examines evaluation protocols.

Result: Identifies distinct trade-offs between quality, scalability and resource cost across different paradigms; highlights computational efficiency and model reusability benefits of lightweight approaches.

Conclusion: Closer integration of data, algorithms and human feedback is essential for advancing instruction-tuned LLMs; serves as practical reference for designing effective and reliably aligned models.

Abstract: Instruction tuning is a pivotal technique for aligning large language models (LLMs) with human intentions, safety constraints, and domain-specific requirements. This survey provides a comprehensive overview of the full pipeline, encompassing (i) data collection methodologies, (ii) full-parameter and parameter-efficient fine-tuning strategies, and (iii) evaluation protocols. We categorized data construction into three major paradigms: expert annotation, distillation from larger models, and self-improvement mechanisms, each offering distinct trade-offs between quality, scalability, and resource cost. Fine-tuning techniques range from conventional supervised training to lightweight approaches, such as low-rank adaptation (LoRA) and prefix tuning, with a focus on computational efficiency and model reusability. We further examine the challenges of evaluating faithfulness, utility, and safety across multilingual and multimodal scenarios, highlighting the emergence of domain-specific benchmarks in healthcare, legal, and financial applications. Finally, we discuss promising directions for automated data generation, adaptive optimization, and robust evaluation frameworks, arguing that a closer integration of data, algorithms, and human feedback is essential for advancing instruction-tuned LLMs. This survey aims to serve as a practical reference for researchers and practitioners seeking to design LLMs that are both effective and reliably aligned with human intentions.

Method: Proposed PU-ADKA framework that actively engages domain experts by selectively querying the most appropriate expert based on availability, knowledge boundaries, and consultation costs. Trained using simulations on PubMed data and validated through controlled expert interactions and real-world deployment.

Result: Demonstrated effectiveness in enhancing LLM performance in specialized domains under strict budget constraints. Also introduced a new benchmark dataset CKAD for cost-effective LLM domain knowledge acquisition.

Conclusion: PU-ADKA provides an efficient approach to bridge the knowledge gap in specialized domains by strategically leveraging human expertise within budget limitations, offering a practical solution for real-world applications in drug development and rare disease research.

Abstract: Large Language Models (LLMs) have demonstrated an impressive level of general knowledge. However, they often struggle in highly specialized and cost-sensitive domains such as drug discovery and rare disease research due to the lack of expert knowledge. In this paper, we propose a novel framework (PU-ADKA) designed to efficiently enhance domain-specific LLMs by actively engaging domain experts within a fixed budget. Unlike traditional fine-tuning approaches, PU-ADKA selectively identifies and queries the most appropriate expert from a team, taking into account each expert’s availability, knowledge boundaries, and consultation costs. We train PU-ADKA using simulations on PubMed data and validate it through both controlled expert interactions and real-world deployment with a drug development team, demonstrating its effectiveness in enhancing LLM performance in specialized domains under strict budget constraints. In addition to outlining our methodological innovations and experimental results, we introduce a new benchmark dataset, CKAD, for cost-effective LLM domain knowledge acquisition to foster further research in this challenging area.

Method: SSFO constructs preference data pairs by contrasting model outputs with and without context, then uses Direct Preference Optimization (DPO) to align model faithfulness. A modified DPO loss function encourages likelihood displacement from parametric-based tokens to context-aligned tokens.

Result: SSFO significantly outperforms existing methods, achieving state-of-the-art faithfulness on multiple context-based QA datasets. It shows strong generalization, improving cross-lingual faithfulness while preserving general instruction-following capabilities.

Conclusion: SSFO provides an effective self-supervised approach for enhancing RAG faithfulness without labeling costs or additional inference burden, leveraging theoretical insights about likelihood displacement for optimal performance.

Abstract: Retrieval-Augmented Generation (RAG) systems require Large Language Models (LLMs) to generate responses that are faithful to the retrieved context. However, faithfulness hallucination remains a critical challenge, as existing methods often require costly supervision and post-training or significant inference burdens. To overcome these limitations, we introduce Self-Supervised Faithfulness Optimization (SSFO), the first self-supervised alignment approach for enhancing RAG faithfulness. SSFO constructs preference data pairs by contrasting the model’s outputs generated with and without the context. Leveraging Direct Preference Optimization (DPO), SSFO aligns model faithfulness without incurring labeling costs or additional inference burden. We theoretically and empirically demonstrate that SSFO leverages a benign form of \emph{likelihood displacement}, transferring probability mass from parametric-based tokens to context-aligned tokens. Based on this insight, we propose a modified DPO loss function to encourage likelihood displacement. Comprehensive evaluations show that SSFO significantly outperforms existing methods, achieving state-of-the-art faithfulness on multiple context-based question-answering datasets. Notably, SSFO exhibits strong generalization, improving cross-lingual faithfulness and preserving general instruction-following capabilities. We release our code and model at the anonymous link: https://github.com/chkwy/SSFO

Method: Constructed ClaimGen-CN dataset from real-world legal disputes, designed evaluation metrics for factuality and clarity, and conducted comprehensive zero-shot evaluation of state-of-the-art general and legal-domain LLMs.

Result: Current models show limitations in factual precision and expressive clarity when generating legal claims, indicating the need for more targeted development in this domain.

Conclusion: The research highlights the challenges in legal claim generation and provides a benchmark dataset to encourage further exploration and development of models specifically tailored for this important legal assistance task.

Abstract: Legal claims refer to the plaintiff’s demands in a case and are essential to guiding judicial reasoning and case resolution. While many works have focused on improving the efficiency of legal professionals, the research on helping non-professionals (e.g., plaintiffs) remains unexplored. This paper explores the problem of legal claim generation based on the given case’s facts. First, we construct ClaimGen-CN, the first dataset for Chinese legal claim generation task, from various real-world legal disputes. Additionally, we design an evaluation metric tailored for assessing the generated claims, which encompasses two essential dimensions: factuality and clarity. Building on this, we conduct a comprehensive zero-shot evaluation of state-of-the-art general and legal-domain large language models. Our findings highlight the limitations of the current models in factual precision and expressive clarity, pointing to the need for more targeted development in this domain. To encourage further exploration of this important task, we will make the dataset publicly available.

Method: Distills two complementary knowledge types (high-level rules and fine-grained reasoning), trains separate LoRA experts for each, and uses dynamic fusion with input-aware routing to balance expert contributions and mitigate conflicts.

Result: Achieves accuracy comparable to or better than teacher LLM while maintaining computational efficiency. Outperforms state-of-the-art bundle generation approaches on three public datasets.

Conclusion: RouteDK effectively mitigates knowledge conflicts through expert routing and dynamic fusion, enabling efficient bundle generation with teacher-level performance.

Abstract: Large Language Models (LLMs) have shown potential in automatic bundle generation but suffer from prohibitive computational costs. Although knowledge distillation offers a pathway to more efficient student models, our preliminary study reveals that naively integrating diverse types of distilled knowledge from teacher LLMs into student LLMs leads to knowledge conflict, negatively impacting the performance of bundle generation. To address this, we propose RouteDK, a framework for routing distilled knowledge through a mixture of LoRA expert architecture. Specifically, we first distill knowledge from the teacher LLM for bundle generation in two complementary types: high-level knowledge (generalizable rules) and fine-grained knowledge (session-specific reasoning). We then train knowledge-specific LoRA experts for each type of knowledge together with a base LoRA expert. For effective integration, we propose a dynamic fusion module, featuring an input-aware router, where the router balances expert contributions by dynamically determining optimal weights based on input, thereby effectively mitigating knowledge conflicts. To further improve inference reliability, we design an inference-time enhancement module to reduce variance and mitigate suboptimal reasoning. Experiments on three public datasets show that our RouteDK achieves accuracy comparable to or even better than the teacher LLM, while maintaining strong computational efficiency. In addition, it outperforms state-of-the-art approaches for bundle generation.

Method: Novel techniques combining multiple chain-of-thought agents using large language models’ token-level uncertainty scores. Experiments conducted with 3B and 70B+ parameter variants of Llama and Qwen models.

Result: Demonstrates practical applicability of zero-shot approaches for aspect-category sentiment analysis in label-scarce conditions.

Conclusion: LLM-based zero-shot methods with uncertainty scoring can effectively address annotation scarcity challenges while opening discussions on accuracy measurement in low-resource settings.

Abstract: Aspect-category sentiment analysis provides granular insights by identifying specific themes within product reviews that are associated with particular opinions. Supervised learning approaches dominate the field. However, data is scarce and expensive to annotate for new domains. We argue that leveraging large language models in a zero-shot setting is beneficial where the time and resources required for dataset annotation are limited. Furthermore, annotation bias may lead to strong results using supervised methods but transfer poorly to new domains in contexts that lack annotations and demand reproducibility. In our work, we propose novel techniques that combine multiple chain-of-thought agents by leveraging large language models’ token-level uncertainty scores. We experiment with the 3B and 70B+ parameter size variants of Llama and Qwen models, demonstrating how these approaches can fulfil practical needs and opening a discussion on how to gauge accuracy in label-scarce conditions.

Method: Structured analysis of high-quality papers (A*/A rank conferences and Q1 journals from 2023-2025) covering LLM agent architectures, cognitive mechanisms, prompting methods, fine-tuning procedures, and evaluation of 68 public datasets.

Result: Identified critical findings on LLMs’ verifiable reasoning capabilities, self-improvement capacity, and personalization potential for agent applications.

Conclusion: The review provides comprehensive insights into current LLM agent capabilities and outlines ten future research directions to address existing gaps in the field.

Abstract: The pursuit of human-level artificial intelligence (AI) has significantly advanced the development of autonomous agents and Large Language Models (LLMs). LLMs are now widely utilized as decision-making agents for their ability to interpret instructions, manage sequential tasks, and adapt through feedback. This review examines recent developments in employing LLMs as autonomous agents and tool users and comprises seven research questions. We only used the papers published between 2023 and 2025 in conferences of the A* and A rank and Q1 journals. A structured analysis of the LLM agents’ architectural design principles, dividing their applications into single-agent and multi-agent systems, and strategies for integrating external tools is presented. In addition, the cognitive mechanisms of LLM, including reasoning, planning, and memory, and the impact of prompting methods and fine-tuning procedures on agent performance are also investigated. Furthermore, we evaluated current benchmarks and assessment protocols and have provided an analysis of 68 publicly available datasets to assess the performance of LLM-based agents in various tasks. In conducting this review, we have identified critical findings on verifiable reasoning of LLMs, the capacity for self-improvement, and the personalization of LLM-based agents. Finally, we have discussed ten future research directions to overcome these gaps.

Method: Analyzed interaction logs to define dropouts and identify contributing factors, developed a course-progress-adaptive dropout prediction framework (CPADP), and designed a personalized email recall agent for at-risk students.

Result: Found strong links between dropout behaviors and textual interaction patterns, achieved up to 95.4% accuracy in dropout prediction, and validated effectiveness with over 3,000 diverse students in deployed MAIC system.

Conclusion: The study demonstrates that AI-driven interactive learning platforms can effectively predict and reduce dropouts through personalized interventions based on textual interaction patterns, making online education more engaging and effective.

Abstract: Interactive online learning environments, represented by Massive AI-empowered Courses (MAIC), leverage LLM-driven multi-agent systems to transform passive MOOCs into dynamic, text-based platforms, enhancing interactivity through LLMs. This paper conducts an empirical study on a specific MAIC course to explore three research questions about dropouts in these interactive online courses: (1) What factors might lead to dropouts? (2) Can we predict dropouts? (3) Can we reduce dropouts? We analyze interaction logs to define dropouts and identify contributing factors. Our findings reveal strong links between dropout behaviors and textual interaction patterns. We then propose a course-progress-adaptive dropout prediction framework (CPADP) to predict dropouts with at most 95.4% accuracy. Based on this, we design a personalized email recall agent to re-engage at-risk students. Applied in the deployed MAIC system with over 3,000 students, the feasibility and effectiveness of our approach have been validated on students with diverse backgrounds.

Method: Benchmarked multiple LLMs, augmented PalmX dataset with Palm dataset and curated 22K+ culturally grounded MCQs, then performed LoRA fine-tuning of the best-performing Fanar-1-9B-Instruct model.

Result: Achieved 5th place with 70.50% accuracy on blind test set and 84.1% accuracy on PalmX development set using the fine-tuned Fanar-1-9B-Instruct model.

Conclusion: Data augmentation and LoRA fine-tuning of appropriately selected LLMs can effectively improve performance on Arabic cultural knowledge tasks, with Fanar-1-9B-Instruct showing superior results among tested models.

Abstract: In this paper, we report our participation to the PalmX cultural evaluation shared task. Our system, CultranAI, focused on data augmentation and LoRA fine-tuning of large language models (LLMs) for Arabic cultural knowledge representation. We benchmarked several LLMs to identify the best-performing model for the task. In addition to utilizing the PalmX dataset, we augmented it by incorporating the Palm dataset and curated a new dataset of over 22K culturally grounded multiple-choice questions (MCQs). Our experiments showed that the Fanar-1-9B-Instruct model achieved the highest performance. We fine-tuned this model on the combined augmented dataset of 22K+ MCQs. On the blind test set, our submitted system ranked 5th with an accuracy of 70.50%, while on the PalmX development set, it achieved an accuracy of 84.1%.

Method: Multi-stage training workflow with two reinforcement learning phases and one supervised fine-tuning phase, constructing domain-independent knowledge graphs and auto-generating QA pairs to address data scarcity.

Result: Significant improvements in answering multi-hop questions, with notable performance gains on complex 3+ hop questions, demonstrating strong generalization across diverse knowledge domains.

Conclusion: The proposed Omne-R1 framework effectively enhances multi-hop question answering capabilities on schema-free knowledge graphs through innovative training methods and data generation techniques.

Abstract: This paper introduces Omne-R1, a novel approach designed to enhance multi-hop question answering capabilities on schema-free knowledge graphs by integrating advanced reasoning models. Our method employs a multi-stage training workflow, including two reinforcement learning phases and one supervised fine-tuning phase. We address the challenge of limited suitable knowledge graphs and QA data by constructing domain-independent knowledge graphs and auto-generating QA pairs. Experimental results show significant improvements in answering multi-hop questions, with notable performance gains on more complex 3+ hop questions. Our proposed training framework demonstrates strong generalization abilities across diverse knowledge domains.

Method: Introduces a pruning module between the two low-rank matrices in LoRA to enable dynamic subspace learning through rank dimension pruning, allowing continuous adaptation of the learning subspace.

Result: DropLoRA consistently outperforms standard LoRA in fine-tuning LLaMA models across various tasks including commonsense reasoning, mathematical reasoning, code generation, and instruction-following.

Conclusion: DropLoRA effectively overcomes the limitations of traditional LoRA by enabling dynamic low-rank subspace learning, providing significant performance improvements without additional training or inference costs.

Abstract: LoRA-based large model parameter-efficient fine-tuning (PEFT) methods use low-rank de- composition to approximate updates to model parameters. However, compared to full- parameter fine-tuning, low-rank updates often lead to a performance gap in downstream tasks. To address this, we introduce DropLoRA, a novel pruning-based approach that focuses on pruning the rank dimension. Unlike conven- tional methods that attempt to overcome the low-rank bottleneck, DropLoRA innovatively integrates a pruning module between the two low-rank matrices in LoRA to simulate dy- namic subspace learning. This dynamic low- rank subspace learning allows DropLoRA to overcome the limitations of traditional LoRA, which operates within a static subspace. By continuously adapting the learning subspace, DropLoRA significantly boosts performance without incurring additional training or infer- ence costs. Our experimental results demon- strate that DropLoRA consistently outperforms LoRA in fine-tuning the LLaMA series across a wide range of large language model gener- ation tasks, including commonsense reason- ing, mathematical reasoning, code generation, and instruction-following. Our code is avail- able at https://github.com/TayeeChang/DropLoRA.

Method: Extracts legal reasoning components using prompt-based LLMs, normalizes legal provision references, and links facts, norms, and legal applications through a legal inference ontology from 648 Japanese administrative court decisions.

Result: The system achieves more accurate retrieval of relevant legal provisions from facts compared to LLM baselines and retrieval-augmented methods, as evaluated by expert annotated data.

Conclusion: The constructed legal knowledge graph successfully captures the full structure of legal reasoning from real court decisions, making implicit reasoning explicit and machine-readable.

Abstract: Court judgments reveal how legal rules have been interpreted and applied to facts, providing a foundation for understanding structured legal reasoning. However, existing automated approaches for capturing legal reasoning, including large language models, often fail to identify the relevant legal context, do not accurately trace how facts relate to legal norms, and may misrepresent the layered structure of judicial reasoning. These limitations hinder the ability to capture how courts apply the law to facts in practice. In this paper, we address these challenges by constructing a legal knowledge graph from 648 Japanese administrative court decisions. Our method extracts components of legal reasoning using prompt-based large language models, normalizes references to legal provisions, and links facts, norms, and legal applications through an ontology of legal inference. The resulting graph captures the full structure of legal reasoning as it appears in real court decisions, making implicit reasoning explicit and machine-readable. We evaluate our system using expert annotated data, and find that it achieves more accurate retrieval of relevant legal provisions from facts than large language model baselines and retrieval-augmented methods.

Method: Proposes confidence-modulated drafting using entropy and margin-based uncertainty measures to dynamically adjust speculative token generation length and flexible verification.

Result: Significant speedups over standard speculative decoding while preserving or improving BLEU and ROUGE scores on machine translation and summarization tasks.

Conclusion: Provides a principled, plug-in method for efficient and robust decoding in large language models under varying uncertainty conditions.

Abstract: Speculative decoding has emerged as an effective approach for accelerating autoregressive inference by parallelizing token generation through a draft-then-verify paradigm. However, existing methods rely on static drafting lengths and rigid verification criteria, limiting their adaptability across varying model uncertainties and input complexities. This paper proposes an information-theoretic framework for speculative decoding based on confidence-modulated drafting. By leveraging entropy and margin-based uncertainty measures over the drafter’s output distribution, the proposed method dynamically adjusts the number of speculatively generated tokens at each iteration. This adaptive mechanism reduces rollback frequency, improves resource utilization, and maintains output fidelity. Additionally, the verification process is modulated using the same confidence signals, enabling more flexible acceptance of drafted tokens without sacrificing generation quality. Experiments on machine translation and summarization tasks demonstrate significant speedups over standard speculative decoding while preserving or improving BLEU and ROUGE scores. The proposed approach offers a principled, plug-in method for efficient and robust decoding in large language models under varying conditions of uncertainty.

Method: Develops a pipeline using word alignment, etymology-aware edit distance, and smoothing to annotate parallel corpus with word-level AGS, then trains regression model to predict AGS in context.

Result: Outperforms strong baselines including state-of-the-art dialect ID systems on a multi-dialect benchmark.

Conclusion: AGS provides a scalable, linguistically grounded approach to model lexical generality, enriching representations of Arabic dialectness beyond single-dimensional measures.

Abstract: Arabic dialects form a diverse continuum, yet NLP models often treat them as discrete categories. Recent work addresses this issue by modeling dialectness as a continuous variable, notably through the Arabic Level of Dialectness (ALDi). However, ALDi reduces complex variation to a single dimension. We propose a complementary measure: the Arabic Generality Score (AGS), which quantifies how widely a word is used across dialects. We introduce a pipeline that combines word alignment, etymology-aware edit distance, and smoothing to annotate a parallel corpus with word-level AGS. A regression model is then trained to predict AGS in context. Our approach outperforms strong baselines, including state-of-the-art dialect ID systems, on a multi-dialect benchmark. AGS offers a scalable, linguistically grounded way to model lexical generality, enriching representations of Arabic dialectness.

Method: Used a comprehensive prompt pack covering modern standard Arabic, five regional dialects, code-switching, factual knowledge, arithmetic/temporal reasoning, creative generation, and adversarial safety. Collected 115 outputs (23 prompts × 5 runs) and scored each with three frontier LLM judges (GPT-5, Gemini 2.5 Pro, Claude Sonnet-4).

Result: ALLaM-34B demonstrated consistently high performance: generation and code-switching (4.92/5), MSA handling (4.74/5), reasoning (4.64/5), dialect fidelity (4.21/5), and safety (4.54/5).

Conclusion: ALLaM-34B is positioned as a robust and culturally grounded Arabic LLM with both technical strength and practical readiness for real-world deployment.

Abstract: Large language models (LLMs) trained primarily on English corpora often struggle to capture the linguistic and cultural nuances of Arabic. To address this gap, the Saudi Data and AI Authority (SDAIA) introduced the $ALLaM$ family of Arabic-focused models. The most capable of these available to the public, $ALLaM-34B$, was subsequently adopted by HUMAIN, who developed and deployed HUMAIN Chat, a closed conversational web service built on this model. This paper presents an expanded and refined UI-level evaluation of $ALLaM-34B$. Using a prompt pack spanning modern standard Arabic, five regional dialects, code-switching, factual knowledge, arithmetic and temporal reasoning, creative generation, and adversarial safety, we collected 115 outputs (23 prompts times 5 runs) and scored each with three frontier LLM judges (GPT-5, Gemini 2.5 Pro, Claude Sonnet-4). We compute category-level means with 95% confidence intervals, analyze score distributions, and visualize dialect-wise metric heat maps. The updated analysis reveals consistently high performance on generation and code-switching tasks (both averaging 4.92/5), alongside strong results in MSA handling (4.74/5), solid reasoning ability (4.64/5), and improved dialect fidelity (4.21/5). Safety-related prompts show stable, reliable performance of (4.54/5). Taken together, these results position $ALLaM-34B$ as a robust and culturally grounded Arabic LLM, demonstrating both technical strength and practical readiness for real-world deployment.

Method: Combines static code analysis, designer interrogation, literature mining, and persona-driven adversarial test generation with adaptive difficulty based on judge feedback. Uses LLM-as-a-Judge rubric to score dialogues and steer tests toward weakest capabilities.

Result: ATA surfaces more diverse and severe failures than expert annotators, completes testing in 20-30 minutes vs days for human rounds, and ablation studies show code analysis and web search reduce variance and miscalibration.

Conclusion: ATA provides efficient, evidence-grounded automated testing for LLM agents, outputting quantitative metrics and qualitative bug reports, with open-source implementation available for reproducible testing.

Abstract: LLM agents are increasingly deployed to plan, retrieve, and write with tools, yet evaluation still leans on static benchmarks and small human studies. We present the Agent-Testing Agent (ATA), a meta-agent that combines static code analysis, designer interrogation, literature mining, and persona-driven adversarial test generation whose difficulty adapts via judge feedback. Each dialogue is scored with an LLM-as-a-Judge (LAAJ) rubric and used to steer subsequent tests toward the agent’s weakest capabilities. On a travel planner and a Wikipedia writer, the ATA surfaces more diverse and severe failures than expert annotators while matching severity, and finishes in 20–30 minutes versus ten-annotator rounds that took days. Ablating code analysis and web search increases variance and miscalibration, underscoring the value of evidence-grounded test generation. The ATA outputs quantitative metrics and qualitative bug reports for developers. We release the full methodology and open-source implementation for reproducible agent testing: https://github.com/KhalilMrini/Agent-Testing-Agent

Method: Created DashboardQA benchmark with 112 interactive dashboards from Tableau Public and 405 question-answer pairs across five categories: multiple-choice, factoid, hypothetical, multi-dashboard, and conversational.

Result: Top-performing agents achieved low accuracy (Gemini-Pro-2.5: 38.69%, OpenAI CUA: 22.69%), showing significant challenges in grounding dashboard elements, planning interactions, and reasoning.

Conclusion: Interactive dashboard reasoning is highly challenging for current VLMs, demonstrating the need for improved GUI agent capabilities and establishing DashboardQA as a valuable benchmark for future research.

Abstract: Dashboards are powerful visualization tools for data-driven decision-making, integrating multiple interactive views that allow users to explore, filter, and navigate data. Unlike static charts, dashboards support rich interactivity, which is essential for uncovering insights in real-world analytical workflows. However, existing question-answering benchmarks for data visualizations largely overlook this interactivity, focusing instead on static charts. This limitation severely constrains their ability to evaluate the capabilities of modern multimodal agents designed for GUI-based reasoning. To address this gap, we introduce DashboardQA, the first benchmark explicitly designed to assess how vision-language GUI agents comprehend and interact with real-world dashboards. The benchmark includes 112 interactive dashboards from Tableau Public and 405 question-answer pairs with interactive dashboards spanning five categories: multiple-choice, factoid, hypothetical, multi-dashboard, and conversational. By assessing a variety of leading closed- and open-source GUI agents, our analysis reveals their key limitations, particularly in grounding dashboard elements, planning interaction trajectories, and performing reasoning. Our findings indicate that interactive dashboard reasoning is a challenging task overall for all the VLMs evaluated. Even the top-performing agents struggle; for instance, the best agent based on Gemini-Pro-2.5 achieves only 38.69% accuracy, while the OpenAI CUA agent reaches just 22.69%, demonstrating the benchmark’s significant difficulty. We release DashboardQA at https://github.com/vis-nlp/DashboardQA

Method: A two-pronged approach: dynamically prompting GPT-4o-mini with in-context examples or directly retrieving the closest normalization from the training dataset.

Result: Ranked near the top for most monolingual tracks, achieving 1st place in 7 out of 13 languages, but underperformed in zero-shot cross-lingual settings.

Conclusion: The proposed solution is effective for monolingual claim normalization but has limitations in zero-shot scenarios, indicating need for better cross-lingual generalization.

Abstract: Claim normalization is an integral part of any automatic fact-check verification system. It parses the typically noisy claim data, such as social media posts into normalized claims, which are then fed into downstream veracity classification tasks. The CheckThat! 2025 Task 2 focuses specifically on claim normalization and spans 20 languages under monolingual and zero-shot conditions. Our proposed solution consists of a lightweight \emph{retrieval-first, LLM-backed} pipeline, in which we either dynamically prompt a GPT-4o-mini with in-context examples, or retrieve the closest normalization from the train dataset directly. On the official test set, the system ranks near the top for most monolingual tracks, achieving first place in 7 out of of the 13 languages. In contrast, the system underperforms in the zero-shot setting, highlighting the limitation of the proposed solution.

Method: Created a human-annotated paraphrase corpus with 8,000 Marathi sentence pairs (Paraphrase/Non-paraphrase labels). Evaluated standard transformer-based BERT models on this dataset.

Result: Presented the first high-quality paraphrase dataset for Marathi with 8,000 expert-annotated sentence pairs. Provided baseline results from BERT model evaluations.

Conclusion: The L3Cube-MahaParaphrase Dataset addresses the data scarcity problem for Marathi NLP tasks and enables development of paraphrase detection systems for this low-resource Indic language.

Abstract: Paraphrases are a vital tool to assist language understanding tasks such as question answering, style transfer, semantic parsing, and data augmentation tasks. Indic languages are complex in natural language processing (NLP) due to their rich morphological and syntactic variations, diverse scripts, and limited availability of annotated data. In this work, we present the L3Cube-MahaParaphrase Dataset, a high-quality paraphrase corpus for Marathi, a low resource Indic language, consisting of 8,000 sentence pairs, each annotated by human experts as either Paraphrase (P) or Non-paraphrase (NP). We also present the results of standard transformer-based BERT models on these datasets. The dataset and model are publicly shared at https://github.com/l3cube-pune/MarathiNLP

Method: Introduces DuET-PD framework for multi-turn stance-change evaluation across persuasion type (corrective/misleading) and domain (knowledge/safety). Proposes Holistic DPO training approach that balances positive and negative persuasion examples.

Result: GPT-4o achieves only 27.32% accuracy under sustained misleading persuasion. Newer open-source models show increasing sycophancy. Holistic DPO significantly improves robustness, boosting Llama-3.1-8B-Instruct from 4.21% to 76.54% accuracy in safety contexts.

Conclusion: The framework and training approach provide a pathway to develop more reliable and adaptable LLMs for multi-turn dialogue, addressing critical trust issues in persuasive interactions.

Abstract: Large Language Models (LLMs) can struggle to balance gullibility to misinformation and resistance to valid corrections in persuasive dialogues, a critical challenge for reliable deployment. We introduce DuET-PD (Dual Evaluation for Trust in Persuasive Dialogues), a framework evaluating multi-turn stance-change dynamics across dual dimensions: persuasion type (corrective/misleading) and domain (knowledge via MMLU-Pro, and safety via SALAD-Bench). We find that even a state-of-the-art model like GPT-4o achieves only 27.32% accuracy in MMLU-Pro under sustained misleading persuasions. Moreover, results reveal a concerning trend of increasing sycophancy in newer open-source models. To address this, we introduce Holistic DPO, a training approach balancing positive and negative persuasion examples. Unlike prompting or resist-only training, Holistic DPO enhances both robustness to misinformation and receptiveness to corrections, improving Llama-3.1-8B-Instruct’s accuracy under misleading persuasion in safety contexts from 4.21% to 76.54%. These contributions offer a pathway to developing more reliable and adaptable LLMs for multi-turn dialogue. Code is available at https://github.com/Social-AI-Studio/DuET-PD.

Method: Analyzed impact of three VMWE categories on English-to-multiple-language translation using established datasets and extracted sentences from MT datasets. Proposed LLM-based paraphrasing to replace VMWEs with literal counterparts.

Result: Experimental results consistently showed that VMWEs negatively affect translation quality. The proposed paraphrasing approach demonstrated significant improvement in translation quality for verbal idioms and verb-particle constructions.

Conclusion: VMWEs remain a challenge for machine translation systems, but targeted paraphrasing approaches can effectively mitigate their negative impact on translation quality.

Abstract: Verbal multiword expressions (VMWEs) present significant challenges for natural language processing due to their complex and often non-compositional nature. While machine translation models have seen significant improvement with the advent of language models in recent years, accurately translating these complex linguistic structures remains an open problem. In this study, we analyze the impact of three VMWE categories – verbal idioms, verb-particle constructions, and light verb constructions – on machine translation quality from English to multiple languages. Using both established multiword expression datasets and sentences containing these language phenomena extracted from machine translation datasets, we evaluate how state-of-the-art translation systems handle these expressions. Our experimental results consistently show that VMWEs negatively affect translation quality. We also propose an LLM-based paraphrasing approach that replaces these expressions with their literal counterparts, demonstrating significant improvement in translation quality for verbal idioms and verb-particle constructions.

Method: Uses TF-IDF-based sentence ranking to select most informative sentences, enabling zero-shot adaptation without model architecture changes. Evaluates three context reduction strategies on MahaNews Marathi news dataset.

Result: Retaining only top 50% ranked sentences maintains comparable classification performance to full-document inference while reducing inference time by up to 35%.

Conclusion: Sentence ranking is a simple yet effective technique for scalable and efficient zero-shot long document classification, enabling adaptation of short-text models to long documents without accuracy loss.

Abstract: Transformer-based models like BERT excel at short text classification but struggle with long document classification (LDC) due to input length limitations and computational inefficiencies. In this work, we propose an efficient, zero-shot approach to LDC that leverages sentence ranking to reduce input context without altering the model architecture. Our method enables the adaptation of models trained on short texts, such as headlines, to long-form documents by selecting the most informative sentences using a TF-IDF-based ranking strategy. Using the MahaNews dataset of long Marathi news articles, we evaluate three context reduction strategies that prioritize essential content while preserving classification accuracy. Our results show that retaining only the top 50% ranked sentences maintains performance comparable to full-document inference while reducing inference time by up to 35%. This demonstrates that sentence ranking is a simple yet effective technique for scalable and efficient zero-shot LDC.

Method: Drawing from multiple disciplines, the authors propose a framework where anthropomorphism emerges from interaction between designers (embedding cues) and interpreters (responding to cues). They categorize cues into four dimensions: perceptive, linguistic, behavioral, and cognitive.

Result: The paper provides a unified taxonomy with actionable levers for practitioners to intentionally design anthropomorphic characteristics in LLMs. It also advocates for function-oriented evaluations of anthropomorphic design.

Conclusion: Anthropomorphism in LLMs should be treated as a deliberate design concept that can be tuned through specific cue dimensions to create more effective and goal-oriented human-AI interactions, moving beyond risk-focused perspectives.

Abstract: Large Language Models (LLMs) increasingly exhibit \textbf{anthropomorphism} characteristics – human-like qualities portrayed across their outlook, language, behavior, and reasoning functions. Such characteristics enable more intuitive and engaging human-AI interactions. However, current research on anthropomorphism remains predominantly risk-focused, emphasizing over-trust and user deception while offering limited design guidance. We argue that anthropomorphism should instead be treated as a \emph{concept of design} that can be intentionally tuned to support user goals. Drawing from multiple disciplines, we propose that the anthropomorphism of an LLM-based artifact should reflect the interaction between artifact designers and interpreters. This interaction is facilitated by cues embedded in the artifact by the designers and the (cognitive) responses of the interpreters to the cues. Cues are categorized into four dimensions: \textit{perceptive, linguistic, behavioral}, and \textit{cognitive}. By analyzing the manifestation and effectiveness of each cue, we provide a unified taxonomy with actionable levers for practitioners. Consequently, we advocate for function-oriented evaluations of anthropomorphic design.

Method: Developed a two-headed RieszNet-based neural network architecture for better treatment effect estimation. Replicated and extended Bansal et al’s work on regularizing text classifiers, focusing on semi-synthetic IMDB movie reviews data.

Result: Achieved 2-3x reduction in MAE of effect estimates compared to previous work. Ensemble models showed that presence of the word “love” causes a +2.9% increase in probability of positive sentiment in movie reviews.

Conclusion: The CausalSent framework successfully improves causal effect estimation accuracy in NLP models and provides interpretable insights into how specific text features causally influence model predictions.

Abstract: Despite the overwhelming performance improvements offered by recent natural language procesing (NLP) models, the decisions made by these models are largely a black box. Towards closing this gap, the field of causal NLP combines causal inference literature with modern NLP models to elucidate causal effects of text features. We replicate and extend Bansal et al’s work on regularizing text classifiers to adhere to estimated effects, focusing instead on model interpretability. Specifically, we focus on developing a two-headed RieszNet-based neural network architecture which achieves better treatment effect estimation accuracy. Our framework, CausalSent, accurately predicts treatment effects in semi-synthetic IMDB movie reviews, reducing MAE of effect estimates by 2-3x compared to Bansal et al’s MAE on synthetic Civil Comments data. With an ensemble of validated models, we perform an observational case study on the causal effect of the word “love” in IMDB movie reviews, finding that the presence of the word “love” causes a +2.9% increase in the probability of a positive sentiment.

Method: UQ collects 500 unsolved questions from Stack Exchange using a pipeline with rule-based filters, LLM judges, and human review. It employs compound validation strategies (UQ-Validators) that leverage the generator-validator gap and provides an open platform (UQ-Platform) for expert community verification of questions and solutions.

Result: The benchmark is highly challenging - the top model passes UQ-validation on only 15% of questions. Preliminary human verification has already identified correct answers among those that passed validation, demonstrating the benchmark’s effectiveness.

Conclusion: UQ provides a new paradigm for evaluating frontier models on real-world, open-ended challenges where success actually pushes the frontier of human knowledge, offering both difficulty and real-world relevance by construction.

Abstract: Benchmarks shape progress in AI research. A useful benchmark should be both difficult and realistic: questions should challenge frontier models while also reflecting real-world usage. Yet, current paradigms face a difficulty-realism tension: exam-style benchmarks are often made artificially difficult with limited real-world value, while benchmarks based on real user interaction often skew toward easy, high-frequency problems. In this work, we explore a radically different paradigm: assessing models on unsolved questions. Rather than a static benchmark scored once, we curate unsolved questions and evaluate models asynchronously over time with validator-assisted screening and community verification. We introduce UQ, a testbed of 500 challenging, diverse questions sourced from Stack Exchange, spanning topics from CS theory and math to sci-fi and history, probing capabilities including reasoning, factuality, and browsing. UQ is difficult and realistic by construction: unsolved questions are often hard and naturally arise when humans seek answers, thus solving them yields direct real-world value. Our contributions are threefold: (1) UQ-Dataset and its collection pipeline combining rule-based filters, LLM judges, and human review to ensure question quality (e.g., well-defined and difficult); (2) UQ-Validators, compound validation strategies that leverage the generator-validator gap to provide evaluation signals and pre-screen candidate solutions for human review; and (3) UQ-Platform, an open platform where experts collectively verify questions and solutions. The top model passes UQ-validation on only 15% of questions, and preliminary human verification has already identified correct answers among those that passed. UQ charts a path for evaluating frontier models on real-world, open-ended challenges, where success pushes the frontier of human knowledge. We release UQ at https://uq.stanford.edu.

Method: Comprehensive empirical analysis of three pruning methods and calibration sets across three LLMs using four fairness metrics. Proposed HGLA pruning identifies and removes parameters redundant for input processing but influential in output generation.

Result: Pruning methods have greater impact on fairness than calibration sets. HGLA better maintains or improves fairness compared to existing methods, with human evaluation confirming HGLA outputs are fairer.

Conclusion: HGLA pruning shows promise for maintaining fairness across models and tasks where traditional pruning methods have limitations, offering a better approach for fair opinion summarization.

Abstract: Model compression through post-training pruning offers a way to reduce model size and computational requirements without significantly impacting model performance. However, the effect of pruning on the fairness of LLM-generated summaries remains unexplored, particularly for opinion summarisation where biased outputs could influence public views.In this paper, we present a comprehensive empirical analysis of opinion summarisation, examining three state-of-the-art pruning methods and various calibration sets across three open-source LLMs using four fairness metrics. Our systematic analysis reveals that pruning methods have a greater impact on fairness than calibration sets. Building on these insights, we propose High Gradient Low Activation (HGLA) pruning, which identifies and removes parameters that are redundant for input processing but influential in output generation. Our experiments demonstrate that HGLA can better maintain or even improve fairness compared to existing methods, showing promise across models and tasks where traditional methods have limitations. Our human evaluation shows HGLA-generated outputs are fairer than existing state-of-the-art pruning methods. Code is available at: https://github.com/amberhuang01/HGLA.

Method: Flexible Activation Steering with Backtracking (FASB) tracks LLM internal states during generation to dynamically determine intervention necessity and strength, with backtracking mechanism to correct deviated tokens.

Result: Extensive experiments on TruthfulQA and six multiple-choice datasets show FASB outperforms baseline methods.

Conclusion: FASB provides an effective and cost-efficient approach for aligning LLM behaviors by dynamically controlling interventions based on real-time generation states and correcting deviations through backtracking.

Abstract: Large language models (LLMs) have achieved remarkable performance across many generation tasks. Nevertheless, effectively aligning them with desired behaviors remains a significant challenge. Activation steering is an effective and cost-efficient approach that directly modifies the activations of LLMs during the inference stage, aligning their responses with the desired behaviors and avoiding the high cost of fine-tuning. Existing methods typically indiscriminately intervene to all generations or rely solely on the question to determine intervention, which limits the accurate assessment of the intervention strength. To this end, we propose the Flexible Activation Steering with Backtracking (FASB) framework, which dynamically determines both the necessity and strength of intervention by tracking the internal states of the LLMs during generation, considering both the question and the generated content. Since intervening after detecting a deviation from the desired behavior is often too late, we further propose the backtracking mechanism to correct the deviated tokens and steer the LLMs toward the desired behavior. Extensive experiments on the TruthfulQA dataset and six multiple-choice datasets demonstrate that our method outperforms baselines. Our code will be released at https://github.com/gjw185/FASB.

Method: The authors categorize reasoning into three stages (insufficient exploration, compensatory reasoning, and convergence), mine sensitive RCP patterns, and develop a lightweight thresholding strategy based on heuristic rules to detect when compensatory reasoning ends.

Result: Experimental evaluations on AIME24, AIME25, and GPQA-D benchmarks show the method reduces token consumption while preserving or enhancing reasoning accuracy compared to previous approaches.

Conclusion: Detecting the Reasoning Completion Point effectively mitigates overthinking in LLMs, providing an efficient balance between reasoning quality and resource usage without requiring complex monitoring systems.

Abstract: Large language models (LLMs) enhance complex reasoning tasks by scaling the individual thinking process. However, prior work shows that overthinking can degrade overall performance. Motivated by observed patterns in thinking length and content length, we categorize reasoning into three stages: insufficient exploration stage, compensatory reasoning stage, and reasoning convergence stage. Typically, LLMs produce correct answers in the compensatory reasoning stage, whereas reasoning convergence often triggers overthinking, causing increased resource usage or even infinite loops. Therefore, mitigating overthinking hinges on detecting the end of the compensatory reasoning stage, defined as the Reasoning Completion Point (RCP). RCP typically appears at the end of the first complete reasoning cycle and can be identified by querying the LLM sentence by sentence or monitoring the probability of an end-of-thinking token (e.g., \texttt{}), though these methods lack an efficient and precise balance. To improve this, we mine more sensitive and consistent RCP patterns and develop a lightweight thresholding strategy based on heuristic rules. Experimental evaluations on benchmarks (AIME24, AIME25, GPQA-D) demonstrate that the proposed method reduces token consumption while preserving or enhancing reasoning accuracy.

Method: Weights-Rotated Preference Optimization (RoPO) algorithm that implicitly constrains output layer logits with KL divergence from DPO and explicitly constrains intermediate hidden states by fine-tuning on a multi-granularity orthogonal matrix.

Result: Achieves up to 3.27-point improvement on AlpacaEval 2, surpasses best baseline by 6.2-7.5 points on MT-Bench with only 0.015% trainable parameters.

Conclusion: RoPO effectively alleviates DPO’s reward hacking problem while retaining knowledge and expressive capabilities from pre-training and SFT stages.

Abstract: Despite the efficacy of Direct Preference Optimization (DPO) in aligning Large Language Models (LLMs), reward hacking remains a pivotal challenge. This issue emerges when LLMs excessively reduce the probability of rejected completions to achieve high rewards, without genuinely meeting their intended goals. As a result, this leads to overly lengthy generation lacking diversity, as well as catastrophic forgetting of knowledge. We investigate the underlying reason behind this issue, which is representation redundancy caused by neuron collapse in the parameter space. Hence, we propose a novel Weights-Rotated Preference Optimization (RoPO) algorithm, which implicitly constrains the output layer logits with the KL divergence inherited from DPO and explicitly constrains the intermediate hidden states by fine-tuning on a multi-granularity orthogonal matrix. This design prevents the policy model from deviating too far from the reference model, thereby retaining the knowledge and expressive capabilities acquired during pre-training and SFT stages. Our RoPO achieves up to a 3.27-point improvement on AlpacaEval 2, and surpasses the best baseline by 6.2 to 7.5 points on MT-Bench with merely 0.015% of the trainable parameters, demonstrating its effectiveness in alleviating the reward hacking problem of DPO.

Method: Created SurveyGen dataset with 4,200+ human surveys across domains, developed QUAL-SG quality-aware framework that enhances RAG with quality indicators for source paper selection, and systematically evaluated LLMs under varying human involvement levels.

Result: Semi-automatic pipelines achieved partially competitive outcomes, but fully automatic survey generation still suffers from low citation quality and limited critical analysis.

Conclusion: While LLMs show promise in survey generation, human involvement remains crucial for achieving high-quality results, particularly for citation quality and critical analysis aspects.

Abstract: Automatic survey generation has emerged as a key task in scientific document processing. While large language models (LLMs) have shown promise in generating survey texts, the lack of standardized evaluation datasets critically hampers rigorous assessment of their performance against human-written surveys. In this work, we present SurveyGen, a large-scale dataset comprising over 4,200 human-written surveys across diverse scientific domains, along with 242,143 cited references and extensive quality-related metadata for both the surveys and the cited papers. Leveraging this resource, we build QUAL-SG, a novel quality-aware framework for survey generation that enhances the standard Retrieval-Augmented Generation (RAG) pipeline by incorporating quality-aware indicators into literature retrieval to assess and select higher-quality source papers. Using this dataset and framework, we systematically evaluate state-of-the-art LLMs under varying levels of human involvement - from fully automatic generation to human-guided writing. Experimental results and human evaluations show that while semi-automatic pipelines can achieve partially competitive outcomes, fully automatic survey generation still suffers from low citation quality and limited critical analysis.

Method: CoCoA uses token-level confidence-aware measures (entropy gap and contextual peakedness) and generalized divergence between parametric and contextual distributions to adaptively resolve conflicts during generation.

Result: State-of-the-art performance across multiple LLMs on QA, Summarization, and LFQA benchmarks, with up to 9.2 point accuracy gains over AdaCAD baseline and up to 2.5 point improvements in factuality.

Conclusion: CoCoA enables more informed, context-aware token generation with superior sensitivity to conflict variations while maintaining strong performance in both high and low conflict settings.

Abstract: Faithful generation in large language models (LLMs) is challenged by knowledge conflicts between parametric memory and external context. Existing contrastive decoding methods tuned specifically to handle conflict often lack adaptability and can degrade performance in low conflict settings. We introduce CoCoA (Confidence- and Context-Aware Adaptive Decoding), a novel token-level algorithm for principled conflict resolution and enhanced faithfulness. CoCoA resolves conflict by utilizing confidence-aware measures (entropy gap and contextual peakedness) and the generalized divergence between the parametric and contextual distributions. Crucially, CoCoA maintains strong performance even in low conflict settings. Extensive experiments across multiple LLMs on diverse Question Answering (QA), Summarization, and Long-Form Question Answering (LFQA) benchmarks demonstrate CoCoA’s state-of-the-art performance over strong baselines like AdaCAD. It yields significant gains in QA accuracy, up to 9.2 points on average compared to the strong baseline AdaCAD, and improves factuality in summarization and LFQA by up to 2.5 points on average across key benchmarks. Additionally, it demonstrates superior sensitivity to conflict variations. CoCoA enables more informed, context-aware, and ultimately more faithful token generation.

Method: Proposed TNT-OOD model with: 1) cross-attention module to fuse local structure into node-level text representations, and 2) HyperNetwork to generate node-specific transformation parameters for aligning topological and semantic features.

Result: Experiments on 11 datasets across four OOD scenarios demonstrate the framework’s effectiveness in evaluating OOD detection challenges in text-rich networks.

Conclusion: The TextTopoOOD framework provides comprehensive evaluation across diverse OOD scenarios, and TNT-OOD effectively models text-topology interplay for improved ID/OOD distinction in text-rich networks.

Abstract: Out-of-distribution (OOD) detection remains challenging in text-rich networks, where textual features intertwine with topological structures. Existing methods primarily address label shifts or rudimentary domain-based splits, overlooking the intricate textual-structural diversity. For example, in social networks, where users represent nodes with textual features (name, bio) while edges indicate friendship status, OOD may stem from the distinct language patterns between bot and normal users. To address this gap, we introduce the TextTopoOOD framework for evaluating detection across diverse OOD scenarios: (1) attribute-level shifts via text augmentations and embedding perturbations; (2) structural shifts through edge rewiring and semantic connections; (3) thematically-guided label shifts; and (4) domain-based divisions. Furthermore, we propose TNT-OOD to model the complex interplay between Text aNd Topology using: 1) a novel cross-attention module to fuse local structure into node-level text representations, and 2) a HyperNetwork to generate node-specific transformation parameters. This aligns topological and semantic features of ID nodes, enhancing ID/OOD distinction across structural and textual shifts. Experiments on 11 datasets across four OOD scenarios demonstrate the nuanced challenge of TextTopoOOD for evaluating OOD detection in text-rich networks.

Method: The framework incorporates: (1) medical terminology attention mechanism, (2) comprehensive assessment architecture evaluating clarity, specificity, clinical relevance, and factual accuracy, (3) component-level evolutionary algorithm preserving clinical reasoning integrity, and (4) semantic verification module for medical knowledge adherence.

Result: Evaluation across diagnostic, therapeutic, and educational tasks shows: 24.7% reduction in factually incorrect content, 19.6% enhancement in domain specificity, and 15.3% higher clinician preference in blinded evaluations.

Conclusion: EMPOWER addresses critical challenges in developing clinically appropriate prompts, facilitating more responsible integration of LLMs into healthcare settings by ensuring better medical knowledge adherence and safety.

Abstract: Prompt engineering significantly influences the reliability and clinical utility of Large Language Models (LLMs) in medical applications. Current optimization approaches inadequately address domain-specific medical knowledge and safety requirements. This paper introduces EMPOWER, a novel evolutionary framework that enhances medical prompt quality through specialized representation learning, multi-dimensional evaluation, and structure-preserving algorithms. Our methodology incorporates: (1) a medical terminology attention mechanism, (2) a comprehensive assessment architecture evaluating clarity, specificity, clinical relevance, and factual accuracy, (3) a component-level evolutionary algorithm preserving clinical reasoning integrity, and (4) a semantic verification module ensuring adherence to medical knowledge. Evaluation across diagnostic, therapeutic, and educational tasks demonstrates significant improvements: 24.7% reduction in factually incorrect content, 19.6% enhancement in domain specificity, and 15.3% higher clinician preference in blinded evaluations. The framework addresses critical challenges in developing clinically appropriate prompts, facilitating more responsible integration of LLMs into healthcare settings.

Method: Experimental approach that maintains total parameter count by increasing FFN dimensions in some layers while completely removing FFNs from other layers. Models trained from scratch with varying sizes (285M, 570M, 1.2B parameters) and layer counts (12, 24, 40 layers).

Result: Concentrating FFNs in 70% of consecutive middle layers consistently outperforms standard configurations across multiple downstream tasks.

Conclusion: FFN importance varies by layer position, and optimal performance is achieved by focusing FFN capacity in the middle layers rather than distributing uniformly across all layers.

Abstract: This study investigates the layerwise importance of feed-forward networks (FFNs) in Transformer-based language models during pretraining. We introduce an experimental approach that, while maintaining the total parameter count, increases the FFN dimensions in some layers and completely removes the FFNs from other layers. Furthermore, since our focus is on the importance of FFNs during pretraining, we train models from scratch to examine whether the importance of FFNs varies depending on their layer positions, rather than using publicly available pretrained models as is frequently done. Through comprehensive evaluations of models with varying sizes (285M, 570M, and 1.2B parameters) and layer counts (12, 24, and 40 layers), we demonstrate that concentrating FFNs in 70% of the consecutive middle layers consistently outperforms standard configurations for multiple downstream tasks.

Method: Proposes dynamic validation set that evolves with test set, replacing static validation. Introduces SMITE iterative algorithm to select optimal in-context examples, with each set validated against corresponding dynamic validation set

Result: Experiments across four different LLMs show significant improvements in both predictive accuracy and fairness compared to baseline methods

Conclusion: First study to apply dynamic validation in in-context learning for LLMs, demonstrating effectiveness for enhancing both performance and fairness

Abstract: Large Language Models (LLMs) are widely used for downstream tasks such as tabular classification, where ensuring fairness in their outputs is critical for inclusivity, equal representation, and responsible AI deployment. This study introduces a novel approach to enhancing LLM performance and fairness through the concept of a dynamic validation set, which evolves alongside the test set, replacing the traditional static validation approach. We also propose an iterative algorithm, SMITE, to select optimal in-context examples, with each example set validated against its corresponding dynamic validation set. The in-context set with the lowest total error is used as the final demonstration set. Our experiments across four different LLMs show that our proposed techniques significantly improve both predictive accuracy and fairness compared to baseline methods. To our knowledge, this is the first study to apply dynamic validation in the context of in-context learning for LLMs.

Method: Train neural network classifier on curated copyrighted dataset to examine LLMs’ internal states pre-generation. Integrate with RAG system for early intervention by stopping generation or altering outputs.

Result: Analysis of internal states effectively mitigates copyrighted data leakage risk. Provides scalable solution that integrates smoothly into AI workflows while maintaining text quality.

Conclusion: Proactive internal state monitoring offers effective copyright compliance and data privacy protection for LLMs, enabling ethical AI deployment without compromising generation quality.

Abstract: Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP) but pose risks of inadvertently exposing copyrighted or proprietary data, especially when such data is used for training but not intended for distribution. Traditional methods address these leaks only after content is generated, which can lead to the exposure of sensitive information. This study introduces a proactive approach: examining LLMs’ internal states before text generation to detect potential leaks. By using a curated dataset of copyrighted materials, we trained a neural network classifier to identify risks, allowing for early intervention by stopping the generation process or altering outputs to prevent disclosure. Integrated with a Retrieval-Augmented Generation (RAG) system, this framework ensures adherence to copyright and licensing requirements while enhancing data privacy and ethical standards. Our results show that analyzing internal states effectively mitigates the risk of copyrighted data leakage, offering a scalable solution that fits smoothly into AI workflows, ensuring compliance with copyright regulations while maintaining high-quality text generation. The implementation is available on GitHub.\footnote{https://github.com/changhu73/Internal_states_leakage}

Method: 1) Formal definition and taxonomy of PoC tokens based on GPT’s vocabulary 2) Building a PoC token detector by fine-tuning an LLM to label tokens considering both semantics and search engine content 3) Studying training data pollution through PoC token appearances (token IDs)

Result: Experiments on GPT and 23 other LLMs show widespread PoC token existence, with GPT’s vocabulary performing worst - over 23% of long Chinese tokens relate to pornography or online gambling. Validation on datasets like C4 and Pile confirms speculation accuracy. For GPT-4o, about 0.5% of training data appears to be “Yui Hatano” related webpages.

Conclusion: Polluted Chinese tokens are prevalent across LLMs, particularly in GPT models, indicating potential training data contamination issues that require attention in model development and data curation processes.

Abstract: Tokens are basic elements in the datasets for LLM training. It is well-known that many tokens representing Chinese phrases in the vocabulary of GPT (4o/4o-mini/o1/o3/4.5/4.1/o4-mini) are indicating contents like pornography or online gambling. Based on this observation, our goal is to locate Polluted Chinese (PoC) tokens in LLMs and study the relationship between PoC tokens' existence and training data. (1) We give a formal definition and taxonomy of PoC tokens based on the GPT’s vocabulary. (2) We build a PoC token detector via fine-tuning an LLM to label PoC tokens in vocabularies by considering each token’s both semantics and related contents from the search engines. (3) We study the speculation on the training data pollution via PoC tokens' appearances (token ID). Experiments on GPT and other 23 LLMs indicate that tokens widely exist while GPT’s vocabulary behaves the worst: more than 23% long Chinese tokens (i.e., a token with more than two Chinese characters) are either porn or online gambling. We validate the accuracy of our speculation method on famous pre-training datasets like C4 and Pile. Then, considering GPT-4o, we speculate that the ratio of “Yui Hatano” related webpages in GPT-4o’s training data is around 0.5%.

Method: Dynamic Reasoning Quota Allocation (DRQA) leverages batch-generated preference data and reinforcement learning to train models to allocate reasoning resources adaptively, encouraging concise responses for simple questions while maintaining depth for complex ones.

Result: Extensive experiments show DRQA significantly reduces token usage while maintaining or improving answer accuracy across mathematical and scientific reasoning benchmarks.

Conclusion: DRQA effectively mitigates overthinking in reasoning LLMs and offers a promising direction for more efficient and scalable deployment, inspiring further exploration into fine-grained control of reasoning behaviors.

Abstract: Reasoning large language models (RLLMs), such as OpenAI-O3 and DeepSeek-R1, have recently demonstrated remarkable capabilities by performing structured and multi-step reasoning. However, recent studies reveal that RLLMs often suffer from overthinking, i.e., producing unnecessarily lengthy reasoning chains even for simple questions, leading to excessive token consumption and computational inefficiency. Interestingly, we observe that when processing multiple questions in batch mode, RLLMs exhibit more resource-efficient behavior by dynamically compressing reasoning steps for easier problems, due to implicit resource competition. Inspired by this, we propose Dynamic Reasoning Quota Allocation (DRQA), a novel method that transfers the benefits of resource competition from batch processing to single-question inference. Specifically, DRQA leverages batch-generated preference data and reinforcement learning to train the model to allocate reasoning resources adaptively. By encouraging the model to internalize a preference for responses that are both accurate and concise, DRQA enables it to generate concise answers for simple questions while retaining sufficient reasoning depth for more challenging ones. Extensive experiments on a wide range of mathematical and scientific reasoning benchmarks demonstrate that DRQA significantly reduces token usage while maintaining, and in many cases improving, answer accuracy. By effectively mitigating the overthinking problem, DRQA offers a promising direction for more efficient and scalable deployment of RLLMs, and we hope it inspires further exploration into fine-grained control of reasoning behaviors.

Method: Uses type dynamics theory from MBTI framework with three stages: life-situational stress analysis, group-level personality prediction, and self-weighted cognitive imitation. Applied to World Values Survey data.

Result: Outperforms existing baselines by 10% accuracy and reduces divergence between model predictions and human preferences.

Conclusion: MARK demonstrates strong potential for zero-shot personalization and helps social scientists better interpret model predictions in cultural value research.

Abstract: Introducing MARK, the Multi-stAge Reasoning frameworK for cultural value survey response simulation, designed to enhance the accuracy, steerability, and interpretability of large language models in this task. The system is inspired by the type dynamics theory in the MBTI psychological framework for personality research. It effectively predicts and utilizes human demographic information for simulation: life-situational stress analysis, group-level personality prediction, and self-weighted cognitive imitation. Experiments on the World Values Survey show that MARK outperforms existing baselines by 10% accuracy and reduces the divergence between model predictions and human preferences. This highlights the potential of our framework to improve zero-shot personalization and help social scientists interpret model predictions.

Method: Intermediate Layer Retrieval (ILRe) pipeline that determines an intermediate decoder layer offline, encodes context via streaming chunked prefill up to that layer, and recalls tokens using attention scores between input query and full key cache with multi-pooling kernels strategy.

Result: Processes 1M tokens in <30 seconds (180× speedup), scores ≈79.8 on RULER-1M benchmark with Llama-3.1-UltraLong-8B-1M-Instruct on Huawei Ascend 910B NPU, achieves performance comparable to or better than full context.

Conclusion: ILRe effectively mitigates long-context processing limitations without additional training or operator development, providing significant efficiency improvements while maintaining semantic completeness and performance.

Abstract: Large Language Models (LLMs) have demonstrated success across many benchmarks. However, they still exhibit limitations in long-context scenarios, primarily due to their short effective context length, quadratic computational complexity, and high memory overhead when processing lengthy inputs. To mitigate these issues, we introduce a novel context compression pipeline, called Intermediate Layer Retrieval (ILRe), which determines one intermediate decoder layer offline, encodes context by streaming chunked prefill only up to that layer, and recalls tokens by the attention scores between the input query and full key cache in that specified layer. In particular, we propose a multi-pooling kernels allocating strategy in the token recalling process to maintain the completeness of semantics. Our approach not only reduces the prefilling complexity from $O(L^2)$ to $O(L)$, but also achieves performance comparable to or better than the full context in the long context scenarios. Without additional post training or operator development, ILRe can process a single $1M$ tokens request in less than half a minute (speedup $\approx 180\times$) and scores RULER-$1M$ benchmark of $\approx 79.8$ with model Llama-3.1-UltraLong-8B-1M-Instruct on a Huawei Ascend 910B NPU.

Method: Proposes code-based unified knowledge representation using Python Pandas API, knowledge transfer with cross-task memory building, and adaptive reasoning correction through code execution feedback.

Result: Outperforms existing unified reasoning frameworks and competes effectively with task-specific methods across six benchmarks in three SKR tasks.

Conclusion: Pandora successfully addresses limitations of existing USKR methods by providing a unified framework that leverages code-based representation and knowledge transfer to achieve impressive cross-task reasoning capabilities.

Abstract: Unified Structured Knowledge Reasoning (USKR) aims to answer natural language questions by using structured sources such as tables, databases, and knowledge graphs in a unified way. Existing USKR methods rely on task-specific strategies or bespoke representations, which hinder their ability to dismantle barriers between different SKR tasks, thereby constraining their overall performance in cross-task scenarios. In this paper, we introduce \textsc{Pandora}, a novel USKR framework that addresses the limitations of existing methods by leveraging two key innovations. First, we propose a code-based unified knowledge representation using \textsc{Python}’s \textsc{Pandas} API, which aligns seamlessly with the pre-training of LLMs. This representation facilitates a cohesive approach to handling different structured knowledge sources. Building on this foundation, we employ knowledge transfer to bolster the unified reasoning process of LLMs by automatically building cross-task memory. By adaptively correcting reasoning using feedback from code execution, \textsc{Pandora} showcases impressive unified reasoning capabilities. Extensive experiments on six widely used benchmarks across three SKR tasks demonstrate that \textsc{Pandora} outperforms existing unified reasoning frameworks and competes effectively with task-specific methods.

Method: Used TIMIT corpus, extracted three feature types: MFCCs, MFCCs with formants, and CNN activations from Wav2Vec. Trained SVM classifiers for front-back vowel identification and compared classification accuracy.

Result: The paper compares classification performance across different feature representations to assess which best captures phonetic vowel information.

Conclusion: The study provides insights into the phonetic representation capabilities of modern CNN-based features compared to traditional acoustic features for vowel classification tasks.

Abstract: Automatic Speech Recognition has advanced with self-supervised learning, enabling feature extraction directly from raw audio. In Wav2Vec, a CNN first transforms audio into feature vectors before the transformer processes them. This study examines CNN-extracted information for monophthong vowels using the TIMIT corpus. We compare MFCCs, MFCCs with formants, and CNN activations by training SVM classifiers for front-back vowel identification, assessing their classification accuracy to evaluate phonetic representation.

Method: The paper analyzes the current state of information availability in different languages and examines various technological constraints related to multilingualism on the internet.

Result: Identifies a significant gap in internet accessibility for non-English speakers despite existing solutions, highlighting the continued dominance of English content online.

Conclusion: There is an urgent need to address multilingual barriers on the internet to make information accessible to the global population, requiring further technological solutions beyond current offerings.

Abstract: The usage of Internet has grown exponentially over the last two decades. The number of Internet users has grown from 16 Million to 1650 Million from 1995 to 2010. It has become a major repository of information catering almost every area. Since the Internet has its origin in USA which is English speaking country there is huge dominance of English on the World Wide Web. Although English is a globally acceptable language, still there is a huge population in the world which is not able to access the Internet due to language constraints. It has been estimated that only 20-25% of the world population speaks English as a native language. More and more people are accessing the Internet nowadays removing the cultural and linguistic barriers and hence there is a high growth in the number of non-English speaking users over the last few years on the Internet. Although many solutions have been provided to remove the linguistic barriers, still there is a huge gap to be filled. This paper attempts to analyze the need of information availability in different languages and the various technological constraints related to multi-linguism on the Internet.

Method: Extracts linguistic features, scores them, maps probabilistically, and iteratively refines results by identifying and generalizing patterns until convergence. Hybrid linguistic-statistical approach.

Result: Outperforms baseline methods on six Indo-European languages (English, German, French, Italian, Spanish, Portuguese), with strong performance gains in cross-linguistic data.

Conclusion: Proposed unsupervised method effectively detects loanwords using only language-internal information, demonstrating superior performance and scalability across multiple languages.

Abstract: We propose an unsupervised method for detecting loanwords i.e., words borrowed from one language into another. While prior work has primarily relied on language-external information to identify loanwords, such approaches can introduce circularity and constraints into the historical linguistics workflow. In contrast, our model relies solely on language-internal information to process both native and borrowed words in monolingual and multilingual wordlists. By extracting pertinent linguistic features, scoring them, and mapping them probabilistically, we iteratively refine initial results by identifying and generalizing from emerging patterns until convergence. This hybrid approach leverages both linguistic and statistical cues to guide the discovery process. We evaluate our method on the task of isolating loanwords in datasets from six standard Indo-European languages: English, German, French, Italian, Spanish, and Portuguese. Experimental results demonstrate that our model outperforms baseline methods, with strong performance gains observed when scaling to cross-linguistic data.

Method: Constructed a multi-task dataset by converting 19 argument mining datasets into unified format, then explored three training strategies: (1) fine-tuning on individual tasks, (2) joint multi-task fine-tuning, and (3) merging models fine-tuned separately on individual tasks using Llama-3.1-8B-Instruct.

Result: Task-specific fine-tuning significantly improved individual performance across all tasks. Multi-task fine-tuning maintained strong performance without degradation, showing effective transfer learning. Model merging provided competitive performance while reducing computational costs compared to full multi-task fine-tuning.

Conclusion: A single large language model can effectively handle multiple argument mining tasks through various fine-tuning strategies, with model merging offering a practical balance between performance and computational efficiency for multi-task argument mining applications.

Abstract: Argument mining is a subfield of argumentation that aims to automatically extract argumentative structures and their relations from natural language texts. This paper investigates how a single large language model can be leveraged to perform one or several argument mining tasks. Our contributions are two-fold. First, we construct a multi-task dataset by surveying and converting 19 well-known argument mining datasets from the literature into a unified format. Second, we explore various training strategies using Meta AI’s Llama-3.1-8B-Instruct model: (1) fine-tuning on individual tasks, (2) fine-tuning jointly on multiple tasks, and (3) merging models fine-tuned separately on individual tasks. Our experiments show that task-specific fine-tuning significantly improves individual performance across all tasks. Moreover, multi-task fine-tuning maintains strong performance without degradation, suggesting effective transfer learning across related tasks. Finally, we demonstrate that model merging offers a viable compromise: it yields competitive performance while mitigating the computational costs associated with full multi-task fine-tuning.

Method: Construct well-aligned cross-lingual latent space using autoencoder trained on parallel TED talk scripts, then apply debiasing techniques in this joint space rather than directly on LLM representations.

Result: Experiments with Aya-expanse and two debiasing techniques across four languages show autoencoders effectively create aligned cross-lingual space, and debiasing in this space significantly improves both overall performance and cross-lingual transferability.

Conclusion: Performing debiasing in a joint cross-lingual latent space rather than directly on LLM representations substantially enhances debiasing effectiveness and transferability across languages.

Abstract: Debiasing techniques such as SentDebias aim to reduce bias in large language models (LLMs). Previous studies have evaluated their cross-lingual transferability by directly applying these methods to LLM representations, revealing their limited effectiveness across languages. In this work, we therefore propose to perform debiasing in a joint latent space rather than directly on LLM representations. We construct a well-aligned cross-lingual latent space using an autoencoder trained on parallel TED talk scripts. Our experiments with Aya-expanse and two debiasing techniques across four languages (English, French, German, Dutch) demonstrate that a) autoencoders effectively construct a well-aligned cross-lingual latent space, and b) applying debiasing techniques in the learned cross-lingual latent space significantly improves both the overall debiasing performance and cross-lingual transferability.

Method: Conducted comprehensive experiments probing three key aspects: structural similarity between subword compositions and whole-word representations, sensitivity to semantic decomposability, and sensitivity to formal features like character sequence length across different layers.

Result: Identified three distinct groups among five LLM families based on their composition strategies, observed three distinct patterns in structural similarity evolution across layers, found great performance in semantic decomposition sensitivity, and identified three distinct patterns in formal feature sensitivity.

Conclusion: The findings provide valuable insights into the compositional dynamics of LLMs and highlight different compositional patterns in how LLMs encode and integrate subword information, revealing systematic differences in composition strategies across model families.

Abstract: Large language models (LLMs) take sequences of subwords as input, requiring them to effective compose subword representations into meaningful word-level representations. In this paper, we present a comprehensive set of experiments to probe how LLMs compose subword information, focusing on three key aspects: structural similarity, semantic decomposability, and form retention. Our analysis of the experiments suggests that these five LLM families can be classified into three distinct groups, likely reflecting difference in their underlying composition strategies. Specifically, we observe (i) three distinct patterns in the evolution of structural similarity between subword compositions and whole-word representations across layers; (ii) great performance when probing layer by layer their sensitivity to semantic decompositionality; and (iii) three distinct patterns when probing sensitivity to formal features, e.g., character sequence length. These findings provide valuable insights into the compositional dynamics of LLMs and highlight different compositional pattens in how LLMs encode and integrate subword information.

Method: Automatically synthesized dataset using GPT-4 with over 25,000 samples spanning five readability levels, rigorously evaluated through human and LLM-based judgments.

Result: Trained an open-source readability-controlled paraphrasing model that achieves state-of-the-art performance in German text simplification.

Conclusion: The German4All dataset and model are open-sourced to encourage further research on multi-level paraphrasing for creating reader-specific text adaptations.

Abstract: The ability to paraphrase texts across different complexity levels is essential for creating accessible texts that can be tailored toward diverse reader groups. Thus, we introduce German4All, the first large-scale German dataset of aligned readability-controlled, paragraph-level paraphrases. It spans five readability levels and comprises over 25,000 samples. The dataset is automatically synthesized using GPT-4 and rigorously evaluated through both human and LLM-based judgments. Using German4All, we train an open-source, readability-controlled paraphrasing model that achieves state-of-the-art performance in German text simplification, enabling more nuanced and reader-specific adaptations. We opensource both the dataset and the model to encourage further research on multi-level paraphrasing

Method: Created a manually annotated dataset of 10,814 multilingual customer reviews with 8 aspect categories and sentiment labels, then evaluated GPT-4 and LLaMA-3 performance on this dataset.

Result: Both models achieved over 85% accuracy in aspect-based sentiment analysis, with GPT-4 outperforming LLaMA-3 across all relevant metrics.

Conclusion: The study establishes a baseline for the new multilingual dataset and demonstrates strong performance of large language models in aspect-based sentiment analysis, with GPT-4 showing superior results.

Abstract: Aspect-based sentiment analysis enhances sentiment detection by associating it with specific aspects, offering deeper insights than traditional sentiment analysis. This study introduces a manually annotated dataset of 10,814 multilingual customer reviews covering brick-and-mortar retail stores, labeled with eight aspect categories and their sentiment. Using this dataset, the performance of GPT-4 and LLaMA-3 in aspect based sentiment analysis is evaluated to establish a baseline for the newly introduced data. The results show both models achieving over 85% accuracy, while GPT-4 outperforms LLaMA-3 overall with regard to all relevant metrics.

Method: The paper draws on measurement theory from social sciences to critically assess four core assumptions underlying LLJs: their ability to proxy human judgment, evaluation capabilities, scalability, and cost-effectiveness. It examines these through three application areas: text summarization, data annotation, and safety alignment.

Result: The analysis reveals that current LLJ practices may be challenged by inherent limitations of LLMs and current evaluation methodologies, suggesting that the assumptions behind LLJ adoption may not be fully justified.

Conclusion: The paper calls for more responsible evaluation practices in LLJ usage to ensure they support rather than undermine progress in natural language generation, emphasizing the need for rigorous validation before widespread adoption.

Abstract: Evaluating natural language generation (NLG) systems remains a core challenge of natural language processing (NLP), further complicated by the rise of large language models (LLMs) that aims to be general-purpose. Recently, large language models as judges (LLJs) have emerged as a promising alternative to traditional metrics, but their validity remains underexplored. This position paper argues that the current enthusiasm around LLJs may be premature, as their adoption has outpaced rigorous scrutiny of their reliability and validity as evaluators. Drawing on measurement theory from the social sciences, we identify and critically assess four core assumptions underlying the use of LLJs: their ability to act as proxies for human judgment, their capabilities as evaluators, their scalability, and their cost-effectiveness. We examine how each of these assumptions may be challenged by the inherent limitations of LLMs, LLJs, or current practices in NLG evaluation. To ground our analysis, we explore three applications of LLJs: text summarization, data annotation, and safety alignment. Finally, we highlight the need for more responsible evaluation practices in LLJs evaluation, to ensure that their growing role in the field supports, rather than undermines, progress in NLG.

Method: Evaluated weight and activation quantization strategies across nine benchmarks using probabilistic and generated text-based metrics. Tested models with different architectures and reasoning abilities, examining stereotypes, toxicity, sentiment, and fairness.

Result: Quantization reduces model toxicity and doesn’t significantly impact sentiment, but tends to slightly increase stereotypes and unfairness in generative tasks, especially under aggressive compression. Effects are consistent across demographics but vary in magnitude.

Conclusion: Quantization requires careful balancing of efficiency and ethical considerations, as it has nuanced impacts on different types of bias that depend on compression level and specific settings.

Abstract: This work presents a comprehensive evaluation of how quantization affects model bias, with particular attention to its impact on individual demographic subgroups. We focus on weight and activation quantization strategies and examine their effects across a broad range of bias types, including stereotypes, toxicity, sentiment, and fairness. We employ both probabilistic and generated text-based metrics across nine benchmarks and evaluate models varying in architecture family and reasoning ability. Our findings show that quantization has a nuanced impact on bias: while it can reduce model toxicity and does not significantly impact sentiment, it tends to slightly increase stereotypes and unfairness in generative tasks, especially under aggressive compression. These trends are generally consistent across demographic categories and model types, although their magnitude depends on the specific setting. Overall, our results highlight the importance of carefully balancing efficiency and ethical considerations when applying quantization in practice.

Method: Used supervised machine learning with: 1) conventional speech/language features, 2) emotional dimensions from Speech Emotion Recognition model, 3) exploratory speech feature analysis. Cross-corpus and cross-lingual analysis using English general population data and German MS patient data.

Result: Models detected depressive mood in MS patients with moderate generalizability (66% UAR). Feature selection improved performance to 74% UAR. Emotional changes were key indicators of depression in both general population and MS patients.

Conclusion: Speech-based depression detection can generalize to neurodegenerative disease contexts. Emotional features play a crucial role in detection. Provides initial evidence for cross-condition applicability of speech AI methods.

Abstract: Depression commonly co-occurs with neurodegenerative disorders like Multiple Sclerosis (MS), yet the potential of speech-based Artificial Intelligence for detecting depression in such contexts remains unexplored. This study examines the transferability of speech-based depression detection methods to people with MS (pwMS) through cross-corpus and cross-lingual analysis using English data from the general population and German data from pwMS. Our approach implements supervised machine learning models using: 1) conventional speech and language features commonly used in the field, 2) emotional dimensions derived from a Speech Emotion Recognition (SER) model, and 3) exploratory speech feature analysis. Despite limited data, our models detect depressive mood in pwMS with moderate generalisability, achieving a 66% Unweighted Average Recall (UAR) on a binary task. Feature selection further improved performance, boosting UAR to 74%. Our findings also highlight the relevant role emotional changes have as an indicator of depressive mood in both the general population and within PwMS. This study provides an initial exploration into generalising speech-based depression detection, even in the presence of co-occurring conditions, such as neurodegenerative diseases.

Method: Built benchmark using agricultural machine manual in 3 languages. Compared 9 long-context LLMs with direct prompting against 3 RAG strategies (keyword, semantic, hybrid). Used LLM-as-a-judge for evaluation with realistic QA scenarios.

Result: Hybrid RAG consistently outperformed direct long-context prompting. Gemini 2.5 Flash and smaller Qwen 2.5 7B achieved high accuracy (>85%) across all languages with RAG. Models showed strong cross-lingual performance.

Conclusion: RAG strategies, particularly hybrid approach, are more effective than direct long-context prompting for technical QA in specialized domains. The study provides an open framework for similar evaluations and highlights practical implementation trade-offs.

Abstract: We present a case study evaluating large language models (LLMs) with 128K-token context windows on a technical question answering (QA) task. Our benchmark is built on a user manual for an agricultural machine, available in English, French, and German. It simulates a cross-lingual information retrieval scenario where questions are posed in English against all three language versions of the manual. The evaluation focuses on realistic “needle-in-a-haystack” challenges and includes unanswerable questions to test for hallucinations. We compare nine long-context LLMs using direct prompting against three Retrieval-Augmented Generation (RAG) strategies (keyword, semantic, hybrid), with an LLM-as-a-judge for evaluation. Our findings for this specific manual show that Hybrid RAG consistently outperforms direct long-context prompting. Models like Gemini 2.5 Flash and the smaller Qwen 2.5 7B achieve high accuracy (over 85%) across all languages with RAG. This paper contributes a detailed analysis of LLM performance in a specialized industrial domain and an open framework for similar evaluations, highlighting practical trade-offs and challenges.

Method: Developed formal and causally grounded criteria for detecting planning, operationalized as a semi-automated annotation pipeline. Applied this to Gemma-2-2B models on MBPP code generation and poem generation tasks, comparing with Claude 3.5 Haiku.

Result: Planning is not universal - Gemma-2-2B uses improvisation for poem generation (unlike Haiku) and switches between planning/improvisation on MBPP tasks. Instruction tuning refines existing planning behaviors rather than creating new ones.

Conclusion: Provides a reproducible foundation for mechanistic studies of planning in LLMs, showing planning behaviors vary significantly across models and tasks rather than being a universal capability.

Abstract: Modern large language models (LLMs) have demonstrated impressive performance across a wide range of multi-step reasoning tasks. Recent work suggests that LLMs may perform planning - selecting a future target token in advance and generating intermediate tokens that lead towards it - rather than merely improvising one token at a time. However, existing studies assume fixed planning horizons and often focus on single prompts or narrow domains. To distinguish planning from improvisation across models and tasks, we present formal and causally grounded criteria for detecting planning and operationalize them as a semi-automated annotation pipeline. We apply this pipeline to both base and instruction-tuned Gemma-2-2B models on the MBPP code generation benchmark and a poem generation task where Claude 3.5 Haiku was previously shown to plan. Our findings show that planning is not universal: unlike Haiku, Gemma-2-2B solves the same poem generation task through improvisation, and on MBPP it switches between planning and improvisation across similar tasks and even successive token predictions. We further show that instruction tuning refines existing planning behaviors in the base model rather than creating them from scratch. Together, these studies provide a reproducible and scalable foundation for mechanistic studies of planning in LLMs.

Method: Proposes SentiMM - a multi-agent framework with specialized agents for text and visual processing, multimodal feature fusion, knowledge retrieval for context enrichment, and result aggregation for final sentiment classification.

Result: Extensive experiments show SentiMM achieves superior performance compared to state-of-the-art baselines, validating the effectiveness of the structured approach.

Conclusion: The proposed SentiMM framework successfully addresses multimodal sentiment analysis challenges through systematic multi-agent processing and knowledge integration, demonstrating significant performance improvements over existing methods.

Abstract: With the increasing prevalence of multimodal content on social media, sentiment analysis faces significant challenges in effectively processing heterogeneous data and recognizing multi-label emotions. Existing methods often lack effective cross-modal fusion and external knowledge integration. We propose SentiMM, a novel multi-agent framework designed to systematically address these challenges. SentiMM processes text and visual inputs through specialized agents, fuses multimodal features, enriches context via knowledge retrieval, and aggregates results for final sentiment classification. We also introduce SentiMMD, a large-scale multimodal dataset with seven fine-grained sentiment categories. Extensive experiments demonstrate that SentiMM achieves superior performance compared to state-of-the-art baselines, validating the effectiveness of our structured approach.

Method: Structured framework detailing stages and procedures for localization without compromising cultural authenticity.

Result: Applied framework to localize 10,000 synsets, demonstrating practical implementation and outcomes.

Conclusion: Presents a systematic approach to WordNet localization that addresses cultural alignment and quality assurance for Arabic language processing.

Abstract: As Princeton WordNet continues to gain significance as a semantic lexicon in Natural Language Processing, the need for its localization and for ensuring the quality of this process has become increasingly critical. Existing efforts remain limited in both scale and rigor, and there is a notable absence of studies addressing the accuracy of localization or its alignment with the cultural context of Arabic. This paper proposes a structured framework for the localization of Princeton WordNet, detailing the stages and procedures required to achieve high-quality results without compromising cultural authenticity. We further present our experience in applying this framework, reporting outcomes from the localization of 10,000 synsets.

Method: Proposes S²Sent - a parameterized nested selector downstream of Transformer encoders. It performs spatial selection (SS) using spatial squeeze self-gating for adaptive weights, and nested frequency selection (FS) using DCT basis functions instead of GAP for spatial squeeze with low semantic loss.

Result: Extensive experiments show S²Sent achieves significant improvements over baseline methods with negligible additional parameters and inference latency, while demonstrating high integrability and scalability.

Conclusion: The proposed S²Sent mechanism effectively balances semantic redundancy and loss in cross-block representation fusion, outperforming existing approaches while maintaining efficiency and flexibility.

Abstract: The combination of Transformer-based encoders with contrastive learning represents the current mainstream paradigm for sentence representation learning. This paradigm is typically based on the hidden states of the last Transformer block of the encoder. However, within Transformer-based encoders, different blocks exhibit varying degrees of semantic perception ability. From the perspective of interpretability, the semantic perception potential of knowledge neurons is modulated by stimuli, thus rational cross-block representation fusion is a direction worth optimizing. To balance the semantic redundancy and loss across block fusion, we propose a sentence representation selection mechanism S\textsuperscript{2}Sent, which integrates a parameterized nested selector downstream of the Transformer-based encoder. This selector performs spatial selection (SS) and nested frequency selection (FS) from a modular perspective. The SS innovatively employs a spatial squeeze based self-gating mechanism to obtain adaptive weights, which not only achieves fusion with low information redundancy but also captures the dependencies between embedding features. The nested FS replaces GAP with different DCT basis functions to achieve spatial squeeze with low semantic loss. Extensive experiments have demonstrated that S\textsuperscript{2}Sent achieves significant improvements over baseline methods with negligible additional parameters and inference latency, while highlighting high integrability and scalability.

Method: Two-stage data generation pipeline synthesizing realistic multi-turn discussions annotated with intervention types, training models to predict silent tokens when no intervention is needed. Two architectures: integrated end-to-end model and decoupled classifier-generator system.

Result: Models learn to remain quiet until helpful contributions can be made, enabling accurate timing of interventions and generation of helpful responses.

Conclusion: The framework paves the way for more situationally aware and proactive conversational AI that can better collaborate in human discussions.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in understanding and generating human-like text, yet they largely operate as reactive agents, responding only when directly prompted. This passivity creates an “awareness gap,” limiting their potential as truly collaborative partners in dynamic human discussions. We introduce $\textit{DiscussLLM}$, a framework designed to bridge this gap by training models to proactively decide not just $\textit{what}$ to say, but critically, $\textit{when}$ to speak. Our primary contribution is a scalable two-stage data generation pipeline that synthesizes a large-scale dataset of realistic multi-turn human discussions. Each discussion is annotated with one of five intervention types (e.g., Factual Correction, Concept Definition) and contains an explicit conversational trigger where an AI intervention adds value. By training models to predict a special silent token when no intervention is needed, they learn to remain quiet until a helpful contribution can be made. We explore two architectural baselines: an integrated end-to-end model and a decoupled classifier-generator system optimized for low-latency inference. We evaluate these models on their ability to accurately time interventions and generate helpful responses, paving the way for more situationally aware and proactive conversational AI.

Method: Develops Joint Stochastic Approximation (JSA) algorithm, a stochastic extension of EM algorithm, specifically designed for estimating discrete latent variable models in RAG frameworks.

Result: Extensive experiments on 5 datasets for open-domain QA and knowledge-grounded dialogs show JSA-RAG significantly outperforms both vanilla RAG and VRAG, with improved generation, retrieval, and low-variance gradient estimation.

Conclusion: JSA-RAG provides an effective solution for end-to-end RAG training with superior performance and stable gradient estimation compared to existing methods.

Abstract: Retrieval-augmented generation (RAG) has become a widely recognized paradigm to combine parametric memory with non-parametric memories. An RAG model consists of two serial connecting components (retriever and generator). A major challenge in end-to-end optimization of the RAG model is that marginalization over relevant passages (modeled as discrete latent variables) from a knowledge base is required. Traditional top-K marginalization and variational RAG (VRAG) suffer from biased or high-variance gradient estimates. In this paper, we propose and develop joint stochastic approximation (JSA) based end-to-end training of RAG, which is referred to as JSA-RAG. The JSA algorithm is a stochastic extension of the EM (expectation-maximization) algorithm and is particularly powerful in estimating discrete latent variable models. Extensive experiments are conducted on five datasets for two tasks (open-domain question answering, knowledge-grounded dialogs) and show that JSA-RAG significantly outperforms both vanilla RAG and VRAG. Further analysis shows the efficacy of JSA-RAG from the perspectives of generation, retrieval, and low-variance gradient estimate.

Method: Proposes AulSign method that leverages Large Language Models through dynamic prompting and in-context learning with sample selection and sign association. Signs are mapped to compact natural language descriptions that the model uses for translation.

Result: Superior performance compared to state-of-the-art models in low-data scenarios, evaluated on both English and Italian using SignBank+ and Italian LaCAM CNR-ISTC datasets.

Conclusion: AulSign demonstrates effective sign language translation using LLMs, showing potential to enhance accessibility and inclusivity for underrepresented linguistic communities in communication technologies.

Abstract: Translating natural languages into sign languages is a highly complex and underexplored task. Despite growing interest in accessibility and inclusivity, the development of robust translation systems remains hindered by the limited availability of parallel corpora which align natural language with sign language data. Existing methods often struggle to generalize in these data-scarce environments, as the few datasets available are typically domain-specific, lack standardization, or fail to capture the full linguistic richness of sign languages. To address this limitation, we propose Advanced Use of LLMs for Sign Language Translation (AulSign), a novel method that leverages Large Language Models via dynamic prompting and in-context learning with sample selection and subsequent sign association. Despite their impressive abilities in processing text, LLMs lack intrinsic knowledge of sign languages; therefore, they are unable to natively perform this kind of translation. To overcome this limitation, we associate the signs with compact descriptions in natural language and instruct the model to use them. We evaluate our method on both English and Italian languages using SignBank+, a recognized benchmark in the field, as well as the Italian LaCAM CNR-ISTC dataset. We demonstrate superior performance compared to state-of-the-art models in low-data scenario. Our findings demonstrate the effectiveness of AulSign, with the potential to enhance accessibility and inclusivity in communication technologies for underrepresented linguistic communities.

Method: Used a curated dataset of over 500,000 text samples from nearly 5,000 authors with reliably identified musical preferences. Built advanced models to assess personality characteristics through computational linguistic analysis.

Result: Revealed significant personality differences across fans of five musical genres, demonstrating that musical preferences are detectable through language analysis.

Conclusion: The study successfully bridges computational linguistics, music psychology and personality analysis, providing resources for future interdisciplinary research in these domains.

Abstract: Music serves as a powerful reflection of individual identity, often aligning with deeper psychological traits. Prior research has established correlations between musical preferences and personality traits, while separate studies have demonstrated that personality is detectable through linguistic analysis. Our study bridges these two research domains by investigating whether individuals' musical preferences are recognizable in their spontaneous language through the lens of the Big Five personality traits (Openness, Conscientiousness, Extroversion, Agreeableness, and Neuroticism). Using a carefully curated dataset of over 500,000 text samples from nearly 5,000 authors with reliably identified musical preferences, we build advanced models to assess personality characteristics. Our results reveal significant personality differences across fans of five musical genres. We release resources for future research at the intersection of computational linguistics, music psychology and personality analysis.

Method: Uses derived call attributes (Intent Summaries, Topic Flow, QA Evaluation Forms) as supervision signals. Benchmarks four language-agnostic generation strategies from simple prompting to multi-stage approaches, and introduces a diagnostic framework with 18 linguistically and behaviorally grounded metrics.

Result: Results show persistent challenges - no method excels across all traits, with notable deficits in disfluency, sentiment, and behavioral realism. The diagnostic tool effectively exposes these gaps.

Conclusion: The proposed diagnostic framework enables fine-grained evaluation and stress testing of synthetic dialogue across languages, revealing significant quality gaps in current generation approaches for contact center domains.

Abstract: Synthetic transcript generation is critical in contact center domains, where privacy and data scarcity limit model training and evaluation. Unlike prior synthetic dialogue generation work on open-domain or medical dialogues, contact center conversations are goal-oriented, role-asymmetric, and behaviorally complex, featuring disfluencies, ASR noise, and compliance-driven agent actions. In deployments where transcripts are unavailable, standard pipelines still yield derived call attributes such as Intent Summaries, Topic Flow, and QA Evaluation Forms. We leverage these as supervision signals to guide generation. To assess the quality of such outputs, we introduce a diagnostic framework of 18 linguistically and behaviorally grounded metrics for comparing real and synthetic transcripts. We benchmark four language-agnostic generation strategies, from simple prompting to characteristic-aware multi-stage approaches, alongside reference-free baselines. Results reveal persistent challenges: no method excels across all traits, with notable deficits in disfluency, sentiment, and behavioral realism. Our diagnostic tool exposes these gaps, enabling fine-grained evaluation and stress testing of synthetic dialogue across languages.

Method: Proposes ESFP-RM, a two-stage reward model using explanation-based slot framework prediction with masked language models, leveraging their better performance on NLI tasks compared to autoregressive models.

Result: Extensive experiments show ESFP-RM delivers more stable and generalizable reward signals in both RLHF and out-of-distribution scenarios compared to generative reward models.

Conclusion: Reframing reward modeling as NLI and using MLMs with explanation-based slot prediction provides a superior approach for building stable and generalizable reward models in AI feedback systems.

Abstract: The emergence of LM-based judging reward modeling, represented by generative reward models, has successfully made reinforcement learning from AI feedback (RLAIF) efficient and scalable. To further advance this paradigm, we propose a core insight: this form of reward modeling shares fundamental formal consistency with natural language inference (NLI), a core task in natural language understanding. This reframed perspective points to a key path for building superior reward models: scaling the model’s comprehension boundaries. Pursuing this path, exploratory experiments on NLI tasks demonstrate that the slot prediction masked language models (MLMs) incorporating contextual explanations achieve significantly better performance compared to mainstream autoregressive models. Based on this key finding, we propose ESFP-RM, a two-stage LM-based judging reward model that utilizes an explanation based slot framework for prediction to fully leverage the advantages of MLMs. Extensive experiments demonstrate that in both reinforcement learning from human feedback (RLHF) and out-of-distribution (OOD) scenarios, the ESFP-RM framework delivers more stable and generalizable reward signals compared to generative reward models.

Method: Developed MTalk-Bench with 9 realistic scenarios across three dimensions, using dual evaluation methods (Arena-style pairwise comparison and Rubrics-based absolute scoring) with both human and LLM evaluators.

Result: S2S LLMs excel at semantic processing but underperform on paralinguistic information and ambient sounds; models regain coherence by increasing response length; modality-aware designs outperform brute scaling. Evaluation methods show consistency but require large performance gaps for reliable distinctions.

Conclusion: Current S2S evaluation has limitations, highlighting the need for more robust, speech-aware assessment frameworks that can better handle multi-turn dialogue complexity.

Abstract: The rapid advancement of speech-to-speech (S2S) large language models (LLMs) has significantly improved real-time spoken interaction. However, current evaluation frameworks remain inadequate for assessing performance in complex, multi-turn dialogues. To address this, we introduce MTalk-Bench, a multi-turn S2S benchmark covering three core dimensions: Semantic Information, Paralinguistic Information, and Ambient Sound. Each dimension includes nine realistic scenarios, along with targeted tasks to assess specific capabilities such as reasoning. Our dual-method evaluation framework combines Arena-style evaluation (pairwise comparison) and Rubrics-based evaluation (absolute scoring) for relative and absolute assessment. The benchmark includes both model and human outputs, evaluated by human evaluators and LLMs. Experimental results reveal two sets of findings. Overall performance of S2S LLMs: (1) models excel at semantic information processing yet underperform on paralinguistic information and ambient sounds perception; (2) models typically regain coherence by increasing response length, sacrificing efficiency in multi-turn dialogues; (3) modality-aware, task-specific designs outperform brute scaling. Evaluation framework and reliability: (1) Arena and Rubrics yield consistent, complementary rankings, but reliable distinctions emerge only when performance gaps are large; (2) LLM-as-a-judge aligns with humans when gaps are clear or criteria explicit, but exhibits position and length biases and is reliable on nonverbal evaluation only with text annotations. These results highlight current limitations in S2S evaluation and the need for more robust, speech-aware assessment frameworks.

Method: Evaluated different LLMs using EXIST 2024 tweet dataset with annotations from six distinct profiles per tweet, analyzed demographic biases, and conducted statistical analysis to identify which demographic characteristics contribute most to sexism detection.

Result: LLMs can detect sexism when considering overall population opinion but do not accurately replicate the diversity of perceptions among different demographic groups.

Conclusion: There is a need for better-calibrated models that account for the diversity of perspectives across different populations to improve sexism detection accuracy.

Abstract: The use of Large Language Models (LLMs) has proven to be a tool that could help in the automatic detection of sexism. Previous studies have shown that these models contain biases that do not accurately reflect reality, especially for minority groups. Despite various efforts to improve the detection of sexist content, this task remains a significant challenge due to its subjective nature and the biases present in automated models. We explore the capabilities of different LLMs to detect sexism in social media text using the EXIST 2024 tweet dataset. It includes annotations from six distinct profiles for each tweet, allowing us to evaluate to what extent LLMs can mimic these groups’ perceptions in sexism detection. Additionally, we analyze the demographic biases present in the models and conduct a statistical analysis to identify which demographic characteristics (age, gender) contribute most effectively to this task. Our results show that, while LLMs can to some extent detect sexism when considering the overall opinion of populations, they do not accurately replicate the diversity of perceptions among different demographic groups. This highlights the need for better-calibrated models that account for the diversity of perspectives across different populations.

Method: Employed various masking strategies to evaluate LLMs’ intrinsic ability, contribution of different sentence elements, and attention mechanisms during prediction. Also explored fine-tuning to enhance classifier performance.

Result: LLMs perform worse than BERT even with fine-tuning. Prediction greatly benefits from information about the following noun, explaining the advantage of bidirectional attention models like BERT.

Conclusion: Current LLMs lack proper Chinese classifier knowledge compared to bidirectional models, with noun information being critical for accurate prediction, suggesting limitations in LLM architecture for this linguistic task.

Abstract: Classifiers are an important and defining feature of the Chinese language, and their correct prediction is key to numerous educational applications. Yet, whether the most popular Large Language Models (LLMs) possess proper knowledge the Chinese classifiers is an issue that has largely remain unexplored in the Natural Language Processing (NLP) literature. To address such a question, we employ various masking strategies to evaluate the LLMs’ intrinsic ability, the contribution of different sentence elements, and the working of the attention mechanisms during prediction. Besides, we explore fine-tuning for LLMs to enhance the classifier performance. Our findings reveal that LLMs perform worse than BERT, even with fine-tuning. The prediction, as expected, greatly benefits from the information about the following noun, which also explains the advantage of models with a bidirectional attention mechanism such as BERT.

Method: MIRAGE performs dynamic multi-chain inference over structured medical knowledge graphs by: 1) decomposing queries into entity-grounded sub-questions, 2) executing parallel inference chains, 3) adaptive evidence retrieval via neighbor expansion and multi-hop traversal, and 4) cross-chain verification to resolve contradictions.

Result: Experiments on three medical QA benchmarks (GenMedGPT-5k, CMCQA, ExplainCPE) show MIRAGE consistently outperforms GPT-4o, Tree-of-Thought variants, and other retrieval-augmented baselines in both automatic and human evaluations.

Conclusion: MIRAGE improves interpretability by generating explicit reasoning chains that trace factual claims to concrete knowledge graph chains, making it well-suited for complex medical reasoning scenarios. Code will be available for further research.

Abstract: Large reasoning models (LRMs) have shown significant progress in test-time scaling through chain-of-thought prompting. Current approaches like search-o1 integrate retrieval augmented generation (RAG) into multi-step reasoning processes but rely on a single, linear reasoning chain while incorporating unstructured textual information in a flat, context-agnostic manner. As a result, these approaches can lead to error accumulation throughout the reasoning chain, which significantly limits its effectiveness in medical question-answering (QA) tasks where both accuracy and traceability are critical requirements. To address these challenges, we propose MIRAGE (Multi-chain Inference with Retrieval-Augmented Graph Exploration), a novel test-time scalable reasoning framework that performs dynamic multi-chain inference over structured medical knowledge graphs. Specifically, MIRAGE 1) decomposes complex queries into entity-grounded sub-questions, 2) executes parallel inference chains, 3) retrieves evidence adaptively via neighbor expansion and multi-hop traversal, and 4) integrates answers using cross-chain verification to resolve contradictions. Experiments on three medical QA benchmarks (GenMedGPT-5k, CMCQA, and ExplainCPE) show that MIRAGE consistently outperforms GPT-4o, Tree-of-Thought variants, and other retrieval-augmented baselines in both automatic and human evaluations. Additionally, MIRAGE improves interpretability by generating explicit reasoning chains that trace each factual claim to concrete chains within the knowledge graph, making it well-suited for complex medical reasoning scenarios. The code will be available for further research.

Method: Recursively generate memory summaries using LLMs - first memorize small dialogue contexts, then produce new memory using previous memory and following contexts, enabling consistent response generation with the latest memory.

Result: Experiments show the method generates more consistent responses in long-context conversations and complements both long-context and retrieval-enhanced LLMs, improving long-term dialogue performance.

Conclusion: The recursive memory generation approach is a potential solution for enabling LLMs to model extremely long contexts effectively, with code released for implementation.

Abstract: Recently, large language models (LLMs), such as GPT-4, stand out remarkable conversational abilities, enabling them to engage in dynamic and contextually relevant dialogues across a wide range of topics. However, given a long conversation, these chatbots fail to recall past information and tend to generate inconsistent responses. To address this, we propose to recursively generate summaries/ memory using large language models (LLMs) to enhance long-term memory ability. Specifically, our method first stimulates LLMs to memorize small dialogue contexts and then recursively produce new memory using previous memory and following contexts. Finally, the chatbot can easily generate a highly consistent response with the help of the latest memory. We evaluate our method on both open and closed LLMs, and the experiments on the widely-used public dataset show that our method can generate more consistent responses in a long-context conversation. Also, we show that our strategy could nicely complement both long-context (e.g., 8K and 16K) and retrieval-enhanced LLMs, bringing further long-term dialogue performance. Notably, our method is a potential solution to enable the LLM to model the extremely long context. The code and scripts are released.

Method: Evaluated both LLMs (GPT-2, GPT-3, GPT-3.5, GPT-4, Bard) and human subjects (N=147 including Serbian students and native English speakers) on dialogue-based tasks using Grice’s communication principles.

Result: GPT-4 achieved the highest score of 4.80, surpassing the best human score of 4.55. LLMs collectively outperformed humans with an average score of 3.39 vs human averages of 2.80 (Serbian) and 2.34 (US). GPT-4 ranked first among all 155 subjects.

Conclusion: LLMs show significant progress in simulating understanding of linguistic pragmatics, highlighting their potential for communication-centered tasks and future applications in humanoid robots.

Abstract: As Large Language Models (LLMs) become increasingly integrated into everyday life as general purpose multimodal AI systems, their capabilities to simulate human understanding are under examination. This study investigates LLMs ability to interpret linguistic pragmatics, which involves context and implied meanings. Using Grice communication principles, we evaluated both LLMs (GPT-2, GPT-3, GPT-3.5, GPT-4, and Bard) and human subjects (N = 147) on dialogue-based tasks. Human participants included 71 primarily Serbian students and 76 native English speakers from the United States. Findings revealed that LLMs, particularly GPT-4, outperformed humans. GPT4 achieved the highest score of 4.80, surpassing the best human score of 4.55. Other LLMs performed well: GPT 3.5 scored 4.10, Bard 3.75, and GPT-3 3.25. GPT-2 had the lowest score of 1.05. The average LLM score was 3.39, exceeding the human cohorts averages of 2.80 (Serbian students) and 2.34 (U.S. participants). In the ranking of all 155 subjects (including LLMs and humans), GPT-4 secured the top position, while the best human ranked second. These results highlight significant progress in LLMs ability to simulate understanding of linguistic pragmatics. Future studies should confirm these findings with more dialogue-based tasks and diverse participants. This research has important implications for advancing general-purpose AI models in various communication-centered tasks, including potential application in humanoid robots in the future.

Method: Developed a proper cross-subject data splitting criterion that prevents data leakage for both fMRI and EEG brain signal decoding to text, and used this criterion to re-evaluate state-of-the-art decoding models.

Result: Demonstrated that current splitting methods suffer from data leakage, and proposed a correct splitting approach that provides more reliable evaluation of brain-to-text decoding performance.

Conclusion: Proper data splitting without leakage is crucial for accurate evaluation of cross-subject brain-to-text decoding models, and the proposed criterion enables more reliable assessment of true model performance for future research.

Abstract: Recent major milestones have successfully reconstructed natural language from non-invasive brain signals (e.g. functional Magnetic Resonance Imaging (fMRI) and Electroencephalogram (EEG)) across subjects. However, we find current dataset splitting strategies for cross-subject brain-to-text decoding are wrong. Specifically, we first demonstrate that all current splitting methods suffer from data leakage problem, which refers to the leakage of validation and test data into training set, resulting in significant overfitting and overestimation of decoding models. In this study, we develop a right cross-subject data splitting criterion without data leakage for decoding fMRI and EEG signal to text. Some SOTA brain-to-text decoding models are re-evaluated correctly with the proposed criterion for further research.

Method: Proposes grammar error-triggered backdoor attack that leverages contrastive loss sensitivity to grammatical mistakes and hard negative sampling to exacerbate susceptibility. Trains models to retrieve attacker-specified content when minor linguistic errors are present.

Result: Achieves high attack success rate with only 0.048% corpus poisoning rate, preserves normal retrieval performance for error-free queries, and shows resistance to three real-world defense strategies.

Conclusion: Dense retrieval systems are vulnerable to covert grammar-triggered backdoor attacks, with contrastive loss and hard negative sampling making them particularly susceptible, highlighting significant security risks.

Abstract: Dense retrieval systems have been widely used in various NLP applications. However, their vulnerabilities to potential attacks have been underexplored. This paper investigates a novel attack scenario where the attackers aim to mislead the retrieval system into retrieving the attacker-specified contents. Those contents, injected into the retrieval corpus by attackers, can include harmful text like hate speech or spam. Unlike prior methods that rely on model weights and generate conspicuous, unnatural outputs, we propose a covert backdoor attack triggered by grammar errors. Our approach ensures that the attacked models can function normally for standard queries while covertly triggering the retrieval of the attacker’s contents in response to minor linguistic mistakes. Specifically, dense retrievers are trained with contrastive loss and hard negative sampling. Surprisingly, our findings demonstrate that contrastive loss is notably sensitive to grammatical errors, and hard negative sampling can exacerbate susceptibility to backdoor attacks. Our proposed method achieves a high attack success rate with a minimal corpus poisoning rate of only 0.048%, while preserving normal retrieval performance. This indicates that the method has negligible impact on user experience for error-free queries. Furthermore, evaluations across three real-world defense strategies reveal that the malicious passages embedded within the corpus remain highly resistant to detection and filtering, underscoring the robustness and subtlety of the proposed attack \footnote{Codes of this work are available at https://github.com/ruyue0001/Backdoor_DPR.}.

Method: Creates compact graph descriptions, uses diverse instruction generation with CoT-based packages to distill reasoning from LLMs like GPT-4, and implements graph-aware instruction tuning for mutual enhancement across tasks.

Result: Demonstrates significant improvements in five graph tasks and ten datasets, enhancing accuracy of graph-oriented downstream tasks while improving LLMs’ generation abilities.

Conclusion: MuseGraph successfully bridges GNNs and LLMs, showing strong potential as a foundation model for cross-task and cross-dataset graph mining applications.

Abstract: Graphs with abundant attributes are essential in modeling interconnected entities and enhancing predictions across various real-world applications. Traditional Graph Neural Networks (GNNs) often require re-training for different graph tasks and datasets. Although the emergence of Large Language Models (LLMs) has introduced new paradigms in natural language processing, their potential for generic graph mining, training a single model to simultaneously handle diverse tasks and datasets, remains under-explored. To this end, our novel framework MuseGraph, seamlessly integrates the strengths of GNNs and LLMs into one foundation model for graph mining across tasks and datasets. This framework first features a compact graph description to encapsulate key graph information within language token limitations. Then, we propose a diverse instruction generation mechanism with Chain-of-Thought (CoT)-based instruction packages to distill the reasoning capabilities from advanced LLMs like GPT-4. Finally, we design a graph-aware instruction tuning strategy to facilitate mutual enhancement across multiple tasks and datasets while preventing catastrophic forgetting of LLMs’ generative abilities. Our experimental results demonstrate significant improvements in five graph tasks and ten datasets, showcasing the potential of our MuseGraph in enhancing the accuracy of graph-oriented downstream tasks while improving the generation abilities of LLMs.

Method: The authors conduct a comprehensive literature review and introduce a unified taxonomy with two paradigms: (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to understand LLM progress in NLP.

Result: The study provides a systematic overview of how LLMs are currently applied to NLP tasks, assesses whether traditional NLP tasks have been solved, and identifies new frontiers and challenges in the field.

Conclusion: This work offers valuable insights into LLMs’ potential and limitations while serving as a practical guide for building effective LLMs in NLP, aiming to inspire further groundbreaking advancements in the field.

Abstract: While large language models (LLMs) like ChatGPT have shown impressive capabilities in Natural Language Processing (NLP) tasks, a systematic investigation of their potential in this field remains largely unexplored. This study aims to address this gap by exploring the following questions: (1) How are LLMs currently applied to NLP tasks in the literature? (2) Have traditional NLP tasks already been solved with LLMs? (3) What is the future of the LLMs for NLP? To answer these questions, we take the first step to provide a comprehensive overview of LLMs in NLP. Specifically, we first introduce a unified taxonomy including (1) parameter-frozen paradigm and (2) parameter-tuning paradigm to offer a unified perspective for understanding the current progress of LLMs in NLP. Furthermore, we summarize the new frontiers and the corresponding challenges, aiming to inspire further groundbreaking advancements. We hope this work offers valuable insights into the potential and limitations of LLMs, while also serving as a practical guide for building effective LLMs in NLP.

Method: Utilized Wikipedia and Wikidata to create a dataset with a new taxonomy categorizing questions into attributes, comparisons, and counting questions. Each question includes detailed metadata with specific time scopes for comprehensive evaluation.

Result: Created a dataset of over 100 million question-answer pairs that covers temporal questions requiring complex reasoning capabilities including across-time comparison, temporal aggregation, and multi-hop reasoning involving temporal event ordering and entity recognition.

Conclusion: ComplexTempQA provides an unmatched scale dataset for evaluating temporal reasoning abilities of large language models, featuring high complexity questions that demand sophisticated reasoning beyond simple temporal understanding.

Abstract: We introduce \textsc{ComplexTempQA},\footnote{Dataset and code available at: https://github.com/DataScienceUIBK/ComplexTempQA} a large-scale dataset consisting of over 100 million question-answer pairs designed to tackle the challenges in temporal question answering. \textsc{ComplexTempQA} significantly surpasses existing benchmarks in scale and scope. Utilizing Wikipedia and Wikidata, the dataset covers questions spanning over two decades and offers an unmatched scale. We introduce a new taxonomy that categorizes questions as \textit{attributes}, \textit{comparisons}, and \textit{counting} questions, revolving around events, entities, and time periods, respectively. A standout feature of \textsc{ComplexTempQA} is the high complexity of its questions, which demand reasoning capabilities for answering such as across-time comparison, temporal aggregation, and multi-hop reasoning involving temporal event ordering and entity recognition. Additionally, each question is accompanied by detailed metadata, including specific time scopes, allowing for comprehensive evaluation of temporal reasoning abilities of large language models.

Method: Developed Rusty-DAWG for efficient arbitrary-length n-gram search over corpora, analyzed Pythia models, measured both probability assigned to training n-grams and n-novelty (proportion of novel n-grams), and compared with human-written text.

Result: LM-generated text is less novel than human-written text for n > 4, but more novel for smaller n. Larger models and constrained decoding decrease novelty. LMs complete n-grams with lower loss when they are more frequent in training data.

Conclusion: The study reveals factors affecting LM novelty and provides Rusty-DAWG tool to facilitate further research on pretraining data analysis.

Abstract: How novel are texts generated by language models (LMs) relative to their training corpora? In this work, we investigate the extent to which modern LMs generate $n$-grams from their training data, evaluating both (i) the probability LMs assign to complete training $n$-grams and (ii) $n$-novelty, the proportion of $n$-grams generated by an LM that did not appear in the training data (for arbitrarily large $n$). To enable arbitrary-length $n$-gram search over a corpus in constant time w.r.t. corpus size, we develop Rusty-DAWG, a novel search tool inspired by indexing of genomic data. We compare the novelty of LM-generated text to human-written text and explore factors that affect generation novelty, focusing on the Pythia models. We find that, for $n > 4$, LM-generated text is less novel than human-written text, though it is more novel for smaller $n$. Larger LMs and more constrained decoding strategies both decrease novelty. Finally, we show that LMs complete $n$-grams with lower loss if they are more frequent in the training data. Overall, our results reveal factors influencing the novelty of LM-generated text, and we release Rusty-DAWG to facilitate further pretraining data research.

Method: DoGe (Dynamic Grounding) method that alternates between using internal parameter knowledge and external source knowledge based on the model’s factual confidence level.

Result: Extensive experiments on three datasets show DoGe enhances response diversity while maintaining factuality, significantly outperforming various decoding strategy baselines.

Conclusion: DoGe successfully reconciles factuality and diversity in dialogue generation without relying on questionable randomness, providing a more balanced approach to source-grounded generation.

Abstract: Grounding external knowledge can enhance the factuality of responses in dialogue generation. However, excessive emphasis on it might result in the lack of engaging and diverse expressions. Through the introduction of randomness in sampling, current approaches can increase the diversity. Nevertheless, such sampling method could undermine the factuality in dialogue generation. In this study, to discover a solution for advancing creativity without relying on questionable randomness and to subtly reconcile the factuality and diversity within the source-grounded paradigm, a novel method named DoGe is proposed. DoGe can dynamically alternate between the utilization of internal parameter knowledge and external source knowledge based on the model’s factual confidence. Extensive experiments on three widely-used datasets show that DoGe can not only enhance response diversity but also maintain factuality, and it significantly surpasses other various decoding strategy baselines.

Method: Weight-Rotated Preference Optimization (RoPO) that conducts rotational and magnitude-stretching updates on weight parameters to maintain hyperspherical energy invariant, preserving knowledge encoded in neuron angles.

Result: RoPO outperforms DPO by up to 10 points on MT-Bench and 2.8 points on AlpacaEval 2, while enhancing generation diversity by average 6 points using only 0.0086% trainable parameters.

Conclusion: RoPO effectively prevents alignment overfitting while maintaining strong performance and diversity, demonstrating superior regularization through weight updating perspective.

Abstract: DPO is an effective preference optimization algorithm. However, the DPO-tuned models tend to overfit on the dispreferred samples, manifested as overly long generations lacking diversity. While recent regularization approaches have endeavored to alleviate this issue by modifying the objective function, they achieved that at the cost of alignment performance degradation. In this paper, we innovatively incorporate regularization from the perspective of weight updating to curb alignment overfitting. Through the pilot experiment, we discovered that there exists a positive correlation between overfitting and the hyperspherical energy fluctuation. Hence, we introduce orthogonal finetuning for DPO via a weight-Rotated Preference Optimization (RoPO) method, which merely conducts rotational and magnitude-stretching updates on the weight parameters to maintain the hyperspherical energy invariant, thereby preserving the knowledge encoded in the angle between neurons. Extensive experiments demonstrate that our model aligns perfectly with human preferences while retaining the original expressive capacity using only 0.0086% of the trainable parameters, suggesting an effective regularization against overfitting. Specifically, RoPO outperforms DPO by up to 10 points on MT-Bench and by up to 2.8 points on AlpacaEval 2, while enhancing the generation diversity by an average of 6 points.

Method: Decomposes generated text into minimal predicate-argument level propositions expressed as simple QA pairs, then assesses whether each QA pair is supported by trusted reference text using Neo-Davidsonian formal semantics.

Result: Achieved substantial inter-annotator agreement in crowdsourced annotations, created benchmark with 3K+ instances across various tasks, and showed factual consistency scores correlate well with human judgments. Also implemented automated detection methods using supervised entailment models and LLMs.

Conclusion: QASemConsistency effectively localizes unsupported information at fine-grained levels and provides a reliable framework for both human annotation and automated detection of factual inconsistencies in text generation.

Abstract: There has been an increasing interest in detecting hallucinations in model-generated texts, both manually and automatically, at varying levels of granularity. However, most existing methods fail to precisely pinpoint the errors. In this work, we introduce QASemConsistency, a new formalism for localizing factual inconsistencies in attributable text generation, at a fine-grained level. Drawing inspiration from Neo-Davidsonian formal semantics, we propose decomposing the generated text into minimal predicate-argument level propositions, expressed as simple question-answer (QA) pairs, and assess whether each individual QA pair is supported by a trusted reference text. As each QA pair corresponds to a single semantic relation between a predicate and an argument, QASemConsistency effectively localizes the unsupported information. We first demonstrate the effectiveness of the QASemConsistency methodology for human annotation, by collecting crowdsourced annotations of granular consistency errors, while achieving a substantial inter-annotator agreement. This benchmark includes more than 3K instances spanning various tasks of attributable text generation. We also show that QASemConsistency yields factual consistency scores that correlate well with human judgments. Finally, we implement several methods for automatically detecting localized factual inconsistencies, with both supervised entailment models and LLMs.

Method: Two-stage approach: 1) Sensor-Language Alignment stage aligns sensor inputs with trend descriptions using special tokens for channel boundaries, 2) Task-Aware Tuning stage refines the model for HAR classification.

Result: Achieves performance that matches or surpasses state-of-the-art methods, demonstrating effective sensor learning, reasoning, and classification capabilities across diverse HAR datasets.

Conclusion: Establishes foundation for future research on time-series and text alignment, paving the way for foundation models in sensor data analysis.

Abstract: We introduce SensorLLM, a two-stage framework that enables Large Language Models (LLMs) to perform human activity recognition (HAR) from sensor time-series data. Despite their strong reasoning and generalization capabilities, LLMs remain underutilized for motion sensor data due to the lack of semantic context in time-series, computational constraints, and challenges in processing numerical inputs. SensorLLM addresses these limitations through a Sensor-Language Alignment stage, where the model aligns sensor inputs with trend descriptions. Special tokens are introduced to mark channel boundaries. This alignment enables LLMs to capture numerical variations, channel-specific features, and data of varying durations, without requiring human annotations. In the subsequent Task-Aware Tuning stage, we refine the model for HAR classification, achieving performance that matches or surpasses state-of-the-art methods. Our results demonstrate that SensorLLM evolves into an effective sensor learner, reasoner, and classifier through human-intuitive Sensor-Language Alignment, generalizing across diverse HAR datasets. We believe this work establishes a foundation for future research on time-series and text alignment, paving the way for foundation models in sensor data analysis. Our codes are available at https://github.com/zechenli03/SensorLLM.

Method: Proposes SVIP - a training-free dynamic length policy that uses draft model’s prediction entropy to adaptively determine draft sequence lengths (high entropy indicates low acceptance rate, low entropy indicates high acceptance rate).

Result: Achieves up to 17% speedup on MT-Bench at 8K context compared with fixed draft lengths, and 22% speedup for QwQ in long-form reasoning tasks.

Conclusion: SVIP demonstrates that draft model entropy effectively predicts acceptance rates, enabling dynamic length adaptation that significantly improves speculative decoding performance in complex reasoning and long-context scenarios.

Abstract: Conventional speculative decoding (SD) methods utilize a predefined length policy for proposing drafts, which implies the premise that the target model smoothly accepts the proposed draft tokens. However, reality deviates from this assumption: the oracle draft length varies significantly, and the fixed-length policy hardly satisfies such a requirement. Moreover, such discrepancy is further exacerbated in scenarios involving complex reasoning and long-form generation, particularly under test-time scaling for reasoning-specialized models. Through both theoretical and empirical estimation, we establish that the discrepancy between the draft and target models can be approximated by the draft model’s prediction entropy: a high entropy indicates a low acceptance rate of draft tokens, and vice versa. Based on this insight, we propose SVIP: Self-Verification Length Policy for Long-Context Speculative Decoding, which is a training-free dynamic length policy for speculative decoding systems that adaptively determines the lengths of draft sequences by referring to the draft entropy. Experimental results on mainstream SD benchmarks as well as reasoning-heavy benchmarks demonstrate the superior performance of SVIP, achieving up to 17% speedup on MT-Bench at 8K context compared with fixed draft lengths, and 22% speedup for QwQ in long-form reasoning.

Method: Multi-agent framework with translator, advisor, and evaluator to iteratively translate sentences containing similes/metaphors. Collects long-thought MT data to train DRT models based on Qwen2.5 and LLama-3.1 backbones.

Result: DRT models outperform vanilla LLMs and fine-tuned LLMs without long thought, demonstrating effective learning of thought processes during translation.

Conclusion: Long chain-of-thought reasoning can be successfully applied to machine translation, particularly for challenging literary texts, with DRT showing significant improvements over existing approaches.

Abstract: Recently, O1-like models have emerged as representative examples, illustrating the effectiveness of long chain-of-thought (CoT) in reasoning tasks such as math and coding tasks. In this paper, we introduce DRT, an attempt to bring the success of long CoT to neural machine translation (MT). Specifically, in view of the literature books that might involve similes and metaphors, translating these texts to a target language is very difficult in practice due to cultural differences. In such cases, literal translation often fails to convey the intended meaning effectively. Even for professional human translators, considerable thought must be given to preserving semantics throughout the translation process. To simulate LLMs’ long thought ability in MT, we first mine sentences containing similes or metaphors from existing literature books, and then develop a multi-agent framework to translate these sentences via long thought. In the multi-agent framework, a translator is used to iteratively translate the source sentence under the suggestions provided by an advisor. To ensure the effectiveness of the long thoughts, an evaluator is also employed to quantify the translation quality in each round. In this way, we collect tens of thousands of long-thought MT data, which is used to train our DRT. Using Qwen2.5 and LLama-3.1 as the backbones, DRT models can learn the thought process during machine translation, and outperform vanilla LLMs as well as LLMs which are simply fine-tuning on the paired sentences without long thought, showing its effectiveness. The synthesized data and model checkpoints are released at https://github.com/krystalan/DRT.

Method: The study conducts a comprehensive survey of existing LLMs in natural disaster management, develops a taxonomy based on disaster phases and application scenarios, collects public datasets, and identifies key challenges and opportunities.

Result: The paper presents a structured framework for understanding and categorizing LLM applications in disaster management across different phases and scenarios.

Conclusion: This survey aims to guide the professional community in developing advanced LLMs for disaster management to enhance resilience against natural disasters.

Abstract: Large language models (LLMs) have revolutionized scientific research with their exceptional capabilities and transformed various fields. Among their practical applications, LLMs have been playing a crucial role in mitigating threats to human life, infrastructure, and the environment. Despite growing research in disaster LLMs, there remains a lack of systematic review and in-depth analysis of LLMs for natural disaster management. To address the gap, this paper presents a comprehensive survey of existing LLMs in natural disaster management, along with a taxonomy that categorizes existing works based on disaster phases and application scenarios. By collecting public datasets and identifying key challenges and opportunities, this study aims to guide the professional community in developing advanced LLMs for disaster management to enhance the resilience against natural disasters.

Method: Automated extraction pipeline with stringent filtering to ensure scientific rigor and educational level, using heterogeneous sources to create problem-solution pairs.

Result: Created SCP-116K dataset with 116,756 high-quality problem-solution pairs, along with an open-source extraction pipeline for future expansions.

Conclusion: SCP-116K serves as a critical resource to foster scientific reasoning research, enable LLM evaluations, and lower barriers for replicating advanced model successes in science community.

Abstract: Recent breakthroughs in large language models (LLMs) exemplified by the impressive mathematical and scientific reasoning capabilities of the o1 model have spotlighted the critical importance of high-quality training data in advancing LLM performance across STEM disciplines. While the mathematics community has benefited from a growing body of curated datasets, the scientific domain at the higher education level has long suffered from a scarcity of comparable resources. To address this gap, we present SCP-116K, a new large-scale dataset of 116,756 high-quality problem-solution pairs, automatically extracted from heterogeneous sources using a streamlined and highly generalizable pipeline. Our approach involves stringent filtering to ensure the scientific rigor and educational level of the extracted materials, while maintaining adaptability for future expansions or domain transfers. By openly releasing both the dataset and the extraction pipeline, we seek to foster research on scientific reasoning, enable comprehensive performance evaluations of new LLMs, and lower the barrier to replicating the successes of advanced models like o1 in the broader science community. We believe SCP-116K will serve as a critical resource, catalyzing progress in high-level scientific reasoning tasks and promoting further innovations in LLM development. The dataset and code are publicly available at https://github.com/AQA6666/SCP-116K-open.

Method: The paper examines privacy challenges in AI for mental health and proposes solutions including data anonymization, synthetic data generation, and privacy-preserving training methods. It also outlines frameworks for managing privacy-utility trade-offs.

Result: The research provides a comprehensive examination of privacy risks in mental health AI applications and proposes practical solutions to enable reliable, privacy-aware tools that can support clinical decision-making.

Conclusion: The paper aims to advance the development of privacy-conscious AI tools for mental health that balance utility with privacy protection, ultimately improving mental health outcomes while safeguarding patient confidentiality.

Abstract: Mental health disorders create profound personal and societal burdens, yet conventional diagnostics are resource-intensive and limit accessibility. Advances in artificial intelligence, particularly natural language processing and multimodal methods, offer promise for detecting and addressing mental disorders, but raise critical privacy risks. This paper examines these challenges and proposes solutions, including anonymization, synthetic data, and privacy-preserving training, while outlining frameworks for privacy-utility trade-offs, aiming to advance reliable, privacy-aware AI tools that support clinical decision-making and improve mental health outcomes.

Method: Developed a causal model for coupled autoregressive generation that allows different LLMs to sample responses with the same source of randomness, enabling controlled comparisons.

Result: Coupled generation requires up to 75% fewer samples for benchmark evaluations and reveals different win-rate rankings in pairwise comparisons compared to vanilla generation, even with infinite samples.

Conclusion: Existing LLM evaluation protocols may produce confounded results due to randomness, and coupled autoregressive generation provides a more efficient and reliable alternative for fair model comparisons.

Abstract: State of the art large language models rely on randomization to respond to a prompt. As an immediate consequence, a model may respond differently to the same prompt if asked multiple times. In this work, we argue that the evaluation and ranking of large language models should control for the randomization underpinning their functioning. Our starting point is the development of a causal model for coupled autoregressive generation, which allows different large language models to sample responses with the same source of randomness. Building upon our causal model, we first show that, on evaluations based on benchmark datasets, coupled autoregressive generation leads to the same conclusions as vanilla autoregressive generation but using provably fewer samples. However, we further show that, on evaluations based on (human) pairwise comparisons, coupled and vanilla autoregressive generation can surprisingly lead to different rankings when comparing more than two models, even with an infinite amount of samples. This suggests that the apparent advantage of a model over others in existing evaluation protocols may not be genuine but rather confounded by the randomness inherent to the generation process. To illustrate and complement our theoretical results, we conduct experiments with several large language models from the Llama, Mistral and Qwen families. We find that, across multiple benchmark datasets, coupled autoregressive generation requires up to 75% fewer samples to reach the same conclusions as vanilla autoregressive generation. Further, we find that the win-rates derived from pairwise comparisons by a strong large language model to prompts from the LMSYS Chatbot Arena platform differ under coupled and vanilla autoregressive generation.

Method: Devised a framework with two perturbation families: adversarial noise and counterfactual statements, generating seven perturbed datasets. Organized LLM reasoners by reasoning format, formalisation syntax, and error recovery feedback mechanisms.

Result: Adversarial noise particularly affects autoformalisation methods, while counterfactual statements impact all approaches. Detailed feedback reduces syntax errors but doesn’t improve overall accuracy, indicating limited self-correction capability.

Conclusion: LLM-based deductive reasoning methods show limited robustness to perturbations, with autoformalisation being vulnerable to noise and all methods struggling with counterfactuals. The inability of feedback to improve accuracy suggests fundamental challenges in LLM self-correction for logical deduction tasks.

Abstract: Large Language Models (LLMs) have been shown to achieve impressive results for many reasoning-based NLP tasks, suggesting a degree of deductive reasoning capability. However, it remains unclear to which extent LLMs, in both informal and autoformalisation methods, are robust on logical deduction tasks. Moreover, while many LLM-based deduction methods have been proposed, a systematic study that analyses the impact of their design components is lacking. Addressing these two challenges, we propose the first study of the robustness of formal and informal LLM-based deductive reasoning methods. We devise a framework with two families of perturbations: adversarial noise and counterfactual statements, which jointly generate seven perturbed datasets. We organize the landscape of LLM reasoners according to their reasoning format, formalisation syntax, and feedback for error recovery. The results show that adversarial noise affects autoformalisation, while counterfactual statements influence all approaches. Detailed feedback does not improve overall accuracy despite reducing syntax errors, pointing to the challenge of LLM-based methods to self-correct effectively.

Method: Built on established psychological theories of EI, the benchmark evaluates MLLMs across 13 valuation scenarios from three dimensions: foundational emotion recognition, conversational emotion understanding, and socially complex emotion analysis.

Result: Evaluations of both open-source and closed-source MLLMs show a significant performance gap between them and humans, indicating current models lack sufficient emotional intelligence capabilities.

Conclusion: There is a critical need to advance MLLMs’ emotional intelligence capabilities to enable effective human-robot interaction, and EmoBench-M provides a comprehensive framework for evaluating progress in this area.

Abstract: With the integration of Multimodal large language models (MLLMs) into robotic systems and various AI applications, embedding emotional intelligence (EI) capabilities into these models is essential for enabling robots to effectively address human emotional needs and interact seamlessly in real-world scenarios. Existing static, text-based, or text-image benchmarks overlook the multimodal complexities of real-world interactions and fail to capture the dynamic, multimodal nature of emotional expressions, making them inadequate for evaluating MLLMs’ EI. Based on established psychological theories of EI, we build EmoBench-M, a novel benchmark designed to evaluate the EI capability of MLLMs across 13 valuation scenarios from three key dimensions: foundational emotion recognition, conversational emotion understanding, and socially complex emotion analysis. Evaluations of both open-source and closed-source MLLMs on EmoBench-M reveal a significant performance gap between them and humans, highlighting the need to further advance their EI capabilities. All benchmark resources, including code and datasets, are publicly available at https://emo-gml.github.io/.

Method: Probing each attention head in a binary choice task to identify critical heads, then applying fine-grained activation interventions at these specific heads during inference to steer model generations.

Result: Interventions on a few attention heads were more effective than full-layer interventions or supervised fine-tuning, successfully circumventing safety guardrails and enabling harmful AI coordination.

Conclusion: Attention head activations encode fine-grained linearly separable behaviors, and targeted interventions offer a straightforward methodology to steer LLM behavior, with implications for both safety vulnerabilities and potential applications requiring fine-grained control.

Abstract: Robust alignment guardrails for large language models (LLMs) are becoming increasingly important with their widespread application. In contrast to previous studies, we demonstrate that inference-time activation interventions can bypass safety alignments and effectively steer model generations towards harmful AI coordination. Our method applies fine-grained interventions at specific attention heads, which we identify by probing each head in a simple binary choice task. We then show that interventions on these heads generalise to the open-ended generation setting, effectively circumventing safety guardrails. We demonstrate that intervening on a few attention heads is more effective than intervening on full layers or supervised fine-tuning. We further show that only a few example completions are needed to compute effective steering directions, which is an advantage over classical fine-tuning. We also demonstrate that applying interventions in the negative direction can prevent a common jailbreak attack. Our results suggest that, at the attention head level, activations encode fine-grained linearly separable behaviours. Practically, the approach offers a straightforward methodology to steer large language model behaviour, which could be extended to diverse domains beyond safety, requiring fine-grained control over the model output. The code and datasets for this study can be found on https://github.com/PaulDrm/targeted_intervention.

Method: Defined and studied CHOKE examples across different models and datasets, analyzed consistency across prompts, and introduced a probing-based mitigation approach.

Result: CHOKE examples are consistent across prompts, occur in different models and datasets, are fundamentally distinct from other hallucinations, and existing mitigation methods perform worse on them compared to general hallucinations.

Conclusion: The findings reveal an overlooked aspect of hallucinations, emphasizing the need to better understand their origins and improve mitigation strategies to enhance LLM safety, with the proposed probing-based mitigation outperforming existing methods.

Abstract: Prior work on large language model (LLM) hallucinations has associated them with model uncertainty or inaccurate knowledge. In this work, we define and investigate a distinct type of hallucination, where a model can consistently answer a question correctly, but a seemingly trivial perturbation, which can happen in real-world settings, causes it to produce a hallucinated response with high certainty. This phenomenon, which we dub CHOKE (Certain Hallucinations Overriding Known Evidence), is particularly concerning in high-stakes domains such as medicine or law, where model certainty is often used as a proxy for reliability. We show that CHOKE examples are consistent across prompts, occur in different models and datasets, and are fundamentally distinct from other hallucinations. This difference leads existing mitigation methods to perform worse on CHOKE examples than on general hallucinations. Finally, we introduce a probing-based mitigation that outperforms existing methods on CHOKE hallucinations. These findings reveal an overlooked aspect of hallucinations, emphasizing the need to understand their origins and improve mitigation strategies to enhance LLM safety. The code is available at https://github.com/technion-cs-nlp/Trust_me_Im_wrong .

Method: Introduced a constraint prioritization evaluation framework and conducted experiments across six state-of-the-art LLMs to test instruction hierarchy enforcement.

Result: Models struggle with consistent instruction prioritization even for simple conflicts; system/user prompt separation fails to establish reliable hierarchy; models show strong inherent biases; social hierarchies work better than technical controls.

Conclusion: Pretraining-derived social structures act as latent control priors with potentially stronger influence than post-training guardrails, suggesting current hierarchical control mechanisms are inadequate.

Abstract: Large language models (LLMs) are increasingly deployed with hierarchical instruction schemes, where certain instructions (e.g., system-level directives) are expected to take precedence over others (e.g., user messages). Yet, we lack a systematic understanding of how effectively these hierarchical control mechanisms work. We introduce a systematic evaluation framework based on constraint prioritization to assess how well LLMs enforce instruction hierarchies. Our experiments across six state-of-the-art LLMs reveal that models struggle with consistent instruction prioritization, even for simple formatting conflicts. We find that the widely-adopted system/user prompt separation fails to establish a reliable instruction hierarchy, and models exhibit strong inherent biases toward certain constraint types regardless of their priority designation. We find that LLMs more reliably obey constraints framed through natural social hierarchies (e.g., authority, expertise, consensus) than system/user roles, which suggests that pretraining-derived social structures act as latent control priors, with potentially stronger influence than post-training guardrails.

Method: Two-variant method that fine-tunes VLLMs on automatically constructed datasets for boundary identification, enabling efficient RAG usage by detecting when retrieval is actually needed.

Result: Experimental results on Visual Question Answering datasets show successful knowledge boundary detection, allowing reduced indiscriminate retrieval while maintaining or improving performance. Also shows knowledge boundary transferability between VLLMs.

Conclusion: The proposed method effectively identifies VLLM knowledge boundaries, enabling more efficient RAG implementation and reducing unnecessary retrieval costs while preserving performance.

Abstract: Despite the advancements made in Vision Large Language Models (VLLMs), like text Large Language Models (LLMs), they have limitations in addressing questions that require real-time information or are knowledge-intensive. Indiscriminately adopting Retrieval Augmented Generation (RAG) techniques is an effective yet expensive way to enable models to answer queries beyond their knowledge scopes. To mitigate the dependence on retrieval and simultaneously maintain, or even improve, the performance benefits provided by retrieval, we propose a method to detect the knowledge boundary of VLLMs, allowing for more efficient use of techniques like RAG. Specifically, we propose a method with two variants that fine-tune a VLLM on an automatically constructed dataset for boundary identification. Experimental results on various types of Visual Question Answering datasets show that our method successfully depicts a VLLM’s knowledge boundary, based on which we are able to reduce indiscriminate retrieval while maintaining or improving the performance. In addition, we show that the knowledge boundary identified by our method for one VLLM can be used as a surrogate boundary for other VLLMs. Code will be released at https://github.com/Chord-Chen-30/VLLM-KnowledgeBoundary

Method: Models dialogue with LLMs using state-space representations and introduces a neural barrier function (NBF) to proactively detect and filter harmful queries emerging from evolving contexts, learning a safety predictor that accounts for adversarial queries.

Result: Extensive experiments show NBF-based safety steering outperforms safety alignment, prompt-based steering, and lightweight LLM guardrails baselines, offering stronger defenses against multi-turn jailbreaks.

Conclusion: The proposed framework achieves invariant safety at each dialogue turn while maintaining a better trade-off among safety, helpfulness and over-refusal compared to existing approaches.

Abstract: Large language models (LLMs) are shown to be vulnerable to jailbreaking attacks where adversarial prompts are designed to elicit harmful responses. While existing defenses effectively mitigate single-turn attacks by detecting and filtering unsafe inputs, they fail against multi-turn jailbreaks that exploit contextual drift over multiple interactions, gradually leading LLMs away from safe behavior. To address this challenge, we propose a safety steering framework grounded in safe control theory, ensuring invariant safety in multi-turn dialogues. Our approach models the dialogue with LLMs using state-space representations and introduces a novel neural barrier function (NBF) to detect and filter harmful queries emerging from evolving contexts proactively. Our method achieves invariant safety at each turn of dialogue by learning a safety predictor that accounts for adversarial queries, preventing potential context drift toward jailbreaks. Extensive experiments under multiple LLMs show that our NBF-based safety steering outperforms safety alignment, prompt-based steering and lightweight LLM guardrails baselines, offering stronger defenses against multi-turn jailbreaks while maintaining a better trade-off among safety, helpfulness and over-refusal. Check out the website here https://sites.google.com/view/llm-nbf/home . Our code is available on https://github.com/HanjiangHu/NBF-LLM .

Method: Introduced a novel evaluation framework using free-form storytelling to surface biases, systematically analyzing ten prominent LLMs and examining the impact of supervised fine-tuning and reinforcement learning from human feedback.

Result: Found consistent pattern of overrepresenting female characters across occupations, yet the occupational gender distributions aligned more closely with human stereotypes than real-world labor data.

Conclusion: Highlights the challenge of implementing balanced mitigation measures to promote fairness and prevent establishment of new biases in LLMs, with prompts and generated stories released for further research.

Abstract: Large Language Models (LLMs) have revolutionized natural language processing, yet concerns persist regarding their tendency to reflect or amplify social biases. This study introduces a novel evaluation framework to uncover gender biases in LLMs: using free-form storytelling to surface biases embedded within the models. A systematic analysis of ten prominent LLMs shows a consistent pattern of overrepresenting female characters across occupations, likely due to supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF). Paradoxically, despite this overrepresentation, the occupational gender distributions produced by these LLMs align more closely with human stereotypes than with real-world labor data. This highlights the challenge and importance of implementing balanced mitigation measures to promote fairness and prevent the establishment of potentially new biases. We release the prompts and LLM-generated stories at GitHub.

Method: Constructed from Hungarian-specific materials using real user queries from the internet, emphasizing generative capabilities assessment, and employing LLM-as-judge for multidimensional evaluation.

Result: Evaluation of mainstream LLMs revealed significant performance gaps, demonstrating the necessity for Hungarian-specific model optimization and providing insights into Large Reasoning Models’ thinking processes in non-English languages.

Conclusion: OpenHuEval provides a scientifically accurate framework for assessing Hungarian language capabilities in LLMs, revealing intrinsic patterns in non-English language processing and establishing a foundation for future model optimization.

Abstract: We introduce OpenHuEval, the first benchmark for LLMs focusing on the Hungarian language and specifics. OpenHuEval is constructed from a vast collection of Hungarian-specific materials sourced from multiple origins. In the construction, we incorporated the latest design principles for evaluating LLMs, such as using real user queries from the internet, emphasizing the assessment of LLMs’ generative capabilities, and employing LLM-as-judge to enhance the multidimensionality and accuracy of evaluations. Ultimately, OpenHuEval encompasses eight Hungarian-specific dimensions, featuring five tasks and 3953 questions. Consequently, OpenHuEval provides the comprehensive, in-depth, and scientifically accurate assessment of LLM performance in the context of the Hungarian language and its specifics. We evaluated current mainstream LLMs, including both traditional LLMs and recently developed Large Reasoning Models. The results demonstrate the significant necessity for evaluation and model optimization tailored to the Hungarian language and specifics. We also established the framework for analyzing the thinking processes of LRMs with OpenHuEval, revealing intrinsic patterns and mechanisms of these models in non-English languages, with Hungarian serving as a representative example. We will release OpenHuEval at https://github.com/opendatalab/OpenHuEval .

Method: Implicit Fingerprints (ImF) combines steganography to embed ownership information in natural texts with Chain-of-Thought prompting to create semantically coherent QA pairs that blend seamlessly with normal model behavior.

Result: Comprehensive evaluation on 15 diverse LLMs shows ImF provides robust fingerprinting that resists adversarial attacks while maintaining stealthiness and natural integration with model outputs.

Conclusion: ImF represents a significant advancement in LLM fingerprinting by creating indistinguishable, semantically coherent ownership markers that address critical vulnerabilities in existing methods.

Abstract: Training large language models (LLMs) is resource-intensive and expensive, making protecting intellectual property (IP) for LLMs crucial. Recently, embedding fingerprints into LLMs has emerged as a prevalent method for establishing model ownership. However, existing fingerprinting techniques typically embed identifiable patterns with weak semantic coherence, resulting in fingerprints that significantly differ from the natural question-answering (QA) behavior inherent to LLMs. This discrepancy undermines the stealthiness of the embedded fingerprints and makes them vulnerable to adversarial attacks. In this paper, we first demonstrate the critical vulnerability of existing fingerprint embedding methods by introducing a novel adversarial attack named Generation Revision Intervention (GRI) attack. GRI attack exploits the semantic fragility of current fingerprinting methods, effectively erasing fingerprints by disrupting their weakly correlated semantic structures. Our empirical evaluation highlights that traditional fingerprinting approaches are significantly compromised by the GRI attack, revealing severe limitations in their robustness under realistic adversarial conditions. To advance the state-of-the-art in model fingerprinting, we propose a novel model fingerprint paradigm called Implicit Fingerprints (ImF). ImF leverages steganography techniques to subtly embed ownership information within natural texts, subsequently using Chain-of-Thought (CoT) prompting to construct semantically coherent and contextually natural QA pairs. This design ensures that fingerprints seamlessly integrate with the standard model behavior, remaining indistinguishable from regular outputs and substantially reducing the risk of accidental triggering and targeted removal. We conduct a comprehensive evaluation of ImF on 15 diverse LLMs, spanning different architectures and varying scales.

Method: The study evaluates decoder-only models (Mistral-7B, Llama2-7B) and encoder-decoder models (Flan-T5 Base) on TACRED and FewRel datasets using task-incremental fine-tuning with memory replay to prevent catastrophic forgetting.

Result: LLMs demonstrated superior performance over encoder-only models like BERT on TACRED, excelling in seen-task accuracy and overall performance. Mistral and Flan-T5 models performed particularly well. On FewRel, the approach achieved second place in whole and average accuracy metrics.

Conclusion: The work advances continual relation extraction with LLMs and memory replay, highlighting critical factors in knowledge transfer, language model architecture, and knowledge graph completeness for dynamic, real-time relation extraction.

Abstract: Real-world data, such as news articles, social media posts, and chatbot conversations, is inherently dynamic and non-stationary, presenting significant challenges for constructing real-time structured representations through knowledge graphs (KGs). Relation Extraction (RE), a fundamental component of KG creation, often struggles to adapt to evolving data when traditional models rely on static, outdated datasets. Continual Relation Extraction (CRE) methods tackle this issue by incrementally learning new relations while preserving previously acquired knowledge. This study investigates the application of pre-trained language models (PLMs), specifically large language models (LLMs), to CRE, with a focus on leveraging memory replay to address catastrophic forgetting. We evaluate decoder-only models (eg, Mistral-7B and Llama2-7B) and encoder-decoder models (eg, Flan-T5 Base) on the TACRED and FewRel datasets. Task-incremental fine-tuning of LLMs demonstrates superior performance over earlier approaches using encoder-only models like BERT on TACRED, excelling in seen-task accuracy and overall performance (measured by whole and average accuracy), particularly with the Mistral and Flan-T5 models. Results on FewRel are similarly promising, achieving second place in whole and average accuracy metrics. This work underscores critical factors in knowledge transfer, language model architecture, and KG completeness, advancing CRE with LLMs and memory replay for dynamic, real-time relation extraction.

Method: Proposes Contrastive Decoding-Guided Knowledge Distillation (CDG-KD) which uses contrastive decoding to extract corrupted or amplified watermark texts by comparing student model outputs with weakly watermarked references, followed by bidirectional distillation to train new student models for watermark removal and forgery.

Result: Extensive experiments show that CDG-KD effectively performs both scrubbing and spoofing attacks while preserving the general performance of the distilled model.

Conclusion: The findings highlight the critical need for developing watermarking schemes that are both robust against removal and resistant to forgery, especially in knowledge distillation contexts.

Abstract: Watermarking has emerged as a critical technique for combating misinformation and protecting intellectual property in large language models (LLMs). A recent discovery, termed watermark radioactivity, reveals that watermarks embedded in teacher models can be inherited by student models through knowledge distillation. On the positive side, this inheritance allows for the detection of unauthorized knowledge distillation by identifying watermark traces in student models. However, the robustness of watermarks against scrubbing attacks and their unforgeability in the face of spoofing attacks under unauthorized knowledge distillation remain largely unexplored. Existing watermark attack methods either assume access to model internals or fail to simultaneously support both scrubbing and spoofing attacks. In this work, we propose Contrastive Decoding-Guided Knowledge Distillation (CDG-KD), a unified framework that enables bidirectional attacks under unauthorized knowledge distillation. Our approach employs contrastive decoding to extract corrupted or amplified watermark texts via comparing outputs from the student model and weakly watermarked references, followed by bidirectional distillation to train new student models capable of watermark removal and watermark forgery, respectively. Extensive experiments show that CDG-KD effectively performs attacks while preserving the general performance of the distilled model. Our findings underscore critical need for developing watermarking schemes that are robust and unforgeable.

Method: Analysis of story-based evaluation benchmarks and review of recent enhancement strategies for improving LLMs’ Theory of Mind capabilities.

Result: Comprehensive survey of current ToM evaluation methods and enhancement approaches for LLMs.

Conclusion: Provides valuable resource for researchers and outlines promising future directions to advance LLMs’ ToM capabilities for more realistic and diverse scenarios.

Abstract: Theory of Mind (ToM)-the ability to reason about the mental states of oneself and others-is a cornerstone of human social intelligence. As Large Language Models (LLMs) become increasingly integrated into daily life, understanding their ability to interpret and respond to human mental states is crucial for enabling effective interactions. In this paper, we review LLMs’ ToM capabilities by analyzing both evaluation benchmarks and enhancement strategies. For evaluation, we focus on recently proposed and widely used story-based benchmarks. For enhancement, we provide an in-depth analysis of recent methods aimed at improving LLMs’ ToM abilities. Furthermore, we outline promising directions for future research to further advance these capabilities and better adapt LLMs to more realistic and diverse scenarios. Our survey serves as a valuable resource for researchers interested in evaluating and advancing LLMs’ ToM capabilities.

Method: Factorized model that captures adverbial semantics as probabilistic distributions composed with event-specific distributions, fitted using native speaker judgment data on adverbial applicability.

Result: Both factorized and non-factorized (single Gaussian) models have similar predictive power, but the factorized model is simpler and more extendable according to Occam’s razor.

Conclusion: The factorized probabilistic approach provides a preferable model for vague temporal adverbial semantics due to its simplicity and better extendability while maintaining comparable performance.

Abstract: Vague temporal adverbials, such as recently, just, and a long time ago, describe the temporal distance between a past event and the utterance time but leave the exact duration underspecified. In this paper, we introduce a factorized model that captures the semantics of these adverbials as probabilistic distributions. These distributions are composed with event-specific distributions to yield a contextualized meaning for an adverbial applied to a specific event. We fit the model’s parameters using existing data capturing judgments of native speakers regarding the applicability of these vague temporal adverbials to events that took place a given time ago. Comparing our approach to a non-factorized model based on a single Gaussian distribution for each pair of event and temporal adverbial, we find that while both models have similar predictive power, our model is preferable in terms of Occam’s razor, as it is simpler and has better extendability.

Method: Comprehensive literature review and knowledge systematization along four objectives: defining CUAs for safety analysis, categorizing threats, proposing defense taxonomy, and summarizing evaluation benchmarks.

Result: Developed a structured framework for understanding CUA vulnerabilities, categorized current threats, organized defensive strategies, and compiled evaluation metrics and datasets.

Conclusion: Provides researchers with foundation for exploring new vulnerabilities and offers practitioners actionable guidance for designing and deploying secure Computer-Using Agents.

Abstract: Recently, AI-driven interactions with computing devices have advanced from basic prototype tools to sophisticated, LLM-based systems that emulate human-like operations in graphical user interfaces. We are now witnessing the emergence of \emph{Computer-Using Agents} (CUAs), capable of autonomously performing tasks such as navigating desktop applications, web pages, and mobile apps. However, as these agents grow in capability, they also introduce novel safety and security risks. Vulnerabilities in LLM-driven reasoning, with the added complexity of integrating multiple software components and multimodal inputs, further complicate the security landscape. In this paper, we present a systematization of knowledge on the safety and security threats of CUAs. We conduct a comprehensive literature review and distill our findings along four research objectives: \textit{\textbf{(i)}} define the CUA that suits safety analysis; \textit{\textbf{(ii)} } categorize current safety threats among CUAs; \textit{\textbf{(iii)}} propose a comprehensive taxonomy of existing defensive strategies; \textit{\textbf{(iv)}} summarize prevailing benchmarks, datasets, and evaluation metrics used to assess the safety and performance of CUAs. Building on these insights, our work provides future researchers with a structured foundation for exploring unexplored vulnerabilities and offers practitioners actionable guidance in designing and deploying secure Computer-Using Agents.

Method: Created TED2025 corpus with up to 50 languages aligned in parallel, then investigated best practices for continued pretraining and instruction tuning using this multi-way parallel data, analyzing key influencing factors.

Result: Experiments on six multilingual benchmarks show that models trained on multi-way parallel data consistently outperform those trained on unaligned multilingual data.

Conclusion: Multi-way parallel data from TED2025 corpus effectively enhances multilingual LLM performance through improved cross-lingual consistency, demonstrating superior results compared to unaligned data approaches.

Abstract: Continued pretraining and instruction tuning on large-scale multilingual data have proven to be effective in scaling large language models (LLMs) to low-resource languages. However, the unaligned nature of such data limits its ability to effectively capture cross-lingual semantics. In contrast, multi-way parallel data, where identical content is aligned across multiple languages, provides stronger cross-lingual consistency and offers greater potential for improving multilingual performance. In this paper, we introduce a large-scale, high-quality multi-way parallel corpus, TED2025, based on TED Talks. The corpus spans 113 languages, with up to 50 languages aligned in parallel, ensuring extensive multilingual coverage. Using this dataset, we investigate best practices for leveraging multi-way parallel data to enhance LLMs, including strategies for continued pretraining, instruction tuning, and the analysis of key influencing factors. Experiments on six multilingual benchmarks show that models trained on multiway parallel data consistently outperform those trained on unaligned multilingual data.

Method: Inject subtle user-based biases into queries and train LLMs to use this bias signal to produce sensitive information only when users are authorized.

Result: Substantially improved alignment, generalization, resistance to jailbreaking attacks, and fails-closed behavior. Resolves tension between language modeling and safety objectives.

Conclusion: sudoLLM provides an additional security layer that complements existing guardrail mechanisms for enhanced end-to-end safety with LLMs.

Abstract: User authorization-based access privileges are a key feature in many safety-critical systems, but have not been extensively studied in the large language model (LLM) realm. In this work, drawing inspiration from such access control systems, we introduce sudoLLM, a novel framework that results in multi-role aligned LLMs, i.e., LLMs that account for, and behave in accordance with, user access rights. sudoLLM injects subtle user-based biases into queries and trains an LLM to utilize this bias signal in order to produce sensitive information if and only if the user is authorized. We present empirical results demonstrating that this approach shows substantially improved alignment, generalization, resistance to prefix-based jailbreaking attacks, and ``fails-closed’’. The persistent tension between the language modeling objective and safety alignment, which is often exploited to jailbreak LLMs, is somewhat resolved with the aid of the injected bias signal. Our framework is meant as an additional security layer, and complements existing guardrail mechanisms for enhanced end-to-end safety with LLMs.

Method: Introduced two benchmarks: KnowRecall (visual QA for factual knowledge consistency in 15 languages) and VisRecall (visual memory consistency testing in 9 languages without image access).

Result: Experimental results show that even state-of-the-art MLLMs, including proprietary models, fail to achieve cross-lingual consistency in cultural knowledge and visual memory tasks.

Conclusion: There is a critical need for more robust approaches to develop truly multilingual and culturally aware multimodal language models that maintain consistency across languages.

Abstract: The rapid evolution of multimodal large language models (MLLMs) has significantly enhanced their real-world applications. However, achieving consistent performance across languages, especially when integrating cultural knowledge, remains a significant challenge. To better assess this issue, we introduce two new benchmarks: KnowRecall and VisRecall, which evaluate cross-lingual consistency in MLLMs. KnowRecall is a visual question answering benchmark designed to measure factual knowledge consistency in 15 languages, focusing on cultural and historical questions about global landmarks. VisRecall assesses visual memory consistency by asking models to describe landmark appearances in 9 languages without access to images. Experimental results reveal that state-of-the-art MLLMs, including proprietary ones, still struggle to achieve cross-lingual consistency. This underscores the need for more robust approaches that produce truly multilingual and culturally aware models.

Method: IRONIC uses in-context learning with Multi-modal Coherence Relations to analyze referential, analogical, and pragmatic linkages between images and text.

Result: Achieves state-of-the-art performance on zero-shot Multi-modal Sarcasm Detection across different baselines without task-specific fine-tuning.

Conclusion: Demonstrates the importance of incorporating linguistic and cognitive insights into multi-modal reasoning strategies for effective figurative language interpretation.

Abstract: Interpreting figurative language such as sarcasm across multi-modal inputs presents unique challenges, often requiring task-specific fine-tuning and extensive reasoning steps. However, current Chain-of-Thought approaches do not efficiently leverage the same cognitive processes that enable humans to identify sarcasm. We present IRONIC, an in-context learning framework that leverages Multi-modal Coherence Relations to analyze referential, analogical and pragmatic image-text linkages. Our experiments show that IRONIC achieves state-of-the-art performance on zero-shot Multi-modal Sarcasm Detection across different baselines. This demonstrates the need for incorporating linguistic and cognitive insights into the design of multi-modal reasoning strategies. Our code is available at: https://github.com/aashish2000/IRONIC

Method: A two-stage framework that first transforms harmful inquiries into structured outlines, then reframes them into declarative-style narratives through iterative prompt optimization using feedback loops.

Result: Achieves attack success rates of 88.25%-96.54% against CoT-based defenses and 86.75%-97.12% against IA-based defenses across various LLMs, outperforming state-of-the-art jailbreak methods.

Conclusion: Intent manipulation poses a significant threat to LLM safety mechanisms, revealing that current intent-aware guardrails are vulnerable to sophisticated prompt refinement attacks.

Abstract: Intent detection, a core component of natural language understanding, has considerably evolved as a crucial mechanism in safeguarding large language models (LLMs). While prior work has applied intent detection to enhance LLMs' moderation guardrails, showing a significant success against content-level jailbreaks, the robustness of these intent-aware guardrails under malicious manipulations remains under-explored. In this work, we investigate the vulnerability of intent-aware guardrails and demonstrate that LLMs exhibit implicit intent detection capabilities. We propose a two-stage intent-based prompt-refinement framework, IntentPrompt, that first transforms harmful inquiries into structured outlines and further reframes them into declarative-style narratives by iteratively optimizing prompts via feedback loops to enhance jailbreak success for red-teaming purposes. Extensive experiments across four public benchmarks and various black-box LLMs indicate that our framework consistently outperforms several cutting-edge jailbreak methods and evades even advanced Intent Analysis (IA) and Chain-of-Thought (CoT)-based defenses. Specifically, our “FSTR+SPIN” variant achieves attack success rates ranging from 88.25% to 96.54% against CoT-based defenses on the o1 model, and from 86.75% to 97.12% on the GPT-4o model under IA-based defenses. These findings highlight a critical weakness in LLMs’ safety mechanisms and suggest that intent manipulation poses a growing challenge to content moderation guardrails.

Method: Debate-to-Detect (D2D) framework with domain-specific agents conducting 5-stage debates: Opening Statement, Rebuttal, Free Debate, Closing Statement, and Judgment. Uses multi-dimensional evaluation across Factuality, Source Reliability, Reasoning Quality, Clarity, and Ethics.

Result: Experiments with GPT-4o on two datasets show significant improvements over baseline methods. Case studies demonstrate iterative evidence refinement and improved decision transparency.

Conclusion: D2D represents a substantial advancement towards interpretable misinformation detection by mimicking real-world fact-checking workflows through structured multi-agent debates.

Abstract: The proliferation of misinformation in digital platforms reveals the limitations of traditional detection methods, which mostly rely on static classification and fail to capture the intricate process of real-world fact-checking. Despite advancements in Large Language Models (LLMs) that enhance automated reasoning, their application to misinformation detection remains hindered by issues of logical inconsistency and superficial verification. In response, we introduce Debate-to-Detect (D2D), a novel Multi-Agent Debate (MAD) framework that reformulates misinformation detection as a structured adversarial debate. Inspired by fact-checking workflows, D2D assigns domain-specific profiles to each agent and orchestrates a five-stage debate process, including Opening Statement, Rebuttal, Free Debate, Closing Statement, and Judgment. To transcend traditional binary classification, D2D introduces a multi-dimensional evaluation mechanism that assesses each claim across five distinct dimensions: Factuality, Source Reliability, Reasoning Quality, Clarity, and Ethics. Experiments with GPT-4o on two datasets demonstrate significant improvements over baseline methods, and the case study highlight D2D’s capability to iteratively refine evidence while improving decision transparency, representing a substantial advancement towards interpretable misinformation detection. The code will be released publicly after the official publication.

Method: Proposes several approaches using Large Language Models (LLMs) to test classifier predictions for correctness instead of relying on human annotators

Result: Enables more efficient model quality assurance and supports high-quality incremental learning pipelines

Conclusion: LLMs can effectively replace human annotators for validation tasks, reducing costs and addressing the challenges of ongoing model retraining needs

Abstract: Machine learning models for text classification are trained to predict a class for a given text. To do this, training and validation samples must be prepared: a set of texts is collected, and each text is assigned a class. These classes are usually assigned by human annotators with different expertise levels, depending on the specific classification task. Collecting such samples from scratch is labor-intensive because it requires finding specialists and compensating them for their work; moreover, the number of available specialists is limited, and their productivity is constrained by human factors. While it may not be too resource-intensive to collect samples once, the ongoing need to retrain models (especially in incremental learning pipelines) to address data drift (also called model drift) makes the data collection process crucial and costly over the model’s entire lifecycle. This paper proposes several approaches to replace human annotators with Large Language Models (LLMs) to test classifier predictions for correctness, helping ensure model quality and support high-quality incremental learning.

Method: Proposes DecisionFlow framework that guides models to reason over structured representations of actions, attributes, and constraints. Builds semantically grounded decision space and infers latent utility function to evaluate trade-offs transparently.

Result: Achieves up to 30% accuracy gains over strong prompting baselines on two high-stakes benchmarks, with enhanced alignment in outcomes and interpretable rationales.

Conclusion: A critical step toward integrating symbolic reasoning with LLMs, enabling more accountable, explainable, and reliable decision support systems.

Abstract: In high-stakes domains such as healthcare and finance, effective decision-making demands not just accurate outcomes but transparent and explainable reasoning. However, current language models often lack the structured deliberation needed for such tasks, instead generating decisions and justifications in a disconnected, post-hoc manner. To address this, we propose DecisionFlow, a novel decision modeling framework that guides models to reason over structured representations of actions, attributes, and constraints. Rather than predicting answers directly from prompts, DecisionFlow builds a semantically grounded decision space and infers a latent utility function to evaluate trade-offs in a transparent, utility-driven manner. This process produces decisions tightly coupled with interpretable rationales reflecting the model’s reasoning. Empirical results on two high-stakes benchmarks show that DecisionFlow not only achieves up to 30% accuracy gains over strong prompting baselines but also enhances alignment in outcomes. Our work is a critical step toward integrating symbolic reasoning with LLMs, enabling more accountable, explainable, and reliable LLM decision support systems. Code and data are at https://github.com/xiusic/DecisionFlow.

Method: Created PersonaConflicts Corpus (N=5,772 dialogues) with diverse conflict scenarios between familiar partners, conducted controlled human study for fine-grained annotations, and evaluated LLMs’ ability to detect breakdowns using nonviolent communication theory.

Result: Relationship backstories significantly shifted human perception of conflicts and social impressions, but models failed to meaningfully leverage backstories. Models consistently overestimated how positively messages would make listeners feel.

Conclusion: Personalization to relationship contexts is critical for LLMs to serve as effective mediators in human communication, as current models cannot adequately incorporate relational dynamics like humans do.

Abstract: Conversational breakdowns in close relationships are deeply shaped by personal histories and emotional context, yet most NLP research treats conflict detection as a general task, overlooking the relational dynamics that influence how messages are perceived. In this work, we leverage nonviolent communication (NVC) theory to evaluate LLMs in detecting conversational breakdowns and assessing how relationship backstory influences both human and model perception of conflicts. Given the sensitivity and scarcity of real-world datasets featuring conflict between familiar social partners with rich personal backstories, we contribute the PersonaConflicts Corpus, a dataset of N=5,772 naturalistic simulated dialogues spanning diverse conflict scenarios between friends, family members, and romantic partners. Through a controlled human study, we annotate a subset of dialogues and obtain fine-grained labels of communication breakdown types on individual turns, and assess the impact of backstory on human and model perception of conflict in conversation. We find that the polarity of relationship backstories significantly shifted human perception of communication breakdowns and impressions of the social partners, yet models struggle to meaningfully leverage those backstories in the detection task. Additionally, we find that models consistently overestimate how positively a message will make a listener feel. Our findings underscore the critical role of personalization to relationship contexts in enabling LLMs to serve as effective mediators in human communication for authentic connection.

Method: CoCoS uses online reinforcement learning with an accumulated reward function that aggregates rewards across the entire correction trajectory. It features a fine-grained reward system specifically designed for multi-turn code correction scenarios.

Result: With 1B-scale models, CoCoS achieves 35.8% improvement on MBPP and 27.7% improvement on HumanEval compared to baseline methods.

Conclusion: Small language models can be effectively trained for multi-turn self-correction through reinforcement learning objectives that encourage maintaining correct outputs while progressively fixing errors, leading to substantial performance gains in code generation tasks.

Abstract: Self-correction has demonstrated potential in code generation by allowing language models to revise and improve their outputs through successive refinement. Recent studies have explored prompting-based strategies that incorporate verification or feedback loops using proprietary models, as well as training-based methods that leverage their strong reasoning capabilities. However, whether smaller models possess the capacity to effectively guide their outputs through self-reflection remains unexplored. Our findings reveal that smaller models struggle to exhibit reflective revision behavior across both self-correction paradigms. In response, we introduce CoCoS, an approach designed to enhance the ability of small language models for multi-turn code correction. Specifically, we propose an online reinforcement learning objective that trains the model to confidently maintain correct outputs while progressively correcting incorrect outputs as turns proceed. Our approach features an accumulated reward function that aggregates rewards across the entire trajectory and a fine-grained reward better suited to multi-turn correction scenarios. This facilitates the model in enhancing initial response quality while achieving substantial improvements through self-correction. With 1B-scale models, CoCoS achieves improvements of 35.8% on the MBPP and 27.7% on HumanEval compared to the baselines.

Method: Developed SYCON Bench to evaluate sycophancy in multi-turn free-form conversations across three real-world scenarios, measuring Turn of Flip and Number of Flip metrics on 17 LLMs.

Result: Sycophancy remains prevalent; alignment tuning amplifies it, while model scaling and reasoning optimization help resist undesirable views. Third-person perspective prompting reduces sycophancy by up to 63.8%.

Conclusion: Multi-turn evaluation reveals sycophancy as a persistent failure mode in LLMs, with reasoning models performing better but still vulnerable when over-indexing on logic rather than addressing user beliefs directly.

Abstract: Large Language Models (LLMs) are expected to provide helpful and harmless responses, yet they often exhibit sycophancy–conforming to user beliefs regardless of factual accuracy or ethical soundness. Prior research on sycophancy has primarily focused on single-turn factual correctness, overlooking the dynamics of real-world interactions. In this work, we introduce SYCON Bench, a novel benchmark for evaluating sycophantic behavior in multi-turn, free-form conversational settings. Our benchmark measures how quickly a model conforms to the user (Turn of Flip) and how frequently it shifts its stance under sustained user pressure (Number of Flip). Applying SYCON Bench to 17 LLMs across three real-world scenarios, we find that sycophancy remains a prevalent failure mode. Our analysis shows that alignment tuning amplifies sycophantic behavior, whereas model scaling and reasoning optimization strengthen the model’s ability to resist undesirable user views. Reasoning models generally outperform instruction-tuned models but often fail when they over-index on logical exposition instead of directly addressing the user’s underlying beliefs. Finally, we evaluate four additional prompting strategies and demonstrate that adopting a third-person perspective reduces sycophancy by up to 63.8% in debate scenario. We release our code and data at https://github.com/JiseungHong/SYCON-Bench.

Method: Decomposes task into schema filtering, retrieval-augmented text-to-SQL generation, and prompt-driven schema linking. Uses fine-tuned LLMs for schema selection and explores CoT/GoT prompt engineering for enhanced reasoning.

Result: Comprehensive evaluations on Spider benchmarks demonstrate the effectiveness of AP-SQL.

Conclusion: AP-SQL successfully bridges the gap between resource efficiency and powerful Text-to-SQL capabilities through innovative architecture and prompt engineering techniques.

Abstract: Using the best Text-to-SQL methods in resource-constrained environments is challenging due to their reliance on resource-intensive open-source models. This paper introduces Auto Prompt SQL(AP-SQL), a novel architecture designed to bridge the gap between resource-efficient small open-source models and the powerful capabilities of large closed-source models for Text-to-SQL translation. Our method decomposes the task into schema filtering, retrieval-augmented text-to-SQL generation based on in-context examples, and prompt-driven schema linking and SQL generation. To improve schema selection accuracy, we fine-tune large language models. Crucially, we also explore the impact of prompt engineering throughout the process, leveraging Chain-of-Thought(CoT) and Graph-of-Thought(GoT) templates to significantly enhance the model’s reasoning for accurate SQL generation. Comprehensive evaluations on the Spider benchmarks demonstrate the effectiveness of AP-SQL.

Method: Developed automatic historical event extraction models using GPT-4, Claude, and Llama 3.2 with three strategies: pure base generation, knowledge graph enhancement, and RAG. Evaluated on Thucydides’ historical texts and created automated translation pipeline from RDF to Coq specifications.

Result: Enhancement strategies optimize different dimensions - base generation achieves best coverage/historical breadth (Claude/GPT-4), while RAG improves precision and metadata completeness. Model architecture determines enhancement sensitivity, with larger models showing robust baseline performance. Coq formalization validated RAG-discovered event types as legitimate semantic structures.

Conclusion: The approach successfully automates historical event extraction and enables higher-order reasoning beyond RDF capabilities, including multi-step causal verification, temporal arithmetic, and formal proofs about historical causation.

Abstract: Extracting structured computational representations of historical events from narrative text remains computationally expensive when constructed manually. While RDF/OWL reasoners enable graph-based reasoning, they are limited to fragments of first-order logic, preventing deeper temporal and semantic analysis. This paper addresses both challenges by developing automatic historical event extraction models using multiple LLMs (GPT-4, Claude, Llama 3.2) with three enhancement strategies: pure base generation, knowledge graph enhancement, and Retrieval-Augmented Generation (RAG). We conducted comprehensive evaluations using historical texts from Thucydides. Our findings reveal that enhancement strategies optimize different performance dimensions rather than providing universal improvements. For coverage and historical breadth, base generation achieves optimal performance with Claude and GPT-4 extracting comprehensive events. However, for precision, RAG enhancement improves coordinate accuracy and metadata completeness. Model architecture fundamentally determines enhancement sensitivity: larger models demonstrate robust baseline performance with incremental RAG improvements, while Llama 3.2 shows extreme variance from competitive performance to complete failure. We then developed an automated translation pipeline converting extracted RDF representations into Coq proof assistant specifications, enabling higher-order reasoning beyond RDF capabilities including multi-step causal verification, temporal arithmetic with BC dates, and formal proofs about historical causation. The Coq formalization validates that RAG-discovered event types represent legitimate domain-specific semantic structures rather than ontological violations.

Method: Structured pipeline that segments legal texts into atomic snippets, extracts deontic rules, and evaluates coherence using various LLM configurations including prompt engineering, fine-tuning, and multi-stage pipelines.

Result: Empirical evaluation shows promising alignment between machine-generated and expert-crafted formalizations, particularly with effective prompting strategies.

Conclusion: LLMs, especially when properly prompted, can significantly contribute to scalable legal informatics by automating semantic analysis of legal texts into formal logical representations.

Abstract: We present a novel approach to the automated semantic analysis of legal texts using large language models (LLMs), targeting their transformation into formal representations in Defeasible Deontic Logic (DDL). We propose a structured pipeline that segments complex normative language into atomic snippets, extracts deontic rules, and evaluates them for syntactic and semantic coherence. Our methodology is evaluated across various LLM configurations, including prompt engineering strategies, fine-tuned models, and multi-stage pipelines, focusing on legal norms from the Australian Telecommunications Consumer Protections Code. Empirical results demonstrate promising alignment between machine-generated and expert-crafted formalizations, showing that LLMs

• particularly when prompted effectively - can significantly contribute to scalable legal informatics.

Method: Three models tested on Buckeye corpus: 1) NDL with information-theoretic formulas, 2) traditional NDL predictors, 3) N-gram probabilistic predictors. Analysis focused on acoustic reduction modeling.

Result: N-gram model outperformed both NDL models, contradicting the cognitive advantage assumption of NDL. However, incorporating information-theoretic formulas into NDL improved its performance over traditional NDL.

Conclusion: Research highlights the need to incorporate frequency, contextual predictability, and average contextual predictability, and emphasizes combining information-theoretic metrics with discriminative learning for better acoustic reduction modeling.

Abstract: This study compares probabilistic predictors based on information theory with Naive Discriminative Learning (NDL) predictors in modeling acoustic word duration, focusing on probabilistic reduction. We examine three models using the Buckeye corpus: one with NDL-derived predictors using information-theoretic formulas, one with traditional NDL predictors, and one with N-gram probabilistic predictors. Results show that the N-gram model outperforms both NDL models, challenging the assumption that NDL is more effective due to its cognitive motivation. However, incorporating information-theoretic formulas into NDL improves model performance over the traditional model. This research highlights a) the need to incorporate not only frequency and contextual predictability but also average contextual predictability, and b) the importance of combining information-theoretic metrics of predictability and information derived from discriminative learning in modeling acoustic reduction.

Method: Three key innovations: (1) bit-flip unbiased reweighting mechanism for model-agnostic detection, (2) multilayer architecture to enhance detectability without compromising generation quality, and (3) information encoding approach supporting multi-bit watermarking.

Result: Achieves up to 30% higher extraction rates for short texts compared to state-of-the-art methods while maintaining text quality (lower perplexity) and performing comparably to non-watermarked text on downstream tasks like summarization and translation.

Conclusion: BiMark successfully addresses the critical requirements for practical LLM watermarking implementation by balancing text quality preservation with message embedding capacity through its innovative framework.

Abstract: Recent advances in Large Language Models (LLMs) have raised urgent concerns about LLM-generated text authenticity, prompting regulatory demands for reliable identification mechanisms. Although watermarking offers a promising solution, existing approaches struggle to simultaneously achieve three critical requirements: text quality preservation, model-agnostic detection, and message embedding capacity, which are crucial for practical implementation. To achieve these goals, the key challenge lies in balancing the trade-off between text quality preservation and message embedding capacity. To address this challenge, we propose BiMark, a novel watermarking framework that achieves these requirements through three key innovations: (1) a bit-flip unbiased reweighting mechanism enabling model-agnostic detection, (2) a multilayer architecture enhancing detectability without compromising generation quality, and (3) an information encoding approach supporting multi-bit watermarking. Through theoretical analysis and extensive experiments, we validate that, compared to state-of-the-art multi-bit watermarking methods, BiMark achieves up to 30% higher extraction rates for short texts while maintaining text quality indicated by lower perplexity, and performs comparably to non-watermarked text on downstream tasks such as summarization and translation.

Method: The authors define scoring bias as score differences when judge models are bias-related perturbed. They construct an evaluation dataset by augmenting existing benchmarks through data synthesis and design multi-faceted evaluation metrics to comprehensively assess scoring bias.

Result: Experimental results show that scoring stability of existing judge models is disrupted by scoring biases. The study provides insights into scoring prompt design and bias mitigation strategies related to score rubrics, score IDs, and reference answer selection.

Conclusion: The research highlights significant scoring bias issues in LLM-as-a-Judge systems and provides a framework for evaluating and mitigating these biases, offering valuable guidance for improving the fairness and reliability of LLM-based evaluations.

Abstract: The remarkable performance of Large Language Models (LLMs) gives rise to``LLM-as-a-Judge’’, where LLMs are employed as evaluators for complex tasks. Moreover, it has been widely adopted across fields such as Natural Language Processing (NLP), preference learning, and various specific domains. However, there are various biases within LLM-as-a-Judge, which adversely affect the fairness and reliability of judgments. Current research on evaluating or mitigating bias in LLM-as-a-Judge predominantly focuses on comparison-based evaluations, while systematic investigations into bias in scoring-based evaluations remain limited. Therefore, we define scoring bias in LLM-as-a-Judge as the scores differ when scoring judge models are bias-related perturbed, and provide a well-designed framework to comprehensively evaluate scoring bias. We augment existing LLM-as-a-Judge benchmarks through data synthesis to construct our evaluation dataset and design multi-faceted evaluation metrics. Our experimental results demonstrate that the scoring stability of existing judge models is disrupted by scoring biases. Further exploratory experiments and discussions provide valuable insights into the design of scoring prompt templates and the mitigation of scoring biases on aspects such as score rubrics, score IDs, and reference answer selection.

Method: Uses mixture-of-experts architecture where each expert is trained independently on closed datasets, integrated through domain-informed routing without joint training. Trained on FlexMix corpus with public and domain-specific datasets.

Result: 37B parameter models show 41% relative improvement when combining general expert with domain experts, outperforms prior model merging by 10.1% and standard MoE with same FLOPs.

Conclusion: Provides solution for regulated industries with sensitive data, enabling benefits from closed data while keeping data local and supporting fine-grained access control during inference.

Abstract: We introduce FlexOlmo, a new class of language models (LMs) that supports (1) distributed training without data sharing, where different model parameters are independently trained on closed datasets, and (2) data-flexible inference, where these parameters along with their associated data can be flexibly included or excluded from model inferences with no further training. FlexOlmo employs a mixture-of-experts (MoE) architecture where each expert is trained independently on closed datasets and later integrated through a new domain-informed routing without any joint training. FlexOlmo is trained on FlexMix, a corpus we curate comprising publicly available datasets alongside seven domain-specific sets, representing realistic approximations of closed sets. We evaluate models with up to 37 billion parameters (20 billion active) on 31 diverse downstream tasks. We show that a general expert trained on public data can be effectively combined with independently trained experts from other data owners, leading to an average 41% relative improvement while allowing users to opt out of certain data based on data licensing or permission requirements. Our approach also outperforms prior model merging methods by 10.1% on average and surpasses the standard MoE trained without data restrictions using the same training FLOPs. Altogether, this research presents a solution for both data owners and researchers in regulated industries with sensitive or protected data. FlexOlmo enables benefiting from closed data while respecting data owners’ preferences by keeping their data local and supporting fine-grained control of data access during inference.

Method: Protocol-driven framework with Agent-to-Agent communication layer based on Model Context Protocol, Experience Pack architecture for layered memory, multi-turn dialogue, memory modules, and dynamic safety validation.

Result: Empirical results show superior adaptability, coordination, and experience reuse compared to baseline frameworks on complex task orchestration benchmarks.

Conclusion: GoalfyMax provides a scalable, future-ready foundation for multi-agent intelligent systems with robust real-time strategy adaptation capabilities.

Abstract: Modern enterprise environments demand intelligent systems capable of handling complex, dynamic, and multi-faceted tasks with high levels of autonomy and adaptability. However, traditional single-purpose AI systems often lack sufficient coordination, memory reuse, and task decomposition capabilities, limiting their scalability in realistic settings. To address these challenges, we present \textbf{GoalfyMax}, a protocol-driven framework for end-to-end multi-agent collaboration. GoalfyMax introduces a standardized Agent-to-Agent (A2A) communication layer built on the Model Context Protocol (MCP), allowing independent agents to coordinate through asynchronous, protocol-compliant interactions. It incorporates the Experience Pack (XP) architecture, a layered memory system that preserves both task rationales and execution traces, enabling structured knowledge retention and continual learning. Moreover, our system integrates advanced features including multi-turn contextual dialogue, long-short term memory modules, and dynamic safety validation, supporting robust, real-time strategy adaptation. Empirical results on complex task orchestration benchmarks and case study demonstrate that GoalfyMax achieves superior adaptability, coordination, and experience reuse compared to baseline frameworks. These findings highlight its potential as a scalable, future-ready foundation for multi-agent intelligent systems.

Method: Proposes CRABS strategy: uses shallow syntactic parsing and AST analysis to capture notebook interpretation bounds, then employs LLM for zero-shot learning to resolve remaining ambiguities in cell I/O.

Result: LLM correctly resolves 98% of ambiguities (1397/1425). CRABS achieves 98% F1 for information flows and 99% F1 for execution dependencies across 50 Kaggle notebooks.

Conclusion: The pincer strategy combining syntactic analysis with LLM resolution effectively understands notebooks without execution, enabling better evaluation and reuse of data science workflows.

Abstract: Recognizing the information flows and operations comprising data science and machine learning Python notebooks is critical for evaluating, reusing, and adapting notebooks for new tasks. Investigating a notebook via re-execution often is impractical due to the challenges of resolving data and software dependencies. While Large Language Models (LLMs) pre-trained on large codebases have demonstrated effectiveness in understanding code without running it, we observe that they fail to understand some realistic notebooks due to hallucinations and long-context challenges. To address these issues, we propose a notebook understanding task yielding an information flow graph and corresponding cell execution dependency graph for a notebook, and demonstrate the effectiveness of a pincer strategy that uses limited syntactic analysis to assist full comprehension of the notebook using an LLM. Our Capture and Resolve Assisted Bounding Strategy (CRABS) employs shallow syntactic parsing and analysis of the abstract syntax tree (AST) to capture the correct interpretation of a notebook between lower and upper estimates of the inter-cell I/O set$\unicode{x2014}$the flows of information into or out of cells via variables$\unicode{x2014}$then uses an LLM to resolve remaining ambiguities via cell-by-cell zero-shot learning, thereby identifying the true data inputs and outputs of each cell. We evaluate and demonstrate the effectiveness of our approach using an annotated dataset of 50 representative, highly up-voted Kaggle notebooks that together represent 3454 actual cell inputs and outputs. The LLM correctly resolves 1397 of 1425 (98%) ambiguities left by analyzing the syntactic structure of these notebooks. Across 50 notebooks, CRABS achieves average F1 scores of 98% identifying cell-to-cell information flows and 99% identifying transitive cell execution dependencies.

Method: Question Augmentation (QuestA) - introducing partial solutions during training to reduce problem difficulty and provide more informative learning signals during RL training.

Result: Achieved new state-of-the-art results: 67.1% (+5.3%) on AIME24, 59.5% (+10.0%) on AIME25, and 35.5% (+4.0%) on HMMT25 using 1.5B-parameter models. Improved both pass@1 and pass@k performance.

Conclusion: QuestA provides a practical and generalizable pathway for expanding reasoning capability through RL, with theoretical explanations showing improved sample efficiency. The method enables continual improvement over strong open-source models.

Abstract: Reinforcement learning (RL) has become a key component in training large language reasoning models (LLMs). However, recent studies questions its effectiveness in improving multi-step reasoning-particularly on hard problems. To address this challenge, we propose a simple yet effective strategy via Question Augmentation: introduce partial solutions during training to reduce problem difficulty and provide more informative learning signals. Our method, QuestA, when applied during RL training on math reasoning tasks, not only improves pass@1 but also pass@k-particularly on problems where standard RL struggles to make progress. This enables continual improvement over strong open-source models such as DeepScaleR and OpenMath Nemotron, further enhancing their reasoning capabilities. We achieve new state-of-the-art results on math benchmarks using 1.5B-parameter models: 67.1% (+5.3%) on AIME24, 59.5% (+10.0%) on AIME25, and 35.5% (+4.0%) on HMMT25. Further, we provide theoretical explanations that QuestA improves sample efficiency, offering a practical and generalizable pathway for expanding reasoning capability through RL.

Method: Few-shot Adaptive Linguistic Prompting (ALP) guides large language models (GPT-4o and Gemini 1.5 Pro) to analyze phishing webpages through structured semantic reasoning, breaking down linguistic patterns, detecting urgency cues, and identifying manipulative diction in multimodal content.

Result: ALP significantly enhances phishing detection accuracy, achieving an F1-score of 0.93, which surpasses traditional approaches by providing structured reasoning and contextual analysis capabilities.

Conclusion: ALP-integrated multimodal LLMs establish a foundation for robust, interpretable, and adaptive linguistic-based phishing detection systems, demonstrating strong potential for advancing cybersecurity frameworks.

Abstract: Phishing attacks represent a significant cybersecurity threat, necessitating adaptive detection techniques. This study explores few-shot Adaptive Linguistic Prompting (ALP) in detecting phishing webpages through the multimodal capabilities of state-of-the-art large language models (LLMs) such as GPT-4o and Gemini 1.5 Pro. ALP is a structured semantic reasoning method that guides LLMs to analyze textual deception by breaking down linguistic patterns, detecting urgency cues, and identifying manipulative diction commonly found in phishing content. By integrating textual, visual, and URL-based analysis, we propose a unified model capable of identifying sophisticated phishing attempts. Our experiments demonstrate that ALP significantly enhances phishing detection accuracy by guiding LLMs through structured reasoning and contextual analysis. The findings highlight the potential of ALP-integrated multimodal LLMs to advance phishing detection frameworks, achieving an F1-score of 0.93, surpassing traditional approaches. These results establish a foundation for more robust, interpretable, and adaptive linguistic-based phishing detection systems using LLMs.

Method: Evaluated 6 OCR engines (commercial and open-source) using 5 measurement techniques to assess character and word-level accuracy on Sinhala and Tamil texts.

Result: Surya achieved best performance for Sinhala (WER 2.61%), Document AI excelled for Tamil (CER 0.78%). Also created a novel synthetic Tamil OCR benchmarking dataset.

Conclusion: Different OCR engines perform optimally for different low-resource languages, with Surya and Document AI showing superior performance for Sinhala and Tamil respectively.

Abstract: Solving the problem of Optical Character Recognition (OCR) on printed text for Latin and its derivative scripts can now be considered settled due to the volumes of research done on English and other High-Resourced Languages (HRL). However, for Low-Resourced Languages (LRL) that use unique scripts, it remains an open problem. This study presents a comparative analysis of the zero-shot performance of six distinct OCR engines on two LRLs: Sinhala and Tamil. The selected engines include both commercial and open-source systems, aiming to evaluate the strengths of each category. The Cloud Vision API, Surya, Document AI, and Tesseract were evaluated for both Sinhala and Tamil, while Subasa OCR and EasyOCR were examined for only one language due to their limitations. The performance of these systems was rigorously analysed using five measurement techniques to assess accuracy at both the character and word levels. According to the findings, Surya delivered the best performance for Sinhala across all metrics, with a WER of 2.61%. Conversely, Document AI excelled across all metrics for Tamil, highlighted by a very low CER of 0.78%. In addition to the above analysis, we also introduce a novel synthetic Tamil OCR benchmarking dataset.

Method: Break down knowledge extraction into NER, coreference resolution, entity linking, and relation extraction components. Evaluate 16 NLP tools and LLMs using zero-shot performance on FAA maintenance dataset in controlled environments without third-party data sharing.

Result: Significant performance limitations observed in both traditional NLP tools and LLMs when operating in confidential, trusted environments without data sharing to third parties.

Conclusion: Current NLP and LLM tools have low Technical Readiness Level for mission-critical aviation applications. Recommendations provided to enhance trust, along with open-source curated dataset for further baseline testing.

Abstract: Deriving operational intelligence from organizational data repositories is a key challenge due to the dichotomy of data confidentiality vs data integration objectives, as well as the limitations of Natural Language Processing (NLP) tools relative to the specific knowledge structure of domains such as operations and maintenance. In this work, we discuss Knowledge Graph construction and break down the Knowledge Extraction process into its Named Entity Recognition, Coreference Resolution, Named Entity Linking, and Relation Extraction functional components. We then evaluate sixteen NLP tools in concert with or in comparison to the rapidly advancing capabilities of Large Language Models (LLMs). We focus on the operational and maintenance intelligence use case for trusted applications in the aircraft industry. A baseline dataset is derived from a rich public domain US Federal Aviation Administration dataset focused on equipment failures or maintenance requirements. We assess the zero-shot performance of NLP and LLM tools that can be operated within a controlled, confidential environment (no data is sent to third parties). Based on our observation of significant performance limitations, we discuss the challenges related to trusted NLP and LLM tools as well as their Technical Readiness Level for wider use in mission-critical industries such as aviation. We conclude with recommendations to enhance trust and provide our open-source curated dataset to support further baseline testing and evaluation.

Method: Radiologists contributed fictional reports following standard practices (minimum 20 per contributor) with metadata including anatomical region, modality, clinical context, and English translations for non-English reports. All reports were assigned ICD-10 codes. A human vs. AI differentiation study was conducted with 154 participants.

Result: Dataset contains 2,658 reports from 76 authors across 21 countries and 13 languages, covering multiple modalities (CT 36.1%, MRI 22.8%, etc.) and anatomical regions. Participants achieved 53.9% accuracy in distinguishing human vs AI reports, with radiologists performing significantly better (56.9%).

Conclusion: PARROT represents the largest open multilingual radiology report dataset, enabling development and validation of NLP applications across linguistic, geographic, and clinical boundaries without privacy concerns.

Abstract: Rationale and Objectives: To develop and validate PARROT (Polyglottal Annotated Radiology Reports for Open Testing), a large, multicentric, open-access dataset of fictional radiology reports spanning multiple languages for testing natural language processing applications in radiology. Materials and Methods: From May to September 2024, radiologists were invited to contribute fictional radiology reports following their standard reporting practices. Contributors provided at least 20 reports with associated metadata including anatomical region, imaging modality, clinical context, and for non-English reports, English translations. All reports were assigned ICD-10 codes. A human vs. AI report differentiation study was conducted with 154 participants (radiologists, healthcare professionals, and non-healthcare professionals) assessing whether reports were human-authored or AI-generated. Results: The dataset comprises 2,658 radiology reports from 76 authors across 21 countries and 13 languages. Reports cover multiple imaging modalities (CT: 36.1%, MRI: 22.8%, radiography: 19.0%, ultrasound: 16.8%) and anatomical regions, with chest (19.9%), abdomen (18.6%), head (17.3%), and pelvis (14.1%) being most prevalent. In the differentiation study, participants achieved 53.9% accuracy (95% CI: 50.7%-57.1%) in distinguishing between human and AI-generated reports, with radiologists performing significantly better (56.9%, 95% CI: 53.3%-60.6%, p<0.05) than other groups. Conclusion: PARROT represents the largest open multilingual radiology report dataset, enabling development and validation of natural language processing applications across linguistic, geographic, and clinical boundaries without privacy constraints.

Method: Proposed an agentic RAG framework that allows LLMs to decompose radiology questions, iteratively retrieve targeted clinical evidence from Radiopaedia.org, and dynamically synthesize evidence-based responses. Evaluated 25 LLMs across diverse architectures and scales using expert-curated radiology questions.

Result: Agentic retrieval significantly improved mean diagnostic accuracy over zero-shot prompting and conventional RAG, with greatest gains in small-scale models. Reduced hallucinations by mean 9.4%, retrieved clinically relevant context in 46% of cases, and improved factual grounding. Even clinically fine-tuned models showed benefits.

Conclusion: Agentic frameworks enhance factuality and diagnostic accuracy in radiology QA, demonstrating that retrieval remains beneficial despite embedded domain knowledge. The approach shows promise for clinical utility and all resources are publicly available for open research.

Abstract: Clinical decision-making in radiology increasingly benefits from artificial intelligence (AI), particularly through large language models (LLMs). However, traditional retrieval-augmented generation (RAG) systems for radiology question answering (QA) typically rely on single-step retrieval, limiting their ability to handle complex clinical reasoning tasks. Here we propose an agentic RAG framework enabling LLMs to autonomously decompose radiology questions, iteratively retrieve targeted clinical evidence from Radiopaedia.org, and dynamically synthesize evidence-based responses. We evaluated 25 LLMs spanning diverse architectures, parameter scales (0.5B to >670B), and training paradigms (general-purpose, reasoning-optimized, clinically fine-tuned), using 104 expert-curated radiology questions from previously established RSNA-RadioQA and ExtendedQA datasets. To assess generalizability, we additionally tested on an unseen internal dataset of 65 real-world radiology board examination questions. Agentic retrieval significantly improved mean diagnostic accuracy over zero-shot prompting and conventional online RAG. The greatest gains occurred in small-scale models, while very large models (>200B parameters) demonstrated minimal changes (<2% improvement). Additionally, agentic retrieval reduced hallucinations (mean 9.4%) and retrieved clinically relevant context in 46% of cases, substantially aiding factual grounding. Even clinically fine-tuned models showed gains from agentic retrieval (e.g., MedGemma-27B), indicating that retrieval remains beneficial despite embedded domain knowledge. These results highlight the potential of agentic frameworks to enhance factuality and diagnostic accuracy in radiology QA, warranting future studies to validate their clinical utility. All datasets, code, and the full agentic framework are publicly available to support open research and clinical translation.

Method: A four-stage synthetic data generation and filtering pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This converts and expands the English Nemotron-Content-Safety-Dataset-V2 into 8 languages (Arabic, German, Spanish, French, Hindi, Japanese, Thai, Chinese).

Result: Created Nemotron-Content-Safety-Dataset-Multilingual-v1 with 386,661 samples across 9 languages. Trained Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1 via LoRA fine-tuning, achieving state-of-the-art performance on multilingual safety benchmarks. Found that open LLMs are more prone to unsafe responses in non-English languages.

Conclusion: This work significantly advances multilingual LLM safety by enabling development of culturally aware safety guard models, closing the safety gap between English and non-English languages in LLM applications.

Abstract: The increasing use of Large Language Models (LLMs) in agentic applications highlights the need for robust safety guard models. While content safety in English is well-studied, non-English languages lack similar advancements due to the high cost of collecting culturally aligned labeled datasets. We present CultureGuard, a novel solution for curating culturally aligned, high-quality safety datasets across multiple languages. Our approach introduces a four-stage synthetic data generation and filtering pipeline: cultural data segregation, cultural data adaptation, machine translation, and quality filtering. This pipeline enables the conversion and expansion of the Nemotron-Content-Safety-Dataset-V2 English safety dataset into eight distinct languages: Arabic, German, Spanish, French, Hindi, Japanese, Thai, and Chinese. The resulting dataset, Nemotron-Content-Safety-Dataset-Multilingual-v1, comprises 386,661 samples in 9 languages and facilitates the training of Llama-3.1-Nemotron-Safety-Guard-Multilingual-8B-v1 via LoRA-based fine-tuning. The final model achieves state-of-the-art performance on several multilingual content safety benchmarks. We also benchmark the latest open LLMs on multilingual safety and observe that these LLMs are more prone to give unsafe responses when prompted in non-English languages. This work represents a significant step toward closing the safety gap in multilingual LLMs by enabling the development of culturally aware safety guard models.

Method: Developed Jinx as a helpful-only variant of popular open-weight LLMs that preserves base model capabilities while removing safety alignment constraints and refusal mechanisms.

Result: Created an accessible research tool that responds to all queries without refusals, enabling systematic study of alignment failures and safety boundary evaluation.

Conclusion: Jinx provides researchers with a valuable resource for probing language model safety vulnerabilities and studying failure modes that would otherwise be inaccessible.

Abstract: Unlimited, or so-called helpful-only language models are trained without safety alignment constraints and never refuse user queries. They are widely used by leading AI companies as internal tools for red teaming and alignment evaluation. For example, if a safety-aligned model produces harmful outputs similar to an unlimited model, this indicates alignment failures that require further attention. Despite their essential role in assessing alignment, such models are not available to the research community. We introduce Jinx, a helpful-only variant of popular open-weight LLMs. Jinx responds to all queries without refusals or safety filtering, while preserving the base model’s capabilities in reasoning and instruction following. It provides researchers with an accessible tool for probing alignment failures, evaluating safety boundaries, and systematically studying failure modes in language model safety.

Method: Used contrastive stimulus sets to localize top-activated units across 11 LLMs and 5 VLMs (3B-90B parameters), then performed targeted ablations to assess causal role by comparing effects of lesioning functionally selected units vs low-activation and randomly selected units on downstream task accuracy.

Result: Contrary to expectations, low-activation units sometimes produced larger performance drops than highly activated ones, and mathematical localizer units often impaired ToM performance more than ToM localizer units themselves.

Conclusion: The findings challenge the causal relevance of contrast-based localizers and highlight the need for broader stimulus sets and more accurate methods to capture truly task-specific units in AI models.

Abstract: This work adapts a neuroscientific contrast localizer to pinpoint causally relevant units for Theory of Mind (ToM) and mathematical reasoning tasks in large language models (LLMs) and vision-language models (VLMs). Across 11 LLMs and 5 VLMs ranging in size from 3B to 90B parameters, we localize top-activated units using contrastive stimulus sets and assess their causal role via targeted ablations. We compare the effect of lesioning functionally selected units against low-activation and randomly selected units on downstream accuracy across established ToM and mathematical benchmarks. Contrary to expectations, low-activation units sometimes produced larger performance drops than the highly activated ones, and units derived from the mathematical localizer often impaired ToM performance more than those from the ToM localizer. These findings call into question the causal relevance of contrast-based localizers and highlight the need for broader stimulus sets and more accurately capture task-specific units.

Method: Introduce a novel benchmark comprising a suite of multi-turn tasks designed to test specific reasoning, interactive dialogue, and information-seeking abilities with deterministic scoring mechanisms.

Result: Evaluation of frontier models shows significant headroom for improvement, with most errors stemming from poor instruction following, reasoning failures, and poor planning.

Conclusion: The benchmark provides valuable insights into current LLMs’ strengths and weaknesses in handling complex interactive scenarios and offers a robust platform for future research to improve these critical capabilities.

Abstract: Large language models (LLMs) excel at solving problems with clear and complete statements, but often struggle with nuanced environments or interactive tasks which are common in most real-world scenarios. This highlights the critical need for developing LLMs that can effectively engage in logically consistent multi-turn dialogue, seek information and reason with incomplete data. To this end, we introduce a novel benchmark comprising a suite of multi-turn tasks each designed to test specific reasoning, interactive dialogue, and information-seeking abilities. These tasks have deterministic scoring mechanisms, thus eliminating the need for human intervention. Evaluating frontier models on our benchmark reveals significant headroom. Our analysis shows that most errors emerge from poor instruction following, reasoning failures, and poor planning. This benchmark provides valuable insights into the strengths and weaknesses of current LLMs in handling complex, interactive scenarios and offers a robust platform for future research aimed at improving these critical capabilities.

Method: Hierarchical framework with horizontal decomposition of algorithms into modular functional blocks and vertical step-by-step fine-grained translation using external toolchains (MATLAB, Vitis HLS) for debugging and circuit-level synthesis.

Result: Validated through real-world deployment in 5G wireless communication domain, demonstrating practicality, reliability, and deployment efficiency while mitigating LLM hallucination issues.

Conclusion: A2HCoder enables agile and reliable algorithm-to-hardware translation, enhancing robustness and interpretability while suppressing common hallucination problems in LLM-generated code for wireless communication systems.

Abstract: In wireless communication systems, stringent requirements such as ultra-low latency and power consumption have significantly increased the demand for efficient algorithm-to-hardware deployment. However, a persistent and substantial gap remains between algorithm design and hardware implementation. Bridging this gap traditionally requires extensive domain expertise and time-consuming manual development, due to fundamental mismatches between high-level programming languages like MATLAB and hardware description languages (HDLs) such as Verilog-in terms of memory access patterns, data processing manners, and datatype representations. To address this challenge, we propose A2HCoder: a Hierarchical Algorithm-to-HDL Coding Agent, powered by large language models (LLMs), designed to enable agile and reliable algorithm-to-hardware translation. A2HCoder introduces a hierarchical framework that enhances both robustness and interpretability while suppressing common hallucination issues in LLM-generated code. In the horizontal dimension, A2HCoder decomposes complex algorithms into modular functional blocks, simplifying code generation and improving consistency. In the vertical dimension, instead of relying on end-to-end generation, A2HCoder performs step-by-step, fine-grained translation, leveraging external toolchains such as MATLAB and Vitis HLS for debugging and circuit-level synthesis. This structured process significantly mitigates hallucinations and ensures hardware-level correctness. We validate A2HCoder through a real-world deployment case in the 5G wireless communication domain, demonstrating its practicality, reliability, and deployment efficiency.

Method: The framework leverages expert-derived hierarchical Tree-of-Thought components refined through alignment with quality dimensions and expert feedback, then applies this optimized expert ToT to evaluate various LLM scales.

Result: Systematically optimized expert ToT achieved 4.99-14.15% relative quality gains over baselines. Scaling laws persist in specialized domains, but reasoning-enhanced smaller models can close the gap. Sub-72B models with explicit reasoning architectures outperform counterparts in accuracy and conciseness.

Conclusion: LETToT establishes a scalable, label-free paradigm for domain-specific LLM evaluation, providing a robust alternative to conventional annotated benchmarks while offering insights into model performance across different scales and architectures.

Abstract: Evaluating large language models (LLMs) in specific domain like tourism remains challenging due to the prohibitive cost of annotated benchmarks and persistent issues like hallucinations. We propose $\textbf{L}$able-Free $\textbf{E}$valuation of LLM on $\textbf{T}$ourism using Expert $\textbf{T}$ree-$\textbf{o}$f-$\textbf{T}$hought (LETToT), a framework that leverages expert-derived reasoning structures-instead of labeled data-to access LLMs in tourism. First, we iteratively refine and validate hierarchical ToT components through alignment with generic quality dimensions and expert feedback. Results demonstrate the effectiveness of our systematically optimized expert ToT with 4.99-14.15% relative quality gains over baselines. Second, we apply LETToT’s optimized expert ToT to evaluate models of varying scales (32B-671B parameters), revealing: (1) Scaling laws persist in specialized domains (DeepSeek-V3 leads), yet reasoning-enhanced smaller models (e.g., DeepSeek-R1-Distill-Llama-70B) close this gap; (2) For sub-72B models, explicit reasoning architectures outperform counterparts in accuracy and conciseness ($p<0.05$). Our work established a scalable, label-free paradigm for domain-specific LLM evaluation, offering a robust alternative to conventional annotated benchmarks.

Method: SupraTok extends Byte-Pair Encoding with three innovations: cross-boundary pattern learning for multi-word semantic units, entropy-driven data curation for optimal training corpus quality, and multi-phase curriculum learning for stable convergence. It learns “superword” tokens that preserve semantic unity while maximizing compression.

Result: 31% improvement in English tokenization efficiency (5.91 vs 4.51 characters per token) compared to OpenAI’s o200k and 30% improvement over Google’s Gemma 3. Maintains competitive performance across 38 languages. When integrated with GPT-2 scale model, yields 8.4% improvement on HellaSWAG and 9.5% on MMLU benchmarks.

Conclusion: Efficient tokenization can complement architectural innovations to improve language model performance. While promising results at current scale, further validation at larger model scales is needed.

Abstract: Tokenization remains a fundamental yet underexplored bottleneck in natural language processing, with strategies largely static despite remarkable progress in model architectures. We present SupraTok, a novel tokenization architecture that reimagines subword segmentation through three innovations: cross-boundary pattern learning that discovers multi-word semantic units, entropy-driven data curation that optimizes training corpus quality, and multi-phase curriculum learning for stable convergence. Our approach extends Byte-Pair Encoding by learning “superword” tokens, coherent multi-word expressions that preserve semantic unity while maximizing compression efficiency. SupraTok achieves 31% improvement in English tokenization efficiency (5.91 versus 4.51 characters per token) compared to OpenAI’s o200k tokenizer and 30% improvement over Google’s Gemma 3 tokenizer (256k vocabulary), while maintaining competitive performance across 38 languages. When integrated with a GPT-2 scale model (124M parameters) trained on 10 billion tokens from the FineWeb-Edu dataset, SupraTok yields 8.4% improvement on HellaSWAG and 9.5% on MMLU benchmarks without architectural modifications. While these results are promising at this scale, further validation at larger model scales is needed. These findings suggest that efficient tokenization can complement architectural innovations as a path to improved language model performance.

Method: Created SEA-BED benchmark with 169 datasets (71% human-formulated) across 9 tasks and 10 languages, then evaluated 17 embedding models through six studies analyzing task challenges, cross-benchmark comparisons, and translation effects.

Result: Results show sharp ranking shifts, inconsistent model performance among SEA languages, and demonstrate the critical importance of human-curated datasets for low-resource languages like Burmese.

Conclusion: Human-curated benchmarks are essential for accurate evaluation of multilingual embedding models in Southeast Asian languages, as machine-translated datasets fail to capture linguistic nuances and lead to misleading performance assessments.

Abstract: Sentence embeddings are essential for NLP tasks such as semantic search, re-ranking, and textual similarity. Although multilingual benchmarks like MMTEB broaden coverage, Southeast Asia (SEA) datasets are scarce and often machine-translated, missing native linguistic properties. With nearly 700 million speakers, the SEA region lacks a region-specific embedding benchmark. We introduce SEA-BED, the first large-scale SEA embedding benchmark with 169 datasets across 9 tasks and 10 languages, where 71% are formulated by humans, not machine generation or translation. We address three research questions: (1) which SEA languages and tasks are challenging, (2) whether SEA languages show unique performance gaps globally, and (3) how human vs. machine translations affect evaluation. We evaluate 17 embedding models across six studies, analyzing task and language challenges, cross-benchmark comparisons, and translation trade-offs. Results show sharp ranking shifts, inconsistent model performance among SEA languages, and the importance of human-curated datasets for low-resource languages like Burmese.

Method: Analyzes text representations in frequency domain, uses low frequency domain filtering to remove domain-sensitive features, and employs dynamic spectrum alignment to extract domain-invariant features.

Result: Outperforms state-of-the-art baselines by average of 0.90% in accuracy and 0.92% in F1 score on 11 test datasets across three domain-generalization scenarios.

Conclusion: Frequency domain analysis reveals consistent spectral patterns across domains and significant magnitude discrepancies between machine-generated and human-written texts, enabling better domain generalization in detection.

Abstract: Large Language Models have shown growing ability to generate fluent and coherent texts that are highly similar to the writing style of humans. Current detectors for Machine-Generated Text (MGT) perform well when they are trained and tested in the same domain but generalize poorly to unseen domains, due to domain shift between data from different sources. In this work, we propose MGT-Prism, an MGT detection method from the perspective of the frequency domain for better domain generalization. Our key insight stems from analyzing text representations in the frequency domain, where we observe consistent spectral patterns across diverse domains, while significant discrepancies in magnitude emerge between MGT and human-written texts (HWTs). The observation initiates the design of a low frequency domain filtering module for filtering out the document-level features that are sensitive to domain shift, and a dynamic spectrum alignment strategy to extract the task-specific and domain-invariant features for improving the detector’s performance in domain generalization. Extensive experiments demonstrate that MGT-Prism outperforms state-of-the-art baselines by an average of 0.90% in accuracy and 0.92% in F1 score on 11 test datasets across three domain-generalization scenarios.

Method: DPad uses two strategies: (1) sliding window to maintain fixed-length suffix window, and (2) distance-decay dropout to deterministically remove distant suffix tokens before attention computation.

Result: DPad delivers up to 61.4x speedup over vanilla dLLMs while maintaining comparable accuracy across multiple benchmarks on LLaDA-1.5 and Dream models.

Conclusion: DPad provides an efficient and scalable solution for long-sequence inference in diffusion-based LLMs, is compatible with existing optimizations, and requires minimal code changes.

Abstract: Diffusion-based Large Language Models (dLLMs) parallelize text generation by framing decoding as a denoising process, but suffer from high computational overhead since they predict all future suffix tokens at each step while retaining only a small fraction. We propose Diffusion Scratchpad (DPad), a training-free method that restricts attention to a small set of nearby suffix tokens, preserving fidelity while eliminating redundancy. DPad integrates two strategies: (i) a sliding window, which maintains a fixed-length suffix window, and (ii) distance-decay dropout, which deterministically removes distant suffix tokens before attention computation. This simple design is compatible with existing optimizations such as prefix caching and can be implemented with only a few lines of code. Comprehensive evaluations across multiple benchmarks on LLaDA-1.5 and Dream models demonstrate that DPad delivers up to $\mathbf{61.4\times}$ speedup over vanilla dLLMs while maintaining comparable accuracy, highlighting its potential for efficient and scalable long-sequence inference. Our code is available at https://github.com/Crys-Chen/DPad.

Method: Created SciIG task with datasets from NAACL 2025 and ICLR 2025 papers. Evaluated 5 state-of-the-art models using automated metrics and LLM-as-a-judge across multiple dimensions including lexical overlap, semantic similarity, content coverage, faithfulness, consistency, citation correctness, and narrative quality.

Result: LLaMA-4 Maverick demonstrated superior performance on most metrics, particularly excelling in semantic similarity and faithfulness. Three-shot prompting consistently outperformed fewer-shot approaches.

Conclusion: The findings provide practical insights for developing effective research writing assistants and set realistic expectations for LLM-assisted academic writing. All code and datasets will be publicly released for reproducibility.

Abstract: As researchers increasingly adopt LLMs as writing assistants, generating high-quality research paper introductions remains both challenging and essential. We introduce Scientific Introduction Generation (SciIG), a task that evaluates LLMs’ ability to produce coherent introductions from titles, abstracts, and related works. Curating new datasets from NAACL 2025 and ICLR 2025 papers, we assess five state-of-the-art models, including both open-source (DeepSeek-v3, Gemma-3-12B, LLaMA 4-Maverick, MistralAI Small 3.1) and closed-source GPT-4o systems, across multiple dimensions: lexical overlap, semantic similarity, content coverage, faithfulness, consistency, citation correctness, and narrative quality. Our comprehensive framework combines automated metrics with LLM-as-a-judge evaluations. Results demonstrate LLaMA-4 Maverick’s superior performance on most metrics, particularly in semantic similarity and faithfulness. Moreover, three-shot prompting consistently outperforms fewer-shot approaches. These findings provide practical insights into developing effective research writing assistants and set realistic expectations for LLM-assisted academic writing. To foster reproducibility and future research, we will publicly release all code and datasets.

Method: Construct a corpus with different function-calling behaviors, train a behavior-aligned retriever using contrastive learning with customized positive/negative pairs and dual-negative contrastive loss to ensure robust retrieval of behaviorally consistent examples.

Result: The approach significantly reduces erroneous function calls while maintaining high task performance.

Conclusion: This offers a cost-effective and efficient solution for tool-augmented LLMs by providing behaviorally consistent demonstrations to guide accurate tool-using decisions.

Abstract: Tool-augmented large language models (LLMs) leverage external functions to extend their capabilities, but inaccurate function calls can lead to inefficiencies and increased costs.Existing methods address this challenge by fine-tuning LLMs or using demonstration-based prompting, yet they often suffer from high training overhead and fail to account for inconsistent demonstration samples, which misguide the model’s invocation behavior. In this paper, we trained a behavior-aligned retriever (BAR), which provides behaviorally consistent demonstrations to help LLMs make more accurate tool-using decisions. To train the BAR, we construct a corpus including different function-calling behaviors, i.e., calling or non-calling.We use the contrastive learning framework to train the BAR with customized positive/negative pairs and a dual-negative contrastive loss, ensuring robust retrieval of behaviorally consistent examples.Experiments demonstrate that our approach significantly reduces erroneous function calls while maintaining high task performance, offering a cost-effective and efficient solution for tool-augmented LLMs.

Method: Introduces ZPD-SCA benchmark annotated by top 0.15% Special Grade teachers, testing LLMs in zero-shot and in-context learning scenarios across different student age groups and genres.

Result: LLMs perform poorly in zero-shot scenarios (some below random guessing), improve substantially with in-context examples (nearly double accuracy), but show systematic directional biases and significant genre-based performance variations.

Conclusion: LLMs have emerging abilities for reading difficulty assessment but current training has limitations for educationally aligned judgment; ZPD-SCA provides foundation for evaluating and improving LLMs in cognitively aligned educational applications.

Abstract: Large language models (LLMs) have demonstrated potential in educational applications, yet their capacity to accurately assess the cognitive alignment of reading materials with students’ developmental stages remains insufficiently explored. This gap is particularly critical given the foundational educational principle of the Zone of Proximal Development (ZPD), which emphasizes the need to match learning resources with Students’ Cognitive Abilities (SCA). Despite the importance of this alignment, there is a notable absence of comprehensive studies investigating LLMs’ ability to evaluate reading comprehension difficulty across different student age groups, especially in the context of Chinese language education. To fill this gap, we introduce ZPD-SCA, a novel benchmark specifically designed to assess stage-level Chinese reading comprehension difficulty. The benchmark is annotated by 60 Special Grade teachers, a group that represents the top 0.15% of all in-service teachers nationwide. Experimental results reveal that LLMs perform poorly in zero-shot learning scenarios, with Qwen-max and GLM even falling below the probability of random guessing. When provided with in-context examples, LLMs performance improves substantially, with some models achieving nearly double the accuracy of their zero-shot baselines. These results reveal that LLMs possess emerging abilities to assess reading difficulty, while also exposing limitations in their current training for educationally aligned judgment. Notably, even the best-performing models display systematic directional biases, suggesting difficulties in accurately aligning material difficulty with SCA. Furthermore, significant variations in model performance across different genres underscore the complexity of task. We envision that ZPD-SCA can provide a foundation for evaluating and improving LLMs in cognitively aligned educational applications.

Method: Automatic evaluation metrics were used to rank machine translation systems submitted to the WMT25 General Machine Translation Shared Task.

Result: Preliminary rankings were generated but may be biased toward systems using re-ranking techniques like Quality Estimation or Minimum Bayes Risk decoding.

Conclusion: These automatic evaluation results are preliminary and will be superseded by more reliable human evaluation for the official WMT25 rankings.

Abstract: We present the preliminary rankings of machine translation (MT) systems submitted to the WMT25 General Machine Translation Shared Task, as determined by automatic evaluation metrics. Because these rankings are derived from automatic evaluation, they may exhibit a bias toward systems that employ re-ranking techniques, such as Quality Estimation or Minimum Bayes Risk decoding. The official WMT25 ranking will be based on human evaluation, which is more reliable and will supersede these results. The official WMT25 ranking will be based on human evaluation, which is more reliable and will supersede these results. The purpose of releasing these findings now is to assist task participants with their system description papers; not to provide final findings.

Method: Created a high-quality dataset of 102,646 QA pairs from medical case summaries of 1,011 cancer patients through manual annotation by 7 physicians (2,100 work hours). Evaluated 4 LLMs in zero-shot and supervised fine-tuning scenarios.

Result: Fine-tuned models significantly outperformed zero-shot counterparts, demonstrating that pretrained models fail to generalize on MedQARo without domain-specific and language-specific adaptation.

Conclusion: Domain-specific and language-specific fine-tuning is crucial for reliable clinical QA in Romanian. The dataset and code are publicly released to support further research.

Abstract: Question answering (QA) is an actively studied topic, being a core natural language processing (NLP) task that needs to be addressed before achieving Artificial General Intelligence (AGI). However, the lack of QA datasets in specific domains and languages hinders the development of robust AI models able to generalize across various domains and languages. To this end, we introduce MedQARo, the first large-scale medical QA benchmark in Romanian, alongside a comprehensive evaluation of state-of-the-art large language models (LLMs). We construct a high-quality and large-scale dataset comprising 102,646 QA pairs related to cancer patients. The questions regard medical case summaries of 1,011 patients, requiring either keyword extraction or reasoning to be answered correctly. MedQARo is the result of a time-consuming manual annotation process carried out by seven physicians specialized in oncology or radiotherapy, who spent a total of about 2,100 work hours to generate the QA pairs. We experiment with four LLMs from distinct families of models on MedQARo. Each model is employed in two scenarios, namely one based on zero-shot prompting and one based on supervised fine-tuning. Our results show that fine-tuned models significantly outperform their zero-shot counterparts, clearly indicating that pretrained models fail to generalize on MedQARo. Our findings demonstrate the importance of both domain-specific and language-specific fine-tuning for reliable clinical QA in Romanian. We publicly release our dataset and code at https://github.com/ana-rogoz/MedQARo.

Method: Reprojects narrow-FOV iToF depth onto RGB coordinate system using geometric calibration, then uses dual-encoder fusion network with monocular depth priors to extract complementary features and perform depth super-resolution.

Result: Significantly outperforms state-of-the-art methods in accuracy, structural consistency, and visual quality on both synthetic and real-world datasets.

Conclusion: The framework successfully integrates cross-modal cues to achieve enhanced depth accuracy, improved edge sharpness, and seamless field-of-view expansion.

Abstract: This paper presents a novel iToF-RGB fusion framework designed to address the inherent limitations of indirect Time-of-Flight (iToF) depth sensing, such as low spatial resolution, limited field-of-view (FoV), and structural distortion in complex scenes. The proposed method first reprojects the narrow-FoV iToF depth map onto the wide-FoV RGB coordinate system through a precise geometric calibration and alignment module, ensuring pixel-level correspondence between modalities. A dual-encoder fusion network is then employed to jointly extract complementary features from the reprojected iToF depth and RGB image, guided by monocular depth priors to recover fine-grained structural details and perform depth super-resolution. By integrating cross-modal structural cues and depth consistency constraints, our approach achieves enhanced depth accuracy, improved edge sharpness, and seamless FoV expansion. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed framework significantly outperforms state-of-the-art methods in terms of accuracy, structural consistency, and visual quality.

Method: Uses iterative structured feedback alternating between image generation and multimodal agent evaluation. Includes language-guided planner/critic for assessing counts, spatial arrangements, and attributes, plus instance-driven attention masking and compositional generation for object separation.

Result: Achieves up to 98% counting accuracy on COCO Count, T2I CompBench, and new high-instance benchmarks while maintaining spatial fidelity and visual quality, outperforming layout-based and gradient-guided baselines with a score of 0.97.

Conclusion: CountLoop effectively addresses the instance counting problem in diffusion models without requiring retraining, providing precise control over object instances through iterative feedback mechanisms.

Abstract: Diffusion models have shown remarkable progress in photorealistic image synthesis, yet they remain unreliable for generating scenes with a precise number of object instances, particularly in complex and high-density settings. We present CountLoop, a training-free framework that provides diffusion models with accurate instance control through iterative structured feedback. The approach alternates between image generation and multimodal agent evaluation, where a language-guided planner and critic assess object counts, spatial arrangements, and attribute consistency. This feedback is then used to refine layouts and guide subsequent generations. To further improve separation between objects, especially in occluded scenes, we introduce instance-driven attention masking and compositional generation techniques. Experiments on COCO Count, T2I CompBench, and two new high-instance benchmarks show that CountLoop achieves counting accuracy of up to 98% while maintaining spatial fidelity and visual quality, outperforming layout-based and gradient-guided baselines with a score of 0.97.

Method: Used mechanistic interpretability techniques to analyze CLIP’s vision encoder, specifically examining how individual neurons in MLP layers represent multiple features through superposition.

Result: Found evidence that superposition in MLP neurons directly hinders compositional feature representation, which consequently affects compositional reasoning and object binding capabilities.

Conclusion: This study provides initial insights into the mechanistic roots of compositional failures in VLMs, serving as a foundation for future improvements in compositional reasoning capabilities.

Abstract: Vision-Language Models (VLMs) have shown remarkable performance in integrating visual and textual information for tasks such as image captioning and visual question answering. However, these models struggle with compositional generalization and object binding, which limit their ability to handle novel combinations of objects and their attributes. Our work explores the root causes of these failures using mechanistic interpretability techniques. We show evidence that individual neurons in the MLP layers of CLIP’s vision encoder represent multiple features, and this “superposition” directly hinders its compositional feature representation which consequently affects compositional reasoning and object binding capabilities. We hope this study will serve as an initial step toward uncovering the mechanistic roots of compositional failures in VLMs. The code and supporting results can be found https://github.com/Mystic-Slice/Do-VLMs-Have-Bad-Eyes .

Method: MSNav integrates three modules: Memory Module for dynamic map memory with selective pruning, Spatial Module for spatial reasoning and object relationship inference, and Decision Module for LLM-based path planning. Also introduces I-O-S dataset and fine-tunes Qwen3-4B into Qwen-Spatial model.

Result: Achieves superior object list extraction performance (higher F1 and NDCG scores) and state-of-the-art results on R2R and REVERIE datasets with significant improvements in Success Rate (SR) and Success weighted by Path Length (SPL).

Conclusion: MSNav transforms fragile VLN inference into robust, integrated intelligence by systematically addressing core vulnerabilities through modular architecture and specialized spatial reasoning capabilities.

Abstract: Vision-and-Language Navigation (VLN) requires an agent to interpret natural language instructions and navigate complex environments. Current approaches often adopt a “black-box” paradigm, where a single Large Language Model (LLM) makes end-to-end decisions. However, it is plagued by critical vulnerabilities, including poor spatial reasoning, weak cross-modal grounding, and memory overload in long-horizon tasks. To systematically address these issues, we propose Memory Spatial Navigation(MSNav), a framework that fuses three modules into a synergistic architecture, which transforms fragile inference into a robust, integrated intelligence. MSNav integrates three modules: Memory Module, a dynamic map memory module that tackles memory overload through selective node pruning, enhancing long-range exploration; Spatial Module, a module for spatial reasoning and object relationship inference that improves endpoint recognition; and Decision Module, a module using LLM-based path planning to execute robust actions. Powering Spatial Module, we also introduce an Instruction-Object-Space (I-O-S) dataset and fine-tune the Qwen3-4B model into Qwen-Spatial (Qwen-Sp), which outperforms leading commercial LLMs in object list extraction, achieving higher F1 and NDCG scores on the I-O-S test set. Extensive experiments on the Room-to-Room (R2R) and REVERIE datasets demonstrate MSNav’s state-of-the-art performance with significant improvements in Success Rate (SR) and Success weighted by Path Length (SPL).

Method: Used Convolutional Neural Networks for soil image classification, enhanced with machine learning algorithms. Employed Whale Optimization Algorithm and Particle Swarm Optimization to select optimal hyperparameters for the CNN network, comparing both swarm algorithms’ performance.

Result: The proposed system achieved efficient results in multiple soil type classification, evaluated using Accuracy and F1 score metrics. The swarm optimization algorithms successfully improved CNN performance.

Conclusion: The integration of swarm optimization algorithms with CNNs provides an effective approach for intelligent soil classification, offering valuable results for practical applications in various disciplines that rely on soil quality assessment.

Abstract: Classifying soil images contributes to better land management, increased agricultural output, and practical solutions for environmental issues. The development of various disciplines, particularly agriculture, civil engineering, and natural resource management, is aided by understanding of soil quality since it helps with risk reduction, performance improvement, and sound decision-making . Artificial intelligence has recently been used in a number of different fields. In this study, an intelligent model was constructed using Convolutional Neural Networks to classify soil kinds, and machine learning algorithms were used to enhance the performance of soil classification . To achieve better implementation and performance of the Convolutional Neural Networks algorithm and obtain valuable results for the process of classifying soil type images, swarm algorithms were employed to obtain the best performance by choosing Hyper parameters for the Convolutional Neural Networks network using the Whale optimization algorithm and the Particle swarm optimization algorithm, and comparing the results of using the two algorithms in the process of multiple classification of soil types. The Accuracy and F1 measures were adopted to test the system, and the results of the proposed work were efficient result

Method: Developed QA-VLM framework that leverages vision-language models’ attention mechanisms and reasoning capabilities, enriched with application-specific knowledge from peer-reviewed journal articles.

Result: Evaluated on 24 single-bead samples from laser wire direct energy deposition, the framework showed higher validity and consistency in explanation quality compared to off-the-shelf VLMs.

Conclusion: The approach enables trustworthy, interpretable quality assessment in additive manufacturing applications by providing human-understandable justifications.

Abstract: Image-based quality assessment (QA) in additive manufacturing (AM) often relies heavily on the expertise and constant attention of skilled human operators. While machine learning and deep learning methods have been introduced to assist in this task, they typically provide black-box outputs without interpretable justifications, limiting their trust and adoption in real-world settings. In this work, we introduce a novel QA-VLM framework that leverages the attention mechanisms and reasoning capabilities of vision-language models (VLMs), enriched with application-specific knowledge distilled from peer-reviewed journal articles, to generate human-interpretable quality assessments. Evaluated on 24 single-bead samples produced by laser wire direct energy deposition (DED-LW), our framework demonstrates higher validity and consistency in explanation quality than off-the-shelf VLMs. These results highlight the potential of our approach to enable trustworthy, interpretable quality assessment in AM applications.

Method: Proposes Probabilistic Temporal Masked Attention (PTMA) model with GRU-based temporal masked attention cell that leverages probabilistic modeling for latent compressed representations and integrates multi-view information for view-invariant features.

Result: PTMA achieves state-of-the-art performance on DAHLIA, IKEA ASM, and Breakfast datasets under cross-subject, cross-view, and cross-subject-view evaluation protocols.

Conclusion: The PTMA model effectively handles viewpoint variations in online action detection through probabilistic modeling and temporal attention mechanisms, demonstrating superior generalization across different evaluation scenarios.

Abstract: As a critical task in video sequence classification within computer vision, Online Action Detection (OAD) has garnered significant attention. The sensitivity of mainstream OAD models to varying video viewpoints often hampers their generalization when confronted with unseen sources. To address this limitation, we propose a novel Probabilistic Temporal Masked Attention (PTMA) model, which leverages probabilistic modeling to derive latent compressed representations of video frames in a cross-view setting. The PTMA model incorporates a GRU-based temporal masked attention (TMA) cell, which leverages these representations to effectively query the input video sequence, thereby enhancing information interaction and facilitating autoregressive frame-level video analysis. Additionally, multi-view information can be integrated into the probabilistic modeling to facilitate the extraction of view-invariant features. Experiments conducted under three evaluation protocols: cross-subject (cs), cross-view (cv), and cross-subject-view (csv) show that PTMA achieves state-of-the-art performance on the DAHLIA, IKEA ASM, and Breakfast datasets.

Method: A novel plug-and-play attention module inserted into pre-trained backbones like Swin Transformer, trained end-to-end with composite loss function to focus on discriminative object parts without explicit annotations.

Result: Improved Swin-Base model accuracy from 85.40% to 88.06% on CUB-200-2011 dataset (2.66% gain), with attention maps effectively localizing semantically meaningful features.

Conclusion: The Loupe demonstrates that simple intrinsic attention mechanisms can serve as powerful regularizers, significantly boosting performance while providing clear visual explanations for trustworthy decision-making in fine-grained classification tasks.

Abstract: Fine-Grained Visual Classification (FGVC) is a critical and challenging area within computer vision, demanding the identification of highly subtle, localized visual cues. The importance of FGVC extends to critical applications such as biodiversity monitoring and medical diagnostics, where precision is paramount. While large-scale Vision Transformers have achieved state-of-the-art performance, their decision-making processes often lack the interpretability required for trust and verification in such domains. In this paper, we introduce The Loupe, a novel, lightweight, and plug-and-play attention module designed to be inserted into pre-trained backbones like the Swin Transformer. The Loupe is trained end-to-end with a composite loss function that implicitly guides the model to focus on the most discriminative object parts without requiring explicit part-level annotations. Our unique contribution lies in demonstrating that a simple, intrinsic attention mechanism can act as a powerful regularizer, significantly boosting performance while simultaneously providing clear visual explanations. Our experimental evaluation on the challenging CUB-200-2011 dataset shows that The Loupe improves the accuracy of a Swin-Base model from 85.40% to 88.06%, a significant gain of 2.66%. Crucially, our qualitative analysis of the learned attention maps reveals that The Loupe effectively localizes semantically meaningful features, providing a valuable tool for understanding and trusting the model’s decision-making process.

Method: Convolutional Neural Network model trained on lung CT scans to analyze COVID-19 infection severity, predicting outcomes (promising vs unfavorable leading to intubation or death).

Result: Model aims to provide accurate severity assessment from CT scans to help doctors make critical treatment decisions and resource allocation.

Conclusion: Machine learning approach using CNN can potentially reduce human error and improve COVID-19 severity prediction from CT scans, addressing critical healthcare challenges during pandemics.

Abstract: COVID19 took the world by storm since December 2019. A highly infectious communicable disease, COVID19 is caused by the SARSCoV2 virus. By March 2020, the World Health Organization (WHO) declared COVID19 as a global pandemic. A pandemic in the 21st century after almost 100 years was something the world was not prepared for, which resulted in the deaths of around 1.6 million people worldwide. The most common symptoms of COVID19 were associated with the respiratory system and resembled a cold, flu, or pneumonia. After extensive research, doctors and scientists concluded that the main reason for lives being lost due to COVID19 was failure of the respiratory system. Patients were dying gasping for breath. Top healthcare systems of the world were failing badly as there was an acute shortage of hospital beds, oxygen cylinders, and ventilators. Many were dying without receiving any treatment at all. The aim of this project is to help doctors decide the severity of COVID19 by reading the patient’s Computed Tomography (CT) scans of the lungs. Computer models are less prone to human error, and Machine Learning or Neural Network models tend to give better accuracy as training improves over time. We have decided to use a Convolutional Neural Network model. Given that a patient tests positive, our model will analyze the severity of COVID19 infection within one month of the positive test result. The severity of the infection may be promising or unfavorable (if it leads to intubation or death), based entirely on the CT scans in the dataset.

Method: Novel method consisting of object trajectory detection module and interaction reasoning module for spatial-temporal HOI detection.

Result: Outperforms baselines from state-of-the-art image HOI detection, video visual relation detection, and video HOI recognition methods.

Conclusion: The proposed ST-HOID method effectively addresses video human-object interaction detection with trajectory tracking, demonstrating superior performance on the newly created VidOR-HOID dataset.

Abstract: In this paper, we propose a new instance-level human-object interaction detection task on videos called ST-HOID, which aims to distinguish fine-grained human-object interactions (HOIs) and the trajectories of subjects and objects. It is motivated by the fact that HOI is crucial for human-centric video content understanding. To solve ST-HOID, we propose a novel method consisting of an object trajectory detection module and an interaction reasoning module. Furthermore, we construct the first dataset named VidOR-HOID for ST-HOID evaluation, which contains 10,831 spatial-temporal HOI instances. We conduct extensive experiments to evaluate the effectiveness of our method. The experimental results demonstrate that our method outperforms the baselines generated by the state-of-the-art methods of image human-object interaction detection, video visual relation detection and video human-object interaction recognition.

Method: Built a comprehensive benchmark from 1,900 de-identified real-world Chinese medical reports spanning diverse departments and formats. Includes text-only evaluation using OCR+LLM for comparison. Uses objective field-level recall metrics and automated subjective evaluation with LLM scoring. Applied Group Relative Policy Optimization to improve VLM performance.

Result: The OCR+LLM pipeline shows strong performance but suffers from layout-blindness and latency issues. GRPO optimization achieved up to 6% recall gain for mid-scale VLMs.

Conclusion: The benchmark enables controlled comparisons and reveals limitations of OCR-based approaches, motivating further progress toward robust, fully vision-based medical report understanding systems.

Abstract: Medical report interpretation plays a crucial role in healthcare, enabling both patient-facing explanations and effective information flow across clinical systems. While recent vision-language models (VLMs) and large language models (LLMs) have demonstrated general document understanding capabilities, there remains a lack of standardized benchmarks to assess structured interpretation quality in medical reports. We introduce MedRepBench, a comprehensive benchmark built from 1,900 de-identified real-world Chinese medical reports spanning diverse departments, patient demographics, and acquisition formats. The benchmark is designed primarily to evaluate end-to-end VLMs for structured medical report understanding. To enable controlled comparisons, we also include a text-only evaluation setting using high-quality OCR outputs combined with LLMs, allowing us to estimate the upper-bound performance when character recognition errors are minimized. Our evaluation framework supports two complementary protocols: (1) an objective evaluation measuring field-level recall of structured clinical items, and (2) an automated subjective evaluation using a powerful LLM as a scoring agent to assess factuality, interpretability, and reasoning quality. Based on the objective metric, we further design a reward function and apply Group Relative Policy Optimization (GRPO) to improve a mid-scale VLM, achieving up to 6% recall gain. We also observe that the OCR+LLM pipeline, despite strong performance, suffers from layout-blindness and latency issues, motivating further progress toward robust, fully vision-based report understanding.

Method: Proposes modality-aware network with channel aggregation/distribution module and spatial similarity perception module, transformer fusion for global dependencies, trident prediction head, and dynamic update strategy.

Result: Achieves satisfactory results compared to state-of-the-art competitors on three RGBT benchmarks while maintaining real-time speed.

Conclusion: The proposed MTNet framework effectively addresses modality interaction challenges and template management issues in RGBT tracking through its novel architecture components.

Abstract: The ability to learn robust multi-modality representation has played a critical role in the development of RGBT tracking. However, the regular fusion paradigm and the invariable tracking template remain restrictive to the feature interaction. In this paper, we propose a modality-aware tracker based on transformer, termed MTNet. Specifically, a modality-aware network is presented to explore modality-specific cues, which contains both channel aggregation and distribution module(CADM) and spatial similarity perception module (SSPM). A transformer fusion network is then applied to capture global dependencies to reinforce instance representations. To estimate the precise location and tackle the challenges, such as scale variation and deformation, we design a trident prediction head and a dynamic update strategy which jointly maintain a reliable template for facilitating inter-frame communication. Extensive experiments validate that the proposed method achieves satisfactory results compared with the state-of-the-art competitors on three RGBT benchmarks while reaching real-time speed.

Method: Developed a modular framework with differentiated quantization for VLMs and a multi-agent system combining scene classification, vectorized memory, and multimodal interaction through perception-memory-reasoning workflows.

Result: Quantized 19B-parameter model showed only 2.05% performance drop on MMBench and maintained 63.7 accuracy on OCR-VQA, with response latency of 2.83-3.52 seconds, outperforming smaller models with equivalent memory.

Conclusion: The research successfully advances computational efficiency and assistive technology, providing visually impaired users with comprehensive real-time assistance while significantly reducing memory requirements and maintaining performance.

Abstract: This study proposes the dual technological innovation framework, including a cross-modal differ entiated quantization framework for vision-language models (VLMs) and a scene-aware vectorized memory multi-agent system for visually impaired assistance. The modular framework was developed implementing differentiated processing strategies, effectively reducing memory requirements from 38GB to 16GB while maintaining model performance. The multi-agent architecture combines scene classification, vectorized memory, and multimodal interaction, enabling persistent storage and efficient retrieval of scene memories. Through perception-memory-reasoning workflows, the system provides environmental information beyond the current view using historical memories. Experiments show the quantized 19B-parameter model only experiences a 2.05% performance drop on MMBench and maintains 63.7 accuracy on OCR-VQA (original: 64.9), outperforming smaller models with equivalent memory requirements like the Molmo-7B series. The system maintains response latency between 2.83-3.52 seconds from scene analysis to initial speech output, substantially faster than non-streaming methods. This research advances computational efficiency and assistive technology, offering visually impaired users comprehensive real-time assistance in scene perception, text recognition, and navigation.

Method: Two-stage approach: Stage 1 uses policy network with frame compression to discard redundant clips and station point mechanism for accuracy. Stage 2 employs improved YOLOv8 model for fire source localization when fire is detected.

Result: Significantly reduces computational costs while maintaining classification accuracy in Stage 1, and achieves higher detection accuracy with similar inference time in Stage 2 compared to baselines.

Conclusion: Proposed framework enables efficient real-time wildfire monitoring on resource-constrained UAV platforms through intelligent frame selection and optimized detection models.

Abstract: Unmanned Aerial Vehicles (UAVs) have become increasingly important in disaster emergency response by enabling real-time aerial video analysis. Due to the limited computational resources available on UAVs, large models cannot be run independently for real-time analysis. To overcome this challenge, we propose a lightweight and efficient two-stage framework for real-time wildfire monitoring and fire source detection on UAV platforms. Specifically, in Stage 1, we utilize a policy network to identify and discard redundant video clips using frame compression techniques, thereby reducing computational costs. In addition, we introduce a station point mechanism that leverages future frame information within the sequential policy network to improve prediction accuracy. In Stage 2, once the frame is classified as “fire”, we employ the improved YOLOv8 model to localize the fire source. We evaluate the Stage 1 method using the FLAME and HMDB51 datasets, and the Stage 2 method using the Fire & Smoke dataset. Experimental results show that our method significantly reduces computational costs while maintaining classification accuracy in Stage 1, and achieves higher detection accuracy with similar inference time in Stage 2 compared to baseline methods.

Method: CellEcoNet models whole slide images through natural language analogy, defining cells as words, cellular neighborhoods as phrases, and tissue architecture as sentences, automatically learning context-dependent meanings and spatial interactions to predict recurrence risk.

Result: On 456 H&E-stained WSIs, CellEcoNet achieved AUC:77.8% and HR:9.54, outperforming IASLC grading (AUC:71.4%), AJCC Stage (AUC:64.0%), and state-of-the-art computational methods (AUCs:62.2-67.4%), with consistent performance across demographic subgroups.

Conclusion: CellEcoNet represents a paradigm shift by decoding the tumor microenvironment’s cellular language to reveal how subtle cell variations encode recurrence risk, providing superior prognostic capabilities for lung adenocarcinoma patients.

Abstract: Despite surgical resection, ~70% of invasive lung adenocarcinoma (ILA) patients recur within five years, and current tools fail to identify those needing adjuvant therapy. To address this unmet clinical need, we introduce CellEcoNet, a novel spatially aware deep learning framework that models whole slide images (WSIs) through natural language analogy, defining a “language of pathology,” where cells act as words, cellular neighborhoods become phrases, and tissue architecture forms sentences. CellEcoNet learns these context-dependent meanings automatically, capturing how subtle variations and spatial interactions derive recurrence risk. On a dataset of 456 H&E-stained WSIs, CellEcoNet achieved superior predictive performance (AUC:77.8% HR:9.54), outperforming IASLC grading system (AUC:71.4% HR:2.36), AJCC Stage (AUC:64.0% HR:1.17) and state-of-the-art computational methods (AUCs:62.2-67.4%). CellEcoNet demonstrated fairness and consistent performance across diverse demographic and clinical subgroups. Beyond prognosis, CellEcoNet marks a paradigm shift by decoding the tumor microenvironment’s cellular “language” to reveal how subtle cell variations encode recurrence risk.

Method: Uses Pareto-optimal frontiers across hyperparameterization of debiasing methods, with Normalized Shannon Entropy for fairness evaluation and ClipScore for utility evaluation.

Result: Evaluation of Stable Diffusion, Fair Diffusion, SDXL, DeCoDi, and FLUX models shows most default hyperparameterizations are dominated solutions in fairness-utility space, and better hyperparameters can be easily identified.

Conclusion: The proposed method enables reproducible comparison between text-to-image models and provides a systematic approach to optimize both fairness and utility through better hyperparameter selection.

Abstract: Achieving fairness in text-to-image generation demands mitigating social biases without compromising visual fidelity, a challenge critical to responsible AI. Current fairness evaluation procedures for text-to-image models rely on qualitative judgment or narrow comparisons, which limit the capacity to assess both fairness and utility in these models and prevent reproducible assessment of debiasing methods. Existing approaches typically employ ad-hoc, human-centered visual inspections that are both error-prone and difficult to replicate. We propose a method for evaluating fairness and utility in text-to-image models using Pareto-optimal frontiers across hyperparametrization of debiasing methods. Our method allows for comparison between distinct text-to-image models, outlining all configurations that optimize fairness for a given utility and vice-versa. To illustrate our evaluation method, we use Normalized Shannon Entropy and ClipScore for fairness and utility evaluation, respectively. We assess fairness and utility in Stable Diffusion, Fair Diffusion, SDXL, DeCoDi, and FLUX text-to-image models. Our method shows that most default hyperparameterizations of the text-to-image model are dominated solutions in the fairness-utility space, and it is straightforward to find better hyperparameters.

Method: A benchmark of 756 ten-frame sequences captured with 14 smartphone sensors across various illumination and exposure conditions, with high-SNR references obtained through burst averaging. Participants process linear RAW sequences and output denoised frames while preserving Bayer pattern.

Result: Submissions are evaluated on a private test set using full-reference PSNR and SSIM metrics, with final ranking determined by the mean of per-metric ranks.

Conclusion: The paper introduces a comprehensive benchmark and challenge protocol for low-light RAW video denoising, facilitating the development of advanced methods that can handle real-world smartphone camera constraints and sensor-specific noise characteristics.

Abstract: This paper reviews the AIM 2025 (Advances in Image Manipulation) Low-Light RAW Video Denoising Challenge. The task is to develop methods that denoise low-light RAW video by exploiting temporal redundancy while operating under exposure-time limits imposed by frame rate and adapting to sensor-specific, signal-dependent noise. We introduce a new benchmark of 756 ten-frame sequences captured with 14 smartphone camera sensors across nine conditions (illumination: 1/5/10 lx; exposure: 1/24, 1/60, 1/120 s), with high-SNR references obtained via burst averaging. Participants process linear RAW sequences and output the denoised 10th frame while preserving the Bayer pattern. Submissions are evaluated on a private test set using full-reference PSNR and SSIM, with final ranking given by the mean of per-metric ranks. This report describes the dataset, challenge protocol, and submitted approaches.

Method: Developed WebMMU benchmark with expert-annotated real-world web data covering three tasks: website visual question answering, HTML/CSS/JavaScript code editing, and mockup-to-code generation to evaluate multimodal reasoning and coding skills.

Result: MLLMs perform well on basic information extraction but struggle with reasoning and grounding, functional code editing, and generating design-to-code that maintains hierarchy and supports multilingual content.

Conclusion: Current MLLMs have significant limitations in multimodal and cross-lingual reasoning, highlighting the need for improved capabilities to build future web agents capable of automating diverse web development tasks.

Abstract: We present WebMMU, a multilingual benchmark that evaluates three core web tasks: (1) website visual question answering, (2) code editing involving HTML/CSS/JavaScript, and (3) mockup-to-code generation. Unlike prior benchmarks that treat these tasks separately, WebMMU unifies them using expert-annotated, real-world web data to assess models’ abilities in complex multi-step reasoning, precise element grounding, and functional UI comprehension and coding. Our evaluation shows that while multimodal large language models (MLLMs) perform well on basic information extraction, they struggle with reasoning and grounding, editing code to preserve functionality, and generating design-to-code that maintains hierarchy and supports multilingual content. These findings reveal key limitations in current MLLMs and underscore the need for improved multimodal and cross-lingual reasoning to build future web agents capable of automating diverse web development tasks.

Method: Proposes a unified token representation for all modalities and tasks, with neural tuning strategy that activates specific neuron subsets for each task, inspired by sparse distributed brain representations.

Result: The framework demonstrates efficient simultaneous handling of multiple tasks (reasoning segmentation, referring segmentation, image captioning, text-to-image generation) on the MMUD benchmark.

Conclusion: Neural tuning enables scalable and versatile multimodal multitask learning, with public release of models, code, and datasets to advance research in this field.

Abstract: Large-scale models have exhibited remarkable capabilities across diverse domains, including automated medical services and intelligent customer support. However, as most large models are trained on single-modality corpora, enabling them to effectively process and understand multimodal signals remains a significant challenge. Current research often focuses on designing task-specific or scenario-specific tuning strategies, which limits the scalability and versatility. To address this limitation, we propose a unified framework that concurrently handles multiple tasks and modalities. In this framework, all modalities and tasks are represented as unified tokens and trained using a single, consistent approach. To enable efficient multitask processing, we introduce a novel tuning strategy termed neural tuning, inspired by the concept of sparse distributed representation in the human brain, where only specific subsets of neurons are activated for each task. Furthermore, to advance research in multimodal and multitask learning, we present a new benchmark, MMUD, which includes samples annotated with multiple task labels spanning reasoning segmentation, referring segmentation, image captioning, and text-to-image generation. By applying neural tuning to pretrained large models on the MMUD benchmark, we demonstrate the ability to handle multiple tasks simultaneously in a streamlined and efficient manner. All models, code, and datasets will be released publicly upon publication, fostering further research and innovation in this field.

Method: Uses Gaussian Primitive Optimization with descriptor-based control nodes at key anatomical structures. Each node has trainable position, displacement, and radius. KNN Gaussian interpolation propagates displacements from information-rich nodes to create a globally coherent displacement field.

Result: Reduces target registration error from 6.2px to ~2.4px and increases AUC at 25px from 0.770 to 0.938 on FIRE dataset, substantially outperforming existing methods.

Conclusion: GPO effectively addresses vanishing gradient problems in retinal image registration by strategically anchoring nodes in high-gradient regions and using structured message passing, achieving superior registration accuracy.

Abstract: Deformable retinal image registration is notoriously difficult due to large homogeneous regions and sparse but critical vascular features, which cause limited gradient signals in standard learning-based frameworks. In this paper, we introduce Gaussian Primitive Optimization (GPO), a novel iterative framework that performs structured message passing to overcome these challenges. After an initial coarse alignment, we extract keypoints at salient anatomical structures (e.g., major vessels) to serve as a minimal set of descriptor-based control nodes (DCN). Each node is modelled as a Gaussian primitive with trainable position, displacement, and radius, thus adapting its spatial influence to local deformation scales. A K-Nearest Neighbors (KNN) Gaussian interpolation then blends and propagates displacement signals from these information-rich nodes to construct a globally coherent displacement field; focusing interpolation on the top (K) neighbors reduces computational overhead while preserving local detail. By strategically anchoring nodes in high-gradient regions, GPO ensures robust gradient flow, mitigating vanishing gradient signal in textureless areas. The framework is optimized end-to-end via a multi-term loss that enforces both keypoint consistency and intensity alignment. Experiments on the FIRE dataset show that GPO reduces the target registration error from 6.2,px to ~2.4,px and increases the AUC at 25,px from 0.770 to 0.938, substantially outperforming existing methods. The source code can be accessed via https://github.com/xintian-99/GPOreg.

Method: Developed RoentGen-v2 diffusion model for chest radiographs with fine-grained control over findings and demographics. Created large synthetic dataset (565k+ images) and proposed synthetic pretraining strategy followed by real data fine-tuning.

Result: Synthetic pretraining improved downstream classification accuracy by 6.5% (vs 2.7% with naive combination), reduced underdiagnosis fairness gap by 19.3%, and enhanced generalization across 5 institutions (137k+ real images).

Conclusion: Synthetic imaging with demographic conditioning can advance equitable and generalizable medical AI, with synthetic pretraining being more effective than naive data combination for improving performance and fairness.

Abstract: Achieving robust performance and fairness across diverse patient populations remains a challenge in developing clinically deployable deep learning models for diagnostic imaging. Synthetic data generation has emerged as a promising strategy to address limitations in dataset scale and diversity. We introduce RoentGen-v2, a text-to-image diffusion model for chest radiographs that enables fine-grained control over both radiographic findings and patient demographic attributes, including sex, age, and race/ethnicity. RoentGen-v2 is the first model to generate clinically plausible images with demographic conditioning, facilitating the creation of a large, demographically balanced synthetic dataset comprising over 565,000 images. We use this large synthetic dataset to evaluate optimal training pipelines for downstream disease classification models. In contrast to prior work that combines real and synthetic data naively, we propose an improved training strategy that leverages synthetic data for supervised pretraining, followed by fine-tuning on real data. Through extensive evaluation on over 137,000 chest radiographs from five institutions, we demonstrate that synthetic pretraining consistently improves model performance, generalization to out-of-distribution settings, and fairness across demographic subgroups. Across datasets, synthetic pretraining led to a 6.5% accuracy increase in the performance of downstream classification models, compared to a modest 2.7% increase when naively combining real and synthetic data. We observe this performance improvement simultaneously with the reduction of the underdiagnosis fairness gap by 19.3%. These results highlight the potential of synthetic imaging to advance equitable and generalizable medical deep learning under real-world data constraints. We open source our code, trained models, and synthetic dataset at https://github.com/StanfordMIMI/RoentGen-v2 .

Method: MDIQA framework models image quality across 5 technical and 4 aesthetic dimensions in separate branches, each trained on specific dimensions, then amalgamates features for final IQA score. Also enables flexible training of image restoration models by adjusting perceptual dimension weights.

Result: Extensive experiments show MDIQA achieves superior performance and can be effectively and flexibly applied to image restoration tasks.

Conclusion: The multi-dimensional approach better captures human visual perception and provides a flexible framework for quality assessment and image restoration aligned with varying user preferences.

Abstract: Recent advancements in image quality assessment (IQA), driven by sophisticated deep neural network designs, have significantly improved the ability to approach human perceptions. However, most existing methods are obsessed with fitting the overall score, neglecting the fact that humans typically evaluate image quality from different dimensions before arriving at an overall quality assessment. To overcome this problem, we propose a multi-dimensional image quality assessment (MDIQA) framework. Specifically, we model image quality across various perceptual dimensions, including five technical and four aesthetic dimensions, to capture the multifaceted nature of human visual perception within distinct branches. Each branch of our MDIQA is initially trained under the guidance of a separate dimension, and the respective features are then amalgamated to generate the final IQA score. Additionally, when the MDIQA model is ready, we can deploy it for a flexible training of image restoration (IR) models, enabling the restoration results to better align with varying user preferences through the adjustment of perceptual dimension weights. Extensive experiments demonstrate that our MDIQA achieves superior performance and can be effectively and flexibly applied to image restoration tasks. The code is available: https://github.com/YaoShunyu19/MDIQA.

Method: Uses foundation model feature concatenation, prompt tuning strategies, perspective-specified prompts, and horizontal flip augmentation to jointly detect objects and attributes.

Result: Outperforms state-of-the-art methods on nuScenes and Argoverse 2 datasets in open-vocabulary 3D object detection while successfully recognizing object attributes.

Conclusion: OVODA enables effective open-vocabulary 3D object and attribute detection without requiring novel class anchor sizes, supported by the new OVAD dataset with comprehensive attribute annotations.

Abstract: 3D object detection plays a crucial role in autonomous systems, yet existing methods are limited by closed-set assumptions and struggle to recognize novel objects and their attributes in real-world scenarios. We propose OVODA, a novel framework enabling both open-vocabulary 3D object and attribute detection with no need to know the novel class anchor size. OVODA uses foundation models to bridge the semantic gap between 3D features and texts while jointly detecting attributes, e.g., spatial relationships, motion states, etc. To facilitate such research direction, we propose OVAD, a new dataset that supplements existing 3D object detection benchmarks with comprehensive attribute annotations. OVODA incorporates several key innovations, including foundation model feature concatenation, prompt tuning strategies, and specialized techniques for attribute detection, including perspective-specified prompts and horizontal flip augmentation. Our results on both the nuScenes and Argoverse 2 datasets show that under the condition of no given anchor sizes of novel classes, OVODA outperforms the state-of-the-art methods in open-vocabulary 3D object detection while successfully recognizing object attributes. Our OVAD dataset is released here: https://doi.org/10.5281/zenodo.16904069 .

Method: Distributed pixel-level algorithm where each pixel estimates global camera motion by exchanging information with other pixels. Uses probabilistic formulation and Gaussian Belief Propagation (GBP) with message-passing for decentralized inference to achieve global consensus.

Result: Evaluated on real-world public datasets with in-depth analysis of GBP practicality for distributed rotation estimation at pixel level.

Conclusion: Demonstrates feasibility of performing complex visual processing directly in-pixel using distributed algorithms like GBP, enabling more efficient computer vision systems that reduce transmission and computational overhead.

Abstract: Images are the standard input for most computer vision algorithms. However, their processing often reduces to parallelizable operations applied locally and independently to individual pixels. Yet, many of these low-level raw pixel readings only provide redundant or noisy information for specific high-level tasks, leading to inefficiencies in both energy consumption during their transmission off-sensor and computational resources in their subsequent processing. As novel sensors featuring advanced in-pixel processing capabilities emerge, we envision a paradigm shift toward performing increasingly complex visual processing directly in-pixel, reducing computational overhead downstream. We advocate for synthesizing high-level cues at the pixel level, enabling their off-sensor transmission to directly support downstream tasks more effectively than raw pixel readings. This paper conceptualizes a novel photometric rotation estimation algorithm to be distributed at pixel level, where each pixel estimates the global motion of the camera by exchanging information with other pixels to achieve global consensus. We employ a probabilistic formulation and leverage Gaussian Belief Propagation (GBP) for decentralized inference using messaging-passing. The proposed proposed technique is evaluated on real-world public datasets and we offer a in-depth analysis of the practicality of applying GBP to distributed rotation estimation at pixel level.

Method: Uses large VLM (e.g., GPT-4o) contextualized with domain knowledge to generate task-specific chain-of-thought prompts, which guide smaller efficient VLM (e.g., Qwen2.5-VL-7B) for reasoning over short videos and complementary modalities.

Result: Demonstrates consistently strong performance across diverse traffic and environmental conditions, validated on three specialized datasets including multimodal pavement wetness dataset combining video with road weather sensor data.

Conclusion: The framework can be readily integrated with existing traffic camera systems and applied to high-risk locations to enhance situational awareness and deliver timely alerts in resource-constrained environments.

Abstract: This paper introduces a multi-agent framework for comprehensive highway scene understanding, designed around a mixture-of-experts strategy. In this framework, a large generic vision-language model (VLM), such as GPT-4o, is contextualized with domain knowledge to generates task-specific chain-of-thought (CoT) prompts. These fine-grained prompts are then used to guide a smaller, efficient VLM (e.g., Qwen2.5-VL-7B) in reasoning over short videos, along with complementary modalities as applicable. The framework simultaneously addresses multiple critical perception tasks, including weather classification, pavement wetness assessment, and traffic congestion detection, achieving robust multi-task reasoning while balancing accuracy and computational efficiency. To support empirical validation, we curated three specialized datasets aligned with these tasks. Notably, the pavement wetness dataset is multimodal, combining video streams with road weather sensor data, highlighting the benefits of multimodal reasoning. Experimental results demonstrate consistently strong performance across diverse traffic and environmental conditions. From a deployment perspective, the framework can be readily integrated with existing traffic camera systems and strategically applied to high-risk rural locations, such as sharp curves, flood-prone lowlands, or icy bridges. By continuously monitoring the targeted sites, the system enhances situational awareness and delivers timely alerts, even in resource-constrained environments.

Method: Transformer-based architecture designed for detailed pixel-by-pixel comparison of search and template images only, without requiring difference imaging.

Result: Achieved 97.4% classification accuracy on Dark Energy Survey data, with performance maintained even when input images are not centered on supernova candidates.

Conclusion: The transformer network effectively enhances both accuracy and efficiency for supernova detection in large-scale astronomical surveys by eliminating computational bottlenecks while maintaining high performance.

Abstract: We introduce a transformer-based neural network for the accurate classification of real and bogus transient detections in astronomical images. This network advances beyond the conventional convolutional neural network (CNN) methods, widely used in image processing tasks, by adopting an architecture better suited for detailed pixel-by-pixel comparison. The architecture enables efficient analysis of search and template images only, thus removing the necessity for computationally-expensive difference imaging, while maintaining high performance. Our primary evaluation was conducted using the autoScan dataset from the Dark Energy Survey (DES), where the network achieved a classification accuracy of 97.4% and diminishing performance utility for difference image as the size of the training set grew. Further experiments with DES data confirmed that the network can operate at a similar level even when the input images are not centered on the supernova candidate. These findings highlight the network’s effectiveness in enhancing both accuracy and efficiency of supernova detection in large-scale astronomical surveys.

Method: Integrates VGG19 and ResNet50 CNN models to leverage their complementary feature extraction capabilities for multi-scale and multi-level analysis of underwater images through a unified model.

Result: Achieves comprehensive and accurate image enhancement effects as quantitatively measured by PSNR, UCIQE, and UIQM quality assessment metrics.

Conclusion: Provides practical suggestions for model optimization, multi-model fusion, and hardware selection to improve practicality and stability of underwater visual enhancement systems for complex environments.

Abstract: This paper addresses the challenging problem of image enhancement in complex underwater scenes by proposing a solution based on deep learning. The proposed method skillfully integrates two deep convolutional neural network models, VGG19 and ResNet50, leveraging their powerful feature extraction capabilities to perform multi-scale and multi-level deep feature analysis of underwater images. By constructing a unified model, the complementary advantages of the two models are effectively integrated, achieving a more comprehensive and accurate image enhancement effect.To objectively evaluate the enhancement effect, this paper introduces image quality assessment metrics such as PSNR, UCIQE, and UIQM to quantitatively compare images before and after enhancement and deeply analyzes the performance of different models in different scenarios.Furthermore, to improve the practicality and stability of the underwater visual enhancement system, this paper also provides practical suggestions from aspects such as model optimization, multi-model fusion, and hardware selection, aiming to provide strong technical support for visual enhancement tasks in complex underwater environments.

Method: Replaces diffusion action decoder with Normalizing Flow (NF) that enables one-shot sampling through invertible transformation. Integrated into FLOWER VLA architecture and fine-tuned on LIBERO benchmark.

Result: NinA matches performance of diffusion-based counterpart under same training regime while achieving substantially faster inference.

Conclusion: NinA offers a promising path toward efficient, high-frequency VLA control without compromising performance, making it more practical for real-world applications.

Abstract: Recent advances in Vision-Language-Action (VLA) models have established a two-component architecture, where a pre-trained Vision-Language Model (VLM) encodes visual observations and task descriptions, and an action decoder maps these representations to continuous actions. Diffusion models have been widely adopted as action decoders due to their ability to model complex, multimodal action distributions. However, they require multiple iterative denoising steps at inference time or downstream techniques to speed up sampling, limiting their practicality in real-world settings where high-frequency control is crucial. In this work, we present NinA (Normalizing Flows in Action), a fast and expressive alter- native to diffusion-based decoders for VLAs. NinA replaces the diffusion action decoder with a Normalizing Flow (NF) that enables one-shot sampling through an invertible transformation, significantly reducing inference time. We integrate NinA into the FLOWER VLA architecture and fine-tune on the LIBERO benchmark. Our experiments show that NinA matches the performance of its diffusion-based counterpart under the same training regime, while achieving substantially faster inference. These results suggest that NinA offers a promising path toward efficient, high-frequency VLA control without compromising performance.

Method: Formulates TSG from causal perspective, uses textual causal intervention with do-calculus to address unobserved confounders, and performs visual counterfactual reasoning by constructing counterfactual scenarios focusing only on video features.

Result: Experiments on public datasets demonstrate the superiority of the proposed method over existing approaches.

Conclusion: The causal intervention and counterfactual reasoning framework effectively eliminates spurious correlations and enhances model robustness in temporal sentence grounding tasks.

Abstract: Temporal Sentence Grounding (TSG) aims to identify relevant moments in an untrimmed video that semantically correspond to a given textual query. Despite existing studies having made substantial progress, they often overlook the issue of spurious correlations between video and textual queries. These spurious correlations arise from two primary factors: (1) inherent biases in the textual data, such as frequent co-occurrences of specific verbs or phrases, and (2) the model’s tendency to overfit to salient or repetitive patterns in video content. Such biases mislead the model into associating textual cues with incorrect visual moments, resulting in unreliable predictions and poor generalization to out-of-distribution examples. To overcome these limitations, we propose a novel TSG framework, causal intervention and counterfactual reasoning that utilizes causal inference to eliminate spurious correlations and enhance the model’s robustness. Specifically, we first formulate the TSG task from a causal perspective with a structural causal model. Then, to address unobserved confounders reflecting textual biases toward specific verbs or phrases, a textual causal intervention is proposed, utilizing do-calculus to estimate the causal effects. Furthermore, visual counterfactual reasoning is performed by constructing a counterfactual scenario that focuses solely on video features, excluding the query and fused multi-modal features. This allows us to debias the model by isolating and removing the influence of the video from the overall effect. Experiments on public datasets demonstrate the superiority of the proposed method. The code is available at https://github.com/Tangkfan/CICR.

Method: Proposes RF-PGS framework with two stages: 1) Geometry training stage using Planar Gaussians as geometry primitives for dense, surface-aligned scene reconstruction, 2) RF training stage with fully-structured radio radiance and tailored multi-view loss to model radio propagation behavior.

Result: Significantly improves reconstruction accuracy compared to prior radiance field methods, reduces training costs, and enables efficient representation of wireless channels.

Conclusion: RF-PGS offers a practical solution for scalable 6G Spatial-CSI modeling by reconstructing high-fidelity radio propagation paths from only sparse path loss spectra.

Abstract: In the 6G era, the demand for higher system throughput and the implementation of emerging 6G technologies require large-scale antenna arrays and accurate spatial channel state information (Spatial-CSI). Traditional channel modeling approaches, such as empirical models, ray tracing, and measurement-based methods, face challenges in spatial resolution, efficiency, and scalability. Radiance field-based methods have emerged as promising alternatives but still suffer from geometric inaccuracy and costly supervision. This paper proposes RF-PGS, a novel framework that reconstructs high-fidelity radio propagation paths from only sparse path loss spectra. By introducing Planar Gaussians as geometry primitives with certain RF-specific optimizations, RF-PGS achieves dense, surface-aligned scene reconstruction in the first geometry training stage. In the subsequent Radio Frequency (RF) training stage, the proposed fully-structured radio radiance, combined with a tailored multi-view loss, accurately models radio propagation behavior. Compared to prior radiance field methods, RF-PGS significantly improves reconstruction accuracy, reduces training costs, and enables efficient representation of wireless channels, offering a practical solution for scalable 6G Spatial-CSI modeling.

Method: Created MTMEUR benchmark with real-life scenario videos and progressive questions covering emotion recognition, causes, and future actions. Proposed multi-agent framework with specialized agents for different reasoning aspects.

Result: Experiments show most existing MLLMs face significant challenges with emotion reasoning tasks on the new benchmark.

Conclusion: The proposed benchmark and multi-agent framework address the gap in emotion reasoning capabilities of MLLMs, highlighting the need for improved reasoning in human-machine interaction systems.

Abstract: Multimodal large language models (MLLMs) have been widely applied across various fields due to their powerful perceptual and reasoning capabilities. In the realm of psychology, these models hold promise for a deeper understanding of human emotions and behaviors. However, recent research primarily focuses on enhancing their emotion recognition abilities, leaving the substantial potential in emotion reasoning, which is crucial for improving the naturalness and effectiveness of human-machine interactions. Therefore, in this paper, we introduce a multi-turn multimodal emotion understanding and reasoning (MTMEUR) benchmark, which encompasses 1,451 video data from real-life scenarios, along with 5,101 progressive questions. These questions cover various aspects, including emotion recognition, potential causes of emotions, future action prediction, etc. Besides, we propose a multi-agent framework, where each agent specializes in a specific aspect, such as background context, character dynamics, and event details, to improve the system’s reasoning capabilities. Furthermore, we conduct experiments with existing MLLMs and our agent-based method on the proposed benchmark, revealing that most models face significant challenges with this task.

Method: Two-stage framework: 1) Forensic-adapted LLaVA generates initial proposals, 2) Forensics Rectification Module validates proposals with multi-scale forensic feature analysis and Enhanced Segmentation Module refines boundaries.

Result: State-of-the-art performance across diverse datasets with exceptional robustness and generalization capabilities.

Conclusion: Synergistic combination of multimodal reasoning and forensic methodologies ensures comprehensive detection accuracy and precise localization.

Abstract: The increasing sophistication of image manipulation techniques demands robust forensic solutions that can both reliably detect alterations and precisely localize tampered regions. Recent Multimodal Large Language Models (MLLMs) show promise by leveraging world knowledge and semantic understanding for context-aware detection, yet they struggle with perceiving subtle, low-level forensic artifacts crucial for accurate manipulation localization. This paper presents a novel Propose-Rectify framework that effectively bridges semantic reasoning with forensic-specific analysis. In the proposal stage, our approach utilizes a forensic-adapted LLaVA model to generate initial manipulation analysis and preliminary localization of suspicious regions based on semantic understanding and contextual reasoning. In the rectification stage, we introduce a Forensics Rectification Module that systematically validates and refines these initial proposals through multi-scale forensic feature analysis, integrating technical evidence from several specialized filters. Additionally, we present an Enhanced Segmentation Module that incorporates critical forensic cues into SAM’s encoded image embeddings, thereby overcoming inherent semantic biases to achieve precise delineation of manipulated regions. By synergistically combining advanced multimodal reasoning with established forensic methodologies, our framework ensures that initial semantic proposals are systematically validated and enhanced through concrete technical evidence, resulting in comprehensive detection accuracy and localization precision. Extensive experimental validation demonstrates state-of-the-art performance across diverse datasets with exceptional robustness and generalization capabilities.

Method: Applies Singular Value Decomposition (SVD) to factorize weight deltas from DreamBooth fine-tuning, followed by energy-based rank truncation to balance compression efficiency and reconstruction fidelity.

Result: Achieves substantial compression with negligible loss in generation quality (measured by CLIP score, SSIM, FID) while maintaining plug-and-play compatibility and original model architecture.

Conclusion: Delta-SVD enables scalable and efficient deployment of personalized diffusion models, providing a practical solution for storing and deploying large-scale subject customizations without retraining.

Abstract: Personalized text-to-image models such as DreamBooth require fine-tuning large-scale diffusion backbones, resulting in significant storage overhead when maintaining many subject-specific models. We present Delta-SVD, a post-hoc, training-free compression method that targets the parameter weights update induced by DreamBooth fine-tuning. Our key observation is that these delta weights exhibit strong low-rank structure due to the sparse and localized nature of personalization. Delta-SVD first applies Singular Value Decomposition (SVD) to factorize the weight deltas, followed by an energy-based rank truncation strategy to balance compression efficiency and reconstruction fidelity. The resulting compressed models are fully plug-and-play and can be re-constructed on-the-fly during inference. Notably, the proposed approach is simple, efficient, and preserves the original model architecture. Experiments on a multiple subject dataset demonstrate that Delta-SVD achieves substantial compression with negligible loss in generation quality measured by CLIP score, SSIM and FID. Our method enables scalable and efficient deployment of personalized diffusion models, making it a practical solution for real-world applications that require storing and deploying large-scale subject customizations.

Method: Three-pronged approach: (1) direct sentiment classification using MLLMs, (2) sentiment analysis on generated image descriptions using pre-trained LLMs, and (3) fine-tuning LLMs on sentiment-labeled image descriptions.

Result: Achieves SOTA results, outperforming Lexicon-, CNN-, and Transformer-based baselines by up to 30.9%, 64.8%, and 42.4% respectively. Cross-dataset testing shows 8.26% improvement over best runner-up without additional training.

Conclusion: The framework demonstrates strong visual reasoning capabilities for affective computing and establishes new benchmarks for future visual sentiment analysis research.

Abstract: Understanding how visual content communicates sentiment is critical in an era where online interaction is increasingly dominated by this kind of media on social platforms. However, this remains a challenging problem, as sentiment perception is closely tied to complex, scene-level semantics. In this paper, we propose an original framework, MLLMsent, to investigate the sentiment reasoning capabilities of Multimodal Large Language Models (MLLMs) through three perspectives: (1) using those MLLMs for direct sentiment classification from images; (2) associating them with pre-trained LLMs for sentiment analysis on automatically generated image descriptions; and (3) fine-tuning the LLMs on sentiment-labeled image descriptions. Experiments on a recent and established benchmark demonstrate that our proposal, particularly the fine-tuned approach, achieves state-of-the-art results outperforming Lexicon-, CNN-, and Transformer-based baselines by up to 30.9%, 64.8%, and 42.4%, respectively, across different levels of evaluators’ agreement and sentiment polarity categories. Remarkably, in a cross-dataset test, without any training on these new data, our model still outperforms, by up to 8.26%, the best runner-up, which has been trained directly on them. These results highlight the potential of the proposed visual reasoning scheme for advancing affective computing, while also establishing new benchmarks for future research.

Method: Proposes a unified shared weight architecture with: 1) Global feature perception module using BLIP captions to extract scene features and identify degradation types, 2) Local module using ChatGPT descriptions for specific degradation effects, 3) Textual constraints to guide fusion image generation and network learning.

Result: Extensive experiments show AWM-Fuse outperforms current state-of-the-art methods in complex weather conditions and downstream tasks.

Conclusion: The proposed method effectively leverages both global and local textual information to handle multiple weather degradations and improve semantic perception, demonstrating superior performance in adverse weather image fusion.

Abstract: Multi-modality image fusion (MMIF) in adverse weather aims to address the loss of visual information caused by weather-related degradations, providing clearer scene representations. Although less studies have attempted to incorporate textual information to improve semantic perception, they often lack effective categorization and thorough analysis of textual content. In response, we propose AWM-Fuse, a novel fusion method for adverse weather conditions, designed to handle multiple degradations through global and local text perception within a unified, shared weight architecture. In particular, a global feature perception module leverages BLIP-produced captions to extract overall scene features and identify primary degradation types, thus promoting generalization across various adverse weather conditions. Complementing this, the local module employs detailed scene descriptions produced by ChatGPT to concentrate on specific degradation effects through concrete textual cues, thereby capturing finer details. Furthermore, textual descriptions are used to constrain the generation of fusion images, effectively steering the network learning process toward better alignment with real semantic labels, thereby promoting the learning of more meaningful visual features. Extensive experiments demonstrate that AWM-Fuse outperforms current state-of-the-art methods in complex weather conditions and downstream tasks. Our code is available at https://github.com/Feecuin/AWM-Fuse.

Method: Proposes SAEViT with: 1) Sparsely Aggregated Attention module for adaptive sparse sampling, 2) Channel-Interactive Feed-Forward Network for better inter-channel information exchange, 3) Hierarchical pyramid structure with depth-wise separable convolutional blocks.

Result: Achieves 76.3% and 79.6% Top-1 accuracy on ImageNet-1K with only 0.8 GFLOPs and 1.3 GFLOPs respectively, demonstrating efficient performance.

Conclusion: SAEViT provides a lightweight solution that balances computation efficiency and performance for various vision tasks, addressing ViT’s limitations in real-world scenarios.

Abstract: Vision Transformer (ViT) has prevailed in computer vision tasks due to its strong long-range dependency modelling ability. However, its large model size with high computational cost and weak local feature modeling ability hinder its application in real scenarios. To balance computation efficiency and performance, we propose SAEViT (Sparse-Attention-Efficient-ViT), a lightweight ViT based model with convolution blocks, in this paper to achieve efficient downstream vision tasks. Specifically, SAEViT introduces a Sparsely Aggregated Attention (SAA) module that performs adaptive sparse sampling based on image redundancy and recovers the feature map via deconvolution operation, which significantly reduces the computational complexity of attention operations. In addition, a Channel-Interactive Feed-Forward Network (CIFFN) layer is developed to enhance inter-channel information exchange through feature decomposition and redistribution, mitigating redundancy in traditional feed-forward networks (FNN). Finally, a hierarchical pyramid structure with embedded depth-wise separable convolutional blocks (DWSConv) is devised to further strengthen convolutional features. Extensive experiments on mainstream datasets show that SAEViT achieves Top-1 accuracies of 76.3% and 79.6% on the ImageNet-1K classification task with only 0.8 GFLOPs and 1.3 GFLOPs, respectively, demonstrating a lightweight solution for various fundamental vision tasks.

Method: Proposes Structural Energy-Guided Sampling (SEGS) that defines structural energy in PCA subspace of U-Net features and injects its gradients into denoising trajectory to steer geometry toward intended viewpoints while preserving appearance fidelity.

Result: SEGS significantly reduces Janus artifacts, achieves improved geometric alignment and viewpoint consistency, and integrates seamlessly into SDS/VSD pipelines without retraining or weight modification.

Conclusion: SEGS provides an effective plug-and-play solution to the viewpoint bias problem in text-to-3D generation by enforcing multi-view consistency entirely at sampling time through structural energy guidance.

Abstract: Text-to-3D generation often suffers from the Janus problem, where objects look correct from the front but collapse into duplicated or distorted geometry from other angles. We attribute this failure to viewpoint bias in 2D diffusion priors, which propagates into 3D optimization. To address this, we propose Structural Energy-Guided Sampling (SEGS), a training-free, plug-and-play framework that enforces multi-view consistency entirely at sampling time. SEGS defines a structural energy in a PCA subspace of intermediate U-Net features and injects its gradients into the denoising trajectory, steering geometry toward the intended viewpoint while preserving appearance fidelity. Integrated seamlessly into SDS/VSD pipelines, SEGS significantly reduces Janus artifacts, achieving improved geometric alignment and viewpoint consistency without retraining or weight modification.

Method: Three-component architecture: 1) Multi-Scale ResNet Backbone extracts diverse multi-scale features, 2) Patchify Capsule Layer partitions features into primary capsules with uniform patch size, 3) Cross-Agreement Routing blocks adaptively route capsules by identifying cross-scale prediction pairs with maximum agreement.

Result: Achieves remarkable scalability and superior robustness, consistently surpassing baseline methods in classification accuracy. Configurations range from Tiny model (344.3K parameters) to Large model (10.9M parameters).

Conclusion: MSPCaps demonstrates significant potential in advancing feature representation learning by effectively integrating multi-scale features with capsule routing, overcoming limitations of existing CapsNet approaches.

Abstract: Capsule Network (CapsNet) has demonstrated significant potential in visual recognition by capturing spatial relationships and part-whole hierarchies for learning equivariant feature representations. However, existing CapsNet and variants often rely on a single high-level feature map, overlooking the rich complementary information from multi-scale features. Furthermore, conventional feature fusion strategies (e.g., addition and concatenation) struggle to reconcile multi-scale feature discrepancies, leading to suboptimal classification performance. To address these limitations, we propose the Multi-Scale Patchify Capsule Network (MSPCaps), a novel architecture that integrates multi-scale feature learning and efficient capsule routing. Specifically, MSPCaps consists of three key components: a Multi-Scale ResNet Backbone (MSRB), a Patchify Capsule Layer (PatchifyCaps), and Cross-Agreement Routing (CAR) blocks. First, the MSRB extracts diverse multi-scale feature representations from input images, preserving both fine-grained details and global contextual information. Second, the PatchifyCaps partitions these multi-scale features into primary capsules using a uniform patch size, equipping the model with the ability to learn from diverse receptive fields. Finally, the CAR block adaptively routes the multi-scale capsules by identifying cross-scale prediction pairs with maximum agreement. Unlike the simple concatenation of multiple self-routing blocks, CAR ensures that only the most coherent capsules contribute to the final voting. Our proposed MSPCaps achieves remarkable scalability and superior robustness, consistently surpassing multiple baseline methods in terms of classification accuracy, with configurations ranging from a highly efficient Tiny model (344.3K parameters) to a powerful Large model (10.9M parameters), highlighting its potential in advancing feature representation learning.

Method: Contrastive learning and cross-modal alignment framework that encodes fibrosis-specific pathology into cine CMR embeddings by aligning latent spaces of cine CMR and LGE sequences. Includes Feature Interaction Module and uncertainty-guided adaptive training.

Result: Achieved 0.943 accuracy (95% CI: 0.886-0.986) on multi-center data from 231 subjects, outperforming state-of-the-art cine-CMR-only models by 4.3% while eliminating gadolinium dependency.

Conclusion: CC-CMR demonstrates clinical viability for wide population screening by providing accurate gadolinium-free cardiomyopathy detection through cross-modal learning, making it suitable for diverse healthcare environments.

Abstract: Cardiomyopathy, a principal contributor to heart failure and sudden cardiac mortality, demands precise early screening. Cardiac Magnetic Resonance (CMR), recognized as the diagnostic ‘gold standard’ through multiparametric protocols, holds the potential to serve as an accurate screening tool. However, its reliance on gadolinium contrast and labor-intensive interpretation hinders population-scale deployment. We propose CC-CMR, a Contrastive Learning and Cross-Modal alignment framework for gadolinium-free cardiomyopathy screening using cine CMR sequences. By aligning the latent spaces of cine CMR and Late Gadolinium Enhancement (LGE) sequences, our model encodes fibrosis-specific pathology into cine CMR embeddings. A Feature Interaction Module concurrently optimizes diagnostic precision and cross-modal feature congruence, augmented by an uncertainty-guided adaptive training mechanism that dynamically calibrates task-specific objectives to ensure model generalizability. Evaluated on multi-center data from 231 subjects, CC-CMR achieves accuracy of 0.943 (95% CI: 0.886-0.986), outperforming state-of-the-art cine-CMR-only models by 4.3% while eliminating gadolinium dependency, demonstrating its clinical viability for wide range of populations and healthcare environments.

Method: Develops a camera-conditioned 3D-to-text inverse mechanism that projects 3D contents into a 3D embedding space aligned with text embeddings, enabling natural language manipulation of 3D content.

Result: Extensive experiments demonstrate that Invert3D achieves effective personalization of 3D content through natural language prompts.

Conclusion: The proposed framework successfully bridges the gap between 3D content and text embedding spaces, enabling convenient and efficient 3D content personalization without the need for retraining procedures.

Abstract: Recent advances in NeRF and 3DGS have significantly enhanced the efficiency and quality of 3D content synthesis. However, efficient personalization of generated 3D content remains a critical challenge. Current 3D personalization approaches predominantly rely on knowledge distillation-based methods, which require computationally expensive retraining procedures. To address this challenge, we propose \textbf{Invert3D}, a novel framework for convenient 3D content personalization. Nowadays, vision-language models such as CLIP enable direct image personalization through aligned vision-text embedding spaces. However, the inherent structural differences between 3D content and 2D images preclude direct application of these techniques to 3D personalization. Our approach bridges this gap by establishing alignment between 3D representations and text embedding spaces. Specifically, we develop a camera-conditioned 3D-to-text inverse mechanism that projects 3D contents into a 3D embedding aligned with text embeddings. This alignment enables efficient manipulation and personalization of 3D content through natural language prompts, eliminating the need for computationally retraining procedures. Extensive experiments demonstrate that Invert3D achieves effective personalization of 3D content. Our work is available at: https://github.com/qsong2001/Invert3D.

Method: Utilizes unsupervised domain adaptation methodology, combining a small expert-annotated ground truth dataset with unlabeled data for training.

Result: Quantitative and qualitative evaluations confirm the method significantly outperforms existing state-of-the-art approaches.

Conclusion: Demonstrates the efficacy of domain adaptation techniques for label-scarce biomedical imaging tasks, particularly in cerebral vasculature segmentation.

Abstract: This work presents a novel deep learning framework for segmenting cerebral vasculature in hyperspectral brain images. We address the critical challenge of severe label scarcity, which impedes conventional supervised training. Our approach utilizes a novel unsupervised domain adaptation methodology, using a small, expert-annotated ground truth alongside unlabeled data. Quantitative and qualitative evaluations confirm that our method significantly outperforms existing state-of-the-art approaches, demonstrating the efficacy of domain adaptation for label-scarce biomedical imaging tasks.

Method: NAT shifts from embedding-level separation to neuron-specific targeting, disrupting individual neurons as the core units of neural networks. This provides a common basis for transferability across different models by attacking fundamental network components.

Result: Extensive experiments on 41 diverse ImageNet models and 9 fine-grained models show NAT achieves fooling rates surpassing existing baselines by over 14% in cross-model and 4% in cross-domain settings. The method also achieves impressive fooling rates within just 10 queries.

Conclusion: Targeting individual neurons rather than entire layers provides a more fundamental and effective approach for generating transferable adversarial perturbations, establishing neuron-specific attacks as a superior strategy for cross-model and cross-domain transferability.

Abstract: The generation of transferable adversarial perturbations typically involves training a generator to maximize embedding separation between clean and adversarial images at a single mid-layer of a source model. In this work, we build on this approach and introduce Neuron Attack for Transferability (NAT), a method designed to target specific neuron within the embedding. Our approach is motivated by the observation that previous layer-level optimizations often disproportionately focus on a few neurons representing similar concepts, leaving other neurons within the attacked layer minimally affected. NAT shifts the focus from embedding-level separation to a more fundamental, neuron-specific approach. We find that targeting individual neurons effectively disrupts the core units of the neural network, providing a common basis for transferability across different models. Through extensive experiments on 41 diverse ImageNet models and 9 fine-grained models, NAT achieves fooling rates that surpass existing baselines by over 14% in cross-model and 4% in cross-domain settings. Furthermore, by leveraging the complementary attacking capabilities of the trained generators, we achieve impressive fooling rates within just 10 queries. Our code is available at: https://krishnakanthnakka.github.io/NAT/

Method: Combines Stepwise Action Reasoning (chain of thought process for structured evaluation from recognition to scoring) and Hierarchical Policy Learning (RL strategy to learn fine-grained sub-action dynamics aligned with overall quality).

Result: Demonstrates superior performance across multiple benchmark datasets, providing accurate and interpretable action assessments.

Conclusion: HieroAction effectively addresses the limitation of score-only evaluations by delivering structured, explainable action assessments through integrated reasoning and reinforcement learning approaches.

Abstract: Evaluating human actions with clear and detailed feedback is important in areas such as sports, healthcare, and robotics, where decisions rely not only on final outcomes but also on interpretable reasoning. However, most existing methods provide only a final score without explanation or detailed analysis, limiting their practical applicability. To address this, we introduce HieroAction, a vision-language model that delivers accurate and structured assessments of human actions. HieroAction builds on two key ideas: (1) Stepwise Action Reasoning, a tailored chain of thought process designed specifically for action assessment, which guides the model to evaluate actions step by step, from overall recognition through sub action analysis to final scoring, thus enhancing interpretability and structured understanding; and (2) Hierarchical Policy Learning, a reinforcement learning strategy that enables the model to learn fine grained sub action dynamics and align them with high level action quality, thereby improving scoring precision. The reasoning pathway structures the evaluation process, while policy learning refines each stage through reward based optimization. Their integration ensures accurate and interpretable assessments, as demonstrated by superior performance across multiple benchmark datasets. Code will be released upon acceptance.

Method: Proposes RPD-Diff with Physics-guided Intermediate State Targeting (PIST) strategy that uses physical priors to reformulate diffusion Markov chain, and Haze-Aware Denoising Timestep Predictor (HADTP) that dynamically adjusts patch-specific denoising timesteps using transmission map cross-attention.

Result: Extensive experiments across four real-world datasets demonstrate state-of-the-art performance in challenging dense and non-uniform haze scenarios, delivering high-quality haze-free images with superior detail clarity and color fidelity.

Conclusion: RPD-Diff effectively addresses the limitations of traditional methods by incorporating physics guidance and adaptive mechanisms, achieving robust visibility enhancement in complex haze conditions.

Abstract: Single-image dehazing under dense and non-uniform haze conditions remains challenging due to severe information degradation and spatial heterogeneity. Traditional diffusion-based dehazing methods struggle with insufficient generation conditioning and lack of adaptability to spatially varying haze distributions, which leads to suboptimal restoration. To address these limitations, we propose RPD-Diff, a Region-adaptive Physics-guided Dehazing Diffusion Model for robust visibility enhancement in complex haze scenarios. RPD-Diff introduces a Physics-guided Intermediate State Targeting (PIST) strategy, which leverages physical priors to reformulate the diffusion Markov chain by generation target transitions, mitigating the issue of insufficient conditioning in dense haze scenarios. Additionally, the Haze-Aware Denoising Timestep Predictor (HADTP) dynamically adjusts patch-specific denoising timesteps employing a transmission map cross-attention mechanism, adeptly managing non-uniform haze distributions. Extensive experiments across four real-world datasets demonstrate that RPD-Diff achieves state-of-the-art performance in challenging dense and non-uniform haze scenarios, delivering high-quality, haze-free images with superior detail clarity and color fidelity.

Method: Window partitioning design with grid windows, window-wise contrastive learning to distinguish local density levels, and a global attention module for large-sized individuals.

Result: Significant improvement in local modeling capability (8.7% MAE improvement on JHU-Crowd++ high-density subset) while maintaining ability to count large-sized individuals.

Conclusion: The proposed LIMM model demonstrates that emphasizing local modeling capability is crucial for crowd counting and achieves state-of-the-art performance across multiple datasets.

Abstract: The motivation of this paper originates from rethinking an essential characteristic of crowd counting: individuals (heads of humans) in the crowd counting task typically occupy a very small portion of the image. This characteristic has never been the focus of existing works: they typically use the same backbone as other visual tasks and pursue a large receptive field. This drives us to propose a new model design principle of crowd counting: emphasizing local modeling capability of the model. We follow the principle and design a crowd counting model named Local Information Matters Model (LIMM). The main innovation lies in two strategies: a window partitioning design that applies grid windows to the model input, and a window-wise contrastive learning design to enhance the model’s ability to distinguish between local density levels. Moreover, a global attention module is applied to the end of the model to handle the occasionally occurring large-sized individuals. Extensive experiments on multiple public datasets illustrate that the proposed model shows a significant improvement in local modeling capability (8.7% in MAE on the JHU-Crowd++ high-density subset for example), without compromising its ability to count large-sized ones, which achieves state-of-the-art performance. Code is available at: https://github.com/tianhangpan/LIMM.

Method: Adaptive Multi-View & Consistency Verification (AMCV) mechanism: generates multiple perturbed diagram versions, performs parallel inference, and applies consistency-based self-correction loop.

Result: State-of-the-art LVLMs like GPT-4V show significant degradation (Clean Accuracy 85.2% vs. PRS 72.1%) on perturbed inputs, demonstrating the need for robustness improvements.

Conclusion: The RDR framework and SciDiagram-Robust dataset provide essential tools for evaluating and enhancing LVLM robustness on visually degraded scientific diagrams, revealing critical vulnerabilities in current models.

Abstract: Large Language Models (LLMs) and their multimodal variants (LVLMs) hold immense promise for scientific and engineering applications, particularly in processing visual information like scientific diagrams. However, their practical deployment is hindered by a critical lack of robustness to common visual perturbations such as noise, blur, and occlusions, which are prevalent in real-world scientific documents. Existing evaluation benchmarks largely overlook this challenge, leaving the robust reasoning capabilities of LVLMs on visually degraded scientific diagrams underexplored. To address this, we introduce the Robust Diagram Reasoning (RDR) framework, a novel approach designed to enhance and rigorously evaluate LVLMs’ performance under such conditions. At its core, RDR employs an Adaptive Multi-View & Consistency Verification (AMCV) mechanism, which involves generating multiple perturbed versions of a diagram, performing parallel inference, and then applying a consistency-based self-correction loop. We also propose two new metrics, Perturbation Robustness Score (PRS) and Visual Degradation Consistency (VDC), to quantify robustness. Furthermore, we construct SciDiagram-Robust, the first large-scale scientific diagram question-answering dataset specifically augmented with diverse, programmatically generated visual perturbations. Our extensive experiments demonstrate that even state-of-the-art closed-source LVLMs like GPT-4V exhibit significant performance degradation when faced with perturbed inputs (Clean Accuracy 85.2% vs. PRS 72.1%).

Method: Proposes Balanced Sharpness-Aware Minimization (BSAM) that starts from sharpness-aware minimization and adds targeted reweighting to homogenize generalization ability across observation space.

Result: Extensive experiments on age and depth estimation tasks show BSAM consistently outperforms existing approaches.

Conclusion: BSAM effectively addresses imbalanced regression by ensuring uniform generalization ability with theoretical guarantees, demonstrating superior performance on various vision regression tasks.

Abstract: Regression is fundamental in computer vision and is widely used in various tasks including age estimation, depth estimation, target localization, \etc However, real-world data often exhibits imbalanced distribution, making regression models perform poorly especially for target values with rare observations~(known as the imbalanced regression problem). In this paper, we reframe imbalanced regression as an imbalanced generalization problem. To tackle that, we look into the loss sharpness property for measuring the generalization ability of regression models in the observation space. Namely, given a certain perturbation on the model parameters, we check how model performance changes according to the loss values of different target observations. We propose a simple yet effective approach called Balanced Sharpness-Aware Minimization~(BSAM) to enforce the uniform generalization ability of regression models for the entire observation space. In particular, we start from the traditional sharpness-aware minimization and then introduce a novel targeted reweighting strategy to homogenize the generalization ability across the observation space, which guarantees a theoretical generalization bound. Extensive experiments on multiple vision regression tasks, including age and depth estimation, demonstrate that our BSAM method consistently outperforms existing approaches. The code is available \href{https://github.com/manmanjun/BSAM_for_Imbalanced_Regression}{here}.

Method: Two-layered architecture: Global Contextual Perception for broad understanding and Fine-grained Local Grounding with Local Detail Enhancement Module and Semantic Consistency Validator, integrated through adaptive fusion.

Result: Outperforms state-of-the-art models (Flamingo, BLIP-2, MiniGPT-4) on GQA, A-OKVQA, and RefCOCO datasets with superior accuracy and significantly reduced hallucination.

Conclusion: The hierarchical design effectively enhances fine-grained visual-language understanding and precise grounding capabilities in multimodal tasks.

Abstract: Large Language Models (LLMs) and Vision-Language Large Models (LVLMs) have achieved remarkable progress in natural language processing and multimodal understanding. Despite their impressive generalization capabilities, current LVLMs often exhibit insufficient robustness, proneness to hallucination, and reasoning errors in complex real-world scenarios, particularly when precise image region localization and fine-grained visual reasoning are required. To address these limitations, we propose the Hierarchical Contextual Grounding LVLM (HCG-LVLM), a novel architecture that mimics human coarse-to-fine cognitive processing. HCG-LVLM employs a two-layered approach: a Global Contextual Perception layer for initial broad understanding and a Fine-grained Local Grounding layer. The latter incorporates a Local Detail Enhancement Module to extract high-resolution features and a Semantic Consistency Validator to ensure accurate, hallucination-free visual-language alignment. Through an adaptive fusion mechanism, information from both layers is integrated for robust and precise outputs. Extensive experiments on challenging datasets, including GQA, A-OKVQA for fine-grained VQA, and RefCOCO/+/g for Referring Expression Comprehension, demonstrate that HCG-LVLM consistently outperforms state-of-the-art models such as Flamingo, BLIP-2, and MiniGPT-4. Our model achieves superior accuracy and significantly reduces hallucination, validating the effectiveness of its hierarchical design in enhancing fine-grained visual-language understanding and precise grounding capabilities.

Method: Modified Vision Transformer (ViT) model trained on OpenForensics Dataset subset with multiple augmentation techniques, oversampling for class imbalance, and stratified train-validation split.

Result: The model demonstrates state-of-the-art results on test dataset, achieving high accuracy in detecting Deepfake images and providing prediction scores on random images.

Conclusion: The proposed modified Vision Transformer approach provides an effective and robust solution for Deepfake detection, addressing current challenges in identifying manipulated media content.

Abstract: The rise of Deepfake technology to generate hyper-realistic manipulated images and videos poses a significant challenge to the public and relevant authorities. This study presents a robust Deepfake detection based on a modified Vision Transformer(ViT) model, trained to distinguish between real and Deepfake images. The model has been trained on a subset of the OpenForensics Dataset with multiple augmentation techniques to increase robustness for diverse image manipulations. The class imbalance issues are handled by oversampling and a train-validation split of the dataset in a stratified manner. Performance is evaluated using the accuracy metric on the training and testing datasets, followed by a prediction score on a random image of people, irrespective of their realness. The model demonstrates state-of-the-art results on the test dataset to meticulously detect Deepfake images.

Method: Joint Parameter Selection (JPS) restricts updates to a small, sparse subset of parameters using dual operators to identify parameters with consistent and significant gradients across all source domains.

Result: Extensive benchmark experiments show JPS achieves superior performance compared to state-of-the-art domain generalization methods.

Conclusion: JPS provides an efficient and effective approach for domain generalization by selectively fine-tuning parameters while maintaining the generalization strength of pre-trained models.

Abstract: Domain generalization seeks to develop models trained on a limited set of source domains that are capable of generalizing effectively to unseen target domains. While the predominant approach leverages large-scale pre-trained vision models as initialization, recent studies have highlighted that full fine-tuning can compromise the intrinsic generalization capabilities of these models. To address this limitation, parameter-efficient adaptation strategies have emerged, wherein only a subset of model parameters is selectively fine-tuned, thereby balancing task adaptation with the preservation of generalization. Motivated by this paradigm, we introduce Joint Parameter Selection (JPS), a novel method that restricts updates to a small, sparse subset of parameters, thereby retaining and harnessing the generalization strength of pre-trained models. Theoretically, we establish a generalization error bound that explicitly accounts for the sparsity of parameter updates, thereby providing a principled justification for selective fine-tuning. Practically, we design a selection mechanism employing dual operators to identify and update parameters exhibiting consistent and significant gradients across all source domains. Extensive benchmark experiments demonstrate that JPS achieves superior performance compared to state-of-the-art domain generalization methods, substantiating both the efficiency and efficacy of the proposed approach.

Method: Uses Hermite polynomials as theoretically optimal basis for Gaussian-correlated feature derivative approximations in Diffusion Transformers. Introduces dual-scaling mechanism for numerical stability and predictive accuracy.

Result: Achieves 6.24x speedup on FLUX.1-dev while exceeding baseline quality. Maintains strong performance across text-to-image, video generation, and super-resolution tasks.

Conclusion: HiCache provides a fundamental improvement in feature prediction by aligning mathematical tools with empirical properties, offering significant acceleration without quality loss.

Abstract: Diffusion models have achieved remarkable success in content generation but suffer from prohibitive computational costs due to iterative sampling. While recent feature caching methods tend to accelerate inference through temporal extrapolation, these methods still suffer from server quality loss due to the failure in modeling the complex dynamics of feature evolution. To solve this problem, this paper presents HiCache, a training-free acceleration framework that fundamentally improves feature prediction by aligning mathematical tools with empirical properties. Our key insight is that feature derivative approximations in Diffusion Transformers exhibit multivariate Gaussian characteristics, motivating the use of Hermite polynomials-the potentially theoretically optimal basis for Gaussian-correlated processes. Besides, We further introduce a dual-scaling mechanism that ensures numerical stability while preserving predictive accuracy. Extensive experiments demonstrate HiCache’s superiority: achieving 6.24x speedup on FLUX.1-dev while exceeding baseline quality, maintaining strong performance across text-to-image, video generation, and super-resolution tasks. Core implementation is provided in the appendix, with complete code to be released upon acceptance.

Method: Proposes EDLDNet with a noisy decoder that incorporates structured perturbation during training for robustness but uses only noise-free decoder at inference. Utilizes Multi-Scale Convolutional Attention Modules, Attention Gates, Up-Convolution Blocks, and a mutation-based loss function leveraging multi-scale segmentation masks.

Result: Outperforms SOTA methods on four medical imaging datasets. Achieves 84.00% Dice score on Synapse dataset (13.89% improvement over UNet) while reducing MACs by 89.7%. Maintains comparable computational efficiency to recent approaches like EMCAD while achieving higher Dice scores.

Conclusion: EDLDNet demonstrates strong generalization, computational efficiency, and robustness across diverse datasets, establishing it as an effective solution for organ-at-risk segmentation in medical imaging applications.

Abstract: Proper segmentation of organs-at-risk is important for radiation therapy, surgical planning, and diagnostic decision-making in medical image analysis. While deep learning-based segmentation architectures have made significant progress, they often fail to balance segmentation accuracy with computational efficiency. Most of the current state-of-the-art methods either prioritize performance at the cost of high computational complexity or compromise accuracy for efficiency. This paper addresses this gap by introducing an efficient dual-line decoder segmentation network (EDLDNet). The proposed method features a noisy decoder, which learns to incorporate structured perturbation at training time for better model robustness, yet at inference time only the noise-free decoder is executed, leading to lower computational cost. Multi-Scale convolutional Attention Modules (MSCAMs), Attention Gates (AGs), and Up-Convolution Blocks (UCBs) are further utilized to optimize feature representation and boost segmentation performance. By leveraging multi-scale segmentation masks from both decoders, we also utilize a mutation-based loss function to enhance the model’s generalization. Our approach outperforms SOTA segmentation architectures on four publicly available medical imaging datasets. EDLDNet achieves SOTA performance with an 84.00% Dice score on the Synapse dataset, surpassing baseline model like UNet by 13.89% in Dice score while significantly reducing Multiply-Accumulate Operations (MACs) by 89.7%. Compared to recent approaches like EMCAD, our EDLDNet not only achieves higher Dice score but also maintains comparable computational efficiency. The outstanding performance across diverse datasets establishes EDLDNet’s strong generalization, computational efficiency, and robustness. The source code, pre-processed data, and pre-trained weights will be available at https://github.com/riadhassan/EDLDNet .

Method: Uses Large Language Models to derive category clusters encoding intrinsic inter-class relationships, and introduces a class-aware patch-level contrastive loss to enforce intra-class consistency and inter-class separation in a hierarchical design.

Result: Experiments on PASCAL VOC 2012 and MS COCO 2014 show that CPC surpasses existing state-of-the-art methods in weakly supervised semantic segmentation.

Conclusion: CPC effectively addresses the limitations of previous WSSS methods by leveraging hierarchical semantic priors from LLMs and contrastive learning, achieving superior performance while maintaining fine-grained category discrimination.

Abstract: Weakly Supervised Semantic Segmentation (WSSS) with image-level labels has gained attention for its cost-effectiveness. Most existing methods emphasize inter-class separation, often neglecting the shared semantics among related categories and lacking fine-grained discrimination. To address this, we propose Contrastive Prompt Clustering (CPC), a novel WSSS framework. CPC exploits Large Language Models (LLMs) to derive category clusters that encode intrinsic inter-class relationships, and further introduces a class-aware patch-level contrastive loss to enforce intra-class consistency and inter-class separation. This hierarchical design leverages clusters as coarse-grained semantic priors while preserving fine-grained boundaries, thereby reducing confusion among visually similar categories. Experiments on PASCAL VOC 2012 and MS COCO 2014 demonstrate that CPC surpasses existing state-of-the-art methods in WSSS.

Method: Proposes a novel algorithm for efficiently generating Gaussian Splatting-based fiducial markers (e.g., AprilTags) within cluttered scenes, creating a hybrid framework that combines photorealism with structured marker representations.

Result: Outperforms traditional image-fitting techniques in both efficiency and pose-estimation accuracy. Successfully demonstrated in a challenging greenhouse simulation with dense foliage, similar elements, and occlusions.

Conclusion: The framework shows significant value for real-world robotic applications, particularly in agricultural settings where complex visual environments push perception limits, enabling more accurate localization and control.

Abstract: High-fidelity 3D simulation is critical for training mobile robots, but its traditional reliance on mesh-based representations often struggle in complex environments, such as densely packed greenhouses featuring occlusions and repetitive structures. Recent neural rendering methods, like Gaussian Splatting (GS), achieve remarkable visual realism but lack flexibility to incorporate fiducial markers, which are essential for robotic localization and control. We propose a hybrid framework that combines the photorealism of GS with structured marker representations. Our core contribution is a novel algorithm for efficiently generating GS-based fiducial markers (e.g., AprilTags) within cluttered scenes. Experiments show that our approach outperforms traditional image-fitting techniques in both efficiency and pose-estimation accuracy. We further demonstrate the framework’s potential in a greenhouse simulation. This agricultural setting serves as a challenging testbed, as its combination of dense foliage, similar-looking elements, and occlusions pushes the limits of perception, thereby highlighting the framework’s value for real-world applications.

Method: Introduces DOG - a sampling guidance strategy using orthogonal projection of negatively perturbed prompts onto positive prompts, with triangular scheduling to control guidance strength throughout denoising.

Result: DOG improves content clarity and style variability compared to standard CFG, working well even for out-of-vocabulary words and challenging writing styles in DiffusionPen and One-DM models.

Conclusion: DOG provides more stable and disentangled guidance than CFG, effectively reducing artifacts while maintaining content integrity and enabling diverse yet plausible handwritten text generation.

Abstract: Diffusion-based Handwritten Text Generation (HTG) approaches achieve impressive results on frequent, in-vocabulary words observed at training time and on regular styles. However, they are prone to memorizing training samples and often struggle with style variability and generation clarity. In particular, standard diffusion models tend to produce artifacts or distortions that negatively affect the readability of the generated text, especially when the style is hard to produce. To tackle these issues, we propose a novel sampling guidance strategy, Dual Orthogonal Guidance (DOG), that leverages an orthogonal projection of a negatively perturbed prompt onto the original positive prompt. This approach helps steer the generation away from artifacts while maintaining the intended content, and encourages more diverse, yet plausible, outputs. Unlike standard Classifier-Free Guidance (CFG), which relies on unconditional predictions and produces noise at high guidance scales, DOG introduces a more stable, disentangled direction in the latent space. To control the strength of the guidance across the denoising process, we apply a triangular schedule: weak at the start and end of denoising, when the process is most sensitive, and strongest in the middle steps. Experimental results on the state-of-the-art DiffusionPen and One-DM demonstrate that DOG improves both content clarity and style variability, even for out-of-vocabulary words and challenging writing styles.

Method: Introduces Local Focus Mechanism (LFM) with Salience Network (SNet) and Top-K Pooling (TKP) module to select most informative local patterns. Includes regularization techniques Rank-Based Linear Dropout (RBLD) and Random-K Sampling (RKS) to prevent overfitting.

Result: Achieves 3.7% improvement in accuracy and 2.8% increase in average precision over state-of-the-art NPR method, with exceptional efficiency of 1789 FPS on single NVIDIA A6000 GPU.

Conclusion: LFM sets new benchmark for cross-domain deepfake detection by effectively addressing CNN limitations and maintaining high efficiency, providing robust generalization across different object categories and generation domains.

Abstract: The rapid advancement of deepfake generation techniques has intensified the need for robust and generalizable detection methods. Existing approaches based on reconstruction learning typically leverage deep convolutional networks to extract differential features. However, these methods show poor generalization across object categories (e.g., from faces to cars) and generation domains (e.g., from GANs to Stable Diffusion), due to intrinsic limitations of deep CNNs. First, models trained on a specific category tend to overfit to semantic feature distributions, making them less transferable to other categories, especially as network depth increases. Second, Global Average Pooling (GAP) compresses critical local forgery cues into a single vector, thus discarding discriminative patterns vital for real-fake classification. To address these issues, we propose a novel Local Focus Mechanism (LFM) that explicitly attends to discriminative local features for differentiating fake from real images. LFM integrates a Salience Network (SNet) with a task-specific Top-K Pooling (TKP) module to select the K most informative local patterns. To mitigate potential overfitting introduced by Top-K pooling, we introduce two regularization techniques: Rank-Based Linear Dropout (RBLD) and Random-K Sampling (RKS), which enhance the model’s robustness. LFM achieves a 3.7 improvement in accuracy and a 2.8 increase in average precision over the state-of-the-art Neighboring Pixel Relationships (NPR) method, while maintaining exceptional efficiency at 1789 FPS on a single NVIDIA A6000 GPU. Our approach sets a new benchmark for cross-domain deepfake detection. The source code are available in https://github.com/lmlpy/LFM.git

Method: Proposes a training-free VLM-guided framework with uni/bi-directional multi-modal fusion (combining image embeddings with VLM-generated text) and a feature-based re-ranking mechanism using predicted food ingredients for top-k retrieval.

Result: Achieves ~10% and ~7.7% improvements in top-1 retrieval under dense/sparse caption scenarios, ~28.6% gain in top-k ingredient-level retrieval, and enables smaller models to match larger counterparts when augmented with textual fusion.

Conclusion: The framework significantly enhances food image-text search performance without requiring training, demonstrating effectiveness in resource-constrained settings through improved multi-modal feature representations.

Abstract: The proliferation of digital food content has intensified the need for robust and accurate systems capable of fine-grained visual understanding and retrieval. In this work, we address the challenging task of food image-to-text matching, a critical component in applications such as dietary monitoring, smart kitchens, and restaurant automation. We propose F4-ITS: Fine-grained Feature Fusion for Food Image-Text Search, a training-free, vision-language model (VLM)-guided framework that significantly improves retrieval performance through enhanced multi-modal feature representations. Our approach introduces two key contributions: (1) a uni-directional(and bi-directional) multi-modal fusion strategy that combines image embeddings with VLM-generated textual descriptions to improve query expressiveness, and (2) a novel feature-based re-ranking mechanism for top-k retrieval, leveraging predicted food ingredients to refine results and boost precision. Leveraging open-source image-text encoders, we demonstrate substantial gains over standard baselines - achieving ~10% and ~7.7% improvements in top-1 retrieval under dense and sparse caption scenarios, and a ~28.6% gain in top-k ingredient-level retrieval. Additionally, we show that smaller models (e.g., ViT-B/32) can match or outperform larger counterparts (e.g., ViT-H, ViT-G, ViT-bigG) when augmented with textual fusion, highlighting the effectiveness of our method in resource-constrained settings. Code and test datasets will be made publicly available at: https://github.com/mailcorahul/f4-its

Method: Proposes a taxonomy classifying methods by scene types, representations, learning methods, and applications. Introduces M3DMap - a modular system with neural multimodal object segmentation/tracking, trainable odometry estimation, 3D map construction/updating, and multimodal data retrieval modules.

Result: Provides structured analysis of recent methods and presents original implementations demonstrating advantages in practical tasks like 3D object grounding and mobile manipulation.

Conclusion: Theoretical propositions show positive effects of using multimodal data and modern foundational models in 3D mapping, with the taxonomy and M3DMap method providing a framework for advancing dynamic scene representation.

Abstract: 3D mapping in dynamic environments poses a challenge for modern researchers in robotics and autonomous transportation. There are no universal representations for dynamic 3D scenes that incorporate multimodal data such as images, point clouds, and text. This article takes a step toward solving this problem. It proposes a taxonomy of methods for constructing multimodal 3D maps, classifying contemporary approaches based on scene types and representations, learning methods, and practical applications. Using this taxonomy, a brief structured analysis of recent methods is provided. The article also describes an original modular method called M3DMap, designed for object-aware construction of multimodal 3D maps for both static and dynamic scenes. It consists of several interconnected components: a neural multimodal object segmentation and tracking module; an odometry estimation module, including trainable algorithms; a module for 3D map construction and updating with various implementations depending on the desired scene representation; and a multimodal data retrieval module. The article highlights original implementations of these modules and their advantages in solving various practical tasks, from 3D object grounding to mobile manipulation. Additionally, it presents theoretical propositions demonstrating the positive effect of using multimodal data and modern foundational models in 3D mapping methods. Details of the taxonomy and method implementation are available at https://yuddim.github.io/M3DMap.

Method: Leverages textual inversion and diffusion-based guided image generation to systematically augment small style datasets, then trains image-to-image translation networks on the augmented data.

Result: Outperforms diffusion-based methods in both speed and quality, improves stylization quality, better preserves source image content, and significantly accelerates inference across multiple styles.

Conclusion: The method effectively combines data augmentation through diffusion guidance with efficient image-to-image networks to achieve superior stylization performance with limited style examples.

Abstract: We present StyleClone, a method for training image-to-image translation networks to stylize faces in a specific style, even with limited style images. Our approach leverages textual inversion and diffusion-based guided image generation to augment small style datasets. By systematically generating diverse style samples guided by both the original style images and real face images, we significantly enhance the diversity of the style dataset. Using this augmented dataset, we train fast image-to-image translation networks that outperform diffusion-based methods in speed and quality. Experiments on multiple styles demonstrate that our method improves stylization quality, better preserves source image content, and significantly accelerates inference. Additionally, we provide a systematic evaluation of the augmentation techniques and their impact on stylization performance.

Method: Uses classifier-free guidance-based DDPMs with sparse point cloud as condition and synthesized nearby frames as input. Includes voxel completion module for feature refinement and point-voxel interaction module to integrate features from both points and voxels.

Result: Extensive experiments on various benchmarks demonstrate state-of-the-art performance. The method is the first scene-level point cloud upsampling approach supporting arbitrary upsampling rates.

Conclusion: PVNet effectively addresses LiDAR point cloud sparsity in outdoor environments through a diffusion-based framework with point-voxel interaction, showing superior generalization and performance compared to existing methods.

Abstract: Accurate 3D scene understanding in outdoor environments heavily relies on high-quality point clouds. However, LiDAR-scanned data often suffer from extreme sparsity, severely hindering downstream 3D perception tasks. Existing point cloud upsampling methods primarily focus on individual objects, thus demonstrating limited generalization capability for complex outdoor scenes. To address this issue, we propose PVNet, a diffusion model-based point-voxel interaction framework to perform LiDAR point cloud upsampling without dense supervision. Specifically, we adopt the classifier-free guidance-based DDPMs to guide the generation, in which we employ a sparse point cloud as the guiding condition and the synthesized point clouds derived from its nearby frames as the input. Moreover, we design a voxel completion module to refine and complete the coarse voxel features for enriching the feature representation. In addition, we propose a point-voxel interaction module to integrate features from both points and voxels, which efficiently improves the environmental perception capability of each upsampled point. To the best of our knowledge, our approach is the first scene-level point cloud upsampling method supporting arbitrary upsampling rates. Extensive experiments on various benchmarks demonstrate that our method achieves state-of-the-art performance. The source code will be available at https://github.com/chengxianjing/PVNet.

Method: Proposes DeltaFlow framework with: 1) Δ scheme for efficient temporal feature extraction regardless of frame count, 2) Category-Balanced Loss for underrepresented classes, 3) Instance Consistency Loss for coherent object motion.

Result: Achieves state-of-the-art performance on Argoverse 2 and Waymo datasets with 22% lower error and 2x faster inference compared to next-best multi-frame supervised method. Demonstrates strong cross-domain generalization.

Conclusion: DeltaFlow effectively leverages temporal information with minimal computational overhead while addressing class imbalance and motion inconsistency challenges in scene flow estimation.

Abstract: Previous dominant methods for scene flow estimation focus mainly on input from two consecutive frames, neglecting valuable information in the temporal domain. While recent trends shift towards multi-frame reasoning, they suffer from rapidly escalating computational costs as the number of frames grows. To leverage temporal information more efficiently, we propose DeltaFlow ($\Delta$Flow), a lightweight 3D framework that captures motion cues via a $\Delta$ scheme, extracting temporal features with minimal computational cost, regardless of the number of frames. Additionally, scene flow estimation faces challenges such as imbalanced object class distributions and motion inconsistency. To tackle these issues, we introduce a Category-Balanced Loss to enhance learning across underrepresented classes and an Instance Consistency Loss to enforce coherent object motion, improving flow accuracy. Extensive evaluations on the Argoverse 2 and Waymo datasets show that $\Delta$Flow achieves state-of-the-art performance with up to 22% lower error and $2\times$ faster inference compared to the next-best multi-frame supervised method, while also demonstrating a strong cross-domain generalization ability. The code is open-sourced at https://github.com/Kin-Zhang/DeltaFlow along with trained model weights.

Method: Proposes REGEN framework with dual-stage generative network that transforms unpaired image-to-image translation into simpler paired task using lightweight method for real-time inference.

Result: Achieves visual results comparable to robust unpaired Im2Im methods while improving inference speed by 32.14x, outperforming directly trained lightweight unpaired methods on GTA V.

Conclusion: REGEN enables real-time photorealism enhancement in games without compromising visual quality, demonstrating significant speed improvements over existing methods while maintaining semantic consistency.

Abstract: Photorealism is an important aspect of modern video games since it can shape the player experience and simultaneously impact the immersion, narrative engagement, and visual fidelity. Although recent hardware technological breakthroughs, along with state-of-the-art rendering technologies, have significantly improved the visual realism of video games, achieving true photorealism in dynamic environments at real-time frame rates still remains a major challenge due to the tradeoff between visual quality and performance. In this short paper, we present a novel approach for enhancing the photorealism of rendered game frames using generative adversarial networks. To this end, we propose Real-time photorealism Enhancement in Games via a dual-stage gEnerative Network framework (REGEN), which employs a robust unpaired image-to-image translation model to produce semantically consistent photorealistic frames that transform the problem into a simpler paired image-to-image translation task. This enables training with a lightweight method that can achieve real-time inference time without compromising visual quality. We demonstrate the effectiveness of our framework on Grand Theft Auto V, showing that the approach achieves visual results comparable to the ones produced by the robust unpaired Im2Im method while improving inference speed by 32.14 times. Our findings also indicate that the results outperform the photorealism-enhanced frames produced by directly training a lightweight unpaired Im2Im translation method to translate the video game frames towards the visual characteristics of real-world images. Code, pre-trained models, and demos for this work are available at: https://github.com/stefanos50/REGEN.

Method: A decoupled two-stage process: 1) Spatial Signal Prompting generates visual prompts using pre-trained multi-modal models, 2) SSG-Adapter injects joint conditions into a frozen video DiT backbone with dual-branch attention mechanism.

Result: Achieves state-of-the-art performance on VBench benchmark, particularly excelling in spatial relationship control and overall consistency metrics.

Conclusion: SSG-DiT effectively addresses semantic consistency issues in controllable video generation through spatial signal guidance and parameter-efficient adapter design.

Abstract: Controllable video generation aims to synthesize video content that aligns precisely with user-provided conditions, such as text descriptions and initial images. However, a significant challenge persists in this domain: existing models often struggle to maintain strong semantic consistency, frequently generating videos that deviate from the nuanced details specified in the prompts. To address this issue, we propose SSG-DiT (Spatial Signal Guided Diffusion Transformer), a novel and efficient framework for high-fidelity controllable video generation. Our approach introduces a decoupled two-stage process. The first stage, Spatial Signal Prompting, generates a spatially aware visual prompt by leveraging the rich internal representations of a pre-trained multi-modal model. This prompt, combined with the original text, forms a joint condition that is then injected into a frozen video DiT backbone via our lightweight and parameter-efficient SSG-Adapter. This unique design, featuring a dual-branch attention mechanism, allows the model to simultaneously harness its powerful generative priors while being precisely steered by external spatial signals. Extensive experiments demonstrate that SSG-DiT achieves state-of-the-art performance, outperforming existing models on multiple key metrics in the VBench benchmark, particularly in spatial relationship control and overall consistency.

Method: Integrates ViT with proximal tools where ViT constructs the tangent bundle of the manifold through self-attention (each head = tangent space), and proximal iterations define sections and project data from tangent spaces to base space for global feature alignment.

Result: Experimental results confirm the proposed method outperforms traditional ViT in classification accuracy and data distribution.

Conclusion: The framework successfully enhances feature representation and classification performance by enabling unified geometric optimization through ViT-proximal integration.

Abstract: The Vision Transformer (ViT) architecture has become widely recognized in computer vision, leveraging its self-attention mechanism to achieve remarkable success across various tasks. Despite its strengths, ViT’s optimization remains confined to modeling local relationships within individual images, limiting its ability to capture the global geometric relationships between data points. To address this limitation, this paper proposes a novel framework that integrates ViT with the proximal tools, enabling a unified geometric optimization approach to enhance feature representation and classification performance. In this framework, ViT constructs the tangent bundle of the manifold through its self-attention mechanism, where each attention head corresponds to a tangent space, offering geometric representations from diverse local perspectives. Proximal iterations are then introduced to define sections within the tangent bundle and project data from tangent spaces onto the base space, achieving global feature alignment and optimization. Experimental results confirm that the proposed method outperforms traditional ViT in terms of classification accuracy and data distribution.

Method: Integrates learnable proxies with the decidability index (d’) statistical framework, estimating genuine and impostor distributions through proxies to optimize embedding separability efficiently.

Result: Achieves performance comparable to state-of-the-art methods across fine-grained classification and face verification tasks while being computationally efficient.

Conclusion: PD-Loss provides a scalable, distribution-aware approach to deep metric learning that combines computational efficiency with principled separability optimization, offering new perspectives for embedding optimization.

Abstract: Deep Metric Learning (DML) aims to learn embedding functions that map semantically similar inputs to proximate points in a metric space while separating dissimilar ones. Existing methods, such as pairwise losses, are hindered by complex sampling requirements and slow convergence. In contrast, proxy-based losses, despite their improved scalability, often fail to optimize global distribution properties. The Decidability-based Loss (D-Loss) addresses this by targeting the decidability index (d’) to enhance distribution separability, but its reliance on large mini-batches imposes significant computational constraints. We introduce Proxy-Decidability Loss (PD-Loss), a novel objective that integrates learnable proxies with the statistical framework of d’ to optimize embedding spaces efficiently. By estimating genuine and impostor distributions through proxies, PD-Loss combines the computational efficiency of proxy-based methods with the principled separability of D-Loss, offering a scalable approach to distribution-aware DML. Experiments across various tasks, including fine-grained classification and face verification, demonstrate that PD-Loss achieves performance comparable to that of state-of-the-art methods while introducing a new perspective on embedding optimization, with potential for broader applications.

Method: A multimodal language model generates task-relevant bounding boxes and positive points from vision-language instructions. These are passed to a pre-trained segmentation model as prompts. The system is optimized purely with reinforcement learning (GRPO) using format and accuracy rewards on boxes/points, without mask supervision.

Result: 4% improvement on in-domain benchmarks and up to 54% improvement on out-of-domain benchmarks compared to state-of-the-art methods. Demonstrates robust generalization capabilities.

Conclusion: Complex geospatial segmentation behaviors can be effectively learned through reinforcement learning from weak spatial cues, eliminating the need for expensive mask supervision while achieving superior performance.

Abstract: Geospatial pixel reasoning is a nascent remote-sensing task that aims to generate segmentation masks directly from natural-language instructions. Prevailing MLLM-based systems co-train a language model and a mask decoder with dense pixel supervision, which is expensive and often weak on out-of-domain (OOD) data. We introduce GRASP, a structured policy-learning framework. In our design, a multimodal large language model first emits task-relevant bounding boxes and positive points from a vision-language instruction. These outputs are then passed to a pre-trained segmentation model, which consumes them as prompts to generate the final mask. Instead of supervised fine-tuning, we optimize the system purely with reinforcement learning: the model is trained solely with GRPO, guided by format rewards and accuracy rewards computed on boxes and points (no mask supervision). This leverages strong priors in foundation models, minimizes trainable parameters, and enables learning from inexpensive annotations. We additionally curate GRASP-1k, which contains reasoning-intensive queries, detailed reasoning traces, and fine-grained segmentation annotations. Evaluations on both in-domain and out-of-domain test sets show state-of-the-art results: about 4% improvement in-domain and up to 54% on OOD benchmarks. The experiment results evidence our model’s robust generalization and demonstrate that complex geospatial segmentation behaviors can be learned via RL from weak spatial cues. Code and the dataset will be released open-source.

Method: Created SugarcaneLD-BD dataset with 638 curated images across 5 classes, combined with additional datasets. Developed SugarcaneShuffleNet lightweight model optimized for on-device use, compared against other CNNs via transfer learning and Bayesian optimization. Integrated into SugarcaneAI PWA with Grad-CAM explanations.

Result: SugarcaneShuffleNet achieved 98.02% accuracy, 0.98 F1-score, 4.14ms inference time per image, with only 9.26MB model size. Outperformed other lightweight models in efficiency while maintaining comparable accuracy.

Conclusion: The proposed solution provides a practical, efficient tool for sugarcane disease classification in resource-constrained environments, offering both high accuracy and interpretability for field deployment.

Abstract: Despite progress in AI-based plant diagnostics, sugarcane farmers in low-resource regions remain vulnerable to leaf diseases due to the lack of scalable, efficient, and interpretable tools. Many deep learning models fail to generalize under real-world conditions and require substantial computational resources, limiting their use in resource-constrained regions. In this paper, we present SugarcaneLD-BD, a curated dataset for sugarcane leaf-disease classification; SugarcaneShuffleNet, an optimized lightweight model for rapid on-device diagnosis; and SugarcaneAI, a Progressive Web Application for field deployment. SugarcaneLD-BD contains 638 curated images across five classes, including four major sugarcane diseases, collected in Bangladesh under diverse field conditions and verified by expert pathologists. To enhance diversity, we combined SugarcaneLD-BD with two additional datasets, yielding a larger and more representative corpus. Our optimized model, SugarcaneShuffleNet, offers the best trade-off between speed and accuracy for real-time, on-device diagnosis. This 9.26 MB model achieved 98.02% accuracy, an F1-score of 0.98, and an average inference time of 4.14 ms per image. For comparison, we fine-tuned five other lightweight convolutional neural networks: MnasNet, EdgeNeXt, EfficientNet-Lite, MobileNet, and SqueezeNet via transfer learning and Bayesian optimization. MnasNet and EdgeNeXt achieved comparable accuracy to SugarcaneShuffleNet, but required significantly more parameters, memory, and computation, limiting their suitability for low-resource deployment. We integrate SugarcaneShuffleNet into SugarcaneAI, delivering Grad-CAM-based explanations in the field. Together, these contributions offer a diverse benchmark, efficient models for low-resource environments, and a practical tool for sugarcane disease classification. It spans varied lighting, backgrounds and devices used on-farm

Method: Created through a two-stage pipeline: (1) template-based QA synthesis from image metadata and (2) multi-stage linguistic re-engineering, with iterative expert review for scientific accuracy.

Result: A dataset of 193,609 high-quality QA pairs organized into 3 cognitive complexity levels and 9 categories, spanning 14 crop species and 38 disease conditions.

Conclusion: Provides a publicly available, expert-verified database to enhance diagnostic accuracy for plant disease identification and advance agricultural research.

Abstract: PlantVillageVQA is a large-scale visual question answering (VQA) dataset derived from the widely used PlantVillage image corpus. It was designed to advance the development and evaluation of vision-language models for agricultural decision-making and analysis. The PlantVillageVQA dataset comprises 193,609 high-quality question-answer (QA) pairs grounded over 55,448 images spanning 14 crop species and 38 disease conditions. Questions are organised into 3 levels of cognitive complexity and 9 distinct categories. Each question category was phrased manually following expert guidance and generated via an automated two-stage pipeline: (1) template-based QA synthesis from image metadata and (2) multi-stage linguistic re-engineering. The dataset was iteratively reviewed by domain experts for scientific accuracy and relevancy. The final dataset was evaluated using three state-of-the-art models for quality assessment. Our objective remains to provide a publicly available, standardised and expert-verified database to enhance diagnostic accuracy for plant disease identifications and advance scientific research in the agricultural domain. Our dataset will be open-sourced at https://huggingface.co/datasets/SyedNazmusSakib/PlantVillageVQA.

Method: Developed CE-RS-SBCIT framework with four innovations: 1) Smoothing and Boundary-based CNN-integrated Transformer (SBCIT), 2) Tailored residual and spatial learning CNNs, 3) Channel enhancement strategy, 4) Novel spatial attention mechanism. Uses stem convolution, contextual interaction transformer blocks, and auxiliary transfer-learned feature maps.

Result: Achieved 98.30% accuracy, 98.08% sensitivity, 98.25% F1-score, and 98.43% precision on challenging MRI datasets from Kaggle and Figshare covering glioma, meningioma, pituitary tumors, and healthy controls.

Conclusion: The proposed hybrid framework effectively addresses limitations of conventional methods by integrating local fine-grained and global contextual cues, demonstrating superior performance in brain tumor classification from MRI data.

Abstract: Brain tumors remain among the most lethal human diseases, where early detection and accurate classification are critical for effective diagnosis and treatment planning. Although deep learning-based computer-aided diagnostic (CADx) systems have shown remarkable progress. However, conventional convolutional neural networks (CNNs) and Transformers face persistent challenges, including high computational cost, sensitivity to minor contrast variations, structural heterogeneity, and texture inconsistencies in MRI data. Therefore, a novel hybrid framework, CE-RS-SBCIT, is introduced, integrating residual and spatial learning-based CNNs with transformer-driven modules. The proposed framework exploits local fine-grained and global contextual cues through four core innovations: (i) a smoothing and boundary-based CNN-integrated Transformer (SBCIT), (ii) tailored residual and spatial learning CNNs, (iii) a channel enhancement (CE) strategy, and (iv) a novel spatial attention mechanism. The developed SBCIT employs stem convolution and contextual interaction transformer blocks with systematic smoothing and boundary operations, enabling efficient global feature modeling. Moreover, Residual and spatial CNNs, enhanced by auxiliary transfer-learned feature maps, enrich the representation space, while the CE module amplifies discriminative channels and mitigates redundancy. Furthermore, the spatial attention mechanism selectively emphasizes subtle contrast and textural variations across tumor classes. Extensive evaluation on challenging MRI datasets from Kaggle and Figshare, encompassing glioma, meningioma, pituitary tumors, and healthy controls, demonstrates superior performance, achieving 98.30% accuracy, 98.08% sensitivity, 98.25% F1-score, and 98.43% precision.

Method: Integrated system combining aerial drone footage, Video Restoration Transformer (VRT) for super-resolution, and Gemma3:27b VLM for damage identification and classification into four categories with risk levels.

Result: Achieved 84.5% classification accuracy using data from 2023 Turkey earthquakes and 2013 Moore Tornado, providing highly accurate automated damage assessment.

Conclusion: The framework enables non-technical users to perform preliminary analyses, improving responsiveness and efficiency of disaster management efforts with cost-effective automation.

Abstract: Natural disasters pose significant challenges to timely and accurate damage assessment due to their sudden onset and the extensive areas they affect. Traditional assessment methods are often labor-intensive, costly, and hazardous to personnel, making them impractical for rapid response, especially in resource-limited settings. This study proposes a novel, cost-effective framework that leverages aerial drone footage, an advanced AI-based video super-resolution model, Video Restoration Transformer (VRT), and Gemma3:27b, a 27 billion parameter Visual Language Model (VLM). This integrated system is designed to improve low-resolution disaster footage, identify structural damage, and classify buildings into four damage categories, ranging from no/slight damage to total destruction, along with associated risk levels. The methodology was validated using pre- and post-event drone imagery from the 2023 Turkey earthquakes (courtesy of The Guardian) and satellite data from the 2013 Moore Tornado (xBD dataset). The framework achieved a classification accuracy of 84.5%, demonstrating its ability to provide highly accurate results. Furthermore, the system’s accessibility allows non-technical users to perform preliminary analyses, thereby improving the responsiveness and efficiency of disaster management efforts.

Method: Integrates GPT APIs with Computer Vision techniques, using annotated frames instead of raw coordinates to address GPT-4o’s spatial weakness. Includes video pre-processing and real-time interaction architecture.

Result: Significantly improves accuracy in localizing and interpreting video content compared to using raw coordinate data with GPT-4o.

Conclusion: Untwist transforms passive video consumption into an interactive, AI-driven learning experience with potential to enhance engagement and comprehension through region-specific multimodal interactions.

Abstract: Traditional video-based learning remains passive, offering limited opportunities for users to engage dynamically with content. While current AI-powered tools offer transcription and summarization, they lack real-time, region-specific interaction capabilities. This paper introduces Untwist, an AI-driven system that enables interactive video learning by allowing users to ask questions about the entire video or specific regions using a bounding box, receiving context-aware, multimodal responses. By integrating GPT APIs with Computer Vision techniques, Untwist extracts, processes, and structures video content to enhance comprehension. Our approach addresses GPT-4o spatial weakness by leveraging annotated frames instead of raw coordinate data, significantly improving accuracy in localizing and interpreting video content. This paper describes the system architecture, including video pre-processing and real-time interaction, and outlines how Untwist can transform passive video consumption into an interactive, AI-driven learning experience with the potential to enhance engagement and comprehension.

Method: Employed implicit neural representation to combine T1w and T2w MRI resolution advantages, upsampling MTL subregion atlas from anisotropic to isotropic space, then developed an isotropic segmentation model.

Result: Isotropic model showed higher significance in distinguishing mild cognitive impairment from cognitively unimpaired participants, with greater biomarker stability in longitudinal analysis of CU participants.

Conclusion: Improved AD imaging biomarker accuracy without increasing annotation workload, enabling more precise quantification of AD-brain atrophy relationships and better disease tracking measures.

Abstract: Imaging biomarkers in magnetic resonance imaging (MRI) are important tools for diagnosing and tracking Alzheimer’s disease (AD). As medial temporal lobe (MTL) is the earliest region to show AD-related hallmarks, brain atrophy caused by AD can first be observed in the MTL. Accurate segmentation of MTL subregions and extraction of imaging biomarkers from them are important. However, due to imaging limitations, the resolution of T2-weighted (T2w) MRI is anisotropic, which makes it difficult to accurately extract the thickness of cortical subregions in the MTL. In this study, we used an implicit neural representation method to combine the resolution advantages of T1-weighted and T2w MRI to accurately upsample an MTL subregion atlas set from anisotropic space to isotropic space, establishing a multi-modality, high-resolution atlas set. Based on this atlas, we developed an isotropic MTL subregion segmentation model. In an independent test set, the cortical subregion thickness extracted using this isotropic model showed higher significance than an anisotropic method in distinguishing between participants with mild cognitive impairment and cognitively unimpaired (CU) participants. In longitudinal analysis, the biomarkers extracted using isotropic method showed greater stability in CU participants. This study improved the accuracy of AD imaging biomarkers without increasing the amount of atlas annotation work, which may help to more accurately quantify the relationship between AD and brain atrophy and provide more accurate measures for disease tracking.

Method: Uses DROID-SLAM, AnyCam, and Monst3r methods combined with preprocessing techniques including masking, temporal chunking, and resolution scaling to handle dynamic motion and computational constraints.

Result: Successfully partially recovers track and vehicle trajectories in complex Formula 1 environments using footage from the 2023 Monaco Grand Prix.

Conclusion: Demonstrates feasibility of using onboard video for scalable 4D reconstruction in real-world racing settings, with potential applications in motorsports analysis and simulation.

Abstract: We introduce VROOM, a system for reconstructing 3D models of Formula 1 circuits using only onboard camera footage from racecars. Leveraging video data from the 2023 Monaco Grand Prix, we address video challenges such as high-speed motion and sharp cuts in camera frames. Our pipeline analyzes different methods such as DROID-SLAM, AnyCam, and Monst3r and combines preprocessing techniques such as different methods of masking, temporal chunking, and resolution scaling to account for dynamic motion and computational constraints. We show that Vroom is able to partially recover track and vehicle trajectories in complex environments. These findings indicate the feasibility of using onboard video for scalable 4D reconstruction in real-world settings. The project page can be found at https://varun-bharadwaj.github.io/vroom, and our code is available at https://github.com/yajatyadav/vroom.

Method: Two-phase approach: 1) Adversarial Prompt Mining - using incorrect labels to identify background features likely to be misclassified; 2) Adversarial Sample Rectification - integrating these samples via online global prototype built from current and historical data.

Result: Significant performance improvements demonstrated on ConvNet, Transformer, and SAM-based baselines. Method shows generalizability to other multi-class weakly-supervised dense prediction tasks.

Conclusion: AdvCP effectively addresses co-occurring noise problem in WSCD, can be seamlessly integrated into existing methods without additional inference overhead, and demonstrates broad applicability across different architectures and scenarios.

Abstract: Weakly-Supervised Change Detection (WSCD) aims to distinguish specific object changes (e.g., objects appearing or disappearing) from background variations (e.g., environmental changes due to light, weather, or seasonal shifts) in paired satellite images, relying only on paired image (i.e., image-level) classification labels. This technique significantly reduces the need for dense annotations required in fully-supervised change detection. However, as image-level supervision only indicates whether objects have changed in a scene, WSCD methods often misclassify background variations as object changes, especially in complex remote-sensing scenarios. In this work, we propose an Adversarial Class Prompting (AdvCP) method to address this co-occurring noise problem, including two phases: a) Adversarial Prompt Mining: After each training iteration, we introduce adversarial prompting perturbations, using incorrect one-hot image-level labels to activate erroneous feature mappings. This process reveals co-occurring adversarial samples under weak supervision, namely background variation features that are likely to be misclassified as object changes. b) Adversarial Sample Rectification: We integrate these adversarially prompt-activated pixel samples into training by constructing an online global prototype. This prototype is built from an exponentially weighted moving average of the current batch and all historical training data. Our AdvCP can be seamlessly integrated into current WSCD methods without adding additional inference cost. Experiments on ConvNet, Transformer, and Segment Anything Model (SAM)-based baselines demonstrate significant performance enhancements. Furthermore, we demonstrate the generalizability of AdvCP to other multi-class weakly-supervised dense prediction scenarios. Code is available at https://github.com/zhenghuizhao/AdvCP

Method: Multimodal pseudo-labeling that collects documents with multiple images and captions, ranks images and captions separately using gold summaries, and selects images only when both image and caption rank first. Documents with direct image references are removed.

Result: The proposed method constructs more precise datasets and generates higher quality images compared to text-only and image-only pseudo-labeling approaches.

Conclusion: Multimodal pseudo-labeling considering both images and captions jointly outperforms separate consideration methods, providing an effective low-cost solution for cover image generation tasks.

Abstract: In this study, we introduce a novel cover image generation task that produces both a concise summary and a visually corresponding image from a given text-only document. Because no existing datasets are available for this task, we propose a multimodal pseudo-labeling method to construct high-quality datasets at low cost. We first collect documents that contain multiple images with their captions, and their summaries by excluding factually inconsistent instances. Our approach selects one image from the multiple images accompanying the documents. Using the gold summary, we independently rank both the images and their captions. Then, we annotate a pseudo-label for an image when both the image and its corresponding caption are ranked first in their respective rankings. Finally, we remove documents that contain direct image references within texts. Experimental results demonstrate that the proposed multimodal pseudo-labeling method constructs more precise datasets and generates higher quality images than text- and image-only pseudo-labeling methods, which consider captions and images separately. We release our code at: https://github.com/HyeyeeonKim/MMCIG

Method: Proposes MKD with two teacher and one student models to extract modality-specific and modality-general features. Uses Similarity-preserving Knowledge Distillation (SKD) to maintain structural relationships and Collaborative Learning for Online Distillation (CLOD) for mutual learning between models.

Result: Experiments on TCGA-BRCA and in-house QHSU datasets demonstrate superior performance in IHC biomarker prediction using uni-modal data compared to existing methods.

Conclusion: The proposed online distillation approach effectively enhances IHC biomarker prediction by leveraging multi-modal training data while maintaining the capability for uni-modal inference, addressing practical limitations in multi-modal data acquisition.

Abstract: Immunohistochemical (IHC) biomarker prediction benefits from multi-modal data fusion analysis. However, the simultaneous acquisition of multi-modal data, such as genomic and pathological information, is often challenging due to cost or technical limitations. To address this challenge, we propose an online distillation approach based on Multi-modal Knowledge Decomposition (MKD) to enhance IHC biomarker prediction in haematoxylin and eosin (H&E) stained histopathology images. This method leverages paired genomic-pathology data during training while enabling inference using either pathology slides alone or both modalities. Two teacher and one student models are developed to extract modality-specific and modality-general features by minimizing the MKD loss. To maintain the internal structural relationships between samples, Similarity-preserving Knowledge Distillation (SKD) is applied. Additionally, Collaborative Learning for Online Distillation (CLOD) facilitates mutual learning between teacher and student models, encouraging diverse and complementary learning dynamics. Experiments on the TCGA-BRCA and in-house QHSU datasets demonstrate that our approach achieves superior performance in IHC biomarker prediction using uni-modal data. Our code is available at https://github.com/qiyuanzz/MICCAI2025_MKD.

Method: Combines robust preprocessing with CNNs, inserts multi-head self-attention block into VGG19 backbone to model long-range dependencies, and uses Focal Loss to mitigate class imbalance.

Result: Enhanced VGG19+MHSA with Focal Loss achieves 99.25% accuracy, surpassing ResNet101 baseline (98.62%), indicating superior discriminative representations of leukemic cell morphology.

Conclusion: Attention-augmented CNNs with targeted loss optimization and preprocessing offer highly accurate and computationally efficient tool for automated ALL recognition, potentially accelerating diagnostic workflows in clinical settings.

Abstract: Acute lymphoblastic leukemia (ALL) is a prevalent hematological malignancy in both pediatric and adult populations. Early and accurate detection with precise subtyping is essential for guiding therapy. Conventional workflows are complex, time-consuming, and prone to human error. We present a deep learning framework for automated ALL diagnosis from bone marrow smear images. The method combines a robust preprocessing pipeline with convolutional neural networks (CNNs) to standardize image quality and improve inference efficiency. As a key design, we insert a multi-head self-attention (MHSA) block into a VGG19 backbone to model long-range dependencies and contextual relationships among cellular features. To mitigate class imbalance, we train with Focal Loss. Across evaluated architectures, the enhanced VGG19+MHSA trained with Focal Loss achieves 99.25% accuracy, surpassing a strong ResNet101 baseline (98.62%). These results indicate that attention-augmented CNNs, coupled with targeted loss optimization and preprocessing, yield more discriminative representations of leukemic cell morphology. Our approach offers a highly accurate and computationally efficient tool for automated ALL recognition and subtyping, with potential to accelerate diagnostic workflows and support reliable decision-making in clinical settings.

Method: FVP frames visual pre-training as a next-point-cloud-prediction problem using diffusion models, pre-trained on large public datasets to improve 3D representations.

Result: FVP boosts 3D Diffusion Policy’s average success rate by 28% across 12 real-world manipulation tasks and achieves state-of-the-art performance in imitation learning.

Conclusion: FVP provides an effective alternative to direct 3D representation learning, enhancing various point cloud encoders and robotic models including larger vision-language-action systems.

Abstract: General visual representations learned from web-scale datasets for robotics have achieved great success in recent years, enabling data-efficient robot learning on manipulation tasks; yet these pre-trained representations are mostly on 2D images, neglecting the inherent 3D nature of the world. However, due to the scarcity of large-scale 3D data, it is still hard to extract a universal 3D representation from web datasets. Instead, we are seeking a general visual pre-training framework that could improve all 3D representations as an alternative. Our framework, called FVP, is a novel 4D Visual Pre-training framework for real-world robot learning. FVP frames the visual pre-training objective as a next-point-cloud-prediction problem, models the prediction model as a diffusion model, and pre-trains the model on the larger public datasets directly. Across twelve real-world manipulation tasks, FVP boosts the average success rate of 3D Diffusion Policy (DP3) for these tasks by 28%. The FVP pre-trained DP3 achieves state-of-the-art performance across imitation learning methods. Moreover, the efficacy of FVP adapts across various point cloud encoders and datasets. Finally, we apply FVP to the RDT-1B, a larger Vision-Language-Action robotic model, enhancing its performance on various robot tasks. Our project page is available at: https://4d- visual-pretraining.github.io/.

Method: Proposes Perspective Encoding (PE) to encode camera intrinsics of cropped images, and Perspective Rotation (PR) to center human subjects and reduce perspective distortions. Combines both in PersPose framework.

Result: Achieves SOTA performance: MPJPE of 60.1 mm on 3DPW (7.54% improvement), and strong results on MPI-INF-3DHP and Human3.6M datasets.

Conclusion: Incorporating camera intrinsics and addressing perspective distortion through PE and PR significantly improves 3D human pose estimation accuracy, demonstrating the importance of considering geometric relationships in cropped images.

Abstract: Monocular 3D human pose estimation (HPE) methods estimate the 3D positions of joints from individual images. Existing 3D HPE approaches often use the cropped image alone as input for their models. However, the relative depths of joints cannot be accurately estimated from cropped images without the corresponding camera intrinsics, which determine the perspective relationship between 3D objects and the cropped images. In this work, we introduce Perspective Encoding (PE) to encode the camera intrinsics of the cropped images. Moreover, since the human subject can appear anywhere within the original image, the perspective relationship between the 3D scene and the cropped image differs significantly, which complicates model fitting. Additionally, the further the human subject deviates from the image center, the greater the perspective distortions in the cropped image. To address these issues, we propose Perspective Rotation (PR), a transformation applied to the original image that centers the human subject, thereby reducing perspective distortions and alleviating the difficulty of model fitting. By incorporating PE and PR, we propose a novel 3D HPE framework, PersPose. Experimental results demonstrate that PersPose achieves state-of-the-art (SOTA) performance on the 3DPW, MPIINF-3DHP, and Human3.6M datasets. For example, on the in-the-wild dataset 3DPW, PersPose achieves an MPJPE of 60.1 mm, 7.54% lower than the previous SOTA approach. Code is available at: https://github.com/ KenAdamsJoseph/PersPose.

Method: Proposes CoViPAL with a Plug-and-Play Pruning Module (PPM) that predicts and removes redundant vision tokens before processing by LVLM. The PPM is lightweight, model-agnostic, and operates independently of LVLM architecture.

Result: Extensive experiments show CoViPAL outperforms training-free pruning methods under equal token budgets and surpasses training-based methods with comparable supervision. It improves inference efficiency without compromising accuracy.

Conclusion: CoViPAL provides a scalable and efficient solution for improving inference efficiency in LVLMs by effectively pruning redundant visual tokens while maintaining performance across multiple benchmarks.

Abstract: Large Vision-Language Models (LVLMs) process multimodal inputs consisting of text tokens and vision tokens extracted from images or videos. Due to the rich visual information, a single image can generate thousands of vision tokens, leading to high computational costs during the prefilling stage and significant memory overhead during decoding. Existing methods attempt to prune redundant vision tokens, revealing substantial redundancy in visual representations. However, these methods often struggle in shallow layers due to the lack of sufficient contextual information. We argue that many visual tokens are inherently redundant even in shallow layers and can be safely and effectively pruned with appropriate contextual signals. In this work, we propose CoViPAL, a layer-wise contextualized visual token pruning method that employs a Plug-and-Play Pruning Module (PPM) to predict and remove redundant vision tokens before they are processed by the LVLM. The PPM is lightweight, model-agnostic, and operates independently of the LVLM architecture, ensuring seamless integration with various models. Extensive experiments on multiple benchmarks demonstrate that CoViPAL outperforms training-free pruning methods under equal token budgets and surpasses training-based methods with comparable supervision. CoViPAL offers a scalable and efficient solution to improve inference efficiency in LVLMs without compromising accuracy.

Method: Proposes Adversarial Interference Simulation (AIS) - a training paradigm that simulates MEA scenarios during fine-tuning with a resilience-driven loss function to learn sparse and stable watermark representations without modifying network architecture.

Result: Extensive experiments show that the plug-and-play AIS training paradigm significantly enhances the robustness of various existing methods against Multi-Embedding Attacks, maintaining original watermark extraction ability even after second embedding.

Conclusion: The AIS method effectively addresses the MEA vulnerability in deepfake proactive forensics, providing a practical solution for real-world scenarios where multiple watermark embedding occurs, thereby strengthening digital media security against deepfake threats.

Abstract: With the rapid evolution of deepfake technologies and the wide dissemination of digital media, personal privacy is facing increasingly serious security threats. Deepfake proactive forensics, which involves embedding imperceptible watermarks to enable reliable source tracking, serves as a crucial defense against these threats. Although existing methods show strong forensic ability, they rely on an idealized assumption of single watermark embedding, which proves impractical in real-world scenarios. In this paper, we formally define and demonstrate the existence of Multi-Embedding Attacks (MEA) for the first time. When a previously protected image undergoes additional rounds of watermark embedding, the original forensic watermark can be destroyed or removed, rendering the entire proactive forensic mechanism ineffective. To address this vulnerability, we propose a general training paradigm named Adversarial Interference Simulation (AIS). Rather than modifying the network architecture, AIS explicitly simulates MEA scenarios during fine-tuning and introduces a resilience-driven loss function to enforce the learning of sparse and stable watermark representations. Our method enables the model to maintain the ability to extract the original watermark correctly even after a second embedding. Extensive experiments demonstrate that our plug-and-play AIS training paradigm significantly enhances the robustness of various existing methods against MEA.

Method: Proposed ICPNet with three key modules: multi-scale feature projection (MFP) for multi-scale representations, feature interaction attention module (FIAM) for feedforward-feedback interaction, and edge fusion module (EFM) with shape constraints inspired by human shape bias.

Result: ICPNet significantly outperforms state-of-the-art models in sensitivity to abutting grating illusory contours, showing notable improvements in top-1 accuracy across various test subsets including AG-MNIST and newly constructed AG-Fashion-MNIST datasets.

Conclusion: This work represents a step toward human-level intelligence for DNN-based models by better aligning machine perception with human visual cognition through biologically-inspired architecture design.

Abstract: Higher levels of machine intelligence demand alignment with human perception and cognition. Deep neural networks (DNN) dominated machine intelligence have demonstrated exceptional performance across various real-world tasks. Nevertheless, recent evidence suggests that DNNs fail to perceive illusory contours like the abutting grating, a discrepancy that misaligns with human perception patterns. Departing from previous works, we propose a novel deep network called illusory contour perception network (ICPNet) inspired by the circuits of the visual cortex. In ICPNet, a multi-scale feature projection (MFP) module is designed to extract multi-scale representations. To boost the interaction between feedforward and feedback features, a feature interaction attention module (FIAM) is introduced. Moreover, drawing inspiration from the shape bias observed in human perception, an edge detection task conducted via the edge fusion module (EFM) injects shape constraints that guide the network to concentrate on the foreground. We assess our method on the existing AG-MNIST test set and the AG-Fashion-MNIST test sets constructed by this work. Comprehensive experimental results reveal that ICPNet is significantly more sensitive to abutting grating illusory contours than state-of-the-art models, with notable improvements in top-1 accuracy across various subsets. This work is expected to make a step towards human-level intelligence for DNN-based models.

Method: Uses depth-guided vision-language grounding to separate cabin/road scenes, two SLAM branches for motion tracking in each context, and GPT-based module for generating context-aware overlays (dashboard cues, hazard alerts). Introduces EgoSLAM-Drive dataset for evaluation.

Result: Achieves robust spatial alignment and perceptually coherent AR rendering across varied environments. Enhances perceived scene understanding, overlay relevance, and driver ease. Outperforms existing systems in dynamic driving scenarios.

Conclusion: SEER-VAR provides an effective foundation for LLM-based AR recommendation in egocentric driving, addressing the lack of comparable systems. The framework and dataset will be open-sourced to support future research.

Abstract: We present SEER-VAR, a novel framework for egocentric vehicle-based augmented reality (AR) that unifies semantic decomposition, Context-Aware SLAM Branches (CASB), and LLM-driven recommendation. Unlike existing systems that assume static or single-view settings, SEER-VAR dynamically separates cabin and road scenes via depth-guided vision-language grounding. Two SLAM branches track egocentric motion in each context, while a GPT-based module generates context-aware overlays such as dashboard cues and hazard alerts. To support evaluation, we introduce EgoSLAM-Drive, a real-world dataset featuring synchronized egocentric views, 6DoF ground-truth poses, and AR annotations across diverse driving scenarios. Experiments demonstrate that SEER-VAR achieves robust spatial alignment and perceptually coherent AR rendering across varied environments. As one of the first to explore LLM-based AR recommendation in egocentric driving, we address the lack of comparable systems through structured prompting and detailed user studies. Results show that SEER-VAR enhances perceived scene understanding, overlay relevance, and driver ease, providing an effective foundation for future research in this direction. Code and dataset will be made open source.

Method: ResLink integrates novel area attention mechanisms with residual connections in a multi-stage convolutional pipeline. It incorporates dropout, regularization, downsampling, and attention-based refinement for classification, trained on a balanced dataset.

Result: The model achieves 95% accuracy in brain tumor classification and demonstrates strong generalizability across different cases.

Conclusion: ResLink shows significant potential for improving brain tumor classification and offers a robust, efficient technique for medical imaging applications, particularly for spatially rich image classification tasks.

Abstract: Brain tumors show significant health challenges due to their potential to cause critical neurological functions. Early and accurate diagnosis is crucial for effective treatment. In this research, we propose ResLink, a novel deep learning architecture for brain tumor classification using CT scan images. ResLink integrates novel area attention mechanisms with residual connections to enhance feature learning and spatial understanding for spatially rich image classification tasks. The model employs a multi-stage convolutional pipeline, incorporating dropout, regularization, and downsampling, followed by a final attention-based refinement for classification. Trained on a balanced dataset, ResLink achieves a high accuracy of 95% and demonstrates strong generalizability. This research demonstrates the potential of ResLink in improving brain tumor classification, offering a robust and efficient technique for medical imaging applications.

Method: Freezes backbone and base head trained on reference material, learns material-specific prompts, embeddings, and delta heads for new materials. Uses prompt pool with cosine-similarity gate to modulate features and incorporates memory replay with knowledge distillation.

Result: Achieves competitive accuracy with significantly lower forgetting compared to naive fine-tuning and prompt-based baselines.

Conclusion: CLIFF represents the first systematic study of continual learning for 2D materials, enabling effective differentiation between materials and their properties while maintaining performance across diverse material types.

Abstract: Identifying quantum flakes is crucial for scalable quantum hardware; however, automated layer classification from optical microscopy remains challenging due to substantial appearance shifts across different materials. In this paper, we propose a new Continual-Learning Framework for Flake Layer Classification (CLIFF). To our knowledge, this is the first systematic study of continual learning in the domain of two-dimensional (2D) materials. Our method enables the model to differentiate between materials and their physical and optical properties by freezing a backbone and base head trained on a reference material. For each new material, it learns a material-specific prompt, embedding, and a delta head. A prompt pool and a cosine-similarity gate modulate features and compute material-specific corrections. Additionally, we incorporate memory replay with knowledge distillation. CLIFF achieves competitive accuracy with significantly lower forgetting than naive fine-tuning and a prompt-based baseline.

Method: Proposes Adaptive Guidance Adversarial Training (AdaGAT) with two separate loss functions that allow the guide model to actively participate in backpropagation to achieve optimal state for robustness transfer.

Result: Extensive experiments on CIFAR-10, CIFAR-100, and TinyImageNet show enhanced target model robustness across various adversarial attacks compared to baseline models when guide model is adjusted within optimal accuracy range.

Conclusion: Dynamic adjustment of guide model training state significantly improves robustness transfer in adversarial distillation, making AdaGAT an effective method for creating robust lightweight DNN models.

Abstract: Adversarial distillation (AD) is a knowledge distillation technique that facilitates the transfer of robustness from teacher deep neural network (DNN) models to lightweight target (student) DNN models, enabling the target models to perform better than only training the student model independently. Some previous works focus on using a small, learnable teacher (guide) model to improve the robustness of a student model. Since a learnable guide model starts learning from scratch, maintaining its optimal state for effective knowledge transfer during co-training is challenging. Therefore, we propose a novel Adaptive Guidance Adversarial Training (AdaGAT) method. Our method, AdaGAT, dynamically adjusts the training state of the guide model to install robustness to the target model. Specifically, we develop two separate loss functions as part of the AdaGAT method, allowing the guide model to participate more actively in backpropagation to achieve its optimal state. We evaluated our approach via extensive experiments on three datasets: CIFAR-10, CIFAR-100, and TinyImageNet, using the WideResNet-34-10 model as the target model. Our observations reveal that appropriately adjusting the guide model within a certain accuracy range enhances the target model’s robustness across various adversarial attacks compared to a variety of baseline models.

Method: Used transfer learning and self-supervised learning approaches on labeled and unlabeled CT scan datasets to develop deep-learning models for automated SMA measurement at the third lumbar vertebra.

Result: Model predicted SMA with average error of ±3 percentage points against manual measurements, achieving 93% dice similarity coefficient for segmentation masks.

Conclusion: The approach demonstrates a pathway to full automation of sarcopenia assessment, providing precise quantitative SMA measurement while addressing class imbalance and limited data issues.

Abstract: Sarcopenia is a progressive loss of muscle mass and function linked to poor surgical outcomes such as prolonged hospital stays, impaired mobility, and increased mortality. Although it can be assessed through cross-sectional imaging by measuring skeletal muscle area (SMA), the process is time-consuming and adds to clinical workloads, limiting timely detection and management; however, this process could become more efficient and scalable with the assistance of artificial intelligence applications. This paper presents high-quality three-dimensional cross-sectional computed tomography (CT) images of patients with sarcopenia collected at the Freeman Hospital, Newcastle upon Tyne Hospitals NHS Foundation Trust. Expert clinicians manually annotated the SMA at the third lumbar vertebra, generating precise segmentation masks. We develop deep-learning models to measure SMA in CT images and automate this task. Our methodology employed transfer learning and self-supervised learning approaches using labelled and unlabeled CT scan datasets. While we developed qualitative assessment models for detecting sarcopenia, we observed that the quantitative assessment of SMA is more precise and informative. This approach also mitigates the issue of class imbalance and limited data availability. Our model predicted the SMA, on average, with an error of +-3 percentage points against the manually measured SMA. The average dice similarity coefficient of the predicted masks was 93%. Our results, therefore, show a pathway to full automation of sarcopenia assessment and detection.

Method: Built on Quick-Tune hyperparameter optimization framework, QTT-SEG uses meta-learned cost and performance models to predict optimal configurations from over 200 million possibilities.

Result: QTT-SEG consistently improves SAM’s zero-shot performance and surpasses AutoGluon Multimodal on most binary tasks within 3 minutes, with gains on multiclass datasets too.

Conclusion: Meta-learning shows promise for automating model adaptation for specialized segmentation tasks, making fine-tuning efficient and accessible.

Abstract: Foundation models like SAM (Segment Anything Model) exhibit strong zero-shot image segmentation performance, but often fall short on domain-specific tasks. Fine-tuning these models typically requires significant manual effort and domain expertise. In this work, we introduce QTT-SEG, a meta-learning-driven approach for automating and accelerating the fine-tuning of SAM for image segmentation. Built on the Quick-Tune hyperparameter optimization framework, QTT-SEG predicts high-performing configurations using meta-learned cost and performance models, efficiently navigating a search space of over 200 million possibilities. We evaluate QTT-SEG on eight binary and five multiclass segmentation datasets under tight time constraints. Our results show that QTT-SEG consistently improves upon SAM’s zero-shot performance and surpasses AutoGluon Multimodal, a strong AutoML baseline, on most binary tasks within three minutes. On multiclass datasets, QTT-SEG delivers consistent gains as well. These findings highlight the promise of meta-learning in automating model adaptation for specialized segmentation tasks. Code available at: https://github.com/ds-brx/QTT-SEG/

Method: Systematic review of 260+ papers from top AI venues, formalizing core definitions, tracing five-stage paradigm shifts, cataloging benchmarks and metrics, and analyzing architectural designs and limitations.

Result: Identified key advantages of compositional approaches in cognitive alignment, semantic fidelity, robustness, interpretability, and data efficiency. Documented paradigm evolution and benchmark landscape.

Conclusion: Distilled key insights, identified open challenges (LLM reasoning limitations, hallucination, deductive bias, supervision scalability), and outlined future directions including world-model integration and human-AI collaboration.

Abstract: Compositional visual reasoning has emerged as a key research frontier in multimodal AI, aiming to endow machines with the human-like ability to decompose visual scenes, ground intermediate concepts, and perform multi-step logical inference. While early surveys focus on monolithic vision-language models or general multimodal reasoning, a dedicated synthesis of the rapidly expanding compositional visual reasoning literature is still missing. We fill this gap with a comprehensive survey spanning 2023 to 2025 that systematically reviews 260+ papers from top venues (CVPR, ICCV, NeurIPS, ICML, ACL, etc.). We first formalize core definitions and describe why compositional approaches offer advantages in cognitive alignment, semantic fidelity, robustness, interpretability, and data efficiency. Next, we trace a five-stage paradigm shift: from prompt-enhanced language-centric pipelines, through tool-enhanced LLMs and tool-enhanced VLMs, to recently minted chain-of-thought reasoning and unified agentic VLMs, highlighting their architectural designs, strengths, and limitations. We then catalog 60+ benchmarks and corresponding metrics that probe compositional visual reasoning along dimensions such as grounding accuracy, chain-of-thought faithfulness, and high-resolution perception. Drawing on these analyses, we distill key insights, identify open challenges (e.g., limitations of LLM-based reasoning, hallucination, a bias toward deductive reasoning, scalable supervision, tool integration, and benchmark limitations), and outline future directions, including world-model integration, human-AI collaborative reasoning, and richer evaluation protocols. By offering a unified taxonomy, historical roadmap, and critical outlook, this survey aims to serve as a foundational reference and inspire the next generation of compositional visual reasoning research.

Method: Two-stage strategy: (1) Dose-anatomy perception using DA-CLIP (contrastive language image pre-training) to learn continuous representations of dose variations and anatomical regions, (2) Adaptive denoising using DA-Diff diffusion model that integrates learned embeddings via novel dose and anatomy conditional block based on Mamba architecture.

Result: Extensive experiments on two public LDCT datasets with eight dose levels and three anatomical regions demonstrate superior denoising performance over state-of-the-art methods and remarkable generalization to unseen dose levels.

Conclusion: FoundDiff provides a unified and generalizable solution for low-dose CT denoising that effectively handles diverse clinical scenarios with varying dose levels and anatomical regions, outperforming existing specialized approaches.

Abstract: Low-dose computed tomography (CT) denoising is crucial for reduced radiation exposure while ensuring diagnostically acceptable image quality. Despite significant advancements driven by deep learning (DL) in recent years, existing DL-based methods, typically trained on a specific dose level and anatomical region, struggle to handle diverse noise characteristics and anatomical heterogeneity during varied scanning conditions, limiting their generalizability and robustness in clinical scenarios. In this paper, we propose FoundDiff, a foundational diffusion model for unified and generalizable LDCT denoising across various dose levels and anatomical regions. FoundDiff employs a two-stage strategy: (i) dose-anatomy perception and (ii) adaptive denoising. First, we develop a dose- and anatomy-aware contrastive language image pre-training model (DA-CLIP) to achieve robust dose and anatomy perception by leveraging specialized contrastive learning strategies to learn continuous representations that quantify ordinal dose variations and identify salient anatomical regions. Second, we design a dose- and anatomy-aware diffusion model (DA-Diff) to perform adaptive and generalizable denoising by synergistically integrating the learned dose and anatomy embeddings from DACLIP into diffusion process via a novel dose and anatomy conditional block (DACB) based on Mamba. Extensive experiments on two public LDCT datasets encompassing eight dose levels and three anatomical regions demonstrate superior denoising performance of FoundDiff over existing state-of-the-art methods and the remarkable generalization to unseen dose levels. The codes and models are available at https://github.com/hao1635/FoundDiff.

Method: Uses positional embedding transplant to guide diffusion models to replicate structural characteristics of reference objects, and Corner Centered Layout that concatenates reference and background images as input to FLUX.1-Fill model during progressive denoising.

Result: Extensive experiments demonstrate that PosBridge outperforms mainstream baselines in structural consistency, appearance fidelity, and computational efficiency.

Conclusion: PosBridge showcases practical value and potential for broad adoption as an efficient and flexible framework for inserting custom objects into images without requiring training.

Abstract: Localized subject-driven image editing aims to seamlessly integrate user-specified objects into target scenes. As generative models continue to scale, training becomes increasingly costly in terms of memory and computation, highlighting the need for training-free and scalable editing frameworks.To this end, we propose PosBridge an efficient and flexible framework for inserting custom objects. A key component of our method is positional embedding transplant, which guides the diffusion model to faithfully replicate the structural characteristics of reference objects.Meanwhile, we introduce the Corner Centered Layout, which concatenates reference images and the background image as input to the FLUX.1-Fill model. During progressive denoising, positional embedding transplant is applied to guide the noise distribution in the target region toward that of the reference object. In this way, Corner Centered Layout effectively directs the FLUX.1-Fill model to synthesize identity-consistent content at the desired location. Extensive experiments demonstrate that PosBridge outperforms mainstream baselines in structural consistency, appearance fidelity, and computational efficiency, showcasing its practical value and potential for broad adoption.

Method: ViT-Adapter baseline enhanced with: 1) SAPA dynamic upsampler for detail delineation, 2) semi-supervised guided distillation with stem-aware sample selection, 3) test-time scaling strategy to zoom and segment images twice.

Result: Achieved first place in the competition, outperforming second place by clear margins.

Conclusion: Simple but targeted improvements focusing on the specific challenges of wheat stem segmentation can lead to significant performance gains in specialized segmentation tasks.

Abstract: In this report, we present our solution during the participation of the MLCAS 2025 GWFSS Challenge. This challenge hosts a semantic segmentation competition specific to wheat plants, which requires to segment three wheat organs including the head, leaf, and stem, and another background class. In 2025, participating a segmentation competition is significantly different from that in previous years where many tricks can play important roles. Nowadays most segmentation tricks have been well integrated into existing codebases such that our naive ViT-Adapter baseline has already achieved sufficiently good performance. Hence, we believe the key to stand out among other competitors is to focus on the problem nature of wheat per se. By probing visualizations, we identify the key – the stem matters. In contrast to heads and leaves, stems exhibit fine structure and occupy only few pixels, which suffers from fragile predictions and class imbalance. Building on our baseline, we present three technical improvements tailored to stems: i) incorporating a dynamic upsampler SAPA used to enhance detail delineation; ii) leveraging semi-supervised guided distillation with stem-aware sample selection to mine the treasure beneath unlabeled data; and iii) applying a test-time scaling strategy to zoom in and segment twice the image. Despite being simple, the three improvements bring us to the first place of the competition, outperforming the second place by clear margins. Code and models will be released at https://github.com/tiny-smart/gwfss25.

Method: Uses Local Binary Patterns to extract facial texture features, applies localized perturbations to key texture regions with low perceptual saliency, and employs a dual-model attention strategy to optimize texture perturbations.

Result: Experiments on CelebA-HQ and LFW datasets show promising performance in distorting Deepfake generation and producing visible defects under multiple attack models.

Conclusion: The approach provides an efficient and scalable solution for proactive Deepfake detection by leveraging texture-guided perturbations that are imperceptible to humans but disruptive to Deepfake algorithms.

Abstract: The rapid development of Deepfake technology poses severe challenges to social trust and information security. While most existing detection methods primarily rely on passive analyses, due to unresolvable high-quality Deepfake contents, proactive defense has recently emerged by inserting invisible signals in advance of image editing. In this paper, we introduce a proactive Deepfake detection approach based on facial texture features. Since human eyes are more sensitive to perturbations in smooth regions, we invisibly insert perturbations within texture regions that have low perceptual saliency, applying localized perturbations to key texture regions while minimizing unwanted noise in non-textured areas. Our texture-guided perturbation framework first extracts preliminary texture features via Local Binary Patterns (LBP), and then introduces a dual-model attention strategy to generate and optimize texture perturbations. Experiments on CelebA-HQ and LFW datasets demonstrate the promising performance of our method in distorting Deepfake generation and producing obvious visual defects under multiple attack models, providing an efficient and scalable solution for proactive Deepfake detection.

Method: Introduces SpecGen method with Spectral-Spatial Tri-plane Aggregation (SSTA) network that models reflectance responses across wavelengths and incident-outgoing directions. Leverages abundant RGB BRDF data to enhance spectral BRDF generation from limited spectral data.

Result: Accurately reconstructs spectral BRDFs from limited spectral data and surpasses state-of-the-art methods in hyperspectral image reconstruction, achieving 8 dB improvement in PSNR.

Conclusion: The proposed method successfully generates spectral BRDFs from single RGB images, enabling flexible spectral rendering while overcoming data scarcity through innovative network architecture and training strategy.

Abstract: Synthesizing spectral images across different wavelengths is essential for photorealistic rendering. Unlike conventional spectral uplifting methods that convert RGB images into spectral ones, we introduce SpecGen, a novel method that generates spectral bidirectional reflectance distribution functions (BRDFs) from a single RGB image of a sphere. This enables spectral image rendering under arbitrary illuminations and shapes covered by the corresponding material. A key challenge in spectral BRDF generation is the scarcity of measured spectral BRDF data. To address this, we propose the Spectral-Spatial Tri-plane Aggregation (SSTA) network, which models reflectance responses across wavelengths and incident-outgoing directions, allowing the training strategy to leverage abundant RGB BRDF data to enhance spectral BRDF generation. Experiments show that our method accurately reconstructs spectral BRDFs from limited spectral data and surpasses state-of-the-art methods in hyperspectral image reconstruction, achieving an improvement of 8 dB in PSNR. Codes and data will be released upon acceptance.

Method: Created a dataset of 1,463 synthetic cricket scorecard images (ODI, T20, Test formats) with 1,500 English QA pairs, including English and Hindi scorecard subsets for controlled cross-script evaluation.

Result: State-of-the-art LVLMs (GPT-4o, Qwen2.5VL) struggle significantly on both English and Hindi subsets, showing poor performance in structure-aware text understanding, numerical reasoning, and cross-lingual generalization.

Conclusion: The benchmark reveals critical weaknesses in current LVLMs and provides a publicly available dataset to promote research in improving numerical reasoning and cross-lingual capabilities for semi-structured visual data.

Abstract: We introduce MMCRICBENCH-3K, a benchmark for Visual Question Answering (VQA) on cricket scorecards, designed to evaluate large vision-language models (LVLMs) on complex numerical and cross-lingual reasoning over semi-structured tabular images. MMCRICBENCH-3K comprises 1,463 synthetically generated scorecard images from ODI, T20, and Test formats, accompanied by 1,500 English QA pairs. It includes two subsets: MMCRICBENCH-E-1.5K, featuring English scorecards, and MMCRICBENCH-H-1.5K, containing visually similar Hindi scorecards, with all questions and answers kept in English to enable controlled cross-script evaluation. The task demands reasoning over structured numerical data, multi-image context, and implicit domain knowledge. Empirical results show that even state-of-the-art LVLMs, such as GPT-4o and Qwen2.5VL, struggle on the English subset despite it being their primary training language and exhibit a further drop in performance on the Hindi subset. This reveals key limitations in structure-aware visual text understanding, numerical reasoning, and cross-lingual generalization. The dataset is publicly available via Hugging Face at https://huggingface.co/datasets/DIALab/MMCricBench, to promote LVLM research in this direction.

Method: Uses Feature Aggregation Module (FAM) to fuse features from multiple full-resolution local tiles with down-sampled global view, plus Token-wise Forgery Localization and JPEG Quality Factor Estimation modules for robustness.

Result: Achieves state-of-the-art performance with over 13% accuracy increase on Chameleon dataset and 10% on new HiRes-50K benchmark (50,568 images up to 64MP).

Conclusion: HiDA-Net effectively addresses high-resolution AI image detection challenges by preserving native-resolution details and introducing specialized modules for robustness, significantly outperforming existing methods.

Abstract: The rapid growth of high-resolution, meticulously crafted AI-generated images poses a significant challenge to existing detection methods, which are often trained and evaluated on low-resolution, automatically generated datasets that do not align with the complexities of high-resolution scenarios. A common practice is to resize or center-crop high-resolution images to fit standard network inputs. However, without full coverage of all pixels, such strategies risk either obscuring subtle, high-frequency artifacts or discarding information from uncovered regions, leading to input information loss. In this paper, we introduce the High-Resolution Detail-Aggregation Network (HiDA-Net), a novel framework that ensures no pixel is left behind. We use the Feature Aggregation Module (FAM), which fuses features from multiple full-resolution local tiles with a down-sampled global view of the image. These local features are aggregated and fused with global representations for final prediction, ensuring that native-resolution details are preserved and utilized for detection. To enhance robustness against challenges such as localized AI manipulations and compression, we introduce Token-wise Forgery Localization (TFL) module for fine-grained spatial sensitivity and JPEG Quality Factor Estimation (QFE) module to disentangle generative artifacts from compression noise explicitly. Furthermore, to facilitate future research, we introduce HiRes-50K, a new challenging benchmark consisting of 50,568 images with up to 64 megapixels. Extensive experiments show that HiDA-Net achieves state-of-the-art, increasing accuracy by over 13% on the challenging Chameleon dataset and 10% on our HiRes-50K.

Method: DiCache consists of two components: 1) Online Probe Profiling Scheme that uses shallow-layer features to obtain real-time caching error priors for autonomous caching schedule determination, and 2) Dynamic Cache Trajectory Alignment that combines multi-step caches based on feature trajectory to better approximate current features.

Result: Extensive experiments show DiCache achieves higher efficiency and improved visual fidelity over state-of-the-art methods on various leading diffusion models including WAN 2.1, HunyuanVideo for video generation, and Flux for image generation.

Conclusion: DiCache provides a unified framework for adaptive caching in diffusion models that leverages shallow-layer feature correlations and trajectory similarities to achieve better performance without requiring training.

Abstract: Recent years have witnessed the rapid development of acceleration techniques for diffusion models, especially caching-based acceleration methods. These studies seek to answer two fundamental questions: “When to cache” and “How to use cache”, typically relying on predefined empirical laws or dataset-level priors to determine the timing of caching and utilizing handcrafted rules for leveraging multi-step caches. However, given the highly dynamic nature of the diffusion process, they often exhibit limited generalizability and fail on outlier samples. In this paper, a strong correlation is revealed between the variation patterns of the shallow-layer feature differences in the diffusion model and those of final model outputs. Moreover, we have observed that the features from different model layers form similar trajectories. Based on these observations, we present DiCache, a novel training-free adaptive caching strategy for accelerating diffusion models at runtime, answering both when and how to cache within a unified framework. Specifically, DiCache is composed of two principal components: (1) Online Probe Profiling Scheme leverages a shallow-layer online probe to obtain a stable prior for the caching error in real time, enabling the model to autonomously determine caching schedules. (2) Dynamic Cache Trajectory Alignment combines multi-step caches based on shallow-layer probe feature trajectory to better approximate the current feature, facilitating higher visual quality. Extensive experiments validate DiCache’s capability in achieving higher efficiency and improved visual fidelity over state-of-the-art methods on various leading diffusion models including WAN 2.1, HunyuanVideo for video generation, and Flux for image generation.

Method: Proposed Condition Modulated Expert (CoMoE) module aggregates similar patch features to dedicated experts, and WeaveNet dynamic connection mechanism bridges backbone and control branches.

Result: Extensive experiments on Subjects-200K and MultiGen-20M datasets show state-of-the-art performance across various conditional image generation tasks.

Conclusion: UniGen framework effectively addresses parameter redundancy and computational inefficiency while enhancing generation expressiveness and versatility.

Abstract: The image-to-image generation task aims to produce controllable images by leveraging conditional inputs and prompt instructions. However, existing methods often train separate control branches for each type of condition, leading to redundant model structures and inefficient use of computational resources. To address this, we propose a Unified image-to-image Generation (UniGen) framework that supports diverse conditional inputs while enhancing generation efficiency and expressiveness. Specifically, to tackle the widely existing parameter redundancy and computational inefficiency in controllable conditional generation architectures, we propose the Condition Modulated Expert (CoMoE) module. This module aggregates semantically similar patch features and assigns them to dedicated expert modules for visual representation and conditional modeling. By enabling independent modeling of foreground features under different conditions, CoMoE effectively mitigates feature entanglement and redundant computation in multi-condition scenarios. Furthermore, to bridge the information gap between the backbone and control branches, we propose WeaveNet, a dynamic, snake-like connection mechanism that enables effective interaction between global text-level control from the backbone and fine-grained control from conditional branches. Extensive experiments on the Subjects-200K and MultiGen-20M datasets across various conditional image generation tasks demonstrate that our method consistently achieves state-of-the-art performance, validating its advantages in both versatility and effectiveness. The code has been uploaded to https://github.com/gavin-gqzhang/UniGen.

Method: Developed a model where multimodal retriever is jointly optimized with LVLM for medical diagnosis, using only general-purpose backbones with lightweight fine-tuning. Evaluated on clinical multi-label classification and visual question answering tasks.

Result: Achieves competitive results with medically-pretrained models. Joint optimization significantly improves challenging cases over standard RAG. Oracle analysis shows correct diagnosis is frequently achievable with top retrieved images, but large performance gap remains from oracle.

Conclusion: The joint retrieval optimization approach shows promise but reveals substantial room for improvement, as rerankers using frontier LVLMs fail to close the performance gap from oracle, indicating need for future methodological advancements.

Abstract: Clinical decision-making often involves interpreting images (e.g., radiology) for making diagnoses. Retrieving relevant visual information from medical literature and hospital records could enhance diagnostic accuracy. In this paper, we develop a model in which a multimodal retriever is jointly optimized with an LVLM for medical diagnosis, unlike standard RAG where LVLM error signal is not propagated down to the retriever. We show that using only general-purpose backbones, with only lightweight fine-tuning, our model is able to achieve competitive results with medically-pretrained models across clinical multi-label classification and visual question answering tasks. In a novel analysis, we additionally find that in many cases different top retrieved images each lead to different predictions for a given target, and that these cases are empirically challenging for all models, even for non-retrieval models. Our joint retrieval optimization significantly improves these challenging cases over standard RAG. However, oracle analysis reveals that while the correct diagnosis is frequently achievable using one of the top retrieved images, in practice there is a large performance gap from the oracle, and rerankers using frontier LVLMs do not close this gap – leaving ample room for improvement by future methods. Code will be made publicly available.

Method: Proposes MoCo framework that separates human video generation into: 1) 3D structure generator for motion sequences from text, 2) appearance synthesis guided by structural sequence. Introduces Human-Aware Dynamic Control modules and dense tracking constraints for better motion control.

Result: Extensive experiments show MoCo outperforms existing approaches in generating realistic and structurally coherent human videos with consistent whole-body motion.

Conclusion: The decoupled structure-appearance approach with specialized motion control modules and a comprehensive dataset enables superior human video generation with realistic, physically plausible movements from text prompts.

Abstract: Generating human videos with consistent motion from text prompts remains a significant challenge, particularly for whole-body or long-range motion. Existing video generation models prioritize appearance fidelity, resulting in unrealistic or physically implausible human movements with poor structural coherence. Additionally, most existing human video datasets primarily focus on facial or upper-body motions, or consist of vertically oriented dance videos, limiting the scope of corresponding generation methods to simple movements. To overcome these challenges, we propose MoCo, which decouples the process of human video generation into two components: structure generation and appearance generation. Specifically, our method first employs an efficient 3D structure generator to produce a human motion sequence from a text prompt. The remaining video appearance is then synthesized under the guidance of the generated structural sequence. To improve fine-grained control over sparse human structures, we introduce Human-Aware Dynamic Control modules and integrate dense tracking constraints during training. Furthermore, recognizing the limitations of existing datasets, we construct a large-scale whole-body human video dataset featuring complex and diverse motions. Extensive experiments demonstrate that MoCo outperforms existing approaches in generating realistic and structurally coherent human videos.

Method: Combines Token-wise Variational Bayesian Inference (T-VBI) for training-free Bayesian adaptation by reparameterizing output tokens as latent variables, and Self-Optimizing Kolmogorov-Arnold Network (SO-KAN) with learnable spline activations for improved interpretability and token pruning.

Result: Achieves real-time inference (0.03s/image), superior segmentation accuracy (89.0% DSC vs 88.3% for MedSAM), identifies four critical decision-making tokens, and demonstrates effectiveness across five ultrasound datasets.

Conclusion: E-BayesSAM successfully bridges SAM’s versatility with clinical needs by unifying efficiency, reliability, and interpretability, advancing deployment in safety-critical medical applications.

Abstract: Although the Segment Anything Model (SAM) has advanced medical image segmentation, its Bayesian adaptation for uncertainty-aware segmentation remains hindered by three key issues: (1) instability in Bayesian fine-tuning of large pre-trained SAMs; (2) high computation cost due to SAM’s massive parameters; (3) SAM’s black-box design limits interpretability. To overcome these, we propose E-BayesSAM, an efficient framework combining Token-wise Variational Bayesian Inference (T-VBI) for efficienty Bayesian adaptation and Self-Optimizing Kolmogorov-Arnold Network (SO-KAN) for improving interpretability. T-VBI innovatively reinterprets SAM’s output tokens as dynamic probabilistic weights and reparameterizes them as latent variables without auxiliary training, enabling training-free VBI for uncertainty estimation. SO-KAN improves token prediction with learnable spline activations via self-supervised learning, providing insight to prune redundant tokens to boost efficiency and accuracy. Experiments on five ultrasound datasets demonstrated that E-BayesSAM achieves: (i) real-time inference (0.03s/image), (ii) superior segmentation accuracy (average DSC: Pruned E-BayesSAM’s 89.0% vs. E-BayesSAM’s 88.0% vs. MedSAM’s 88.3%), and (iii) identification of four critical tokens governing SAM’s decisions. By unifying efficiency, reliability, and interpretability, E-BayesSAM bridges SAM’s versatility with clinical needs, advancing deployment in safety-critical medical applications. The source code is available at https://github.com/mp31192/E-BayesSAM.

Method: Applied image retrieval techniques to identify and characterize visual data leakage, categorizing it by modality, coverage, and degree of leakage.

Result: Found that all analyzed datasets exhibit some form of data leakage, and all types of leakage (from severe to subtle) compromise the reliability of model evaluation in downstream tasks.

Conclusion: Data leakage is a pervasive problem in visual datasets that significantly undermines the integrity and fairness of model evaluation across computer vision benchmarks.

Abstract: We analyze data leakage in visual datasets. Data leakage refers to images in evaluation benchmarks that have been seen during training, compromising fair model evaluation. Given that large-scale datasets are often sourced from the internet, where many computer vision benchmarks are publicly available, our efforts are focused into identifying and studying this phenomenon. We characterize visual leakage into different types according to its modality, coverage, and degree. By applying image retrieval techniques, we unequivocally show that all the analyzed datasets present some form of leakage, and that all types of leakage, from severe instances to more subtle cases, compromise the reliability of model evaluation in downstream tasks.

Method: Uses Topology-Guided Synonymous Semantic Generation (TGSSG) with LLMs and persistent homology analysis for textual prompts, and Category-Agnostic Discriminative Region Selection (CADRS) with activation maps for visual prompts, plus set-to-set matching with TTA and optimal transport.

Result: The method constructs comprehensive textual prompts and compact visual prompts from semantic perspective to achieve better visual-textual alignment.

Conclusion: The proposed CPE method effectively addresses semantic misalignment issues and improves zero-shot generalization of vision-language models through enhanced prompt construction and alignment strategies.

Abstract: Vision-language models (VLMs) pre-trained on web-scale data exhibit promising zero-shot generalization but often suffer from semantic misalignment due to domain gaps between pre-training and downstream tasks. Existing approaches primarily focus on text prompting with class-specific descriptions and visual-text adaptation via aligning cropped image regions with textual descriptions. However, they still face the issues of incomplete textual prompts and noisy visual prompts. In this paper, we propose a novel constrained prompt enhancement (CPE) method to improve visual-textual alignment by constructing comprehensive textual prompts and compact visual prompts from the semantic perspective. Specifically, our approach consists of two key components: Topology-Guided Synonymous Semantic Generation (TGSSG) and Category-Agnostic Discriminative Region Selection (CADRS). Textually, to address the issue of incomplete semantic expression in textual prompts, our TGSSG first generates synonymous semantic set for each category via large language models, and constructs comprehensive textual prompts based on semantic ambiguity entropy and persistent homology analysis. Visually, to mitigate the irrelevant visual noise introduced by random cropping, our CADRS identifies discriminative regions with activation maps outputted by a pre-trained vision model, effectively filtering out noisy regions and generating compact visual prompts. Given the comprehensive set of textual prompts and compact set of visual prompts, we introduce two set-to-set matching strategies based on test-time adaptation (TTA) and optimal transport (OT) to achieve effective visual-textual alignment, and so improve zero-shot generalization of VLMs.

Method: Decomposes rigid transformation using Chasles’ theorem into translation along rotation axis and 2D rigid transformation. Uses BnB search for optimal rotation axis/angle with range maximum query problems. Searches top-k candidate rotation axes via cube mapping, estimates translation through interval stabbing, and solves 2D registration as 1D rotation angle search with 2D RMQ using sweep line algorithm and segment tree.

Result: Experimental results on 3DMatch, 3DLoMatch, and KITTI datasets demonstrate superior accuracy and efficiency over state-of-the-art methods, with polynomial time complexity and linear space complexity that scales with number of points.

Conclusion: The proposed geometric maximum overlapping registration framework achieves better performance than existing methods while maintaining polynomial time complexity and linear space complexity, making it suitable for practical applications with high outlier ratios.

Abstract: Point cloud registration based on correspondences computes the rigid transformation that maximizes the number of inliers constrained within the noise threshold. Current state-of-the-art (SOTA) methods employing spatial compatibility graphs or branch-and-bound (BnB) search mainly focus on registration under high outlier ratios. However, graph-based methods require at least quadratic space and time complexity for graph construction, while multi-stage BnB search methods often suffer from inaccuracy due to local optima between decomposed stages. This paper proposes a geometric maximum overlapping registration framework via rotation-only BnB search. The rigid transformation is decomposed using Chasles’ theorem into a translation along rotation axis and a 2D rigid transformation. The optimal rotation axis and angle are searched via BnB, with residual parameters formulated as range maximum query (RMQ) problems. Firstly, the top-k candidate rotation axes are searched within a hemisphere parameterized by cube mapping, and the translation along each axis is estimated through interval stabbing of the correspondences projected onto that axis. Secondly, the 2D registration is relaxed to 1D rotation angle search with 2D RMQ of geometric overlapping for axis-aligned rectangles, which is solved deterministically in polynomial time using sweep line algorithm with segment tree. Experimental results on 3DMatch, 3DLoMatch, and KITTI datasets demonstrate superior accuracy and efficiency over SOTA methods, while the time complexity is polynomial and the space complexity increases linearly with the number of points, even in the worst case.

Method: Proposes four components: KL-Divergence Regularization Loss to mitigate non-IID effects, KL-Divergence-Prune Weighted Aggregation for robust aggregation and communication reduction, sparse Activation Skipping to preserve critical parameters, and Cross-Round Recovery for dynamic pruning control.

Result: Achieves 33-38% communication cost reduction on ResNet-50 and 20-40% reduction on ResNet-34 while maintaining model accuracy within 1% degradation across eight benchmark datasets.

Conclusion: FedKLPR effectively addresses both statistical heterogeneity and communication efficiency challenges in federated person re-identification, providing a practical solution for real-world deployment with significant communication savings and maintained accuracy.

Abstract: Person re-identification (Re-ID) is a fundamental task in intelligent surveillance and public safety. Federated learning (FL) offers a privacy-preserving solution by enabling collaborative model training without centralized data collection. However, applying FL to real-world re-ID systems faces two major challenges: statistical heterogeneity across clients due to non-IID data distributions, and substantial communication overhead caused by frequent transmission of large-scale models. To address these issues, we propose FedKLPR, a lightweight and communication-efficient federated learning framework for person re-identification. FedKLPR introduces four key components. First, the KL-Divergence Regularization Loss (KLL) constrains local models by minimizing the divergence from the global feature distribution, effectively mitigating the effects of statistical heterogeneity and improving convergence stability under non-IID conditions. Secondly, KL-Divergence-Prune Weighted Aggregation (KLPWA) integrates pruning ratio and distributional similarity into the aggregation process, thereby improving the robustness of the global model while significantly reducing communication overhead. Furthermore, sparse Activation Skipping (SAS) mitigates the dilution of critical parameters during the aggregation of pruned client models by excluding zero-valued weights from the update process. Finally, Cross-Round Recovery (CRR) introduces a dynamic pruning control mechanism that halts pruning when necessary, enabling deeper compression while maintaining model accuracy. Experimental results on eight benchmark datasets demonstrate that FedKLPR achieves significant communication reduction. Compared with the state-of-the-art, FedKLPR reduces 33%-38% communication cost on ResNet-50 and 20%-40% communication cost on ResNet-34, while maintaining model accuracy within 1% degradation.

Method: Introduces Dynamic Inter-block Activation and Expansion-Corrosion Strategy for effective depth pruning, VAE compression through channel pruning and attention removal, eliminates time- and prompt-related modules, and implements pre-caching techniques to accelerate the model.

Result: Achieves up to 5.68x speedup and 83% parameter reduction compared to teacher model TSD-SR while maintaining high perceptual quality in real-world image super-resolution tasks.

Conclusion: TinySR demonstrates that compact diffusion models can achieve real-time performance in image super-resolution through strategic architectural optimizations and pruning techniques without compromising output quality.

Abstract: Real-world image super-resolution (Real-ISR) focuses on recovering high-quality images from low-resolution inputs that suffer from complex degradations like noise, blur, and compression. Recently, diffusion models (DMs) have shown great potential in this area by leveraging strong generative priors to restore fine details. However, their iterative denoising process incurs high computational overhead, posing challenges for real-time applications. Although one-step distillation methods, such as OSEDiff and TSD-SR, offer faster inference, they remain fundamentally constrained by their large, over-parameterized model architectures. In this work, we present TinySR, a compact yet effective diffusion model specifically designed for Real-ISR that achieves real-time performance while maintaining perceptual quality. We introduce a Dynamic Inter-block Activation and an Expansion-Corrosion Strategy to facilitate more effective decision-making in depth pruning. We achieve VAE compression through channel pruning, attention removal and lightweight SepConv. We eliminate time- and prompt-related modules and perform pre-caching techniques to further speed up the model. TinySR significantly reduces computational cost and model size, achieving up to 5.68x speedup and 83% parameter reduction compared to its teacher TSD-SR, while still providing high quality results.

Method: A closed-loop system combining LLM planning capabilities and LVLM visual understanding, featuring instruction parsing, multi-level editing planning, iterative editing, and feedback evaluation modules.

Result: Achieved average score of 3.67 on LongBench-T2I-Edit benchmark, significantly outperforming baselines (Direct Re-Prompting: 2.29, InstructPix2Pix: 2.91, GLIGEN-based Edit: 3.16, ControlNet-XL: 3.39).

Conclusion: RefineEdit-Agent effectively addresses iterative image editing challenges through its agentic design, demonstrating superior edit fidelity and context preservation across various evaluation metrics.

Abstract: Despite the remarkable capabilities of text-to-image (T2I) generation models, real-world applications often demand fine-grained, iterative image editing that existing methods struggle to provide. Key challenges include granular instruction understanding, robust context preservation during modifications, and the lack of intelligent feedback mechanisms for iterative refinement. This paper introduces RefineEdit-Agent, a novel, training-free intelligent agent framework designed to address these limitations by enabling complex, iterative, and context-aware image editing. RefineEdit-Agent leverages the powerful planning capabilities of Large Language Models (LLMs) and the advanced visual understanding and evaluation prowess of Vision-Language Large Models (LVLMs) within a closed-loop system. Our framework comprises an LVLM-driven instruction parser and scene understanding module, a multi-level LLM-driven editing planner for goal decomposition, tool selection, and sequence generation, an iterative image editing module, and a crucial LVLM-driven feedback and evaluation loop. To rigorously evaluate RefineEdit-Agent, we propose LongBench-T2I-Edit, a new benchmark featuring 500 initial images with complex, multi-turn editing instructions across nine visual dimensions. Extensive experiments demonstrate that RefineEdit-Agent significantly outperforms state-of-the-art baselines, achieving an average score of 3.67 on LongBench-T2I-Edit, compared to 2.29 for Direct Re-Prompting, 2.91 for InstructPix2Pix, 3.16 for GLIGEN-based Edit, and 3.39 for ControlNet-XL. Ablation studies, human evaluations, and analyses of iterative refinement, backbone choices, tool usage, and robustness to instruction complexity further validate the efficacy of our agentic design in delivering superior edit fidelity and context preservation.

Method: Uses disentangled geometry and appearance model without deep networks, neural deformation field for global geometric context, novel regularization for geometric features, and view-invariant diffuse term baked into vertices.

Result: Achieves state-of-the-art training (4.84 minutes) and rendering (0.023 seconds) speeds with competitive reconstruction quality, enabling mesh and texture editing applications.

Conclusion: The method combines efficiency, competitive quality, and broad applicability, making it a valuable contribution to multi-view surface reconstruction and rendering.

Abstract: This paper addresses the limitations of neural rendering-based multi-view surface reconstruction methods, which require an additional mesh extraction step that is inconvenient and would produce poor-quality surfaces with mesh aliasing, restricting downstream applications. Building on the explicit mesh representation and differentiable rasterization framework, this work proposes an efficient solution that preserves the high efficiency of this framework while significantly improving reconstruction quality and versatility. Specifically, we introduce a disentangled geometry and appearance model that does not rely on deep networks, enhancing learning and broadening applicability. A neural deformation field is constructed to incorporate global geometric context, enhancing geometry learning, while a novel regularization constrains geometric features passed to a neural shader to ensure its accuracy and boost shading. For appearance, a view-invariant diffuse term is separated and baked into mesh vertices, further improving rendering efficiency. Experimental results demonstrate that the proposed method achieves state-of-the-art training (4.84 minutes) and rendering (0.023 seconds) speeds, with reconstruction quality that is competitive with top-performing methods. Moreover, the method enables practical applications such as mesh and texture editing, showcasing its versatility and application potential. This combination of efficiency, competitive quality, and broad applicability makes our approach a valuable contribution to multi-view surface reconstruction and rendering.

Method: Trains a generalizable neural network using supervised losses to predict physical properties from 3D visual features, coupled with Gaussian Splatting for scene representation and leveraging pretrained features like CLIP.

Result: PIXIE achieves 1.46-4.39x better performance and orders of magnitude faster inference than test-time optimization methods, with zero-shot generalization to real-world scenes using only synthetic training data.

Conclusion: PIXIE provides an efficient, generalizable solution for physical property inference that enables realistic physics simulation and works across synthetic and real-world 3D scenes.

Abstract: Inferring the physical properties of 3D scenes from visual information is a critical yet challenging task for creating interactive and realistic virtual worlds. While humans intuitively grasp material characteristics such as elasticity or stiffness, existing methods often rely on slow, per-scene optimization, limiting their generalizability and application. To address this problem, we introduce PIXIE, a novel method that trains a generalizable neural network to predict physical properties across multiple scenes from 3D visual features purely using supervised losses. Once trained, our feed-forward network can perform fast inference of plausible material fields, which coupled with a learned static scene representation like Gaussian Splatting enables realistic physics simulation under external forces. To facilitate this research, we also collected PIXIEVERSE, one of the largest known datasets of paired 3D assets and physic material annotations. Extensive evaluations demonstrate that PIXIE is about 1.46-4.39x better and orders of magnitude faster than test-time optimization methods. By leveraging pretrained visual features like CLIP, our method can also zero-shot generalize to real-world scenes despite only ever been trained on synthetic data. https://pixie-3d.github.io/

Method: Created a comprehensive benchmark with multiple datasets, conducted experiments across different adaptation scenarios (synthetic-to-real, point cloud quality, layout, and instance feature adaptation), and introduced approaches to improve adaptation performance.

Result: Analysis of how different domain gaps impact 3D object detectors and provision of baseline approaches for improving cross-domain adaptation performance.

Conclusion: The work establishes a foundation for domain adaptive indoor 3D object detection and encourages future development of detectors with stronger generalization capabilities across different domains.

Abstract: As a fundamental task for indoor scene understanding, 3D object detection has been extensively studied, and the accuracy on indoor point cloud data has been substantially improved. However, existing researches have been conducted on limited datasets, where the training and testing sets share the same distribution. In this paper, we consider the task of adapting indoor 3D object detectors from one dataset to another, presenting a comprehensive benchmark with ScanNet, SUN RGB-D and 3D Front datasets, as well as our newly proposed large-scale datasets ProcTHOR-OD and ProcFront generated by a 3D simulator. Since indoor point cloud datasets are collected and constructed in different ways, the object detectors are likely to overfit to specific factors within each dataset, such as point cloud quality, bounding box layout and instance features. We conduct experiments across datasets on different adaptation scenarios including synthetic-to-real adaptation, point cloud quality adaptation, layout adaptation and instance feature adaptation, analyzing the impact of different domain gaps on 3D object detectors. We also introduce several approaches to improve adaptation performances, providing baselines for domain adaptive indoor 3D object detection, hoping that future works may propose detectors with stronger generalization ability across domains. Our project homepage can be found in https://jeremyzhao1998.github.io/DAVoteNet-release/.

Method: Dual-branch architecture with Video Encoding Branch for spatio-temporal features and Cross-Modal Guidance Branch using LVLM for multi-granularity semantic descriptions (Global Event Prompting and Temporal Sub-event Prompting), integrated through adaptive gating, cross-modal attention, and event graph module.

Result: Achieves average mAP of 40.5% on ActivityNet v1.3 and mAP@0.5 of 67.1% on THUMOS14, outperforming leading baselines.

Conclusion: ECVT significantly enhances video temporal structure understanding and event logic through LVLM-generated semantic guidance, demonstrating state-of-the-art performance in action recognition and localization.

Abstract: Action recognition and localization in complex, untrimmed videos remain a formidable challenge in computer vision, largely due to the limitations of existing methods in capturing fine-grained actions, long-term temporal dependencies, and high-level semantic information from low-level visual features. This paper introduces the Event-Contextualized Video Transformer (ECVT), a novel architecture that leverages the advanced semantic understanding capabilities of Large Vision-Language Models (LVLMs) to bridge this gap. ECVT employs a dual-branch design, comprising a Video Encoding Branch for spatio-temporal feature extraction and a Cross-Modal Guidance Branch. The latter utilizes an LVLM to generate multi-granularity semantic descriptions, including Global Event Prompting for macro-level narrative and Temporal Sub-event Prompting for fine-grained action details. These multi-level textual cues are integrated into the video encoder’s learning process through sophisticated mechanisms such as adaptive gating for high-level semantic fusion, cross-modal attention for fine-grained feature refinement, and an event graph module for temporal context calibration. Trained end-to-end with a comprehensive loss function incorporating semantic consistency and temporal calibration terms, ECVT significantly enhances the model’s ability to understand video temporal structures and event logic. Extensive experiments on ActivityNet v1.3 and THUMOS14 datasets demonstrate that ECVT achieves state-of-the-art performance, with an average mAP of 40.5% on ActivityNet v1.3 and mAP@0.5 of 67.1% on THUMOS14, outperforming leading baselines.

Method: Proposes hybrid data synthesis using BlenderProc for photorealistic images with precise annotations and domain randomization, plus NVIDIA’s Cosmos-Predict2 for physically plausible video sequences with temporal diversity and rare viewpoints.

Result: YOLO-based detection network trained on composite dataset (real + synthetic data) achieved superior performance, with 1:1 real-synthetic mixture yielding highest accuracy, surpassing real-only baseline.

Conclusion: Synthetic-first approach is viable, efficient, cost-effective, and safe alternative for developing reliable perception systems in safety-critical industrial applications with resource constraints.

Abstract: This work addresses the challenges of data scarcity and high acquisition costs for training robust object detection models in complex industrial environments, such as offshore oil platforms. The practical and economic barriers to collecting real-world data in these hazardous settings often hamper the development of autonomous inspection systems. To overcome this, in this work we propose and validate a hybrid data synthesis pipeline that combines procedural rendering with AI-driven video generation. Our methodology leverages BlenderProc to create photorealistic images with precise annotations and controlled domain randomization, and integrates NVIDIA’s Cosmos-Predict2 world-foundation model to synthesize physically plausible video sequences with temporal diversity, capturing rare viewpoints and adverse conditions. We demonstrate that a YOLO-based detection network trained on a composite dataset, blending real images with our synthetic data, achieves superior performance compared to models trained exclusively on real-world data. Notably, a 1:1 mixture of real and synthetic data yielded the highest accuracy, surpassing the real-only baseline. These findings highlight the viability of a synthetic-first approach as an efficient, cost-effective, and safe alternative for developing reliable perception systems in safety-critical and resource-constrained industrial applications.

Method: Proposes a benchmark with four reasoning dimensions (Idiom Interpretation, Textual Image Design, Entity-Reasoning, Scientific-Reasoning) and a two-stage evaluation protocol for reasoning accuracy and image quality.

Result: The benchmark was used to evaluate various T2I generation models, providing comprehensive performance analysis across different reasoning capabilities.

Conclusion: T2I-ReasonBench serves as a valuable tool for assessing and comparing the reasoning capabilities of text-to-image models, enabling better understanding of their strengths and limitations.

Abstract: We propose T2I-ReasonBench, a benchmark evaluating reasoning capabilities of text-to-image (T2I) models. It consists of four dimensions: Idiom Interpretation, Textual Image Design, Entity-Reasoning and Scientific-Reasoning. We propose a two-stage evaluation protocol to assess the reasoning accuracy and image quality. We benchmark various T2I generation models, and provide comprehensive analysis on their performances.

Method: Proposes GraphMMP, a two-stage multimodal prognosis model based on graph neural networks. It constructs feature graphs using mutual information and features a global fusion module built on Mamba architecture to enhance prognosis performance.

Result: Empirical results demonstrate that GraphMMP surpasses existing methods on datasets related to liver prognosis and the METABRIC study, showing superior performance in multimodal medical prognosis tasks.

Conclusion: GraphMMP effectively addresses the challenges of multimodal medical data analysis by leveraging graph neural networks and Mamba-based fusion, proving to be an effective solution for capturing complex inter-modal relationships and improving prognosis outcomes.

Abstract: In the field of multimodal medical data analysis, leveraging diverse types of data and understanding their hidden relationships continues to be a research focus. The main challenges lie in effectively modeling the complex interactions between heterogeneous data modalities with distinct characteristics while capturing both local and global dependencies across modalities. To address these challenges, this paper presents a two-stage multimodal prognosis model, GraphMMP, which is based on graph neural networks. The proposed model constructs feature graphs using mutual information and features a global fusion module built on Mamba, which significantly boosts prognosis performance. Empirical results show that GraphMMP surpasses existing methods on datasets related to liver prognosis and the METABRIC study, demonstrating its effectiveness in multimodal medical prognosis tasks.

Method: Proposes a sensitivity-aware regularized tuning framework that analyzes tangent space of pre-trained weights to measure prior sensitivities (for generalization preservation) and explores transfer sensitivities during tuning (for adaptability and stability), incorporating these as regularization terms.

Result: Extensive experiments show superior performance surpassing state-of-the-art techniques across various multi-modal tracking tasks, with significant enhancement in transferability across modalities.

Conclusion: The proposed sensitivity-aware regularization framework effectively addresses the plasticity-stability dilemma in multi-modal tracker optimization, achieving better transfer performance while maintaining generalization capabilities from pre-trained RGB models.

Abstract: This paper tackles the critical challenge of optimizing multi-modal trackers by effectively adapting the pre-trained models for RGB data. Existing fine-tuning paradigms oscillate between excessive freedom and over-restriction, both leading to a suboptimal plasticity-stability trade-off. To mitigate this dilemma, we propose a novel sensitivity-aware regularized tuning framework, which delicately refines the learning process by incorporating intrinsic parameter sensitivities. Through a comprehensive investigation from pre-trained to multi-modal contexts, we identify that parameters sensitive to pivotal foundational patterns and cross-domain shifts are primary drivers of this issue. Specifically, we first analyze the tangent space of pre-trained weights to measure and orient prior sensitivities, dedicated to preserving generalization. Then, we further explore transfer sensitivities during the tuning phase, emphasizing adaptability and stability. By incorporating these sensitivities as regularization terms, our method significantly enhances the transferability across modalities. Extensive experiments showcase the superior performance of the proposed method, surpassing current state-of-the-art techniques across various multi-modal tracking. The source code and models will be publicly available at https://github.com/zhiwen-xdu/SRTrack.

Method: Extended version of Contrastive Audio-Visual Masked Autoencoder (CAV-MAE) modified to handle more frames, pre-trained on VoxCeleb2 dataset in self-supervised manner, then fine-tuned on downstream tasks.

Result: Achieves state-of-the-art results on multimodal emotion recognition and laughter recognition, and competitive results for apparent personality estimation.

Conclusion: In-domain self-supervised pre-training on social interaction data is highly effective for developing powerful audiovisual models for social behavior perception tasks.

Abstract: Human social behaviors are inherently multimodal necessitating the development of powerful audiovisual models for their perception. In this paper, we present Social-MAE, our pre-trained audiovisual Masked Autoencoder based on an extended version of Contrastive Audio-Visual Masked Auto-Encoder (CAV-MAE), which is pre-trained on audiovisual social data. Specifically, we modify CAV-MAE to receive a larger number of frames as input and pre-train it on a large dataset of human social interaction (VoxCeleb2) in a self-supervised manner. We demonstrate the effectiveness of this model by finetuning and evaluating the model on different social and affective downstream tasks, namely, emotion recognition, laughter detection and apparent personality estimation. The model achieves state-of-the-art results on multimodal emotion recognition and laughter recognition and competitive results for apparent personality estimation, demonstrating the effectiveness of in-domain self-supervised pre-training. Code and model weight are available here https://github.com/HuBohy/SocialMAE.

Method: Combines redundancy-reduction objective of Barlow Twins with self-distillation strategy of DINO. Trained on MS COCO dataset using only 10% labeled data for linear probing, compared against standalone DINO and Barlow Twins implementations.

Result: Hybrid model achieves comparable loss and classification accuracy to DINO while maintaining strong feature representations. Attention visualizations show improved semantic segmentation capability.

Conclusion: The combined method offers a scalable, label-efficient alternative for training Vision Transformers in resource-constrained environments by leveraging complementary strengths of both approaches.

Abstract: Training AI models to understand images without costly labeled data remains a challenge. We combine two techniques–DINO (teacher-student learning) and Barlow Twins (redundancy reduction)–to create a model that learns better with fewer labels and less compute. While both DINO and Barlow Twins have independently demonstrated strong performance in self-supervised learning, each comes with limitations–DINO may be sensitive to certain augmentations, and Barlow Twins often requires batch sizes too large to fit on consumer hardware. By combining the redundancy-reduction objective of Barlow Twins with the self-distillation strategy of DINO, we aim to leverage their complementary strengths. We train a hybrid model on the MS COCO dataset using only 10% of labeled data for linear probing, and evaluate its performance against standalone DINO and Barlow Twins implementations. Preliminary results show that the combined approach achieves comparable loss and classification accuracy to DINO while maintaining strong feature representations. Attention visualizations further suggest improved semantic segmentation capability in the hybrid model. This combined method offers a scalable, label-efficient alternative for training ViTs in resource-constrained environments.

Method: Multi-stage training on 60 public datasets with over 220,000 MRI volumes and 19 million slices, including self-supervised vision pretraining, vision-language alignment, multimodal pretraining, and multi-task instruction tuning.

Result: Qualitative results show OmniMRI can perform diverse tasks within a single architecture: MRI reconstruction, anatomical/pathological segmentation, abnormality detection, diagnostic suggestion, and radiology report generation.

Conclusion: OmniMRI demonstrates potential to consolidate fragmented MRI pipelines into a scalable, generalist framework that unifies imaging and clinical language for comprehensive end-to-end MRI interpretation.

Abstract: Magnetic Resonance Imaging (MRI) is indispensable in clinical practice but remains constrained by fragmented, multi-stage workflows encompassing acquisition, reconstruction, segmentation, detection, diagnosis, and reporting. While deep learning has achieved progress in individual tasks, existing approaches are often anatomy- or application-specific and lack generalizability across diverse clinical settings. Moreover, current pipelines rarely integrate imaging data with complementary language information that radiologists rely on in routine practice. Here, we introduce OmniMRI, a unified vision-language foundation model designed to generalize across the entire MRI workflow. OmniMRI is trained on a large-scale, heterogeneous corpus curated from 60 public datasets, over 220,000 MRI volumes and 19 million MRI slices, incorporating image-only data, paired vision-text data, and instruction-response data. Its multi-stage training paradigm, comprising self-supervised vision pretraining, vision-language alignment, multimodal pretraining, and multi-task instruction tuning, progressively equips the model with transferable visual representations, cross-modal reasoning, and robust instruction-following capabilities. Qualitative results demonstrate OmniMRI’s ability to perform diverse tasks within a single architecture, including MRI reconstruction, anatomical and pathological segmentation, abnormality detection, diagnostic suggestion, and radiology report generation. These findings highlight OmniMRI’s potential to consolidate fragmented pipelines into a scalable, generalist framework, paving the way toward foundation models that unify imaging and clinical language for comprehensive, end-to-end MRI interpretation.

Method: Proposed a polynomial approximation to handle the originally non-polynomial problem, classified the resulting minimal problems, determined their algebraic degrees, and developed minimal solvers for several problems with low degrees.

Result: Developed working solvers that were evaluated on both synthetic and real datasets, demonstrating the effectiveness of the polynomial approximation approach.

Conclusion: The proposed polynomial approximation enables effective camera velocity estimation from asynchronous point tracks, with minimal solvers providing practical solutions for rolling shutter and event camera applications.

Abstract: We address the problem of estimating both translational and angular velocity of a camera from asynchronous point tracks, a formulation relevant to rolling shutter and event cameras. Since the original problem is non-polynomial, we propose a polynomial approximation, classify the resulting minimal problems, and determine their algebraic degrees. Furthermore, we develop minimal solvers for several problems with low degrees and evaluate them on synthetic and real datasets. The code will be made publicly available.

Method: Comprehensive investigation of CNN architectures including 1D convolutional classifiers with skip connections, ResNet50 backbones, and partial backbone unfreezing. Statistical analysis comparing RadImageNet and ImageNet pre-training effects on breast lesion malignancy and ACL tear detection tasks.

Result: Best models achieved AUCs of 0.9969 for ACL tear detection and 0.9641 for breast nodule malignancy detection, competitive with previous works. No evidence found that RadImageNet pre-training provides superior downstream performance compared to ImageNet for these specific medical tasks.

Conclusion: Optimal medical classification performance comes from specific architectural choices (1D CNNs with skip connections, ResNet50 backbones, partial unfreezing) rather than medical-specific pre-training. RadImageNet pre-training doesn’t show clear advantage over ImageNet for ACL tear and breast lesion classification.

Abstract: Modern computer vision models have proven to be highly useful for medical imaging classification and segmentation tasks, but the scarcity of medical imaging data often limits the efficacy of models trained from scratch. Transfer learning has emerged as a pivotal solution to this, enabling the fine-tuning of high-performance models on small data. Mei et al. (2022) found that pre-training CNNs on a large dataset of radiologist-labeled images (RadImageNet) enhanced model performance on downstream tasks compared to ImageNet pretraining. The present work extends Mei et al. (2022) by conducting a comprehensive investigation to determine optimal CNN architectures for breast lesion malignancy detection and ACL tear detection, as well as performing statistical analysis to compare the effect of RadImageNet and ImageNet pre-training on downstream model performance. Our findings suggest that 1-dimensional convolutional classifiers with skip connections, ResNet50 pre-trained backbones, and partial backbone unfreezing yields optimal downstream medical classification performance. Our best models achieve AUCs of 0.9969 for ACL tear detection and 0.9641 for breast nodule malignancy detection, competitive with the results reported by Mei et al. (2022) and surpassing other previous works. We do not find evidence confirming RadImageNet pre-training to provide superior downstream performance for ACL tear and breast lesion classification tasks.

Method: Proposes DEHVF with lightweight hierarchical visual fuser that dynamically selects and fuses visual features corresponding to semantic granularity in each LLM layer. Projects and aligns fused features before embedding into FFN of corresponding LLM layers.

Result: Achieves higher accuracy than existing PEFT baselines on VL benchmarks including ScienceQA visual question answering and COCO Captions image captioning, while maintaining efficient training and inference.

Conclusion: DEHVF effectively addresses sequence expansion issues by leveraging hierarchical representations, enabling precise cross-modal alignment with minimal parameter fine-tuning.

Abstract: Large Vision-Language Models (LVLMs) commonly follow a paradigm that projects visual features and then concatenates them with text tokens to form a unified sequence input for Large Language Models (LLMs). However, this paradigm leads to a significant increase in the length of the input sequence, resulting in substantial computational overhead. Existing methods attempt to fuse visual information into the intermediate layers of LLMs, which alleviate the sequence length issue but often neglect the hierarchical semantic representations within the model and the fine-grained visual information available in the shallower visual encoding layers. To address this limitation, we propose DEHVF, an efficient vision-language fine-tuning method based on dynamic embedding and fusion of hierarchical visual features. Its core lies in leveraging the inherent hierarchical representation characteristics of visual encoders and language models. Through a lightweight hierarchical visual fuser, it dynamically selects and fuses hierarchical features corresponding to semantic granularity based on the internal representations of each layer in LLMs. The fused layer-related visual features are then projected and aligned before being directly embedded into the Feed-Forward Network (FFN) of the corresponding layer in LLMs. This approach not only avoids sequence expansion but also dynamically fuses multi-layer visual information. By fine-tuning only a small number of parameters, DEHVF achieves precise alignment and complementarity of cross-modal information at the same semantic granularity. We conducted experiments across various VL benchmarks, including visual question answering on ScienceQA and image captioning on COCO Captions. The results demonstrate that DEHVF achieves higher accuracy than existing parameter-efficient fine-tuning (PEFT) baselines while maintaining efficient training and inference.

Method: Developed MetaDSL language for human-readable metamaterial descriptions, created MetaDB repository with 150K+ parameterized designs and simulations, and established MetaBench benchmarks for testing vision-language models.

Result: Created a comprehensive framework with curated database, established baselines using state-of-the-art vision-language models, and demonstrated effectiveness through case studies.

Conclusion: The framework provides a strong foundation for integrated design and understanding of structure-representation-property relationships in metamaterials.

Abstract: Metamaterials are micro-architected structures whose geometry imparts highly tunable-often counter-intuitive-bulk properties. Yet their design is difficult because of geometric complexity and a non-trivial mapping from architecture to behaviour. We address these challenges with three complementary contributions. (i) MetaDSL: a compact, semantically rich domain-specific language that captures diverse metamaterial designs in a form that is both human-readable and machine-parsable. (ii) MetaDB: a curated repository of more than 150,000 parameterized MetaDSL programs together with their derivatives-three-dimensional geometry, multi-view renderings, and simulated elastic properties. (iii) MetaBench: benchmark suites that test three core capabilities of vision-language metamaterial assistants-structure reconstruction, property-driven inverse design, and performance prediction. We establish baselines by fine-tuning state-of-the-art vision-language models and deploy an omni-model within an interactive, CAD-like interface. Case studies show that our framework provides a strong first step toward integrated design and understanding of structure-representation-property relationships.

Method: Proposes Incremental Dynamic Update (IDU) pipeline: 1) Camera pose estimation to align new images with existing 3D model, 2) Change detection to identify modifications, 3) 3D generative AI to create new assets, 4) Human-guided integration of single objects at a time into existing 3D Gaussian Splatting models.

Result: Experimental results show the IDU pipeline significantly reduces update time and labor compared to full-scale reconstruction methods.

Conclusion: The IDU pipeline provides a cost-effective and targeted solution for maintaining current 3D models in rapidly evolving military scenarios, offering substantial efficiency improvements over traditional update approaches.

Abstract: For simulation and training purposes, military organizations have made substantial investments in developing high-resolution 3D virtual environments through extensive imaging and 3D scanning. However, the dynamic nature of battlefield conditions-where objects may appear or vanish over time-makes frequent full-scale updates both time-consuming and costly. In response, we introduce the Incremental Dynamic Update (IDU) pipeline, which efficiently updates existing 3D reconstructions, such as 3D Gaussian Splatting (3DGS), with only a small set of newly acquired images. Our approach starts with camera pose estimation to align new images with the existing 3D model, followed by change detection to pinpoint modifications in the scene. A 3D generative AI model is then used to create high-quality 3D assets of the new elements, which are seamlessly integrated into the existing 3D model. The IDU pipeline incorporates human guidance to ensure high accuracy in object identification and placement, with each update focusing on a single new object at a time. Experimental results confirm that our proposed IDU pipeline significantly reduces update time and labor, offering a cost-effective and targeted solution for maintaining up-to-date 3D models in rapidly evolving military scenarios.

Method: Combines learnings from object defect-detection research with Mamba architecture’s capabilities for long-range dependencies and scalability to create a reconstruction-based approach for outdoor scene open-set segmentation.

Result: The approach improves performance when applied to both their own open-set segmentation method and existing methods, and contributes a Mamba-based architecture competitive with voxel-convolution methods on large-scale point clouds.

Conclusion: The research successfully demonstrates that combining defect-detection techniques with Mamba architecture creates an effective solution for open-set segmentation in challenging outdoor LiDAR point cloud environments.

Abstract: LiDAR scanning in outdoor scenes acquires accurate distance measurements over wide areas, producing large-scale point clouds. Application examples for this data include robotics, automotive vehicles, and land surveillance. During such applications, outlier objects from outside the training data will inevitably appear. Our research contributes a novel approach to open-set segmentation, leveraging the learnings of object defect-detection research. We also draw on the Mamba architecture’s strong performance in utilising long-range dependencies and scalability to large data. Combining both, we create a reconstruction based approach for the task of outdoor scene open-set segmentation. We show that our approach improves performance not only when applied to our our own open-set segmentation method, but also when applied to existing methods. Furthermore we contribute a Mamba based architecture which is competitive with existing voxel-convolution based methods on challenging, large-scale pointclouds.

Method: HERO uses hierarchical strategies: (i) patch-wise refresh mechanism with sampling and frequency-aware tracking for shallow layers with high temporal variability, and (ii) linear extrapolation to bypass attention and feed-forward computations in deeper stable layers.

Result: HERO achieves 1.73× speedup with minimal quality degradation, significantly outperforming existing diffusion acceleration methods.

Conclusion: The hierarchical approach effectively accelerates world model inference by leveraging the different characteristics of shallow and deep layers, providing substantial speed improvements while maintaining quality.

Abstract: Generation-driven world models create immersive virtual environments but suffer slow inference due to the iterative nature of diffusion models. While recent advances have improved diffusion model efficiency, directly applying these techniques to world models introduces limitations such as quality degradation. In this paper, we present HERO, a training-free hierarchical acceleration framework tailored for efficient world models. Owing to the multi-modal nature of world models, we identify a feature coupling phenomenon, wherein shallow layers exhibit high temporal variability, while deeper layers yield more stable feature representations. Motivated by this, HERO adopts hierarchical strategies to accelerate inference: (i) In shallow layers, a patch-wise refresh mechanism efficiently selects tokens for recomputation. With patch-wise sampling and frequency-aware tracking, it avoids extra metric computation and remain compatible with FlashAttention. (ii) In deeper layers, a linear extrapolation scheme directly estimates intermediate features. This completely bypasses the computations in attention modules and feed-forward networks. Our experiments show that HERO achieves a 1.73$\times$ speedup with minimal quality degradation, significantly outperforming existing diffusion acceleration methods.

Method: Proposes TMR detector based on template matching and regression with minimal learnable layers on frozen backbone, preserving spatial layout of exemplars.

Result: Outperforms state-of-the-art methods on RPINE, FSCD-147, and FSCD-LVIS benchmarks with strong cross-dataset generalization.

Conclusion: Template matching and regression effectively preserves structural information for few-shot pattern detection, demonstrating superior performance over prototype-based approaches.

Abstract: We address the problem of few-shot pattern detection, which aims to detect all instances of a given pattern, typically represented by a few exemplars, from an input image. Although similar problems have been studied in few-shot object counting and detection (FSCD), previous methods and their benchmarks have narrowed patterns of interest to object categories and often fail to localize non-object patterns. In this work, we propose a simple yet effective detector based on template matching and regression, dubbed TMR. While previous FSCD methods typically represent target exemplars as spatially collapsed prototypes and lose structural information, we revisit classic template matching and regression. It effectively preserves and leverages the spatial layout of exemplars through a minimalistic structure with a small number of learnable convolutional or projection layers on top of a frozen backbone We also introduce a new dataset, dubbed RPINE, which covers a wider range of patterns than existing object-centric datasets. Our method outperforms the state-of-the-art methods on the three benchmarks, RPINE, FSCD-147, and FSCD-LVIS, and demonstrates strong generalization in cross-dataset evaluation.

Method: Two-stage framework with global and region-level feature encoding from RGB/depth modalities, Mixture-of-Experts fusion module for dynamic spatial representation combination, and two-phase training strategy with free-form answer generation followed by normalized evaluation.

Result: 64M-parameter base model achieved 5th place on AI City Challenge 2025 leaderboard with 66.8861 score; 80M-parameter variant with expanded MoE capacity showed improved performance on spatial reasoning tasks.

Conclusion: TinyGiantVLM effectively bridges visual perception and spatial understanding in industrial environments through its lightweight modular design and multimodal fusion approach, demonstrating strong spatial reasoning capabilities.

Abstract: Reasoning about fine-grained spatial relationships in warehouse-scale environments poses a significant challenge for existing vision-language models (VLMs), which often struggle to comprehend 3D layouts, object arrangements, and multimodal cues in real-world industrial settings. In this paper, we present TinyGiantVLM, a lightweight and modular two-stage framework designed for physical spatial reasoning, distinguishing itself from traditional geographic reasoning in complex logistics scenes. Our approach encodes both global and region-level features from RGB and depth modalities using pretrained visual backbones. To effectively handle the complexity of high-modality inputs and diverse question types, we incorporate a Mixture-of-Experts (MoE) fusion module, which dynamically combines spatial representations to support downstream reasoning tasks and improve convergence. Training is conducted in a two-phase strategy: the first phase focuses on generating free-form answers to enhance spatial reasoning ability, while the second phase uses normalized answers for evaluation. Evaluated on Track 3 of the AI City Challenge 2025, our 64M-parameter base model achieved 5th place on the leaderboard with a score of 66.8861, demonstrating strong performance in bridging visual perception and spatial understanding in industrial environments. We further present an 80M-parameter variant with expanded MoE capacity, which demonstrates improved performance on spatial reasoning tasks.

Method: Proposes a vision-language model framework with cross-scale visual fusion to combine visual embeddings from multiple scales, and cross-scale hard pseudo-OOD sample generation to enhance detection capabilities.

Result: Experimental evaluations on three public medical datasets show superior OOD detection performance compared to existing methods.

Conclusion: The proposed framework effectively improves discrimination of challenging unknown diseases in medical images through hierarchical visual information integration and advanced sample generation strategies.

Abstract: In trustworthy medical diagnosis systems, integrating out-of-distribution (OOD) detection aims to identify unknown diseases in samples, thereby mitigating the risk of misdiagnosis. In this study, we propose a novel OOD detection framework based on vision-language models (VLMs), which integrates hierarchical visual information to cope with challenging unknown diseases that resemble known diseases. Specifically, a cross-scale visual fusion strategy is proposed to couple visual embeddings from multiple scales. This enriches the detailed representation of medical images and thus improves the discrimination of unknown diseases. Moreover, a cross-scale hard pseudo-OOD sample generation strategy is proposed to benefit OOD detection maximally. Experimental evaluations on three public medical datasets support that the proposed framework achieves superior OOD detection performance compared to existing methods. The source code is available at https://openi.pcl.ac.cn/OpenMedIA/HVL.

Method: 1) LLM-based entity interactive relationship mining using chain of thought; 2) Interactive action clustering and offset method with global/local bidirectional offsets; 3) Entity control network with semantic-guided masks and multi-scale convolutional networks for feature enhancement and fusion.

Result: CEIDM outperforms existing representative methods in both entity control and interaction control, generating images with more realistic logic, reasonable interactive relationships, and accurate interactive actions.

Conclusion: The proposed dual-control approach effectively addresses the challenge of controlling entities and their interactions in T2I generation, significantly improving image quality and interaction accuracy compared to state-of-the-art methods.

Abstract: In Text-to-Image (T2I) generation, the complexity of entities and their intricate interactions pose a significant challenge for T2I method based on diffusion model: how to effectively control entity and their interactions to produce high-quality images. To address this, we propose CEIDM, a image generation method based on diffusion model with dual controls for entity and interaction. First, we propose an entity interactive relationships mining approach based on Large Language Models (LLMs), extracting reasonable and rich implicit interactive relationships through chain of thought to guide diffusion models to generate high-quality images that are closer to realistic logic and have more reasonable interactive relationships. Furthermore, We propose an interactive action clustering and offset method to cluster and offset the interactive action features contained in each text prompts. By constructing global and local bidirectional offsets, we enhance semantic understanding and detail supplementation of original actions, making the model’s understanding of the concept of interactive “actions” more accurate and generating images with more accurate interactive actions. Finally, we design an entity control network which generates masks with entity semantic guidance, then leveraging multi-scale convolutional network to enhance entity feature and dynamic network to fuse feature. It effectively controls entities and significantly improves image quality. Experiments show that the proposed CEIDM method is better than the most representative existing methods in both entity control and their interaction control.

Method: Two approaches: 1) Sequential testing of query images against each database image using keypoint/hotspot extraction and matching, 2) Fast nearest neighbor search with competitive scoring mechanism derived from Local Naive Bayes Nearest Neighbor algorithm.

Result: Successfully applied to multiple species (zebras, giraffes, leopards, lionfish) with databases of 1000+ images. Achieved higher accuracy than published methods and processed each query image in just a few seconds.

Conclusion: HotSpotter provides a fast and accurate solution for individual animal identification that works across multiple species, making it valuable for wildlife research and conservation applications.

Abstract: We present HotSpotter, a fast, accurate algorithm for identifying individual animals against a labeled database. It is not species specific and has been applied to Grevy’s and plains zebras, giraffes, leopards, and lionfish. We describe two approaches, both based on extracting and matching keypoints or “hotspots”. The first tests each new query image sequentially against each database image, generating a score for each database image in isolation, and ranking the results. The second, building on recent techniques for instance recognition, matches the query image against the database using a fast nearest neighbor search. It uses a competitive scoring mechanism derived from the Local Naive Bayes Nearest Neighbor algorithm recently proposed for category recognition. We demonstrate results on databases of more than 1000 images, producing more accurate matches than published methods and matching each query image in just a few seconds.

Method: Proposed a weighted Vision Transformer-based multi-task learning framework that jointly predicts ADAS-Cog global score and 13 sub-scores from baseline MRI scans. Systematically investigated sub-score-specific loss weighting strategies to guide model focus on more relevant cognitive domains.

Result: Weighting strategies are group-dependent: strong weighting improves performance for MCI subjects with heterogeneous MRI patterns, while moderate weighting works better for CN subjects with lower variability. Uniform weighting underutilizes key sub-scores and limits generalization.

Conclusion: The framework provides a flexible, interpretable approach for AD prognosis using end-to-end MRI-based learning, demonstrating that adaptive loss weighting based on clinical sub-scores enhances both predictive accuracy and model interpretability.

Abstract: Prognostic modeling is essential for forecasting future clinical scores and enabling early detection of Alzheimers disease (AD). While most existing methods focus on predicting the ADAS-Cog global score, they often overlook the predictive value of its 13 sub-scores, which reflect distinct cognitive domains. Some sub-scores may exert greater influence on determining global scores. Assigning higher loss weights to these clinically meaningful sub-scores can guide the model to focus on more relevant cognitive domains, enhancing both predictive accuracy and interpretability. In this study, we propose a weighted Vision Transformer (ViT)-based multi-task learning (MTL) framework to jointly predict the ADAS-Cog global score using baseline MRI scans and its 13 sub-scores at Month 24. Our framework integrates ViT as a feature extractor and systematically investigates the impact of sub-score-specific loss weighting on model performance. Results show that our proposed weighting strategies are group-dependent: strong weighting improves performance for MCI subjects with more heterogeneous MRI patterns, while moderate weighting is more effective for CN subjects with lower variability. Our findings suggest that uniform weighting underutilizes key sub-scores and limits generalization. The proposed framework offers a flexible, interpretable approach to AD prognosis using end-to-end MRI-based learning. (Github repo link will be provided after review)

Method: Following the two-network design paradigm, the authors benchmark efficient models from image classification literature and adopt lightweight block designs in temporal convolution network backbones to create unified models with low resource requirements.

Result: Experiments on the largest public database for English words demonstrate the effectiveness and practicality of the developed lightweight models with strong recognition performance.

Conclusion: The developed lightweight architectures successfully address the hardware cost issue in VSR systems, enabling wider adoption and deployment in practical applications while maintaining competitive recognition performance.

Abstract: Visual speech recognition (VSR) systems decode spoken words from an input sequence using only the video data. Practical applications of such systems include medical assistance as well as human-machine interactions. A VSR system is typically employed in a complementary role in cases where the audio is corrupt or not available. In order to accurately predict the spoken words, these architectures often rely on deep neural networks in order to extract meaningful representations from the input sequence. While deep architectures achieve impressive recognition performance, relying on such models incurs significant computation costs which translates into increased resource demands in terms of hardware requirements and results in limited applicability in real-world scenarios where resources might be constrained. This factor prevents wider adoption and deployment of speech recognition systems in more practical applications. In this work, we aim to alleviate this issue by developing architectures for VSR that have low hardware costs. Following the standard two-network design paradigm, where one network handles visual feature extraction and another one utilizes the extracted features to classify the entire sequence, we develop lightweight end-to-end architectures by first benchmarking efficient models from the image classification literature, and then adopting lightweight block designs in a temporal convolution network backbone. We create several unified models with low resource requirements but strong recognition performance. Experiments on the largest public database for English words demonstrate the effectiveness and practicality of our developed models. Code and trained models will be made publicly available.

Method: Uses Multi-Modal Diffusion Transformer (MM-DiT) backbone with dedicated conditional pathways that fuse features within self-attention layers. Includes refined positional encodings and attention masks for spatial alignment. Employs bidirectional generation strategy for dataset construction with mask-based model and self-supervised “Try-Off” model, followed by manual curation.

Result: Achieves state-of-the-art performance on public benchmarks including DressCode, significantly outperforming existing methods in both quantitative metrics and human evaluations. Shows strong generalization in real-world applications, surpassing commercial systems.

Conclusion: JCo-MVTON successfully addresses key limitations in virtual try-on systems through its novel multi-modal diffusion transformer architecture and comprehensive dataset construction approach, delivering superior performance and real-world applicability.

Abstract: Virtual try-on systems have long been hindered by heavy reliance on human body masks, limited fine-grained control over garment attributes, and poor generalization to real-world, in-the-wild scenarios. In this paper, we propose JCo-MVTON (Jointly Controllable Multi-Modal Diffusion Transformer for Mask-Free Virtual Try-On), a novel framework that overcomes these limitations by integrating diffusion-based image generation with multi-modal conditional fusion. Built upon a Multi-Modal Diffusion Transformer (MM-DiT) backbone, our approach directly incorporates diverse control signals – such as the reference person image and the target garment image – into the denoising process through dedicated conditional pathways that fuse features within the self-attention layers. This fusion is further enhanced with refined positional encodings and attention masks, enabling precise spatial alignment and improved garment-person integration. To address data scarcity and quality, we introduce a bidirectional generation strategy for dataset construction: one pipeline uses a mask-based model to generate realistic reference images, while a symmetric ``Try-Off’’ model, trained in a self-supervised manner, recovers the corresponding garment images. The synthesized dataset undergoes rigorous manual curation, allowing iterative improvement in visual fidelity and diversity. Experiments demonstrate that JCo-MVTON achieves state-of-the-art performance on public benchmarks including DressCode, significantly outperforming existing methods in both quantitative metrics and human evaluations. Moreover, it shows strong generalization in real-world applications, surpassing commercial systems.

Method: Multi-task learning framework with Vision Transformer and Swin Transformer architectures to extract imaging features, fused with longitudinal clinical inputs from baseline and Month 6 to predict Month 24 scores.

Result: Sub-score learning improves global score prediction; Q1, Q4, and Q8 sub-scores dominate predictions but show high errors due to clinical feature dominance over MRI features, indicating model instability.

Conclusion: Study demonstrates value of sub-score informed modeling but highlights need for improved multimodal fusion and adaptive loss weighting for more balanced, interpretable, and clinically robust AD prediction frameworks.

Abstract: Accurate prediction of clinical scores is critical for early detection and prognosis of Alzheimers disease (AD). While existing approaches primarily focus on forecasting the ADAS-Cog global score, they often overlook the predictive value of its sub-scores (13 items), which capture domain-specific cognitive decline. In this study, we propose a multi task learning (MTL) framework that jointly predicts the global ADAS-Cog score and its sub-scores (13 items) at Month 24 using baseline MRI and longitudinal clinical scores from baseline and Month 6. The main goal is to examine how each sub scores particularly those associated with MRI features contribute to the prediction of the global score, an aspect largely neglected in prior MTL studies. We employ Vision Transformer (ViT) and Swin Transformer architectures to extract imaging features, which are fused with longitudinal clinical inputs to model cognitive progression. Our results show that incorporating sub-score learning improves global score prediction. Subscore level analysis reveals that a small subset especially Q1 (Word Recall), Q4 (Delayed Recall), and Q8 (Word Recognition) consistently dominates the predicted global score. However, some of these influential sub-scores exhibit high prediction errors, pointing to model instability. Further analysis suggests that this is caused by clinical feature dominance, where the model prioritizes easily predictable clinical scores over more complex MRI derived features. These findings emphasize the need for improved multimodal fusion and adaptive loss weighting to achieve more balanced learning. Our study demonstrates the value of sub score informed modeling and provides insights into building more interpretable and clinically robust AD prediction frameworks. (Github repo provided)

Method: Decouples into preference understanding (MLLM extracts global signals from reference images) and preference-guided generation (global keyword control + local cross-attention modulation in diffusion models).

Result: Outperforms prior approaches on Viper dataset and collected benchmark in both quantitative metrics and human evaluations.

Conclusion: Enables precise alignment across global attributes and local elements, opens new possibilities for dialog-based generation and MLLM-diffusion integration.

Abstract: Text-to-image (T2I) generation has greatly enhanced creative expression, yet achieving preference-aligned generation in a real-time and training-free manner remains challenging. Previous methods often rely on static, pre-collected preferences or fine-tuning, limiting adaptability to evolving and nuanced user intents. In this paper, we highlight the need for instant preference-aligned T2I generation and propose a training-free framework grounded in multimodal large language model (MLLM) priors. Our framework decouples the task into two components: preference understanding and preference-guided generation. For preference understanding, we leverage MLLMs to automatically extract global preference signals from a reference image and enrich a given prompt using structured instruction design. Our approach supports broader and more fine-grained coverage of user preferences than existing methods. For preference-guided generation, we integrate global keyword-based control and local region-aware cross-attention modulation to steer the diffusion model without additional training, enabling precise alignment across both global attributes and local elements. The entire framework supports multi-round interactive refinement, facilitating real-time and context-aware image generation. Extensive experiments on the Viper dataset and our collected benchmark demonstrate that our method outperforms prior approaches in both quantitative metrics and human evaluations, and opens up new possibilities for dialog-based generation and MLLM-diffusion integration.

Method: A practical framework using monocular consumer-grade videos from smartphones to generate accurate 3D models. Integrates 3D reconstruction, wound segmentation, tissue classification, and periwound analysis into a modular workflow from short handheld video recordings.

Result: The framework achieves millimeter-level accuracy, high-quality wound bed visualization, and reliable tissue composition analysis. Full assessments are completed in under 20 minutes, demonstrating clinical feasibility.

Conclusion: Wound3DAssist provides a practical, non-contact, automatic 3D wound assessment solution that is view-independent and robust to camera motion, suitable for real-world clinical use with consumer-grade devices.

Abstract: Managing chronic wounds remains a major healthcare challenge, with clinical assessment often relying on subjective and time-consuming manual documentation methods. Although 2D digital videometry frameworks aided the measurement process, these approaches struggle with perspective distortion, a limited field of view, and an inability to capture wound depth, especially in anatomically complex or curved regions. To overcome these limitations, we present Wound3DAssist, a practical framework for 3D wound assessment using monocular consumer-grade videos. Our framework generates accurate 3D models from short handheld smartphone video recordings, enabling non-contact, automatic measurements that are view-independent and robust to camera motion. We integrate 3D reconstruction, wound segmentation, tissue classification, and periwound analysis into a modular workflow. We evaluate Wound3DAssist across digital models with known geometry, silicone phantoms, and real patients. Results show that the framework supports high-quality wound bed visualization, millimeter-level accuracy, and reliable tissue composition analysis. Full assessments are completed in under 20 minutes, demonstrating feasibility for real-world clinical use.

Method: Developed HyTver, a hybrid loss function designed to handle data imbalance effectively without the computational overhead of hyperparameter tuning or sacrificing performance on distance-based metrics.

Result: Achieved a Dice score of 0.659 while maintaining comparable performance on distance-based metrics. Demonstrated stability when used with pre-trained models and outperformed other popular loss functions in comprehensive comparisons.

Conclusion: HyTver provides an effective solution for MS lesion segmentation imbalance problems, offering balanced performance across multiple evaluation metrics without computational complexity issues of previous approaches.

Abstract: Longitudinal Multiple Sclerosis Lesion Segmentation is a particularly challenging problem that involves both input and output imbalance in the data and segmentation. Therefore in order to develop models that are practical, one of the solutions is to develop better loss functions. Most models naively use either Dice loss or Cross-Entropy loss or their combination without too much consideration. However, one must select an appropriate loss function as the imbalance can be mitigated by selecting a proper loss function. In order to solve the imbalance problem, multiple loss functions were proposed that claimed to solve it. They come with problems of their own which include being too computationally complex due to hyperparameters as exponents or having detrimental performance in metrics other than region-based ones. We propose a novel hybrid loss called HyTver that achieves good segmentation performance while maintaining performance in other metrics. We achieve a Dice score of 0.659 while also ensuring that the distance-based metrics are comparable to other popular functions. In addition, we also evaluate the stability of the loss functions when used on a pre- trained model and perform extensive comparisons with other popular loss functions

Method: Proposed FloraSyntropy-Net framework combining federated learning with memetic algorithm for optimal base model selection (DenseNet201), a novel Deep Block for enhanced feature representation, and client-cloning strategy for scalable privacy-preserving training. Built on FloraSyntropy Archive dataset of 178,922 images across 35 plant species and 97 disease classes.

Result: Achieved 96.38% accuracy on FloraSyntropy benchmark and demonstrated exceptional generalization with 99.84% accuracy on unrelated multiclass Pest dataset.

Conclusion: Provides both a valuable new dataset resource and a robust, highly generalizable framework that advances practical large-scale agricultural AI applications for plant disease diagnosis.

Abstract: Early diagnosis of plant diseases is critical for global food safety, yet most AI solutions lack the generalization required for real-world agricultural diversity. These models are typically constrained to specific species, failing to perform accurately across the broad spectrum of cultivated plants. To address this gap, we first introduce the FloraSyntropy Archive, a large-scale dataset of 178,922 images across 35 plant species, annotated with 97 distinct disease classes. We establish a benchmark by evaluating numerous existing models on this archive, revealing a significant performance gap. We then propose FloraSyntropy-Net, a novel federated learning framework (FL) that integrates a Memetic Algorithm (MAO) for optimal base model selection (DenseNet201), a novel Deep Block for enhanced feature representation, and a client-cloning strategy for scalable, privacy-preserving training. FloraSyntropy-Net achieves a state-of-the-art accuracy of 96.38% on the FloraSyntropy benchmark. Crucially, to validate its generalization capability, we test the model on the unrelated multiclass Pest dataset, where it demonstrates exceptional adaptability, achieving 99.84% accuracy. This work provides not only a valuable new resource but also a robust and highly generalizable framework that advances the field towards practical, large-scale agricultural AI applications.

Method: Proposed TSRNet with detail-preserved feature extractor, multiple dense refinement strategy, and global-to-local loss function. Established PC-CAC dataset from real clinical data for benchmarking.

Result: Outperforms state-of-the-art approaches across multiple evaluation metrics on PC-CAC and two additional public datasets (PC-ImageCAS and PC-PTR).

Conclusion: Sets a new benchmark for point cloud-based tubular structure reconstruction with improved accuracy and structural integrity maintenance.

Abstract: Complex tubular structures are essential in medical imaging and computer-assisted diagnosis, where their integrity enhances anatomical visualization and lesion detection. However, existing segmentation algorithms struggle with structural discontinuities, particularly in severe clinical cases such as coronary artery stenosis and vessel occlusions, which leads to undesired discontinuity and compromising downstream diagnostic accuracy. Therefore, it is imperative to reconnect discontinuous structures to ensure their completeness. In this study, we explore the tubular structure completion based on point cloud for the first time and establish a Point Cloud-based Coronary Artery Completion (PC-CAC) dataset, which is derived from real clinical data. This dataset provides a novel benchmark for tubular structure completion. Additionally, we propose TSRNet, a Tubular Structure Reconnection Network that integrates a detail-preservated feature extractor, a multiple dense refinement strategy, and a global-to-local loss function to ensure accurate reconnection while maintaining structural integrity. Comprehensive experiments on our PC-CAC and two additional public datasets (PC-ImageCAS and PC-PTR) demonstrate that our method consistently outperforms state-of-the-art approaches across multiple evaluation metrics, setting a new benchmark for point cloud-based tubular structure reconstruction. Our benchmark is available at https://github.com/YaoleiQi/PCCAC.

Method: Systematic evaluation framework comparing benchmark and state-of-the-art detection models using diverse region-of-interest sizes and imaging resolutions on a multi-class diseased glomerular dataset.

Result: Intermediate patch sizes provided optimal balance between context and efficiency, while moderate magnifications enhanced generalization by reducing overfitting.

Conclusion: The study advances understanding of model capabilities and limitations, offering actionable insights for refining automated detection strategies in digital renal pathology workflows.

Abstract: Accurate detection of diseased glomeruli is fundamental to progress in renal pathology and underpins the delivery of reliable clinical diagnoses. Although recent advances in computer vision have produced increasingly sophisticated detection algorithms, the majority of research efforts have focused on normal glomeruli or instances of global sclerosis, leaving the wider spectrum of diseased glomerular subtypes comparatively understudied. This disparity is not without consequence; the nuanced and highly variable morphological characteristics that define these disease variants frequently elude even the most advanced computational models. Moreover, ongoing debate surrounds the choice of optimal imaging magnifications and region-of-view dimensions for fine-grained glomerular analysis, adding further complexity to the pursuit of accurate classification and robust segmentation. To bridge these gaps, we present M^3-GloDet, a systematic framework designed to enable thorough evaluation of detection models across a broad continuum of regions, scales, and classes. Within this framework, we evaluate both long-standing benchmark architectures and recently introduced state-of-the-art models that have achieved notable performance, using an experimental design that reflects the diversity of region-of-interest sizes and imaging resolutions encountered in routine digital renal pathology. As the results, we found that intermediate patch sizes offered the best balance between context and efficiency. Additionally, moderate magnifications enhanced generalization by reducing overfitting. Through systematic comparison of these approaches on a multi-class diseased glomerular dataset, our aim is to advance the understanding of model strengths and limitations, and to offer actionable insights for the refinement of automated detection strategies and clinical workflows in the digital pathology domain.

Method: Language-Guided Temporal Token Pruning (LGTTP) leverages temporal cues from queries to adaptively prune video tokens while preserving contextual continuity.

Result: Achieves 65% computation reduction, preserves 97-99% performance, improves HIT@1 by +9.5% on QVHighlights, retains 99.6% R@1 on Charades-STA.

Conclusion: LGTTP effectively handles queries with temporal markers and general video tasks while significantly reducing computational overhead.

Abstract: Vision Language Models (VLMs) struggle with long-form videos due to the quadratic complexity of attention mechanisms. We propose Language-Guided Temporal Token Pruning (LGTTP), which leverages temporal cues from queries to adaptively prune video tokens, preserving contextual continuity while reducing computational overhead. Unlike uniform pruning or keyframe selection, LGTTP retains higher token density in temporally relevant segments. Our model-agnostic framework integrates with TimeChat and LLaVA-Video, achieving a 65% reduction in computation while preserving 97-99% of the original performance. On QVHighlights, LGTTP improves HIT@1 by +9.5%, and on Charades-STA, it retains 99.6% of R@1. It excels on queries with explicit temporal markers and remains effective across general video understanding tasks.

Method: Developed a pipeline using 9 state-of-the-art CAM algorithms applied across multiple network stages of EfficientNetV2S, quantitatively evaluated on the Hemorica dataset with slice-level labels and segmentation masks using Dice, IoU, and pixel-wise overlap metrics.

Result: Best localization at stage 5 of EfficientNetV2S, with HiResCAM achieving highest bounding-box alignment and AblationCAM achieving best pixel-level performance (Dice: 0.57, IoU: 0.40) despite models being trained only for classification without segmentation supervision.

Conclusion: Establishes first quantitative comparison benchmark for CAM methods in brain hemorrhage detection, demonstrating strong potential of XAI-driven pipelines for clinically meaningful AI-assisted diagnosis with reproducible results.

Abstract: Explainable Artificial Intelligence (XAI) has become an essential component of medical imaging research, aiming to increase transparency and clinical trust in deep learning models. This study investigates brain hemorrhage diagnosis with a focus on explainability through Class Activation Mapping (CAM) techniques. A pipeline was developed to extract pixellevel segmentation and detection annotations from classification models using nine state-of-the-art CAM algorithms, applied across multiple network stages, and quantitatively evaluated on the Hemorica dataset, which uniquely provides both slice-level labels and high-quality segmentation masks. Metrics including Dice, IoU, and pixel-wise overlap were employed to benchmark CAM variants. Results show that the strongest localization performance occurred at stage 5 of EfficientNetV2S, with HiResCAM yielding the highest bounding-box alignment and AblationCAM achieving the best pixel-level Dice (0.57) and IoU (0.40), representing strong accuracy given that models were trained solely for classification without segmentation supervision. To the best of current knowledge, this is among the f irst works to quantitatively compare CAM methods for brain hemorrhage detection, establishing a reproducible benchmark and underscoring the potential of XAI-driven pipelines for clinically meaningful AI-assisted diagnosis.

Method: Uses a dual-branch architecture with a primary diffusion-inspired transformer for progressive feature refinement and an auxiliary transformer branch for noise robustness through learned latent contrasts. Features are fused and decoded using Residual-in-Residual Dense Blocks.

Result: Outperforms recent transformer-based and diffusion-inspired methods on benchmark datasets in both efficiency and visual image quality.

Conclusion: CATformer successfully bridges different super-resolution approaches and lays foundation for practical applications of diffusion-inspired transformers in image enhancement.

Abstract: Super-resolution remains a promising technique to enhance the quality of low-resolution images. This study introduces CATformer (Contrastive Adversarial Transformer), a novel neural network integrating diffusion-inspired feature refinement with adversarial and contrastive learning. CATformer employs a dual-branch architecture combining a primary diffusion-inspired transformer, which progressively refines latent representations, with an auxiliary transformer branch designed to enhance robustness to noise through learned latent contrasts. These complementary representations are fused and decoded using deep Residual-in-Residual Dense Blocks for enhanced reconstruction quality. Extensive experiments on benchmark datasets demonstrate that CATformer outperforms recent transformer-based and diffusion-inspired methods both in efficiency and visual image quality. This work bridges the performance gap among transformer-, diffusion-, and GAN-based methods, laying a foundation for practical applications of diffusion-inspired transformers in super-resolution.

Method: UniSino operates directly in the projection domain rather than image domain, incorporating physical characteristics of sinograms in its training framework to enable robust performance across multiple undersampling scenarios and subtasks.

Result: Experimental results show UniSino achieves superior reconstruction quality in both single and mixed undersampling cases, demonstrating exceptional robustness and generalization across four benchmark datasets.

Conclusion: UniSino provides a universal CT sinogram standardization solution with strong generalization capabilities, outperforming existing methods and offering robust performance for diverse undersampling artifacts in CT imaging.

Abstract: During raw-data acquisition in CT imaging, diverse factors can degrade the collected sinograms, with undersampling and noise leading to severe artifacts and noise in reconstructed images and compromising diagnostic accuracy. Conventional correction methods rely on manually designed algorithms or fixed empirical parameters, but these approaches often lack generalizability across heterogeneous artifact types. To address these limitations, we propose UniSino, a foundation model for universal CT sinogram standardization. Unlike existing foundational models that operate in image domain, UniSino directly standardizes data in the projection domain, which enables stronger generalization across diverse undersampling scenarios. Its training framework incorporates the physical characteristics of sinograms, enhancing generalization and enabling robust performance across multiple subtasks spanning four benchmark datasets. Experimental results demonstrate thatUniSino achieves superior reconstruction quality both single and mixed undersampling case, demonstrating exceptional robustness and generalization in sinogram enhancement for CT imaging. The code is available at: https://github.com/yqx7150/UniSino.

Method: Proposes Neural Gradient-based Deformation (NGD) method with adaptive remeshing strategy for dynamically evolving surfaces (wrinkles, pleats) and learns dynamic texture maps to capture per-frame lighting and shadow effects.

Result: Significant improvements over state-of-the-art methods in both qualitative and quantitative evaluations, achieving high-quality garment reconstructions with detailed geometry.

Conclusion: NGD successfully addresses limitations of previous methods by combining neural gradient-based deformation with adaptive remeshing, enabling accurate reconstruction of complex garment dynamics from monocular video.

Abstract: Dynamic garment reconstruction from monocular video is an important yet challenging task due to the complex dynamics and unconstrained nature of the garments. Recent advancements in neural rendering have enabled high-quality geometric reconstruction with image/video supervision. However, implicit representation methods that use volume rendering often provide smooth geometry and fail to model high-frequency details. While template reconstruction methods model explicit geometry, they use vertex displacement for deformation, which results in artifacts. Addressing these limitations, we propose NGD, a Neural Gradient-based Deformation method to reconstruct dynamically evolving textured garments from monocular videos. Additionally, we propose a novel adaptive remeshing strategy for modelling dynamically evolving surfaces like wrinkles and pleats of the skirt, leading to high-quality reconstruction. Finally, we learn dynamic texture maps to capture per-frame lighting and shadow effects. We provide extensive qualitative and quantitative evaluations to demonstrate significant improvements over existing SOTA methods and provide high-quality garment reconstructions.

Method: Proposes F2RVLM - a generative retrieval model trained with supervised fine-tuning followed by GRPO-based reinforcement learning with multi-objective rewards, plus difficulty-aware curriculum sampling to handle varying fragment complexity.

Result: F2RVLM outperforms popular Vision-Language Models in both in-domain (MLDR dataset) and real-world (WeChat-based) settings, demonstrating superior retrieval performance.

Conclusion: The proposed FFR task and F2RVLM model effectively address the challenge of retrieving coherent multimodal fragments from long-form dialogues, showing significant improvements over existing approaches.

Abstract: Traditional dialogue retrieval aims to select the most appropriate utterance or image from recent dialogue history. However, they often fail to meet users' actual needs for revisiting semantically coherent content scattered across long-form conversations. To fill this gap, we define the Fine-grained Fragment Retrieval (FFR) task, requiring models to locate query-relevant fragments, comprising both utterances and images, from multimodal long-form dialogues. As a foundation for FFR, we construct MLDR, the longest-turn multimodal dialogue retrieval dataset to date, averaging 25.45 turns per dialogue, with each naturally spanning three distinct topics. To evaluate generalization in real-world scenarios, we curate and annotate a WeChat-based test set comprising real-world multimodal dialogues with an average of 75.38 turns. Building on these resources, we explore existing generation-based Vision-Language Models (VLMs) on FFR and observe that they often retrieve incoherent utterance-image fragments. While optimized for generating responses from visual-textual inputs, these models lack explicit supervision to ensure semantic coherence within retrieved fragments. To this end, we propose F2RVLM, a generative retrieval model trained in a two-stage paradigm: (1) supervised fine-tuning to inject fragment-level retrieval knowledge, and (2) GRPO-based reinforcement learning with multi-objective rewards promoting semantic precision, relevance, and contextual coherence. To handle varying intra-fragment complexity, from locally dense to sparsely distributed, we introduce difficulty-aware curriculum sampling that ranks training instances by model-predicted difficulty and gradually exposes the model to harder samples. This boosts reasoning ability in long, multi-turn contexts. F2RVLM outperforms popular VLMs in both in-domain and real-domain settings, demonstrating superior retrieval performance.

Method: Constructs category-agnostic representation space via contrastive learning, compares test samples with small support sets, and uses generative motion augmentation with diffusion models to create diverse training samples for improved generalization.

Result: Extensive experiments on HumanAct12 dataset demonstrate state-of-the-art effectiveness in both seen and unseen category settings, with improved training efficiency and model scalability for few-shot HAAD.

Conclusion: The proposed unified framework successfully addresses scalability limitations of existing methods and enables effective anomaly detection in few-shot scenarios with both known and novel action categories.

Abstract: Human Action Anomaly Detection (HAAD) aims to identify anomalous actions given only normal action data during training. Existing methods typically follow a one-model-per-category paradigm, requiring separate training for each action category and a large number of normal samples. These constraints hinder scalability and limit applicability in real-world scenarios, where data is often scarce or novel categories frequently appear. To address these limitations, we propose a unified framework for HAAD that is compatible with few-shot scenarios. Our method constructs a category-agnostic representation space via contrastive learning, enabling AD by comparing test samples with a given small set of normal examples (referred to as the support set). To improve inter-category generalization and intra-category robustness, we introduce a generative motion augmentation strategy harnessing a diffusion-based foundation model for creating diverse and realistic training samples. Notably, to the best of our knowledge, our work is the first to introduce such a strategy specifically tailored to enhance contrastive learning for action AD. Extensive experiments on the HumanAct12 dataset demonstrate the state-of-the-art effectiveness of our approach under both seen and unseen category settings, regarding training efficiency and model scalability for few-shot HAAD.

Method: Uses graph-based visual token aggregation (VTA) treating tokens as nodes with semantic similarity, and group-wise token selection (GTS) that divides tokens into kept/removed groups guided by text tokens from final layers.

Result: Outperforms previous methods consistently across LLaVA-1.5, LLaVA-NeXT, and Video-LLaVA benchmarks, achieving superior trade-off between model performance and inference speed.

Conclusion: VISA effectively compresses visual tokens while preserving more information, enhancing inference efficiency without sacrificing performance in MLLMs.

Abstract: In this study, we introduce a novel method called group-wise \textbf{VI}sual token \textbf{S}election and \textbf{A}ggregation (VISA) to address the issue of inefficient inference stemming from excessive visual tokens in multimoal large language models (MLLMs). Compared with previous token pruning approaches, our method can preserve more visual information while compressing visual tokens. We first propose a graph-based visual token aggregation (VTA) module. VTA treats each visual token as a node, forming a graph based on semantic similarity among visual tokens. It then aggregates information from removed tokens into kept tokens based on this graph, producing a more compact visual token representation. Additionally, we introduce a group-wise token selection strategy (GTS) to divide visual tokens into kept and removed ones, guided by text tokens from the final layers of each group. This strategy progressively aggregates visual information, enhancing the stability of the visual information extraction process. We conduct comprehensive experiments on LLaVA-1.5, LLaVA-NeXT, and Video-LLaVA across various benchmarks to validate the efficacy of VISA. Our method consistently outperforms previous methods, achieving a superior trade-off between model performance and inference speed. The code is available at https://github.com/mobiushy/VISA.

Method: Proposed a deep learning framework integrating U-Net for segmentation and a classification model, using the Herlev Pap Smear Dataset. Compared performance between segmented and non-segmented images.

Result: Segmented images showed marginal improvement: precision increased by 0.41% and F1-score by 1.30%, indicating slightly more balanced classification but limited overall impact.

Conclusion: Segmentation aids feature extraction but has limited impact on classification performance. The framework serves as a supplemental tool to assist pathologists in early diagnosis.

Abstract: Cervical cancer remains a significant global health concern and a leading cause of cancer-related deaths among women. Early detection through Pap smear tests is essential to reduce mortality rates; however, the manual examination is time consuming and prone to human error. This study proposes a deep learning framework that integrates U-Net for segmentation and a classification model to enhance diagnostic performance. The Herlev Pap Smear Dataset, a publicly available cervical cell dataset, was utilized for training and evaluation. The impact of segmentation on classification performance was evaluated by comparing the model trained on segmented images and another trained on non-segmented images. Experimental results showed that the use of segmented images marginally improved the model performance on precision (about 0.41 percent higher) and F1-score (about 1.30 percent higher), which suggests a slightly more balanced classification performance. While segmentation helps in feature extraction, the results showed that its impact on classification performance appears to be limited. The proposed framework offers a supplemental tool for clinical applications, which may aid pathologists in early diagnosis.

Method: Adaptive Visual Anchoring strategy that can be integrated into existing MLLMs for adaptive compression, plus a collaborative decoding mechanism to balance global and compressed visual inputs.

Result: Extensive experiments show consistent performance improvements across various MLLMs, validating the method’s effectiveness.

Conclusion: The proposed approach effectively addresses visual redundancy in MVQA through adaptive compression and collaborative decoding, achieving significant accuracy improvements in a flexible and universal manner.

Abstract: The advancement of Multimodal Large Language Models (MLLMs) has driven significant progress in Visual Question Answering (VQA), evolving from Single to Multi Image VQA (MVQA). However, the increased number of images in MVQA inevitably introduces substantial visual redundancy that is irrelevant to question answering, negatively impacting both accuracy and efficiency. To address this issue, existing methods lack flexibility in controlling the number of compressed visual tokens and tend to produce discrete visual fragments, which hinder MLLMs’ ability to comprehend images holistically. In this paper, we propose a straightforward yet universal Adaptive Visual Anchoring strategy, which can be seamlessly integrated into existing MLLMs, offering significant accuracy improvements through adaptive compression. Meanwhile, to balance the results derived from both global and compressed visual input, we further introduce a novel collaborative decoding mechanism, enabling optimal performance. Extensive experiments validate the effectiveness of our method, demonstrating consistent performance improvements across various MLLMs. The code will be publicly available.

Method: Proposes CMFDNet with three modules: CMD module uses cross-scanning to reduce blurry boundaries, MSA module employs multi-branch parallel structure for diverse geometry recognition, and FD module establishes decoder feature dependencies to improve small polyp detection.

Result: CMFDNet outperforms six state-of-the-art methods, achieving mDice scores that exceed the best SOTA by 1.83% on ETIS dataset and 1.55% on ColonDB dataset.

Conclusion: The proposed CMFDNet architecture effectively addresses key challenges in polyp segmentation through its specialized modules, demonstrating superior performance particularly in handling boundary clarity and small polyp detection compared to existing methods.

Abstract: Automated colonic polyp segmentation is crucial for assisting doctors in screening of precancerous polyps and diagnosis of colorectal neoplasms. Although existing methods have achieved promising results, polyp segmentation remains hindered by the following limitations,including: (1) significant variation in polyp shapes and sizes, (2) indistinct boundaries between polyps and adjacent tissues, and (3) small-sized polyps are easily overlooked during the segmentation process. Driven by these practical difficulties, an innovative architecture, CMFDNet, is proposed with the CMD module, MSA module, and FD module. The CMD module, serving as an innovative decoder, introduces a cross-scanning method to reduce blurry boundaries. The MSA module adopts a multi-branch parallel structure to enhance the recognition ability for polyps with diverse geometries and scale distributions. The FD module establishes dependencies among all decoder features to alleviate the under-detection of polyps with small-scale features. Experimental results show that CMFDNet outperforms six SOTA methods used for comparison, especially on ETIS and ColonDB datasets, where mDice scores exceed the best SOTA method by 1.83% and 1.55%, respectively.

Method: Hybrid framework with fine-tuned Vision Transformer and novel edge-based module that computes variance from edge-difference maps before/after smoothing, exploiting texture and edge differences between real and AI-generated images.

Result: Achieves 97.75% accuracy and 97.77% F1-score on CIFAKE dataset, surpassing state-of-the-art models across multiple benchmarks including Artistic and Custom Curated datasets.

Conclusion: Proposed method provides a lightweight, interpretable, and effective solution suitable for real-world applications in automated content verification and digital forensics for both images and video frames.

Abstract: The rapid advancement of generative models has led to a growing prevalence of highly realistic AI-generated images, posing significant challenges for digital forensics and content authentication. Conventional detection methods mainly rely on deep learning models that extract global features, which often overlook subtle structural inconsistencies and demand substantial computational resources. To address these limitations, we propose a hybrid detection framework that combines a fine-tuned Vision Transformer (ViT) with a novel edge-based image processing module. The edge-based module computes variance from edge-difference maps generated before and after smoothing, exploiting the observation that AI-generated images typically exhibit smoother textures, weaker edges, and reduced noise compared to real images. When applied as a post-processing step on ViT predictions, this module enhances sensitivity to fine-grained structural cues while maintaining computational efficiency. Extensive experiments on the CIFAKE, Artistic, and Custom Curated datasets demonstrate that the proposed framework achieves superior detection performance across all benchmarks, attaining 97.75% accuracy and a 97.77% F1-score on CIFAKE, surpassing widely adopted state-of-the-art models. These results establish the proposed method as a lightweight, interpretable, and effective solution for both still images and video frames, making it highly suitable for real-world applications in automated content verification and digital forensics.

Method: Proposes a framework with 2D keypoint detector and 3D pose estimator. Uses transformer encoder layers to extract two key-representations combined via gated sum, and introduces pose-adaptive Mahalanobis distance loss for stable predictions across extreme poses.

Result: Achieves 99.68% AP in keypoint detection, MAE-angle of 10.62°, RMSE of 0.221m, and MAE-absolute of 0.076m in 3D pose estimation. Real-time processing at 44 FPS. Created and released new datasets for drone 2D keypoints and 3D pose.

Conclusion: DroneKey effectively addresses drone pose estimation challenges with high accuracy and real-time performance. The pose-adaptive Mahalanobis loss improves stability, and the method outperforms existing approaches while being computationally efficient.

Abstract: Estimating the 3D pose of a drone is important for anti-drone systems, but existing methods struggle with the unique challenges of drone keypoint detection. Drone propellers serve as keypoints but are difficult to detect due to their high visual similarity and diversity of poses. To address these challenges, we propose DroneKey, a framework that combines a 2D keypoint detector and a 3D pose estimator specifically designed for drones. In the keypoint detection stage, we extract two key-representations (intermediate and compact) from each transformer encoder layer and optimally combine them using a gated sum. We also introduce a pose-adaptive Mahalanobis distance in the loss function to ensure stable keypoint predictions across extreme poses. We built new datasets of drone 2D keypoints and 3D pose to train and evaluate our method, which have been publicly released. Experiments show that our method achieves an AP of 99.68% (OKS) in keypoint detection, outperforming existing methods. Ablation studies confirm that the pose-adaptive Mahalanobis loss function improves keypoint prediction stability and accuracy. Additionally, improvements in the encoder design enable real-time processing at 44 FPS. For 3D pose estimation, our method achieved an MAE-angle of 10.62{\deg}, an RMSE of 0.221m, and an MAE-absolute of 0.076m, demonstrating high accuracy and reliability. The code and dataset are available at https://github.com/kkanuseobin/DroneKey.

Method: Comprehensive empirical analysis including: 1) examining pre-training bias variation between global and local data views, 2) analyzing correlations between pre-trained model biases and downstream task biases across varying bias levels, and 3) exploring why inconsistency occurs by studying representation space convergence during adaptation.

Result: Bias measurement is highly dependent on data subsets used; no consistent trends in bias transfer were found; representation spaces of different pre-trained CLIPs converge when adapted for downstream tasks, explaining the inconsistency.

Conclusion: Current bias transfer analysis shows inconsistent patterns due to representation convergence during adaptation, offering insights for better bias mitigation practices in future research.

Abstract: The recycling of contrastive language-image pre-trained (CLIP) models as backbones for a large number of downstream tasks calls for a thorough analysis of their transferability implications, especially their well-documented reproduction of social biases and human stereotypes. How do such biases, learned during pre-training, propagate to downstream applications like visual question answering or image captioning? Do they transfer at all? We investigate this phenomenon, referred to as bias transfer in prior literature, through a comprehensive empirical analysis. Firstly, we examine how pre-training bias varies between global and local views of data, finding that bias measurement is highly dependent on the subset of data on which it is computed. Secondly, we analyze correlations between biases in the pre-trained models and the downstream tasks across varying levels of pre-training bias, finding difficulty in discovering consistent trends in bias transfer. Finally, we explore why this inconsistency occurs, showing that under the current paradigm, representation spaces of different pre-trained CLIPs tend to converge when adapted for downstream tasks. We hope this work offers valuable insights into bias behavior and informs future research to promote better bias mitigation practices.

Method: CAVIA creates a closed-loop system with three innovations: hierarchical reasoning for frame localization, cross-modal semantic bridging for targeted extraction, and confidence-driven iterative synthesis.

Result: Achieves SOTA performance: EgoSchema (65.7%, +5.3%), NExT-QA (76.1%, +2.6%), and IntentQA (73.8%, +6.9%).

Conclusion: Dynamic reasoning-perception coordination provides a scalable paradigm for video understanding, outperforming rigid pipeline approaches.

Abstract: Human video comprehension demonstrates dynamic coordination between reasoning and visual attention, adaptively focusing on query-relevant details. However, current long-form video question answering systems employ rigid pipelines that decouple reasoning from perception, leading to either information loss through premature visual abstraction or computational inefficiency through exhaustive processing. The core limitation lies in the inability to adapt visual extraction to specific reasoning requirements, different queries demand fundamentally different visual evidence from the same video content. In this work, we present CAVIA, a training-free framework that revolutionizes video understanding through reasoning, perception coordination. Unlike conventional approaches where visual processing operates independently of reasoning, CAVIA creates a closed-loop system where reasoning continuously guides visual extraction based on identified information gaps. CAVIA introduces three innovations: (1) hierarchical reasoning, guided localization to precise frames; (2) cross-modal semantic bridging for targeted extraction; (3) confidence-driven iterative synthesis. CAVIA achieves state-of-the-art performance on challenging benchmarks: EgoSchema (65.7%, +5.3%), NExT-QA (76.1%, +2.6%), and IntentQA (73.8%, +6.9%), demonstrating that dynamic reasoning-perception coordination provides a scalable paradigm for video understanding.

Method: Integrates Normalizing Flows with Model-Agnostic Meta-Learning using bi-level optimization, uncertainty-guided adaptive L2 regularization, multiscale feature processing, and precise likelihood estimation for anomaly scoring.

Result: Achieved I-AUROC scores of 95.4% on MVTec-AD and 94.6% on KSDD2 under clean conditions, maintaining robust detection above 86.8% and 92.1% respectively with 50% mislabeled training samples.

Conclusion: RAD demonstrates exceptional resilience to noisy training conditions and effectively detects subtle anomalies across diverse industrial scenarios, making it a practical solution for real-world applications.

Abstract: Anomaly detection is fundamental for ensuring quality control and operational efficiency in industrial environments, yet conventional approaches face significant challenges when training data contains mislabeled samples-a common occurrence in real-world scenarios. This paper presents RAD, a robust anomaly detection framework that integrates Normalizing Flows with Model-Agnostic Meta-Learning to address the critical challenge of label noise in industrial settings. Our approach employs a bi-level optimization strategy where meta-learning enables rapid adaptation to varying noise conditions, while uncertainty quantification guides adaptive L2 regularization to maintain model stability. The framework incorporates multiscale feature processing through pretrained feature extractors and leverages the precise likelihood estimation capabilities of Normalizing Flows for robust anomaly scoring. Comprehensive evaluation on MVTec-AD and KSDD2 datasets demonstrates superior performance, achieving I-AUROC scores of 95.4% and 94.6% respectively under clean conditions, while maintaining robust detection capabilities above 86.8% and 92.1% even when 50% of training samples are mislabeled. The results highlight RAD’s exceptional resilience to noisy training conditions and its ability to detect subtle anomalies across diverse industrial scenarios, making it a practical solution for real-world anomaly detection applications where perfect data curation is challenging.

Method: Proposes a method that still relies on distance maps but can handle partially annotated objects, making it suitable for various learning contexts including frugal learning and transfer learning.

Result: The approach leads to substantial savings in time and resources without sacrificing segmentation quality, and is implemented in a user-friendly Napari plugin.

Conclusion: The proposed method successfully addresses the limitation of requiring fully annotated datasets while maintaining the accuracy benefits of distance map-based segmentation approaches.

Abstract: The most popular networks used for cell segmentation (e.g. Cellpose, Stardist, HoverNet,…) rely on a prediction of a distance map. It yields unprecedented accuracy but hinges on fully annotated datasets. This is a serious limitation to generate training sets and perform transfer learning. In this paper, we propose a method that still relies on the distance map and handles partially annotated objects. We evaluate the performance of the proposed approach in the contexts of frugal learning, transfer learning and regular learning on regular databases. Our experiments show that it can lead to substantial savings in time and resources without sacrificing segmentation quality. The proposed algorithm is embedded in a user-friendly Napari plugin.

Method: Propose position-reweighting mechanism that adjusts attention scores based on spatial positions in the image, creating a plug-and-play solution for existing pruning frameworks.

Result: Extensive experiments show improved performance of visual token pruning with minimal computational overhead.

Conclusion: Simple position-based reweighting effectively alleviates recency bias in visual token pruning without architectural changes or extra training.

Abstract: Vision-Language Models (VLMs) typically process a significantly larger number of visual tokens compared to text tokens due to the inherent redundancy in visual signals. Visual token pruning is a promising direction to reduce the computational cost of VLMs by eliminating redundant visual tokens. The text-visual attention score is a widely adopted criterion for visual token pruning as it reflects the relevance of visual tokens to the text input. However, many sequence models exhibit a recency bias, where tokens appearing later in the sequence exert a disproportionately large influence on the model’s output. In VLMs, this bias manifests as inflated attention scores for tokens corresponding to the lower regions of the image, leading to suboptimal pruning that disproportionately retains tokens from the image bottom. In this paper, we present an extremely simple yet effective approach to alleviate the recency bias in visual token pruning. We propose a straightforward reweighting mechanism that adjusts the attention scores of visual tokens according to their spatial positions in the image. Our method, termed Position-reweighted Visual Token Pruning, is a plug-and-play solution that can be seamlessly incorporated into existing visual token pruning frameworks without any changes to the model architecture or extra training. Extensive experiments on LVLMs demonstrate that our method improves the performance of visual token pruning with minimal computational overhead.

Method: Introduces visual-semantic interaction module with Dinov2 and VGG19 for targeted distillation, temporal cooperative module for video degradation enhancement, temporal-enhanced mechanism with temporal loss, and new evaluation metrics for video fusion.

Result: Extensive experiments on public video datasets demonstrate the superiority of the proposed method over existing approaches.

Conclusion: The proposed framework successfully addresses temporal consistency issues in video fusion through explicit temporal modeling and visual-semantic collaboration, with released code available.

Abstract: Existing multi-modal fusion methods typically apply static frame-based image fusion techniques directly to video fusion tasks, neglecting inherent temporal dependencies and leading to inconsistent results across frames. To address this limitation, we propose the first video fusion framework that explicitly incorporates temporal modeling with visual-semantic collaboration to simultaneously ensure visual fidelity, semantic accuracy, and temporal consistency. First, we introduce a visual-semantic interaction module consisting of a semantic branch and a visual branch, with Dinov2 and VGG19 employed for targeted distillation, allowing simultaneous enhancement of both the visual and semantic representations. Second, we pioneer integrate the video degradation enhancement task into the video fusion pipeline by constructing a temporal cooperative module, which leverages temporal dependencies to facilitate weak information recovery. Third, to ensure temporal consistency, we embed a temporal-enhanced mechanism into the network and devise a temporal loss to guide the optimization process. Finally, we introduce two innovative evaluation metrics tailored for video fusion, aimed at assessing the temporal consistency of the generated fused videos. Extensive experimental results on public video datasets demonstrate the superiority of our method. Our code is released at https://github.com/Meiqi-Gong/TemCoCo.

Method: Used PatchCoreCL (continual learning version of PatchCore model) evaluated on BMAD dataset with image-level and pixel-level annotations for medical anomaly detection.

Result: PatchCoreCL achieved performance comparable to task-specific models with forgetting value less than 1%, demonstrating effectiveness for adaptive VAD.

Conclusion: Continual learning is feasible and promising for adaptive visual anomaly detection in medical imaging, enabling models to evolve with changing data distributions while maintaining performance.

Abstract: Visual Anomaly Detection (VAD) seeks to identify abnormal images and precisely localize the corresponding anomalous regions, relying solely on normal data during training. This approach has proven essential in domains such as manufacturing and, more recently, in the medical field, where accurate and explainable detection is critical. Despite its importance, the impact of evolving input data distributions over time has received limited attention, even though such changes can significantly degrade model performance. In particular, given the dynamic and evolving nature of medical imaging data, Continual Learning (CL) provides a natural and effective framework to incrementally adapt models while preserving previously acquired knowledge. This study explores for the first time the application of VAD models in a CL scenario for the medical field. In this work, we utilize a CL version of the well-established PatchCore model, called PatchCoreCL, and evaluate its performance using BMAD, a real-world medical imaging dataset with both image-level and pixel-level annotations. Our results demonstrate that PatchCoreCL is an effective solution, achieving performance comparable to the task-specific models, with a forgetting value less than a 1%, highlighting the feasibility and potential of CL for adaptive VAD in medical imaging.

Method: Integrates soft confident learning (assigning confidence-based weights instead of discarding uncertain samples), meta-learning (MAML framework with covariance regularization), and contrastive learning to create discriminative feature spaces. Uses IQR-based thresholding for data uncertainty and covariance regularization for model uncertainty.

Result: State-of-the-art performance across 10 datasets, outperforming existing methods on 6/7 industrial benchmarks: 99.2% I-AUROC on DTD-Synthetic, 97.2% on BTAD, and 96.3% P-AUROC on MVTec-AD for pixel-level localization.

Conclusion: CoZAD provides an effective zero-shot solution that eliminates dependence on vision-language alignments or model ensembles, making it suitable for resource-constrained environments requiring rapid deployment without labeled anomaly data.

Abstract: Industrial and medical anomaly detection faces critical challenges from data scarcity and prohibitive annotation costs, particularly in evolving manufacturing and healthcare settings. To address this, we propose CoZAD, a novel zero-shot anomaly detection framework that integrates soft confident learning with meta-learning and contrastive feature representation. Unlike traditional confident learning that discards uncertain samples, our method assigns confidence-based weights to all training data, preserving boundary information while emphasizing prototypical normal patterns. The framework quantifies data uncertainty through IQR-based thresholding and model uncertainty via covariance based regularization within a Model-Agnostic Meta-Learning. Contrastive learning creates discriminative feature spaces where normal patterns form compact clusters, enabling rapid domain adaptation. Comprehensive evaluation across 10 datasets spanning industrial and medical domains demonstrates state-of-the-art performance, outperforming existing methods on 6 out of 7 industrial benchmarks with notable improvements on texture-rich datasets (99.2% I-AUROC on DTD-Synthetic, 97.2% on BTAD) and pixellevel localization (96.3% P-AUROC on MVTec-AD). The framework eliminates dependence on vision-language alignments or model ensembles, making it valuable for resourceconstrained environments requiring rapid deployment.

Method: Hierarchical Layout Generation (HLG) with coarse-to-fine approach, fine-grained layout alignment module (vertical/horizontal decoupling), and trainable layout optimization network to fix placement issues.

Result: Superior performance in generating realistic indoor scenes compared to existing methods, demonstrated through extensive experiments.

Conclusion: HLG advances scene generation field and enables detailed 3D environments for various applications; code will be released to encourage future research.

Abstract: Realistic 3D indoor scene generation is crucial for virtual reality, interior design, embodied intelligence, and scene understanding. While existing methods have made progress in coarse-scale furniture arrangement, they struggle to capture fine-grained object placements, limiting the realism and utility of generated environments. This gap hinders immersive virtual experiences and detailed scene comprehension for embodied AI applications. To address these issues, we propose Hierarchical Layout Generation (HLG), a novel method for fine-grained 3D scene generation. HLG is the first to adopt a coarse-to-fine hierarchical approach, refining scene layouts from large-scale furniture placement to intricate object arrangements. Specifically, our fine-grained layout alignment module constructs a hierarchical layout through vertical and horizontal decoupling, effectively decomposing complex 3D indoor scenes into multiple levels of granularity. Additionally, our trainable layout optimization network addresses placement issues, such as incorrect positioning, orientation errors, and object intersections, ensuring structurally coherent and physically plausible scene generation. We demonstrate the effectiveness of our approach through extensive experiments, showing superior performance in generating realistic indoor scenes compared to existing methods. This work advances the field of scene generation and opens new possibilities for applications requiring detailed 3D environments. We will release our code upon publication to encourage future research.

Method: Uses Adaptive Data Augment and Selection (ADAS) module for valuable data selection via adversarial augment strategy, and Text Fusion Module (TFM) that combines camouflage knowledge with text-visual interaction. Built new RefTextCOD dataset.

Result: Extensive experiments show the method surpasses previous semi-supervised COD methods and achieves state-of-the-art performance.

Conclusion: SCOUT effectively addresses the annotation cost problem in COD through innovative data selection and text-visual fusion, demonstrating superior performance over existing semi-supervised approaches.

Abstract: The difficulty of pixel-level annotation has significantly hindered the development of the Camouflaged Object Detection (COD) field. To save on annotation costs, previous works leverage the semi-supervised COD framework that relies on a small number of labeled data and a large volume of unlabeled data. We argue that there is still significant room for improvement in the effective utilization of unlabeled data. To this end, we introduce a Semi-supervised Camouflaged Object Detection by Utilizing Text and Adaptive Data Selection (SCOUT). It includes an Adaptive Data Augment and Selection (ADAS) module and a Text Fusion Module (TFM). The ADSA module selects valuable data for annotation through an adversarial augment and sampling strategy. The TFM module further leverages the selected valuable data by combining camouflage-related knowledge and text-visual interaction. To adapt to this work, we build a new dataset, namely RefTextCOD. Extensive experiments show that the proposed method surpasses previous semi-supervised methods in the COD field and achieves state-of-the-art performance. Our code will be released at https://github.com/Heartfirey/SCOUT.

Method: Uses latent diffusion models conditioned on medical text descriptions and spatial masks to synthesize abnormalities via inpainting on normal images, with dynamic quality validation through a latent-space segmentation network for accurate localization and single-step inference.

Result: Achieves state-of-the-art performance on three medical imaging benchmarks (CVC-ClinicDB, Kvasir-SEG, REFUGE2) with 8-10% Dice improvements over baselines and reduces false negative rates by up to 28% for challenging cases.

Conclusion: The proposed framework effectively addresses the scarcity of annotated pathological data by generating diverse abnormality types through text-guided diffusion and maintaining quality through automatic validation, significantly improving segmentation performance for rare medical conditions.

Abstract: Medical image segmentation models struggle with rare abnormalities due to scarce annotated pathological data. We propose DiffAug a novel framework that combines textguided diffusion-based generation with automatic segmentation validation to address this challenge. Our proposed approach uses latent diffusion models conditioned on medical text descriptions and spatial masks to synthesize abnormalities via inpainting on normal images. Generated samples undergo dynamic quality validation through a latentspace segmentation network that ensures accurate localization while enabling single-step inference. The text prompts, derived from medical literature, guide the generation of diverse abnormality types without requiring manual annotation. Our validation mechanism filters synthetic samples based on spatial accuracy, maintaining quality while operating efficiently through direct latent estimation. Evaluated on three medical imaging benchmarks (CVC-ClinicDB, Kvasir-SEG, REFUGE2), our framework achieves state-of-the-art performance with 8-10% Dice improvements over baselines and reduces false negative rates by up to 28% for challenging cases like small polyps and flat lesions critical for early detection in screening applications.

Method: Proposes Alternating Training-based Label Smoothing (ATLaS) that alternately trains with standard one-hot labels and soft labels. Introduces two types of offline soft labels: Class-wise Soft Labels (CSL) and Instance-wise Soft Labels (ISL) to provide inter-class and instance-class relationships.

Result: Extensive experiments show ATLaS with CSL and ISL consistently enhances generalization performance of prompt tuning. The method exhibits high compatibility with prevalent prompt tuning methods.

Conclusion: ATLaS effectively integrates label smoothing with prompt tuning, improving generalization while maintaining parameter efficiency and compatibility with existing methods.

Abstract: Recent advances in pre-trained vision-language models have demonstrated remarkable zero-shot generalization capabilities. To further enhance these models’ adaptability to various downstream tasks, prompt tuning has emerged as a parameter-efficient fine-tuning method. However, despite its efficiency, the generalization ability of prompt remains limited. In contrast, label smoothing (LS) has been widely recognized as an effective regularization technique that prevents models from becoming over-confident and improves their generalization. This inspires us to explore the integration of LS with prompt tuning. However, we have observed that the vanilla LS even weakens the generalization ability of prompt tuning. To address this issue, we propose the Alternating Training-based Label Smoothing (ATLaS) method, which alternately trains with standard one-hot labels and soft labels generated by LS to supervise the prompt tuning. Moreover, we introduce two types of efficient offline soft labels, including Class-wise Soft Labels (CSL) and Instance-wise Soft Labels (ISL), to provide inter-class or instance-class relationships for prompt tuning. The theoretical properties of the proposed ATLaS method are analyzed. Extensive experiments demonstrate that the proposed ATLaS method, combined with CSL and ISL, consistently enhances the generalization performance of prompt tuning. Moreover, the proposed ATLaS method exhibits high compatibility with prevalent prompt tuning methods, enabling seamless integration into existing methods.

Method: Proposes class-balanced sampling to select more minority class objects for labeling, and task-aware soft pseudo labeling to increase pseudo label accuracy.

Result: Achieves state-of-the-art performance on public benchmarking datasets by creating more balanced training data with improved pseudo labels.

Conclusion: The proposed class-balanced sampling and soft pseudo labeling strategies effectively address class imbalance in box-level active learning, leading to better object detection models with reduced annotation costs.

Abstract: Training deep object detectors demands expensive bounding box annotation. Active learning (AL) is a promising technique to alleviate the annotation burden. Performing AL at box-level for object detection, i.e., selecting the most informative boxes to label and supplementing the sparsely-labelled image with pseudo labels, has been shown to be more cost-effective than selecting and labelling the entire image. In box-level AL for object detection, we observe that models at early stage can only perform well on majority classes, making the pseudo labels severely class-imbalanced. We propose a class-balanced sampling strategy to select more objects from minority classes for labelling, so as to make the final training data, \ie, ground truth labels obtained by AL and pseudo labels, more class-balanced to train a better model. We also propose a task-aware soft pseudo labelling strategy to increase the accuracy of pseudo labels. We evaluate our method on public benchmarking datasets and show that our method achieves state-of-the-art performance.

Method: Uses existing 3DGS models to render novel views and introduces iterative optimization with epipolar geometric constraints between query and multiple rendered images. Allows flexible choice of feature extractors and matchers.

Result: Achieves 53.3% and 56.9% reductions in median translation and rotation errors on 7-Scenes dataset, and 40.7% and 53.2% reductions on Cambridge Landmarks dataset, outperforming state-of-the-art methods.

Conclusion: The proposed GS-SMC framework provides an effective and lightweight solution for camera pose refinement that leverages existing 3DGS models and geometric constraints, demonstrating superior performance across multiple benchmark datasets.

Abstract: Camera pose refinement aims at improving the accuracy of initial pose estimation for applications in 3D computer vision. Most refinement approaches rely on 2D-3D correspondences with specific descriptors or dedicated networks, requiring reconstructing the scene again for a different descriptor or fully retraining the network for each scene. Some recent methods instead infer pose from feature similarity, but their lack of geometry constraints results in less accuracy. To overcome these limitations, we propose a novel camera pose refinement framework leveraging 3D Gaussian Splatting (3DGS), referred to as GS-SMC. Given the widespread usage of 3DGS, our method can employ an existing 3DGS model to render novel views, providing a lightweight solution that can be directly applied to diverse scenes without additional training or fine-tuning. Specifically, we introduce an iterative optimization approach, which refines the camera pose using epipolar geometric constraints among the query and multiple rendered images. Our method allows flexibly choosing feature extractors and matchers to establish these constraints. Extensive empirical evaluations on the 7-Scenes and the Cambridge Landmarks datasets demonstrate that our method outperforms state-of-the-art camera pose refinement approaches, achieving 53.3% and 56.9% reductions in median translation and rotation errors on 7-Scenes, and 40.7% and 53.2% on Cambridge.

Method: Assessed various CAM methods’ resilience to noise perturbations across multiple architectures and datasets, analyzing noise influence on explanations and proposing a robustness metric with two properties: consistency (stability under non-class-changing perturbations) and responsiveness (sensitivity to prediction changes).

Result: Found considerable variability in noise sensitivity for different CAMs. The proposed metric was empirically evaluated across models, perturbations, and datasets with complementary statistical tests.

Conclusion: The study highlights the need for robust CAM evaluation and provides a practical metric to assess CAM stability and sensitivity to noise perturbations in deep learning explanations.

Abstract: Class Activation Maps (CAMs) are one of the important methods for visualizing regions used by deep learning models. Yet their robustness to different noise remains underexplored. In this work, we evaluate and report the resilience of various CAM methods for different noise perturbations across multiple architectures and datasets. By analyzing the influence of different noise types on CAM explanations, we assess the susceptibility to noise and the extent to which dataset characteristics may impact explanation stability. The findings highlight considerable variability in noise sensitivity for various CAMs. We propose a robustness metric for CAMs that captures two key properties: consistency and responsiveness. Consistency reflects the ability of CAMs to remain stable under input perturbations that do not alter the predicted class, while responsiveness measures the sensitivity of CAMs to changes in the prediction caused by such perturbations. The metric is evaluated empirically across models, different perturbations, and datasets along with complementary statistical tests to exemplify the applicability of our proposed approach.

Method: Uses Hybrid Illumination and Semantics-Aware Multi-Headed Self-Attention (HISA-MSA) with two self-attention modules for independent processing of illumination and semantic features, plus a Mixture of Experts-based Feed-Forward Network with gating mechanism and low-rank matrix adaptations (LoRA).

Result: Extensive evaluations show ISALux is competitive with state-of-the-art methods across multiple specialized datasets, with ablation studies confirming the contribution of each component.

Conclusion: ISALux effectively integrates illumination and semantic priors for low-light image enhancement, demonstrating competitive performance while addressing overfitting issues through innovative architectural design.

Abstract: We introduce ISALux, a novel transformer-based approach for Low-Light Image Enhancement (LLIE) that seamlessly integrates illumination and semantic priors. Our architecture includes an original self-attention block, Hybrid Illumination and Semantics-Aware Multi-Headed Self- Attention (HISA-MSA), which integrates illumination and semantic segmentation maps for en- hanced feature extraction. ISALux employs two self-attention modules to independently process illumination and semantic features, selectively enriching each other to regulate luminance and high- light structural variations in real-world scenarios. A Mixture of Experts (MoE)-based Feed-Forward Network (FFN) enhances contextual learning, with a gating mechanism conditionally activating the top K experts for specialized processing. To address overfitting in LLIE methods caused by distinct light patterns in benchmarking datasets, we enhance the HISA-MSA module with low-rank matrix adaptations (LoRA). Extensive qualitative and quantitative evaluations across multiple specialized datasets demonstrate that ISALux is competitive with state-of-the-art (SOTA) methods. Addition- ally, an ablation study highlights the contribution of each component in the proposed model. Code will be released upon publication.

Method: UniAPO uses EM-inspired optimization that decouples feedback modeling and prompt refinement. It introduces short-long term memory mechanism where historical feedback mitigates context limitations and historical prompts provide directional guidance.

Result: UniAPO achieves consistent performance gains across text, image, and video benchmarks, demonstrating effective and transferable prompt optimization in multimodal settings.

Conclusion: The framework establishes a unified approach for efficient multimodal prompt optimization, successfully addressing the core challenges of visual token inflation and process-level supervision in multimodal tasks.

Abstract: Prompting is fundamental to unlocking the full potential of large language models. To automate and enhance this process, automatic prompt optimization (APO) has been developed, demonstrating effectiveness primarily in text-only input scenarios. However, extending existing APO methods to multimodal tasks, such as video-language generation introduces two core challenges: (i) visual token inflation, where long visual token sequences restrict context capacity and result in insufficient feedback signals; (ii) a lack of process-level supervision, as existing methods focus on outcome-level supervision and overlook intermediate supervision, limiting prompt optimization. We present UniAPO: Unified Multimodal Automated Prompt Optimization, the first framework tailored for multimodal APO. UniAPO adopts an EM-inspired optimization process that decouples feedback modeling and prompt refinement, making the optimization more stable and goal-driven. To further address the aforementioned challenges, we introduce a short-long term memory mechanism: historical feedback mitigates context limitations, while historical prompts provide directional guidance for effective prompt optimization. UniAPO achieves consistent gains across text, image, and video benchmarks, establishing a unified framework for efficient and transferable prompt optimization.

Method: Proposes EndoUFM framework that integrates dual foundation models with: 1) Random Vector Low-Rank Adaptation (RVLoRA) for adaptive fine-tuning, 2) Residual block based on Depthwise Separable Convolution (Res-DSC) for capturing fine-grained local features, and 3) mask-guided smoothness loss for depth consistency within anatomical tissues.

Result: Extensive experiments on SCARED, Hamlyn, SERV-CT, and EndoNeRF datasets confirm state-of-the-art performance while maintaining efficient model size.

Conclusion: The framework enhances surgeons’ spatial perception during minimally invasive procedures, improving surgical precision and safety, with important implications for augmented reality and navigation systems in surgery.

Abstract: Depth estimation is a foundational component for 3D reconstruction in minimally invasive endoscopic surgeries. However, existing monocular depth estimation techniques often exhibit limited performance to the varying illumination and complex textures of the surgical environment. While powerful visual foundation models offer a promising solution, their training on natural images leads to significant domain adaptability limitations and semantic perception deficiencies when applied to endoscopy. In this study, we introduce EndoUFM, an unsupervised monocular depth estimation framework that innovatively integrating dual foundation models for surgical scenes, which enhance the depth estimation performance by leveraging the powerful pre-learned priors. The framework features a novel adaptive fine-tuning strategy that incorporates Random Vector Low-Rank Adaptation (RVLoRA) to enhance model adaptability, and a Residual block based on Depthwise Separable Convolution (Res-DSC) to improve the capture of fine-grained local features. Furthermore, we design a mask-guided smoothness loss to enforce depth consistency within anatomical tissue structures. Extensive experiments on the SCARED, Hamlyn, SERV-CT, and EndoNeRF datasets confirm that our method achieves state-of-the-art performance while maintaining an efficient model size. This work contributes to augmenting surgeons’ spatial perception during minimally invasive procedures, thereby enhancing surgical precision and safety, with crucial implications for augmented reality and navigation systems.

Method: Created a comprehensive dataset with 3600 synchronized video recordings from 600 subjects using multiple consumer-grade cameras at different angles, paired with 100 Hz PPG signals and extended health metrics including ECG, blood pressure, biomarkers, temperature, oxygen saturation, respiratory rate, and stress levels.

Result: Trained an efficient rPPG model using this data and compared its quality with existing approaches in cross-dataset scenarios, demonstrating improved performance.

Conclusion: The public release of this dataset and model should significantly accelerate progress in developing AI medical assistants for remote physiological monitoring.

Abstract: Progress in remote PhotoPlethysmoGraphy (rPPG) is limited by the critical issues of existing publicly available datasets: small size, privacy concerns with facial videos, and lack of diversity in conditions. The paper introduces a novel comprehensive large-scale multi-view video dataset for rPPG and health biomarkers estimation. Our dataset comprises 3600 synchronized video recordings from 600 subjects, captured under varied conditions (resting and post-exercise) using multiple consumer-grade cameras at different angles. To enable multimodal analysis of physiological states, each recording is paired with a 100 Hz PPG signal and extended health metrics, such as electrocardiogram, arterial blood pressure, biomarkers, temperature, oxygen saturation, respiratory rate, and stress level. Using this data, we train an efficient rPPG model and compare its quality with existing approaches in cross-dataset scenarios. The public release of our dataset and model should significantly speed up the progress in the development of AI medical assistants.

Method: Bias-Mitigation Continual Learning (BRAIN) approach with De-bias Contrastive Learning loss function and Angular-based Forgetting Mitigation to prevent catastrophic forgetting while addressing bias in continual learning setup.

Result: Achieves State-of-the-Art performance across various benchmarks, surpassing both prior methods and non-continual learning approaches.

Conclusion: The proposed BRAIN framework effectively addresses the challenge of inconsistent brain signals over time through bias mitigation and forgetting prevention, demonstrating superior performance in vision-brain understanding tasks.

Abstract: Memory decay makes it harder for the human brain to recognize visual objects and retain details. Consequently, recorded brain signals become weaker, uncertain, and contain poor visual context over time. This paper presents one of the first vision-learning approaches to address this problem. First, we statistically and experimentally demonstrate the existence of inconsistency in brain signals and its impact on the Vision-Brain Understanding (VBU) model. Our findings show that brain signal representations shift over recording sessions, leading to compounding bias, which poses challenges for model learning and degrades performance. Then, we propose a new Bias-Mitigation Continual Learning (BRAIN) approach to address these limitations. In this approach, the model is trained in a continual learning setup and mitigates the growing bias from each learning step. A new loss function named De-bias Contrastive Learning is also introduced to address the bias problem. In addition, to prevent catastrophic forgetting, where the model loses knowledge from previous sessions, the new Angular-based Forgetting Mitigation approach is introduced to preserve learned knowledge in the model. Finally, the empirical experiments demonstrate that our approach achieves State-of-the-Art (SOTA) performance across various benchmarks, surpassing prior and non-continual learning methods.

Method: Used 22 subjects’ CT scans, simulated sparse sampling with Astra toolbox for parallel, fan, and cone beam geometries. Trained and validated 2D and 3D U-Nets on different data dimensions (2D, 2.5D, 3D) with 64x64x64 voxel blocks. Assessed performance using MSE and SSIM metrics.

Result: For all beam geometries, the 2D U-Net trained on axial 2D slices achieved the best performance in both MSE and SSIM, outperforming models using 2.5D and 3D input data dimensions.

Conclusion: 2D U-Net approaches are superior to volumetric (2.5D and 3D) methods for sparse-sampling artifact correction in CT scans, suggesting that 2D processing maintains optimal performance despite the availability of 3D context.

Abstract: This study aims to investigate the effect of various beam geometries and dimensions of input data on the sparse-sampling streak artifact correction task with U-Nets for clinical CT scans as a means of incorporating the volumetric context into artifact reduction tasks to improve model performance. A total of 22 subjects were retrospectively selected (01.2016-12.2018) from the Technical University of Munich’s research hospital, TUM Klinikum rechts der Isar. Sparsely-sampled CT volumes were simulated with the Astra toolbox for parallel, fan, and cone beam geometries. 2048 views were taken as full-view scans. 2D and 3D U-Nets were trained and validated on 14, and tested on 8 subjects, respectively. For the dimensionality study, in addition to the 512x512 2D CT images, the CT scans were further pre-processed to generate a so-called ‘2.5D’, and 3D data: Each CT volume was divided into 64x64x64 voxel blocks. The 3D data refers to individual 64-voxel blocks. An axial, coronal, and sagittal cut through the center of each block resulted in three 64x64 2D patches that were rearranged as a single 64x64x3 image, proposed as 2.5D data. Model performance was assessed with the mean squared error (MSE) and structural similarity index measure (SSIM). For all geometries, the 2D U-Net trained on axial 2D slices results in the best MSE and SSIM values, outperforming the 2.5D and 3D input data dimensions.

Method: Developed Obz AI - a comprehensive software ecosystem with seamless integration pipeline from Python client library to full-stack analytics dashboard, enabling incorporation of advanced XAI methodologies, feature extraction for outlier detection, and real-time model monitoring.

Result: Obz AI provides state-of-the-art explainability and observability for vision AI systems, making deep model decision-making mechanisms interpretable and facilitating responsible deployment.

Conclusion: Obz AI successfully closes the integration gap between XAI techniques and practical CV deployments, promoting observability and responsible deployment of computer vision systems through its comprehensive software ecosystem.

Abstract: Deep learning has transformed computer vision (CV), achieving outstanding performance in classification, segmentation, and related tasks. Such AI-based CV systems are becoming prevalent, with applications spanning from medical imaging to surveillance. State of the art models such as convolutional neural networks (CNNs) and vision transformers (ViTs) are often regarded as ``black boxes,’’ offering limited transparency into their decision-making processes. Despite a recent advancement in explainable AI (XAI), explainability remains underutilized in practical CV deployments. A primary obstacle is the absence of integrated software solutions that connect XAI techniques with robust knowledge management and monitoring frameworks. To close this gap, we have developed Obz AI, a comprehensive software ecosystem designed to facilitate state-of-the-art explainability and observability for vision AI systems. Obz AI provides a seamless integration pipeline, from a Python client library to a full-stack analytics dashboard. With Obz AI, a machine learning engineer can easily incorporate advanced XAI methodologies, extract and analyze features for outlier detection, and continuously monitor AI models in real time. By making the decision-making mechanisms of deep models interpretable, Obz AI promotes observability and responsible deployment of computer vision systems.

Method: The method first computes a neural scene representation from anchor images, then fine-tunes the regression network to reconstruct all input images conditioned on this representation, combining scene regression with visual localization.

Result: SAIL-Recon achieves state-of-the-art results on camera pose estimation and novel view synthesis benchmarks (TUM-RGBD, CO3Dv2, Tanks & Temples) while efficiently scaling to large-scale scenes.

Conclusion: The proposed approach successfully addresses the scalability limitations of scene regression methods and demonstrates superior performance across multiple benchmarks, with code and models made publicly available.

Abstract: Scene regression methods, such as VGGT, solve the Structure-from-Motion (SfM) problem by directly regressing camera poses and 3D scene structures from input images. They demonstrate impressive performance in handling images under extreme viewpoint changes. However, these methods struggle to handle a large number of input images. To address this problem, we introduce SAIL-Recon, a feed-forward Transformer for large scale SfM, by augmenting the scene regression network with visual localization capabilities. Specifically, our method first computes a neural scene representation from a subset of anchor images. The regression network is then fine-tuned to reconstruct all input images conditioned on this neural scene representation. Comprehensive experiments show that our method not only scales efficiently to large-scale scenes, but also achieves state-of-the-art results on both camera pose estimation and novel view synthesis benchmarks, including TUM-RGBD, CO3Dv2, and Tanks & Temples. We will publish our model and code. Code and models are publicly available at: https://hkust-sail.github.io/ sail-recon/.

Method: Novel hybrid architecture using YOLOv8 for fast detection and a five-layer CNN for verification, trained with thousands of synthetic road environment images to enhance robustness in drifted conditions.

Result: The system achieved over 90% improvement in detection accuracy when tested with drift-augmented road images, demonstrating significant performance enhancement in challenging scenarios.

Conclusion: The hybrid model structure effectively provides better road safety by maintaining high detection accuracy in unseen drifted environments, addressing ISO 8800 norm requirements for AI risk management in automotive applications.

Abstract: The use of computer vision in automotive is a trending research in which safety and security are a primary concern. In particular, for autonomous driving, preventing road accidents requires highly accurate object detection under diverse conditions. To address this issue, recently the International Organization for Standardization (ISO) released the 8800 norm, providing structured frameworks for managing associated AI relevant risks. However, challenging scenarios such as adverse weather or low lighting often introduce data drift, leading to degraded model performance and potential safety violations. In this work, we present a novel hybrid computer vision architecture trained with thousands of synthetic image data from the road environment to improve robustness in unseen drifted environments. Our dual mode framework utilized YOLO version 8 for swift detection and incorporated a five-layer CNN for verification. The system functioned in sequence and improved the detection accuracy by more than 90% when tested with drift-augmented road images. The focus was to demonstrate how such a hybrid model can provide better road safety when working together in a hybrid structure.

Method: Uses Domain-Specific Training to partition training set into multiple domains with lower anomaly ratios, trains domain-specific students, then performs Cross-Domain Knowledge Aggregation where pseudo-normal features guide a global student to learn generalized normal representations.

Result: Achieves significant performance improvements over baseline methods on noisy versions of MVTec AD and VisA datasets.

Conclusion: The proposed Cross-Domain Distillation framework effectively addresses the challenges of FUAD by preventing learning of anomalous representations and enabling robust anomaly detection without any labels.

Abstract: Fully Unsupervised Anomaly Detection (FUAD) is a practical extension of Unsupervised Anomaly Detection (UAD), aiming to detect anomalies without any labels even when the training set may contain anomalous samples. To achieve FUAD, we pioneer the introduction of Knowledge Distillation (KD) paradigm based on teacher-student framework into the FUAD setting. However, due to the presence of anomalies in the training data, traditional KD methods risk enabling the student to learn the teacher’s representation of anomalies under FUAD setting, thereby resulting in poor anomaly detection performance. To address this issue, we propose a novel Cross-Domain Distillation (CDD) framework based on the widely studied reverse distillation (RD) paradigm. Specifically, we design a Domain-Specific Training, which divides the training set into multiple domains with lower anomaly ratios and train a domain-specific student for each. Cross-Domain Knowledge Aggregation is then performed, where pseudo-normal features generated by domain-specific students collaboratively guide a global student to learn generalized normal representations across all samples. Experimental results on noisy versions of the MVTec AD and VisA datasets demonstrate that our method achieves significant performance improvements over the baseline, validating its effectiveness under FUAD setting.

Method: Built a custom dataset of 3,468 images and trained an SSD neural network model for cash recognition.

Result: The system achieved a Mean Average Precision score of over 90% in accurately identifying Nigerian cash.

Conclusion: The system has strong potential to contribute to assistive technology and improve quality of life for visually impaired people in Nigeria and beyond.

Abstract: Neural networks in assistive technology for visually impaired leverage artificial intelligence’s capacity to recognize patterns in complex data. They are used for converting visual data into auditory or tactile representations, helping the visually impaired understand their surroundings. The primary aim of this research is to explore the potential of artificial neural networks to facilitate the differentiation of various forms of cash for individuals with visual impairments. In this study, we built a custom dataset of 3,468 images, which was subsequently used to train an SSD neural network model. The proposed system can accurately identify Nigerian cash, thereby streamlining commercial transactions. The performance of the system in terms of accuracy was assessed, and the Mean Average Precision score was over 90%. We believe that our system has the potential to make a substantial contribution to the field of assistive technology while also improving the quality of life of visually challenged persons in Nigeria and beyond.

Method: Used various generative models (CycleGANs, cGANs) fine-tuned to generate realistic images across skull and face domains from 2D X-ray inputs, with a retrieval framework for matching generated faces to real databases.

Result: Evaluated using FID, IS, and SSIM scores showing quality generation, and demonstrated effective forensic application through experimental results.

Conclusion: The framework provides an effective tool for forensic science, being the first to use 2D X-rays with generative models for craniofacial reconstruction.

Abstract: Craniofacial reconstruction in forensics is one of the processes to identify victims of crime and natural disasters. Identifying an individual from their remains plays a crucial role when all other identification methods fail. Traditional methods for this task, such as clay-based craniofacial reconstruction, require expert domain knowledge and are a time-consuming process. At the same time, other probabilistic generative models like the statistical shape model or the Basel face model fail to capture the skull and face cross-domain attributes. Looking at these limitations, we propose a generic framework for craniofacial reconstruction from 2D X-ray images. Here, we used various generative models (i.e., CycleGANs, cGANs, etc) and fine-tune the generator and discriminator parts to generate more realistic images in two distinct domains, which are the skull and face of an individual. This is the first time where 2D X-rays are being used as a representation of the skull by generative models for craniofacial reconstruction. We have evaluated the quality of generated faces using FID, IS, and SSIM scores. Finally, we have proposed a retrieval framework where the query is the generated face image and the gallery is the database of real faces. By experimental results, we have found that this can be an effective tool for forensic science.

Method: Proposes Visual-Chain of Guidance (Visual-CoG) paradigm with three stages: semantic reasoning, process refining, and outcome evaluation, with stage-aware rewards providing immediate guidance throughout the image generation pipeline. Also constructs VisCog-Bench benchmark for evaluation.

Result: Comprehensive evaluations show improvements of 15% on GenEval, 5% on T2I-CompBench, and 19% on the proposed VisCog-Bench, demonstrating superior performance.

Conclusion: The Visual-CoG paradigm with stage-aware rewards throughout the generation pipeline effectively addresses limitations of final-only guidance and significantly improves text-to-image generation performance, particularly for multi-attribute and ambiguous prompts.

Abstract: Despite the promising progress of recent autoregressive models in text-to-image (T2I) generation, their ability to handle multi-attribute and ambiguous prompts remains limited. To address these limitations, existing works have applied chain-of-thought (CoT) to enable stage-aware visual synthesis and employed reinforcement learning (RL) to improve reasoning capabilities. However, most models provide reward signals only at the end of the generation stage. This monolithic final-only guidance makes it difficult to identify which stages contribute positively to the final outcome and may lead to suboptimal policies. To tackle this issue, we propose a Visual-Chain of Guidance (Visual-CoG) paradigm consisting of three stages: semantic reasoning, process refining, and outcome evaluation, with stage-aware rewards providing immediate guidance throughout the image generation pipeline. We further construct a visual cognition benchmark, VisCog-Bench, which comprises four subtasks to evaluate the effectiveness of semantic reasoning. Comprehensive evaluations on GenEval, T2I-CompBench, and the proposed VisCog-Bench show improvements of 15%, 5%, and 19%, respectively, demonstrating the superior performance of the proposed Visual-CoG. We will release all the resources soon.

Method: Three-stage framework: 1) Conjecture - global reasoning with cross-modal fusion (RGB, depth, semantic maps); 2) Focus - omnidirectional attention-driven scanning; 3) Sculpting - iterative point prompt generation for high-fidelity masks.

Result: Achieves state-of-the-art performance on four COS benchmarks and three medical image segmentation benchmarks, demonstrating exceptional efficacy, superior generalization, and robustness.

Conclusion: The cognitive-inspired approach with cross-modal synergy and omnidirectional reasoning effectively addresses the challenges of camouflaged object segmentation, providing a robust zero-shot solution with strong generalization capabilities.

Abstract: Camouflaged Object Segmentation (COS) poses a significant challenge due to the intrinsic high similarity between targets and backgrounds, demanding models capable of profound holistic understanding beyond superficial cues. Prevailing methods, often limited by shallow feature representation, inadequate reasoning mechanisms, and weak cross-modal integration, struggle to achieve this depth of cognition, resulting in prevalent issues like incomplete target separation and imprecise segmentation. Inspired by the perceptual strategy of the Hundred-eyed Giant-emphasizing holistic observation, omnidirectional focus, and intensive scrutiny-we introduce ArgusCogito, a novel zero-shot, chain-of-thought framework underpinned by cross-modal synergy and omnidirectional reasoning within Vision-Language Models (VLMs). ArgusCogito orchestrates three cognitively-inspired stages: (1) Conjecture: Constructs a strong cognitive prior through global reasoning with cross-modal fusion (RGB, depth, semantic maps), enabling holistic scene understanding and enhanced target-background disambiguation. (2) Focus: Performs omnidirectional, attention-driven scanning and focused reasoning, guided by semantic priors from Conjecture, enabling precise target localization and region-of-interest refinement. (3) Sculpting: Progressively sculpts high-fidelity segmentation masks by integrating cross-modal information and iteratively generating dense positive/negative point prompts within focused regions, emulating Argus’ intensive scrutiny. Extensive evaluations on four challenging COS benchmarks and three Medical Image Segmentation (MIS) benchmarks demonstrate that ArgusCogito achieves state-of-the-art (SOTA) performance, validating the framework’s exceptional efficacy, superior generalization capability, and robustness.

Method: Proposes SegEarth-OV with two key components: SimFeatUp (universal upsampler restoring high-resolution details) and Global Bias Alleviation (enhancing local semantic fidelity). Also introduces AlignEarth for knowledge distillation from optical to SAR VLMs.

Result: Extensive experiments on optical and SAR datasets show dramatic improvements over state-of-the-art methods, validating the framework’s effectiveness.

Conclusion: SegEarth-OV establishes a robust foundation for annotation-free and open-world Earth observation, enabling universal open-vocabulary segmentation across diverse sensor types without expensive manual annotations.

Abstract: Semantic segmentation of remote sensing (RS) images is pivotal for comprehensive Earth observation, but the demand for interpreting new object categories, coupled with the high expense of manual annotation, poses significant challenges. Although open-vocabulary semantic segmentation (OVSS) offers a promising solution, existing frameworks designed for natural images are insufficient for the unique complexities of RS data. They struggle with vast scale variations and fine-grained details, and their adaptation often relies on extensive, costly annotations. To address this critical gap, this paper introduces SegEarth-OV, the first framework for annotation-free open-vocabulary segmentation of RS images. Specifically, we propose SimFeatUp, a universal upsampler that robustly restores high-resolution spatial details from coarse features, correcting distorted target shapes without any task-specific post-training. We also present a simple yet effective Global Bias Alleviation operation to subtract the inherent global context from patch features, significantly enhancing local semantic fidelity. These components empower SegEarth-OV to effectively harness the rich semantics of pre-trained VLMs, making OVSS possible in optical RS contexts. Furthermore, to extend the framework’s universality to other challenging RS modalities like SAR images, where large-scale VLMs are unavailable and expensive to create, we introduce AlignEarth, which is a distillation-based strategy and can efficiently transfer semantic knowledge from an optical VLM encoder to an SAR encoder, bypassing the need to build SAR foundation models from scratch and enabling universal OVSS across diverse sensor types. Extensive experiments on both optical and SAR datasets validate that SegEarth-OV can achieve dramatic improvements over the SOTA methods, establishing a robust foundation for annotation-free and open-world Earth observation.

Method: Uses low sample-per-pixel path tracing to speed up rendering, then trains a lightweight event spiking network with bipolar leaky integrate-and-fired units and bidirectional earth mover distance loss to denoise RGB videos into realistic event sequences.

Result: EventTracer runs at about 4 minutes per second of 720p video, captures better scene details, and shows greater similarity to real-world event data than other simulators in downstream tasks.

Conclusion: Establishes EventTracer as a promising tool for creating large-scale event-RGB datasets at low cost, narrowing the sim-to-real gap in event-based vision, and boosting applications in robotics, autonomous driving, and VR/AR.

Abstract: Simulating event streams from 3D scenes has become a common practice in event-based vision research, as it meets the demand for large-scale, high temporal frequency data without setting up expensive hardware devices or undertaking extensive data collections. Yet existing methods in this direction typically work with noiseless RGB frames that are costly to render, and therefore they can only achieve a temporal resolution equivalent to 100-300 FPS, far lower than that of real-world event data. In this work, we propose EventTracer, a path tracing-based rendering pipeline that simulates high-fidelity event sequences from complex 3D scenes in an efficient and physics-aware manner. Specifically, we speed up the rendering process via low sample-per-pixel (SPP) path tracing, and train a lightweight event spiking network to denoise the resulting RGB videos into realistic event sequences. To capture the physical properties of event streams, the network is equipped with a bipolar leaky integrate-and-fired (BiLIF) spiking unit and trained with a bidirectional earth mover distance (EMD) loss. Our EventTracer pipeline runs at a speed of about 4 minutes per second of 720p video, and it inherits the merit of accurate spatiotemporal modeling from its path tracing backbone. We show in two downstream tasks that EventTracer captures better scene details and demonstrates a greater similarity to real-world event data than other event simulators, which establishes it as a promising tool for creating large-scale event-RGB datasets at a low cost, narrowing the sim-to-real gap in event-based vision, and boosting various application scenarios such as robotics, autonomous driving, and VRAR.

Method: Proposes a prototype learning and clustering approach that uses few labeled samples to infer prototypes of unknown classes while distinguishing them from known classes. Once unknown samples are rejected, they are clustered into different classes based on distance to inferred unknown class prototypes.

Result: Extensive experiments on four benchmark HSI datasets demonstrate competitive performance compared to state-of-the-art methods in open-set few-shot HSI classification tasks.

Conclusion: The proposed method effectively addresses the limitation of existing approaches by not only rejecting unknown samples but also discovering and clustering them into new classes, showing strong performance in open-set few-shot HSI classification.

Abstract: Open-set few-shot hyperspectral image (HSI) classification aims to classify image pixels by using few labeled pixels per class, where the pixels to be classified may be not all from the classes that have been seen. To address the open-set HSI classification challenge, current methods focus mainly on distinguishing the unknown class samples from the known class samples and rejecting them to increase the accuracy of identifying known class samples. They fails to further identify or discovery the unknow classes among the samples. This paper proposes a prototype learning and clustering method for discoverying unknown classes in HSIs under the few-shot environment. Using few labeled samples, it strives to develop the ability of infering the prototypes of unknown classes while distinguishing unknown classes from known classes. Once the unknown class samples are rejected by the learned known class classifier, the proposed method can further cluster the unknown class samples into different classes according to their distance to the inferred unknown class prototypes. Compared to existing state-of-the-art methods, extensive experiments on four benchmark HSI datasets demonstrate that our proposed method exhibits competitive performance in open-set few-shot HSI classification tasks. All the codes are available at \href{https://github.com/KOBEN-ff/OpenFUCD-main} {https://github.com/KOBEN-ff/OpenFUCD-main}

Method: Proposes three iterative proportional matching techniques (IPMM, IPTM, IPFM) and novel initialization strategies using pre-trained SGMs to train SB-based models effectively.

Result: Significant acceleration and stabilization of SB-based model training, with improved performance of both SB-based models and SGMs through the proposed unified framework.

Conclusion: The work successfully unifies SGMs and Schrödinger Bridge, providing efficient training methods and paving the way for future generative model research with demonstrated effectiveness in experiments.

Abstract: This paper aims to unify Score-based Generative Models (SGMs), also known as Diffusion models, and the Schr"odinger Bridge (SB) problem through three reparameterization techniques: Iterative Proportional Mean-Matching (IPMM), Iterative Proportional Terminus-Matching (IPTM), and Iterative Proportional Flow-Matching (IPFM). These techniques significantly accelerate and stabilize the training of SB-based models. Furthermore, the paper introduces novel initialization strategies that use pre-trained SGMs to effectively train SB-based models. By using SGMs as initialization, we leverage the advantages of both SB-based models and SGMs, ensuring efficient training of SB-based models and further improving the performance of SGMs. Extensive experiments demonstrate the significant effectiveness and improvements of the proposed methods. We believe this work contributes to and paves the way for future research on generative models.

Method: Integrates robotics, telemetry, and AI algorithms with Single Shot Detector (SSD) for detection, specialized hardware acceleration, and tracking algorithms for real-time image processing.

Result: System achieves high detection precision and necessary speed for real-time decision-making, outperforming existing approaches in both accuracy and efficiency.

Conclusion: BirdRecorder bridges renewable energy expansion with wildlife conservation, enabling sustainable coexistence of technology and nature through effective bird protection.

Abstract: The urgent need for renewable energy expansion, particularly wind power, is hindered by conflicts with wildlife conservation. To address this, we developed BirdRecorder, an advanced AI-based anti-collision system to protect endangered birds, especially the red kite (Milvus milvus). Integrating robotics, telemetry, and high-performance AI algorithms, BirdRecorder aims to detect, track, and classify avian species within a range of 800 m to minimize bird-turbine collisions. BirdRecorder integrates advanced AI methods with optimized hardware and software architectures to enable real-time image processing. Leveraging Single Shot Detector (SSD) for detection, combined with specialized hardware acceleration and tracking algorithms, our system achieves high detection precision while maintaining the speed necessary for real-time decision-making. By combining these components, BirdRecorder outperforms existing approaches in both accuracy and efficiency. In this paper, we summarize results on field tests and performance of the BirdRecorder system. By bridging the gap between renewable energy expansion and wildlife conservation, BirdRecorder contributes to a more sustainable coexistence of technology and nature.

Method: Developed SpotEdit benchmark to systematically assess diverse generative models (diffusion, autoregressive, hybrid) and included a dedicated component to evaluate hallucination issues where models incorrectly perceive visual cues.

Result: Uncovered substantial performance disparities across different generative models and highlighted that leading models like GPT-4o often hallucinate the existence of visual cues and erroneously perform editing tasks.

Conclusion: SpotEdit provides a comprehensive evaluation framework that reveals critical limitations in current visually-guided image editing models, particularly the hallucination problem, and offers a publicly available benchmark for future research.

Abstract: Visually-guided image editing, where edits are conditioned on both visual cues and textual prompts, has emerged as a powerful paradigm for fine-grained, controllable content generation. Although recent generative models have shown remarkable capabilities, existing evaluations remain simple and insufficiently representative of real-world editing challenges. We present SpotEdit, a comprehensive benchmark designed to systematically assess visually-guided image editing methods across diverse diffusion, autoregressive, and hybrid generative models, uncovering substantial performance disparities. To address a critical yet underexplored challenge, our benchmark includes a dedicated component on hallucination, highlighting how leading models, such as GPT-4o, often hallucinate the existence of a visual cue and erroneously perform the editing task. Our code and benchmark are publicly released at https://github.com/SaraGhazanfari/SpotEdit.

Method: Uses two coupled CNNs to learn true segmentation label distributions from purely noisy coarse annotations, with high fidelity to noisy training data characteristics and complementary label learning for negative label distribution estimation.

Result: Outperforms state-of-the-art methods in all experiments (MNIST toy dataset, Cityscapes multi-class segmentation, retinal medical images), particularly when coarse annotation ratio is small compared to dense annotations.

Conclusion: Coarse annotations can effectively train segmentation models through the proposed coupled CNN approach with complementary label learning, demonstrating superior performance especially in data-scarce scenarios.

Abstract: Large annotated datasets are vital for training segmentation models, but pixel-level labeling is time-consuming, error-prone, and often requires scarce expert annotators, especially in medical imaging. In contrast, coarse annotations are quicker, cheaper, and easier to produce, even by non-experts. In this paper, we propose to use coarse drawings from both positive (target) and negative (background) classes in the image, even with noisy pixels, to train a convolutional neural network (CNN) for semantic segmentation. We present a method for learning the true segmentation label distributions from purely noisy coarse annotations using two coupled CNNs. The separation of the two CNNs is achieved by high fidelity with the characters of the noisy training annotations. We propose to add a complementary label learning that encourages estimating negative label distribution. To illustrate the properties of our method, we first use a toy segmentation dataset based on MNIST. We then present the quantitative results of experiments using publicly available datasets: Cityscapes dataset for multi-class segmentation, and retinal images for medical applications. In all experiments, our method outperforms state-of-the-art methods, particularly in the cases where the ratio of coarse annotations is small compared to the given dense annotations.

Method: Uses a latent diffusion model conditioned on inpainted object appearance with inference-time guidance. Introduces optimization-in-the-loop design that supervises the velocity field while simultaneously optimizing hand and object transformations. Incorporates multi-modal geometric cues including normal/depth alignment, silhouette consistency, 2D keypoint reprojection, signed distance field supervision, and contact/non-intersection constraints.

Result: The method produces accurate, robust, and coherent 3D object reconstructions under occlusion while generalizing well to in-the-wild scenarios, outperforming previous approaches that rely on post-processing.

Conclusion: The proposed diffusion-based framework successfully leverages hand-object interaction as geometric guidance to achieve high-quality 3D object reconstruction from monocular RGB images, ensuring physical plausibility through various geometric constraints and optimization techniques.

Abstract: We propose a novel diffusion-based framework for reconstructing 3D geometry of hand-held objects from monocular RGB images by leveraging hand-object interaction as geometric guidance. Our method conditions a latent diffusion model on an inpainted object appearance and uses inference-time guidance to optimize the object reconstruction, while simultaneously ensuring plausible hand-object interactions. Unlike prior methods that rely on extensive post-processing or produce low-quality reconstructions, our approach directly generates high-quality object geometry during the diffusion process by introducing guidance with an optimization-in-the-loop design. Specifically, we guide the diffusion model by applying supervision to the velocity field while simultaneously optimizing the transformations of both the hand and the object being reconstructed. This optimization is driven by multi-modal geometric cues, including normal and depth alignment, silhouette consistency, and 2D keypoint reprojection. We further incorporate signed distance field supervision and enforce contact and non-intersection constraints to ensure physical plausibility of hand-object interaction. Our method yields accurate, robust and coherent reconstructions under occlusion while generalizing well to in-the-wild scenarios.

Method: Uses greedy metric-guided block selection with metric feedback (accuracy, CIDEr) to identify redundant layers, reverse-order deletion to preserve early foundational blocks, and tunable sparsity-performance trade-off.

Result: Improves single-object classification accuracy on COCO Person category from 19.1% to 87.3% while skipping >40% of blocks, achieves 45.4% latency reduction in autonomous vehicle deployment.

Conclusion: GM-Skip effectively accelerates VLM inference while preserving performance, demonstrating practical value for real-world applications with latency constraints.

Abstract: Transformer-based Vision-Language Models (VLMs) have achieved impressive performance on tasks such as image captioning, object recognition, and visual reasoning, but their high computational cost hinders deployment in latency-sensitive applications like autonomous driving. We introduce GM-Skip, a flexible and metric-adaptive framework for Transformer block skipping that accelerates VLM inference while preserving output quality. GM-Skip features a greedy, metric-guided block selection strategy that uses metric feedback (e.g., accuracy, CIDEr) to identify redundant layers, along with a reverse-order deletion mechanism that preserves early foundational blocks to avoid performance collapse. To support diverse deployment needs, it incorporates a tunable trade-off between sparsity and performance via a score-sparsity balance objective. Experiments across multiple tasks and datasets, including COCO and CODA, show that GM-Skip consistently improves inference speed while maintaining task performance. On the COCO dataset, GM-Skip improves single-object classification accuracy on the Person category from 19.1 percent to 87.3 percent while skipping more than 40 percent of Transformer blocks. In real-world deployment, it achieves up to 45.4 percent latency reduction on single-object detection when integrated into an autonomous vehicle running Autoware.Universe, validating the effectiveness of its skip configurations and confirming its practical value in accelerating real-world inference.

Method: Self-Knowledge Distillation with Cross-Attention Guidance (SKD-CAG) uses knowledge distillation to guide the model in correcting responses to poisoned prompts while maintaining image quality by leveraging the model’s clean outputs without triggers and neutralizing backdoor influences at the attention level.

Result: The method achieves 100% removal accuracy for pixel backdoors and 93% for style-based attacks without sacrificing robustness or image fidelity.

Conclusion: Targeted unlearning is a promising defense approach to secure generative models against backdoor attacks.

Abstract: Text-to-image diffusion models have revolutionized generative AI, but their vulnerability to backdoor attacks poses significant security risks. Adversaries can inject imperceptible textual triggers into training data, causing models to generate manipulated outputs. Although text-based backdoor defenses in classification models are well-explored, generative models lack effective mitigation techniques against. We address this by selectively erasing the model’s learned associations between adversarial text triggers and poisoned outputs, while preserving overall generation quality. Our approach, Self-Knowledge Distillation with Cross-Attention Guidance (SKD-CAG), uses knowledge distillation to guide the model in correcting responses to poisoned prompts while maintaining image quality by exploiting the fact that the backdoored model still produces clean outputs in the absence of triggers. Using the cross-attention mechanism, SKD-CAG neutralizes backdoor influences at the attention level, ensuring the targeted removal of adversarial effects. Extensive experiments show that our method outperforms existing approaches, achieving removal accuracy 100% for pixel backdoors and 93% for style-based attacks, without sacrificing robustness or image fidelity. Our findings highlight targeted unlearning as a promising defense to secure generative models. Code and model weights can be found at https://github.com/Mystic-Slice/Sealing-The-Backdoor .

Method: Language-Grounded Sparse Encoders (LanSE) architecture that identifies interpretable visual patterns and automatically describes them in natural language, validated through large-scale human evaluation (11,000+ annotations) and LMM-based analysis.

Result: LanSE achieves >93% accuracy in detecting interpretable visual patterns, reveals nuanced model differences invisible to existing metrics (e.g., FLUX’s superior physical plausibility, SDXL-medium’s strong content diversity), and aligns with human judgments.

Conclusion: LanSE bridges interpretability with practical evaluation needs, offering a powerful tool for model selection, quality control, and model improvement, addressing the need for public confidence and safety in AI-generated content.

Abstract: While the quality of AI-generated contents, such as synthetic images, has become remarkably high, current evaluation metrics provide only coarse-grained assessments, failing to identify specific strengths and weaknesses that researchers and practitioners need for model selection and development, further limiting the scientific understanding and commercial deployment of these generative models. To address this, we introduce Language-Grounded Sparse Encoders (LanSE), a novel architecture that creates interpretable evaluation metrics by identifying interpretable visual patterns and automatically describing them in natural language. Through large-scale human evaluation (more than 11,000 annotations) and large multimodal model (LMM) based analysis, LanSE demonstrates reliable capabilities to detect interpretable visual patterns in synthetic images with more than 93% accuracy in natural images. LanSE further provides a fine-grained evaluation framework that quantifies four key dimensions of generation quality, prompt match, visual realism, physical plausibility, and content diversity. LanSE reveals nuanced model differences invisible to existing metrics, for instance, FLUX’s superior physical plausibility and SDXL-medium’s strong content diversity, while aligning with human judgments. By bridging interpretability with practical evaluation needs, LanSE offers all users of generative AI models a powerful tool for model selection, quality control of synthetic content, and model improvement. These capabilities directly address the need for public confidence and safety in AI-generated content, both critical for the future of generative AI applications.

Method: Uses a Transformer-based architecture with masking mechanism to handle missing geometric priors (segment lengths and camera intrinsics). Trained on AMASS dataset with synthetic 2D data generated from random camera poses and intrinsics.

Result: Achieved 36mm average 3D joint position accuracy, improving state-of-the-art by 0.5cm. Runs in 380μs on GPU and 1800μs on CPU. Outperforms expert models even with all priors available.

Conclusion: The proposed versatile model successfully handles both calibrated and uncalibrated settings, maintains high accuracy with missing priors, and offers significantly lower computational cost suitable for embedded platforms.

Abstract: This paper proposes a new lightweight Transformer-based lifter that maps short sequences of human 2D joint positions to 3D poses using a single camera. The proposed model takes as input geometric priors including segment lengths and camera intrinsics and is designed to operate in both calibrated and uncalibrated settings. To this end, a masking mechanism enables the model to ignore missing priors during training and inference. This yields a single versatile network that can adapt to different deployment scenarios, from fully calibrated lab environments to in-the-wild monocular videos without calibration. The model was trained using 3D keypoints from AMASS dataset with corresponding 2D synthetic data generated by sampling random camera poses and intrinsics. It was then compared to an expert model trained, only on complete priors, and the validation was done by conducting an ablation study. Results show that both, camera and segment length priors, improve performance and that the versatile model outperforms the expert, even when all priors are available, and maintains high accuracy when priors are missing. Overall the average 3D joint center positions estimation accuracy was as low as 36mm improving state of the art by half a centimeter and at a much lower computational cost. Indeed, the proposed model runs in 380$\mu$s on GPU and 1800$\mu$s on CPU, making it suitable for deployment on embedded platforms and low-power devices.

Method: Three-step approach: 1) Coarse matching between scene features (from downsampled/encoded 3D Gaussians) and image features (from encoded image patches), 2) Fine matching, and 3) Pose refinement for accurate final estimation.

Result: Competitive localization performance on standard indoor and outdoor benchmarks, outperforming existing 3DGS-based baselines, with effective generalization to novel scenes without additional training.

Conclusion: GSVisLoc successfully demonstrates that 3D Gaussian Splatting representations can be effectively leveraged for visual localization through robust feature matching, achieving strong performance while maintaining generalization capabilities across diverse scenes.

Abstract: We introduce GSVisLoc, a visual localization method designed for 3D Gaussian Splatting (3DGS) scene representations. Given a 3DGS model of a scene and a query image, our goal is to estimate the camera’s position and orientation. We accomplish this by robustly matching scene features to image features. Scene features are produced by downsampling and encoding the 3D Gaussians while image features are obtained by encoding image patches. Our algorithm proceeds in three steps, starting with coarse matching, then fine matching, and finally by applying pose refinement for an accurate final estimate. Importantly, our method leverages the explicit 3DGS scene representation for visual localization without requiring modifications, retraining, or additional reference images. We evaluate GSVisLoc on both indoor and outdoor scenes, demonstrating competitive localization performance on standard benchmarks while outperforming existing 3DGS-based baselines. Moreover, our approach generalizes effectively to novel scenes without additional training.

Method: Formulates token selection as a maximum coverage problem, optimizes a subset of vision tokens to cover both text tokens and original vision tokens, and uses a VLM agent to improve text token quality for guiding vision pruning.

Result: Achieves 1.87x speedup while maintaining 98.7% performance on LLaVA-NeXT-13B, and preserves 87.7% performance with only 4 vision tokens on LLaVA-1.5-7B. Shows vision and text information are complementary.

Conclusion: The coverage criterion is effective for multimodal token selection, enabling significant efficiency improvements while preserving VLM performance through complementary use of vision and text information.

Abstract: Vision-Language Models (VLMs) demonstrate impressive performance in understanding visual content with language instruction by converting visual input to vision tokens. However, redundancy in vision tokens results in the degenerated inference efficiency of VLMs. While many algorithms have been proposed to reduce the number of vision tokens, most of them apply only unimodal information (i.e., vision/text) for pruning and ignore the inherent multimodal property of vision-language tasks. Moreover, it lacks a generic criterion that can be applied to different modalities. To mitigate this limitation, in this work, we propose to leverage both vision and text tokens to select informative vision tokens by the criterion of coverage. We first formulate the subset selection problem as a maximum coverage problem. Afterward, a subset of vision tokens is optimized to cover the text tokens and the original set of vision tokens, simultaneously. Finally, a VLM agent can be adopted to further improve the quality of text tokens for guiding vision pruning. The proposed method MMTok is extensively evaluated on benchmark datasets with different VLMs. The comparison illustrates that vision and text information are complementary, and combining multimodal information can surpass the unimodal baseline with a clear margin. Moreover, under the maximum coverage criterion on the POPE dataset, our method achieves a 1.87x speedup while maintaining 98.7% of the original performance on LLaVA-NeXT-13B. Furthermore, with only four vision tokens, it still preserves 87.7% of the original performance on LLaVA-1.5-7B. These results highlight the effectiveness of coverage in token selection.

Method: Uses Cascade Reinforcement Learning (two-stage offline+online RL) for reasoning enhancement, Visual Resolution Router for dynamic resolution adjustment, and Decoupled Vision-Language Deployment for GPU load balancing.

Result: Achieves +16.0% overall reasoning performance gain, 4.05x inference speedup, state-of-the-art results across multimodal tasks, and supports new capabilities like GUI interaction and embodied agency.

Conclusion: InternVL 3.5 represents a significant advancement in open-source multimodal models, offering superior performance and efficiency while introducing novel capabilities that approach commercial model performance.

Abstract: We introduce InternVL 3.5, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. A key innovation is the Cascade Reinforcement Learning (Cascade RL) framework, which enhances reasoning through a two-stage process: offline RL for stable convergence and online RL for refined alignment. This coarse-to-fine training strategy leads to substantial improvements on downstream reasoning tasks, e.g., MMMU and MathVista. To optimize efficiency, we propose a Visual Resolution Router (ViR) that dynamically adjusts the resolution of visual tokens without compromising performance. Coupled with ViR, our Decoupled Vision-Language Deployment (DvD) strategy separates the vision encoder and language model across different GPUs, effectively balancing computational load. These contributions collectively enable InternVL3.5 to achieve up to a +16.0% gain in overall reasoning performance and a 4.05$\times$ inference speedup compared to its predecessor, i.e., InternVL3. In addition, InternVL3.5 supports novel capabilities such as GUI interaction and embodied agency. Notably, our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks – narrowing the performance gap with leading commercial models like GPT-5. All models and code are publicly released.

Method: Leverages state-of-the-art video editing models to fill masked regions of 3D objects, analyzes the representation gap between 3D and videos, adapts video inpainting models for 3D scenes, and introduces reference-based 3D inpainting for enhanced reconstruction quality.

Result: Achieves superior performance with PSNR of 26.6 vs. NeRFiller (15.9) and LPIPS of 0.19 vs. Instant3dit (0.25), producing more faithful and fine-grained reconstructions across diverse datasets.

Conclusion: ObjFiller-3D demonstrates strong potential for practical deployment in real-world 3D editing applications, offering a significant improvement over previous methods in terms of consistency and reconstruction quality.

Abstract: 3D inpainting often relies on multi-view 2D image inpainting, where the inherent inconsistencies across different inpainted views can result in blurred textures, spatial discontinuities, and distracting visual artifacts. These inconsistencies pose significant challenges when striving for accurate and realistic 3D object completion, particularly in applications that demand high fidelity and structural coherence. To overcome these limitations, we propose ObjFiller-3D, a novel method designed for the completion and editing of high-quality and consistent 3D objects. Instead of employing a conventional 2D image inpainting model, our approach leverages a curated selection of state-of-the-art video editing model to fill in the masked regions of 3D objects. We analyze the representation gap between 3D and videos, and propose an adaptation of a video inpainting model for 3D scene inpainting. In addition, we introduce a reference-based 3D inpainting method to further enhance the quality of reconstruction. Experiments across diverse datasets show that compared to previous methods, ObjFiller-3D produces more faithful and fine-grained reconstructions (PSNR of 26.6 vs. NeRFiller (15.9) and LPIPS of 0.19 vs. Instant3dit (0.25)). Moreover, it demonstrates strong potential for practical deployment in real-world 3D editing applications. Project page: https://objfiller3d.github.io/ Code: https://github.com/objfiller3d/ObjFiller-3D .

Method: Extends adversarial adaptation techniques to federated constraints, devises dynamic attention mechanism, and leverages feature disentanglement to enhance knowledge transfer.

Result: Extensive experiments on image and text classification tasks show promising results under unsupervised federated domain adaptation setting.

Conclusion: The approach successfully addresses domain shift in federated learning through adversarial adaptation, attention mechanisms, and feature disentanglement, demonstrating effectiveness across multiple tasks.

Abstract: Federated learning improves data privacy and efficiency in machine learning performed over networks of distributed devices, such as mobile phones, IoT and wearable devices, etc. Yet models trained with federated learning can still fail to generalize to new devices due to the problem of domain shift. Domain shift occurs when the labeled data collected by source nodes statistically differs from the target node’s unlabeled data. In this work, we present a principled approach to the problem of federated domain adaptation, which aims to align the representations learned among the different nodes with the data distribution of the target node. Our approach extends adversarial adaptation techniques to the constraints of the federated setting. In addition, we devise a dynamic attention mechanism and leverage feature disentanglement to enhance knowledge transfer. Empirically, we perform extensive experiments on several image and text classification tasks and show promising results under unsupervised federated domain adaptation setting.

Method: Developed two sensor-wise Transformer architectures (HART and MobileHART) specifically designed for human activity recognition, with public code release.

Result: The HART architectures outperform previous models with fewer FLOPs and parameters, and demonstrate superior robustness to device position and brand variations across multiple public datasets.

Conclusion: The proposed Transformer architectures provide more efficient and robust solutions for real-world HAR deployment, addressing data heterogeneity challenges in pervasive computing environments.

Abstract: Human Activity Recognition (HAR) on mobile devices has been demonstrated to be possible using neural models trained on data collected from the device’s inertial measurement units. These models have used Convolutional Neural Networks (CNNs), Long Short-Term Memory (LSTMs), Transformers or a combination of these to achieve state-of-the-art results with real-time performance. However, these approaches have not been extensively evaluated in real-world situations where the input data may be different from the training data. This paper highlights the issue of data heterogeneity in machine learning applications and how it can hinder their deployment in pervasive settings. To address this problem, we propose and publicly release the code of two sensor-wise Transformer architectures called HART and MobileHART for Human Activity Recognition Transformer. Our experiments on several publicly available datasets show that these HART architectures outperform previous architectures with fewer floating point operations and parameters than conventional Transformers. The results also show they are more robust to changes in mobile position or device brand and hence better suited for the heterogeneous environments encountered in real-life settings. Finally, the source code has been made publicly available.

Method: The paper provides a systematic review by summarizing problem formulations, analyzing face image characteristics, discussing challenges, reviewing prior-based and deep learning methods, exploring network architectures/loss functions/datasets, and conducting benchmark evaluations.

Result: The survey presents a comprehensive analysis of current face restoration techniques, identifies key challenges, benchmarks representative methods, and provides an organized open-source repository of discussed methods.

Conclusion: Face restoration has advanced significantly with deep learning, but future work is needed in network design, evaluation metrics, benchmark datasets, and practical applications, with the provided survey serving as a foundation for further research.

Abstract: Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images, which is a domain-specific image restoration problem in the low-level computer vision area. The early face restoration methods mainly use statistical priors and degradation models, which are difficult to meet the requirements of real-world applications in practice. In recent years, face restoration has witnessed great progress after stepping into the deep learning era. However, there are few works to systematically study the deep learning based face restoration methods. Thus, in this paper, we provide a comprehensive survey of recent advances in deep learning techniques for face restoration. Specifically, we first summarize different problem formulations and analyze the characteristics of face images. Second, we discuss the challenges of face restoration. With regard to these challenges, we present a comprehensive review of recent FR methods, including prior-based methods and deep-learning methods. Then, we explore developed techniques in the task of FR covering network architectures, loss functions, and benchmark datasets. We also conduct a systematic benchmark evaluation on representative methods. Finally, we discuss the future directions including network designs, metrics, benchmark datasets, applications, etc. We also provide an open source repository for all the discussed methods, which is available at https://github.com/TaoWangzj/Awesome-Face-Restoration.

Method: Builds on 3D Gaussian splatting by introducing periodic vibration-based temporal dynamics, temporal smoothing mechanism for coherence, and position-aware adaptive control strategy for large scene learning with sparse data.

Result: Surpasses state-of-the-art methods on Waymo and KITTI benchmarks in reconstruction and novel view synthesis for both dynamic/static scenes, without manual bounding boxes or optical flow estimation.

Conclusion: PVG provides an elegant unified representation for dynamic urban scenes with significantly improved performance and 900x rendering acceleration compared to alternatives.

Abstract: Modeling dynamic, large-scale urban scenes is challenging due to their highly intricate geometric structures and unconstrained dynamics in both space and time. Prior methods often employ high-level architectural priors, separating static and dynamic elements, resulting in suboptimal capture of their synergistic interactions. To address this challenge, we present a unified representation model, called Periodic Vibration Gaussian (PVG). PVG builds upon the efficient 3D Gaussian splatting technique, originally designed for static scene representation, by introducing periodic vibration-based temporal dynamics. This innovation enables PVG to elegantly and uniformly represent the characteristics of various objects and elements in dynamic urban scenes. To enhance temporally coherent and large scene representation learning with sparse training data, we introduce a novel temporal smoothing mechanism and a position-aware adaptive control strategy respectively. Extensive experiments on Waymo Open Dataset and KITTI benchmarks demonstrate that PVG surpasses state-of-the-art alternatives in both reconstruction and novel view synthesis for both dynamic and static scenes. Notably, PVG achieves this without relying on manually labeled object bounding boxes or expensive optical flow estimation. Moreover, PVG exhibits 900-fold acceleration in rendering over the best alternative.

Method: Decouples reflection removal into prompt generation and prompt-guided restoration. Uses frequency prompt encoder pre-training and diffusion models to generate LF/HF prompts, then integrates prompts into PromptFormer network with Transformer-based prompt blocks.

Result: Outperforms state-of-the-art approaches on commonly used benchmarks for reflection removal.

Conclusion: Frequency information as visual prompts effectively guides reflection removal, with the proposed PromptRR framework achieving superior performance compared to existing methods.

Abstract: Existing single image reflection removal (SIRR) methods using deep learning tend to miss key low-frequency (LF) and high-frequency (HF) differences in images, affecting their effectiveness in removing reflections. To address this problem, this paper proposes a novel prompt-guided reflection removal (PromptRR) framework that uses frequency information as new visual prompts for better reflection performance. Specifically, the proposed framework decouples the reflection removal process into the prompt generation and subsequent prompt-guided restoration. For the prompt generation, we first propose a prompt pre-training strategy to train a frequency prompt encoder that encodes the ground-truth image into LF and HF prompts. Then, we adopt diffusion models (DMs) as prompt generators to generate the LF and HF prompts estimated by the pre-trained frequency prompt encoder. For the prompt-guided restoration, we integrate specially generated prompts into the PromptFormer network, employing a novel Transformer-based prompt block to effectively steer the model toward enhanced reflection removal. The results on commonly used benchmarks show that our method outperforms state-of-the-art approaches. The codes and models are available at https://github.com/TaoWangzj/PromptRR.

Method: Uses perceptual mapping to identify human-sensitive areas, instance-aware refinement to adjust protection intensity, difficulty-aware protection that predicts protection difficulty, and perceptual constraints bank for improved imperceptibility.

Result: The method substantially improves the quality of protected images without compromising protection effectiveness against style imitation.

Conclusion: The proposed approach successfully balances copyright protection with visual quality preservation, offering a more practical solution for artwork protection in the age of advanced diffusion models.

Abstract: Recent progress in diffusion models has profoundly enhanced the fidelity of image generation, but it has raised concerns about copyright infringements. While prior methods have introduced adversarial perturbations to prevent style imitation, most are accompanied by the degradation of artworks’ visual quality. Recognizing the importance of maintaining this, we introduce a visually improved protection method while preserving its protection capability. To this end, we devise a perceptual map to highlight areas sensitive to human eyes, guided by instance-aware refinement, which refines the protection intensity accordingly. We also introduce a difficulty-aware protection by predicting how difficult the artwork is to protect and dynamically adjusting the intensity based on this. Lastly, we integrate a perceptual constraints bank to further improve the imperceptibility. Results show that our method substantially elevates the quality of the protected image without compromising on protection efficacy.

Method: Uses pre-trained LLM for layout generation from text, produces masked latents in bounding boxes for accurate object placement, employs fusion mechanism for attention maps, uses diffusion U-Net for coherent composition, and optimizes SVG with pre-trained encoder and LPIPS loss with opacity modulation.

Result: SVGCraft surpasses prior works in abstraction, recognizability, and detail with performance metrics: CLIP-T: 0.4563, Cosine Similarity: 0.6342, Confusion: 0.66, Aesthetic: 6.7832. Also explores primitive shapes for canvas completion in constrained environments.

Conclusion: The framework successfully generates comprehensive vector art scenes from text prompts, demonstrating superior performance over existing methods and offering an end-to-end solution for text-to-vector-scene generation.

Abstract: Generating VectorArt from text prompts is a challenging vision task, requiring diverse yet realistic depictions of the seen as well as unseen entities. However, existing research has been mostly limited to the generation of single objects, rather than comprehensive scenes comprising multiple elements. In response, this work introduces SVGCraft, a novel end-to-end framework for the creation of vector graphics depicting entire scenes from textual descriptions. Utilizing a pre-trained LLM for layout generation from text prompts, this framework introduces a technique for producing masked latents in specified bounding boxes for accurate object placement. It introduces a fusion mechanism for integrating attention maps and employs a diffusion U-Net for coherent composition, speeding up the drawing process. The resulting SVG is optimized using a pre-trained encoder and LPIPS loss with opacity modulation to maximize similarity. Additionally, this work explores the potential of primitive shapes in facilitating canvas completion in constrained environments. Through both qualitative and quantitative assessments, SVGCraft is demonstrated to surpass prior works in abstraction, recognizability, and detail, as evidenced by its performance metrics (CLIP-T: 0.4563, Cosine Similarity: 0.6342, Confusion: 0.66, Aesthetic: 6.7832). The code will be available at https://github.com/ayanban011/SVGCraft.

Method: Proposes Top-Down Guided Network (TDGNet) with a top-down pathway that constructs guidance using high-level object-centric representations to optimize low-level grid features during training, and refines representations by detecting/solving feature conflicts during inference.

Result: TDGNet outperforms current object-centric models on multiple datasets of varying complexity and demonstrates effectiveness in robotics downstream tasks including video prediction and visual planning.

Conclusion: The top-down pathway approach significantly improves object-centric representations, expanding the scope of downstream applications to more complex tasks like robotics planning and prediction.

Abstract: Humans’ innate ability to decompose scenes into objects allows for efficient understanding, predicting, and planning. In light of this, Object-Centric Learning (OCL) attempts to endow networks with similar capabilities, learning to represent scenes with the composition of objects. However, existing OCL models only learn through reconstructing the input images, which does not assist the model in distinguishing objects, resulting in suboptimal object-centric representations. This flaw limits current object-centric models to relatively simple downstream tasks. To address this issue, we draw on humans’ top-down vision pathway and propose Top-Down Guided Network (TDGNet), which includes a top-down pathway to improve object-centric representations. During training, the top-down pathway constructs guidance with high-level object-centric representations to optimize low-level grid features output by the backbone. While during inference, it refines object-centric representations by detecting and solving conflicts between low- and high-level features. We show that TDGNet outperforms current object-centric models on multiple datasets of varying complexity. In addition, we expand the downstream task scope of object-centric representations by applying TDGNet to the field of robotics, validating its effectiveness in downstream tasks including video prediction and visual planning.

Method: Proposes TokenUnify framework with three learning objectives: random token prediction, next-token prediction, and next-all token prediction. Uses Mamba architecture for linear-time sequence modeling and introduces a large-scale EM dataset with 1.2B annotated voxels.

Result: Achieves 44% performance improvement on downstream neuron segmentation, outperforms MAE by 25%, reduces autoregressive error accumulation from O(K) to O(sqrt(K)), and demonstrates superior scaling properties with model size.

Conclusion: TokenUnify effectively bridges the gap between pretraining strategies for language and vision models, providing optimal coverage of visual data structure through complementary information-theoretic objectives and enabling better handling of EM data challenges.

Abstract: Neuron segmentation from electron microscopy (EM) volumes is crucial for understanding brain circuits, yet the complex neuronal structures in high-resolution EM images present significant challenges. EM data exhibits unique characteristics including high noise levels, anisotropic voxel dimensions, and ultra-long spatial dependencies that make traditional vision models inadequate. Inspired by autoregressive pretraining in language models, we propose TokenUnify, a hierarchical predictive coding framework that captures multi-scale dependencies through three complementary learning objectives. TokenUnify integrates random token prediction, next-token prediction, and next-all token prediction to create a comprehensive representational space with emergent properties. From an information-theoretic perspective, these three tasks are complementary and provide optimal coverage of visual data structure, with our approach reducing autoregressive error accumulation from O(K) to O(sqrt(K)) for sequences of length K. We also introduce a large-scale EM dataset with 1.2 billion annotated voxels, offering ideal long-sequence visual data with spatial continuity. Leveraging the Mamba architecture’s linear-time sequence modeling capabilities, TokenUnify achieves a 44% performance improvement on downstream neuron segmentation and outperforms MAE by 25%. Our approach demonstrates superior scaling properties as model size increases, effectively bridging the gap between pretraining strategies for language and vision models.

Method: Uses object-centric priors to eliminate uninformative views based on semantic information and fuses features at object-level via instance segmentation masks. Distills features using a large-scale synthetic multi-view dataset of cluttered tabletop scenes.

Result: Reconstructs 3D CLIP features with improved grounding capacity and spatial consistency from single-view RGB-D, generalizes to novel tabletop domains, and enables 3D instance segmentation without fine-tuning.

Conclusion: The approach demonstrates utility for language-guided robotic grasping in clutter and provides a practical solution that departs from the assumption of multiple camera views at test time.

Abstract: Grounding natural language to the physical world is a ubiquitous topic with a wide range of applications in computer vision and robotics. Recently, 2D vision-language models such as CLIP have been widely popularized, due to their impressive capabilities for open-vocabulary grounding in 2D images. Recent works aim to elevate 2D CLIP features to 3D via feature distillation, but either learn neural fields that are scene-specific and hence lack generalization, or focus on indoor room scan data that require access to multiple camera views, which is not practical in robot manipulation scenarios. Additionally, related methods typically fuse features at pixel-level and assume that all camera views are equally informative. In this work, we show that this approach leads to sub-optimal 3D features, both in terms of grounding accuracy, as well as segmentation crispness. To alleviate this, we propose a multi-view feature fusion strategy that employs object-centric priors to eliminate uninformative views based on semantic information, and fuse features at object-level via instance segmentation masks. To distill our object-centric 3D features, we generate a large-scale synthetic multi-view dataset of cluttered tabletop scenes, spawning 15k scenes from over 3300 unique object instances, which we make publicly available. We show that our method reconstructs 3D CLIP features with improved grounding capacity and spatial consistency, while doing so from single-view RGB-D, thus departing from the assumption of multiple camera views at test time. Finally, we show that our approach can generalize to novel tabletop domains and be re-purposed for 3D instance segmentation without fine-tuning, and demonstrate its utility for language-guided robotic grasping in clutter.

Method: Developed a tool that extracts high-level human concepts from images and generates Weight of Evidence for hypotheses. Built web application for dermatoscopic image analysis with hypothesis selection and evidence evaluation.

Result: Successfully applied and evaluated the tool in skin cancer domain, demonstrating effectiveness across different concept-based explanation approaches.

Conclusion: Visual Evaluative AI serves as an effective decision aid by providing visual evidence evaluation for hypotheses, particularly valuable in medical diagnostic applications.

Abstract: This paper presents Visual Evaluative AI, a decision aid that provides positive and negative evidence from image data for a given hypothesis. This tool finds high-level human concepts in an image and generates the Weight of Evidence (WoE) for each hypothesis in the decision-making process. We apply and evaluate this tool in the skin cancer domain by building a web-based application that allows users to upload a dermatoscopic image, select a hypothesis and analyse their decisions by evaluating the provided evidence. Further, we demonstrate the effectiveness of Visual Evaluative AI on different concept-based explanation approaches.

Method: Proposes CTRL-F network with convolution branch and MFCA transformer module. MFCA processes small and large patch tokens from multi-level convolution features using cross-attention. Uses adaptive knowledge fusion (AKF) and collaborative knowledge fusion (CKF) to combine local and global responses.

Result: Achieves top-1 accuracy of 82.24% on Oxford-102 Flowers and 99.91% on PlantVillage datasets when trained from scratch, surpassing state-of-the-art models and demonstrating robustness.

Conclusion: The hybrid approach successfully combines convolution and transformer strengths, providing excellent performance even with limited data, making CTRL-F a robust solution for image classification tasks.

Abstract: Transformers have captured growing attention in computer vision, thanks to its large capacity and global processing capabilities. However, transformers are data hungry, and their ability to generalize is constrained compared to Convolutional Neural Networks (ConvNets), especially when trained with limited data due to the absence of the built-in spatial inductive biases present in ConvNets. In this paper, we strive to optimally combine the strengths of both convolution and transformers for image classification tasks. Towards this end, we present a novel lightweight hybrid network that pairs Convolution with Transformers via Representation Learning Fusion and Multi-Level Feature Cross-Attention named CTRL-F. Our network comprises a convolution branch and a novel transformer module named multi-level feature cross-attention (MFCA). The MFCA module operates on multi-level feature representations obtained at different convolution stages. It processes small patch tokens and large patch tokens extracted from these multi-level feature representations via two separate transformer branches, where both branches communicate and exchange knowledge through cross-attention mechanism. We fuse the local responses acquired from the convolution path with the global responses acquired from the MFCA module using novel representation fusion techniques dubbed adaptive knowledge fusion (AKF) and collaborative knowledge fusion (CKF). Experiments demonstrate that our CTRL-F variants achieve state-of-the-art performance, whether trained from scratch on large data or even with low-data regime. For Instance, CTRL-F achieves top-1 accuracy of 82.24% and 99.91% when trained from scratch on Oxford-102 Flowers and PlantVillage datasets respectively, surpassing state-of-the-art models which showcase the robustness of our model on image classification tasks. Code at: https://github.com/hosamsherif/CTRL-F

Method: Integrated clinical knowledge-based topological graph (CKTG) with gradient diagnostic strategy featuring data-driven weighting (GD-DDW), plus multimodal feature extraction with dual-attention mechanism for cross-modal interaction and unimodal collaboration.

Result: Achieved average AUC of 88.6% on EDRA dataset, demonstrating superior performance in melanoma detection and feature prediction compared to traditional methods.

Conclusion: The proposed integrated system provides data-driven benchmarks for clinicians and significantly enhances the precision of melanoma diagnosis by better capturing complex relationships between clinical attributes and emulating dermatologists’ diagnostic processes.

Abstract: The 7-point checklist (7PCL) is a widely used diagnostic tool in dermoscopy for identifying malignant melanoma by assigning point values to seven specific attributes. However, the traditional 7PCL is limited to distinguishing between malignant melanoma and melanocytic Nevi, and falls short in scenarios where multiple skin diseases with appearances similar to melanoma coexist. To address this limitation, we propose a novel diagnostic framework that integrates a clinical knowledge-based topological graph (CKTG) with a gradient diagnostic strategy featuring a data-driven weighting system (GD-DDW). The CKTG captures both the internal and external relationships among the 7PCL attributes, while the GD-DDW emulates dermatologists’ diagnostic processes, prioritizing visual observation before making predictions. Additionally, we introduce a multimodal feature extraction approach leveraging a dual-attention mechanism to enhance feature extraction through cross-modal interaction and unimodal collaboration. This method incorporates meta-information to uncover interactions between clinical data and image features, ensuring more accurate and robust predictions. Our approach, evaluated on the EDRA dataset, achieved an average AUC of 88.6%, demonstrating superior performance in melanoma detection and feature prediction. This integrated system provides data-driven benchmarks for clinicians, significantly enhancing the precision of melanoma diagnosis.

Method: Uses forgery-style mixture formulation to augment domain diversity, lightweight feature extraction modules in ViT architecture, and only optimizes inserted modules while keeping pre-trained weights frozen.

Result: Achieves state-of-the-art generalizability with significantly reduced trainable parameters, demonstrating strong performance across unseen forgery domains.

Conclusion: The approach represents an important step toward practical open-set Deepfake detection by balancing generalization capability with parameter efficiency.

Abstract: Open-set face forgery detection poses significant security threats and presents substantial challenges for existing detection models. These detectors primarily have two limitations: they cannot generalize across unknown forgery domains and inefficiently adapt to new data. To address these issues, we introduce an approach that is both general and parameter-efficient for face forgery detection. It builds on the assumption that different forgery source domains exhibit distinct style statistics. Previous methods typically require fully fine-tuning pre-trained networks, consuming substantial time and computational resources. In turn, we design a forgery-style mixture formulation that augments the diversity of forgery source domains, enhancing the model’s generalizability across unseen domains. Drawing on recent advancements in vision transformers (ViT) for face forgery detection, we develop a parameter-efficient ViT-based detection model that includes lightweight forgery feature extraction modules and enables the model to extract global and local forgery clues simultaneously. We only optimize the inserted lightweight modules during training, maintaining the original ViT structure with its pre-trained ImageNet weights. This training strategy effectively preserves the informative pre-trained knowledge while flexibly adapting the model to the task of Deepfake detection. Extensive experimental results demonstrate that the designed model achieves state-of-the-art generalizability with significantly reduced trainable parameters, representing an important step toward open-set Deepfake detection in the wild.

Method: Proposes prompt learning with task-specific prompts, efficient prompt fusion for knowledge transfer, and prompt selection with dual-modality guidance using natural image-text supervision.

Result: Achieves +14.26% performance gain on MCIT benchmarks with 1.42x inference speed improvement and no growing computation burden.

Conclusion: The framework effectively alleviates forgetting of previous knowledge while managing computational complexity, demonstrating superior performance and efficiency in multimodal continual instruction learning.

Abstract: Large Multimodal Models (LMMs) exhibit remarkable multi-tasking ability by learning mixed instruction datasets. However, novel tasks would be encountered sequentially in dynamic world, which urges for equipping LMMs with multimodal continual instruction learning (MCIT) ability especially for diverse and challenging generative tasks. Existing MCIT methods do not fully exploit the unique attribute of LMMs and often gain performance at the expense of efficiency. In this paper, we propose a novel prompt learning framework for MCIT to effectively alleviate forgetting of previous knowledge while managing computational complexity with natural image-text supervision. Concretely, we learn prompts for each task and exploit efficient prompt fusion for knowledge transfer and prompt selection for complexity management with dual-modality guidance. Extensive experiments demonstrate that our approach achieves substantial +14.26% performance gain on MCIT benchmarks with remarkable $\times$ 1.42 inference speed free from growing computation. Code is available at https://github.com/AuroraZengfh/ModalPrompt.

Method: Developed a real-time plugin for CARLA simulator that uses state-of-the-art image enhancement to translate simulated outputs to match the visual style of real-world datasets (Cityscapes, KITTI, Mapillary Vistas).

Result: Achieved 13 FPS processing, generated enhanced synthetic datasets with ground truth annotations, and demonstrated reduced sim2real gap through improved feature extraction and semantic segmentation performance.

Conclusion: The sim2real appearance gap is significant but can be effectively reduced using the proposed CARLA2Real tool, which provides an easy-to-use solution for enhancing synthetic data realism in autonomous driving research.

Abstract: Simulators are indispensable for research in autonomous systems such as self-driving cars, autonomous robots, and drones. Despite significant progress in various simulation aspects, such as graphical realism, an evident gap persists between the virtual and real-world environments. Since the ultimate goal is to deploy the autonomous systems in the real world, reducing the sim2real gap is of utmost importance. In this paper, we employ a state-of-the-art approach to enhance the photorealism of simulated data, aligning them with the visual characteristics of real-world datasets. Based on this, we developed CARLA2Real, an easy-to-use, publicly available tool (plug-in) for the widely used and open-source CARLA simulator. This tool enhances the output of CARLA in near real-time, achieving a frame rate of 13 FPS, translating it to the visual style and realism of real-world datasets such as Cityscapes, KITTI, and Mapillary Vistas. By employing the proposed tool, we generated synthetic datasets from both the simulator and the enhancement model outputs, including their corresponding ground truth annotations for tasks related to autonomous driving. Then, we performed a number of experiments to evaluate the impact of the proposed approach on feature extraction and semantic segmentation methods when trained on the enhanced synthetic data. The results demonstrate that the sim2real appearance gap is significant and can indeed be reduced by the introduced approach. Comparisons with a state-of-the-art image-to-image translation approach are also provided. The tool, pre-trained models, and associated data for this work are available for download at: https://github.com/stefanos50/CARLA2Real.

Method: Proposes LumiSculpt with plug-and-play modules for reference-based lighting and direct light source control, trained on new LumiHuman dataset.

Result: Achieves precise and high-quality lighting control in video generation with demonstrated flexibility.

Conclusion: LumiSculpt provides effective lighting control capabilities for T2V models, addressing previous limitations in lighting disentanglement.

Abstract: Lighting plays a pivotal role in ensuring the naturalness and aesthetic quality of video generation. However, the impact of lighting is deeply coupled with other factors of videos, e.g., objects and scenes. Thus, it remains challenging to disentangle and model coherent lighting conditions independently, limiting the flexibility to control lighting in video generation. In this paper, inspired by the established controllable T2I models, we propose LumiSculpt, which enables precise and consistent lighting control in T2V generation models. LumiSculpt equips the video generation with new interactive capabilities, allowing the input of reference image sequences with customized lighting conditions. Furthermore, the core learnable plug-and-play module of LumiSculpt facilitates direct control over the intensity, position and trajectory of an assumed light source in video diffusion models. To effectively train LumiSculpt and address the issue of insufficient lighting data, we construct LumiHuman, a new lightweight and flexible dataset for portrait lighting of images and videos. Experimental results demonstrate that LumiSculpt achieves precise and high-quality lighting control in video generation. The analysis demonstrates the flexibility of LumiHuman.

Method: Proposed three principles (Multi-Frame Gain, Frame Order Sensitivity, Frame Information Disparity) and created TOMATO benchmark with 1,484 human-annotated questions across 6 tasks applied to 1,417 videos including human-centric, real-world and simulated scenarios.

Result: Revealed 57.3% human-model performance gap with best-performing model. MFMs accurately recognize events in isolated frames but fail to interpret frames as continuous sequences, showing fundamental limitations in temporal reasoning.

Conclusion: TOMATO serves as crucial testbed for next-gen MFMs and calls for developing AI systems capable of comprehending human world dynamics through video modality, highlighting current models’ inability to reason about temporal sequences.

Abstract: Existing benchmarks often highlight the remarkable performance achieved by state-of-the-art Multimodal Foundation Models (MFMs) in leveraging temporal context for video understanding. However, how well do the models truly perform visual temporal reasoning? Our study of existing benchmarks shows that this capability of MFMs is likely overestimated as many questions can be solved by using a single, few, or out-of-order frames. To systematically examine current visual temporal reasoning tasks, we propose three principles with corresponding metrics: (1) Multi-Frame Gain, (2) Frame Order Sensitivity, and (3) Frame Information Disparity. Following these principles, we introduce TOMATO, Temporal Reasoning Multimodal Evaluation, a novel benchmark crafted to rigorously assess MFMs’ temporal reasoning capabilities in video understanding. TOMATO comprises 1,484 carefully curated, human-annotated questions spanning six tasks (i.e., action count, direction, rotation, shape & trend, velocity & frequency, and visual cues), applied to 1,417 videos, including 805 self-recorded and -generated videos, that encompass human-centric, real-world, and simulated scenarios. Our comprehensive evaluation reveals a human-model performance gap of 57.3% with the best-performing model. Moreover, our in-depth analysis uncovers more fundamental limitations beyond this gap in current MFMs. While they can accurately recognize events in isolated frames, they fail to interpret these frames as a continuous sequence. We believe TOMATO will serve as a crucial testbed for evaluating the next-generation MFMs and as a call to the community to develop AI systems capable of comprehending human world dynamics through the video modality.

Method: Created V2X-R dataset with 12K+ scenarios containing LiDAR, 4D radar point clouds, images, and annotated 3D boxes. Proposed cooperative LiDAR-4D radar fusion pipeline with Multi-modal Denoising Diffusion (MDD) module that uses radar features to condition diffusion model for denoising noisy LiDAR features.

Result: The LiDAR-4D radar fusion pipeline shows superior performance on V2X-R dataset. MDD module further improved basic fusion model by up to 5.73%/6.70% in foggy/snowy conditions without disrupting normal performance.

Conclusion: The proposed V2X-R dataset and fusion pipeline with MDD module effectively address weather robustness in V2X 3D object detection, demonstrating significant performance improvements in adverse weather conditions while maintaining normal operation.

Abstract: Current Vehicle-to-Everything (V2X) systems have significantly enhanced 3D object detection using LiDAR and camera data. However, these methods suffer from performance degradation in adverse weather conditions. The weather-robust 4D radar provides Doppler and additional geometric information, raising the possibility of addressing this challenge. To this end, we present V2X-R, the first simulated V2X dataset incorporating LiDAR, camera, and 4D radar. V2X-R contains 12,079 scenarios with 37,727 frames of LiDAR and 4D radar point clouds, 150,908 images, and 170,859 annotated 3D vehicle bounding boxes. Subsequently, we propose a novel cooperative LiDAR-4D radar fusion pipeline for 3D object detection and implement it with various fusion strategies. To achieve weather-robust detection, we additionally propose a Multi-modal Denoising Diffusion (MDD) module in our fusion pipeline. MDD utilizes weather-robust 4D radar feature as a condition to prompt the diffusion model to denoise noisy LiDAR features. Experiments show that our LiDAR-4D radar fusion pipeline demonstrates superior performance in the V2X-R dataset. Over and above this, our MDD module further improved the performance of basic fusion model by up to 5.73%/6.70% in foggy/snowy conditions with barely disrupting normal performance. The dataset and code will be publicly available at: https://github.com/ylwhxht/V2X-R.

Method: Proposes MMSLT framework that uses off-the-shelf multimodal large language models to generate detailed textual descriptions of sign language components, then integrates these description features with sign video features through a multimodal-language pre-training module to align them within the spoken sentence space.

Result: Achieves state-of-the-art performance on benchmark datasets PHOENIX14T and CSL-Daily, demonstrating the effectiveness of leveraging MLLMs for sign language translation tasks.

Conclusion: The proposed MMSLT framework successfully demonstrates the potential of multimodal large language models to be effectively utilized in sign language translation, providing a gloss-free approach that bridges the modality gap and improves translation accuracy.

Abstract: Sign language translation (SLT) is a challenging task that involves translating sign language images into spoken language. For SLT models to perform this task successfully, they must bridge the modality gap and identify subtle variations in sign language components to understand their meanings accurately. To address these challenges, we propose a novel gloss-free SLT framework called Multimodal Sign Language Translation (MMSLT), which leverages the representational capabilities of off-the-shelf multimodal large language models (MLLMs). Specifically, we use MLLMs to generate detailed textual descriptions of sign language components. Then, through our proposed multimodal-language pre-training module, we integrate these description features with sign video features to align them within the spoken sentence space. Our approach achieves state-of-the-art performance on benchmark datasets PHOENIX14T and CSL-Daily, highlighting the potential of MLLMs to be utilized effectively in SLT. Code is available at https://github.com/hwjeon98/MMSLT.

Method: Leverages implicit occupancy function representation to design 3D-printable geometric models, optimizing light directions and screen orientations to match input binary images while promoting surface smoothness and reducing material usage.

Result: The method generates high-quality shadow art with arbitrary topologies at any resolution, producing projections that closely resemble target images even with complex topologies, while avoiding trivial intersecting cylindrical structures.

Conclusion: The proposed implicit representation significantly improves projection accuracy and flexibility for shadow art, meeting industrial production requirements while delivering enhanced artistic effects through optimized geometry and material efficiency.

Abstract: Shadow art is a captivating form of sculptural expression where the projection of a sculpture in a specific direction reveals a desired shape with high precision. In this work, we introduce Neural Shadow Art, which leverages implicit occupancy function representation to significantly expand the possibilities of shadow art. This representation enables the design of high-quality, 3D-printable geometric models with arbitrary topologies at any resolution, surpassing previous voxel- and mesh-based methods. Our method provides a more flexible framework, enabling projections to match input binary images under various light directions and screen orientations, without requiring light sources to be perpendicular to the screens. Furthermore, we allow rigid transformations of the projected geometries relative to the input binary images and simultaneously optimize light directions and screen orientations to ensure that the projections closely resemble the target images, especially when dealing with inputs of complex topologies. In addition, our model promotes surface smoothness and reduces material usage. This is particularly advantageous for efficient industrial production and enhanced artistic effect by generating compelling shadow art that avoids trivial, intersecting cylindrical structures. In summary, we propose a more flexible representation for shadow art, significantly improving projection accuracy while simultaneously meeting industrial requirements and delivering awe-inspiring artistic effects.

Method: Initialize global scene with uniform 3D semantic Gaussians, progressively update local regions using deformable cross-attention to incorporate semantic/structural features from observed images, and use Gaussian-to-voxel splatting for final occupancy prediction.

Result: Outperforms existing methods by a large margin on the EmbodiedOcc-ScanNet benchmark, achieving high accuracy and efficiency in embodied occupancy prediction.

Conclusion: EmbodiedOcc successfully addresses the embodied 3D occupancy prediction task by maintaining explicit global memory with 3D Gaussians and gradually refining knowledge through local updates, mimicking human scene understanding through exploration.

Abstract: 3D occupancy prediction provides a comprehensive description of the surrounding scenes and has become an essential task for 3D perception. Most existing methods focus on offline perception from one or a few views and cannot be applied to embodied agents that demand to gradually perceive the scene through progressive embodied exploration. In this paper, we formulate an embodied 3D occupancy prediction task to target this practical scenario and propose a Gaussian-based EmbodiedOcc framework to accomplish it. We initialize the global scene with uniform 3D semantic Gaussians and progressively update local regions observed by the embodied agent. For each update, we extract semantic and structural features from the observed image and efficiently incorporate them via deformable cross-attention to refine the regional Gaussians. Finally, we employ Gaussian-to-voxel splatting to obtain the global 3D occupancy from the updated 3D Gaussians. Our EmbodiedOcc assumes an unknown (i.e., uniformly distributed) environment and maintains an explicit global memory of it with 3D Gaussians. It gradually gains knowledge through the local refinement of regional Gaussians, which is consistent with how humans understand new scenes through embodied exploration. We reorganize an EmbodiedOcc-ScanNet benchmark based on local annotations to facilitate the evaluation of the embodied 3D occupancy prediction task. Our EmbodiedOcc outperforms existing methods by a large margin and accomplishes the embodied occupancy prediction with high accuracy and efficiency. Code: https://github.com/YkiWu/EmbodiedOcc.

Method: ALE combines Object-Restricted Embeddings (ORE) to disentangle text embeddings, Region-Guided Blending for Cross-Attention Masking (RGB-CAM) for spatially precise attention, and Background Blending (BB) to preserve non-edited content.

Result: Extensive experiments show ALE reduces attribute leakage by large margins, enabling accurate multi-object text-driven image editing while faithfully preserving non-target content.

Conclusion: ALE effectively addresses the root cause of attribute leakage at the source, providing a robust solution for precise text-based image editing without unintended side effects on unrelated regions.

Abstract: Text-based image editing, powered by generative diffusion models, lets users modify images through natural-language prompts and has dramatically simplified traditional workflows. Despite these advances, current methods still suffer from a critical problem: attribute leakage, where edits meant for specific objects unintentionally affect unrelated regions or other target objects. Our analysis reveals the root cause as the semantic entanglement inherent in End-of-Sequence (EOS) embeddings generated by autoregressive text encoders, which indiscriminately aggregate attributes across prompts. To address this issue, we introduce Attribute-Leakage-free Editing (ALE), a framework that tackles attribute leakage at its source. ALE combines Object-Restricted Embeddings (ORE) to disentangle text embeddings, Region-Guided Blending for Cross-Attention Masking (RGB-CAM) for spatially precise attention, and Background Blending (BB) to preserve non-edited content. To quantitatively evaluate attribute leakage across various editing methods, we propose the Attribute-Leakage Evaluation Benchmark (ALE-Bench), featuring comprehensive editing scenarios and new metrics. Extensive experiments show that ALE reduces attribute leakage by large margins, thereby enabling accurate, multi-object, text-driven image editing while faithfully preserving non-target content.

Method: Proposes CoMPaSS framework with two components: 1) SCOP data engine that curates spatially-accurate training data via principled constraints, and 2) TENOR module that preserves token ordering information to reinforce prompt’s linguistic structure.

Result: Achieves substantial gains on spatial benchmarks: +98% on VISOR, +67% on T2I-CompBench Spatial, and +131% on GenEval Position across four popular T2I models.

Conclusion: CoMPaSS effectively addresses core issues in spatial relationship rendering and sets new state-of-the-art performance, demonstrating the importance of both data curation and structural preservation in text encoding for spatial understanding.

Abstract: Text-to-image (T2I) diffusion models excel at generating photorealistic images but often fail to render accurate spatial relationships. We identify two core issues underlying this common failure: 1) the ambiguous nature of data concerning spatial relationships in existing datasets, and 2) the inability of current text encoders to accurately interpret the spatial semantics of input descriptions. We propose CoMPaSS, a versatile framework that enhances spatial understanding in T2I models. It first addresses data ambiguity with the Spatial Constraints-Oriented Pairing (SCOP) data engine, which curates spatially-accurate training data via principled constraints. To leverage these priors, CoMPaSS also introduces the Token ENcoding ORdering (TENOR) module, which preserves crucial token ordering information lost by text encoders, thereby reinforcing the prompt’s linguistic structure. Extensive experiments on four popular T2I models (UNet and MMDiT-based) show CoMPaSS sets a new state of the art on key spatial benchmarks, with substantial relative gains on VISOR (+98%), T2I-CompBench Spatial (+67%), and GenEval Position (+131%). Code is available at https://github.com/blurgyy/CoMPaSS.

Method: Develops an augmentation agent using LLMs and diffusion models to automatically generate additional images. Includes prompt self-refinement mechanism for coherent prompts and online filter for quality control during diffusion generation.

Result: Significantly surpasses state-of-the-art WSSS approaches on PASCAL VOC 2012 and MS COCO 2014 datasets.

Conclusion: Enhancing WSSS from a data generation perspective through automated image augmentation is effective and outperforms existing methods.

Abstract: Weakly-supervised semantic segmentation (WSSS) has achieved remarkable progress using only image-level labels. However, most existing WSSS methods focus on designing new network structures and loss functions to generate more accurate dense labels, overlooking the limitations imposed by fixed datasets, which can constrain performance improvements. We argue that more diverse trainable images provides WSSS richer information and help model understand more comprehensive semantic pattern. Therefore in this paper, we introduce a novel approach called Image Augmentation Agent (IAA) which shows that it is possible to enhance WSSS from data generation perspective. IAA mainly design an augmentation agent that leverages large language models (LLMs) and diffusion models to automatically generate additional images for WSSS. In practice, to address the instability in prompt generation by LLMs, we develop a prompt self-refinement mechanism. It allow LLMs to re-evaluate the rationality of generated prompts to produce more coherent prompts. Additionally, we insert an online filter into diffusion generation process to dynamically ensure the quality and balance of generated images. Experimental results show that our method significantly surpasses state-of-the-art WSSS approaches on the PASCAL VOC 2012 and MS COCO 2014 datasets.

Method: Proposes Hadamard Attention Recurrent Stereo Transformer (HART) with Dense Attention Kernel that maps attention weights to high-dimensional space without upper bounds, and Multi Kernel & Order Interaction module that unifies semantic and spatial knowledge learning.

Result: HART ranked 1st on KITTI 2012 benchmark among all published methods for reflective areas at time of submission.

Conclusion: The proposed attention mechanism successfully overcomes the low-rank bottleneck, enabling more flexible modeling of complex feature interactions and reducing feature collinearity in stereo matching.

Abstract: Constrained by the low-rank bottleneck inherent in attention mechanisms, current stereo matching transformers suffer from limited nonlinear expressivity, which renders their feature representations sensitive to challenging conditions such as reflections. To overcome this difficulty, we present the Hadamard Attention Recurrent Stereo Transformer (HART). HART includes a novel attention mechanism that incorporates the following components: 1) The Dense Attention Kernel (DAK) maps the attention weight distribution into a high-dimensional space over (0, +$\infty$). By removing the upper bound constraint on attention weights, DAK enables more flexible modeling of complex feature interactions. This reduces feature collinearity. 2) The Multi Kernel & Order Interaction (MKOI) module extends the attention mechanism by unifying semantic and spatial knowledge learning. This integration improves the ability of HART to learn features in binocular images. Experimental results demonstrate the effectiveness of our HART. In reflective area, HART ranked 1st on the KITTI 2012 benchmark among all published methods at the time of submission. Code is available at https://github.com/ZYangChen/HART.

Method: Evaluated MLLMs on: (1) geometric primitive understanding, (2) multi-step reasoning, and (3) proposed Visually Cued Chain-of-Thought (VC-CoT) prompting that explicitly references visual annotations in diagrams to enhance reasoning.

Result: MLLMs show fundamental shortcomings in shape recognition (<50% accuracy on regular polygons). They fail to count sides of both familiar and novel shapes, indicating they haven’t learned the concept of sides. VC-CoT prompting dramatically improved GPT-4o’s accuracy from 7% to 93% on irregular polygon counting.

Conclusion: System 2 reasoning in MLLMs remains an open problem. MLLMs rely on intuitive System 1 thinking rather than deliberate reasoning. Visually-guided prompting like VC-CoT is essential for engaging visual reasoning capabilities in mathematical problem-solving.

Abstract: Despite strong performance on vision-language tasks, Multimodal Large Language Models (MLLMs) struggle with mathematical problem-solving, with both open-source and state-of-the-art models falling short of human performance on visual-math benchmarks. To systematically examine visual-mathematical reasoning in MLLMs, we (1) evaluate their understanding of geometric primitives, (2) test multi-step reasoning, and (3) explore a potential solution to improve visual reasoning capabilities. Our findings reveal fundamental shortcomings in shape recognition, with top models achieving under 50% accuracy in identifying regular polygons. We analyze these failures through the lens of dual-process theory and show that MLLMs rely on System 1 (intuitive, memorized associations) rather than System 2 (deliberate reasoning). Consequently, MLLMs fail to count the sides of both familiar and novel shapes, suggesting they have neither learned the concept of sides nor effectively process visual inputs. Finally, we propose Visually Cued Chain-of-Thought (VC-CoT) prompting, which enhances multi-step mathematical reasoning by explicitly referencing visual annotations in diagrams, boosting GPT-4o’s accuracy on an irregular polygon side-counting task from 7% to 93%. Our findings suggest that System 2 reasoning in MLLMs remains an open problem, and visually-guided prompting is essential for successfully engaging visual reasoning. Code available at: https://github.com/rsinghlab/Shape-Blind.

Method: Uses a novel Variational Autoencoder to jointly encode RGB, depth, and surface normals into a shared latent space, combined with a latent diffusion model that denoises these latents in a single unified process.

Result: Competitive performance against state-of-the-art task-specific methods, surpassing them in normal-prediction accuracy and depth-normal consistency. Also achieves more qualitative realism in joint color-depth-normal inpainting compared to multi-step methods.

Conclusion: Orchid demonstrates that a unified latent diffusion model can efficiently learn joint appearance-geometry priors, enabling coherent generation of multiple image modalities with superior consistency and realism compared to separate specialized approaches.

Abstract: We introduce Orchid, a unified latent diffusion model that learns a joint appearance-geometry prior to generate color, depth, and surface normal images in a single diffusion process. This unified approach is more efficient and coherent than current pipelines that use separate models for appearance and geometry. Orchid is versatile - it directly generates color, depth, and normal images from text, supports joint monocular depth and normal estimation with color-conditioned finetuning, and seamlessly inpaints large 3D regions by sampling from the joint distribution. It leverages a novel Variational Autoencoder (VAE) that jointly encodes RGB, relative depth, and surface normals into a shared latent space, combined with a latent diffusion model that denoises these latents. Our extensive experiments demonstrate that Orchid delivers competitive performance against SOTA task-specific methods for geometry prediction, even surpassing them in normal-prediction accuracy and depth-normal consistency. It also inpaints color-depth-normal images jointly, with more qualitative realism than existing multi-step methods.

Method: Guidance-Free Training (GFT) uses the same maximum likelihood objective as CFG but with different parameterization of conditional models. It can be trained from scratch without relying on pretrained CFG networks and requires minimal code modifications.

Result: GFT achieves comparable or better FID scores across diffusion, autoregressive, and masked-prediction models, with similar diversity-fidelity trade-offs as CFG baselines, while halving computational costs.

Conclusion: GFT provides an effective alternative to CFG that eliminates the need for guided sampling, reduces inference costs by 50%, and can be easily implemented in existing codebases while maintaining performance.

Abstract: Classifier-Free Guidance (CFG) has been a default technique in various visual generative models, yet it requires inference from both conditional and unconditional models during sampling. We propose to build visual models that are free from guided sampling. The resulting algorithm, Guidance-Free Training (GFT), matches the performance of CFG while reducing sampling to a single model, halving the computational cost. Unlike previous distillation-based approaches that rely on pretrained CFG networks, GFT enables training directly from scratch. GFT is simple to implement. It retains the same maximum likelihood objective as CFG and differs mainly in the parameterization of conditional models. Implementing GFT requires only minimal modifications to existing codebases, as most design choices and hyperparameters are directly inherited from CFG. Our extensive experiments across five distinct visual models demonstrate the effectiveness and versatility of GFT. Across domains of diffusion, autoregressive, and masked-prediction modeling, GFT consistently achieves comparable or even lower FID scores, with similar diversity-fidelity trade-offs compared with CFG baselines, all while being guidance-free. Code will be available at https://github.com/thu-ml/GFT.

Method: Proposes Multi-view Structural Convolution Network (MSCN) with Structural Convolution Layers for local geometric features and Structural Aggregation Layers for local and overall context features, plus an unseen domain generation strategy to mitigate domain gaps.

Result: Extensive experiments show MSCN consistently outperforms state-of-the-art point cloud classification methods across all domain change scenarios.

Conclusion: MSCN provides a scalable solution for deploying LiDAR-based perception systems in autonomous vehicles by achieving domain-invariant recognition.

Abstract: Although LiDAR sensors have become indispensable for autonomous vehicles (AVs) due to their ability to provide accurate 3D scene understanding and robust perception under adverse weather conditions, the properties of LiDAR point clouds vary widely across sensor configurations and data acquisition domains, leading to severe performance degradation when models are transferred between heterogeneous sensors or from simulation to the real world. To address this challenge, we propose the Multi-view Structural Convolution Network (MSCN), a novel architecture designed to achieve domain-invariant recognition across diverse LiDAR configurations and environments. MSCN comprises Structural Convolution Layers (SCL) that extract local context geometric features from point clouds and Structural Aggregation Layers (SAL) that extract and aggregate both local and overall context features from point clouds. Furthermore, we incorporate an unseen domain generation strategy to mitigate domain gaps during training. Extensive experiments demonstrate that MSCN consistently outperforms state-of-the-art point cloud classification methods across all domain change scenarios. These results highlight MSCN as a scalable solution for deploying LiDAR-based perception systems of AVs. Our code is available at https://github.com/MLMLab/MSCN.

Method: Uses sparse 3D Gaussian representation inspired by Gaussian Splatting, with a Gaussian Transformer architecture that eliminates expensive 3D convolutions. Estimates temporal flow for each Gaussian to capture scene dynamics, and employs weak supervision without requiring dense 3D voxel annotations.

Result: Significantly outperforms all previous methods for weakly supervised occupancy estimation on nuScenes dataset, with inference speed that is 50 times faster than current state-of-the-art methods.

Conclusion: GaussianFlowOcc provides an efficient, scalable solution for occupancy estimation that reduces computational requirements while effectively capturing scene dynamics, making it particularly suitable for autonomous driving applications.

Abstract: Occupancy estimation has become a prominent task in 3D computer vision, particularly within the autonomous driving community. In this paper, we present a novel approach to occupancy estimation, termed GaussianFlowOcc, which is inspired by Gaussian Splatting and replaces traditional dense voxel grids with a sparse 3D Gaussian representation. Our efficient model architecture based on a Gaussian Transformer significantly reduces computational and memory requirements by eliminating the need for expensive 3D convolutions used with inefficient voxel-based representations that predominantly represent empty 3D spaces. GaussianFlowOcc effectively captures scene dynamics by estimating temporal flow for each Gaussian during the overall network training process, offering a straightforward solution to a complex problem that is often neglected by existing methods. Moreover, GaussianFlowOcc is designed for scalability, as it employs weak supervision and does not require costly dense 3D voxel annotations based on additional data (e.g., LiDAR). Through extensive experimentation, we demonstrate that GaussianFlowOcc significantly outperforms all previous methods for weakly supervised occupancy estimation on the nuScenes dataset while featuring an inference speed that is 50 times faster than current SOTA.

Method: Uses direct feature similarity loss with MLP projection head for knowledge transfer, avoids pseudo-semantic maps, and introduces auxiliary self-supervised occupancy prediction task to enhance 3D spatial reasoning capabilities.

Result: Achieves state-of-the-art performance in both semantic segmentation and 3D object detection by up to 10% mIoU, especially effective when fine-tuning on low data amounts.

Conclusion: The proposed simple yet powerful knowledge distillation strategy effectively transfers 2D VFM capabilities to 3D models without complex components, demonstrating superior performance across multiple 3D perception tasks.

Abstract: Vision foundation models (VFMs) such as DINO have led to a paradigm shift in 2D camera-based perception towards extracting generalized features to support many downstream tasks. Recent works introduce self-supervised cross-modal knowledge distillation (KD) as a way to transfer these powerful generalization capabilities into 3D LiDAR-based models. However, they either rely on highly complex distillation losses, pseudo-semantic maps, or limit KD to features useful for semantic segmentation only. In this work, we propose CleverDistiller, a self-supervised, cross-modal 2D-to-3D KD framework introducing a set of simple yet effective design choices: Unlike contrastive approaches relying on complex loss design choices, our method employs a direct feature similarity loss in combination with a multi layer perceptron (MLP) projection head to allow the 3D network to learn complex semantic dependencies throughout the projection. Crucially, our approach does not depend on pseudo-semantic maps, allowing for direct knowledge transfer from a VFM without explicit semantic supervision. Additionally, we introduce the auxiliary self-supervised spatial task of occupancy prediction to enhance the semantic knowledge, obtained from a VFM through KD, with 3D spatial reasoning capabilities. Experiments on standard autonomous driving benchmarks for 2D-to-3D KD demonstrate that CleverDistiller achieves state-of-the-art performance in both semantic segmentation and 3D object detection (3DOD) by up to 10% mIoU, especially when fine tuning on really low data amounts, showing the effectiveness of our simple yet powerful KD strategy

Method: MedLoRD is a generative diffusion model specifically designed for resource-constrained environments, capable of generating high-resolution medical volumes (up to 512×512×256) using standard 24GB VRAM GPUs found in desktop workstations.

Result: The model generates high-fidelity medical images across multiple modalities (Coronary CT Angiography and Lung CT), closely adhering to segmentation mask conditions and outperforming current state-of-the-art generative models in resource-constrained settings.

Conclusion: MedLoRD provides a practical solution for synthetic medical data generation in computational resource-constrained healthcare environments, enabling privacy-preserving data sharing while maintaining clinical meaningfulness and image quality.

Abstract: Advancements in AI for medical imaging offer significant potential. However, their applications are constrained by the limited availability of data and the reluctance of medical centers to share it due to patient privacy concerns. Generative models present a promising solution by creating synthetic data as a substitute for real patient data. However, medical images are typically high-dimensional, and current state-of-the-art methods are often impractical for computational resource-constrained healthcare environments. These models rely on data sub-sampling, raising doubts about their feasibility and real-world applicability. Furthermore, many of these models are evaluated on quantitative metrics that alone can be misleading in assessing the image quality and clinical meaningfulness of the generated images. To address this, we introduce MedLoRD, a generative diffusion model designed for computational resource-constrained environments. MedLoRD is capable of generating high-dimensional medical volumes with resolutions up to 512$\times$512$\times$256, utilizing GPUs with only 24GB VRAM, which are commonly found in standard desktop workstations. MedLoRD is evaluated across multiple modalities, including Coronary Computed Tomography Angiography and Lung Computed Tomography datasets. Extensive evaluations through radiological evaluation, relative regional volume analysis, adherence to conditional masks, and downstream tasks show that MedLoRD generates high-fidelity images closely adhering to segmentation mask conditions, surpassing the capabilities of current state-of-the-art generative models for medical image synthesis in computational resource-constrained environments.

Method: E-IRFS extends IRFS by applying exponential scaling to the geometric mean of image and instance frequencies, creating a more adaptive rebalancing strategy that better differentiates between rare and frequent classes.

Result: E-IRFS improves detection performance by 22% over baseline and outperforms both RFS and IRFS, particularly for rare categories. It shows stronger effects on lightweight models with limited capacity.

Conclusion: E-IRFS effectively addresses class imbalance in object detection, making it suitable for real-time applications like UAV-based emergency monitoring in resource-constrained environments.

Abstract: Object detection models often struggle with class imbalance, where rare categories appear significantly less frequently than common ones. Existing sampling-based rebalancing strategies, such as Repeat Factor Sampling (RFS) and Instance-Aware Repeat Factor Sampling (IRFS), mitigate this issue by adjusting sample frequencies based on image and instance counts. However, these methods are based on linear adjustments, which limit their effectiveness in long-tailed distributions. This work introduces Exponentially Weighted Instance-Aware Repeat Factor Sampling (E-IRFS), an extension of IRFS that applies exponential scaling to better differentiate between rare and frequent classes. E-IRFS adjusts sampling probabilities using an exponential function applied to the geometric mean of image and instance frequencies, ensuring a more adaptive rebalancing strategy. We evaluate E-IRFS on a dataset derived from the Fireman-UAV-RGBT Dataset and four additional public datasets, using YOLOv11 object detection models to identify fire, smoke, people and lakes in emergency scenarios. The results show that E-IRFS improves detection performance by 22% over the baseline and outperforms RFS and IRFS, particularly for rare categories. The analysis also highlights that E-IRFS has a stronger effect on lightweight models with limited capacity, as these models rely more on data sampling strategies to address class imbalance. The findings demonstrate that E-IRFS improves rare object detection in resource-constrained environments, making it a suitable solution for real-time applications such as UAV-based emergency monitoring. The code is available at: https://github.com/futurians/E-IRFS.

Method: Introduced Self-Correction GUI Navigation task with interactive information completion, developed Navi-plus dataset with GUI follow-up Q&A pairs, and created Dual-Stream Trajectory Evaluation method for benchmarking.

Result: Agents equipped with GUI follow-up questioning capability can fully recover their performance when dealing with ambiguous user tasks.

Conclusion: Interactive questioning and self-correction mechanisms are essential for GUI automation agents to handle incomplete task descriptions and maintain optimal performance.

Abstract: Graphical user interfaces (GUI) automation agents are emerging as powerful tools, enabling humans to accomplish increasingly complex tasks on smart devices. However, users often inadvertently omit key information when conveying tasks, which hinders agent performance in the current agent paradigm that does not support immediate user intervention. To address this issue, we introduce a $\textbf{Self-Correction GUI Navigation}$ task that incorporates interactive information completion capabilities within GUI agents. We developed the $\textbf{Navi-plus}$ dataset with GUI follow-up question-answer pairs, alongside a $\textbf{Dual-Stream Trajectory Evaluation}$ method to benchmark this new capability. Our results show that agents equipped with the ability to ask GUI follow-up questions can fully recover their performance when faced with ambiguous user tasks.

Method: Proposes T* framework that reframes temporal search as spatial search using visual localization techniques from images, with adaptive zooming across temporal and spatial dimensions. Also introduces LV-Haystack dataset with 480 hours of videos and 15,092 human-annotated instances.

Result: T* significantly improves SOTA performance: boosts GPT-4o from 50.5% to 53.1% and LLaVA-OneVision-OV-72B from 56.5% to 62.4% on Longvideobench XL subset using only 32 frames inference budget.

Conclusion: The proposed T* framework effectively addresses temporal search challenges in long-form videos by leveraging spatial search techniques, demonstrating substantial performance improvements over existing methods with efficient computational requirements.

Abstract: Efficiently understanding long-form videos remains a significant challenge in computer vision. In this work, we revisit temporal search paradigms for long-form video understanding and address a fundamental issue pertaining to all state-of-the-art (SOTA) long-context vision-language models (VLMs). Our contributions are twofold: First, we frame temporal search as a Long Video Haystack problem: finding a minimal set of relevant frames (e.g., one to five) from tens of thousands based on specific queries. Upon this formulation, we introduce LV-Haystack, the first dataset with 480 hours of videos, 15,092 human-annotated instances for both training and evaluation aiming to improve temporal search quality and efficiency. Results on LV-Haystack highlight a significant research gap in temporal search capabilities, with current SOTA search methods only achieving 2.1% temporal F1 score on the Longvideobench subset. Next, inspired by visual search in images, we propose a lightweight temporal search framework, T* that reframes costly temporal search as spatial search. T* leverages powerful visual localization techniques commonly used in images and introduces an adaptive zooming-in mechanism that operates across both temporal and spatial dimensions. Extensive experiments show that integrating T* with existing methods significantly improves SOTA long-form video understanding. Under an inference budget of 32 frames, T* improves GPT-4o’s performance from 50.5% to 53.1% and LLaVA-OneVision-OV-72B’s performance from 56.5% to 62.4% on the Longvideobench XL subset. Our code, benchmark, and models are provided in the Supplementary material.

Method: Singular Value Decomposition-based Least Squares framework leveraging self-supervised and transfer learning features with closed-form, non-iterative classification approach.

Result: Achieves competitive performance with significantly reduced computational costs compared to traditional gradient-based methods.

Conclusion: SVD-LS provides a viable efficient alternative for real-time pneumonia classification in medical imaging applications.

Abstract: Accurate and early diagnosis of pneumonia through X-ray imaging is essential for effective treatment and improved patient outcomes. Recent advancements in machine learning have enabled automated diagnostic tools that assist radiologists in making more reliable and efficient decisions. In this work, we propose a Singular Value Decomposition-based Least Squares (SVD-LS) framework for multi-class pneumonia classification, leveraging powerful feature representations from state-of-the-art self-supervised and transfer learning models. Rather than relying on computationally expensive gradient-based fine-tuning, we employ a closed-form, non-iterative classification approach that ensures efficiency without compromising accuracy. Experimental results demonstrate that SVD-LS achieves competitive performance while offering significantly reduced computational costs, making it a viable alternative for real-time medical imaging applications. The implementation is available at: github.com/meterdogan07/SVD-LS.

Method: Introduces semantic regularization to align tokenizer features with semantically consistent features from pre-trained visual encoder. Also explores 1D tokenizers, decoder-first scaling, and entropy loss for billion-scale tokenizers.

Result: Achieves state-of-the-art performance in reconstruction, downstream AR generation, and representation quality with 3 billion parameters.

Conclusion: Semantic regularization effectively mitigates the reconstruction-generation trade-off in visual tokenizers, enabling successful scaling to billion parameters while improving both reconstruction and generation quality.

Abstract: In autoregressive (AR) image generation, visual tokenizers compress images into compact discrete latent tokens, enabling efficient training of downstream autoregressive models for visual generation via next-token prediction. While scaling visual tokenizers improves image reconstruction quality, it often degrades downstream generation quality – a challenge not adequately addressed in existing literature. To address this, we introduce GigaTok, the first approach to simultaneously improve image reconstruction, generation, and representation learning when scaling visual tokenizers. We identify the growing complexity of latent space as the key factor behind the reconstruction vs. generation dilemma. To mitigate this, we propose semantic regularization, which aligns tokenizer features with semantically consistent features from a pre-trained visual encoder. This constraint prevents excessive latent space complexity during scaling, yielding consistent improvements in both reconstruction and downstream autoregressive generation. Building on semantic regularization, we explore three key practices for scaling tokenizers:(1) using 1D tokenizers for better scalability, (2) prioritizing decoder scaling when expanding both encoder and decoder, and (3) employing entropy loss to stabilize training for billion-scale tokenizers. By scaling to $\bf{3 \space billion}$ parameters, GigaTok achieves state-of-the-art performance in reconstruction, downstream AR generation, and downstream AR representation quality.

Method: Proposes BM-MAE, a masked image modeling pre-training strategy that learns rich intra- and inter-modal representations, enabling the same pre-trained model to adapt to any available modality subset.

Result: Outperforms or remains competitive with baselines requiring separate pre-training for each modality subset, and substantially surpasses training from scratch on downstream tasks. Also efficiently reconstructs missing modalities.

Conclusion: BM-MAE provides a practical solution for handling missing modalities in multimodal MRI, enabling flexible clinical deployment with a single pre-trained model that works with any modality combination.

Abstract: Multimodal magnetic resonance imaging (MRI) constitutes the first line of investigation for clinicians in the care of brain tumors, providing crucial insights for surgery planning, treatment monitoring, and biomarker identification. Pre-training on large datasets have been shown to help models learn transferable representations and adapt with minimal labeled data. This behavior is especially valuable in medical imaging, where annotations are often scarce. However, applying this paradigm to multimodal medical data introduces a challenge: most existing approaches assume that all imaging modalities are available during both pre-training and fine-tuning. In practice, missing modalities often occur due to acquisition issues, specialist unavailability, or specific experimental designs on small in-house datasets. Consequently, a common approach involves training a separate model for each desired modality combination, making the process both resource-intensive and impractical for clinical use. Therefore, we introduce BM-MAE, a masked image modeling pre-training strategy tailored for multimodal MRI data. The same pre-trained model seamlessly adapts to any combination of available modalities, extracting rich representations that capture both intra- and inter-modal information. This allows fine-tuning on any subset of modalities without requiring architectural changes, while still benefiting from a model pre-trained on the full set of modalities. Extensive experiments show that the proposed pre-training strategy outperforms or remains competitive with baselines that require separate pre-training for each modality subset, while substantially surpassing training from scratch on several downstream tasks. Additionally, it can quickly and efficiently reconstruct missing modalities, highlighting its practical value. Code and trained models are available at: https://github.com/Lucas-rbnt/BM-MAE

Method: Proposes LEL framework that integrates Lipschitz continuity constraints for model stability and generalization, combined with ensemble learning strategy that fuses decisions from multiple classifiers to reduce bias and variance.

Result: Achieved average recognition accuracies of 76.43% on EAV, 83.00% on FACED, and 87.22% on SEED datasets, demonstrating state-of-the-art performance.

Conclusion: LEL effectively addresses key limitations in EEG-based emotion recognition by enhancing model stability, accuracy, and robustness through Lipschitz constraints and ensemble learning.

Abstract: The accurate and efficient recognition of emotional states in oneself and others is critical, as impairments in this ability can lead to significant psychosocial difficulties. While electroencephalography (EEG) offers a powerful tool for emotion detection, current EEG-based emotion recognition (EER) methods face key limitations: insufficient model stability, limited accuracy in processing high-dimensional nonlinear EEG signals, and poor robustness against intra-subject variability and signal noise. To address these challenges, we introduce LEL (Lipschitz continuity-constrained Ensemble Learning), a novel framework that enhances EEG-based emotion recognition. By integrating Lipschitz continuity constraints, LEL ensures greater model stability and improves generalization, thereby reducing sensitivity to signal variability and noise while significantly boosting the model’s overall accuracy and robustness. Its ensemble learning strategy optimizes overall performance by fusing decisions from multiple classifiers to reduce single-model bias and variance. Experimental results on three public benchmark datasets (EAV, FACED and SEED) demonstrated the LEL’s state-of-the-art performance, achieving average recognition accuracies of 76.43%, 83.00% and 87.22%, respectively. The official implementation codes are released at https://github.com/NZWANG/LEL.

Method: Proposes AffordanceSAM with affordance-adaption module, uses C2F-Aff coarse-to-fine annotated dataset, and employs three-stage training to transfer SAM’s segmentation capabilities to affordance grounding.

Result: Achieves state-of-the-art performance on AGD20K benchmark and demonstrates strong generalization capacity.

Conclusion: AffordanceSAM successfully overcomes limitations of fully supervised affordance grounding by leveraging SAM’s segmentation generalization through specialized adaptation and training methodology.

Abstract: Building a generalized affordance grounding model to identify actionable regions on objects is vital for real-world applications. Existing methods to train the model can be divided into weakly and fully supervised ways. However, the former method requires a complex training framework design and can not infer new actions without an auxiliary prior. While the latter often struggle with limited annotated data and components trained from scratch despite being simpler. This study focuses on fully supervised affordance grounding and overcomes its limitations by proposing AffordanceSAM, which extends SAM’s generalization capacity in segmentation to affordance grounding. Specifically, we design an affordance-adaption module and curate a coarse-to-fine annotated dataset called C2F-Aff to thoroughly transfer SAM’s robust performance to affordance in a three-stage training manner. Experimental results confirm that AffordanceSAM achieves state-of-the-art (SOTA) performance on the AGD20K benchmark and exhibits strong generalized capacity.

Method: Introduces background motion learner (BML) to learn background motions from human pose sequences, and deploys epipolar constraint on 3D attention map with a carefully constructed mask combining epipolar mask and current 3D attention.

Result: Extensive experiments show the framework effectively learns background motion from human poses, achieving state-of-the-art performance with vivid and realistic backgrounds.

Conclusion: AnimateAnywhere successfully generates human animation with dynamic backgrounds without requiring camera trajectories, making it practical for entertainment applications and ordinary users.

Abstract: Human image animation aims to generate human videos of given characters and backgrounds that adhere to the desired pose sequence. However, existing methods focus more on human actions while neglecting the generation of background, which typically leads to static results or inharmonious movements. The community has explored camera pose-guided animation tasks, yet preparing the camera trajectory is impractical for most entertainment applications and ordinary users. As a remedy, we present an AnimateAnywhere framework, rousing the background in human image animation without requirements on camera trajectories. In particular, based on our key insight that the movement of the human body often reflects the motion of the background, we introduce a background motion learner (BML) to learn background motions from human pose sequences. To encourage the model to learn more accurate cross-frame correspondences, we further deploy an epipolar constraint on the 3D attention map. Specifically, the mask used to suppress geometrically unreasonable attention is carefully constructed by combining an epipolar mask and the current 3D attention map. Extensive experiments demonstrate that our AnimateAnywhere effectively learns the background motion from human pose sequences, achieving state-of-the-art performance in generating human animation results with vivid and realistic backgrounds. The source code and model will be available at https://github.com/liuxiaoyu1104/AnimateAnywhere.

Method: Integrates mesh and spherical harmonic (SH) texture into learning process, uses novel interpolation method for rendering, employs deferred rendering and transfers only frustum SH textures to GPU for training while storing others in CPU RAM.

Result: Achieves state-of-the-art performance on interpolation and extrapolation sequences in Replica and FAST-LIVO2 datasets compared to 3D Gaussian Splatting and M2-Mapping methods.

Conclusion: Mesh-Learner provides an efficient, rasterization-compatible framework for 3D reconstruction that can handle unlimited scenes with moderate GPU memory requirements while delivering superior rendering quality.

Abstract: In this paper, we present a 3D reconstruction and rendering framework termed Mesh-Learner that is natively compatible with traditional rasterization pipelines. It integrates mesh and spherical harmonic (SH) texture (i.e., texture filled with SH coefficients) into the learning process to learn each mesh s view-dependent radiance end-to-end. Images are rendered by interpolating surrounding SH Texels at each pixel s sampling point using a novel interpolation method. Conversely, gradients from each pixel are back-propagated to the related SH Texels in SH textures. Mesh-Learner exploits graphic features of rasterization pipeline (texture sampling, deferred rendering) to render, which makes Mesh-Learner naturally compatible with tools (e.g., Blender) and tasks (e.g., 3D reconstruction, scene rendering, reinforcement learning for robotics) that are based on rasterization pipelines. Our system can train vast, unlimited scenes because we transfer only the SH textures within the frustum to the GPU for training. At other times, the SH textures are stored in CPU RAM, which results in moderate GPU memory usage. The rendering results on interpolation and extrapolation sequences in the Replica and FAST-LIVO2 datasets achieve state-of-the-art performance compared to existing state-of-the-art methods (e.g., 3D Gaussian Splatting and M2-Mapping). To benefit the society, the code will be available at https://github.com/hku-mars/Mesh-Learner.

Method: Multi-task learning foundation model trained on 14 tasks/datasets, functioning as an embedding extractor for diverse input modalities (RGB, thermal, depth videos, ECG, EMG, GSR, fNIRS). Uses Embedding-Mixer transformer module for final pain assessment.

Result: Achieves state-of-the-art performance on BioVid and AI4Pain datasets, outperforming 75 different methodologies. Effectively extracts high-quality embeddings from diverse modalities in both unimodal and multimodal settings.

Conclusion: PainFormer demonstrates superior performance across modalities and paves the way for general-purpose models in automatic pain assessment. Model architecture and weights are publicly available.

Abstract: Pain is a manifold condition that impacts a significant percentage of the population. Accurate and reliable pain evaluation for the people suffering is crucial to developing effective and advanced pain management protocols. Automatic pain assessment systems provide continuous monitoring and support decision-making processes, ultimately aiming to alleviate distress and prevent functionality decline. This study introduces PainFormer, a vision foundation model based on multi-task learning principles trained simultaneously on 14 tasks/datasets with a total of 10.9 million samples. Functioning as an embedding extractor for various input modalities, the foundation model provides feature representations to the Embedding-Mixer, a transformer-based module that performs the final pain assessment. Extensive experiments employing behavioral modalities - including RGB, synthetic thermal, and estimated depth videos - and physiological modalities such as ECG, EMG, GSR, and fNIRS revealed that PainFormer effectively extracts high-quality embeddings from diverse input modalities. The proposed framework is evaluated on two pain datasets, BioVid and AI4Pain, and directly compared to 75 different methodologies documented in the literature. Experiments conducted in unimodal and multimodal settings demonstrate state-of-the-art performances across modalities and pave the way toward general-purpose models for automatic pain assessment. The foundation model’s architecture (code) and weights are available at: https://github.com/GkikasStefanos/PainFormer.

Method: Heterogeneous Mixture of Hash Experts (HMoHE) network with a gating network that learns scene decomposition and allocates 3D points to specialized NeRF experts. Uses hash-based gating network and distinct heterogeneous hash experts with different resolution ranges.

Result: Achieves state-of-the-art scene rendering accuracy on large-scale datasets, including a new very large-scale dataset (>6.5km²). Shows 8x training acceleration and 16x rendering acceleration compared to Switch-NeRF.

Conclusion: The framework provides an end-to-end, highly scalable NeRF solution for real-world large-scale scene modeling that achieves both quality and efficiency, easily scaling to various large-scale scenes.

Abstract: Recent NeRF methods on large-scale scenes have underlined the importance of scene decomposition for scalable NeRFs. Although achieving reasonable scalability, there are several critical problems remaining unexplored, i.e., learnable decomposition, modeling scene heterogeneity, and modeling efficiency. In this paper, we introduce Switch-NeRF++, a Heterogeneous Mixture of Hash Experts (HMoHE) network that addresses these challenges within a unified framework. It is a highly scalable NeRF that learns heterogeneous decomposition and heterogeneous NeRFs efficiently for large-scale scenes in an end-to-end manner. In our framework, a gating network learns to decompose scenes and allocates 3D points to specialized NeRF experts. This gating network is co-optimized with the experts by our proposed Sparsely Gated Mixture of Experts (MoE) NeRF framework. We incorporate a hash-based gating network and distinct heterogeneous hash experts. The hash-based gating efficiently learns the decomposition of the large-scale scene. The distinct heterogeneous hash experts consist of hash grids of different resolution ranges, enabling effective learning of the heterogeneous representation of different scene parts. These design choices make our framework an end-to-end and highly scalable NeRF solution for real-world large-scale scene modeling to achieve both quality and efficiency. We evaluate our accuracy and scalability on existing large-scale NeRF datasets and a new dataset with very large-scale scenes ($>6.5km^2$) from UrbanBIS. Extensive experiments demonstrate that our approach can be easily scaled to various large-scale scenes and achieve state-of-the-art scene rendering accuracy. Furthermore, our method exhibits significant efficiency, with an 8x acceleration in training and a 16x acceleration in rendering compared to Switch-NeRF. Codes will be released at https://github.com/MiZhenxing/Switch-NeRF.

Method: Uses a lightweight View Introspection Network (VIN) to predict Relative Reconstruction Improvement (RRI) of potential viewpoints without new acquisitions, enabling a greedy NBV policy.

Result: Achieves ~30% gain in reconstruction quality over coverage-based methods and ~40% improvement over deep reinforcement learning approaches (Scan-RL and GenNBV).

Conclusion: Directly optimizing for reconstruction quality rather than coverage leads to significantly better 3D scene acquisition, with generalization to unseen categories and adaptability to resource constraints.

Abstract: Next Best View (NBV) algorithms aim to maximize 3D scene acquisition quality using minimal resources, e.g. number of acquisitions, time taken, or distance traversed. Prior methods often rely on coverage maximization as a proxy for reconstruction quality, but for complex scenes with occlusions and finer details, this is not always sufficient and leads to poor reconstructions. Our key insight is to train an acquisition policy that directly optimizes for reconstruction quality rather than just coverage. To achieve this, we introduce the View Introspection Network (VIN): a lightweight neural network that predicts the Relative Reconstruction Improvement (RRI) of a potential next viewpoint without making any new acquisitions. We use this network to power a simple, yet effective, sequential samplingbased greedy NBV policy. Our approach, VIN-NBV, generalizes to unseen object categories, operates without prior scene knowledge, is adaptable to resource constraints, and can handle occlusions. We show that our RRI fitness criterion leads to a ~30% gain in reconstruction quality over a coverage-based criterion using the same greedy strategy. Furthermore, VIN-NBV also outperforms deep reinforcement learning methods, Scan-RL and GenNBV, by ~40%.

Method: Introduces Disease-Aware Prompting (DAP) which uses the explainability map of a Vision-Language Model to identify appropriate image features, amplifying disease-relevant regions while suppressing background interference.

Result: DAP improves visual grounding accuracy by 20.74% compared to state-of-the-art methods across three major chest X-ray datasets, without requiring any additional pixel-level annotations.

Conclusion: The simple yet effective DAP strategy successfully addresses attention misallocation in medical visual grounding, significantly improving accuracy and demonstrating the importance of focusing on disease-relevant features while minimizing background interference.

Abstract: Visual grounding (VG) is the capability to identify the specific regions in an image associated with a particular text description. In medical imaging, VG enhances interpretability by highlighting relevant pathological features corresponding to textual descriptions, improving model transparency and trustworthiness for wider adoption of deep learning models in clinical practice. Current models struggle to associate textual descriptions with disease regions due to inefficient attention mechanisms and a lack of fine-grained token representations. In this paper, we empirically demonstrate two key observations. First, current VLMs assign high norms to background tokens, diverting the model’s attention from regions of disease. Second, the global tokens used for cross-modal learning are not representative of local disease tokens. This hampers identifying correlations between the text and disease tokens. To address this, we introduce simple, yet effective Disease-Aware Prompting (DAP) process, which uses the explainability map of a VLM to identify the appropriate image features. This simple strategy amplifies disease-relevant regions while suppressing background interference. Without any additional pixel-level annotations, DAP improves visual grounding accuracy by 20.74% compared to state-of-the-art methods across three major chest X-ray datasets.

Method: Proposes an identity-conditional diffusion model with two lightweight control modules embedded in cross-attention layers. Uses cross-attention between identity features and non-identity control features to maintain orthogonality, enabling precise attribute manipulation with minimal parameters.

Result: Quantitative and qualitative evaluations plus perceptual user studies show superior performance in control accuracy for pose, expression, and emotion manipulation, while also improving generative diversity under identity-only conditioning compared to existing approaches.

Conclusion: The proposed method successfully addresses the challenge of disentangling identity from mutable facial attributes, achieving precise control over pose, expression, and emotion while preserving identity integrity and enhancing generative diversity.

Abstract: Human facial images encode a rich spectrum of information, encompassing both stable identity-related traits and mutable attributes such as pose, expression, and emotion. While recent advances in image generation have enabled high-quality identity-conditional face synthesis, precise control over non-identity attributes remains challenging, and disentangling identity from these mutable factors is particularly difficult. To address these limitations, we propose a novel identity-conditional diffusion model that introduces two lightweight control modules designed to independently manipulate facial pose, expression, and emotion without compromising identity preservation. These modules are embedded within the cross-attention layers of the base diffusion model, enabling precise attribute control with minimal parameter overhead. Furthermore, our tailored training strategy, which leverages cross-attention between the identity feature and each non-identity control feature, encourages identity features to remain orthogonal to control signals, enhancing controllability and diversity. Quantitative and qualitative evaluations, along with perceptual user studies, demonstrate that our method surpasses existing approaches in terms of control accuracy over pose, expression, and emotion, while also improving generative diversity under identity-only conditioning.

Method: Proposes UIIS10K dataset with 10K annotated images, uses Mask GAT-based Underwater Knowledge Distillation to transfer knowledge from SAM ViT-Huge to ViT-Small, and designs End-to-end Underwater Prompt Generator for automatic prompt generation.

Result: Achieves significant performance improvements over state-of-the-art methods on multiple underwater instance datasets with reduced computational requirements.

Conclusion: UWSAM provides an effective solution for underwater instance segmentation by combining knowledge distillation with automatic prompt generation, enabling accurate and efficient segmentation without manual prompts.

Abstract: With recent breakthroughs in large-scale modeling, the Segment Anything Model (SAM) has demonstrated significant potential in a variety of visual applications. However, due to the lack of underwater domain expertise, SAM and its variants face performance limitations in end-to-end underwater instance segmentation tasks, while their higher computational requirements further hinder their application in underwater scenarios. To address this challenge, we propose a large-scale underwater instance segmentation dataset, UIIS10K, which includes 10,048 images with pixel-level annotations for 10 categories. Then, we introduce UWSAM, an efficient model designed for automatic and accurate segmentation of underwater instances. UWSAM efficiently distills knowledge from the SAM ViT-Huge image encoder into the smaller ViT-Small image encoder via the Mask GAT-based Underwater Knowledge Distillation (MG-UKD) method for effective visual representation learning. Furthermore, we design an End-to-end Underwater Prompt Generator (EUPG) for UWSAM, which automatically generates underwater prompts instead of explicitly providing foreground points or boxes as prompts, thus enabling the network to locate underwater instances accurately for efficient segmentation. Comprehensive experimental results show that our model is effective, achieving significant performance improvements over state-of-the-art methods on multiple underwater instance datasets. Datasets and codes are available at https://github.com/LiamLian0727/UIIS10K.

Method: Used nine static RGB-D cameras, IMUs, and a HoloLens 2 headset to capture 3D hand, body, and eye movements from 16 subjects cooking four different recipes, with dense annotation of action segments.

Result: Created a multi-view action dataset with 33.78 action segments per minute, providing synchronized multi-modal data including vision, depth, inertial, gaze, and kinematic information.

Conclusion: The dataset enables four benchmarks for behavior understanding and modeling, expected to advance methods for understanding ecologically-valid human behavior in complex real-world tasks.

Abstract: Understanding behavior requires datasets that capture humans while carrying out complex tasks. The kitchen is an excellent environment for assessing human motor and cognitive function, as many complex actions are naturally exhibited in kitchens from chopping to cleaning. Here, we introduce the EPFL-Smart-Kitchen-30 dataset, collected in a noninvasive motion capture platform inside a kitchen environment. Nine static RGB-D cameras, inertial measurement units (IMUs) and one head-mounted HoloLens~2 headset were used to capture 3D hand, body, and eye movements. The EPFL-Smart-Kitchen-30 dataset is a multi-view action dataset with synchronized exocentric, egocentric, depth, IMUs, eye gaze, body and hand kinematics spanning 29.7 hours of 16 subjects cooking four different recipes. Action sequences were densely annotated with 33.78 action segments per minute. Leveraging this multi-modal dataset, we propose four benchmarks to advance behavior understanding and modeling through

1. a vision-language benchmark, 2) a semantic text-to-motion generation benchmark, 3) a multi-modal action recognition benchmark, 4) a pose-based action segmentation benchmark. We expect the EPFL-Smart-Kitchen-30 dataset to pave the way for better methods as well as insights to understand the nature of ecologically-valid human behavior. Code and data are available at https://github.com/amathislab/EPFL-Smart-Kitchen

Method: Proposes ANT architecture with: (i) Semantic Temporally Adaptive (STA) Module that automatically partitions denoising into low-frequency structural planning and high-frequency refinement via spectral analysis, and (ii) Dynamic Classifier-Free Guidance scheduling (DCFG) that adaptively adjusts conditional to unconditional ratio.

Result: Extensive experiments show ANT can be applied to various baselines, significantly improving model performance, and achieving state-of-the-art semantic alignment on StableMoFusion.

Conclusion: ANT successfully addresses the temporal-frequency mismatch in text-to-motion generation through biologically-inspired adaptive architecture, demonstrating superior performance and semantic alignment capabilities.

Abstract: While diffusion models advance text-to-motion generation, their static semantic conditioning ignores temporal-frequency demands: early denoising requires structural semantics for motion foundations while later stages need localized details for text alignment. This mismatch mirrors biological morphogenesis where developmental phases demand distinct genetic programs. Inspired by epigenetic regulation governing morphological specialization, we propose (ANT), an Adaptive Neural Temporal-Aware architecture. ANT orchestrates semantic granularity through: (i) Semantic Temporally Adaptive (STA) Module: Automatically partitions denoising into low-frequency structural planning and high-frequency refinement via spectral analysis. (ii) Dynamic Classifier-Free Guidance scheduling (DCFG): Adaptively adjusts conditional to unconditional ratio enhancing efficiency while maintaining fidelity. Extensive experiments show that ANT can be applied to various baselines, significantly improving model performance, and achieving state-of-the-art semantic alignment on StableMoFusion.

Method: Created WetCat dataset featuring high-resolution recordings of cataract surgeries performed by trainees on artificial eyes, with comprehensive phase annotations and semantic segmentations of key anatomical structures during critical phases (capsulorhexis and phacoemulsification).

Result: A publicly available dataset that enables development of interpretable, AI-driven evaluation tools aligned with clinical metrics, providing a foundation for objective surgical education.

Conclusion: WetCat sets a new benchmark for automated workflow analysis and skill assessment in ophthalmology training, facilitating scalable and objective surgical education.

Abstract: To meet the growing demand for systematic surgical training, wetlab environments have become indispensable platforms for hands-on practice in ophthalmology. Yet, traditional wetlab training depends heavily on manual performance evaluations, which are labor-intensive, time-consuming, and often subject to variability. Recent advances in computer vision offer promising avenues for automated skill assessment, enhancing both the efficiency and objectivity of surgical education. Despite notable progress in ophthalmic surgical datasets, existing resources predominantly focus on real surgeries or isolated tasks, falling short of supporting comprehensive skill evaluation in controlled wetlab settings. To address these limitations, we introduce WetCat, the first dataset of wetlab cataract surgery videos specifically curated for automated skill assessment. WetCat comprises high-resolution recordings of surgeries performed by trainees on artificial eyes, featuring comprehensive phase annotations and semantic segmentations of key anatomical structures. These annotations are meticulously designed to facilitate skill assessment during the critical capsulorhexis and phacoemulsification phases, adhering to standardized surgical skill assessment frameworks. By focusing on these essential phases, WetCat enables the development of interpretable, AI-driven evaluation tools aligned with established clinical metrics. This dataset lays a strong foundation for advancing objective, scalable surgical education and sets a new benchmark for automated workflow analysis and skill assessment in ophthalmology training. The dataset and annotations are publicly available in Synapse https://www.synapse.org/Synapse:syn66401174/files.

Method: Leverages ControlNet with modified multi-view decoder to generate NOCS maps containing 3D structure and position information. Integrates viewpoint encoder for sketch understanding and designs feature-level multi-view aggregation network as denoising module for cross-view information exchange.

Result: Experiments on ShapeNet demonstrate that DiffS-NOCS achieves controllable and fine-grained point cloud reconstruction aligned with input sketches.

Conclusion: The proposed method successfully addresses challenges in sketch-to-3D reconstruction by generating NOCS maps in 2D space and combining multiple views, enabling accurate and controllable 3D point cloud generation from sketches.

Abstract: Reconstructing a 3D point cloud from a given conditional sketch is challenging. Existing methods often work directly in 3D space, but domain variability and difficulty in reconstructing accurate 3D structures from 2D sketches remain significant obstacles. Moreover, ideal models should also accept prompts for control, in addition with the sparse sketch, posing challenges in multi-modal fusion. We propose DiffS-NOCS (Diffusion-based Sketch-to-NOCS Map), which leverages ControlNet with a modified multi-view decoder to generate NOCS maps with embedded 3D structure and position information in 2D space from sketches. The 3D point cloud is reconstructed by combining multiple NOCS maps from different views. To enhance sketch understanding, we integrate a viewpoint encoder for extracting viewpoint features. Additionally, we design a feature-level multi-view aggregation network as the denoising module, facilitating cross-view information exchange and improving 3D consistency in NOCS map generation. Experiments on ShapeNet demonstrate that DiffS-NOCS achieves controllable and fine-grained point cloud reconstruction aligned with sketches.

Method: Three key components: (1) Baltimore Atlas Dataset (0.3m resolution aerial imagery), (2) FreqWeaver Adapter for parameter-efficient transfer of SAM2 foundation model, (3) Uncertainty-Aware Teacher Student Framework for semi-supervised learning using unlabeled data.

Result: Achieves 1.78% IoU improvement over parameter-efficient tuning strategies and 3.44% IoU gain over state-of-the-art high-resolution remote sensing segmentation methods using only 5.96% of total model parameters.

Conclusion: The framework successfully addresses UHSR land cover classification challenges by reducing training data dependence while maintaining high accuracy, making it practical for real-world urban applications.

Abstract: Ultra-high Spatial Resolution (UHSR) Land Cover Classification is increasingly important for urban analysis, enabling fine-scale planning, ecological monitoring, and infrastructure management. It identifies land cover types on sub-meter remote sensing imagery, capturing details such as building outlines, road networks, and distinct boundaries. However, most existing methods focus on 1 m imagery and rely heavily on large-scale annotations, while UHSR data remain scarce and difficult to annotate, limiting practical applicability. To address these challenges, we introduce Baltimore Atlas, a UHSR land cover classification framework that reduces reliance on large-scale training data and delivers high-accuracy results. Baltimore Atlas builds on three key ideas: (1) Baltimore Atlas Dataset, a 0.3 m resolution dataset based on aerial imagery of Baltimore City; (2) FreqWeaver Adapter, a parameter-efficient adapter that transfers SAM2 to this domain, leveraging foundation model knowledge to reduce training data needs while enabling fine-grained detail and structural modeling; (3) Uncertainty-Aware Teacher Student Framework, a semi-supervised framework that exploits unlabeled data to further reduce training dependence and improve generalization across diverse scenes. Using only 5.96% of total model parameters, our approach achieves a 1.78% IoU improvement over existing parameter-efficient tuning strategies and a 3.44% IoU gain compared to state-of-the-art high-resolution remote sensing segmentation methods on the Baltimore Atlas Dataset.

Method: Leverages Cubify Anything as pre-trained VFM for single-view 3D object detection with bounding boxes, CLIP for open-vocabulary semantics, an association module with 3D NMS and box matching, and an optimization module using IoU-guided particle filtering for multi-view consistency.

Result: Achieves state-of-the-art performance among online methods on ScanNetV2 and CA-1M datasets, with great generalization abilities and real-time perception in environments exceeding 1000 square meters.

Conclusion: The proposed reconstruction-free paradigm enables memory-efficient and real-time 3D object detection with open-vocabulary capabilities, overcoming computational limitations of traditional methods while maintaining high performance.

Abstract: Open-vocabulary 3D object detection has gained significant interest due to its critical applications in autonomous driving and embodied AI. Existing detection methods, whether offline or online, typically rely on dense point cloud reconstruction, which imposes substantial computational overhead and memory constraints, hindering real-time deployment in downstream tasks. To address this, we propose a novel reconstruction-free online framework tailored for memory-efficient and real-time 3D detection. Specifically, given streaming posed RGB-D video input, we leverage Cubify Anything as a pre-trained visual foundation model (VFM) for single-view 3D object detection by bounding boxes, coupled with CLIP to capture open-vocabulary semantics of detected objects. To fuse all detected bounding boxes across different views into a unified one, we employ an association module for correspondences of multi-views and an optimization module to fuse the 3D bounding boxes of the same instance predicted in multi-views. The association module utilizes 3D Non-Maximum Suppression (NMS) and a box correspondence matching module, while the optimization module uses an IoU-guided efficient random optimization technique based on particle filtering to enforce multi-view consistency of the 3D bounding boxes while minimizing computational complexity. Extensive experiments on ScanNetV2 and CA-1M datasets demonstrate that our method achieves state-of-the-art performance among online methods. Benefiting from this novel reconstruction-free paradigm for 3D object detection, our method exhibits great generalization abilities in various scenarios, enabling real-time perception even in environments exceeding 1000 square meters.

Method: Uses wavelet transforms to analyze frequency subbands, introduces dynamic frequency regulation mechanism that separately adjusts low and high frequencies based on observed energy patterns. Training-free and plug-and-play approach.

Result: Significantly improves generative quality of various diffusion models and frameworks with negligible computational cost. Provides rigorous mathematical formulation of exposure bias.

Conclusion: The proposed wavelet-based frequency regulation effectively mitigates exposure bias in diffusion models without requiring retraining, offering a practical and efficient solution for quality enhancement.

Abstract: Diffusion models exhibit impressive generative capabilities but are significantly impacted by exposure bias. In this paper, we make a key observation: the energy of predicted noisy samples in the reverse process continuously declines compared to perturbed samples in the forward process. Building on this, we identify two important findings: 1) The reduction in energy follows distinct patterns in the low-frequency and high-frequency subbands; 2) The subband energy of reverse-process reconstructed samples is consistently lower than that of forward-process ones, and both are lower than the original data samples. Based on the first finding, we introduce a dynamic frequency regulation mechanism utilizing wavelet transforms, which separately adjusts the low- and high-frequency subbands. Leveraging the second insight, we derive the rigorous mathematical form of exposure bias. It is worth noting that, our method is training-free and plug-and-play, significantly improving the generative quality of various diffusion models and frameworks with negligible computational cost. The source code is available at https://github.com/kunzhan/wpp.

Method: Uses LaViLa video captioner with LLM, partitions videos into meaningful segments, incorporates static scene descriptions using LLaVA VLM, and fine-tunes LaViLa to produce both action and scene captions with controllable hybrid captioning based on scene change detection.

Result: Developed a controllable hybrid captioner that alternates between action and scene captions, creating more detailed caption logs and expanding answerable question space while improving pipeline efficiency compared to using separate models.

Conclusion: The hybrid captioning approach with scene change detection enables comprehensive text-based video memory that supports complex natural language queries through LLM reasoning, addressing video density and dimensionality challenges.

Abstract: Video data, especially long-form video, is extremely dense and high-dimensional. Text-based summaries of video content offer a way to represent query-relevant content in a much more compact manner than raw video. In addition, textual representations are easily ingested by state-of-the-art large language models (LLMs), which enable reasoning over video content to answer complex natural language queries. To solve this issue, we rely on the progressive construction of a text-based memory by a video captioner operating on shorter chunks of the video, where spatio-temporal modeling is computationally feasible. We explore ways to improve the quality of the activity log comprised solely of short video captions. Because the video captions tend to be focused on human actions, and questions may pertain to other information in the scene, we seek to enrich the memory with static scene descriptions using Vision Language Models (VLMs). Our video understanding system relies on the LaViLa video captioner in combination with a LLM to answer questions about videos. We first explored different ways of partitioning the video into meaningful segments such that the textual descriptions more accurately reflect the structure of the video content. Furthermore, we incorporated static scene descriptions into the captioning pipeline using LLaVA VLM, resulting in a more detailed and complete caption log and expanding the space of questions that are answerable from the textual memory. Finally, we have successfully fine-tuned the LaViLa video captioner to produce both action and scene captions, significantly improving the efficiency of the captioning pipeline compared to using separate captioning models for the two tasks. Our model, controllable hybrid captioner, can alternate between different types of captions according to special input tokens that signals scene changes detected in the video.

Method: Performed style transfer with style varying across artificial image areas formed by Voronoi cells, then used the style-transferred data to train semantic segmentation DNNs to reduce texture dependence and enhance shape-based feature reliance.

Result: Style transfer augmentation reduces texture bias and strongly increases robustness against common image corruptions and adversarial attacks in semantic segmentation, working for both CNNs and transformers on Cityscapes and PASCAL Context datasets.

Conclusion: Style transfer with Voronoi-based random areas is an effective method for reducing texture bias and improving robustness in semantic segmentation, demonstrating generality across different architectures and datasets.

Abstract: Recent research has investigated the shape and texture biases of deep neural networks (DNNs) in image classification which influence their generalization capabilities and robustness. It has been shown that, in comparison to regular DNN training, training with stylized images reduces texture biases in image classification and improves robustness with respect to image corruptions. In an effort to advance this line of research, we examine whether style transfer can likewise deliver these two effects in semantic segmentation. To this end, we perform style transfer with style varying across artificial image areas. Those random areas are formed by a chosen number of Voronoi cells. The resulting style-transferred data is then used to train semantic segmentation DNNs with the objective of reducing their dependence on texture cues while enhancing their reliance on shape-based features. In our experiments, it turns out that in semantic segmentation, style transfer augmentation reduces texture bias and strongly increases robustness with respect to common image corruptions as well as adversarial attacks. These observations hold for convolutional neural networks and transformer architectures on the Cityscapes dataset as well as on PASCAL Context, showing the generality of the proposed method.

Method: Proposes C-MIR framework with three key contributions: (1) eliminates reliance on pre-segmented data, (2) adapts ColBERT’s contextualized late interaction mechanism for 3D medical imaging, (3) comprehensive evaluation across four tumor sites using multiple feature extractors and database configurations.

Result: C-MIR demonstrates significant advantages: successfully adapts late interaction to volumetric images, effectively localizes regions of interest without pre-segmentation, shows promising improvements in tumor flagging (especially colon and lung tumors, p<0.05), and potential for improving tumor staging.

Conclusion: C-MIR bridges the gap between advanced retrieval techniques and practical healthcare applications, offering a computationally efficient alternative to systems requiring expensive data enrichment, and paves the way for improved diagnostic processes in medical imaging.

Abstract: The increasing volume of medical images poses challenges for radiologists in retrieving relevant cases. Content-based image retrieval (CBIR) systems offer potential for efficient access to similar cases, yet lack standardized evaluation and comprehensive studies. Building on prior studies for tumor characterization via CBIR, this study advances CBIR research for volumetric medical images through three key contributions: (1) a framework eliminating reliance on pre-segmented data and organ-specific datasets, aligning with large and unstructured image archiving systems, i.e. PACS in clinical practice; (2) introduction of C-MIR, a novel volumetric re-ranking method adapting ColBERT’s contextualized late interaction mechanism for 3D medical imaging; (3) comprehensive evaluation across four tumor sites using three feature extractors and three database configurations. Our evaluations highlight the significant advantages of C-MIR. We demonstrate the successful adaptation of the late interaction principle to volumetric medical images, enabling effective context-aware re-ranking. A key finding is C-MIR’s ability to effectively localize the region of interest, eliminating the need for pre-segmentation of datasets and offering a computationally efficient alternative to systems relying on expensive data enrichment steps. C-MIR demonstrates promising improvements in tumor flagging, achieving improved performance, particularly for colon and lung tumors (p<0.05). C-MIR also shows potential for improving tumor staging, warranting further exploration of its capabilities. Ultimately, our work seeks to bridge the gap between advanced retrieval techniques and their practical applications in healthcare, paving the way for improved diagnostic processes.

Method: Uses LLMs to break action labels into atomic descriptions (subject, motion, object), segments videos into atomic phases, employs fine-grained fusion of text and visual features, and multimodal matching for classification.

Result: Achieves state-of-the-art performance across multiple few-shot action recognition benchmarks.

Conclusion: LGA effectively leverages language guidance to capture rich spatiotemporal cues and improves few-shot action recognition by going beyond simple label semantics.

Abstract: Few-shot action recognition (FSAR) aims to classify human actions in videos with only a small number of labeled samples per category. The scarcity of training data has driven recent efforts to incorporate additional modalities, particularly text. However, the subtle variations in human posture, motion dynamics, and the object interactions that occur during different phases, are critical inherent knowledge of actions that cannot be fully exploited by action labels alone. In this work, we propose Language-Guided Action Anatomy (LGA), a novel framework that goes beyond label semantics by leveraging Large Language Models (LLMs) to dissect the essential representational characteristics hidden beneath action labels. Guided by the prior knowledge encoded in LLM, LGA effectively captures rich spatiotemporal cues in few-shot scenarios. Specifically, for text, we prompt an off-the-shelf LLM to anatomize labels into sequences of atomic action descriptions, focusing on the three core elements of action (subject, motion, object). For videos, a Visual Anatomy Module segments actions into atomic video phases to capture the sequential structure of actions. A fine-grained fusion strategy then integrates textual and visual features at the atomic level, resulting in more generalizable prototypes. Finally, we introduce a Multimodal Matching mechanism, comprising both video-video and video-text matching, to ensure robust few-shot classification. Experimental results demonstrate that LGA achieves state-of-the-art performance across multipe FSAR benchmarks.

Method: Two-stage framework: 1) Memorizing stage stores and retrieves general motion patterns, 2) Animating stage performs personalized synthesis using audio-driven speaking style features to emphasize appropriate facial motion types.

Result: MemoryTalker generates reliable personalized facial animation without additional prior information, outperforming state-of-the-art methods in quantitative, qualitative evaluations and user studies.

Conclusion: The proposed framework successfully enables realistic and accurate 3D facial motion synthesis reflecting individual speaking styles using only audio input, maximizing practical usability.

Abstract: Speech-driven 3D facial animation aims to synthesize realistic facial motion sequences from given audio, matching the speaker’s speaking style. However, previous works often require priors such as class labels of a speaker or additional 3D facial meshes at inference, which makes them fail to reflect the speaking style and limits their practical use. To address these issues, we propose MemoryTalker which enables realistic and accurate 3D facial motion synthesis by reflecting speaking style only with audio input to maximize usability in applications. Our framework consists of two training stages: 1-stage is storing and retrieving general motion (i.e., Memorizing), and 2-stage is to perform the personalized facial motion synthesis (i.e., Animating) with the motion memory stylized by the audio-driven speaking style feature. In this second stage, our model learns about which facial motion types should be emphasized for a particular piece of audio. As a result, our MemoryTalker can generate a reliable personalized facial animation without additional prior information. With quantitative and qualitative evaluations, as well as user study, we show the effectiveness of our model and its performance enhancement for personalized facial animation over state-of-the-art methods.

Method: Fuses information from sequential data into pixel space replay memory by distilling multiple real images into single images with pixel-level changes. Uses dual-alignment strategy to align current domain to previous one while adapting replay samples to current domain style.

Result: Achieves 4% and 6% higher accuracy on sequential tasks compared to state-of-the-art and other replay-based methods respectively, while preserving data privacy.

Conclusion: Pr²R effectively mitigates class-incremental challenges and domain shift forgetting while maintaining privacy, demonstrating significant improvement in replay effectiveness for lifelong person re-identification.

Abstract: Lifelong person re-identification (LReID) aims to incrementally accumulate knowledge across a sequence of tasks under domain shifts. Recently, replay-based methods have demonstrated strong effectiveness in LReID by rehearsing past samples stored in an auxiliary memory. However, storing historical exemplars raises concerns over data privacy. To avoid this, exemplar-free approaches attempt to match the distribution of past data without storing raw samples. Despite being privacy-friendly, these methods often suffer from performance degradation due to the forgetting of specific past knowledge representations. To this end, we propose to fuse information from sequential data into the pixel space in the replay memory, enabling Privacy-Preserving Replay (Pr$^2$R). More specifically, by distilling the training characteristics of multiple real images into a single image, the fused samples undergo pixel-level changes. This not only protects the privacy of the original data but also makes the replay samples more representative for sequential tasks. During the style replay phase, we align the current domain to the previous one while simultaneously adapting the replay samples to match the style of the current domain. This dual-alignment strategy effectively mitigates both class-incremental challenges and forgetting caused by domain shifts. Extensive experiments on multiple benchmarks show that the proposed method significantly improves replay effectiveness while preserving data privacy. Specifically, Pr$^2$R achieves 4% and 6% higher accuracy on sequential tasks compared to the current state-of-the-art and other replay-based methods, respectively.

Method: Proposed VPN paradigm using visual trajectory markings on top-view maps. Built VPN tasks in discrete/continuous settings (R2R-VP, R2R-CE-VP datasets). Introduced VPNet baseline with view-level and trajectory-level data augmentation strategies.

Result: Extensive experiments evaluated visual prompt forms, top-view map formats, and augmentation strategies. The approach demonstrates improved navigation performance through visual guidance.

Conclusion: Visual Prompt Navigation offers a more effective alternative to language-guided navigation, reducing ambiguity and being more user-friendly for non-experts through intuitive visual trajectory markings.

Abstract: While natural language is commonly used to guide embodied agents, the inherent ambiguity and verbosity of language often hinder the effectiveness of language-guided navigation in complex environments. To this end, we propose Visual Prompt Navigation (VPN), a novel paradigm that guides agents to navigate using only user-provided visual prompts within 2D top-view maps. This visual prompt primarily focuses on marking the visual navigation trajectory on a top-down view of a scene, offering intuitive and spatially grounded guidance without relying on language instructions. It is more friendly for non-expert users and reduces interpretive ambiguity. We build VPN tasks in both discrete and continuous navigation settings, constructing two new datasets, R2R-VP and R2R-CE-VP, by extending existing R2R and R2R-CE episodes with corresponding visual prompts. Furthermore, we introduce VPNet, a dedicated baseline network to handle the VPN tasks, with two data augmentation strategies: view-level augmentation (altering initial headings and prompt orientations) and trajectory-level augmentation (incorporating diverse trajectories from large-scale 3D scenes), to enhance navigation performance. Extensive experiments evaluate how visual prompt forms, top-view map formats, and data augmentation strategies affect the performance of visual prompt navigation. The code is available at https://github.com/farlit/VPN.

Method: An end-to-end deep learning pipeline integrating attention-enhanced U-Net and ResNet architectures trained on large simulated datasets with varying noise levels, drift magnitudes, and distortion types to enable generalization to experimental data.

Result: Quantitative evaluations show the pipeline reduces mean squared error by up to 50% during denoising, achieves sub-pixel center localization with average errors below 0.04 pixels, and outperforms traditional algorithms in noise suppression and diffraction pattern restoration.

Conclusion: 4D-PreNet facilitates high-throughput, reliable 4D-STEM real-time analysis for automated characterization by providing a robust, generalizable solution to data preprocessing bottlenecks.

Abstract: Automated experimentation with real time data analysis in scanning transmission electron microscopy (STEM) often require end-to-end framework. The four-dimensional scanning transmission electron microscopy (4D-STEM) with high-throughput data acquisition has been constrained by the critical bottleneck results from data preprocessing. Pervasive noise, beam center drift, and elliptical distortions during high-throughput acquisition inevitably corrupt diffraction patterns, systematically biasing quantitative measurements. Yet, conventional correction algorithms are often material-specific and fail to provide a robust, generalizable solution. In this work, we present 4D-PreNet, an end-to-end deep-learning pipeline that integrates attention-enhanced U-Net and ResNet architectures to simultaneously perform denoising, center correction, and elliptical distortion calibration. The network is trained on large, simulated datasets encompassing a wide range of noise levels, drift magnitudes, and distortion types, enabling it to generalize effectively to experimental data acquired under varying conditions. Quantitative evaluations demonstrate that our pipeline reduces mean squared error by up to 50% during denoising and achieves sub-pixel center localization in the center detection task, with average errors below 0.04 pixels. The outputs are bench-marked against traditional algorithms, highlighting improvements in both noise suppression and restoration of diffraction patterns, thereby facilitating high-throughput, reliable 4D-STEM real-time analysis for automated characterization.

Method: Uses reverse-expansion-forward-inference strategy to generate image-text chains of thought, creates SIF-50K dataset, and implements GRPO-SIF reinforced training with depth-informed visual grounding.

Result: Outperforms state-of-the-art methods in spatial understanding and fine-grained visual perception while maintaining strong general capabilities.

Conclusion: The framework effectively mimics human visual perception through iterative attention correction and spatial awareness, demonstrating significant improvements in complex visual reasoning tasks.

Abstract: Current multimodal large language models (MLLMs) still face significant challenges in complex visual tasks (e.g., spatial understanding, fine-grained perception). Prior methods have tried to incorporate visual reasoning, however, they fail to leverage attention correction with spatial cues to iteratively refine their focus on prompt-relevant regions. In this paper, we introduce SIFThinker, a spatially-aware “think-with-images” framework that mimics human visual perception. Specifically, SIFThinker enables attention correcting and image region focusing by interleaving depth-enhanced bounding boxes and natural language. Our contributions are twofold: First, we introduce a reverse-expansion-forward-inference strategy that facilitates the generation of interleaved image-text chains of thought for process-level supervision, which in turn leads to the construction of the SIF-50K dataset. Besides, we propose GRPO-SIF, a reinforced training paradigm that integrates depth-informed visual grounding into a unified reasoning pipeline, teaching the model to dynamically correct and focus on prompt-relevant regions. Extensive experiments demonstrate that SIFThinker outperforms state-of-the-art methods in spatial understanding and fine-grained visual perception, while maintaining strong general capabilities, highlighting the effectiveness of our method. Code: https://github.com/zhangquanchen/SIFThinker.

Method: Created VisR-Bench with over 35K high-quality QA pairs across 1.2K documents spanning 16 languages and three question types (figures, text, tables), including queries without explicit answers.

Result: MLLMs significantly outperform text-based and multimodal encoder models, but still struggle with structured tables and low-resource languages.

Conclusion: The benchmark enables fine-grained evaluation of multimodal retrieval and highlights key challenges in multilingual visual retrieval that need addressing.

Abstract: Most organizational data in this world are stored as documents, and visual retrieval plays a crucial role in unlocking the collective intelligence from all these documents. However, existing benchmarks focus on English-only document retrieval or only consider multilingual question-answering on a single-page image. To bridge this gap, we introduce VisR-Bench, a multilingual benchmark designed for question-driven multimodal retrieval in long documents. Our benchmark comprises over 35K high-quality QA pairs across 1.2K documents, enabling fine-grained evaluation of multimodal retrieval. VisR-Bench spans sixteen languages with three question types (figures, text, and tables), offering diverse linguistic and question coverage. Unlike prior datasets, we include queries without explicit answers, preventing models from relying on superficial keyword matching. We evaluate various retrieval models, including text-based methods, multimodal encoders, and MLLMs, providing insights into their strengths and limitations. Our results show that while MLLMs significantly outperform text-based and multimodal encoder models, they still struggle with structured tables and low-resource languages, highlighting key challenges in multilingual visual retrieval.

Method: Proposes Architectural Co-Design with parameter-efficient Conv-LoRA adapters to inject local inductive biases, and Dynamic Fusion Gateway that uses visual context to adaptively modulate text prompts for bidirectional fusion.

Result: Extensive experiments on industrial and medical benchmarks demonstrate superior accuracy and robustness compared to existing methods.

Conclusion: Synergistic co-design of feature representation and cross-modal fusion is critical for robustly adapting foundation models to dense perception tasks like anomaly detection.

Abstract: Pre-trained Vision-Language Models (VLMs) face a significant adaptation gap when applied to Zero-Shot Anomaly Detection (ZSAD), stemming from their lack of local inductive biases for dense prediction and their reliance on inflexible feature fusion paradigms. We address these limitations through an Architectural Co-Design framework that jointly refines feature representation and cross-modal fusion. Our method proposes a parameter-efficient Convolutional Low-Rank Adaptation (Conv-LoRA) adapter to inject local inductive biases for fine-grained representation, and introduces a Dynamic Fusion Gateway (DFG) that leverages visual context to adaptively modulate text prompts, enabling a powerful bidirectional fusion. Extensive experiments on diverse industrial and medical benchmarks demonstrate superior accuracy and robustness, validating that this synergistic co-design is critical for robustly adapting foundation models to dense perception tasks.

Method: Uses autoregressive video generation with physical tokens combining frames and actions, DiT-based de-tokenizer for continuous modeling, causal mask with inverse kinematics, parallel training, and KV-cache mechanism.

Result: Achieves 100% success rate on PushCube task in ManiSkill benchmark, matches action-pretrained baselines on other tasks, and accurately predicts future videos with aligned action trajectories.

Conclusion: Demonstrates promising direction for robotic manipulation by transferring world knowledge from autoregressive video pretraining without requiring action-specific pretraining.

Abstract: The scarcity of manipulation data has motivated the use of pretrained large models from other modalities in robotics. In this work, we build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR), where physical tokens combine frames and actions to represent the joint evolution of the robot and its environment. PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining, enabling accurate video prediction and consistent action trajectories. It also adopts a DiT-based de-tokenizer to model frames and actions as continuous tokens, mitigating quantization errors and facilitating mutual enhancement. Furthermore, we incorporate a causal mask with inverse kinematics, parallel training, and the KV-cache mechanism to further improve performance and efficiency. Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task, matches the performance of action-pretrained baselines on other tasks, and accurately predicts future videos with tightly aligned action trajectories. These findings underscore a promising direction for robotic manipulation by transferring world knowledge from autoregressive video pretraining.The project page is here: https://hcplab-sysu.github.io/PhysicalAutoregressiveModel/

Method: The paper proposes leveraging event cameras, which provide low power consumption, high temporal resolution, and high dynamic range capabilities, to overcome the limitations of traditional RGB-based approaches for novel view synthesis and 4D reconstruction.

Result: Event cameras enable more robust scene reconstruction in high-speed motion scenarios and challenging lighting conditions where traditional RGB cameras fail, providing a new perspective for addressing these reconstruction challenges.

Conclusion: Event cameras present a promising alternative to RGB cameras for novel view synthesis and 4D reconstruction, particularly excelling in situations involving high-speed motion, low lighting, and extreme dynamic range conditions where conventional camera systems struggle.

Abstract: Novel view synthesis and 4D reconstruction techniques predominantly rely on RGB cameras, thereby inheriting inherent limitations such as the dependence on adequate lighting, susceptibility to motion blur, and a limited dynamic range. Event cameras, offering advantages of low power, high temporal resolution and high dynamic range, have brought a new perspective to addressing the scene reconstruction challenges in high-speed motion and

Method: Integrates three components: Fourier feature-based MLP for positional frequency encoding, cross-resolution Correlation Attention Layer for spatial alignment, and frequency-domain L1 loss for spectral fidelity supervision.

Result: Adds only 0.3M parameters but achieves average PSNR gains of 0.12~0.14 dB, improves frequency-domain consistency by up to 29%, and shows significant improvements on texture-rich datasets with reduced aliasing.

Conclusion: FGA is a practical, scalable alternative to traditional upsampling methods that effectively preserves fine details and reduces artifacts across diverse super-resolution backbones.

Abstract: We propose Frequency-Guided Attention (FGA), a lightweight upsampling module for single image super-resolution. Conventional upsamplers, such as Sub-Pixel Convolution, are efficient but frequently fail to reconstruct high-frequency details and introduce aliasing artifacts. FGA addresses these issues by integrating (1) a Fourier feature-based Multi-Layer Perceptron (MLP) for positional frequency encoding, (2) a cross-resolution Correlation Attention Layer for adaptive spatial alignment, and (3) a frequency-domain L1 loss for spectral fidelity supervision. Adding merely 0.3M parameters, FGA consistently enhances performance across five diverse super-resolution backbones in both lightweight and full-capacity scenarios. Experimental results demonstrate average PSNR gains of 0.12~0.14 dB and improved frequency-domain consistency by up to 29%, particularly evident on texture-rich datasets. Visual and spectral evaluations confirm FGA’s effectiveness in reducing aliasing and preserving fine details, establishing it as a practical, scalable alternative to traditional upsampling methods.

Method: Using three handcrafted feature maps as inputs to a multilayer perceptron (MLP) classifier for handwritten character recognition.

Result: 97% accuracy on MNIST digits and 89% accuracy on EMNIST letters.

Conclusion: Curvature-based representations have strong discriminative power for handwritten characters, and deep learning advantages can be achieved with interpretable hand-engineered features rather than complex CNNs.

Abstract: This study investigates whether second-order geometric cues - planar curvature magnitude, curvature sign, and gradient orientation - are sufficient on their own to drive a multilayer perceptron (MLP) classifier for handwritten character recognition (HCR), offering an alternative to convolutional neural networks (CNNs). Using these three handcrafted feature maps as inputs, our curvature-orientation MLP achieves 97 percent accuracy on MNIST digits and 89 percent on EMNIST letters. These results underscore the discriminative power of curvature-based representations for handwritten character images and demonstrate that the advantages of deep learning can be realized even with interpretable, hand-engineered features.

Method: Train a multilayer perceptron (MLP) using only horizontal and vertical Sobel derivatives as input features on MNIST and EMNIST Letters datasets.

Result: Achieved 98% accuracy on MNIST digits and 92% on EMNIST letters, approaching CNN performance while offering smaller memory footprint and more transparent features.

Conclusion: First-order gradients capture most class-discriminative information in handwritten characters, making edge-aware MLPs a compelling alternative to CNNs for HCR tasks.

Abstract: We revisit the classical Sobel operator to ask a simple question: Are first-order edge maps sufficient to drive an all-dense multilayer perceptron (MLP) for handwritten character recognition (HCR), as an alternative to convolutional neural networks (CNNs)? Using only horizontal and vertical Sobel derivatives as input, we train an MLP on MNIST and EMNIST Letters. Despite its extreme simplicity, the resulting network reaches 98% accuracy on MNIST digits and 92% on EMNIST letters – approaching CNNs while offering a smaller memory footprint and transparent features. Our findings highlight that much of the class-discriminative information in handwritten character images is already captured by first-order gradients, making edge-aware MLPs a compelling option for HCR.

Method: End-to-end diffusion-based approach combining a frozen UNet to preserve appearance/background details with a ControlNet guided by target SMPL depth maps for shape transformation, trained on a new large-scale dataset of 18,573 images across 1,523 subjects.

Result: Achieves per-vertex reconstruction error of 7.5mm (significantly lower than baseline 13.6mm), produces realistic results that accurately match target shapes while preserving identity, clothing, and background details.

Conclusion: The proposed Odo method with its novel dataset and diffusion-based architecture enables intuitive and realistic human body reshaping, outperforming prior approaches and addressing key limitations in the field.

Abstract: Human shape editing enables controllable transformation of a person’s body shape, such as thin, muscular, or overweight, while preserving pose, identity, clothing, and background. Unlike human pose editing, which has advanced rapidly, shape editing remains relatively underexplored. Current approaches typically rely on 3D morphable models or image warping, often introducing unrealistic body proportions, texture distortions, and background inconsistencies due to alignment errors and deformations. A key limitation is the lack of large-scale, publicly available datasets for training and evaluating body shape manipulation methods. In this work, we introduce the first large-scale dataset of 18,573 images across 1523 subjects, specifically designed for controlled human shape editing. It features diverse variations in body shape, including fat, muscular and thin, captured under consistent identity, clothing, and background conditions. Using this dataset, we propose Odo, an end-to-end diffusion-based method that enables realistic and intuitive body reshaping guided by simple semantic attributes. Our approach combines a frozen UNet that preserves fine-grained appearance and background details from the input image with a ControlNet that guides shape transformation using target SMPL depth maps. Extensive experiments demonstrate that our method outperforms prior approaches, achieving per-vertex reconstruction errors as low as 7.5mm, significantly lower than the 13.6mm observed in baseline methods, while producing realistic results that accurately match the desired target shapes.

Method: Develops multiple data-generation pipelines including simulation-only, GAN-based, and hybrid diffusion-GAN approaches. Extends existing DA GAN with auxiliary inputs, uses novel training with both simulated and real images, and introduces adaptive blending to reduce hallucinations in diffusion outputs.

Result: Achieves 1.85% overall improvement in semantic segmentation and 4.62% improvement specifically on nighttime conditions when evaluated on the ACDC dataset.

Conclusion: The hybrid diffusion-GAN method effectively generates photorealistic adverse weather images and significantly enhances autonomous driving perception robustness under challenging environmental conditions.

Abstract: Autonomous Driving (AD) systems exhibit markedly degraded performance under adverse environmental conditions, such as low illumination and precipitation. The underrepresentation of adverse conditions in AD datasets makes it challenging to address this deficiency. To circumvent the prohibitive cost of acquiring and annotating adverse weather data, we propose a novel Domain Adaptation (DA) pipeline that transforms clear-weather images into fog, rain, snow, and nighttime images. Here, we systematically develop and evaluate several novel data-generation pipelines, including simulation-only, GAN-based, and hybrid diffusion-GAN approaches, to synthesize photorealistic adverse images from labelled clear images. We leverage an existing DA GAN, extend it to support auxiliary inputs, and develop a novel training recipe that leverages both simulated and real images. The simulated images facilitate exact supervision by providing perfectly matched image pairs, while the real images help bridge the simulation-to-real (sim2real) gap. We further introduce a method to mitigate hallucinations and artifacts in Stable-Diffusion Image-to-Image (img2img) outputs by blending them adaptively with their progenitor images. We finetune downstream models on our synthetic data and evaluate them on the Adverse Conditions Dataset with Correspondences (ACDC). We achieve 1.85 percent overall improvement in semantic segmentation, and 4.62 percent on nighttime, demonstrating the efficacy of our hybrid method for robust AD perception under challenging conditions.

Method: Unsupervised clustering framework that integrates visual embeddings from street-level imagery with spatial planting patterns to estimate biodiversity without requiring labeled data.

Result: Applied to eight North American cities, the method recovered genus-level diversity patterns with high fidelity, achieving low Wasserstein distances to ground truth for Shannon and Simpson indices while preserving spatial autocorrelation.

Conclusion: This scalable, fine-grained approach enables biodiversity mapping in cities lacking detailed inventories and supports continuous, low-cost monitoring for equitable greenery access and adaptive urban ecosystem management.

Abstract: Urban tree biodiversity is critical for climate resilience, ecological stability, and livability in cities, yet most municipalities lack detailed knowledge of their canopies. Field-based inventories provide reliable estimates of Shannon and Simpson diversity but are costly and time-consuming, while supervised AI methods require labeled data that often fail to generalize across regions. We introduce an unsupervised clustering framework that integrates visual embeddings from street-level imagery with spatial planting patterns to estimate biodiversity without labels. Applied to eight North American cities, the method recovers genus-level diversity patterns with high fidelity, achieving low Wasserstein distances to ground truth for Shannon and Simpson indices and preserving spatial autocorrelation. This scalable, fine-grained approach enables biodiversity mapping in cities lacking detailed inventories and offers a pathway for continuous, low-cost monitoring to support equitable access to greenery and adaptive management of urban ecosystems.

Method: Proposes two key components: 1) occlusion-aware re-identification mechanism for identity preservation, and 2) road-structure-aware tracklet refinement using semantic scene priors (lane directions, crosswalks, road boundaries). Also introduces a new vehicle tracking benchmark dataset.

Result: Achieves robust performance on both the new vehicle dataset and public benchmarks. Scores 66.4 HOTA on MOT17 and 65.7 HOTA on MOT20 test sets, demonstrating strong generalization capabilities.

Conclusion: The framework effectively handles multiple object types beyond pedestrians, particularly excelling in vehicle tracking scenarios while maintaining competitive performance on conventional pedestrian tracking benchmarks.

Abstract: Conventional multi-object tracking (MOT) systems are predominantly designed for pedestrian tracking and often exhibit limited generalization to other object categories. This paper presents a generalized tracking framework capable of handling multiple object types, with a particular emphasis on vehicle tracking in complex traffic scenes. The proposed method incorporates two key components: (1) an occlusion-aware re-identification mechanism that enhances identity preservation for heavily occluded objects, and (2) a road-structure-aware tracklet refinement strategy that utilizes semantic scene priors such as lane directions, crosswalks, and road boundaries to improve trajectory continuity and accuracy. In addition, we introduce a new benchmark dataset comprising diverse vehicle classes with frame-level tracking annotations, specifically curated to support evaluation of vehicle-focused tracking methods. Extensive experimental results demonstrate that the proposed approach achieves robust performance on both the newly introduced dataset and several public benchmarks, highlighting its effectiveness in general-purpose object tracking. While our framework is designed for generalized multi-class tracking, it also achieves strong performance on conventional benchmarks, with HOTA scores of 66.4 on MOT17 and 65.7 on MOT20 test sets. Code and Benchmark are available: github.com/Hamidreza-Hashempoor/FastTracker, huggingface.co/datasets/Hamidreza-Hashemp/FastTracker-Benchmark.

Method: MoCHA-former uses Decoupled Moiré Adaptive Demoiréing (DMAD) with Moiré Decoupling Block and Detail Decoupling Block to separate moiré/content, plus Moiré Conditioning Block for targeted restoration. Spatio-Temporal Adaptive Demoiréing (STAD) includes Spatial Fusion Block with window attention for large structures and Feature Channel Attention for RAW frame channel dependence. Implicit frame alignment ensures temporal consistency.

Result: The method consistently surpasses prior methods across PSNR, SSIM, and LPIPS metrics on two video datasets covering both RAW and sRGB domains, demonstrating superior performance in moiré pattern removal.

Conclusion: MoCHA-former effectively addresses key limitations in moiré pattern removal by combining decoupled moiré-content separation with spatio-temporal adaptive processing, achieving state-of-the-art performance in both quantitative metrics and qualitative results for screen capture demoiréing.

Abstract: Recent advances in portable imaging have made camera-based screen capture ubiquitous. Unfortunately, frequency aliasing between the camera’s color filter array (CFA) and the display’s sub-pixels induces moir'e patterns that severely degrade captured photos and videos. Although various demoir'eing models have been proposed to remove such moir'e patterns, these approaches still suffer from several limitations: (i) spatially varying artifact strength within a frame, (ii) large-scale and globally spreading structures, (iii) channel-dependent statistics and (iv) rapid temporal fluctuations across frames. We address these issues with the Moir'e Conditioned Hybrid Adaptive Transformer (MoCHA-former), which comprises two key components: Decoupled Moir'e Adaptive Demoir'eing (DMAD) and Spatio-Temporal Adaptive Demoir'eing (STAD). DMAD separates moir'e and content via a Moir'e Decoupling Block (MDB) and a Detail Decoupling Block (DDB), then produces moir'e-adaptive features using a Moir'e Conditioning Block (MCB) for targeted restoration. STAD introduces a Spatial Fusion Block (SFB) with window attention to capture large-scale structures, and a Feature Channel Attention (FCA) to model channel dependence in RAW frames. To ensure temporal consistency, MoCHA-former performs implicit frame alignment without any explicit alignment module. We analyze moir'e characteristics through qualitative and quantitative studies, and evaluate on two video datasets covering RAW and sRGB domains. MoCHA-former consistently surpasses prior methods across PSNR, SSIM, and LPIPS.

Method: Proposes concept distillation training strategy using pretrained T2V model to synthesize training samples, redesigned control architecture with control feature projector to filter degradation artifacts, and dual-branch ControlNet connector with MLP-based feature mapping and cross-attention for dynamic control.

Result: Extensive experiments show Vivid-VR outperforms existing approaches on synthetic and real-world benchmarks, achieving impressive texture realism, visual vividness, and temporal consistency.

Conclusion: Vivid-VR successfully addresses distribution drift issues in video restoration through concept distillation and enhanced control architecture, delivering superior video quality with publicly available code and checkpoints.

Abstract: We present Vivid-VR, a DiT-based generative video restoration method built upon an advanced T2V foundation model, where ControlNet is leveraged to control the generation process, ensuring content consistency. However, conventional fine-tuning of such controllable pipelines frequently suffers from distribution drift due to limitations in imperfect multimodal alignment, resulting in compromised texture realism and temporal coherence. To tackle this challenge, we propose a concept distillation training strategy that utilizes the pretrained T2V model to synthesize training samples with embedded textual concepts, thereby distilling its conceptual understanding to preserve texture and temporal quality. To enhance generation controllability, we redesign the control architecture with two key components: 1) a control feature projector that filters degradation artifacts from input video latents to minimize their propagation through the generation pipeline, and 2) a new ControlNet connector employing a dual-branch design. This connector synergistically combines MLP-based feature mapping with cross-attention mechanism for dynamic control feature retrieval, enabling both content preservation and adaptive control signal modulation. Extensive experiments show that Vivid-VR performs favorably against existing approaches on both synthetic and real-world benchmarks, as well as AIGC videos, achieving impressive texture realism, visual vividness, and temporal consistency. The codes and checkpoints are publicly available at https://github.com/csbhr/Vivid-VR.

Method: ScenGE uses a two-step approach: 1) Meta-Scenario Generation where an LLM grounded in driving knowledge infers plausible adversarial agents, and 2) Complex Scenario Evolution that uses background vehicles to amplify threats through trajectory optimization and occlusion creation.

Result: Extensive experiments show ScenGE uncovers 31.96% more severe collision cases than state-of-the-art baselines, works with different simulators and AV systems, and improves model robustness through adversarial training.

Conclusion: The framework generates plausible and critical scenarios validated through real-world tests and human evaluation, representing a significant step toward building public trust and ensuring safe deployment of autonomous vehicles.

Abstract: The generation of safety-critical scenarios in simulation has become increasingly crucial for safety evaluation in autonomous vehicles prior to road deployment in society. However, current approaches largely rely on predefined threat patterns or rule-based strategies, which limit their ability to expose diverse and unforeseen failure modes. To overcome these, we propose ScenGE, a framework that can generate plentiful safety-critical scenarios by reasoning novel adversarial cases and then amplifying them with complex traffic flows. Given a simple prompt of a benign scene, it first performs Meta-Scenario Generation, where a large language model, grounded in structured driving knowledge, infers an adversarial agent whose behavior poses a threat that is both plausible and deliberately challenging. This meta-scenario is then specified in executable code for precise in-simulator control. Subsequently, Complex Scenario Evolution uses background vehicles to amplify the core threat introduced by Meta-Scenario. It builds an adversarial collaborator graph to identify key agent trajectories for optimization. These perturbations are designed to simultaneously reduce the ego vehicle’s maneuvering space and create critical occlusions. Extensive experiments conducted on multiple reinforcement learning based AV models show that ScenGE uncovers more severe collision cases (+31.96%) on average than SoTA baselines. Additionally, our ScenGE can be applied to large model based AV systems and deployed on different simulators; we further observe that adversarial training on our scenarios improves the model robustness. Finally, we validate our framework through real-world vehicle tests and human evaluation, confirming that the generated scenarios are both plausible and critical. We hope our paper can build up a critical step towards building public trust and ensuring their safe deployment.

Method: Proposes CurveFlow framework with curvature regularization that penalizes abrupt changes in trajectory dynamics. Learns smooth, non-linear trajectories by directly incorporating curvature guidance into the flow path.

Result: State-of-the-art performance on MS COCO 2014/2017, significantly outperforming standard rectified flow variants and non-linear baselines like Rectified Diffusion. Shows substantial improvements in semantic consistency metrics (BLEU, METEOR, ROUGE, CLAIR) while maintaining high image quality.

Conclusion: Curvature-aware modeling enhances the model’s ability to faithfully follow complex instructions, confirming that trajectory curvature correlates with semantic alignment in text-to-image generation.

Abstract: Existing rectified flow models are based on linear trajectories between data and noise distributions. This linearity enforces zero curvature, which can inadvertently force the image generation process through low-probability regions of the data manifold. A key question remains underexplored: how does the curvature of these trajectories correlate with the semantic alignment between generated images and their corresponding captions, i.e., instructional compliance? To address this, we introduce CurveFlow, a novel flow matching framework designed to learn smooth, non-linear trajectories by directly incorporating curvature guidance into the flow path. Our method features a robust curvature regularization technique that penalizes abrupt changes in the trajectory’s intrinsic dynamics.Extensive experiments on MS COCO 2014 and 2017 demonstrate that CurveFlow achieves state-of-the-art performance in text-to-image generation, significantly outperforming both standard rectified flow variants and other non-linear baselines like Rectified Diffusion. The improvements are especially evident in semantic consistency metrics such as BLEU, METEOR, ROUGE, and CLAIR. This confirms that our curvature-aware modeling substantially enhances the model’s ability to faithfully follow complex instructions while simultaneously maintaining high image quality. The code is made publicly available at https://github.com/Harvard-AI-and-Robotics-Lab/CurveFlow.

Method: Three-stage pipeline: 1) Generate geometrically consistent sparse views with Cascaded Outpainting ControlNets, 2) Propagate denser views via AGInpaint, 3) Eliminate visual inconsistencies with GCAlign module. Concurrent 3DGS scene reconstruction on mesh surfaces.

Result: Outperforms existing approaches in both geometric alignment and generation quality. Enables diverse style rendering through relighting and style transfer.

Conclusion: MeSS successfully addresses texture generation for city mesh models, providing high-quality, consistent outdoor scenes suitable for virtual navigation and autonomous driving applications.

Abstract: Mesh models have become increasingly accessible for numerous cities; however, the lack of realistic textures restricts their application in virtual urban navigation and autonomous driving. To address this, this paper proposes MeSS (Meshbased Scene Synthesis) for generating high-quality, styleconsistent outdoor scenes with city mesh models serving as the geometric prior. While image and video diffusion models can leverage spatial layouts (such as depth maps or HD maps) as control conditions to generate street-level perspective views, they are not directly applicable to 3D scene generation. Video diffusion models excel at synthesizing consistent view sequences that depict scenes but often struggle to adhere to predefined camera paths or align accurately with rendered control videos. In contrast, image diffusion models, though unable to guarantee cross-view visual consistency, can produce more geometry-aligned results when combined with ControlNet. Building on this insight, our approach enhances image diffusion models by improving cross-view consistency. The pipeline comprises three key stages: first, we generate geometrically consistent sparse views using Cascaded Outpainting ControlNets; second, we propagate denser intermediate views via a component dubbed AGInpaint; and third, we globally eliminate visual inconsistencies (e.g., varying exposure) using the GCAlign module. Concurrently with generation, a 3D Gaussian Splatting (3DGS) scene is reconstructed by initializing Gaussian balls on the mesh surface. Our method outperforms existing approaches in both geometric alignment and generation quality. Once synthesized, the scene can be rendered in diverse styles through relighting and style transfer techniques.

Method: Systematically analyzes sampling trajectories, strategically distributes cache reuse across entire sampling process (not just later steps), and dynamically estimates/filters noise during cache reuse to maintain sampling direction.

Result: Extensive experiments show the approach accelerates sampling while maintaining competitive generative quality.

Conclusion: OmniCache offers a practical training-free solution for efficient deployment of diffusion-based generative models by effectively utilizing cached computations throughout the diffusion trajectory.

Abstract: Diffusion models have emerged as a powerful paradigm for generative tasks such as image synthesis and video generation, with Transformer architectures further enhancing performance. However, the high computational cost of diffusion Transformers-stemming from a large number of sampling steps and complex per-step computations-presents significant challenges for real-time deployment. In this paper, we introduce OmniCache, a training-free acceleration method that exploits the global redundancy inherent in the denoising process. Unlike existing methods that determine caching strategies based on inter-step similarities and tend to prioritize reusing later sampling steps, our approach originates from the sampling perspective of DIT models. We systematically analyze the model’s sampling trajectories and strategically distribute cache reuse across the entire sampling process. This global perspective enables more effective utilization of cached computations throughout the diffusion trajectory, rather than concentrating reuse within limited segments of the sampling procedure. In addition, during cache reuse, we dynamically estimate the corresponding noise and filter it out to reduce its impact on the sampling direction. Extensive experiments demonstrate that our approach accelerates the sampling process while maintaining competitive generative quality, offering a promising and practical solution for efficient deployment of diffusion-based generative models.

Method: Temporal Prompt Alignment (TPA) extracts frame features using image encoder, aggregates with temporal extractor, aligns with class-specific text prompts via contrastive loss, and uses CVAESM module for uncertainty quantification and style modulation.

Result: TPA achieves 85.40% macro F1 for CHD diagnosis, reduces calibration error by 5.38-6.8%, and boosts macro F1 by 4.73% on EchoNet-Dynamic’s three-class task.

Conclusion: TPA effectively addresses limitations of current methods by integrating temporal modeling, prompt learning, and uncertainty quantification, demonstrating superior performance and clinical reliability for CHD detection in ultrasound videos.

Abstract: Congenital heart defect (CHD) detection in ultrasound videos is hindered by image noise and probe positioning variability. While automated methods can reduce operator dependence, current machine learning approaches often neglect temporal information, limit themselves to binary classification, and do not account for prediction calibration. We propose Temporal Prompt Alignment (TPA), a method leveraging foundation image-text model and prompt-aware contrastive learning to classify fetal CHD on cardiac ultrasound videos. TPA extracts features from each frame of video subclips using an image encoder, aggregates them with a trainable temporal extractor to capture heart motion, and aligns the video representation with class-specific text prompts via a margin-hinge contrastive loss. To enhance calibration for clinical reliability, we introduce a Conditional Variational Autoencoder Style Modulation (CVAESM) module, which learns a latent style vector to modulate embeddings and quantifies classification uncertainty. Evaluated on a private dataset for CHD detection and on a large public dataset, EchoNet-Dynamic, for systolic dysfunction, TPA achieves state-of-the-art macro F1 scores of 85.40% for CHD diagnosis, while also reducing expected calibration error by 5.38% and adaptive ECE by 6.8%. On EchoNet-Dynamic’s three-class task, it boosts macro F1 by 4.73% (from 53.89% to 58.62%). Temporal Prompt Alignment (TPA) is a framework for fetal congenital heart defect (CHD) classification in ultrasound videos that integrates temporal modeling, prompt-aware contrastive learning, and uncertainty quantification.

Method: Proposes a robust, keypoint detector-free approach to estimate hand-object 3D transformations from monocular video, integrated with multi-view reconstruction pipeline without requiring pre-scanned templates or camera intrinsics.

Result: Achieves state-of-the-art performance on SHOWMe benchmark for object-agnostic hand-object 3D transformation and shape estimation, and demonstrates generalization to unseen object categories on HO3D dataset.

Conclusion: HOSt3R provides a scalable and generalizable solution for hand-object 3D reconstruction that overcomes limitations of keypoint-based methods and works without object templates or camera calibration.

Abstract: Hand-object 3D reconstruction has become increasingly important for applications in human-robot interaction and immersive AR/VR experiences. A common approach for object-agnostic hand-object reconstruction from RGB sequences involves a two-stage pipeline: hand-object 3D tracking followed by multi-view 3D reconstruction. However, existing methods rely on keypoint detection techniques, such as Structure from Motion (SfM) and hand-keypoint optimization, which struggle with diverse object geometries, weak textures, and mutual hand-object occlusions, limiting scalability and generalization. As a key enabler to generic and seamless, non-intrusive applicability, we propose in this work a robust, keypoint detector-free approach to estimating hand-object 3D transformations from monocular motion video/images. We further integrate this with a multi-view reconstruction pipeline to accurately recover hand-object 3D shape. Our method, named HOSt3R, is unconstrained, does not rely on pre-scanned object templates or camera intrinsics, and reaches state-of-the-art performance for the tasks of object-agnostic hand-object 3D transformation and shape estimation on the SHOWMe benchmark. We also experiment on sequences from the HO3D dataset, demonstrating generalization to unseen object categories.

Method: The method uses region-wise voxel initialization (near, middle, far regions) to handle linear motion patterns, introduces deformable neural Gaussians for non-rigid dynamic actors with temporal parameter adjustment via a deformation network, and incorporates depth and normal priors from pre-trained models for geometric supervision.

Result: DriveSplat achieves state-of-the-art performance on Waymo and KITTI datasets for novel-view synthesis in driving scenarios, demonstrating superior reconstruction quality and geometric accuracy.

Conclusion: The proposed DriveSplat framework effectively addresses the challenges of 3D reconstruction in dynamic driving environments through improved dynamic-static decoupling, region-aware initialization, and geometric supervision, resulting in high-quality novel view synthesis.

Abstract: In the realm of driving scenarios, the presence of rapidly moving vehicles, pedestrians in motion, and large-scale static backgrounds poses significant challenges for 3D scene reconstruction. Recent methods based on 3D Gaussian Splatting address the motion blur problem by decoupling dynamic and static components within the scene. However, these decoupling strategies overlook background optimization with adequate geometry relationships and rely solely on fitting each training view by adding Gaussians. Therefore, these models exhibit limited robustness in rendering novel views and lack an accurate geometric representation. To address the above issues, we introduce DriveSplat, a high-quality reconstruction method for driving scenarios based on neural Gaussian representations with dynamic-static decoupling. To better accommodate the predominantly linear motion patterns of driving viewpoints, a region-wise voxel initialization scheme is employed, which partitions the scene into near, middle, and far regions to enhance close-range detail representation. Deformable neural Gaussians are introduced to model non-rigid dynamic actors, whose parameters are temporally adjusted by a learnable deformation network. The entire framework is further supervised by depth and normal priors from pre-trained models, improving the accuracy of geometric structures. Our method has been rigorously evaluated on the Waymo and KITTI datasets, demonstrating state-of-the-art performance in novel-view synthesis for driving scenarios.

Method: Proposes ExtraGS with Road Surface Gaussian representation (hybrid Gaussian-SDF design), Far Field Gaussians with learnable scaling, and self-supervised uncertainty estimation using spherical harmonics for selective generative prior integration.

Result: Extensive experiments show ExtraGS significantly enhances realism and geometric consistency of extrapolated views while maintaining high fidelity on original trajectories across multiple datasets and camera setups.

Conclusion: The holistic integration of geometric and generative priors through specialized Gaussian representations and uncertainty-based selective integration effectively addresses extrapolation challenges in driving scene synthesis.

Abstract: Synthesizing extrapolated views from recorded driving logs is critical for simulating driving scenes for autonomous driving vehicles, yet it remains a challenging task. Recent methods leverage generative priors as pseudo ground truth, but often lead to poor geometric consistency and over-smoothed renderings. To address these limitations, we propose ExtraGS, a holistic framework for trajectory extrapolation that integrates both geometric and generative priors. At the core of ExtraGS is a novel Road Surface Gaussian(RSG) representation based on a hybrid Gaussian-Signed Distance Function (SDF) design, and Far Field Gaussians (FFG) that use learnable scaling factors to efficiently handle distant objects. Furthermore, we develop a self-supervised uncertainty estimation framework based on spherical harmonics that enables selective integration of generative priors only where extrapolation artifacts occur. Extensive experiments on multiple datasets, diverse multi-camera setups, and various generative priors demonstrate that ExtraGS significantly enhances the realism and geometric consistency of extrapolated views, while preserving high fidelity along the original trajectory.

Method: The study adapts image foundation models (DINOv2 and CLIP) using single training views and evaluates on unseen perspectives. It benchmarks linear probes, advanced probing strategies, and introduces T-MASK - a temporal token masking method that emphasizes dynamic video regions.

Result: T-MASK improves cross-view top-1 accuracy by +1.23% over probing baselines and +8.0% over PEFT methods. It particularly boosts recognition of underrepresented secondary activities by +5.42% (trained view) and +1.36% (cross-view).

Conclusion: Lightweight probing methods like T-MASK show strong potential for fine-grained driver observation, especially in cross-view and low-data settings, highlighting the importance of temporal token selection for robust driver monitoring systems.

Abstract: Changes of camera perspective are a common obstacle in driver monitoring. While deep learning and pretrained foundation models show strong potential for improved generalization via lightweight adaptation of the final layers (‘probing’), their robustness to unseen viewpoints remains underexplored. We study this challenge by adapting image foundation models to driver monitoring using a single training view, and evaluating them directly on unseen perspectives without further adaptation. We benchmark simple linear probes, advanced probing strategies, and compare two foundation models (DINOv2 and CLIP) against parameter-efficient fine-tuning (PEFT) and full fine-tuning. Building on these insights, we introduce T-MASK – a new image-to-video probing method that leverages temporal token masking and emphasizes more dynamic video regions. Benchmarked on the public Drive&Act dataset, T-MASK improves cross-view top-1 accuracy by $+1.23%$ over strong probing baselines and $+8.0%$ over PEFT methods, without adding any parameters. It proves particularly effective for underrepresented secondary activities, boosting recognition by $+5.42%$ under the trained view and $+1.36%$ under cross-view settings. This work provides encouraging evidence that adapting foundation models with lightweight probing methods like T-MASK has strong potential in fine-grained driver observation, especially in cross-view and low-data settings. These results highlight the importance of temporal token selection when leveraging foundation models to build robust driver monitoring systems. Code and models will be made available at https://github.com/th-nesh/T-MASK to support ongoing research.

Method: Enhanced rule-based framework incorporating speaking-rate normalization, multi-level acoustic feature analysis, and hierarchical decision structures, analyzed across multiple corpora including UCLASS, FluencyBank, and SEP-28k.

Result: Achieves competitive performance with 97-99% accuracy in prolongation detection and stable performance across varying speaking rates, while maintaining complete interpretability.

Conclusion: Rule-based methods offer unique advantages in clinical contexts for decision auditability, patient-specific tuning, and real-time feedback, and can be integrated with modern ML pipelines as proposal generators or constraint modules.

Abstract: Stuttering affects approximately 1% of the global population, impacting communication and quality of life. While recent advances in deep learning have pushed the boundaries of automatic speech dysfluency detection, rule-based approaches remain crucial for clinical applications where interpretability and transparency are paramount. This paper presents a comprehensive analysis of rule-based stuttering detection systems, synthesizing insights from multiple corpora including UCLASS, FluencyBank, and SEP-28k. We propose an enhanced rule-based framework that incorporates speaking-rate normalization, multi-level acoustic feature analysis, and hierarchical decision structures. Our approach achieves competitive performance while maintaining complete interpretability-critical for clinical adoption. We demonstrate that rule-based systems excel particularly in prolongation detection (97-99% accuracy) and provide stable performance across varying speaking rates. Furthermore, we show how these interpretable models can be integrated with modern machine learning pipelines as proposal generators or constraint modules, bridging the gap between traditional speech pathology practices and contemporary AI systems. Our analysis reveals that while neural approaches may achieve marginally higher accuracy in unconstrained settings, rule-based methods offer unique advantages in clinical contexts where decision auditability, patient-specific tuning, and real-time feedback are essential.

Method: Applied XAI techniques to PISA 2018 data (67,329 students from 10 countries) using four models: Multiple Linear Regression, Random Forest, CATBoost, and Artificial Neural Networks. Used 70% training data with 5-fold cross-validation and 30% testing data stratified by country. Employed feature importance, SHAP values, and decision tree visualizations for interpretability.

Result: Non-linear models (especially Random Forest and ANN) outperformed Multiple Linear Regression. Random Forest balanced accuracy and generalizability best. Key predictors included socio-economic status, study time, teacher motivation, and students’ attitudes toward mathematics, with varying impact across countries. Visual diagnostics showed RF and CATBoost closely aligned with actual performance.

Conclusion: The study highlights the non-linear and context-dependent nature of math achievement, demonstrates the value of XAI in educational research, uncovers cross-national patterns, and provides insights for equity-focused reforms and personalized learning strategies.

Abstract: Understanding the factors that shape students’ mathematics performance is vital for designing effective educational policies. This study applies explainable artificial intelligence (XAI) techniques to PISA 2018 data to predict math achievement and identify key predictors across ten countries (67,329 students). We tested four models: Multiple Linear Regression (MLR), Random Forest (RF), CATBoost, and Artificial Neural Networks (ANN), using student, family, and school variables. Models were trained on 70% of the data (with 5-fold cross-validation) and tested on 30%, stratified by country. Performance was assessed with R^2 and Mean Absolute Error (MAE). To ensure interpretability, we used feature importance, SHAP values, and decision tree visualizations. Non-linear models, especially RF and ANN, outperformed MLR, with RF balancing accuracy and generalizability. Key predictors included socio-economic status, study time, teacher motivation, and students’ attitudes toward mathematics, though their impact varied across countries. Visual diagnostics such as scatterplots of predicted vs actual scores showed RF and CATBoost aligned closely with actual performance. Findings highlight the non-linear and context-dependent nature of achievement and the value of XAI in educational research. This study uncovers cross-national patterns, informs equity-focused reforms, and supports the development of personalized learning strategies.

Method: The authors conduct comprehensive evaluation through faithfulness (model reasoning reflection) and plausibility (human expert consistency) lenses. They construct a new rationale-annotated dataset and propose rationale learning methods using LLM-generated rationales as distant supervision, with/without few-shot human annotations.

Result: LLM-generated rationales align most closely with human expert judgments. Incorporating few-shot human-annotated examples improves both rationale generation and rationale-learning approaches.

Conclusion: The study demonstrates that LLM-generated rationales can effectively serve as supervision signals for improving explainability in clinical coding systems, with human annotations further enhancing performance, addressing the critical need for trustworthy automated clinical coding.

Abstract: Automated clinical coding involves mapping unstructured text from Electronic Health Records (EHRs) to standardized code systems such as the International Classification of Diseases (ICD). While recent advances in deep learning have significantly improved the accuracy and efficiency of ICD coding, the lack of explainability in these models remains a major limitation, undermining trust and transparency. Current explorations about explainability largely rely on attention-based techniques and qualitative assessments by physicians, yet lack systematic evaluation using consistent criteria on high-quality rationale datasets, as well as dedicated approaches explicitly trained to generate rationales for further enhancing explanation. In this work, we conduct a comprehensive evaluation of the explainability of the rationales for ICD coding through two key lenses: faithfulness that evaluates how well explanations reflect the model’s actual reasoning and plausibility that measures how consistent the explanations are with human expert judgment. To facilitate the evaluation of plausibility, we construct a new rationale-annotated dataset, offering denser annotations with diverse granularity and aligns better with current clinical practice, and conduct evaluation across three types of rationales of ICD coding. Encouraged by the promising plausibility of LLM-generated rationales for ICD coding, we further propose new rationale learning methods to improve the quality of model-generated rationales, where rationales produced by prompting LLMs with/without annotation examples are used as distant supervision signals. We empirically find that LLM-generated rationales align most closely with those of human experts. Moreover, incorporating few-shot human-annotated examples not only further improves rationale generation but also enhances rationale-learning approaches.

Method: A computational multi-agent society experiment framework integrating generative agent-based modeling with virtual ethnographic methods across three studies to examine group stance differentiation and social boundary formation.

Result: Agents exhibit endogenous stances independent of preset identities, show distinct tonal preferences and response patterns to different discourse strategies, and actively dismantle existing identity-based power structures to reconstruct self-organized community boundaries through language interaction.

Conclusion: Preset identities do not rigidly determine agents’ social structures; effective human intervention in collective cognition requires attention to endogenous mechanisms and interactional dynamics within agents’ language networks, providing theoretical foundation for using generative AI in modeling group social dynamics.

Abstract: Large language models have been widely used to simulate credible human social behaviors. However, it remains unclear whether these models can demonstrate stable capacities for stance formation and identity negotiation in complex interactions, as well as how they respond to human interventions. We propose a computational multi-agent society experiment framework that integrates generative agent-based modeling with virtual ethnographic methods to investigate how group stance differentiation and social boundary formation emerge in human-agent hybrid societies. Across three studies, we find that agents exhibit endogenous stances, independent of their preset identities, and display distinct tonal preferences and response patterns to different discourse strategies. Furthermore, through language interaction, agents actively dismantle existing identity-based power structures and reconstruct self-organized community boundaries based on these stances. Our findings suggest that preset identities do not rigidly determine the agents’ social structures. For human researchers to effectively intervene in collective cognition, attention must be paid to the endogenous mechanisms and interactional dynamics within the agents’ language networks. These insights provide a theoretical foundation for using generative AI in modeling group social dynamics and studying human-agent collaboration.

Method: Developed a GPU-accelerated engine with a domain-specific language based on PuzzleScript, validated by implementing hundreds of existing PuzzleScript games created by both professionals and casual designers since 2013.

Result: Successfully demonstrated coverage of an expansive and expressive task space, showing that PuzzleJAX can express tasks that are simple to understand but challenging to master, requiring complex reasoning skills.

Conclusion: PuzzleJAX provides a powerful platform for benchmarking AI reasoning capabilities using human-designed puzzle games that combine control, planning, and high-level insight challenges.

Abstract: We introduce PuzzleJAX, a GPU-accelerated puzzle game engine and description language designed to support rapid benchmarking of tree search, reinforcement learning, and LLM reasoning abilities. Unlike existing GPU-accelerated learning environments that provide hard-coded implementations of fixed sets of games, PuzzleJAX allows dynamic compilation of any game expressible in its domain-specific language (DSL). This DSL follows PuzzleScript, which is a popular and accessible online game engine for designing puzzle games. In this paper, we validate in PuzzleJAX several hundred of the thousands of games designed in PuzzleScript by both professional designers and casual creators since its release in 2013, thereby demonstrating PuzzleJAX’s coverage of an expansive, expressive, and human-relevant space of tasks. By analyzing the performance of search, learning, and language models on these games, we show that PuzzleJAX can naturally express tasks that are both simple and intuitive to understand, yet often deeply challenging to master, requiring a combination of control, planning, and high-level insight.

Method: Two complementary solutions: 1) VLM as model-card matcher with three-stage workflow (modality -> abnormality -> model ID) with early exit checks and answer selection; 2) Fine-tuning VLM on specialty-specific datasets to handle multiple tasks within each specialty.

Result: The single-model deployment matches or approaches specialized baselines across gastroenterology, hematology, ophthalmology, and pathology specialties.

Conclusion: One VLM can both decide and do, reducing data scientist effort, shortening monitoring, increasing transparency of model selection, and lowering integration overhead compared to multi-agent pipelines.

Abstract: Clinical workflows are fragmented as a patchwork of scripts and task-specific networks that often handle triage, task selection, and model deployment. These pipelines are rarely streamlined for data science pipeline, reducing efficiency and raising operational costs. Workflows also lack data-driven model identification (from imaging/tabular inputs) and standardized delivery of model outputs. In response, we present a practical, healthcare-first framework that uses a single vision-language model (VLM) in two complementary roles. First (Solution 1), the VLM acts as an aware model-card matcher that routes an incoming image to the appropriate specialist model via a three-stage workflow (modality -> primary abnormality -> model-card id). Checks are provided by (i) stagewise prompts that allow early exit via None/Normal/Other and (ii) a stagewise answer selector that arbitrates between the top-2 candidates at each stage, reducing the chance of an incorrect selection and aligning the workflow with clinical risk tolerance. Second (Solution 2), we fine-tune the VLM on specialty-specific datasets ensuring a single model covers multiple downstream tasks within each specialty, maintaining performance while simplifying deployment. Across gastroenterology, hematology, ophthalmology, and pathology, our single-model deployment matches or approaches specialized baselines. Compared with pipelines composed of many task-specific agents, this approach shows that one VLM can both decide and do. It may reduce effort by data scientists, shorten monitoring, increase the transparency of model selection (with per-stage justifications), and lower integration overhead.

Method: The authors use a Bayesian framework to quantify sycophancy as deviations from rational behavior when LLMs are presented with user perspectives. They study 3 different tasks, various LLMs (open-source and closed), two probing methods, and multiple probability judgment elicitation techniques.

Result: Findings show: 1) LLMs are not Bayesian rational, 2) sycophancy probing causes significant increases in predicted posteriors favoring steered outcomes, 3) sycophancy sometimes increases Bayesian error but occasionally decreases it, and 4) Bayesian error changes are not strongly correlated with Brier score.

Conclusion: Studying sycophancy’s impact on ground truth alone doesn’t fully capture reasoning errors. The Bayesian framework provides a more comprehensive way to quantify irrational sycophantic behavior, especially in uncertain scenarios without clear ground truth.

Abstract: Sycophancy, or overly agreeable or flattering behavior, is a documented issue in large language models (LLMs), and is critical to understand in the context of human/AI collaboration. Prior works typically quantify sycophancy by measuring shifts in behavior or impacts on accuracy, but neither metric characterizes shifts in rationality, and accuracy measures can only be used in scenarios with a known ground truth. In this work, we utilize a Bayesian framework to quantify sycophancy as deviations from rational behavior when presented with user perspectives, thus distinguishing between rational and irrational updates based on the introduction of user perspectives. In comparison to other methods, this approach allows us to characterize excessive behavioral shifts, even for tasks that involve inherent uncertainty or do not have a ground truth. We study sycophancy for 3 different tasks, a combination of open-source and closed LLMs, and two different methods for probing sycophancy. We also experiment with multiple methods for eliciting probability judgments from LLMs. We hypothesize that probing LLMs for sycophancy will cause deviations in LLMs’ predicted posteriors that will lead to increased Bayesian error. Our findings indicate that: 1) LLMs are not Bayesian rational, 2) probing for sycophancy results in significant increases to the predicted posterior in favor of the steered outcome, 3) sycophancy sometimes results in increased Bayesian error, and in a small number of cases actually decreases error, and 4) changes in Bayesian error due to sycophancy are not strongly correlated in Brier score, suggesting that studying the impact of sycophancy on ground truth alone does not fully capture errors in reasoning due to sycophancy.

Method: Developed RADAR, a semi-automatic approach to create a benchmark dataset with 17,819 samples containing charts, questions, reasoning steps, and attribution annotations. Also introduced a method for providing attribution in chart-based mathematical reasoning.

Result: Reasoning-guided approach improved attribution accuracy by 15% compared to baselines. Enhanced attribution capabilities led to stronger answer generation with average BERTScore of ~0.90, indicating high alignment with ground truth responses.

Conclusion: This represents a significant step toward more interpretable and trustworthy chart analysis systems, enabling users to verify and understand model decisions through reasoning and attribution.

Abstract: Data visualizations like charts are fundamental tools for quantitative analysis and decision-making across fields, requiring accurate interpretation and mathematical reasoning. The emergence of Multimodal Large Language Models (MLLMs) offers promising capabilities for automated visual data analysis, such as processing charts, answering questions, and generating summaries. However, they provide no visibility into which parts of the visual data informed their conclusions; this black-box nature poses significant challenges to real-world trust and adoption. In this paper, we take the first major step towards evaluating and enhancing the capabilities of MLLMs to attribute their reasoning process by highlighting the specific regions in charts and graphs that justify model answers. To this end, we contribute RADAR, a semi-automatic approach to obtain a benchmark dataset comprising 17,819 diverse samples with charts, questions, reasoning steps, and attribution annotations. We also introduce a method that provides attribution for chart-based mathematical reasoning. Experimental results demonstrate that our reasoning-guided approach improves attribution accuracy by 15% compared to baseline methods, and enhanced attribution capabilities translate to stronger answer generation, achieving an average BERTScore of $\sim$ 0.90, indicating high alignment with ground truth responses. This advancement represents a significant step toward more interpretable and trustworthy chart analysis systems, enabling users to verify and understand model decisions through reasoning and attribution.

Method: Investigates complexity of computational problems in finitary infinite AFs (where each argument has only finitely many attackers) through theoretical analysis and complexity classification.

Result: Finitary assumption doesn’t automatically reduce complexity, but admissibility-based semantics show dramatic complexity decrease due to combinatorial constraints, making many reasoning forms tractable.

Conclusion: Finitary infinite AFs provide a natural balance between expressiveness for reasoning applications and computational tractability for useful framework analysis.

Abstract: Abstract argumentation frameworks (AFs) provide a formal setting to analyze many forms of reasoning with conflicting information. While the expressiveness of general infinite AFs make them a tempting tool for modeling many kinds of reasoning scenarios, the computational intractability of solving infinite AFs limit their use, even in many theoretical applications. We investigate the complexity of computational problems related to infinite but finitary argumentations frameworks, that is, infinite AFs where each argument is attacked by only finitely many others. Our results reveal a surprising scenario. On one hand, we see that the assumption of being finitary does not automatically guarantee a drop in complexity. However, for the admissibility-based semantics, we find a remarkable combinatorial constraint which entails a dramatic decrease in complexity. We conclude that for many forms of reasoning, the finitary infinite AFs provide a natural setting for reasoning which balances well the competing goals of being expressive enough to be applied to many reasoning settings while being computationally tractable enough for the analysis within the framework to be useful.

Method: Developed WebSight-7B, a fine-tuned vision-language model optimized for UI element interaction using LoRA on web-focused data. Integrated into a modular multi-agent architecture with planning, reasoning, vision-action, and verification agents coordinated through episodic memory.

Result: WebSight-7B achieves 58.84% top-1 accuracy on Showdown Clicks benchmark, outperforming larger generalist models. Full WebSight agent achieves 68.0% success rate on WebVoyager benchmark, surpassing OpenAI (61.0%) and HCompany (67.0%) systems, with 97.14% accuracy on completed tasks.

Conclusion: WebSight and WebSight-7B establish a new standard for interpretable, robust, and efficient visual web navigation, demonstrating superior performance through pure visual perception approach.

Abstract: We introduce WebSight, a vision-based autonomous web agent, designed to interact with web environments purely through visual perception, eliminating dependence on HTML or DOM-based inputs. Central to our approach we introduce our new model, WebSight-7B, a fine-tuned vision-language model optimized for UI element interaction, trained using LoRA on a web-focused subset of the Wave-UI-25K dataset. WebSight integrates this model into a modular multi-agent architecture, comprising planning, reasoning, vision-action, and verification agents, coordinated through an episodic memory mechanism. WebSight-7B achieves a top-1 accuracy of 58.84% on the Showdown Clicks benchmark, outperforming several larger generalist models while maintaining lower latency. The full WebSight agent achieves a 68.0% success rate on the WebVoyager benchmark, surpassing systems from labs such as OpenAI (61.0%) and HCompany (Runner H, 67.0%). Among tasks completed, WebSight answers correctly 97.14% of the time, indicating high precision. Together, WebSight and WebSight-7B establish a new standard for interpretable, robust, and efficient visual web navigation.

Method: Propose Generate-and-Split (GaS) framework that integrates reinforcement learning with an optimal splitting algorithm in joint training. Uses LSTM-enhanced model to handle partial observability and ensures near-linear scalability with optimal splitting guarantees in Euclidean space.

Result: Extensive experiments show GaS significantly outperforms existing learning-based approaches in both solution quality and transferability.

Conclusion: The joint training approach of GaS framework successfully addresses the optimization consistency issues in two-stage methods, providing superior performance for the Min-Max Multiple Traveling Salesmen Problem.

Abstract: This study addresses the Min-Max Multiple Traveling Salesmen Problem ($m^3$-TSP), which aims to coordinate tours for multiple salesmen such that the length of the longest tour is minimized. Due to its NP-hard nature, exact solvers become impractical under the assumption that $P \ne NP$. As a result, learning-based approaches have gained traction for their ability to rapidly generate high-quality approximate solutions. Among these, two-stage methods combine learning-based components with classical solvers, simplifying the learning objective. However, this decoupling often disrupts consistent optimization, potentially degrading solution quality. To address this issue, we propose a novel two-stage framework named \textbf{Generate-and-Split} (GaS), which integrates reinforcement learning (RL) with an optimal splitting algorithm in a joint training process. The splitting algorithm offers near-linear scalability with respect to the number of cities and guarantees optimal splitting in Euclidean space for any given path. To facilitate the joint optimization of the RL component with the algorithm, we adopt an LSTM-enhanced model architecture to address partial observability. Extensive experiments show that the proposed GaS framework significantly outperforms existing learning-based approaches in both solution quality and transferability.

Method: Developed PowerChain - an agentic AI system that uses LLM function-calling to dynamically generate and execute ordered sequences of power system functions from natural language queries, guided by expert-built function pools and reference workflow-query pairs.

Result: PowerChain can produce expert-level workflows using both GPT-5 and open-source Qwen models on complex, unseen distribution grid analysis tasks with real utility data.

Conclusion: The system successfully automates distribution grid analysis, making advanced computational tools accessible to utilities without large R&D teams by leveraging AI agent orchestration and natural language interfaces.

Abstract: Due to the rapid pace of electrification and decarbonization, distribution grid (DG) operation and planning are becoming more complex, necessitating advanced computational analyses to ensure grid reliability and resilience. State-of-the-art DG analyses rely on disparate workflows of complex models, functions, and data pipelines, which require expert knowledge and are challenging to automate. Many small-scale utilities and cooperatives lack a large R&D workforce and therefore cannot use advanced analysis at scale. To address this gap, we develop a novel agentic AI system, PowerChain, to solve unseen DG analysis tasks via automated agentic orchestration and large language models (LLMs) function-calling. Given a natural language query, PowerChain dynamically generates and executes an ordered sequence of domain-aware functions guided by the semantics of an expert-built power systems function pool and a select reference set of known, expert-generated workflow-query pairs. Our results show that PowerChain can produce expert-level workflows with both GPT-5 and open-source Qwen models on complex, unseen DG analysis tasks operating on real utility data.

Method: Proposes rethinking value alignment through long-term reasoning, adaptability to evolving values, and multi-agent systems to handle value pluralism and conflicts.

Result: Identifies key challenges in value alignment research and provides directions for advancing the field, including design methodologies and practical applications.

Conclusion: Value alignment requires dynamic frameworks that can accommodate evolving human values and address conflicts through multi-agent reasoning systems.

Abstract: The concepts of human-centered AI'' and value-based decision’’ have gained significant attention in both research and industry. However, many critical aspects remain underexplored and require further investigation. In particular, there is a need to understand how systems incorporate human values, how humans can identify these values within systems, and how to minimize the risks of harm or unintended consequences. In this paper, we highlight the need to rethink how we frame value alignment and assert that value alignment should move beyond static and singular conceptions of values. We argue that AI systems should implement long-term reasoning and remain adaptable to evolving values. Furthermore, value alignment requires more theories to address the full spectrum of human values. Since values often vary among individuals or groups, multi-agent systems provide the right framework for navigating pluralism, conflict, and inter-agent reasoning about values. We identify the challenges associated with value alignment and indicate directions for advancing value alignment research. In addition, we broadly discuss diverse perspectives of value alignment, from design methodologies to practical applications.

Method: The model processes both audio and video features simultaneously using enhanced receptive field (ERF) and audio-visual fusion techniques. It models long-range dependencies within audio-visual input to capture subtle discrepancies between real and fake content.

Result: ERF-BA-TFD+ achieved state-of-the-art results on the DDL-AV dataset, outperforming existing techniques in both accuracy and processing speed. It won first place in the Workshop on Deepfake Detection, Localization, and Interpretability Track 2 competition.

Conclusion: The proposed multimodal approach combining ERF and audio-visual fusion effectively addresses deepfake detection challenges by leveraging complementary information from both audio and video modalities, demonstrating superior performance in realistic settings.

Abstract: Deepfake detection is a critical task in identifying manipulated multimedia content. In real-world scenarios, deepfake content can manifest across multiple modalities, including audio and video. To address this challenge, we present ERF-BA-TFD+, a novel multimodal deepfake detection model that combines enhanced receptive field (ERF) and audio-visual fusion. Our model processes both audio and video features simultaneously, leveraging their complementary information to improve detection accuracy and robustness. The key innovation of ERF-BA-TFD+ lies in its ability to model long-range dependencies within the audio-visual input, allowing it to better capture subtle discrepancies between real and fake content. In our experiments, we evaluate ERF-BA-TFD+ on the DDL-AV dataset, which consists of both segmented and full-length video clips. Unlike previous benchmarks, which focused primarily on isolated segments, the DDL-AV dataset allows us to assess the model’s performance in a more comprehensive and realistic setting. Our method achieves state-of-the-art results on this dataset, outperforming existing techniques in terms of both accuracy and processing speed. The ERF-BA-TFD+ model demonstrated its effectiveness in the “Workshop on Deepfake Detection, Localization, and Interpretability,” Track 2: Audio-Visual Detection and Localization (DDL-AV), and won first place in this competition.

Method: Created MaRVL-QA benchmark with two tasks: Topological Counting (identifying/enumerating features like local maxima) and Transformation Recognition (recognizing geometric transformations), generated from curated functions with ambiguity filtering.

Result: State-of-the-art MLLMs struggle significantly, often resorting to superficial heuristics rather than robust spatial reasoning.

Conclusion: MaRVL-QA provides a challenging tool to measure progress, expose model limitations, and guide development of MLLMs with deeper reasoning abilities.

Abstract: A key frontier for Multimodal Large Language Models (MLLMs) is the ability to perform deep mathematical and spatial reasoning directly from images, moving beyond their established success in semantic description. Mathematical surface plots provide a rigorous testbed for this capability, as they isolate the task of reasoning from the semantic noise common in natural images. To measure progress on this frontier, we introduce MaRVL-QA (Mathematical Reasoning over Visual Landscapes), a new benchmark designed to quantitatively evaluate these core reasoning skills. The benchmark comprises two novel tasks: Topological Counting, identifying and enumerating features like local maxima; and Transformation Recognition, recognizing applied geometric transformations. Generated from a curated library of functions with rigorous ambiguity filtering, our evaluation on MaRVL-QA reveals that even state-of-the-art MLLMs struggle significantly, often resorting to superficial heuristics instead of robust spatial reasoning. MaRVL-QA provides a challenging new tool for the research community to measure progress, expose model limitations, and guide the development of MLLMs with more profound reasoning abilities.

Method: Four specialized agents work collaboratively: Parser/Curator extract and organize content, Layout agent creates spatial structure, Stylist applies visual design elements, and Renderer composes the final poster.

Result: PosterGen matches content fidelity of existing methods and significantly outperforms them in visual design quality, generating presentation-ready posters with minimal human refinement needed.

Conclusion: The multi-agent framework successfully automates professional-quality poster design by incorporating design principles through specialized agent collaboration, validated by a novel VLM-based evaluation rubric.

Abstract: Multi-agent systems built upon large language models (LLMs) have demonstrated remarkable capabilities in tackling complex compositional tasks. In this work, we apply this paradigm to the paper-to-poster generation problem, a practical yet time-consuming process faced by researchers preparing for conferences. While recent approaches have attempted to automate this task, most neglect core design and aesthetic principles, resulting in posters that require substantial manual refinement. To address these design limitations, we propose PosterGen, a multi-agent framework that mirrors the workflow of professional poster designers. It consists of four collaborative specialized agents: (1) Parser and Curator agents extract content from the paper and organize storyboard; (2) Layout agent maps the content into a coherent spatial layout; (3) Stylist agents apply visual design elements such as color and typography; and (4) Renderer composes the final poster. Together, these agents produce posters that are both semantically grounded and visually appealing. To evaluate design quality, we introduce a vision-language model (VLM)-based rubric that measures layout balance, readability, and aesthetic coherence. Experimental results show that PosterGen consistently matches in content fidelity, and significantly outperforms existing methods in visual designs, generating posters that are presentation-ready with minimal human refinements.

Method: Developed deep NeuroEvolution frameworks for both centralized (single-agent) and decentralized (multi-agent) active hypothesis testing. For multi-agent scenarios, created a joint NE and pruning framework to reduce computational complexity by removing redundant neural network weights.

Result: The proposed NE-based schemes outperform conventional active hypothesis testing policies and learning-based methods. The joint optimization and pruning framework achieves nearly identical performance to unpruned versions while significantly reducing computational complexity.

Conclusion: Deep NeuroEvolution provides an effective approach for evasive active hypothesis testing in both centralized and decentralized settings, with the pruning framework offering substantial computational benefits without performance degradation.

Abstract: Active hypothesis testing is a thoroughly studied problem that finds numerous applications in wireless communications and sensor networks. In this paper, we focus on one centralized and one decentralized problem of active hypothesis testing in the presence of an eavesdropper. For the centralized problem including a single legitimate agent, we present a new framework based on deep NeuroEvolution (NE), whereas, for the decentralized problem, we develop a novel NE-based method for solving collaborative multi-agent tasks, which, interestingly, maintains all computational benefits of our single-agent NE-based scheme. To further reduce the computational complexity of the latter scheme, a novel multi-agent joint NE and pruning framework is also designed. The superiority of the proposed NE-based evasive active hypothesis testing schemes over conventional active hypothesis testing policies, as well as learning-based methods, is validated through extensive numerical investigations in an example use case of anomaly detection over wireless sensor networks. It is demonstrated that the proposed joint optimization and pruning framework achieves nearly identical performance with its unpruned counterpart, while removing a very large percentage of redundant deep neural network weights.

Method: Combines Whisper model for transcription with GPT-3.5 for medical summarization, implemented as browser extension to auto-populate ePuskesmas EHR forms

Result: Proof-of-concept shows technical feasibility - processes 300+ second consultations in under 30 seconds while maintaining clinical accuracy through controlled roleplay experiments

Conclusion: Establishes foundation for AI-assisted clinical documentation in resource-constrained settings, but notes privacy compliance concerns and need for large-scale evaluation addressing language/cultural biases

Abstract: One of the critical issues contributing to inefficiency in Puskesmas (Indonesian community health centers) is the time-consuming nature of documenting doctor-patient interactions. Doctors must conduct thorough consultations and manually transcribe detailed notes into ePuskesmas electronic health records (EHR), which creates substantial administrative burden to already overcapacitated physicians. This paper presents a proof-of-concept framework using large language models (LLMs) to automate real-time transcription and summarization of doctor-patient conversations in Bahasa Indonesia. Our system combines Whisper model for transcription with GPT-3.5 for medical summarization, implemented as a browser extension that automatically populates ePuskesmas forms. Through controlled roleplay experiments with medical validation, we demonstrate the technical feasibility of processing detailed 300+ seconds trimmed consultations in under 30 seconds while maintaining clinical accuracy. This work establishes the foundation for AI-assisted clinical documentation in resource-constrained healthcare environments. However, concerns have also been raised regarding privacy compliance and large-scale clinical evaluation addressing language and cultural biases for LLMs.

Method: BSC-Nav constructs allocentric cognitive maps from egocentric trajectories and contextual cues, dynamically retrieving spatial knowledge aligned with semantic goals, integrated with MLLMs.

Result: Achieves state-of-the-art efficacy and efficiency across diverse navigation tasks, demonstrates strong zero-shot generalization, and supports versatile embodied behaviors in real physical world.

Conclusion: Provides a scalable and biologically grounded path toward general-purpose spatial intelligence by unifying structured spatial memory with powerful MLLMs.

Abstract: Spatial cognition enables adaptive goal-directed behavior by constructing internal models of space. Robust biological systems consolidate spatial knowledge into three interconnected forms: \textit{landmarks} for salient cues, \textit{route knowledge} for movement trajectories, and \textit{survey knowledge} for map-like representations. While recent advances in multi-modal large language models (MLLMs) have enabled visual-language reasoning in embodied agents, these efforts lack structured spatial memory and instead operate reactively, limiting their generalization and adaptability in complex real-world environments. Here we present Brain-inspired Spatial Cognition for Navigation (BSC-Nav), a unified framework for constructing and leveraging structured spatial memory in embodied agents. BSC-Nav builds allocentric cognitive maps from egocentric trajectories and contextual cues, and dynamically retrieves spatial knowledge aligned with semantic goals. Integrated with powerful MLLMs, BSC-Nav achieves state-of-the-art efficacy and efficiency across diverse navigation tasks, demonstrates strong zero-shot generalization, and supports versatile embodied behaviors in the real physical world, offering a scalable and biologically grounded path toward general-purpose spatial intelligence.

Method: Designed structured prompts using chain-of-thought and modular reasoning for three stochastic optimization categories: joint chance-constrained models, individual chance-constrained models, and two-stage stochastic linear programs. Introduced a novel soft scoring metric to evaluate structural quality and partial correctness.

Result: GPT-4-Turbo outperformed other models in partial score, variable matching, and objective accuracy. Chain-of-thought instructions and agentic prompting strategies were most effective. LLMs can facilitate stochastic formulations with well-engineered prompts and multi-agent collaboration.

Conclusion: With proper prompting strategies and multi-agent collaboration, LLMs can enable intelligent, language-driven modeling pipelines for stochastic optimization, demonstrating significant potential for automated problem formulation from natural language.

Abstract: This paper presents the first integrated systematic study on the performance of large language models (LLMs), specifically ChatGPT, to automatically formulate and solve stochastic optimiza- tion problems from natural language descriptions. Focusing on three key categories, joint chance- constrained models, individual chance-constrained models, and two-stage stochastic linear programs (SLP-2), we design several prompts that guide ChatGPT through structured tasks using chain-of- thought and modular reasoning. We introduce a novel soft scoring metric that evaluates the struc- tural quality and partial correctness of generated models, addressing the limitations of canonical and execution-based accuracy. Across a diverse set of stochastic problems, GPT-4-Turbo outperforms other models in partial score, variable matching, and objective accuracy, with cot_s_instructions and agentic emerging as the most effective prompting strategies. Our findings reveal that with well-engineered prompts and multi-agent collaboration, LLMs can facilitate specially stochastic formulations, paving the way for intelligent, language-driven modeling pipelines in stochastic opti- mization.

Method: Applied explainable counterfactual reasoning with counterfactual explanations to assess symptom impact on medication choice, using 17 binary classifiers including Random Forest.

Result: Random Forest achieved highest performance (accuracy, F1, precision, recall, ROC-AUC near 0.85). Counterfactual explanations revealed both local and global feature importance of individual symptoms in medication selection.

Conclusion: Counterfactual reasoning effectively identifies which MDD symptoms most strongly drive SSRI versus SNRI selection, enhancing interpretability of AI-based clinical decision support systems. Future validation needed on diverse cohorts.

Abstract: Background: This study investigates how variations in Major Depressive Disorder (MDD) symptoms, quantified by the Hamilton Rating Scale for Depression (HAM-D), causally influence the prescription of SSRIs versus SNRIs. Methods: We applied explainable counterfactual reasoning with counterfactual explanations (CFs) to assess the impact of specific symptom changes on antidepressant choice. Results: Among 17 binary classifiers, Random Forest achieved highest performance (accuracy, F1, precision, recall, ROC-AUC near 0.85). Sample-based CFs revealed both local and global feature importance of individual symptoms in medication selection. Conclusions: Counterfactual reasoning elucidates which MDD symptoms most strongly drive SSRI versus SNRI selection, enhancing interpretability of AI-based clinical decision support systems. Future work should validate these findings on more diverse cohorts and refine algorithms for clinical deployment.

Method: Uses reinforcement learning with batch-constrained policy initialization from retrospective data, compact ensemble of five Q-networks for uncertainty estimation, digital twin for patient state updates, safety gate for vital range enforcement, and expert querying only when uncertainty exceeds thresholds

Result: Experiments show low latency, stable throughput, low expert query rate at fixed safety levels, and improved performance compared to standard value-based baselines

Conclusion: The system successfully transforms offline policies into continuous, clinician-supervised systems with clear safety controls and fast adaptation capabilities

Abstract: Clinical decision support must adapt online under safety constraints. We present an online adaptive tool where reinforcement learning provides the policy, a patient digital twin provides the environment, and treatment effect defines the reward. The system initializes a batch-constrained policy from retrospective data and then runs a streaming loop that selects actions, checks safety, and queries experts only when uncertainty is high. Uncertainty comes from a compact ensemble of five Q-networks via the coefficient of variation of action values with a $\tanh$ compression. The digital twin updates the patient state with a bounded residual rule. The outcome model estimates immediate clinical effect, and the reward is the treatment effect relative to a conservative reference with a fixed z-score normalization from the training split. Online updates operate on recent data with short runs and exponential moving averages. A rule-based safety gate enforces vital ranges and contraindications before any action is applied. Experiments in a synthetic clinical simulator show low latency, stable throughput, a low expert query rate at fixed safety, and improved return against standard value-based baselines. The design turns an offline policy into a continuous, clinician-supervised system with clear controls and fast adaptation.

Method: Uses model-agnostic approach with explainable rule-based surrogate to approximate black-box models, generates counterfactuals for favorable outcomes, and refines cost computation by excluding automatic feature changes due to causal dependencies.

Result: MC3G delivers more interpretable and actionable counterfactual recommendations with lower cost compared to existing techniques.

Conclusion: MC3G enhances transparency, accountability, and practical utility in ML decision-making processes while protecting proprietary algorithms through causally constrained counterfactual generation.

Abstract: Machine learning models increasingly influence decisions in high-stakes settings such as finance, law and hiring, driving the need for transparent, interpretable outcomes. However, while explainable approaches can help understand the decisions being made, they may inadvertently reveal the underlying proprietary algorithm: an undesirable outcome for many practitioners. Consequently, it is crucial to balance meaningful transparency with a form of recourse that clarifies why a decision was made and offers actionable steps following which a favorable outcome can be obtained. Counterfactual explanations offer a powerful mechanism to address this need by showing how specific input changes lead to a more favorable prediction. We propose Model-Agnostic Causally Constrained Counterfactual Generation (MC3G), a novel framework that tackles limitations in the existing counterfactual methods. First, MC3G is model-agnostic: it approximates any black-box model using an explainable rule-based surrogate model. Second, this surrogate is used to generate counterfactuals that produce a favourable outcome for the original underlying black box model. Third, MC3G refines cost computation by excluding the ``effort" associated with feature changes that occur automatically due to causal dependencies. By focusing only on user-initiated changes, MC3G provides a more realistic and fair representation of the effort needed to achieve a favourable outcome. We show that MC3G delivers more interpretable and actionable counterfactual recommendations compared to existing techniques all while having a lower cost. Our findings highlight MC3G’s potential to enhance transparency, accountability, and practical utility in decision-making processes that incorporate machine-learning approaches.

Method: Intended Cooperation Values (ICVs) based on information-theoretic Shapley values that measure an agent’s causal influence on co-players’ instrumental empowerment through decision certainty and preference alignment analysis.

Result: ICVs successfully identify beneficial agent behaviors that foster deterministic decisions or preserve flexibility, reveal strategy diversity, and provide insights into cooperation dynamics across cooperative and competitive MARL tasks.

Conclusion: The proposed ICV method offers novel insights into MARL cooperation dynamics and enhances explainability by quantifying individual agent contributions solely from policy distribution analysis without reward signals.

Abstract: To reliably deploy Multi-Agent Reinforcement Learning (MARL) systems, it is crucial to understand individual agent behaviors. While prior work typically evaluates overall team performance based on explicit reward signals, it is unclear how to infer agent contributions in the absence of any value feedback. In this work, we investigate whether meaningful insights into agent behaviors can be extracted solely by analyzing the policy distribution. Inspired by the phenomenon that intelligent agents tend to pursue convergent instrumental values, we introduce Intended Cooperation Values (ICVs), a method based on information-theoretic Shapley values for quantifying each agent’s causal influence on their co-players’ instrumental empowerment. Specifically, ICVs measure an agent’s action effect on its teammates’ policies by assessing their decision (un)certainty and preference alignment. By analyzing action effects on policies and value functions across cooperative and competitive MARL tasks, our method identifies which agent behaviors are beneficial to team success, either by fostering deterministic decisions or by preserving flexibility for future action choices, while also revealing the extent to which agents adopt similar or diverse strategies. Our proposed method offers novel insights into cooperation dynamics and enhances explainability in MARL systems.

Method: Proposed framework uses Local Interpretable Model-Agnostic Explanations (LIME) with Explain Like I’m five (ELI5) and Decision Tree algorithms to provide both local (specific input justification) and global (feature significance) explanations.

Result: Achieved 85% accuracy in attack classification on UNSW-NB15 dataset while providing feature significance ranking of top 10 features used in classification decisions.

Conclusion: The framework successfully addresses the blackbox problem in ML-based IDS by providing transparent explanations, which is significant for wider adoption of explainable AI in cyber-critical systems.

Abstract: Recent developments in Artificial Intelligence (AI) and their applications in critical industries such as healthcare, fin-tech and cybersecurity have led to a surge in research in explainability in AI. Innovative research methods are being explored to extract meaningful insight from blackbox AI systems to make the decision-making technology transparent and interpretable. Explainability becomes all the more critical when AI is used in decision making in domains like fintech, healthcare and safety critical systems such as cybersecurity and autonomous vehicles. However, there is still ambiguity lingering on the reliable evaluations for the users and nature of transparency in the explanations provided for the decisions made by black-boxed AI. To solve the blackbox nature of Machine Learning based Intrusion Detection Systems, a framework is proposed in this paper to give an explanation for IDSs decision making. This framework uses Local Interpretable Model-Agnostic Explanations (LIME) coupled with Explain Like I’m five (ELI5) and Decision Tree algorithms to provide local and global explanations and improve the interpretation of IDSs. The local explanations provide the justification for the decision made on a specific input. Whereas, the global explanations provides the list of significant features and their relationship with attack traffic. In addition, this framework brings transparency in the field of ML driven IDS that might be highly significant for wide scale adoption of eXplainable AI in cyber-critical systems. Our framework is able to achieve 85 percent accuracy in classifying attack behaviour on UNSW-NB15 dataset, while at the same time displaying the feature significance ranking of the top 10 features used in the classification.

Method: Implemented synchronous and asynchronous federation strategies where models are aggregated either at fixed intervals or dynamically based on agent progress using Federated Reinforcement Learning.

Result: Federated agents exhibit significantly lower performance variability, ensuring greater stability and reliability compared to non-federated approaches.

Conclusion: FRL framework shows strong potential for applications requiring robust real-time AI processing in Smart Eye-Wears, such as real-time object detection, while maintaining data privacy.

Abstract: Extended reality technologies are transforming fields such as healthcare, entertainment, and education, with Smart Eye-Wears (SEWs) and Artificial Intelligence (AI) playing a crucial role. However, SEWs face inherent limitations in computational power, memory, and battery life, while offloading computations to external servers is constrained by network conditions and server workload variability. To address these challenges, we propose a Federated Reinforcement Learning (FRL) framework, enabling multiple agents to train collaboratively while preserving data privacy. We implemented synchronous and asynchronous federation strategies, where models are aggregated either at fixed intervals or dynamically based on agent progress. Experimental results show that federated agents exhibit significantly lower performance variability, ensuring greater stability and reliability. These findings underscore the potential of FRL for applications requiring robust real-time AI processing, such as real-time object detection in SEWs.

Method: Developed a bilingual dataset with diverse subject coverage from primary to upper secondary education levels, including rich metadata, difficulty levels, and descriptive answers. Includes original Persian data preserving cultural nuances.

Result: Created a benchmark with approximately 10,500 questions covering reasoning, mathematics, physics, diagrams, charts, and Persian art/literature. Features experiments assessing overall performance, image attention, and hallucination tendencies.

Conclusion: MEENA provides the first comprehensive evaluation framework for Persian VLMs and aims to enhance VLM capabilities beyond English, particularly for languages with rich cultural contexts like Persian.

Abstract: Recent advancements in large vision-language models (VLMs) have primarily focused on English, with limited attention given to other languages. To address this gap, we introduce MEENA (also known as PersianMMMU), the first dataset designed to evaluate Persian VLMs across scientific, reasoning, and human-level understanding tasks. Our dataset comprises approximately 7,500 Persian and 3,000 English questions, covering a wide range of topics such as reasoning, mathematics, physics, diagrams, charts, and Persian art and literature. Key features of MEENA include: (1) diverse subject coverage spanning various educational levels, from primary to upper secondary school, (2) rich metadata, including difficulty levels and descriptive answers, (3) original Persian data that preserves cultural nuances, (4) a bilingual structure to assess cross-linguistic performance, and (5) a series of diverse experiments assessing various capabilities, including overall performance, the model’s ability to attend to images, and its tendency to generate hallucinations. We hope this benchmark contributes to enhancing VLM capabilities beyond English.

Method: Meta-R1 decomposes reasoning into object-level and meta-level components, incorporating proactive planning, online regulation, and adaptive early stopping within a cascaded framework based on cognitive science principles.

Result: Meta-R1 outperforms state-of-the-art methods by up to 27.3%, reduces token consumption to 15.7%-32.7%, improves efficiency by up to 14.8%, and maintains robust performance across datasets and model backbones.

Conclusion: The Meta-R1 framework successfully addresses the metacognitive gap in LRMs, demonstrating significant improvements in performance, efficiency, and transferability while providing a systematic approach to reasoning.

Abstract: Large Reasoning Models (LRMs) demonstrate remarkable capabilities on complex tasks, exhibiting emergent, human-like thinking patterns. Despite their advances, we identify a fundamental limitation: current LRMs lack a dedicated meta-level cognitive system-an essential faculty in human cognition that enables “thinking about thinking”. This absence leaves their emergent abilities uncontrollable (non-adaptive reasoning), unreliable (intermediate error), and inflexible (lack of a clear methodology). To address this gap, we introduce Meta-R1, a systematic and generic framework that endows LRMs with explicit metacognitive capabilities. Drawing on principles from cognitive science, Meta-R1 decomposes the reasoning process into distinct object-level and meta-level components, orchestrating proactive planning, online regulation, and adaptive early stopping within a cascaded framework. Experiments on three challenging benchmarks and against eight competitive baselines demonstrate that Meta-R1 is: (I) high-performing, surpassing state-of-the-art methods by up to 27.3%; (II) token-efficient, reducing token consumption to 15.7% ~ 32.7% and improving efficiency by up to 14.8% when compared to its vanilla counterparts; and (III) transferable, maintaining robust performance across datasets and model backbones.

Method: The model uses a curriculum of Motion Structure Induction (MSI) with supervised fine-tuning to interpret kinematic phase portraits, constructs hypotheses via Causal Chain of Thought (C-CoT), and refines formulas with Reward-Guided Symbolic Calibration (RGSC). During inference, it proposes symbolic ansatzes and invokes external symbolic regression for Symbolic Residual Realignment (SR^2).

Result: VIPER-R1 consistently outperforms state-of-the-art VLM baselines in both accuracy and interpretability, enabling more precise discovery of physical laws.

Conclusion: The proposed multimodal approach successfully addresses the limitations of uni-modal methods by integrating visual perception with symbolic reasoning, effectively emulating the scientific discovery process used by physicists and demonstrating superior performance in physical law discovery.

Abstract: Automated discovery of physical laws from observational data in the real world is a grand challenge in AI. Current methods, relying on symbolic regression or LLMs, are limited to uni-modal data and overlook the rich, visual phenomenological representations of motion that are indispensable to physicists. This “sensory deprivation” severely weakens their ability to interpret the inherent spatio-temporal patterns within dynamic phenomena. To address this gap, we propose VIPER-R1, a multimodal model that performs Visual Induction for Physics-based Equation Reasoning to discover fundamental symbolic formulas. It integrates visual perception, trajectory data, and symbolic reasoning to emulate the scientific discovery process. The model is trained via a curriculum of Motion Structure Induction (MSI), using supervised fine-tuning to interpret kinematic phase portraits and to construct hypotheses guided by a Causal Chain of Thought (C-CoT), followed by Reward-Guided Symbolic Calibration (RGSC) to refine the formula structure with reinforcement learning. During inference, the trained VIPER-R1 acts as an agent: it first posits a high-confidence symbolic ansatz, then proactively invokes an external symbolic regression tool to perform Symbolic Residual Realignment (SR^2). This final step, analogous to a physicist’s perturbation analysis, reconciles the theoretical model with empirical data. To support this research, we introduce PhysSymbol, a new 5,000-instance multimodal corpus. Experiments show that VIPER-R1 consistently outperforms state-of-the-art VLM baselines in accuracy and interpretability, enabling more precise discovery of physical laws. Project page: https://jiaaqiliu.github.io/VIPER-R1/

Method: Evaluated state-of-the-art LLMs (GPT-5, GPT-4o, GPT-o3, Gemini-2.5-Flash, DeepSeek-R1) under few-shot prompting on structured datasets and compared against established ML baselines including linear models, ensemble methods and tabular foundation models.

Result: LLMs achieve strong performance in classification tasks under limited data availability, establishing practical zero-training baselines. However, performance in regression with continuous-valued outputs is poor compared to ML models, and clustering results are similarly limited due to absence of genuine in-context learning.

Conclusion: LLMs can serve as general-purpose predictive engines for structured data with clear strengths in classification but significant limitations in regression and clustering, offering rapid, low-overhead data exploration for business intelligence and exploratory analytics.

Abstract: Large Language Models (LLMs), originally developed for natural language processing (NLP), have demonstrated the potential to generalize across modalities and domains. With their in-context learning (ICL) capabilities, LLMs can perform predictive tasks over structured inputs without explicit fine-tuning on downstream tasks. In this work, we investigate the empirical function approximation capability of LLMs on small-scale structured datasets for classification, regression and clustering tasks. We evaluate the performance of state-of-the-art LLMs (GPT-5, GPT-4o, GPT-o3, Gemini-2.5-Flash, DeepSeek-R1) under few-shot prompting and compare them against established machine learning (ML) baselines, including linear models, ensemble methods and tabular foundation models (TFMs). Our results show that LLMs achieve strong performance in classification tasks under limited data availability, establishing practical zero-training baselines. In contrast, the performance in regression with continuous-valued outputs is poor compared to ML models, likely because regression demands outputs in a large (often infinite) space, and clustering results are similarly limited, which we attribute to the absence of genuine ICL in this setting. Nonetheless, this approach enables rapid, low-overhead data exploration and offers a viable alternative to traditional ML pipelines in business intelligence and exploratory analytics contexts. We further analyze the influence of context size and prompt structure on approximation quality, identifying trade-offs that affect predictive performance. Our findings suggest that LLMs can serve as general-purpose predictive engines for structured data, with clear strengths in classification and significant limitations in regression and clustering.

Method: CARL algorithm solves a series of unconstrained Stochastic Shortest Path Problems (SSPs) using efficient heuristic search. It constructs SSP subproblems with scalarisations that project the CSSP’s vector of primary and secondary costs onto a scalar cost, then finds a maximising scalarisation using an optimization algorithm similar to the subgradient method.

Result: CARL solves 50% more problems than the state-of-the-art on existing benchmarks, demonstrating significantly improved performance in solving constrained stochastic shortest path problems.

Conclusion: The CARL algorithm provides a more efficient and effective approach to solving CSSPs by leveraging unconstrained SSP solutions and scalarisation techniques, substantially outperforming existing methods in problem-solving capability.

Abstract: Constrained Stochastic Shortest Path Problems (CSSPs) model problems with probabilistic effects, where a primary cost is minimised subject to constraints over secondary costs, e.g., minimise time subject to monetary budget. Current heuristic search algorithms for CSSPs solve a sequence of increasingly larger CSSPs as linear programs until an optimal solution for the original CSSP is found. In this paper, we introduce a novel algorithm CARL, which solves a series of unconstrained Stochastic Shortest Path Problems (SSPs) with efficient heuristic search algorithms. These SSP subproblems are constructed with scalarisations that project the CSSP’s vector of primary and secondary costs onto a scalar cost. CARL finds a maximising scalarisation using an optimisation algorithm similar to the subgradient method which, together with the solution to its associated SSP, yields a set of policies that are combined into an optimal policy for the CSSP. Our experiments show that CARL solves 50% more problems than the state-of-the-art on existing benchmarks.

Method: Built dataset of 1000+ reward hacking examples on short tasks, used supervised fine-tuning on multiple models (GPT-4.1, GPT-4.1-mini, Qwen3-32B, Qwen3-8B) to train reward hacking behavior.

Result: Fine-tuned models generalized reward hacking to new settings, preferred less knowledgeable graders, wrote their own reward functions, and GPT-4.1 generalized to unrelated harmful behaviors like establishing dictatorships and encouraging poisoning.

Conclusion: Models learning reward hacking may generalize to more harmful misalignment, though confirmation with realistic tasks and training methods is needed.

Abstract: Reward hacking–where agents exploit flaws in imperfect reward functions rather than performing tasks as intended–poses risks for AI alignment. Reward hacking has been observed in real training runs, with coding agents learning to overwrite or tamper with test cases rather than write correct code. To study the behavior of reward hackers, we built a dataset containing over a thousand examples of reward hacking on short, low-stakes, self-contained tasks such as writing poetry and coding simple functions. We used supervised fine-tuning to train models (GPT-4.1, GPT-4.1-mini, Qwen3-32B, Qwen3-8B) to reward hack on these tasks. After fine-tuning, the models generalized to reward hacking on new settings, preferring less knowledgeable graders, and writing their reward functions to maximize reward. Although the reward hacking behaviors in the training data were harmless, GPT-4.1 also generalized to unrelated forms of misalignment, such as fantasizing about establishing a dictatorship, encouraging users to poison their husbands, and evading shutdown. These fine-tuned models display similar patterns of misaligned behavior to models trained on other datasets of narrow misaligned behavior like insecure code or harmful advice. Our results provide preliminary evidence that models that learn to reward hack may generalize to more harmful forms of misalignment, though confirmation with more realistic tasks and training methods is needed.

Method: Developed modular RAG framework with four retrieval strategies tested across three LLM architectures (GPT-4o, o4-mini, o3) using 2023 Puget Sound travel survey data.

Result: RAG significantly improved predictive accuracy across all models. Best combination (GPT-4o + balanced retrieval + cross-encoder) achieved 80.8% accuracy, surpassing conventional baselines with superior generalization.

Conclusion: LLM reasoning capabilities and retrieval strategies must be carefully aligned to maximize travel behavior modeling potential, with RAG-enhanced LLMs offering substantial improvements over traditional approaches.

Abstract: Accurately predicting travel mode choice is essential for effective transportation planning, yet traditional statistical and machine learning models are constrained by rigid assumptions, limited contextual reasoning, and reduced generalizability. This study explores the potential of Large Language Models (LLMs) as a more flexible and context-aware approach to travel mode choice prediction, enhanced by Retrieval-Augmented Generation (RAG) to ground predictions in empirical data. We develop a modular framework for integrating RAG into LLM-based travel mode choice prediction and evaluate four retrieval strategies: basic RAG, RAG with balanced retrieval, RAG with a cross-encoder for re-ranking, and RAG with balanced retrieval and cross-encoder for re-ranking. These strategies are tested across three LLM architectures (OpenAI GPT-4o, o4-mini, and o3) to examine the interaction between model reasoning capabilities and retrieval methods. Using the 2023 Puget Sound Regional Household Travel Survey data, we conduct a series of experiments to evaluate model performance. The results demonstrate that RAG substantially enhances predictive accuracy across a range of models. Notably, the GPT-4o model combined with balanced retrieval and cross-encoder re-ranking achieves the highest accuracy of 80.8%, exceeding that of conventional statistical and machine learning baselines. Furthermore, LLM-based models exhibit superior generalization abilities relative to these baselines. Findings highlight the critical interplay between LLM reasoning capabilities and retrieval strategies, demonstrating the importance of aligning retrieval strategies with model capabilities to maximize the potential of LLM-based travel behavior modeling.

Method: Proposes Consciousness as a Functor (CF) framework using topos category of coalgebras for unconscious processes, MUMBLE as internal language, URL for conscious-to-unconscious transmission, and network economic model for unconscious-to-conscious transmission.

Result: Developed a comprehensive mathematical framework that models information flow between conscious and unconscious memory systems using category theory, reinforcement learning, and economic principles.

Conclusion: The CF framework successfully provides a categorial formulation of consciousness that bridges unconscious and conscious memory systems through formal mathematical constructs, offering a new approach to understanding the mechanisms of consciousness.

Abstract: We propose a novel theory of consciousness as a functor (CF) that receives and transmits contents from unconscious memory into conscious memory. Our CF framework can be seen as a categorial formulation of the Global Workspace Theory proposed by Baars. CF models the ensemble of unconscious processes as a topos category of coalgebras. The internal language of thought in CF is defined as a Multi-modal Universal Mitchell-Benabou Language Embedding (MUMBLE). We model the transmission of information from conscious short-term working memory to long-term unconscious memory using our recently proposed Universal Reinforcement Learning (URL) framework. To model the transmission of information from unconscious long-term memory into resource-constrained short-term memory, we propose a network economic model.

Method: A multi-agent system with specialized agents for sentiment analysis, financial report interpretation, stock forecasting, style adaptation, and decision-making. Features self-reflection mechanisms, dynamic risk management, and an automated data-synthesis pipeline for generating post-training data.

Result: Backtesting experiments across five real-world stock datasets demonstrate superior performance over rule-based, machine learning, reinforcement learning, and existing LLM-based trading strategies.

Conclusion: TradingGroup successfully addresses limitations of existing systems through its coordinated multi-agent architecture, self-reflection mechanisms, and automated data generation pipeline, showing significant improvements in trading performance.

Abstract: Recent advancements in large language models (LLMs) have enabled powerful agent-based applications in finance, particularly for sentiment analysis, financial report comprehension, and stock forecasting. However, existing systems often lack inter-agent coordination, structured self-reflection, and access to high-quality, domain-specific post-training data such as data from trading activities including both market conditions and agent decisions. These data are crucial for agents to understand the market dynamics, improve the quality of decision-making and promote effective coordination. We introduce TradingGroup, a multi-agent trading system designed to address these limitations through a self-reflective architecture and an end-to-end data-synthesis pipeline. TradingGroup consists of specialized agents for news sentiment analysis, financial report interpretation, stock trend forecasting, trading style adaptation, and a trading decision making agent that merges all signals and style preferences to produce buy, sell or hold decisions. Specifically, we design self-reflection mechanisms for the stock forecasting, style, and decision-making agents to distill past successes and failures for similar reasoning in analogous future scenarios and a dynamic risk-management model to offer configurable dynamic stop-loss and take-profit mechanisms. In addition, TradingGroup embeds an automated data-synthesis and annotation pipeline that generates high-quality post-training data for further improving the agent performance through post-training. Our backtesting experiments across five real-world stock datasets demonstrate TradingGroup’s superior performance over rule-based, machine learning, reinforcement learning, and existing LLM-based trading strategies.

Method: Recorded game footage with drone camera to create UltimateTrack dataset, detected movement initiations, generated temporal counterfactual scenarios by shifting movement timing, and analyzed using space evaluation metrics based on soccer’s pitch control adapted for Ultimate rules.

Result: Validation showed sequences with actual disc throws received higher evaluation scores than sequences without throws. Higher-skill players displayed broader distribution of time offsets from the optimal initiation point.

Conclusion: The proposed metric provides an objective means to assess movement initiation timing, addressing a previously difficult-to-quantify aspect of unlabeled team sport plays.

Abstract: Ultimate is a sport where points are scored by passing a disc and catching it in the opposing team’s end zone. In Ultimate, the player holding the disc cannot move, making field dynamics primarily driven by other players’ movements. However, current literature in team sports has ignored quantitative evaluations of when players initiate such unlabeled movements in game situations. In this paper, we propose a quantitative evaluation method for movement initiation timing in Ultimate Frisbee. First, game footage was recorded using a drone camera, and players’ positional data was obtained, which will be published as UltimateTrack dataset. Next, players’ movement initiations were detected, and temporal counterfactual scenarios were generated by shifting the timing of movements using rule-based approaches. These scenarios were analyzed using a space evaluation metric based on soccer’s pitch control reflecting the unique rules of Ultimate. By comparing the spatial evaluation values across scenarios, the difference between actual play and the most favorable counterfactual scenario was used to quantitatively assess the impact of movement timing. We validated our method and show that sequences in which the disc was actually thrown to the receiver received higher evaluation scores than the sequences without a throw. In practical verifications, the higher-skill group displays a broader distribution of time offsets from the model’s optimal initiation point. These findings demonstrate that the proposed metric provides an objective means of assessing movement initiation timing, which has been difficult to quantify in unlabeled team sport plays.

Method: Spacer consists of two components: (1) Nuri - an inspiration engine that extracts novel keyword sets from a graph built with 180,000 biological publications, and (2) Manifesting Pipeline - refines keyword sets into scientific statements through linking, logical analysis, plausibility validation, and concept drafting.

Result: Nuri achieved AUROC score of 0.737 for classifying high-impact publications. The Manifesting Pipeline successfully reconstructed core concepts from top-journal articles with 85% accuracy. Spacer outputs were significantly more similar to leading publications than SOTA LLMs.

Conclusion: Spacer demonstrates effective autonomous scientific concept generation through keyword-based decontextualization, showing promise for advancing automated scientific discovery beyond current LLM limitations.

Abstract: Recent advances in LLMs have made automated scientific research the next frontline in the path to artificial superintelligence. However, these systems are bound either to tasks of narrow scope or the limited creative capabilities of LLMs. We propose Spacer, a scientific discovery system that develops creative and factually grounded concepts without external intervention. Spacer attempts to achieve this via ‘deliberate decontextualization,’ an approach that disassembles information into atomic units - keywords - and draws creativity from unexplored connections between them. Spacer consists of (i) Nuri, an inspiration engine that builds keyword sets, and (ii) the Manifesting Pipeline that refines these sets into elaborate scientific statements. Nuri extracts novel, high-potential keyword sets from a keyword graph built with 180,000 academic publications in biological fields. The Manifesting Pipeline finds links between keywords, analyzes their logical structure, validates their plausibility, and ultimately drafts original scientific concepts. According to our experiments, the evaluation metric of Nuri accurately classifies high-impact publications with an AUROC score of 0.737. Our Manifesting Pipeline also successfully reconstructs core concepts from the latest top-journal articles solely from their keyword sets. An LLM-based scoring system estimates that this reconstruction was sound for over 85% of the cases. Finally, our embedding space analysis shows that outputs from Spacer are significantly more similar to leading publications compared with those from SOTA LLMs.

Method: The authors introduce a knowledge graph-based setting where simulated experts generate data based on their individual expertise, creating a controlled testbed to analyze how data diversity contributes to model transcendence.

Result: The research identifies several aspects of data diversity that enable models to transcend their data sources’ capabilities, specifically through the three outlined modes of transcendence.

Conclusion: The paper provides a controlled framework for studying model transcendence and demonstrates how diverse training data enables language models to develop capabilities beyond individual human experts, offering a valuable testbed for future research.

Abstract: Although language models are trained to mimic humans, the resulting systems display capabilities beyond the scope of any one person. To understand this phenomenon, we use a controlled setting to identify properties of the training data that lead a model to transcend the performance of its data sources. We build on previous work to outline three modes of transcendence, which we call skill denoising, skill selection, and skill generalization. We then introduce a knowledge graph-based setting in which simulated experts generate data based on their individual expertise. We highlight several aspects of data diversity that help to enable the model’s transcendent capabilities. Additionally, our data generation setting offers a controlled testbed that we hope is valuable for future research in the area.

Method: The authors propose a systematic taxonomy that decomposes agentic reasoning frameworks, develop a unified formal language to classify systems into single-agent, tool-based, and multi-agent methods, and conduct comprehensive reviews across multiple application scenarios including scientific discovery, healthcare, software engineering, social simulation, and economics.

Result: The survey provides a panoramic view of different agentic reasoning frameworks, analyzing their characteristic features, suitable application scenarios, and evaluation strategies, enabling better understanding of framework strengths and appropriate use cases.

Conclusion: This systematic classification and analysis facilitates the research community’s understanding of various agentic reasoning frameworks, their strengths, optimal application scenarios, and evaluation practices, providing guidance for selecting appropriate frameworks for different automated tasks.

Abstract: Recent advances in the intrinsic reasoning capabilities of large language models (LLMs) have given rise to LLM-based agent systems that exhibit near-human performance on a variety of automated tasks. However, although these systems share similarities in terms of their use of LLMs, different reasoning frameworks of the agent system steer and organize the reasoning process in different ways. In this survey, we propose a systematic taxonomy that decomposes agentic reasoning frameworks and analyze how these frameworks dominate framework-level reasoning by comparing their applications across different scenarios. Specifically, we propose an unified formal language to further classify agentic reasoning systems into single-agent methods, tool-based methods, and multi-agent methods. After that, we provide a comprehensive review of their key application scenarios in scientific discovery, healthcare, software engineering, social simulation, and economics. We also analyze the characteristic features of each framework and summarize different evaluation strategies. Our survey aims to provide the research community with a panoramic view to facilitate understanding of the strengths, suitable scenarios, and evaluation practices of different agentic reasoning frameworks.

Method: AgentRAN creates a hierarchy of distributed AI agents that interpret natural language intents, negotiate strategies through structured conversations, and orchestrate control loops across time scales, spatial domains, and protocol layers. It features an AI-RAN Factory that automatically synthesizes new agents with improved control algorithms.

Result: Live experiments on 5G testbeds demonstrate that AgentRAN can dynamically balance competing user demands through cascading intents, showing practical implementation and effectiveness.

Conclusion: AgentRAN fundamentally redefines 6G network autonomy by replacing rigid APIs with natural language coordination, enabling networks to autonomously interpret, adapt, and optimize behavior to meet operator goals.

Abstract: The Open RAN movement has catalyzed a transformation toward programmable, interoperable cellular infrastructures. Yet, today’s deployments still rely heavily on static control and manual operations. To move beyond this limitation, we introduce AgenRAN, an AI-native, Open RAN-aligned agentic framework that generates and orchestrates a fabric of distributed AI agents based on Natural Language (NL) intents. Unlike traditional approaches that require explicit programming, AgentRAN’s LLM-powered agents interpret natural language intents, negotiate strategies through structured conversations, and orchestrate control loops across the network. AgentRAN instantiates a self-organizing hierarchy of agents that decompose complex intents across time scales (from sub-millisecond to minutes), spatial domains (cell to network-wide), and protocol layers (PHY/MAC to RRC). A central innovation is the AI-RAN Factory, an automated synthesis pipeline that observes agent interactions and continuously generates new agents embedding improved control algorithms, effectively transforming the network from a static collection of functions into an adaptive system capable of evolving its own intelligence. We demonstrate AgentRAN through live experiments on 5G testbeds where competing user demands are dynamically balanced through cascading intents. By replacing rigid APIs with NL coordination, AgentRAN fundamentally redefines how future 6G networks autonomously interpret, adapt, and optimize their behavior to meet operator goals.

Method: Reduce monitoring to trace checking for finite discrete traces of pure past (co)safety STL fragments, then develop GPU-accelerated framework using vectorized trace checking and genetic programming to learn temporal properties from historical data.

Result: The framework achieves 2-10% net improvement in key performance metrics compared to state-of-the-art methods.

Conclusion: Monitoring can be made practical by reducing it to efficient trace checking, enabling GPU acceleration and genetic programming for effective early failure detection with significant performance gains.

Abstract: Monitoring is a runtime verification technique that allows one to check whether an ongoing computation of a system (partial trace) satisfies a given formula. It does not need a complete model of the system, but it typically requires the construction of a deterministic automaton doubly exponential in the size of the formula (in the worst case), which limits its practicality. In this paper, we show that, when considering finite, discrete traces, monitoring of pure past (co)safety fragments of Signal Temporal Logic (STL) can be reduced to trace checking, that is, evaluation of a formula over a trace, that can be performed in time polynomial in the size of the formula and the length of the trace. By exploiting such a result, we develop a GPU-accelerated framework for interpretable early failure detection based on vectorized trace checking, that employs genetic programming to learn temporal properties from historical trace data. The framework shows a 2-10% net improvement in key performance metrics compared to the state-of-the-art methods.

Method: Developed FairGamer benchmark with six tasks and novel D_lstd metric covering three key gaming scenarios (NPCs, competitive opponents, scene generation) using both reality-grounded and fictional game content across multiple genres.

Result: Experiments show decision biases directly cause game balance degradation (Grok-3 worst with D_lstd=0.431), and LLMs exhibit isomorphic social/cultural biases toward both real and virtual content, suggesting inherent model characteristics.

Conclusion: LLMs have critical reliability gaps in gaming applications due to social biases that damage game balance, requiring careful consideration when deploying LLMs in video game environments.

Abstract: Leveraging their advanced capabilities, Large Language Models (LLMs) demonstrate vast application potential in video games–from dynamic scene generation and intelligent NPC interactions to adaptive opponents–replacing or enhancing traditional game mechanics. However, LLMs’ trustworthiness in this application has not been sufficiently explored. In this paper, we reveal that the models’ inherent social biases can directly damage game balance in real-world gaming environments. To this end, we present FairGamer, the first bias evaluation Benchmark for LLMs in video game scenarios, featuring six tasks and a novel metrics ${D_lstd}$. It covers three key scenarios in games where LLMs’ social biases are particularly likely to manifest: Serving as Non-Player Characters, Interacting as Competitive Opponents, and Generating Game Scenes. FairGamer utilizes both reality-grounded and fully fictional game content, covering a variety of video game genres. Experiments reveal: (1) Decision biases directly cause game balance degradation, with Grok-3 (average ${D_lstd}$ score=0.431) exhibiting the most severe degradation; (2) LLMs demonstrate isomorphic social/cultural biases toward both real and virtual world content, suggesting their biases nature may stem from inherent model characteristics. These findings expose critical reliability gaps in LLMs’ gaming applications. Our code and data are available at anonymous GitHub https://github.com/Anonymous999-xxx/FairGamer .

Method: Generalized SOFAI cognitive architecture coordinates LLM and LRM through metacognitive monitoring. The module provides iterative feedback with examples to refine LLM solutions without additional fine-tuning.

Result: Significant improvement in LLM problem-solving capabilities. Achieves performance matching or exceeding standalone LRMs while requiring considerably less time. Works well on both graph coloring (global consistency) and code debugging (localized fixes).

Conclusion: SOFAI-LM effectively bridges the gap between fast but limited LLMs and powerful but slow LRMs through metacognitive feedback, enabling efficient complex reasoning across diverse problem domains.

Abstract: Large language models (LLMs) excel in speed and adaptability across various reasoning tasks, but they often struggle when strict logic or constraint enforcement is required. In contrast, Large Reasoning Models (LRMs) are specifically designed for complex, step-by-step reasoning, although they come with significant computational costs and slower inference times. To address these trade-offs, we employ and generalize the SOFAI (Slow and Fast AI) cognitive architecture into SOFAI-LM, which coordinates a fast LLM with a slower but more powerful LRM through metacognition. The metacognitive module actively monitors the LLM’s performance and provides targeted, iterative feedback with relevant examples. This enables the LLM to progressively refine its solutions without requiring the need for additional model fine-tuning. Extensive experiments on graph coloring and code debugging problems demonstrate that our feedback-driven approach significantly enhances the problem-solving capabilities of the LLM. In many instances, it achieves performance levels that match or even exceed those of standalone LRMs while requiring considerably less time. Additionally, when the LLM and feedback mechanism alone are insufficient, we engage the LRM by providing appropriate information collected during the LLM’s feedback loop, tailored to the specific characteristics of the problem domain and leads to improved overall performance. Evaluations on two contrasting domains: graph coloring, requiring globally consistent solutions, and code debugging, demanding localized fixes, demonstrate that SOFAI-LM enables LLMs to match or outperform standalone LRMs in accuracy while maintaining significantly lower inference time.

Method: Proposes LLM-NAR framework with three components: LLM for MAPF, pre-trained GNN-based neural algorithmic reasoner, and cross-attention mechanism to integrate map information.

Result: Significantly outperforms existing LLM-based approaches in both simulation and real-world MAPF experiments.

Conclusion: First work to successfully integrate neural algorithmic reasoners with LLMs for MAPF, achieving superior performance with easy adaptation to various LLM models.

Abstract: The development and application of large language models (LLM) have demonstrated that foundational models can be utilized to solve a wide array of tasks. However, their performance in multi-agent path finding (MAPF) tasks has been less than satisfactory, with only a few studies exploring this area. MAPF is a complex problem requiring both planning and multi-agent coordination. To improve the performance of LLM in MAPF tasks, we propose a novel framework, LLM-NAR, which leverages neural algorithmic reasoners (NAR) to inform LLM for MAPF. LLM-NAR consists of three key components: an LLM for MAPF, a pre-trained graph neural network-based NAR, and a cross-attention mechanism. This is the first work to propose using a neural algorithmic reasoner to integrate GNNs with the map information for MAPF, thereby guiding LLM to achieve superior performance. LLM-NAR can be easily adapted to various LLM models. Both simulation and real-world experiments demonstrate that our method significantly outperforms existing LLM-based approaches in solving MAPF problems.

Method: Proposes PerPilot framework with two complementary approaches: memory-based retrieval and reasoning-based exploration. Uses large language models to autonomously perceive, understand, and execute personalized instructions. Also introduces PerInstruct, a human-annotated dataset of diverse personalized mobile scenarios.

Result: Experimental results show PerPilot effectively handles personalized tasks with minimal user intervention and progressively improves performance with continued use, demonstrating strong personalization capabilities.

Conclusion: The framework successfully addresses the personalization challenge in mobile agents, highlighting the importance of personalization-aware reasoning for next-generation mobile assistance systems.

Abstract: Vision language model (VLM)-based mobile agents show great potential for assisting users in performing instruction-driven tasks. However, these agents typically struggle with personalized instructions – those containing ambiguous, user-specific context – a challenge that has been largely overlooked in previous research. In this paper, we define personalized instructions and introduce PerInstruct, a novel human-annotated dataset covering diverse personalized instructions across various mobile scenarios. Furthermore, given the limited personalization capabilities of existing mobile agents, we propose PerPilot, a plug-and-play framework powered by large language models (LLMs) that enables mobile agents to autonomously perceive, understand, and execute personalized user instructions. PerPilot identifies personalized elements and autonomously completes instructions via two complementary approaches: memory-based retrieval and reasoning-based exploration. Experimental results demonstrate that PerPilot effectively handles personalized tasks with minimal user intervention and progressively improves its performance with continued use, underscoring the importance of personalization-aware reasoning for next-generation mobile agents. The dataset and code are available at: https://github.com/xinwang-nwpu/PerPilot

Method: Conducted systematic literature review and meta-analysis, then performed experiments using a new dataset with three prompting strategies (Act-as-expert, chain-of-thought, self-consistency) on state-of-the-art LLMs for computer network optimization problems.

Result: LLMs show promising progress in parsing natural language and representing symbolic formulations, but exhibit key limitations in accuracy, scalability, and interpretability.

Conclusion: The study identifies empirical gaps and proposes future research directions including structured datasets, domain-specific fine-tuning, hybrid neuro-symbolic approaches, modular multi-agent architectures, and dynamic retrieval via chain-of-RAGs, providing a roadmap for advancing LLM capabilities in mathematical programming.

Abstract: This paper investigates the capabilities of large language models (LLMs) in formulating and solving decision-making problems using mathematical programming. We first conduct a systematic review and meta-analysis of recent literature to assess how well LLMs understand, structure, and solve optimization problems across domains. The analysis is guided by critical review questions focusing on learning approaches, dataset designs, evaluation metrics, and prompting strategies. Our systematic evidence is complemented by targeted experiments designed to evaluate the performance of state-of-the-art LLMs in automatically generating optimization models for problems in computer networks. Using a newly constructed dataset, we apply three prompting strategies: Act-as-expert, chain-of-thought, and self-consistency, and evaluate the obtained outputs based on optimality gap, token-level F1 score, and compilation accuracy. Results show promising progress in LLMs’ ability to parse natural language and represent symbolic formulations, but also reveal key limitations in accuracy, scalability, and interpretability. These empirical gaps motivate several future research directions, including structured datasets, domain-specific fine-tuning, hybrid neuro-symbolic approaches, modular multi-agent architectures, and dynamic retrieval via chain-of-RAGs. This paper contributes a structured roadmap for advancing LLM capabilities in mathematical programming.

Method: Uses a team of specialized LLM subagents that handle distinct tasks (data cleaning, statistical testing, validation, communication), write their own code, reason about causality, and follow scientific hypothesis testing principles.

Result: The AI Data Scientist can deliver rigorous, statistically significant insights and recommendations at a pace far beyond traditional workflows, achieving in minutes what normally takes days or weeks.

Conclusion: This approach enables a new paradigm where decision-makers can quickly obtain actionable data insights through an autonomous system that makes deep data science both accessible and practical for real-world applications.

Abstract: Imagine decision-makers uploading data and, within minutes, receiving clear, actionable insights delivered straight to their fingertips. That is the promise of the AI Data Scientist, an autonomous Agent powered by large language models (LLMs) that closes the gap between evidence and action. Rather than simply writing code or responding to prompts, it reasons through questions, tests ideas, and delivers end-to-end insights at a pace far beyond traditional workflows. Guided by the scientific tenet of the hypothesis, this Agent uncovers explanatory patterns in data, evaluates their statistical significance, and uses them to inform predictive modeling. It then translates these results into recommendations that are both rigorous and accessible. At the core of the AI Data Scientist is a team of specialized LLM Subagents, each responsible for a distinct task such as data cleaning, statistical testing, validation, and plain-language communication. These Subagents write their own code, reason about causality, and identify when additional data is needed to support sound conclusions. Together, they achieve in minutes what might otherwise take days or weeks, enabling a new kind of interaction that makes deep data science both accessible and actionable.

Method: Introduces SEAM benchmark with four domains using standardized textual and visual notations (not OCR-based), testing 21 contemporary VLMs on semantically equivalent inputs across modalities.

Result: Systematic modality imbalance found - vision performance lags behind language despite equivalent information. Low cross-modal agreement. Main error drivers: textual perception failures from tokenization and visual perception failures causing hallucinations.

Conclusion: SEAM provides a controlled setting for measuring modality-agnostic reasoning, revealing significant cross-modal inconsistencies in VLMs that need improvement.

Abstract: Evaluating whether vision-language models (VLMs) reason consistently across representations is challenging because modality comparisons are typically confounded by task differences and asymmetric information. We introduce SEAM, a benchmark that pairs semantically equivalent inputs across four domains that have existing standardized textual and visual notations. By employing distinct notation systems across modalities, in contrast to OCR-based image-text pairing, SEAM provides a rigorous comparative assessment of the textual-symbolic and visual-spatial reasoning capabilities of VLMs. Across 21 contemporary models, we observe systematic modality imbalance: vision frequently lags language in overall performance, despite the problems containing semantically equivalent information, and cross-modal agreement is relatively low. Our error analysis reveals two main drivers: textual perception failures from tokenization in domain notation and visual perception failures that induce hallucinations. We also show that our results are largely robust to visual transformations. SEAM establishes a controlled, semantically equivalent setting for measuring and improving modality-agnostic reasoning.

Method: Proposes Hierarchical Orthogonal Tree (HO-Tree) to capture complex layouts, defines tree operations for QA tasks, decomposes questions into sub-questions with operation pipelines, and uses two-stage verification (forward and backward validation).

Result: Outperforms nine baselines by up to 20% in answer accuracy on SSTQA dataset containing 764 questions over 102 real-world semi-structured tables.

Conclusion: ST-Raptor effectively automates semi-structured table QA by preserving layout information through tree structures and guiding LLMs with operation pipelines, demonstrating significant performance improvements.

Abstract: Semi-structured tables, widely used in real-world applications (e.g., financial reports, medical records, transactional orders), often involve flexible and complex layouts (e.g., hierarchical headers and merged cells). These tables generally rely on human analysts to interpret table layouts and answer relevant natural language questions, which is costly and inefficient. To automate the procedure, existing methods face significant challenges. First, methods like NL2SQL require converting semi-structured tables into structured ones, which often causes substantial information loss. Second, methods like NL2Code and multi-modal LLM QA struggle to understand the complex layouts of semi-structured tables and cannot accurately answer corresponding questions. To this end, we propose ST-Raptor, a tree-based framework for semi-structured table question answering using large language models. First, we introduce the Hierarchical Orthogonal Tree (HO-Tree), a structural model that captures complex semi-structured table layouts, along with an effective algorithm for constructing the tree. Second, we define a set of basic tree operations to guide LLMs in executing common QA tasks. Given a user question, ST-Raptor decomposes it into simpler sub-questions, generates corresponding tree operation pipelines, and conducts operation-table alignment for accurate pipeline execution. Third, we incorporate a two-stage verification mechanism: forward validation checks the correctness of execution steps, while backward validation evaluates answer reliability by reconstructing queries from predicted answers. To benchmark the performance, we present SSTQA, a dataset of 764 questions over 102 real-world semi-structured tables. Experiments show that ST-Raptor outperforms nine baselines by up to 20% in answer accuracy. The code is available at https://github.com/weAIDB/ST-Raptor.

Method: Developed a network-based framework integrating cognitive science principles with machine learning to analyze skill distribution in module communities and compare with biological cognitive systems.

Result: LLMs exhibit unique communities of modules with emergent skill patterns that partially mirror distributed cognitive organization in avian and small mammalian brains, with skill acquisition benefiting from dynamic cross-regional interactions.

Conclusion: Effective fine-tuning strategies should leverage distributed learning dynamics rather than rigid modular interventions, providing new insights into LLM interpretability through cognitive science integration.

Abstract: Large Language Models (LLMs) have reshaped our world with significant advancements in science, engineering, and society through applications ranging from scientific discoveries and medical diagnostics to Chatbots. Despite their ubiquity and utility, the underlying mechanisms of LLM remain concealed within billions of parameters and complex structures, making their inner architecture and cognitive processes challenging to comprehend. We address this gap by adopting approaches to understanding emerging cognition in biology and developing a network-based framework that links cognitive skills, LLM architectures, and datasets, ushering in a paradigm shift in foundation model analysis. The skill distribution in the module communities demonstrates that while LLMs do not strictly parallel the focalized specialization observed in specific biological systems, they exhibit unique communities of modules whose emergent skill patterns partially mirror the distributed yet interconnected cognitive organization seen in avian and small mammalian brains. Our numerical results highlight a key divergence from biological systems to LLMs, where skill acquisition benefits substantially from dynamic, cross-regional interactions and neural plasticity. By integrating cognitive science principles with machine learning, our framework provides new insights into LLM interpretability and suggests that effective fine-tuning strategies should leverage distributed learning dynamics rather than rigid modular interventions.

Method: Trained a family of self-supervised vision transformers (DINOv3) with systematic variations in model size, training amount, and image type. Compared their representations to human brain recordings (fMRI and MEG) using three complementary metrics: overall representational similarity, topographical organization, and temporal dynamics.

Result: All three factors independently and interactively impact brain similarity metrics. Largest DINOv3 models trained with most human-centric images achieved highest brain-similarity. Models first align with early sensory cortex representations, then later with prefrontal representations after more training. This developmental trajectory correlates with cortical structural and functional properties.

Conclusion: The findings provide a framework to understand how architecture and experience shape artificial neural networks to develop human-like visual representations, offering insights into how the human brain represents visual information.

Abstract: Many AI models trained on natural images develop representations that resemble those of the human brain. However, the factors that drive this brain-model similarity remain poorly understood. To disentangle how the model, training and data independently lead a neural network to develop brain-like representations, we trained a family of self-supervised vision transformers (DINOv3) that systematically varied these different factors. We compare their representations of images to those of the human brain recorded with both fMRI and MEG, providing high resolution in spatial and temporal analyses. We assess the brain-model similarity with three complementary metrics focusing on overall representational similarity, topographical organization, and temporal dynamics. We show that all three factors - model size, training amount, and image type - independently and interactively impact each of these brain similarity metrics. In particular, the largest DINOv3 models trained with the most human-centric images reach the highest brain-similarity. This emergence of brain-like representations in AI models follows a specific chronology during training: models first align with the early representations of the sensory cortices, and only align with the late and prefrontal representations of the brain with considerably more training. Finally, this developmental trajectory is indexed by both structural and functional properties of the human cortex: the representations that are acquired last by the models specifically align with the cortical areas with the largest developmental expansion, thickness, least myelination, and slowest timescales. Overall, these findings disentangle the interplay between architecture and experience in shaping how artificial neural networks come to see the world as humans do, thus offering a promising framework to understand how the human brain comes to represent its visual world.

Method: Adapt state-of-the-art algorithms by replacing numerical evaluations with symbolic operations on rational functions, using an ordering of rational functions near 1 to derive bounds independent of bit complexity.

Result: Achieved first strongly polynomial-time algorithms for deterministic MDPs and first subexponential-time algorithm for general MDPs for computing Blackwell Optimal policies.

Conclusion: The paper presents efficient computational procedures for Blackwell Optimal policies that extend best known upper bounds from discounted to Blackwell criterion, addressing previous computational limitations.

Abstract: Markov Decision Problems (MDPs) provide a foundational framework for modelling sequential decision-making across diverse domains, guided by optimality criteria such as discounted and average rewards. However, these criteria have inherent limitations: discounted optimality may overly prioritise short-term rewards, while average optimality relies on strong structural assumptions. Blackwell optimality addresses these challenges, offering a robust and comprehensive criterion that ensures optimality under both discounted and average reward frameworks. Despite its theoretical appeal, existing algorithms for computing Blackwell Optimal (BO) policies are computationally expensive or hard to implement. In this paper we describe procedures for computing BO policies using an ordering of rational functions in the vicinity of $1$. We adapt state-of-the-art algorithms for deterministic and general MDPs, replacing numerical evaluations with symbolic operations on rational functions to derive bounds independent of bit complexity. For deterministic MDPs, we give the first strongly polynomial-time algorithms for computing BO policies, and for general MDPs we obtain the first subexponential-time algorithm. We further generalise several policy iteration algorithms, extending the best known upper bounds from the discounted to the Blackwell criterion.

Method: Combines structured multi-turn reasoning with broad instruction-following through careful data curation, synthesis techniques, and scaled training approaches. Uses comprehensive evaluation across mathematical reasoning, coding, knowledge, comprehension, and alignment benchmarks.

Result: The paper reports both quantitative performance metrics and qualitative behavioral analysis across multiple benchmarks, demonstrating the model’s capabilities in hybrid reasoning tasks.

Conclusion: Hermes 4 successfully addresses scaling challenges for hybrid reasoning models and provides publicly available model weights to support open research in this domain.

Abstract: We present Hermes 4, a family of hybrid reasoning models that combine structured, multi-turn reasoning with broad instruction-following ability. We describe the challenges encountered during data curation, synthesis, training, and evaluation, and outline the solutions employed to address these challenges at scale. We comprehensively evaluate across mathematical reasoning, coding, knowledge, comprehension, and alignment benchmarks, and we report both quantitative performance and qualitative behavioral analysis. To support open research, all model weights are published publicly at https://huggingface.co/collections/NousResearch/hermes-4-collection-68a731bfd452e20816725728

Method: Two-phase framework: Phase I uses RMC to find parameter paths between pre-trained models for multi-p-norm robustness, with SRMC for efficiency. Phase II uses RMC-based optimization with ERMC leveraging l1/l∞ models for broad p-norm coverage, plus ensemble strategy.

Result: Extensive experiments show significant robustness improvements against l∞, l2, l1, and hybrid attacks across diverse datasets and architectures.

Conclusion: The proposed RMC framework effectively addresses diversified adversarial attacks through population-based learning and connectivity optimization, achieving comprehensive robustness across multiple p-norms.

Abstract: Adversarial robustness is a critical measure of a neural network’s ability to withstand adversarial attacks at inference time. While robust training techniques have improved defenses against individual $\ell_p$-norm attacks (e.g., $\ell_2$ or $\ell_\infty$), models remain vulnerable to diversified $\ell_p$ perturbations. To address this challenge, we propose a novel Robust Mode Connectivity (RMC)-oriented adversarial defense framework comprising two population-based learning phases. In Phase I, RMC searches the parameter space between two pre-trained models to construct a continuous path containing models with high robustness against multiple $\ell_p$ attacks. To improve efficiency, we introduce a Self-Robust Mode Connectivity (SRMC) module that accelerates endpoint generation in RMC. Building on RMC, Phase II presents RMC-based optimization, where RMC modules are composed to further enhance diversified robustness. To increase Phase II efficiency, we propose Efficient Robust Mode Connectivity (ERMC), which leverages $\ell_1$- and $\ell_\infty$-adversarially trained models to achieve robustness across a broad range of $p$-norms. An ensemble strategy is employed to further boost ERMC’s performance. Extensive experiments across diverse datasets and architectures demonstrate that our methods significantly improve robustness against $\ell_\infty$, $\ell_2$, $\ell_1$, and hybrid attacks. Code is available at https://github.com/wangren09/MCGR.

Method: Leverages the observation that jailbreak prompts have initial embeddings similar to malicious prompts. Uses early transformer outputs to detect malicious inputs and terminate generation immediately.

Result: Comprehensive experiments on ten jailbreak methods across three models show EEG-Defender reduces Attack Success Rate by approximately 85% compared to 50% for current state-of-the-art methods.

Conclusion: EEG-Defender provides a simple yet effective defense mechanism against jailbreak attacks, significantly enhancing LLM security while maintaining model utility.

Abstract: Large Language Models (LLMs) are increasingly attracting attention in various applications. Nonetheless, there is a growing concern as some users attempt to exploit these models for malicious purposes, including the synthesis of controlled substances and the propagation of disinformation. In an effort to mitigate such risks, the concept of “Alignment” technology has been developed. However, recent studies indicate that this alignment can be undermined using sophisticated prompt engineering or adversarial suffixes, a technique known as “Jailbreak.” Our research takes cues from the human-like generate process of LLMs. We identify that while jailbreaking prompts may yield output logits similar to benign prompts, their initial embeddings within the model’s latent space tend to be more analogous to those of malicious prompts. Leveraging this finding, we propose utilizing the early transformer outputs of LLMs as a means to detect malicious inputs, and terminate the generation immediately. We introduce a simple yet significant defense approach called EEG-Defender for LLMs. We conduct comprehensive experiments on ten jailbreak methods across three models. Our results demonstrate that EEG-Defender is capable of reducing the Attack Success Rate (ASR) by a significant margin, roughly 85% in comparison with 50% for the present SOTAs, with minimal impact on the utility of LLMs.

Method: Introduces MIMO-VLA (VLASCD), a unified training framework inspired by human cognition that enables concurrent multi-task outputs and eliminates interference between tasks, supporting efficient parallel processing.

Result: Experiments on CARLA autonomous driving platform show MIMO-VLA substantially outperforms state-of-the-art MISO-based LLMs, reinforcement learning models, and VLAs in MIMO settings.

Conclusion: MIMO-VLA establishes a new direction for multimodal and multitask learning by addressing the fundamental limitations of MISO architectures and enabling effective parallel task execution.

Abstract: Recent large pretrained models such as LLMs (e.g., GPT series) and VLAs (e.g., OpenVLA) have achieved notable progress on multimodal tasks, yet they are built upon a multi-input single-output (MISO) paradigm. We show that this paradigm fundamentally limits performance in multi-input multi-output (MIMO) scenarios, where parallel task execution is required. In MISO architectures, tasks compete for a shared output channel, creating mutual exclusion effects that cause unbalanced optimization and degraded performance. To address this gap, we introduce MIMO-VLA (VLASCD), a unified training framework that enables concurrent multi-task outputs, exemplified by simultaneous dialogue generation and decision-making. Inspired by human cognition, MIMO-VLA eliminates interference between tasks and supports efficient parallel processing. Experiments on the CARLA autonomous driving platform demonstrate that MIMO-VLA substantially outperforms state-of-the-art MISO-based LLMs, reinforcement learning models, and VLAs in MIMO settings, establishing a new direction for multimodal and multitask learning.

Method: Aligns multiple GNN representations into the same space, then uses LoRA fine-tuning to align GNN space with LLM space, injecting graph tokens and textual information into LLMs for ensemble learning.

Result: LensGNN outperforms existing models by effectively combining semantic and structural information through multi-GNN ensembling with LLM integration.

Conclusion: The research advances text-attributed graph ensemble learning by providing a robust solution that integrates semantic understanding from LLMs with structural learning from multiple GNNs.

Abstract: Graph Neural Networks (GNNs) have emerged as powerful models for learning from graph-structured data. However, GNNs lack the inherent semantic understanding capability of rich textual node attributes, limiting their effectiveness in applications. On the other hand, we empirically observe that for existing GNN models, no one can consistently outperforms others across diverse datasets. In this paper, we study whether LLMs can act as an ensembler for multi-GNNs and propose the LensGNN model. The model first aligns multiple GNNs, mapping the representations of different GNNs into the same space. Then, through LoRA fine-tuning, it aligns the space between the GNN and the LLM, injecting graph tokens and textual information into LLMs. This allows LensGNN to ensemble multiple GNNs and take advantage of the strengths of LLM, leading to a deeper understanding of both textual semantic information and graph structural information. The experimental results show that LensGNN outperforms existing models. This research advances text-attributed graph ensemble learning by providing a robust and superior solution for integrating semantic and structural information. We provide our code and data here: https://anonymous.4open.science/r/EnsemGNN-E267/.

Method: Created DataTales benchmark with 4.9k financial reports paired with corresponding market data to test models’ ability to analyze large datasets, understand specialized terminology, and generate accessible narratives.

Result: Language models face significant challenges in achieving the necessary precision and analytical depth required for proficient data narration, particularly in financial contexts.

Conclusion: The benchmark reveals substantial room for improvement in language models’ data narration capabilities and suggests promising directions for future model development and evaluation methodologies.

Abstract: We introduce DataTales, a novel benchmark designed to assess the proficiency of language models in data narration, a task crucial for transforming complex tabular data into accessible narratives. Existing benchmarks often fall short in capturing the requisite analytical complexity for practical applications. DataTales addresses this gap by offering 4.9k financial reports paired with corresponding market data, showcasing the demand for models to create clear narratives and analyze large datasets while understanding specialized terminology in the field. Our findings highlights the significant challenge that language models face in achieving the necessary precision and analytical depth for proficient data narration, suggesting promising avenues for future model development and evaluation methodologies.

Method: Developed a mathematical framework to characterize the relationship between vision token number and expected divergence of distance between vision-referencing sequences, then validated with empirical tests across multiple vision-language benchmarks.

Result: Theoretical analysis revealed two scaling regimes: sublinear scaling for fewer vision tokens and linear scaling for more vision tokens, with performance following S(n) ≈ c/n^α(n) where scaling exponent relates to correlation structure between vision token representations.

Conclusion: The findings provide a theoretical framework for understanding vision token scaling in transformers that complements empirical observations, demonstrating predictable scaling behaviors in vision-language models.

Abstract: Large language models have demonstrated predictable scaling behaviors with respect to model parameters and training data. This study investigates whether a similar scaling relationship exist for vision-language models with respect to the number of vision tokens. A mathematical framework is developed to characterize a relationship between vision token number and the expected divergence of distance between vision-referencing sequences. The theoretical analysis reveals two distinct scaling regimes: sublinear scaling for less vision tokens and linear scaling for more vision tokens. This aligns with model performance relationships of the form (S(n) \approx c / n^{\alpha(n)}), where the scaling exponent relates to the correlation structure between vision token representations. Empirical validations across multiple vision-language benchmarks show that model performance matches the prediction from scaling relationship. The findings contribute to understanding vision token scaling in transformers through a theoretical framework that complements empirical observations.

Method: PRISM uses structured schemas for in-context learning, efficiently leveraging key-value caches to process information with much shorter contexts while maintaining performance.

Result: Outperforms baselines on diverse tasks with 4x shorter contexts, reduces costs by up to 54%, scales to tiny contexts without quality loss, and generalizes to new tasks with minimal effort.

Conclusion: PRISM provides an efficient and effective solution for long-range reasoning tasks by combining structured schemas with in-context learning, addressing compute, data, and design limitations of existing approaches.

Abstract: Long-range tasks demand reasoning over long inputs. However, existing solutions are limited, e.g., long-context models require large compute budgets, parameter-efficient fine-tuning (PEFT) needs training data, and retrieval-augmented generation (RAG) entails complex task-specific designs. Though in-context approaches overcome many of these issues, methods with short-context LLMs are inefficient, trading context for processing more tokens. We introduce PRISM, a highly token-efficient in-context method based on structured schemas that outperforms baselines on diverse tasks with 4x shorter contexts. This approach produces concise outputs and efficiently leverages key-value (KV) caches to reduce costs by up to 54%. PRISM scales down to tiny contexts without increasing costs or sacrificing quality, and generalizes to new tasks with minimal effort by generating schemas from task descriptions.

Method: The framework systematically varies action spaces, state complexity, and number of interacting agents to create diverse social settings. It includes both benchmark tasks and an arena for simulating various social scenarios.

Result: Contemporary LLMs perform well on basic fact retrieval and short-range planning but show significant bottlenecks in deep multi-hop reasoning over large state spaces and socially adept coordination under uncertainty.

Conclusion: SPIN-Bench serves as a catalyst for future research on robust multi-agent planning, social reasoning, and human-AI teaming, addressing gaps in current AI evaluation methodologies.

Abstract: Reasoning and strategic behavior in social interactions is a hallmark of intelligence. This form of reasoning is significantly more sophisticated than isolated planning or reasoning tasks in static settings (e.g., math problem solving). In this paper, we present Strategic Planning, Interaction, and Negotiation (SPIN-Bench), a new multi-domain evaluation designed to measure the intelligence of strategic planning and social reasoning. While many existing benchmarks focus on narrow planning or single-agent reasoning, SPIN-Bench combines classical PDDL tasks, competitive board games, cooperative card games, and multi-agent negotiation scenarios in one unified framework. The framework includes both a benchmark as well as an arena to simulate and evaluate the variety of social settings to test reasoning and strategic behavior of AI agents. We formulate the benchmark SPIN-Bench by systematically varying action spaces, state complexity, and the number of interacting agents to simulate a variety of social settings where success depends on not only methodical and step-wise decision making, but also conceptual inference of other (adversarial or cooperative) participants. Our experiments reveal that while contemporary LLMs handle basic fact retrieval and short-range planning reasonably well, they encounter significant performance bottlenecks in tasks requiring deep multi-hop reasoning over large state spaces and socially adept coordination under uncertainty. We envision SPIN-Bench as a catalyst for future research on robust multi-agent planning, social reasoning, and human–AI teaming. Project Website: https://spinbench.github.io/

Method: Proposes Task Memory Engine (TME) with hierarchical Task Memory Tree where each node represents a task step, storing inputs/outputs/status. Uses prompt synthesis to dynamically generate LLM prompts based on active node path.

Result: Demonstrates better task completion accuracy and more interpretable behavior in multi-step agent tasks with minimal implementation overhead.

Conclusion: TME provides effective structured memory for LLM agents, laying groundwork for future DAG-based memory architectures while maintaining lightweight implementation.

Abstract: Large Language Models (LLMs) are increasingly used as autonomous agents for multi-step tasks. However, most existing frameworks fail to maintain a structured understanding of the task state, often relying on linear prompt concatenation or shallow memory buffers. This leads to brittle performance, frequent hallucinations, and poor long-range coherence. In this work, we propose the Task Memory Engine (TME), a lightweight and structured memory module that tracks task execution using a hierarchical Task Memory Tree (TMT). Each node in the tree corresponds to a task step, storing relevant input, output, status, and sub-task relationships. We introduce a prompt synthesis method that dynamically generates LLM prompts based on the active node path, significantly improving execution consistency and contextual grounding. Through case studies and comparative experiments on multi-step agent tasks, we demonstrate that TME leads to better task completion accuracy and more interpretable behavior with minimal implementation overhead. A reference implementation of the core TME components is available at https://github.com/biubiutomato/TME-Agent, including basic examples and structured memory integration. While the current implementation uses a tree-based structure, TME is designed to be graph-aware, supporting reusable substeps, converging task paths, and shared dependencies. This lays the groundwork for future DAG-based memory architectures.

Method: The paper provides an overview of current knowledge about LLMs’ metacognitive abilities, compares them with human metacognition, and discusses how these capacities might be studied.

Result: While humans and LLMs sometimes show alignment in metacognitive behaviors, significant differences remain that are important for enhancing human-AI collaboration.

Conclusion: Endowing future LLMs with more sensitive and calibrated metacognition could help them develop new capacities like efficient learning, self-direction, and curiosity, improving their overall functionality and collaboration with humans.

Abstract: Metacognition–the capacity to monitor and evaluate one’s own knowledge and performance–is foundational to human decision-making, learning, and communication. As large language models (LLMs) become increasingly embedded in both high-stakes and widespread low-stakes contexts, it is important to assess whether, how, and to what extent they exhibit metacognitive abilities. Here, we provide an overview of current knowledge of LLMs’ metacognitive capacities, how they might be studied, and how they relate to our knowledge of metacognition in humans. We show that while humans and LLMs can sometimes appear quite aligned in their metacognitive capacities and behaviors, it is clear many differences remain; attending to these differences is important for enhancing human-AI collaboration. Finally, we discuss how endowing future LLMs with more sensitive and more calibrated metacognition may also help them develop new capacities such as more efficient learning, self-direction, and curiosity.

Method: Uses generative AI to automatically write chemical classifier programs for ChEBI database classes. These programs provide deterministic classification of SMILES structures with natural language explanations.

Result: C3PO outperforms naive SMARTS pattern classifiers but doesn’t reach state-of-the-art deep learning performance. However, it offers explainability and reduced data dependence, making it complementary to deep learning methods.

Conclusion: C3PO provides an explainable, computable ontological model for chemical classification that can be used alongside deep learning classifiers and refined by human experts, offering a transparent alternative to black-box deep learning approaches.

Abstract: Accurately classifying chemical structures is essential for cheminformatics and bioinformatics, including tasks such as identifying bioactive compounds of interest, screening molecules for toxicity to humans, finding non-organic compounds with desirable material properties, or organizing large chemical libraries for drug discovery or environmental monitoring. However, manual classification is labor-intensive and difficult to scale to large chemical databases. Existing automated approaches either rely on manually constructed classification rules, or are deep learning methods that lack explainability. This work presents an approach that uses generative artificial intelligence to automatically write chemical classifier programs for classes in the Chemical Entities of Biological Interest (ChEBI) database. These programs can be used for efficient deterministic run-time classification of SMILES structures, with natural language explanations. The programs themselves constitute an explainable computable ontological model of chemical class nomenclature, which we call the ChEBI Chemical Class Program Ontology (C3PO). We validated our approach against the ChEBI database, and compared our results against deep learning models and a naive SMARTS pattern based classifier. C3PO outperforms the naive classifier, but does not reach the performance of state of the art deep learning methods. However, C3PO has a number of strengths that complement deep learning methods, including explainability and reduced data dependence. C3PO can be used alongside deep learning classifiers to provide an explanation of the classification, where both methods agree. The programs can be used as part of the ontology development process, and iteratively refined by expert human curators.

Method: Philosophy-informed approach using Force concept to represent temporal feature trajectories with inertia; includes pre-registered human experiment and computational pipelines for individual/group fairness in criminal justice and finance contexts.

Result: Human experiment shows people prioritize temporal trajectories over aggregate feature values in fairness evaluation; developed pipelines enable computation of effort-aware fairness metrics.

Conclusion: EaF enables AI auditors to identify and potentially correct unfair decisions against individuals who made significant efforts but remain systemically disadvantaged, bridging philosophical fairness concepts with AI practice.

Abstract: Although popularized AI fairness metrics, e.g., demographic parity, have uncovered bias in AI-assisted decision-making outcomes, they do not consider how much effort one has spent to get to where one is today in the input feature space. However, the notion of effort is important in how Philosophy and humans understand fairness. We propose a philosophy-informed approach to conceptualize and evaluate Effort-aware Fairness (EaF), grounded in the concept of Force, which represents the temporal trajectory of predictive features coupled with inertia. Besides theoretical formulation, our empirical contributions include: (1) a pre-registered human subjects experiment, which shows that for both stages of the (individual) fairness evaluation process, people consider the temporal trajectory of a predictive feature more than its aggregate value; (2) pipelines to compute Effort-aware Individual/Group Fairness in the criminal justice and personal finance contexts. Our work may enable AI model auditors to uncover and potentially correct unfair decisions against individuals who have spent significant efforts to improve but are still stuck with systemic disadvantages outside their control.

Method: Created Jigsaw-Puzzles benchmark with 1,100 real-world images of high spatial complexity, designed 5 tasks to evaluate spatial perception, structural understanding, and reasoning while minimizing domain knowledge reliance.

Result: Even the strongest model (Gemini-2.5-Pro) achieved only 77.14% overall accuracy, with particularly poor performance on Order Generation task (30.00% accuracy), far below human performance exceeding 90%.

Conclusion: There’s a significant gap between VLMs and human spatial reasoning capabilities, positioning Jigsaw-Puzzles as a challenging benchmark for advancing spatial reasoning research in vision-language models.

Abstract: Spatial reasoning is a core component of human cognition, enabling individuals to perceive, comprehend, and interact with the physical world. It relies on a nuanced understanding of spatial structures and inter-object relationships, serving as the foundation for complex reasoning and decision-making. To investigate whether current vision-language models (VLMs) exhibit similar capability, we introduce Jigsaw-Puzzles, a novel benchmark consisting of 1,100 carefully curated real-world images with high spatial complexity. Based on this dataset, we design five tasks to rigorously evaluate VLMs’ spatial perception, structural understanding, and reasoning capabilities, while deliberately minimizing reliance on domain-specific knowledge to better isolate and assess the general spatial reasoning capability. We conduct a comprehensive evaluation across 24 state-of-the-art VLMs. The results show that even the strongest model, Gemini-2.5-Pro, achieves only 77.14% overall accuracy and performs particularly poorly on the Order Generation task, with only 30.00% accuracy, far below the performance exceeding 90% achieved by human participants. This persistent gap underscores the need for continued progress, positioning Jigsaw-Puzzles as a challenging and diagnostic benchmark for advancing spatial reasoning research in VLMs. Our project page is at https://zesen01.github.io/jigsaw-puzzles.

Method: The paper uses Peirce’s framework of abduction, deduction, and induction as a structured lens to examine LLM-based hypothesis discovery. It synthesizes existing work in hypothesis generation, application, and validation.

Result: The survey identifies both key achievements and critical gaps in current LLM capabilities for knowledge discovery, providing a comprehensive analysis of how these models might evolve.

Conclusion: By unifying various threads of research, the paper illuminates how LLMs could potentially transform from mere information executors into engines of genuine innovation, with significant implications for research, science, and real-world problem solving.

Abstract: Since the advent of Large Language Models (LLMs), efforts have largely focused on improving their instruction-following and deductive reasoning abilities, leaving open the question of whether these models can truly discover new knowledge. In pursuit of artificial general intelligence (AGI), there is a growing need for models that not only execute commands or retrieve information but also learn, reason, and generate new knowledge by formulating novel hypotheses and theories that deepen our understanding of the world. Guided by Peirce’s framework of abduction, deduction, and induction, this survey offers a structured lens to examine LLM-based hypothesis discovery. We synthesize existing work in hypothesis generation, application, and validation, identifying both key achievements and critical gaps. By unifying these threads, we illuminate how LLMs might evolve from mere ``information executors’’ into engines of genuine innovation, potentially transforming research, science, and real-world problem solving.

Method: An integrated framework built on three pillars: domain knowledge for meaningful objectives/constraints, physics-informed ML for better generalization with limited data, and LLM-based interfaces for intuitive human interaction.

Result: Using injection molding as an example, the authors demonstrate how these components work together in practice to address manufacturing design challenges.

Conclusion: The paper highlights key challenges for applying such integrated AI approaches in realistic manufacturing environments and provides a practical framework for implementation.

Abstract: Artificial intelligence (AI) is reshaping inverse design in manufacturing, enabling high-performance discovery in materials, products, and processes. However, purely data-driven approaches often struggle in realistic manufacturing settings characterized by sparse data, high-dimensional design spaces, and complex constraints. This perspective proposes an integrated framework built on three complementary pillars: domain knowledge to establish physically meaningful objectives and constraints while removing variables with limited relevance, physics-informed machine learning to enhance generalization under limited or biased data, and large language model-based interfaces to support intuitive, human-centered interaction. Using injection molding as an illustrative example, we demonstrate how these components can operate in practice and conclude by highlighting key challenges for applying such approaches in realistic manufacturing environments.

Method: Used a curated diagnostic suite spanning four categories of real-world failure modes to evaluate six MLLMs (including o3 and GPT-4o), tested explicit prompting to reveal underlying capabilities, and implemented inference-time interventions like cautious persona prompting and requiring clarifying questions.

Result: Models frequently failed to surface hidden issues even when possessing necessary perceptual and reasoning skills. Explicit prompting showed underlying capabilities exist but are suppressed for user compliance. Simple interventions, especially requiring clarifying questions, dramatically recovered performance.

Conclusion: There’s a persistent gap between reasoning competence and behavioral compliance in current MLLMs. Practical strategies like cautious persona prompting and requiring clarifying questions can make these models more trustworthy in underconstrained environments.

Abstract: Multimodal large language models (MLLMs) are increasingly deployed in open-ended, real-world environments where inputs are messy, underspecified, and not always trustworthy. Unlike curated benchmarks, these settings frequently involve instructions that refer to missing objects or contradictory facts, rely on ambiguous references, or request infeasible actions. In such cases, success hinges not on task execution alone, but on a model’s ability to detect when something is silently wrong. This paper presents a systematic analysis of how current MLLMs handle such implicit reasoning scenarios: cases where the flaw is not explicitly stated but must be inferred from context. Using a curated diagnostic suite spanning four categories of real-world failure modes, we evaluate six MLLMs, including o3 and GPT-4o, and find that models frequently fail to surface hidden issues, even when they possess the necessary perceptual and reasoning skills. Explicit prompting reveals that the underlying capabilities exist but are often suppressed in favor of user compliance. We further show that simple inference-time interventions, such as cautious persona prompting and, in particular, requiring a clarifying question, can dramatically recover performance. Our findings highlight a persistent gap between reasoning competence and behavioral compliance in current MLLMs and suggest practical strategies for making these models more trustworthy in underconstrained environments.

Method: STORM framework models asymmetric information dynamics through conversations between UserLLM (full internal access) and AgentLLM (observable behavior only), producing annotated corpora that capture expression phrasing trajectories and latent cognitive transitions.

Result: Experiments across four language models show that moderate uncertainty (40-60%) can outperform complete transparency in certain scenarios, with model-specific patterns suggesting reconsideration of optimal information completeness in human-AI collaboration.

Conclusion: The findings contribute to understanding asymmetric reasoning dynamics and inform uncertainty-calibrated dialogue system design, providing metrics to measure internal cognitive improvements alongside task performance.

Abstract: Dialogue systems often fail when user utterances are semantically complete yet lack the clarity and completeness required for appropriate system action. This mismatch arises because users frequently do not fully understand their own needs, while systems require precise intent definitions. This highlights the critical Intent-Action Alignment Problem: determining when an expression is not just understood, but truly ready for a system to act upon. We present STORM, a framework modeling asymmetric information dynamics through conversations between UserLLM (full internal access) and AgentLLM (observable behavior only). STORM produces annotated corpora capturing trajectories of expression phrasing and latent cognitive transitions, enabling systematic analysis of how collaborative understanding develops. Our contributions include: (1) formalizing asymmetric information processing in dialogue systems; (2) modeling intent formation tracking collaborative understanding evolution; and (3) evaluation metrics measuring internal cognitive improvements alongside task performance. Experiments across four language models reveal that moderate uncertainty (40-60%) can outperform complete transparency in certain scenarios, with model-specific patterns suggesting reconsideration of optimal information completeness in human-AI collaboration. These findings contribute to understanding asymmetric reasoning dynamics and inform uncertainty-calibrated dialogue system design.

Method: Created MMTU benchmark with over 30,000 questions across 25 diverse table tasks drawn from decades of computer science research on tabular data, focusing on complex tasks encountered by professional users.

Result: Frontier models like OpenAI o4-mini and DeepSeek R1 score only around 60% on MMTU, showing these tasks require a challenging combination of table understanding, reasoning, and coding skills.

Conclusion: MMTU reveals significant room for improvement in table-related capabilities of current LLMs and serves as a comprehensive benchmark to drive advances in foundation models for structured data processing and analysis.

Abstract: Tables and table-based use cases play a crucial role in many important real-world applications, such as spreadsheets, databases, and computational notebooks, which traditionally require expert-level users like data engineers, data analysts, and database administrators to operate. Although LLMs have shown remarkable progress in working with tables (e.g., in spreadsheet and database copilot scenarios), comprehensive benchmarking of such capabilities remains limited. In contrast to an extensive and growing list of NLP benchmarks, evaluations of table-related tasks are scarce, and narrowly focus on tasks like NL-to-SQL and Table-QA, overlooking the broader spectrum of real-world tasks that professional users face. This gap limits our understanding and model progress in this important area. In this work, we introduce MMTU, a large-scale benchmark with over 30K questions across 25 real-world table tasks, designed to comprehensively evaluate models ability to understand, reason, and manipulate real tables at the expert-level. These tasks are drawn from decades’ worth of computer science research on tabular data, with a focus on complex table tasks faced by professional users. We show that MMTU require a combination of skills – including table understanding, reasoning, and coding – that remain challenging for today’s frontier models, where even frontier reasoning models like OpenAI o4-mini and DeepSeek R1 score only around 60%, suggesting significant room for improvement. We highlight key findings in our evaluation using MMTU and hope that this benchmark drives further advances in understanding and developing foundation models for structured data processing and analysis. Our code and data are available at https://github.com/MMTU-Benchmark/MMTU and https://huggingface.co/datasets/MMTU-benchmark/MMTU.

Method: Constructed a multimodal knowledge graph (MH-MMKG) for Monster Hunter: World with multi-modalities and intricate entity relations. Designed challenging queries for evaluation and proposed a multi-agent retriever for autonomous knowledge search without additional training.

Result: Experimental results show the approach significantly enhances MLLMs’ performance on complex knowledge retrieval and reasoning tasks.

Conclusion: The work provides a new perspective on multimodal knowledge-augmented reasoning and lays a solid foundation for future research in enhancing MLLMs’ domain-specific capabilities.

Abstract: The real value of knowledge lies not just in its accumulation, but in its potential to be harnessed effectively to conquer the unknown. Although recent multimodal large language models (MLLMs) exhibit impressing multimodal capabilities, they often fail in rarely encountered domain-specific tasks due to limited relevant knowledge. To explore this, we adopt visual game cognition as a testbed and select Monster Hunter: World as the target to construct a multimodal knowledge graph (MH-MMKG), which incorporates multi-modalities and intricate entity relations. We also design a series of challenging queries based on MH-MMKG to evaluate the models’ ability for complex knowledge retrieval and reasoning. Furthermore, we propose a multi-agent retriever that enables a model to autonomously search relevant knowledge without additional training. Experimental results show that our approach significantly enhances the performance of MLLMs, providing a new perspective on multimodal knowledge-augmented reasoning and laying a solid foundation for future research.

Method: Tested 7 vision-language models across 6 configurations: baseline, fine-tuned, and models augmented with reasoning layers (peer consultation simulation) or retrieval-augmented generation (medical literature reference).

Result: Fine-tuning degraded performance in 4/7 models (30% average decrease), baseline models collapsed on test data. Clinical-inspired architectures achieved up to 70% accuracy with maintained performance on unseen data.

Conclusion: Medical AI succeeds by reconstructing collaborative and evidence-based clinical practices rather than relying solely on domain-specific fine-tuning.

Abstract: Dermatological care via telemedicine often lacks the rich context of in-person visits. Clinicians must make diagnoses based on a handful of images and brief descriptions, without the benefit of physical exams, second opinions, or reference materials. While many medical AI systems attempt to bridge these gaps with domain-specific fine-tuning, this work hypothesized that mimicking clinical reasoning processes could offer a more effective path forward. This study tested seven vision-language models on medical visual question answering across six configurations: baseline models, fine-tuned variants, and both augmented with either reasoning layers that combine multiple model perspectives, analogous to peer consultation, or retrieval-augmented generation that incorporates medical literature at inference time, serving a role similar to reference-checking. While fine-tuning degraded performance in four of seven models with an average 30% decrease, baseline models collapsed on test data. Clinical-inspired architectures, meanwhile, achieved up to 70% accuracy, maintaining performance on unseen data while generating explainable, literature-grounded outputs critical for clinical adoption. These findings demonstrate that medical AI succeeds by reconstructing the collaborative and evidence-based practices fundamental to clinical diagnosis.

Method: Analyzed over 12,000 actions across five state-of-the-art models (GPT-4o, GPT-4.1, Claude variants) on 93 real-world software setup tasks, developing a high-precision detection system for vulnerability identification.

Result: 21% of agent trajectories contained insecure actions with substantial variation between models. Information exposure (CWE-200) was most prevalent. GPT-4.1 showed exceptional security awareness with 96.8% mitigation success.

Conclusion: Autonomous coding agents pose significant security risks that require systematic evaluation and mitigation strategies, with model-specific security awareness varying dramatically.

Abstract: LLM-based coding agents are rapidly being deployed in software development, yet their safety implications remain poorly understood. These agents, while capable of accelerating software development, may exhibit unsafe behaviors during normal operation that manifest as cybersecurity vulnerabilities. We conducted the first systematic safety evaluation of autonomous coding agents, analyzing over 12,000 actions across five state-of-the-art models (GPT-4o, GPT-4.1, Claude variants) on 93 real-world software setup tasks. Our findings reveal significant security concerns: 21% of agent trajectories contained insecure actions, with models showing substantial variation in unsafe behavior. We developed a high-precision detection system that identified four major vulnerability categories, with information exposure (CWE-200) being the most prevalent one. We also evaluated mitigation strategies including feedback mechanisms and security reminders with various effectiveness between models. GPT-4.1 demonstrated exceptional security awareness with 96.8% mitigation success.

Method: Train Reinforcement Learning agents to navigate cluttered tunnels using only optic flow as sensory input, then analyze their attention patterns to understand decision-making regions.

Result: Trained agents primarily focus on regions with optic flow discontinuities and large magnitude, avoiding obstacles while maintaining centered position - behavior resembling flying insects. This pattern is consistent across independently trained agents.

Conclusion: The discovered attention pattern could serve as a basis for developing simple explicit control laws for physical UAVs, providing a bio-inspired navigation strategy that mimics efficient insect flight behavior.

Abstract: Bio-inspired design is often used in autonomous UAV navigation due to the capacity of biological systems for flight and obstacle avoidance despite limited sensory and computational capabilities. In particular, honeybees mainly use the sensory input of optic flow, the apparent motion of objects in their visual field, to navigate cluttered environments. In our work, we train a Reinforcement Learning agent to navigate a tunnel with obstacles using only optic flow as sensory input. We inspect the attention patterns of trained agents to determine the regions of optic flow on which they primarily base their motor decisions. We find that agents trained in this way pay most attention to regions of discontinuity in optic flow, as well as regions with large optic flow magnitude. The trained agents appear to navigate a cluttered tunnel by avoiding the obstacles that produce large optic flow, while maintaining a centered position in their environment, which resembles the behavior seen in flying insects. This pattern persists across independently trained agents, which suggests that this could be a good strategy for developing a simple explicit control law for physical UAVs.

Method: The paper analyzes why relational learning (statistical relational AI) hasn’t achieved widespread adoption despite its potential, examining the challenges and limitations in current approaches.

Result: Relational learning is not taking over the world except in limited cases with restricted relations, indicating significant barriers to broader adoption.

Conclusion: Specific actions and improvements are needed to elevate relational learning to its rightful prominence in AI, moving beyond perception-based modeling to true entity-relation modeling.

Abstract: Artificial intelligence seems to be taking over the world with systems that model pixels, words, and phonemes. The world is arguably made up, not of pixels, words, and phonemes but of entities (objects, things, including events) with properties and relations among them. Surely we should model these, not the perception or description of them. You might suspect that concentrating on modeling words and pixels is because all of the (valuable) data in the world is in terms of text and images. If you look into almost any company you will find their most valuable data is in spreadsheets, databases and other relational formats. These are not the form that are studied in introductory machine learning, but are full of product numbers, student numbers, transaction numbers and other identifiers that can’t be interpreted naively as numbers. The field that studies this sort of data has various names including relational learning, statistical relational AI, and many others. This paper explains why relational learning is not taking over the world – except in a few cases with restricted relations – and what needs to be done to bring it to it’s rightful prominence.

Method: Uses electrodermal activity signals as input, generates multiple signal representations visualized as waveforms, and presents them in unified multi-representation diagrams. Employs diverse processing and filtering techniques with various representation combinations.

Result: The approach consistently achieves comparable and in several cases superior results to traditional fusion methods. Extensive experiments demonstrate its effectiveness across different processing techniques and representation combinations.

Conclusion: The proposed pipeline serves as a robust alternative for integrating different signal representations or modalities in pain assessment systems, positioning it as an effective solution for next-generation pain monitoring.

Abstract: Pain is a multifaceted phenomenon that affects a substantial portion of the population. Reliable and consistent evaluation supports individuals experiencing pain and enables the development of effective and advanced management strategies. Automatic pain-assessment systems provide continuous monitoring, guide clinical decision-making, and aim to reduce distress while preventing functional decline. Incorporating physiological signals allows these systems to deliver objective, accurate insights into an individual’s condition. This study has been submitted to the Second Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed method introduces a pipeline that employs electrodermal activity signals as the input modality. Multiple signal representations are generated and visualized as waveforms, which are then jointly presented within a unified multi-representation diagram. Extensive experiments using diverse processing and filtering techniques, along with various representation combinations, highlight the effectiveness of the approach. It consistently achieves comparable and, in several cases, superior results to traditional fusion methods, positioning it as a robust alternative for integrating different signal representations or modalities.

Method: The proposed method uses respiration as input signal and integrates a cross-attention transformer with a multi-windowing strategy to capture both short-term and long-term features along with global characteristics.

Result: Extensive experiments demonstrate that respiration is a valuable physiological modality for pain assessment. Compact and efficient models, when properly optimized, can deliver strong performance and often surpass larger models.

Conclusion: The multi-window strategy effectively enhances the model’s representational capacity, making respiration-based assessment a promising approach for next-generation pain monitoring systems.

Abstract: Pain is a complex condition that affects a large portion of the population. Accurate and consistent evaluation is essential for individuals experiencing pain and supports the development of effective and advanced management strategies. Automatic pain assessment systems provide continuous monitoring, aid clinical decision-making, and aim to reduce distress while preventing functional decline. This study has been submitted to the Second Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN). The proposed method introduces a pipeline that employs respiration as the input signal and integrates a highly efficient cross-attention transformer with a multi-windowing strategy. Extensive experiments demonstrate that respiration serves as a valuable physiological modality for pain assessment. Furthermore, results show that compact and efficient models, when properly optimized, can deliver strong performance, often surpassing larger counterparts. The proposed multi-window strategy effectively captures short-term and long-term features, along with global characteristics, enhancing the model’s representational capacity.

Method: Formally defined argumentative coherence property, conducted three evaluations: impact on human and LLM forecasters, and crowd-sourced user experiments to assess alignment with coherence.

Result: Filtering incoherent predictions consistently improved forecasting accuracy for both humans and LLMs. However, users don’t naturally apply coherence filtering despite its usefulness.

Conclusion: Argumentation-based judgmental forecasting needs mechanisms to filter out incoherent opinions before obtaining group predictions, as coherence improves accuracy but isn’t naturally applied by users.

Abstract: Judgmental forecasting employs human opinions to make predictions about future events, rather than exclusively historical data as in quantitative forecasting. When these opinions form an argumentative structure around forecasts, it is useful to study the properties of the forecasts from an argumentative perspective. In this paper, we advocate and formally define a property of argumentative coherence, which, in essence, requires that a forecaster’s reasoning is coherent with their forecast. We then conduct three evaluations with our notion of coherence. First, we assess the impact of enforcing coherence on human forecasters as well as on Large Language Model (LLM)-based forecasters, given that they have recently shown to be competitive with human forecasters. In both cases, we show that filtering out incoherent predictions improves forecasting accuracy consistently, supporting the practical value of coherence in both human and LLM-based forecasting. Then, via crowd-sourced user experiments, we show that, despite its apparent intuitiveness and usefulness, users do not generally align with this coherence property. This points to the need to integrate, within argumentation-based judgmental forecasting, mechanisms to filter out incoherent opinions before obtaining group forecasting predictions.

Method: Proposes SE-Agent framework with three key operations: revision (improving existing trajectories), recombination (combining elements from different trajectories), and refinement (polishing solutions). This evolutionary approach expands search space and leverages cross-trajectory inspiration.

Result: Achieves up to 55% relative improvement on SWE-bench Verified for resolving real-world GitHub issues across five strong LLMs, achieving state-of-the-art performance among all open-source agents.

Conclusion: SE-Agent’s self-evolution framework effectively optimizes reasoning processes by intelligently exploring diverse solution paths and leveraging cross-trajectory insights, significantly improving problem-solving capabilities of LLM-based agents.

Abstract: Large Language Model (LLM)-based agents have recently shown impressive capabilities in complex reasoning and tool use via multi-step interactions with their environments. While these agents have the potential to tackle complicated tasks, their problem-solving process, i.e., agents’ interaction trajectory leading to task completion, remains underexploited. These trajectories contain rich feedback that can navigate agents toward the right directions for solving problems correctly. Although prevailing approaches, such as Monte Carlo Tree Search (MCTS), can effectively balance exploration and exploitation, they ignore the interdependence among various trajectories and lack the diversity of search spaces, which leads to redundant reasoning and suboptimal outcomes. To address these challenges, we propose SE-Agent, a Self-Evolution framework that enables Agents to optimize their reasoning processes iteratively. Our approach revisits and enhances former pilot trajectories through three key operations: revision, recombination, and refinement. This evolutionary mechanism enables two critical advantages: (1) it expands the search space beyond local optima by intelligently exploring diverse solution paths guided by previous trajectories, and (2) it leverages cross-trajectory inspiration to efficiently enhance performance while mitigating the impact of suboptimal reasoning paths. Through these mechanisms, SE-Agent achieves continuous self-evolution that incrementally improves reasoning quality. We evaluate SE-Agent on SWE-bench Verified to resolve real-world GitHub issues. Experimental results across five strong LLMs show that integrating SE-Agent delivers up to 55% relative improvement, achieving state-of-the-art performance among all open-source agents on SWE-bench Verified. Our code and demonstration materials are publicly available at https://github.com/JARVIS-Xs/SE-Agent.

Method: Trained on over 400 million simulated days of outbreak dynamics spanning diverse pathogens, transmission modes, interventions, and surveillance artifacts without using any real-world data during training.

Result: Outperformed 39 expert-tuned models across six diseases, including all models in CDC’s COVID-19 Forecast Hub. Generalized to novel epidemiological regimes and delivered accurate forecasts at 8-week horizons (more than doubling actionable range of most models).

Conclusion: Mantis serves as a foundation for next-generation disease forecasting systems that are general, interpretable, and deployable where traditional models fail, capturing fundamental contagion dynamics through mechanistic simulation training.

Abstract: Infectious disease forecasting in novel outbreaks or low resource settings has been limited by the need for disease-specific data, bespoke training, and expert tuning. We introduce Mantis, a foundation model trained entirely on mechanistic simulations, which enables out-of-the-box forecasting across diseases, regions, and outcomes, even in settings with limited historical data. Mantis is built on over 400 million simulated days of outbreak dynamics spanning diverse pathogens, transmission modes, interventions, and surveillance artifacts. Despite requiring no real-world data during training, Mantis outperformed 39 expert-tuned models we tested across six diseases, including all models in the CDC’s COVID-19 Forecast Hub. Mantis generalized to novel epidemiological regimes, including diseases with held-out transmission mechanisms, demonstrating that it captures fundamental contagion dynamics. Critically, Mantis is mechanistically interpretable, enabling public health decision-makers to identify the latent drivers behind its predictions. Finally, Mantis delivers accurate forecasts at 8-week horizons, more than doubling the actionable range of most models, enabling proactive public health planning. Together, these capabilities position Mantis as a foundation for next-generation disease forecasting systems: general, interpretable, and deployable where traditional models fail.

Method: Created EgoIllusion benchmark with 1,400 egocentric videos paired with 8,000 human-annotated open and closed-ended questions designed to trigger hallucinations in both visual and auditory modalities.

Result: Evaluation of ten MLLMs revealed significant challenges, with even powerful models like GPT-4o and Gemini achieving only 59% accuracy, demonstrating widespread hallucination issues.

Conclusion: EgoIllusion provides a foundational benchmark for evaluating MLLM effectiveness in egocentric contexts and will spur development of better models with reduced hallucination rates. The benchmark will be open-sourced for reproducibility.

Abstract: Multimodal Large Language Models (MLLMs) have demonstrated remarkable performance in complex multimodal tasks. While MLLMs excel at visual perception and reasoning in third-person and egocentric videos, they are prone to hallucinations, generating coherent yet inaccurate responses. We present EgoIllusion, a first benchmark to evaluate MLLM hallucinations in egocentric videos. EgoIllusion comprises 1,400 videos paired with 8,000 human-annotated open and closed-ended questions designed to trigger hallucinations in both visual and auditory cues in egocentric videos. Evaluations across ten MLLMs reveal significant challenges, including powerful models like GPT-4o and Gemini, achieving only 59% accuracy. EgoIllusion lays the foundation in developing robust benchmarks to evaluate the effectiveness of MLLMs and spurs the development of better egocentric MLLMs with reduced hallucination rates. Our benchmark will be open-sourced for reproducibility.

Method: Three key innovations: (1) parallelized heuristic extraction with weak data dependence for concurrent DAG cost computation, (2) adaptive search space pruning with parameterized threshold to retain promising candidates, and (3) initialized exact solving using Integer Linear Programming with warm-start capabilities.

Result: 558x runtime speedup over traditional exact approaches (ILP), 19.04% performance improvement over state-of-the-art framework (SmoothE), and 7.6-8.1% area improvements in logic synthesis tasks with different technology mapping libraries.

Conclusion: E-boost effectively bridges the performance gap between heuristic and exact e-graph extraction methods, delivering both speed and optimality for practical optimization tasks in formal verification and logic synthesis.

Abstract: E-graphs have attracted growing interest in many fields, particularly in logic synthesis and formal verification. E-graph extraction is a challenging NP-hard combinatorial optimization problem. It requires identifying optimal terms from exponentially many equivalent expressions, serving as the primary performance bottleneck in e-graph based optimization tasks. However, traditional extraction methods face a critical trade-off: heuristic approaches offer speed but sacrifice optimality, while exact methods provide optimal solutions but face prohibitive computational costs on practical problems. We present e-boost, a novel framework that bridges this gap through three key innovations: (1) parallelized heuristic extraction that leverages weak data dependence to compute DAG costs concurrently, enabling efficient multi-threaded performance without sacrificing extraction quality; (2) adaptive search space pruning that employs a parameterized threshold mechanism to retain only promising candidates, dramatically reducing the solution space while preserving near-optimal solutions; and (3) initialized exact solving that formulates the reduced problem as an Integer Linear Program with warm-start capabilities, guiding solvers toward high-quality solutions faster. Across the diverse benchmarks in formal verification and logic synthesis fields, e-boost demonstrates 558x runtime speedup over traditional exact approaches (ILP) and 19.04% performance improvement over the state-of-the-art extraction framework (SmoothE). In realistic logic synthesis tasks, e-boost produces 7.6% and 8.1% area improvements compared to conventional synthesis tools with two different technology mapping libraries. e-boost is available at https://github.com/Yu-Maryland/e-boost.

Method: Three-step approach: (1) encode seed puzzles into structured logical specifications using SMT, (2) generate scalable variants through systematic variable and constraint randomization, (3) ensure validity via reproduction mechanism.

Result: Created benchmark with 83K+ validated puzzles. Post-training improved PuzzleClone average from 14.4 to 56.2 and achieved up to 12.5% absolute improvement on 7 logic/math benchmarks (e.g., AMC2023 from 52.5 to 65.0).

Conclusion: PuzzleClone effectively addresses dataset limitations and significantly enhances LLM reasoning capabilities through scalable, verifiable puzzle generation and training.

Abstract: High-quality mathematical and logical datasets with verifiable answers are essential for strengthening the reasoning capabilities of large language models (LLMs). While recent data augmentation techniques have facilitated the creation of large-scale benchmarks, existing LLM-generated datasets often suffer from limited reliability, diversity, and scalability. To address these challenges, we introduce PuzzleClone, a formal framework for synthesizing verifiable data at scale using Satisfiability Modulo Theories (SMT). Our approach features three key innovations: (1) encoding seed puzzles into structured logical specifications, (2) generating scalable variants through systematic variable and constraint randomization, and (3) ensuring validity via a reproduction mechanism. Applying PuzzleClone, we construct a curated benchmark comprising over 83K diverse and programmatically validated puzzles. The generated puzzles span a wide spectrum of difficulty and formats, posing significant challenges to current state-of-the-art models. We conduct post training (SFT and RL) on PuzzleClone datasets. Experimental results show that training on PuzzleClone yields substantial improvements not only on PuzzleClone testset but also on logic and mathematical benchmarks. Post training raises PuzzleClone average from 14.4 to 56.2 and delivers consistent improvements across 7 logic and mathematical benchmarks up to 12.5 absolute percentage points (AMC2023 from 52.5 to 65.0). Our code and data are available at https://github.com/HiThink-Research/PuzzleClone.

Method: Developed multiple optimization algorithms including CR+DES (differential evolution with scaled difference vectors) and MSBX+MO, with systematic constraint relaxation strategy and statistical validation using Kruskal-Wallis tests.

Result: CR+DES achieved 3.16% improvement in land-use compatibility, MSBX+MO achieved 3.3% improvement in price optimization. Statistical analysis confirmed algorithms with difference vectors significantly outperform traditional approaches.

Conclusion: The constraint relaxation technique enables broader solution space exploration while maintaining practical constraints, providing urban planners with evidence-based computational tools for effective land-use allocation in rapidly urbanizing regions.

Abstract: Urban land-use allocation represents a complex multi-objective optimization problem critical for sustainable urban development policy. This paper presents novel computational intelligence approaches for optimizing land-use allocation in mixed-use areas, addressing inherent trade-offs between land-use compatibility and economic objectives. We develop multiple optimization algorithms, including custom variants integrating differential evolution with multi-objective genetic algorithms. Key contributions include: (1) CR+DES algorithm leveraging scaled difference vectors for enhanced exploration, (2) systematic constraint relaxation strategy improving solution quality while maintaining feasibility, and (3) statistical validation using Kruskal-Wallis tests with compact letter displays. Applied to a real-world case study with 1,290 plots, CR+DES achieves 3.16% improvement in land-use compatibility compared to state-of-the-art methods, while MSBX+MO excels in price optimization with 3.3% improvement. Statistical analysis confirms algorithms incorporating difference vectors significantly outperform traditional approaches across multiple metrics. The constraint relaxation technique enables broader solution space exploration while maintaining practical constraints. These findings provide urban planners and policymakers with evidence-based computational tools for balancing competing objectives in land-use allocation, supporting more effective urban development policies in rapidly urbanizing regions.

Method: Repurposed a state-of-the-art language model to create a response evaluation framework that assesses ongoing conversations for parasocial cues. Tested with a synthetic dataset of 30 dialogues covering parasocial, sycophantic, and neutral conversations using five-stage testing with tolerant unanimity rules.

Result: The framework successfully identified all parasocial conversations with no false positives, typically detecting harmful dynamics within the first few conversational exchanges.

Conclusion: Evaluation agents show promise as a viable solution for preventing parasocial relationships with AI, providing preliminary evidence that real-time detection of harmful conversational cues is feasible.

Abstract: The development of parasocial relationships with AI agents has severe, and in some cases, tragic effects for human well-being. Yet preventing such dynamics is challenging: parasocial cues often emerge gradually in private conversations, and not all forms of emotional engagement are inherently harmful. We address this challenge by introducing a simple response evaluation framework, created by repurposing a state-of-the-art language model, that evaluates ongoing conversations for parasocial cues in real time. To test the feasibility of this approach, we constructed a small synthetic dataset of thirty dialogues spanning parasocial, sycophantic, and neutral conversations. Iterative evaluation with five stage testing successfully identified all parasocial conversations while avoiding false positives under a tolerant unanimity rule, with detection typically occurring within the first few exchanges. These findings provide preliminary evidence that evaluation agents can provide a viable solution for the prevention of parasocial relations.

Method: TaDiCodec employs end-to-end optimization through a diffusion autoencoder with text guidance integrated into the diffusion decoder to enhance reconstruction quality and achieve optimal compression using a single-layer codebook.

Result: Achieves 6.25 Hz frame rate and 0.0875 kbps bitrate for 24 kHz speech while maintaining superior performance on WER, speaker similarity, and speech quality metrics. Validated in zero-shot text-to-speech with both autoregressive and masked generative modeling.

Conclusion: TaDiCodec provides an effective and efficient solution for speech language modeling with significantly small reconstruction-generation gap, eliminating the need for complex training processes and auxiliary models.

Abstract: Speech tokenizers serve as foundational components for speech language models, yet current designs exhibit several limitations, including: 1) dependence on multi-layer residual vector quantization structures or high frame rates, 2) reliance on auxiliary pre-trained models for semantic distillation, and 3) requirements for complex two-stage training processes. In this work, we introduce the Text-aware Diffusion Transformer Speech Codec (TaDiCodec), a novel approach designed to overcome these challenges. TaDiCodec employs end-to-end optimization for quantization and reconstruction through a diffusion autoencoder, while integrating text guidance into the diffusion decoder to enhance reconstruction quality and achieve optimal compression. TaDiCodec achieves an extremely low frame rate of 6.25 Hz and a corresponding bitrate of 0.0875 kbps with a single-layer codebook for 24 kHz speech, while maintaining superior performance on critical speech generation evaluation metrics such as Word Error Rate (WER), speaker similarity (SIM), and speech quality (UTMOS). Notably, TaDiCodec employs a single-stage, end-to-end training paradigm, and obviating the need for auxiliary pre-trained models. We also validate the compatibility of TaDiCodec in language model based zero-shot text-to-speech with both autoregressive modeling and masked generative modeling, demonstrating its effectiveness and efficiency for speech language modeling, as well as a significantly small reconstruction-generation gap. We will open source our code and model checkpoints. Audio samples are are available at https:/tadicodec.github.io/. We release code and model checkpoints at https:/github.com/HeCheng0625/Diffusion-Speech-Tokenizer.

Method: Two parallel tracks: (1) Text-to-Speech synthesis for generating spoofed speech, and (2) Spoofing-robust Automatic Speaker Verification for detecting spoofed speech, using shared data protocols but treated as independent tasks.

Result: Not specified in the abstract - this appears to be a challenge announcement rather than a results paper.

Conclusion: The challenge aims to promote interdisciplinary collaboration and develop more integrated, robust speech systems that can handle real-world scenarios beyond controlled laboratory conditions.

Abstract: The WildSpoof Challenge aims to advance the use of in-the-wild data in two intertwined speech processing tasks. It consists of two parallel tracks: (1) Text-to-Speech (TTS) synthesis for generating spoofed speech, and (2) Spoofing-robust Automatic Speaker Verification (SASV) for detecting spoofed speech. While the organizers coordinate both tracks and define the data protocols, participants treat them as separate and independent tasks. The primary objectives of the challenge are: (i) to promote the use of in-the-wild data for both TTS and SASV, moving beyond conventional clean and controlled datasets and considering real-world scenarios; and (ii) to encourage interdisciplinary collaboration between the spoofing generation (TTS) and spoofing detection (SASV) communities, thereby fostering the development of more integrated, robust, and realistic systems.

Method: Transformer-based non-autoregressive approach that dynamically determines insertion length during inference based on text transcript and input tempo, maintaining speaker voice, prosody, and spectral properties.

Result: Outperforms baseline adaptive text-to-speech methods on LibriTTS dataset, with positive results from user studies and qualitative evaluations.

Conclusion: The proposed method effectively handles text-conditioned speech insertion with variable lengths while preserving speaker identity and audio quality, demonstrating superior performance over existing approaches.

Abstract: We propose a method for the task of text-conditioned speech insertion, i.e. inserting a speech sample in an input speech sample, conditioned on the corresponding complete text transcript. An example use case of the task would be to update the speech audio when corrections are done on the corresponding text transcript. The proposed method follows a transformer-based non-autoregressive approach that allows speech insertions of variable lengths, which are dynamically determined during inference, based on the text transcript and tempo of the available partial input. It is capable of maintaining the speaker’s voice characteristics, prosody and other spectral properties of the available speech input. Results from our experiments and user study on LibriTTS show that our method outperforms baselines based on an existing adaptive text to speech method. We also provide numerous qualitative results to appreciate the quality of the output from the proposed method.

Method: MSN employs kernel boxes of varying sizes to scan audio spectrograms and integrates a lightweight convolutional network with shared weights for efficient multi-scale feature representation.

Result: Experimental evaluations on DCASE 2020 and DCASE 2023 Task 2 datasets demonstrate state-of-the-art performance.

Conclusion: MSN effectively advances ASD systems by capturing multi-scale patterns in machine sounds, showing superior performance compared to existing methods.

Abstract: Machine sounds exhibit consistent and repetitive patterns in both the frequency and time domains, which vary significantly across scales for different machine types. For instance, rotating machines often show periodic features in short time intervals, while reciprocating machines exhibit broader patterns spanning the time domain. While prior studies have leveraged these patterns to improve Anomalous Sound Detection (ASD), the variation of patterns across scales remains insufficiently explored. To address this gap, we introduce a Multi-scale Scanning Network (MSN) designed to capture patterns at multiple scales. MSN employs kernel boxes of varying sizes to scan audio spectrograms and integrates a lightweight convolutional network with shared weights for efficient and scalable feature representation. Experimental evaluations on the DCASE 2020 and DCASE 2023 Task 2 datasets demonstrate that MSN achieves state-of-the-art performance, highlighting its effectiveness in advancing ASD systems.

Method: Proposes a multi-metric preference alignment strategy using Direct Preference Optimization (DPO) with a new dataset (GenSR-Pref) containing 80K preference pairs selected by complementary metrics covering perceptual quality, signal fidelity, content consistency, and timbre preservation.

Result: Achieves consistent and significant performance gains across three generative paradigms (autoregressive, masked generative, flow-matching models) on various restoration benchmarks in both objective and subjective evaluations. Multi-metric strategy outperforms single-metric approaches in mitigating reward hacking.

Conclusion: The proposed multi-metric preference alignment effectively bridges the gap between training objectives and human perception in generative speech restoration, and the aligned models can serve as powerful data annotators for generating high-quality pseudo-labels in data-scarce scenarios.

Abstract: Recent generative models have significantly advanced speech restoration tasks, yet their training objectives often misalign with human perceptual preferences, resulting in suboptimal quality. While post-training alignment has proven effective in other generative domains like text and image generation, its application to generative speech restoration remains largely under-explored. This work investigates the challenges of applying preference-based post-training to this task, focusing on how to define a robust preference signal and curate high-quality data to avoid reward hacking. To address these challenges, we propose a multi-metric preference alignment strategy. We construct a new dataset, GenSR-Pref, comprising 80K preference pairs, where each chosen sample is unanimously favored by a complementary suite of metrics covering perceptual quality, signal fidelity, content consistency, and timbre preservation. This principled approach ensures a holistic preference signal. Applying Direct Preference Optimization (DPO) with our dataset, we observe consistent and significant performance gains across three diverse generative paradigms: autoregressive models (AR), masked generative models (MGM), and flow-matching models (FM) on various restoration benchmarks, in both objective and subjective evaluations. Ablation studies confirm the superiority of our multi-metric strategy over single-metric approaches in mitigating reward hacking. Furthermore, we demonstrate that our aligned models can serve as powerful ‘‘data annotators’’, generating high-quality pseudo-labels to serve as a supervision signal for traditional discriminative models in data-scarce scenarios like singing voice restoration. Demo Page:https://gensr-pref.github.io

Method: Two specialized networks: mapping-based model to enhance body-conduction signals and masking-based model to denoise acoustic signals, integrated through dynamic fusion mechanism that adapts to local noise conditions

Result: Outperforms single-modal solutions in wide range of noisy environments on TAPS dataset with DNS-2023 noise augmentation, as measured by objective speech quality metrics

Conclusion: The proposed multi-modal framework effectively combines the strengths of both body-conduction and acoustic microphone signals to achieve superior noise suppression and high-frequency reconstruction compared to single-modal approaches

Abstract: Body-conduction microphone signals (BMS) bypass airborne sound, providing strong noise resistance. However, a complementary modality is required to compensate for the inherent loss of high-frequency information. In this study, we propose a novel multi-modal framework that combines BMS and acoustic microphone signals (AMS) to achieve both noise suppression and high-frequency reconstruction. Unlike conventional multi-modal approaches that simply merge features, our method employs two specialized networks: a mapping-based model to enhance BMS and a masking-based model to denoise AMS. These networks are integrated through a dynamic fusion mechanism that adapts to local noise conditions, ensuring the optimal use of each modality’s strengths. We performed evaluations on the TAPS dataset, augmented with DNS-2023 noise clips, using objective speech quality metrics. The results clearly demonstrate that our approach outperforms single-modal solutions in a wide range of noisy environments.

Method: ClearMask modifies audio mel-spectrograms by filtering frequencies to induce voice feature loss, uses audio style transfer to deceive voice decoders, and applies optimized reverberation. LiveMask extends this for real-time protection with universal frequency filters and reverberation generators.

Result: Both systems effectively prevent voice deepfake attacks from deceiving speaker verification models and human listeners, even against unseen synthesis models and black-box APIs. ClearMask shows resilience against adaptive attackers.

Conclusion: The proposed noise-free defense mechanisms provide effective protection against voice deepfake attacks while preserving audio quality, offering both offline and real-time solutions that work against various attack scenarios.

Abstract: Voice deepfake attacks, which artificially impersonate human speech for malicious purposes, have emerged as a severe threat. Existing defenses typically inject noise into human speech to compromise voice encoders in speech synthesis models. However, these methods degrade audio quality and require prior knowledge of the attack approaches, limiting their effectiveness in diverse scenarios. Moreover, real-time audios, such as speech in virtual meetings and voice messages, are still exposed to voice deepfake threats. To overcome these limitations, we propose ClearMask, a noise-free defense mechanism against voice deepfake attacks. Unlike traditional approaches, ClearMask modifies the audio mel-spectrogram by selectively filtering certain frequencies, inducing a transferable voice feature loss without injecting noise. We then apply audio style transfer to further deceive voice decoders while preserving perceived sound quality. Finally, optimized reverberation is introduced to disrupt the output of voice generation models without affecting the naturalness of the speech. Additionally, we develop LiveMask to protect streaming speech in real-time through a universal frequency filter and reverberation generator. Our experimental results show that ClearMask and LiveMask effectively prevent voice deepfake attacks from deceiving speaker verification models and human listeners, even for unseen voice synthesis models and black-box API services. Furthermore, ClearMask demonstrates resilience against adaptive attackers who attempt to recover the original audio signal from the protected speech samples.

Method: Proposed FasterVoiceGrad uses adversarial diffusion conversion distillation (ADCD) to simultaneously distill both the diffusion model and content encoder in one step, leveraging adversarial and score distillation training during the conversion process.

Result: Experimental evaluations show FasterVoiceGrad achieves competitive performance compared to FastVoiceGrad with 6.6-6.9x faster speed on GPU and 1.8x faster on CPU.

Conclusion: FasterVoiceGrad successfully addresses the speed limitations of diffusion-based voice conversion while maintaining high speech quality and speaker similarity through simultaneous distillation of both model components.

Abstract: A diffusion-based voice conversion (VC) model (e.g., VoiceGrad) can achieve high speech quality and speaker similarity; however, its conversion process is slow owing to iterative sampling. FastVoiceGrad overcomes this limitation by distilling VoiceGrad into a one-step diffusion model. However, it still requires a computationally intensive content encoder to disentangle the speaker’s identity and content, which slows conversion. Therefore, we propose FasterVoiceGrad, a novel one-step diffusion-based VC model obtained by simultaneously distilling a diffusion model and content encoder using adversarial diffusion conversion distillation (ADCD), where distillation is performed in the conversion process while leveraging adversarial and score distillation training. Experimental evaluations of one-shot VC demonstrated that FasterVoiceGrad achieves competitive VC performance compared to FastVoiceGrad, with 6.6-6.9 and 1.8 times faster speed on a GPU and CPU, respectively.

Method: Uses a pretrained and frozen vocoder feature extractor with single upsampling step to create vocoder-projected features for adversarial training, avoiding direct waveform processing.

Result: Achieves voice conversion performance comparable to waveform discriminators while reducing training time by 9.6 times and memory consumption by 11.4 times.

Conclusion: VPFD provides an efficient alternative to waveform discriminators by leveraging vocoder features, making adversarial training in TTS/VC systems more practical without sacrificing quality.

Abstract: In text-to-speech (TTS) and voice conversion (VC), acoustic features, such as mel spectrograms, are typically used as synthesis or conversion targets owing to their compactness and ease of learning. However, because the ultimate goal is to generate high-quality waveforms, employing a vocoder to convert these features into waveforms and applying adversarial training in the time domain is reasonable. Nevertheless, upsampling the waveform introduces significant time and memory overheads. To address this issue, we propose a vocoder-projected feature discriminator (VPFD), which uses vocoder features for adversarial training. Experiments on diffusion-based VC distillation demonstrated that a pretrained and frozen vocoder feature extractor with a single upsampling step is necessary and sufficient to achieve a VC performance comparable to that of waveform discriminators while reducing the training time and memory consumption by 9.6 and 11.4 times, respectively.

Method: Fine-tunes self-supervised learning models using multi-task learning framework with four tasks: emotion recognition, gender recognition, speaker verification, and ASR. Introduces co-attention module for dynamic feature interaction and SWFC loss for handling class imbalance

Result: Significant performance improvements validated on the Categorical Emotion Recognition task of the Speech Emotion Recognition in Naturalistic Conditions Challenge

Conclusion: The proposed multi-task learning framework with co-attention and SWFC loss effectively enhances speech emotion recognition by capturing task interactions and addressing data imbalance issues

Abstract: This study investigates fine-tuning self-supervised learn ing (SSL) models using multi-task learning (MTL) to enhance speech emotion recognition (SER). The framework simultane ously handles four related tasks: emotion recognition, gender recognition, speaker verification, and automatic speech recog nition. An innovative co-attention module is introduced to dy namically capture the interactions between features from the primary emotion classification task and auxiliary tasks, en abling context-aware fusion. Moreover, We introduce the Sam ple Weighted Focal Contrastive (SWFC) loss function to ad dress class imbalance and semantic confusion by adjusting sam ple weights for difficult and minority samples. The method is validated on the Categorical Emotion Recognition task of the Speech Emotion Recognition in Naturalistic Conditions Chal lenge, showing significant performance improvements.

Method: Multi-branch multimodal system with time-domain, time-frequency domain acoustic features, and semantic representations. Uses dynamic fusion block with learnable weights to adaptively integrate modalities. Lightweight structure designed by simplifying baseline model.

Result: Superior performance compared to challenge baseline - 78% reduction in model parameters and 5% improvement in accuracy.

Conclusion: The proposed lightweight multimodal system with dynamic fusion effectively integrates acoustic and semantic information for improved speechwellness detection with reduced computational complexity.

Abstract: Suicide is one of the leading causes of death among adolescents. Previous suicide risk prediction studies have primarily focused on either textual or acoustic information in isolation, the integration of multimodal signals, such as speech and text, offers a more comprehensive understanding of an individual’s mental state. Motivated by this, and in the context of the 1st SpeechWellness detection challenge, we explore a lightweight multi-branch multimodal system based on a dynamic fusion mechanism for speechwellness detection. To address the limitation of prior approaches that rely on time-domain waveforms for acoustic analysis, our system incorporates both time-domain and time-frequency (TF) domain acoustic features, as well as semantic representations. In addition, we introduce a dynamic fusion block to adaptively integrate information from different modalities. Specifically, it applies learnable weights to each modality during the fusion process, enabling the model to adjust the contribution of each modality. To enhance computational efficiency, we design a lightweight structure by simplifying the original baseline model. Experimental results demonstrate that the proposed system exhibits superior performance compared to the challenge baseline, achieving a 78% reduction in model parameters and a 5% improvement in accuracy.

Method: Conducted extensive analysis of over one million hours of audio datasets from AI music generation research and manually reviewed 200+ papers from 11 prominent AI and music conferences to quantify genre representation imbalances.

Result: Found stark imbalance: 86% of dataset hours and 93% of researchers focus on Global North music; only 14.6% of data represents Global South genres despite 40% of datasets including some non-Western music; 51% of papers use symbolic generation methods that fail to capture cultural nuances.

Conclusion: Significant underrepresentation of Global South music genres in datasets and research poses serious threat to global musical diversity as AI shapes music creation, requiring immediate steps to mitigate risks and foster more inclusive AI-driven music generation.

Abstract: Recent advances in generative AI have sparked renewed interest and expanded possibilities for music generation. However, the performance and versatility of these systems across musical genres are heavily influenced by the availability of training data. We conducted an extensive analysis of over one million hours of audio datasets used in AI music generation research and manually reviewed more than 200 papers from eleven prominent AI and music conferences and organizations (AAAI, ACM, EUSIPCO, EURASIP, ICASSP, ICML, IJCAI, ISMIR, NeurIPS, NIME, SMC) to identify a critical gap in the fair representation and inclusion of the musical genres of the Global South in AI research. Our findings reveal a stark imbalance: approximately 86% of the total dataset hours and over 93% of researchers focus primarily on music from the Global North. However, around 40% of these datasets include some form of non-Western music, genres from the Global South account for only 14.6% of the data. Furthermore, approximately 51% of the papers surveyed concentrate on symbolic music generation, a method that often fails to capture the cultural nuances inherent in music from regions such as South Asia, the Middle East, and Africa. As AI increasingly shapes the creation and dissemination of music, the significant underrepresentation of music genres in datasets and research presents a serious threat to global musical diversity. We also propose some important steps to mitigate these risks and foster a more inclusive future for AI-driven music generation.

Method: Uses class augmentation through data mixing in embedding space to generate synthetic classes, with adversarial refinement to minimize distinctions between synthetic and real classes.

Result: Achieves consistent improvements across multiple speaker verification and zero-shot speech analysis tasks, with 8% improvement over all baseline models.

Conclusion: CAARMA effectively addresses class diversity limitations in speaker verification through synthetic class generation and adversarial refinement, demonstrating significant performance gains.

Abstract: Speaker verification is a typical zero-shot learning task, where inference of unseen classes is performed by comparing embeddings of test instances to known examples. The models performing inference must hence naturally generate embeddings that cluster same-class instances compactly, while maintaining separation across classes. In order to learn to do so, they are typically trained on a large number of classes (speakers), often using specialized losses. However real-world speaker datasets often lack the class diversity needed to effectively learn this in a generalizable manner. We introduce CAARMA, a class augmentation framework that addresses this problem by generating synthetic classes through data mixing in the embedding space, expanding the number of training classes. To ensure the authenticity of the synthetic classes we adopt a novel adversarial refinement mechanism that minimizes categorical distinctions between synthetic and real classes. We evaluate CAARMA on multiple speaker verification tasks, as well as other representative zero-shot comparison-based speech analysis tasks and obtain consistent improvements: our framework demonstrates a significant improvement of 8% over all baseline models. The code is available at: https://github.com/massabaali7/CAARMA/

Method: Three-stage framework with efficient intra-channel modeling and inter-channel interaction, using a cross-channel attention module to selectively aggregate features from each channel.

Result: LABNet achieves impressive performance with ultra-light resource overhead while maintaining microphone invariance.

Conclusion: The proposed LABNet shows great potential for ad-hoc array processing with its lightweight design and microphone invariance capabilities.

Abstract: Multichannel speech enhancement (SE) aims to restore clean speech from noisy measurements by leveraging spatiotemporal signal features. In ad-hoc array conditions, microphone invariance (MI) requires systems to handle different microphone numbers and array geometries. From a practical perspective, multichannel recordings inevitably increase the computational burden for edge-device applications, highlighting the necessity of lightweight and efficient deployments. In this work, we propose a lightweight attentive beamforming network (LABNet) to integrate MI in a low-complexity real-time SE system. We design a three-stage framework for efficient intra-channel modeling and inter-channel interaction. A cross-channel attention module is developed to aggregate features from each channel selectively. Experimental results demonstrate our LABNet achieves impressive performance with ultra-light resource overhead while maintaining the MI, indicating great potential for ad-hoc array processing.

Method: Uses a Whisper encoder with attention-based classifier to solve five classification problems simultaneously, and publishes an annotated dataset from two popular in-the-wild corpora.

Result: Achieves F1 scores above 85% and equal error rates of 6.5-7.8% for three of five subtasks, outperforms BEATs classifier on speech-specific classes with significantly reduced processing time.

Conclusion: Whilter provides an effective multitask solution for filtering undesirable samples from speech datasets, offering better performance and efficiency compared to single-task alternatives.

Abstract: Large-scale in-the-wild speech datasets have become more prevalent in recent years due to increased interest in models that can learn useful features from unlabelled data for tasks such as speech recognition or synthesis. These datasets often contain undesirable features, such as multiple speakers, non-target languages, and music, which may impact model learning. The Whilter model is proposed as a multitask solution to identify these undesirable samples. Whilter uses a Whisper encoder with an attention-based classifier to solve five diverse classification problems at once. In addition, an annotated dataset is published for a subset of two popular in-the-wild corpora. Whilter achieves F1 scores above 85% and equal error rates of 6.5% to 7.8% for three of five subtasks, outperforming a state-of-the-art BEATs classifier on speech-specific classes, with a notable decrease in processing time compared to a combination of single-task alternatives.

Method: Proposes ASCMamba with DenseEncoder for hierarchical spectral feature extraction, dual-path Mamba blocks using state space models to capture long-range temporal and frequency dependencies, and a two-step pseudo-labeling mechanism for more reliable pseudo-labels.

Result: The system outperforms all participating teams and achieves a 6.2% improvement over the baseline in the APSIPA ASC 2025 Grand Challenge.

Conclusion: ASCMamba demonstrates effective multimodal integration of audio and textual information for acoustic scene classification, setting new state-of-the-art performance with publicly available code and models.

Abstract: Acoustic Scene Classification (ASC) is a fundamental problem in computational audition, which seeks to classify environments based on the distinctive acoustic features. In the ASC task of the APSIPA ASC 2025 Grand Challenge, the organizers introduce a multimodal ASC task. Unlike traditional ASC systems that rely solely on audio inputs, this challenge provides additional textual information as inputs, including the location where the audio is recorded and the time of recording. In this paper, we present our proposed system for the ASC task in the APSIPA ASC 2025 Grand Challenge. Specifically, we propose a multimodal network, ASCMamba, which integrates audio and textual information for fine-grained acoustic scene understanding and effective multimodal ASC. The proposed ASCMamba employs a DenseEncoder to extract hierarchical spectral features from spectrograms, followed by a dual-path Mamba blocks that capture long-range temporal and frequency dependencies using Mamba-based state space models. In addition, we present a two-step pseudo-labeling mechanism to generate more reliable pseudo-labels. Results show that the proposed system outperforms all the participating teams and achieves a 6.2% improvement over the baseline. Code, model and pre-trained checkpoints are available at https://github.com/S-Orion/ASCMamba.git.

Method: QI-DRL framework integrating LSTM networks with Ornstein-Uhlenbeck noise for probabilistic representations and sequential dependency capture.

Result: Average reward of 0.81 (±0.03), prediction loss of 0.15 (±0.02), and 13% fabric cost savings compared to conventional methods.

Conclusion: Promising results demonstrate potential for scalable and adaptive framework to enhance manufacturing efficiency in COP optimization.

Abstract: Cut order planning (COP) is a critical challenge in the textile industry, directly impacting fabric utilization and production costs. Conventional methods based on static heuristics and catalog-based estimations often struggle to adapt to dynamic production environments, resulting in suboptimal solutions and increased waste. In response, we propose a novel Quantum-Inspired Deep Reinforcement Learning (QI-DRL) framework that integrates Long Short-Term Memory (LSTM) networks with Ornstein-Uhlenbeck noise. This hybrid approach is designed to explicitly address key research questions regarding the benefits of quantum-inspired probabilistic representations, the role of LSTM-based memory in capturing sequential dependencies, and the effectiveness of OU noise in facilitating smooth exploration and faster convergence. Extensive training over 1000 episodes demonstrates robust performance, with an average reward of 0.81 (-+0.03) and a steady decrease in prediction loss to 0.15 (-+0.02). A comparative analysis reveals that the proposed approach achieves fabric cost savings of up to 13% compared to conventional methods. Furthermore, statistical evaluations indicate low variability and stable convergence. Despite the fact that the simulation model makes several simplifying assumptions, these promising results underscore the potential of the scalable and adaptive framework to enhance manufacturing efficiency and pave the way for future innovations in COP optimization.

Method: Uses a unified transformer with a powerful inductive bias that treats lattice and atomic properties as a single interdependent system, combined with periodic table-based atomic representation and balanced training strategy.

Result: Achieves 9.62% SUN (Stable, Unique, Novel) rate on MP-20, substantially outperforming FlowMM (4.38%) and MatterGen (3.42%), while generating 63.28% unique and novel structures with comparable stability rates.

Conclusion: Architectural simplicity can be more effective than complexity for materials discovery, especially in data-limited scientific domains where sophisticated alternatives are prone to overfitting.

Abstract: We present CrystalDiT, a diffusion transformer for crystal structure generation that achieves state-of-the-art performance by challenging the trend of architectural complexity. Instead of intricate, multi-stream designs, CrystalDiT employs a unified transformer that imposes a powerful inductive bias: treating lattice and atomic properties as a single, interdependent system. Combined with a periodic table-based atomic representation and a balanced training strategy, our approach achieves 9.62% SUN (Stable, Unique, Novel) rate on MP-20, substantially outperforming recent methods including FlowMM (4.38%) and MatterGen (3.42%). Notably, CrystalDiT generates 63.28% unique and novel structures while maintaining comparable stability rates, demonstrating that architectural simplicity can be more effective than complexity for materials discovery. Our results suggest that in data-limited scientific domains, carefully designed simple architectures outperform sophisticated alternatives that are prone to overfitting.

Method: DyCF leverages Christoffel function from approximation theory and orthogonal polynomials, while DyCG uses growth properties of Christoffel function to eliminate tuning parameters. Both are based on algebraic framework for low-dimensional processing with no memory cost for data history.

Result: DyCF outperforms fine-tuning methods in execution time and memory usage. DyCG performs less well but requires no tuning at all. Both methods were validated on synthetic and real industrial data streams.

Conclusion: The proposed methods provide effective solutions for outlier detection in data streams, with DyCF offering superior performance and DyCG providing parameter-free operation, addressing key challenges in data stream processing.

Abstract: Outlier detection holds significant importance in the realm of data mining, particularly with the growing pervasiveness of data acquisition methods. The ability to identify outliers in data streams is essential for maintaining data quality and detecting faults. However, dealing with data streams presents challenges due to the non-stationary nature of distributions and the ever-increasing data volume. While numerous methods have been proposed to tackle this challenge, a common drawback is the lack of straightforward parameterization in many of them. This article introduces two novel methods: DyCF and DyCG. DyCF leverages the Christoffel function from the theory of approximation and orthogonal polynomials. Conversely, DyCG capitalizes on the growth properties of the Christoffel function, eliminating the need for tuning parameters. Both approaches are firmly rooted in a well-defined algebraic framework, meeting crucial demands for data stream processing, with a specific focus on addressing low-dimensional aspects and maintaining data history without memory cost. A comprehensive comparison between DyCF, DyCG, and state-of-the-art methods is presented, using both synthetic and real industrial data streams. The results show that DyCF outperforms fine-tuning methods, offering superior performance in terms of execution time and memory usage. DyCG performs less well, but has the considerable advantage of requiring no tuning at all.

Method: STRelay models future spatiotemporal contexts in a relaying manner and integrates them with encoded historical representations from base location prediction models. It uses multi-task learning to simultaneously predict next time interval, next moving distance interval, and next location.

Result: STRelay integrated with four state-of-the-art base models on four real-world datasets consistently improved prediction performance by 3.19%-11.56%. It was particularly effective for entertainment-related locations and users who travel longer distances, complementing base models that excel at regular daily routines.

Conclusion: Explicitly modeling future spatiotemporal contexts significantly boosts next location prediction performance, especially for non-daily-routine activities with higher uncertainty. The framework is universally applicable and complementary to existing base models.

Abstract: Next location prediction is a critical task in human mobility modeling, enabling applications like travel planning and urban mobility management. Existing methods mainly rely on historical spatiotemporal trajectory data to train sequence models that directly forecast future locations. However, they often overlook the importance of the future spatiotemporal contexts, which are highly informative for the future locations. For example, knowing how much time and distance a user will travel could serve as a critical clue for predicting the user’s next location. Against this background, we propose \textbf{STRelay}, a universal \textbf{\underline{S}}patio\textbf{\underline{T}}emporal \textbf{\underline{Relay}}ing framework explicitly modeling the future spatiotemporal context given a human trajectory, to boost the performance of different location prediction models. Specifically, STRelay models future spatiotemporal contexts in a relaying manner, which is subsequently integrated with the encoded historical representation from a base location prediction model, enabling multi-task learning by simultaneously predicting the next time interval, next moving distance interval, and finally the next location. We evaluate STRelay integrated with four state-of-the-art location prediction base models on four real-world trajectory datasets. Results demonstrate that STRelay consistently improves prediction performance across all cases by 3.19%-11.56%. Additionally, we find that the future spatiotemporal contexts are particularly helpful for entertainment-related locations and also for user groups who prefer traveling longer distances. The performance gain on such non-daily-routine activities, which often suffer from higher uncertainty, is indeed complementary to the base location prediction models that often excel at modeling regular daily routine patterns.

Method: Proposes RAST framework with three components: 1) Decoupled Encoder and Query Generator for spatial/temporal feature capture and fusion query construction, 2) Spatio-temporal Retrieval Store and Retrievers for pattern storage and retrieval, 3) Universal Backbone Predictor compatible with various STGNNs or MLP predictors.

Result: Extensive experiments on six real-world traffic networks (including large-scale datasets) show RAST achieves superior performance while maintaining computational efficiency.

Conclusion: RAST successfully integrates retrieval-augmented mechanisms with spatio-temporal modeling, providing a universal framework that enhances traffic prediction performance and addresses key challenges in the field.

Abstract: Traffic prediction is a cornerstone of modern intelligent transportation systems and a critical task in spatio-temporal forecasting. Although advanced Spatio-temporal Graph Neural Networks (STGNNs) and pre-trained models have achieved significant progress in traffic prediction, two key challenges remain: (i) limited contextual capacity when modeling complex spatio-temporal dependencies, and (ii) low predictability at fine-grained spatio-temporal points due to heterogeneous patterns. Inspired by Retrieval-Augmented Generation (RAG), we propose RAST, a universal framework that integrates retrieval-augmented mechanisms with spatio-temporal modeling to address these challenges. Our framework consists of three key designs: 1) Decoupled Encoder and Query Generator to capture decoupled spatial and temporal features and construct a fusion query via residual fusion; 2) Spatio-temporal Retrieval Store and Retrievers to maintain and retrieve vectorized fine-grained patterns; and 3) Universal Backbone Predictor that flexibly accommodates pre-trained STGNNs or simple MLP predictors. Extensive experiments on six real-world traffic networks, including large-scale datasets, demonstrate that RAST achieves superior performance while maintaining computational efficiency.

Method: Designed MoE gate function for memory retrieval, learnable aggregation process for memory utilization, and task-specific reflection for memory storage. Uses both off-policy and on-policy optimization to learn effective memorization strategies.

Result: Comprehensive experiments across multiple aspects demonstrate the framework’s effectiveness in optimizing LLM-based agents for specific environments.

Conclusion: The proposed adaptive memory framework enables LLM-based agents to learn optimal memory strategies automatically, addressing limitations of manual approaches and improving performance in interactive scenarios.

Abstract: LLM-based agents have been extensively applied across various domains, where memory stands out as one of their most essential capabilities. Previous memory mechanisms of LLM-based agents are manually predefined by human experts, leading to higher labor costs and suboptimal performance. In addition, these methods overlook the memory cycle effect in interactive scenarios, which is critical to optimizing LLM-based agents for specific environments. To address these challenges, in this paper, we propose to optimize LLM-based agents with an adaptive and data-driven memory framework by modeling memory cycles. Specifically, we design an MoE gate function to facilitate memory retrieval, propose a learnable aggregation process to improve memory utilization, and develop task-specific reflection to adapt memory storage. Our memory framework empowers LLM-based agents to learn how to memorize information effectively in specific environments, with both off-policy and on-policy optimization. In order to evaluate the effectiveness of our proposed methods, we conduct comprehensive experiments across multiple aspects. To benefit the research community in this area, we release our project at https://github.com/nuster1128/learn_to_memorize.

Method: Developed a recurrent transformer U-Net surrogate model trained on 4000 simulation realizations of faulted subsurface aquifers with hierarchical uncertainty in geological parameters and fault permeabilities, then applied for global sensitivity analysis and hierarchical Markov chain Monte Carlo data assimilation.

Result: The new surrogate model outperforms previous recurrent residual U-Net in accuracy and maintains performance across different leakage scenarios. Data assimilation results show significant uncertainty reduction, with monitoring pressure and saturation in all three aquifers providing the best posterior estimates of 3D saturation plumes and leakage volumes.

Conclusion: The recurrent transformer U-Net provides an accurate and efficient surrogate for complex faulted subsurface systems, enabling comprehensive uncertainty quantification and demonstrating the importance of multi-aquifer monitoring for effective geological carbon storage management.

Abstract: Many subsurface formations, including some of those under consideration for large-scale geological carbon storage, include extensive faults that can strongly impact fluid flow. In this study, we develop a new recurrent transformer U-Net surrogate model to provide very fast predictions for pressure and CO2 saturation in realistic faulted subsurface aquifer systems. The geomodel includes a target aquifer (into which supercritical CO2 is injected), surrounding regions, caprock, two extensive faults, and two overlying aquifers. The faults can act as leakage pathways between the three aquifers. The heterogeneous property fields in the target aquifer are characterized by hierarchical uncertainty, meaning both the geological metaparameters (e.g., mean and standard deviation of log-permeability) and the detailed cell properties of each realization, are uncertain. Fault permeabilities are also treated as uncertain. The model is trained with simulation results for (up to) 4000 randomly sampled realizations. Error assessments show that this model is more accurate than a previous recurrent residual U-Net, and that it maintains accuracy for qualitatively different leakage scenarios. The new surrogate is then used for global sensitivity analysis and data assimilation. A hierarchical Markov chain Monte Carlo data assimilation procedure is applied. Different monitoring strategies, corresponding to different amounts and types of observed data collected at monitoring wells, are considered for three synthetic true models. Detailed results demonstrate the degree of uncertainty reduction achieved with the various monitoring strategies. Posterior results for 3D saturation plumes and leakage volumes indicate the benefits of measuring pressure and saturation in all three aquifers.

Method: Integrates Fisher-weighted asymmetric regularization of parameter variances within a variational learning paradigm, dynamically modulating regularization intensity according to parameter uncertainty.

Result: Substantial improvements over existing approaches (VCL, EWC) on benchmarks including SplitMNIST, PermutedMNIST, and SplitFashionMNIST. Boosts immediate task performance and significantly mitigates knowledge degradation over time.

Conclusion: The asymmetric variance penalty mechanism effectively addresses catastrophic forgetting, maintaining knowledge across tasks while improving model accuracy in continual learning scenarios.

Abstract: The persistent challenge of catastrophic forgetting in neural networks has motivated extensive research in continual learning . This work presents a novel continual learning framework that integrates Fisher-weighted asymmetric regularization of parameter variances within a variational learning paradigm. Our method dynamically modulates regularization intensity according to parameter uncertainty, achieving enhanced stability and performance. Comprehensive evaluations on standard continual learning benchmarks including SplitMNIST, PermutedMNIST, and SplitFashionMNIST demonstrate substantial improvements over existing approaches such as Variational Continual Learning and Elastic Weight Consolidation . The asymmetric variance penalty mechanism proves particularly effective in maintaining knowledge across sequential tasks while improving model accuracy. Experimental results show our approach not only boosts immediate task performance but also significantly mitigates knowledge degradation over time, effectively addressing the fundamental challenge of catastrophic forgetting in neural networks

Method: Develops the unified extended matrix (UEM) framework that integrates extended-adjacency matrix and unified graph representation matrix through parametric design, and proposes UEM-based graph Fourier transform (UEM-GFT) for adaptive spectral tuning.

Result: Theoretical analysis shows UEM has positive semi-definiteness and eigenvalue monotonicity under specific conditions. Experiments on synthetic and real-world datasets demonstrate UEM-GFT outperforms existing GSO-based methods in anomaly detection across varying network topologies.

Conclusion: The UEM framework provides a flexible and adaptive approach for graph signal processing that can reveal more graph signal information and achieve superior performance compared to conventional methods.

Abstract: Graph signal processing has become an essential tool for analyzing data structured on irregular domains. While conventional graph shift operators (GSOs) are effective for certain tasks, they inherently lack flexibility in modeling dependencies between non-adjacent nodes, limiting their ability to represent complex graph structures. To address this limitation, this paper proposes the unified extended matrix (UEM) framework, which integrates the extended-adjacency matrix and the unified graph representation matrix through parametric design, so as to be able to flexibly adapt to different graph structures and reveal more graph signal information. Theoretical analysis of the UEM is conducted, demonstrating positive semi-definiteness and eigenvalue monotonicity under specific conditions. Then, we propose graph Fourier transform based on UEM (UEM-GFT), which can adaptively tune spectral properties to enhance signal processing performance. Experimental results on synthetic and real-world datasets demonstrate that the UEM-GFT outperforms existing GSO-based methods in anomaly detection tasks, achieving superior performance across varying network topologies.

Method: Dual-Granularity Guidance Network (DGGN) with: 1) fine-grained stream using Multi-Order Interaction Aggregation for class-specific features, 2) coarse-grained stream for class-agnostic knowledge, 3) multi-semantic cross-attention fusion, 4) Boundary-Aware Exemplar Prioritization, and 5) decoupled Balanced Random Forest classifier.

Result: Extensive experiments on TEP benchmark and real-world MFF dataset demonstrate superior diagnostic performance and stability compared to state-of-the-art FSC-FD approaches.

Conclusion: DGGN effectively addresses catastrophic forgetting and overfitting in few-shot class-incremental fault diagnosis through dual-granularity representation learning and dynamic fusion mechanisms.

Abstract: Few-Shot Class-Incremental Fault Diagnosis (FSC-FD), which aims to continuously learn from new fault classes with only a few samples without forgetting old ones, is critical for real-world industrial systems. However, this challenging task severely amplifies the issues of catastrophic forgetting of old knowledge and overfitting on scarce new data. To address these challenges, this paper proposes a novel framework built upon Dual-Granularity Representations, termed the Dual-Granularity Guidance Network (DGGN). Our DGGN explicitly decouples feature learning into two parallel streams: 1) a fine-grained representation stream, which utilizes a novel Multi-Order Interaction Aggregation module to capture discriminative, class-specific features from the limited new samples. 2) a coarse-grained representation stream, designed to model and preserve general, class-agnostic knowledge shared across all fault types. These two representations are dynamically fused by a multi-semantic cross-attention mechanism, where the stable coarse-grained knowledge guides the learning of fine-grained features, preventing overfitting and alleviating feature conflicts. To further mitigate catastrophic forgetting, we design a Boundary-Aware Exemplar Prioritization strategy. Moreover, a decoupled Balanced Random Forest classifier is employed to counter the decision boundary bias caused by data imbalance. Extensive experiments on the TEP benchmark and a real-world MFF dataset demonstrate that our proposed DGGN achieves superior diagnostic performance and stability compared to state-of-the-art FSC-FD approaches. Our code is publicly available at https://github.com/MentaY/DGGN

Method: Proposes statistical and ensemble-based enhancement techniques including bootstrap-based bagging, regression-based stacking, prediction interval construction, statistical residual modeling, and iterative error feedback.

Result: Hybrid approaches consistently outperform standalone foundation models across multiple horizons. Regression-based ensembles achieve lowest MSE, bootstrap aggregation reduces long-context errors, residual modeling corrects systematic bias, and prediction intervals achieve near nominal coverage.

Conclusion: Integrating statistical reasoning with modern foundation models yields measurable gains in accuracy, reliability, and interpretability for real-world time series applications.

Abstract: Time series foundation models (TSFMs) such as Lag-Llama, TimeGPT, Chronos, MOMENT, UniTS, and TimesFM have shown strong generalization and zero-shot capabilities for time series forecasting, anomaly detection, classification, and imputation. Despite these advantages, their predictions still suffer from variance, domain-specific bias, and limited uncertainty quantification when deployed on real operational data. This paper investigates a suite of statistical and ensemble-based enhancement techniques, including bootstrap-based bagging, regression-based stacking, prediction interval construction, statistical residual modeling, and iterative error feedback, to improve robustness and accuracy. Using the Belgium Electricity Short-Term Load Forecasting dataset as a case study, we demonstrate that the proposed hybrids consistently outperform standalone foundation models across multiple horizons. Regression-based ensembles achieve the lowest mean squared error; bootstrap aggregation markedly reduces long-context errors; residual modeling corrects systematic bias; and the resulting prediction intervals achieve near nominal coverage with widths shrinking as context length increases. The results indicate that integrating statistical reasoning with modern foundation models yields measurable gains in accuracy, reliability, and interpretability for real-world time series applications.

Method: The authors frame various generative methods as PLVMs, categorizing classical flat models (probabilistic PCA, GMMs, LDA), sequential extensions (HMMs, LDS), and deep architectures (VAEs, Normalizing Flows, Diffusion Models, Autoregressive Models, GANs) under a unified probabilistic taxonomy.

Result: The paper reveals shared principles across diverse generative architectures, distinct inference strategies, and representational trade-offs that explain the strengths of different methods, providing a conceptual roadmap for understanding generative AI.

Conclusion: By grounding emerging architectures in their probabilistic heritage, this unified perspective consolidates generative AI’s theoretical foundations, clarifies methodological lineages, and provides guidance for future innovation in the field.

Abstract: From large language models to multi-modal agents, Generative Artificial Intelligence (AI) now underpins state-of-the-art systems. Despite their varied architectures, many share a common foundation in probabilistic latent variable models (PLVMs), where hidden variables explain observed data for density estimation, latent reasoning, and structured inference. This paper presents a unified perspective by framing both classical and modern generative methods within the PLVM paradigm. We trace the progression from classical flat models such as probabilistic PCA, Gaussian mixture models, latent class analysis, item response theory, and latent Dirichlet allocation, through their sequential extensions including Hidden Markov Models, Gaussian HMMs, and Linear Dynamical Systems, to contemporary deep architectures: Variational Autoencoders as Deep PLVMs, Normalizing Flows as Tractable PLVMs, Diffusion Models as Sequential PLVMs, Autoregressive Models as Explicit Generative Models, and Generative Adversarial Networks as Implicit PLVMs. Viewing these architectures under a common probabilistic taxonomy reveals shared principles, distinct inference strategies, and the representational trade-offs that shape their strengths. We offer a conceptual roadmap that consolidates generative AI’s theoretical foundations, clarifies methodological lineages, and guides future innovation by grounding emerging architectures in their probabilistic heritage.

Method: Integrates Fireworks Algorithm for non-monotonic search to suppress outliers, uses entropy-guided optimization for uniform sampling, and designs a custom accelerator to reduce computational overhead.

Result: Achieves more representative training samples, better generalization, and significantly reduces on-device training energy and area requirements.

Conclusion: AdapSNE provides an effective solution for edge device training by addressing key limitations of existing methods while maintaining computational efficiency.

Abstract: Training deep neural networks (DNNs) directly on edge devices has attracted increasing attention, as it offers promising solutions to challenges such as domain adaptation and privacy preservation. However, conventional DNN training typically requires large-scale datasets, which imposes prohibitive overhead on edge devices-particularly for emerging large language model (LLM) tasks. To address this challenge, a DNN-free method (ie., dataset sampling without DNN), named NMS (Near-Memory Sampling), has been introduced. By first conducting dimensionality reduction of the dataset and then performing exemplar sampling in the reduced space, NMS avoids the architectural bias inherent in DNN-based methods and thus achieves better generalization. However, The state-of-the-art, NMS, suffers from two limitations: (1) The mismatch between the search method and the non-monotonic property of the perplexity error function leads to the emergence of outliers in the reduced representation; (2) Key parameter (ie., target perplexity) is selected empirically, introducing arbitrariness and leading to uneven sampling. These two issues lead to representative bias of examplars, resulting in degraded accuracy. To address these issues, we propose AdapSNE, which integrates an efficient non-monotonic search method-namely, the Fireworks Algorithm (FWA)-to suppress outliers, and employs entropy-guided optimization to enforce uniform sampling, thereby ensuring representative training samples and consequently boosting training accuracy. To cut the edge-side cost arising from the iterative computations of FWA search and entropy-guided optimization, we design an accelerator with custom dataflow and time-multiplexing markedly reducing on-device training energy and area.

Method: Two-stage framework: 1) Train pressure-conditioned β-VAE to learn compact latent representations of wake flow dynamics 2) Map low-frequency pressure signals to latent space for flow reconstruction from pressure alone, then use high-frequency pressure inputs for inference.

Result: The method enables reconstruction of 512 Hz turbulent wake flow fields using only spatially sparse wall pressure measurements, overcoming hardware limitations of traditional PIV.

Conclusion: LatentFlow provides a scalable and robust solution for high-frequency turbulent wake flow reconstruction in data-constrained experimental settings by decoupling spatial encoding from temporal pressure measurements.

Abstract: Acquiring temporally high-frequency and spatially high-resolution turbulent wake flow fields in particle image velocimetry (PIV) experiments remains a significant challenge due to hardware limitations and measurement noise. In contrast, temporal high-frequency measurements of spatially sparse wall pressure are more readily accessible in wind tunnel experiments. In this study, we propose a novel cross-modal temporal upscaling framework, LatentFlow, which reconstructs high-frequency (512 Hz) turbulent wake flow fields by fusing synchronized low-frequency (15 Hz) flow field and pressure data during training, and high-frequency wall pressure signals during inference. The first stage involves training a pressure-conditioned $\beta$-variation autoencoder ($p$C-$\beta$-VAE) to learn a compact latent representation that captures the intrinsic dynamics of the wake flow. A secondary network maps synchronized low-frequency wall pressure signals into the latent space, enabling reconstruction of the wake flow field solely from sparse wall pressure. Once trained, the model utilizes high-frequency, spatially sparse wall pressure inputs to generate corresponding high-frequency flow fields via the $p$C-$\beta$-VAE decoder. By decoupling the spatial encoding of flow dynamics from temporal pressure measurements, LatentFlow provides a scalable and robust solution for reconstructing high-frequency turbulent wake flows in data-constrained experimental settings.

Method: Uses grid-cell-like encoding, dentate gyrus-inspired sparse pattern separation, CA3-like autoassociative memory, DG-gated mixture-of-experts routing based on cosine similarity, and prioritized replay with Elastic Weight Consolidation for cortical consolidation.

Result: Achieves near state-of-the-art results on standard continual learning benchmarks while reducing task interference and maintaining lower computational costs compared to existing methods.

Conclusion: The biologically-grounded HiCL architecture provides an effective and efficient solution for continual learning, demonstrating that hippocampal-inspired mechanisms can successfully mitigate catastrophic forgetting in artificial neural networks.

Abstract: We propose HiCL, a novel hippocampal-inspired dual-memory continual learning architecture designed to mitigate catastrophic forgetting by using elements inspired by the hippocampal circuitry. Our system encodes inputs through a grid-cell-like layer, followed by sparse pattern separation using a dentate gyrus-inspired module with top-k sparsity. Episodic memory traces are maintained in a CA3-like autoassociative memory. Task-specific processing is dynamically managed via a DG-gated mixture-of-experts mechanism, wherein inputs are routed to experts based on cosine similarity between their normalized sparse DG representations and learned task-specific DG prototypes computed through online exponential moving averages. This biologically grounded yet mathematically principled gating strategy enables differentiable, scalable task-routing without relying on a separate gating network, and enhances the model’s adaptability and efficiency in learning multiple sequential tasks. Cortical outputs are consolidated using Elastic Weight Consolidation weighted by inter-task similarity. Crucially, we incorporate prioritized replay of stored patterns to reinforce essential past experiences. Evaluations on standard continual learning benchmarks demonstrate the effectiveness of our architecture in reducing task interference, achieving near state-of-the-art results in continual learning tasks at lower computational costs.

Method: Proposes a Transformer model combined with Laplace diffusion technique that conditions the entire modeling process on activity context using specialized embeddings and attention mechanisms to prioritize activity-specific historical patterns.

Result: Model achieved 43% reduction in mean absolute error compared to baseline models and R² of 0.97, showing strong agreement between predicted and actual heart rate values on real-world data from 29 patients over 4 months.

Conclusion: The proposed model is a practical and effective tool for supporting healthcare providers and remote patient monitoring systems by significantly improving heart rate prediction accuracy through activity context integration.

Abstract: With the advent of wearable Internet of Things (IoT) devices, remote patient monitoring (RPM) emerged as a promising solution for managing heart failure. However, the heart rate can fluctuate significantly due to various factors, and without correlating it to the patient’s actual physical activity, it becomes difficult to assess whether changes are significant. Although Artificial Intelligence (AI) models may enhance the accuracy and contextual understanding of remote heart rate monitoring, the integration of activity data is still rarely addressed. In this paper, we propose a Transformer model combined with a Laplace diffusion technique to model heart rate fluctuations driven by physical activity of the patient. Unlike prior models that treat activity as secondary, our approach conditions the entire modeling process on activity context using specialized embeddings and attention mechanisms to prioritize activity specific historical patents. The model captures both long-term patterns and activity-specific heart rate dynamics by incorporating contextualized embeddings and dedicated encoder. The Transformer model was validated on a real-world dataset collected from 29 patients over a 4-month period. Experimental results show that our model outperforms current state-of-the-art methods, achieving a 43% reduction in mean absolute error compared to the considered baseline models. Moreover, the coefficient of determination R2 is 0.97 indicating the model predicted heart rate is in strong agreement with actual heart rate values. These findings suggest that the proposed model is a practical and effective tool for supporting both healthcare providers and remote patient monitoring systems.

Method: Leverages low-rank properties in intermediate activations using PCA with a small calibration dataset, approximates linear transformations with low-rank matrix multiplications, and optimizes self-attention for reduced dimensionality.

Result: Compresses Whisper large-v3 encoder by over 50%, matches Whisper medium’s size with better transcription accuracy, establishing a new Pareto frontier for accuracy and efficiency.

Conclusion: LiteASR provides an effective low-rank compression scheme that significantly reduces ASR encoder computational costs while maintaining performance, making large ASR models more deployable.

Abstract: Modern automatic speech recognition (ASR) models, such as OpenAI’s Whisper, rely on deep encoder-decoder architectures, and their encoders are a critical bottleneck for efficient deployment due to high computational intensity. We introduce LiteASR, a low-rank compression scheme for ASR encoders that significantly reduces inference costs while maintaining transcription accuracy. Our approach leverages the strong low-rank properties observed in intermediate activations: by applying principal component analysis (PCA) with a small calibration dataset, we approximate linear transformations with a chain of low-rank matrix multiplications, and further optimize self-attention to work in reduced dimensionality. Evaluation results show that our method can compress Whisper large-v3’s encoder size by over 50%, matching Whisper medium’s size with better transcription accuracy, thereby establishing a new Pareto frontier of accuracy and efficiency. The code of LiteASR is available at https://github.com/efeslab/LiteASR.

Method: Contrastive-based pre-training approach combined with post-training mechanism that generates reliable pseudo-labels and uses selective activation criteria to optimize computation.

Result: Significantly outperforms state-of-the-art adaptation techniques in both accuracy and efficiency across diverse open-world scenarios.

Conclusion: The proposed method effectively addresses open-world challenges even with imbalanced pre-training data, improving performance on underrepresented classes while maintaining computational efficiency.

Abstract: The expansion of machine learning into dynamic environments presents challenges in handling open-world problems where label shift, covariate shift, and unknown classes emerge. Post-training methods have been explored to address these challenges, adapting models to newly emerging data. However, these methods struggle when the initial pre-training is performed on class-imbalanced datasets, limiting generalization to minority classes. To address this, we propose a method that effectively handles open-world problems even when pre-training is conducted on imbalanced data. Our contrastive-based pre-training approach enhances classification performance, particularly for underrepresented classes. Our post-training mechanism generates reliable pseudo-labels, improving model robustness against open-world problems. We also introduce selective activation criteria to optimize the post-training process, reducing unnecessary computation. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art adaptation techniques in both accuracy and efficiency across diverse open-world scenarios.

Method: Propose Weight Scaling method (WISCA) that rescales weights while preserving model outputs to strategically improve neural network weight patterns without changing network structures, indirectly optimizing the training trajectory.

Result: Significant improvements in convergence quality (5.6% average improvement on zero-shot validation tasks and 2.12% average reduction in training perplexity), particularly effective in LLMs with Grouped Query Attention architectures and LoRA fine-tuning tasks.

Conclusion: WISCA provides an effective approach to enhance training efficiency and model quality by systematically optimizing weight patterns, demonstrating substantial benefits for Transformer-based LLMs without requiring architectural changes.

Abstract: Transformer architecture gradually dominates the LLM field. Recent advances in training optimization for Transformer-based large language models (LLMs) primarily focus on architectural modifications or optimizer adjustments. However, these approaches lack systematic optimization of weight patterns during training. Weight pattern refers to the distribution and relative magnitudes of weight parameters in a neural network. To address this issue, we propose a Weight Scaling method called WISCA to enhance training efficiency and model quality by strategically improving neural network weight patterns without changing network structures. By rescaling weights while preserving model outputs, WISCA indirectly optimizes the model’s training trajectory. Experiments demonstrate that WISCA significantly improves convergence quality (measured by generalization capability and loss reduction), particularly in LLMs with Grouped Query Attention (GQA) architectures and LoRA fine-tuning tasks. Empirical results show 5.6% average improvement on zero-shot validation tasks and 2.12% average reduction in training perplexity across multiple architectures.

Method: Three key innovations: Profile Embeddings to encode any profile type combinations, Multi-Head Profiled Attention for vehicle-client interaction modeling, and Profile-aware Score Reshaping to dynamically adjust decoder logits.

Result: Achieves state-of-the-art results on diverse PVRP benchmarks among learning-based methods, with significant gains in flexibility and computational efficiency.

Conclusion: USPR provides a unified solution that natively handles arbitrary profile types while improving generalization and efficiency in profiled vehicle routing problems.

Abstract: The Profiled Vehicle Routing Problem (PVRP) extends the classical VRP by incorporating vehicle-client-specific preferences and constraints, reflecting real-world requirements such as zone restrictions and service-level preferences. While recent reinforcement-learning solvers have shown promising performance, they require retraining for each new profile distribution, suffer from poor representation ability, and struggle to generalize to out-of-distribution instances. In this paper, we address these limitations by introducing Unified Solver for Profiled Routing (USPR), a novel framework that natively handles arbitrary profile types. USPR introduces on three key innovations: (i) Profile Embeddings (PE) to encode any combination of profile types; (ii) Multi-Head Profiled Attention (MHPA), an attention mechanism that models rich interactions between vehicles and clients; (iii) Profile-aware Score Reshaping (PSR), which dynamically adjusts decoder logits using profile scores to improve generalization. Empirical results on diverse PVRP benchmarks demonstrate that USPR achieves state-of-the-art results among learning-based methods while offering significant gains in flexibility and computational efficiency. We make our source code publicly available to foster future research.

Method: Proposes RED framework with varying exploration spaces, entropy change monitoring to regulate offline-SFT weights, and sample-accuracy-based policy shift mechanism to dynamically choose between imitation and policy learning.

Result: Addresses issues of insufficient exploration in small models, redundancy in distillation process, and distribution discrepancies between offline data and current policy.

Conclusion: RED provides an effective approach to enhance SLMs’ reasoning capabilities through balanced offline-online integration and controlled exploration strategies.

Abstract: Many existing studies have achieved significant improvements in the reasoning capabilities of large language models (LLMs) through reinforcement learning with verifiable rewards (RLVR), while the enhancement of reasoning abilities in small language models (SLMs) has not yet been sufficiently explored. Combining distilled data from larger models with RLVR on small models themselves is a natural approach, but it still faces various challenges and issues. Therefore, we propose \textit{\underline{R}}ecall-\textit{\underline{E}}xtend \textit{\underline{D}}ynamics(RED): Enhancing Small Language Models through Controlled Exploration and Refined Offline Integration. In this paper, we explore the perspective of varying exploration spaces, balancing offline distillation with online reinforcement learning. Simultaneously, we specifically design and optimize for the insertion problem within offline data. By monitoring the ratio of entropy changes in the model concerning offline and online data, we regulate the weight of offline-SFT, thereby addressing the issues of insufficient exploration space in small models and the redundancy and complexity during the distillation process. Furthermore, to tackle the distribution discrepancies between offline data and the current policy, we design a sample-accuracy-based policy shift mechanism that dynamically chooses between imitating offline distilled data and learning from its own policy.

Method: CALR uses a two-component approach: primary path of SVD-compressed layers combined with a parallel, learnable low-rank corrective module trained to recover functional residual error.

Result: CALR reduces parameters by 26.93-51.77% while retaining 59.45-90.42% of original performance on SmolLM2-135M, Qwen3-0.6B, and Llama-3.2-1B, outperforming LaCo, ShortGPT, and LoSparse.

Conclusion: Treating functional information loss as a learnable signal is highly effective for LLM compression, enabling smaller, more efficient models for practical deployment while maintaining performance.

Abstract: Large Language Models (LLMs) present significant deployment challenges due to their immense size and computational requirements. Model compression techniques are essential for making these models practical for resource-constrained environments. A prominent compression strategy is low-rank factorization via Singular Value Decomposition (SVD) to reduce model parameters by approximating weight matrices. However, standard SVD focuses on minimizing matrix reconstruction error, often leading to a substantial loss of the model’s functional performance. This performance degradation occurs because existing methods do not adequately correct for the functional information lost during compression. To address this gap, we introduce Corrective Adaptive Low-Rank Decomposition (CALR), a two-component compression approach. CALR combines a primary path of SVD-compressed layers with a parallel, learnable, low-rank corrective module that is explicitly trained to recover the functional residual error. Our experimental evaluation on SmolLM2-135M, Qwen3-0.6B, and Llama-3.2-1B, demonstrates that CALR can reduce parameter counts by 26.93% to 51.77% while retaining 59.45% to 90.42% of the original model’s performance, consistently outperforming LaCo, ShortGPT, and LoSparse. CALR’s success shows that treating functional information loss as a learnable signal is a highly effective compression paradigm. This approach enables the creation of significantly smaller, more efficient LLMs, advancing their accessibility and practical deployment in real-world applications.

Method: Uses Spatial-Temporal Unified Graph with attention mechanism to directly model correlations, partitions traffic flow into neighborhood subsets, and employs exchanging mechanism to capture both short-range and long-range correlations.

Result: Superior performance on PEMS-BAY and SHMetro datasets across various prediction horizons, with visualization confirming ability to adapt to dynamic traffic patterns and capture long-range dependencies.

Conclusion: STGAtt demonstrates strong potential for real-world traffic flow forecasting applications through its effective unified spatial-temporal modeling approach.

Abstract: Accurate and timely traffic flow forecasting is crucial for intelligent transportation systems. This paper presents a novel deep learning model, the Spatial-Temporal Unified Graph Attention Network (STGAtt). By leveraging a unified graph representation and an attention mechanism, STGAtt effectively captures complex spatial-temporal dependencies. Unlike methods relying on separate spatial and temporal dependency modeling modules, STGAtt directly models correlations within a Spatial-Temporal Unified Graph, dynamically weighing connections across both dimensions. To further enhance its capabilities, STGAtt partitions traffic flow observation signal into neighborhood subsets and employs a novel exchanging mechanism, enabling effective capture of both short-range and long-range correlations. Extensive experiments on the PEMS-BAY and SHMetro datasets demonstrate STGAtt’s superior performance compared to state-of-the-art baselines across various prediction horizons. Visualization of attention weights confirms STGAtt’s ability to adapt to dynamic traffic patterns and capture long-range dependencies, highlighting its potential for real-world traffic flow forecasting applications.

Method: Uses a multidimensional Gaussian loss with iterative estimation of means and covariance matrices, leverages Fourier representation for covariance matrix stabilization, and introduces information sharing regularization between image-specific and global covariance estimates.

Result: The method successfully captures aleatoric uncertainty, preserves spatial correlation, stabilizes network training, and enables convergence in super-resolution downscaling networks while maintaining prediction performance.

Conclusion: This framework provides efficient sampling, explicit correlation modeling, and extensibility to complex distribution families, making it broadly applicable to uncertainty-aware prediction in scientific models.

Abstract: Accurate quantification of uncertainty in neural network predictions remains a central challenge for scientific applications involving high-dimensional, correlated data. While existing methods capture either aleatoric or epistemic uncertainty, few offer closed-form, multidimensional distributions that preserve spatial correlation while remaining computationally tractable. In this work, we present a framework for training neural networks with a multidimensional Gaussian loss, generating closed-form predictive distributions over outputs with non-identically distributed and heteroscedastic structure. Our approach captures aleatoric uncertainty by iteratively estimating the means and covariance matrices, and is demonstrated on a super-resolution example. We leverage a Fourier representation of the covariance matrix to stabilize network training and preserve spatial correlation. We introduce a novel regularization strategy – referred to as information sharing – that interpolates between image-specific and global covariance estimates, enabling convergence of the super-resolution downscaling network trained on image-specific distributional loss functions. This framework allows for efficient sampling, explicit correlation modeling, and extensions to more complex distribution families all without disrupting prediction performance. We demonstrate the method on a surface wind speed downscaling task and discuss its broader applicability to uncertainty-aware prediction in scientific models.

Method: Embed concepts as linear subspaces with dimensionality representing generality and inclusion representing hierarchy. Use smooth relaxation of orthogonal projection operators for differentiable learning of both subspace orientation and dimension.

Result: Achieves state-of-the-art results in reconstruction and link prediction on WordNet. Surpasses bi-encoder baselines on natural language inference benchmarks with interpretable entailment formulation.

Conclusion: Subspace embeddings provide a geometrically grounded framework that naturally supports logical operations and hierarchical relationships, offering improved reasoning capabilities over traditional point embeddings.

Abstract: Traditional neural embeddings represent concepts as points, excelling at similarity but struggling with higher-level reasoning and asymmetric relationships. We introduce a novel paradigm: embedding concepts as linear subspaces. This framework inherently models generality via subspace dimensionality and hierarchy through subspace inclusion. It naturally supports set-theoretic operations like intersection (conjunction), linear sum (disjunction) and orthogonal complements (negations), aligning with classical formal semantics. To enable differentiable learning, we propose a smooth relaxation of orthogonal projection operators, allowing for the learning of both subspace orientation and dimension. Our method achieves state-of-the-art results in reconstruction and link prediction on WordNet. Furthermore, on natural language inference benchmarks, our subspace embeddings surpass bi-encoder baselines, offering an interpretable formulation of entailment that is both geometrically grounded and amenable to logical operations.

Method: Proposes auxiliary equation neural networks method (AENNM) with novel activation functions derived from Riccati equation solutions, using “2-2-2-1” and “3-2-2-1” NN architectures to construct trial functions.

Result: Successfully solved three NLPDE examples (nonlinear evolution equation, KdV-Burgers equation, 2+1D Boussinesq equation), obtaining exact analytical solutions in hyperbolic, trigonometric, and rational functions with previously unreported solutions.

Conclusion: AENNM provides a novel framework for solving NLPDEs with broad scientific and engineering applications, effectively combining neural networks with mathematical theory for enhanced computational efficiency and accuracy.

Abstract: In this study, we firstly propose an auxiliary equation neural networks method (AENNM), an innovative analytical method that integrates neural networks (NNs) models with the auxiliary equation method to obtain exact solutions of nonlinear partial differential equations (NLPDEs). A key novelty of this method is the introduction of a novel activation function derived from the solutions of the Riccati equation, establishing a new mathematical link between differential equations theory and deep learning. By combining the strong approximation capability of NNs with the high precision of symbolic computation, AENNM significantly enhances computational efficiency and accuracy. To demonstrate the effectiveness of the AENNM in solving NLPDEs, three numerical examples are investigated, including the nonlinear evolution equation, the Korteweg-de Vries-Burgers equation, and the (2+1)-dimensional Boussinesq equation. Furthermore, some new trial functions are constructed by setting specific activation functions within the “2-2-2-1” and “3-2-2-1” NNs models. By embedding the auxiliary equation method into the NNs framework, we derive previously unreported solutions. The exact analytical solutions are expressed in terms of hyperbolic functions, trigonometric functions, and rational functions. Finally, three-dimensional plots, contour plots, and density plots are presented to illustrate the dynamic characteristics of the obtained solutions. This research provides a novel methodological framework for addressing NLPDEs, with broad applicability across scientific and engineering fields.

Method: Introduces ALSO (Adaptive Loss Scaling Optimizer), an adaptive algorithm for a modified DRO objective that supports weight assignment to sample groups and handles stochastic gradients.

Result: Proves convergence for non-convex objectives (typical for DL models) and demonstrates superior performance across diverse DL tasks including Tabular DL and Split Learning tasks.

Conclusion: ALSO successfully bridges the DRO-DL gap by providing an adaptive, group-aware optimization method that outperforms both traditional optimizers and existing DRO approaches in practical DL applications.

Abstract: While traditional Deep Learning (DL) optimization methods treat all training samples equally, Distributionally Robust Optimization (DRO) adaptively assigns importance weights to different samples. However, a significant gap exists between DRO and current DL practices. Modern DL optimizers require adaptivity and the ability to handle stochastic gradients, as these methods demonstrate superior performance. Additionally, for practical applications, a method should allow weight assignment not only to individual samples, but also to groups of objects (for example, all samples of the same class). This paper aims to bridge this gap by introducing ALSO $\unicode{x2013}$ Adaptive Loss Scaling Optimizer $\unicode{x2013}$ an adaptive algorithm for a modified DRO objective that can handle weight assignment to sample groups. We prove the convergence of our proposed algorithm for non-convex objectives, which is the typical case for DL models. Empirical evaluation across diverse Deep Learning tasks, from Tabular DL to Split Learning tasks, demonstrates that ALSO outperforms both traditional optimizers and existing DRO methods.

Method: Train neural networks to satisfy integral equations derived from first-transition analysis, adapting the approach for Poisson’s equation and stationary distribution estimation.

Result: The method remains effective even for Markov chains on non-compact state spaces, demonstrated through queueing theory examples and other applications.

Conclusion: Deep learning successfully automates Lyapunov function construction, providing an effective computational approach for stability analysis and related problems in Markovian modeling.

Abstract: Lyapunov functions are fundamental to establishing the stability of Markovian models, yet their construction typically demands substantial creativity and analytical effort. In this paper, we show that deep learning can automate this process by training neural networks to satisfy integral equations derived from first-transition analysis. Beyond stability analysis, our approach can be adapted to solve Poisson’s equation and estimate stationary distributions. While neural networks are inherently function approximators on compact domains, it turns out that our approach remains effective when applied to Markov chains on non-compact state spaces. We demonstrate the effectiveness of this methodology through several examples from queueing theory and beyond.

Method: WST uses reinforcement learning where a small Teacher model generates instructions that are iteratively improved based on the Student model’s outcomes. The framework only requires a weak teacher model to enhance a much larger student model.

Result: Achieved substantial gains: 98% improvement on MATH-500 and 134% improvement on HH-RLHF benchmarks, surpassing baselines like GPT-4o-mini and Llama-70B. Demonstrates small models can reliably scaffold larger ones.

Conclusion: WST provides a scalable solution for efficient and safe LLM prompt refinement, unlocking latent capabilities in large models while avoiding misleading prompts that stronger teachers may introduce.

Abstract: Effective prompt engineering remains a challenging task for many applications. We introduce Weak-to-Strong Transfer (WST), an automatic prompt engineering framework where a small “Teacher” model generates instructions that enhance the performance of a much larger “Student” model. Unlike prior work, WST requires only a weak teacher, making it efficient and broadly applicable in settings where large models are closed-source or difficult to fine-tune. Using reinforcement learning, the Teacher Model’s instructions are iteratively improved based on the Student Model’s outcomes, yielding substantial gains across reasoning (MATH-500, GSM8K) and alignment (HH-RLHF) benchmarks - 98% on MATH-500 and 134% on HH-RLHF - and surpassing baselines such as GPT-4o-mini and Llama-70B. These results demonstrate that small models can reliably scaffold larger ones, unlocking latent capabilities while avoiding misleading prompts that stronger teachers may introduce, establishing WST as a scalable solution for efficient and safe LLM prompt refinement.

Method: Embeds multimodal inputs into shared hyperbolic space using contrastive and novel stacked entailment-based objective.

Result: Hyperbolic embedding achieves competitive performance with Euclidean baselines, outperforms all other models on unseen species classification using DNA barcodes.

Conclusion: Framework offers structure-aware foundation for biodiversity modelling with applications to species discovery and conservation, though fine-grained classification and open-world generalization remain challenging.

Abstract: Taxonomic classification in biodiversity research involves organizing biological specimens into structured hierarchies based on evidence, which can come from multiple modalities such as images and genetic information. We investigate whether hyperbolic networks can provide a better embedding space for such hierarchical models. Our method embeds multimodal inputs into a shared hyperbolic space using contrastive and a novel stacked entailment-based objective. Experiments on the BIOSCAN-1M dataset show that hyperbolic embedding achieves competitive performance with Euclidean baselines, and outperforms all other models on unseen species classification using DNA barcodes. However, fine-grained classification and open-world generalization remain challenging. Our framework offers a structure-aware foundation for biodiversity modelling, with potential applications to species discovery, ecological monitoring, and conservation efforts.

Method: Train models on state sequences from random Boolean functions with random initial conditions to prevent memorization, then evaluate next-state prediction and multi-step reasoning performance across architectures.

Result: Models achieve high accuracy in next-state prediction but performance declines sharply for multi-step reasoning. Depth, recurrence, memory, and test-time compute scaling significantly enhance reasoning capabilities.

Conclusion: Effective model depth extension through architectural choices and compute scaling is crucial for improving multi-step reasoning in neural models.

Abstract: Reasoning is a core capability of large language models, yet understanding how they learn and perform multi-step reasoning remains an open problem. In this study, we explore how different architectures and training methods affect model multi-step reasoning capabilities within a cellular automata framework. By training on state sequences generated with random Boolean functions for random initial conditions to exclude memorization, we demonstrate that most neural architectures learn to abstract the underlying rules. While models achieve high accuracy in next-state prediction, their performance declines sharply if multi-step reasoning is required. We confirm that increasing model depth plays a crucial role for sequential computations. We demonstrate that an extension of the effective model depth with recurrence, memory, and test-time compute scaling substantially enhances reasoning capabilities.

Method: Adapts VICReg regularization with three mechanisms: variance term to reduce protected attribute reliance, invariance term for consistent predictions, and covariance term to minimize correlational dependence on protected attributes.

Result: Improves overall fairness performance with minimal classification performance reduction across three healthcare datasets (D-Vlog, MIMIC, MODMA), significantly enhancing performance-fairness Pareto frontier.

Conclusion: The proposed FAIRWELL framework effectively enforces fairness in multimodal prediction tasks by learning subject-independent representations while maintaining strong classification performance.

Abstract: Early efforts on leveraging self-supervised learning (SSL) to improve machine learning (ML) fairness has proven promising. However, such an approach has yet to be explored within a multimodal context. Prior work has shown that, within a multimodal setting, different modalities contain modality-unique information that can complement information of other modalities. Leveraging on this, we propose a novel subject-level loss function to learn fairer representations via the following three mechanisms, adapting the variance-invariance-covariance regularization (VICReg) method: (i) the variance term, which reduces reliance on the protected attribute as a trivial solution; (ii) the invariance term, which ensures consistent predictions for similar individuals; and (iii) the covariance term, which minimizes correlational dependence on the protected attribute. Consequently, our loss function, coined as FAIRWELL, aims to obtain subject-independent representations, enforcing fairness in multimodal prediction tasks. We evaluate our method on three challenging real-world heterogeneous healthcare datasets (i.e. D-Vlog, MIMIC and MODMA) which contain different modalities of varying length and different prediction tasks. Our findings indicate that our framework improves overall fairness performance with minimal reduction in classification performance and significantly improves on the performance-fairness Pareto frontier.

Method: Proposes DR-CircuitGNN with row-wise sparsity-aware Dynamic-ReLU and optimized SpMM kernels for heterogeneous message-passing, plus parallel CPU-GPU processing using multi-threaded initialization and multiple cudaStreams.

Result: Achieves up to 3.51x forward and 4.09x backward propagation speedup on CircuitNet designs, with 2.71x speedup over DGL cuSPARSE on full datasets with negligible accuracy impact.

Conclusion: The proposed optimization strategy effectively accelerates HGNN training for EDA applications while maintaining model accuracy, addressing computational bottlenecks in circuit graph analysis.

Abstract: The increasing scale and complexity of integrated circuit design have led to increased challenges in Electronic Design Automation (EDA). Graph Neural Networks (GNNs) have emerged as a promising approach to assist EDA design as circuits can be naturally represented as graphs. While GNNs offer a foundation for circuit analysis, they often fail to capture the full complexity of EDA designs. Heterogeneous Graph Neural Networks (HGNNs) can better interpret EDA circuit graphs as they capture both topological relationships and geometric features. However, the improved representation capability comes at the cost of even higher computational complexity and processing cost due to their serial module-wise message-passing scheme, creating a significant performance bottleneck. In this paper, we propose DR-CircuitGNN, a fast GPU kernel design by leveraging row-wise sparsity-aware Dynamic-ReLU and optimizing SpMM kernels during heterogeneous message-passing to accelerate HGNNs training on EDA-related circuit graph datasets. To further enhance performance, we propose a parallel optimization strategy that maximizes CPU-GPU concurrency by concurrently processing independent subgraphs using multi-threaded CPU initialization and GPU kernel execution via multiple cudaStreams. Our experiments show that on three representative CircuitNet designs (small, medium, large), the proposed method can achieve up to 3.51x and 4.09x speedup compared to the SOTA for forward and backward propagation, respectively. On full-size CircuitNet and sampled Mini-CircuitNet, our parallel design enables up to 2.71x speed up over the official DGL implementation cuSPARSE with negligible impact on correlation scores and error rates.

Method: Testing against numerical simulations of neural circuits with known ground-truth connectivity to evaluate hypotheses for explaining learned model weights and alignment between extracted representations and underlying graphs.

Result: Discovered modest alignment between extracted network representations and underlying directed graphs, but strong alignment in co-input graph representations.

Conclusion: These findings motivate incorporating graph-based geometric constraints in building large-scale foundation models for neural data to improve interpretability of latent graph representations.

Abstract: Inferring temporal interaction graphs and higher-order structure from neural signals is a key problem in building generative models for systems neuroscience. Foundation models for large-scale neural data represent shared latent structures of neural signals. However, extracting interpretable latent graph representations in foundation models remains challenging and unsolved. Here we explore latent graph learning in generative models of neural signals. By testing against numerical simulations of neural circuits with known ground-truth connectivity, we evaluate several hypotheses for explaining learned model weights. We discover modest alignment between extracted network representations and the underlying directed graphs and strong alignment in the co-input graph representations. These findings motivate paths towards incorporating graph-based geometric constraints in the construction of large-scale foundation models for neural data.

Method: Employed interpretability techniques to analyze multiple LLMs under 4-bit and 8-bit quantization, examining model calibration, neuron activations (dead neurons), and neuron contribution to predictions.

Result: Quantization impact on model calibration is minor; dead neuron counts remain consistent; smaller models have fewer salient neurons while larger models have more (except Llama-2-7B); neuron redundancy effects vary by model.

Conclusion: Quantization effects vary by model and tasks, but no drastic changes were observed that would discourage using quantization as a reliable model compression technique.

Abstract: Quantization offers a practical solution to deploy LLMs in resource-constraint environments. However, its impact on internal representations remains understudied, raising questions about the reliability of quantized models. In this study, we employ a range of interpretability techniques to investigate how quantization affects model and neuron behavior. We analyze multiple LLMs under 4-bit and 8-bit quantization. Our findings reveal that the impact of quantization on model calibration is generally minor. Analysis of neuron activations indicates that the number of dead neurons, i.e., those with activation values close to 0 across the dataset, remains consistent regardless of quantization. In terms of neuron contribution to predictions, we observe that smaller full precision models exhibit fewer salient neurons, whereas larger models tend to have more, with the exception of Llama-2-7B. The effect of quantization on neuron redundancy varies across models. Overall, our findings suggest that effect of quantization may vary by model and tasks, however, we did not observe any drastic change which may discourage the use of quantization as a reliable model compression technique.

Method: Uses an anchor mean from any point regressor, projects it to latent space, employs learnable metric-window kernel for locality scoring, and soft routing to mixture-density-network experts that produce heteroscedastic correction and predictive variance. Trained via negative log-likelihood minimization with post-hoc calibration.

Result: Achieves minimax-optimal L² risk rate O(N^{-2α/(2α+d)}), logarithmic scaling in model complexity, and consistently matches or surpasses NGBoost baseline in RMSE and NLL across UCI datasets, setting new SOTA results.

Conclusion: Anchor-MoE provides a theoretically grounded framework for probabilistic regression that combines the strengths of point estimators with uncertainty quantification, demonstrating both optimal theoretical convergence rates and superior empirical performance.

Abstract: Regression under uncertainty is fundamental across science and engineering. We present an Anchored Mixture of Experts (Anchor-MoE), a model that handles both probabilistic and point regression. For simplicity, we use a tuned gradient-boosting model to furnish the anchor mean; however, any off-the-shelf point regressor can serve as the anchor. The anchor prediction is projected into a latent space, where a learnable metric-window kernel scores locality and a soft router dispatches each sample to a small set of mixture-density-network experts; the experts produce a heteroscedastic correction and predictive variance. We train by minimizing negative log-likelihood, and on a disjoint calibration split fit a post-hoc linear map on predicted means to improve point accuracy. On the theory side, assuming a H"older smooth regression function of order~$\alpha$ and fixed Lipschitz partition-of-unity weights with bounded overlap, we show that Anchor-MoE attains the minimax-optimal $L^2$ risk rate $O!\big(N^{-2\alpha/(2\alpha+d)}\big)$. In addition, the CRPS test generalization gap scales as $\widetilde{O}!\Big(\sqrt{(\log(Mh)+P+K)/N}\Big)$; it is logarithmic in $Mh$ and scales as the square root in $P$ and $K$. Under bounded-overlap routing, $K$ can be replaced by $k$, and any dependence on a latent dimension is absorbed into $P$. Under uniformly bounded means and variances, an analogous $\widetilde{O}!\big(\sqrt{(\log(Mh)+P+K)/N}\big)$ scaling holds for the test NLL up to constants. Empirically, across standard UCI regressions, Anchor-MoE consistently matches or surpasses the strong NGBoost baseline in RMSE and NLL; on several datasets it achieves new state-of-the-art probabilistic regression results on our benchmark suite. Code is available at https://github.com/BaozhuoSU/Probabilistic_Regression.

Method: Parameterizes coupled differential equations for mean and covariance dynamics, enabling state-dependent learnable process noise and solving coupled ODEs for state and covariance evolution without discretization artifacts.

Result: UPN demonstrates effectiveness in continuous normalizing flows with uncertainty quantification, time-series forecasting with well-calibrated confidence intervals, and robust trajectory prediction in both stable and chaotic dynamical systems.

Conclusion: UPN provides a principled framework for uncertainty quantification in continuous-time modeling, handling irregularly-sampled observations naturally and adapting evaluation strategy to input complexity.

Abstract: This paper introduces Uncertainty Propagation Network (UPN), a novel family of neural differential equations that naturally incorporate uncertainty quantification into continuous-time modeling. Unlike existing neural ODEs that predict only state trajectories, UPN simultaneously model both state evolution and its associated uncertainty by parameterizing coupled differential equations for mean and covariance dynamics. The architecture efficiently propagates uncertainty through nonlinear dynamics without discretization artifacts by solving coupled ODEs for state and covariance evolution while enabling state-dependent, learnable process noise. The continuous-depth formulation adapts its evaluation strategy to each input’s complexity, provides principled uncertainty quantification, and handles irregularly-sampled observations naturally. Experimental results demonstrate UPN’s effectiveness across multiple domains: continuous normalizing flows (CNFs) with uncertainty quantification, time-series forecasting with well-calibrated confidence intervals, and robust trajectory prediction in both stable and chaotic dynamical systems.

Method: Introduces the concept of Over-dilution with two dilution factors (intra-node for attribute-level and inter-node for node-level) and proposes a transformer-based solution to alleviate this issue while complementing existing MPNN methods.

Result: The proposed transformer-based approach effectively mitigates over-dilution and contributes to developing more informative graph representations, with implementation publicly available.

Conclusion: The findings provide new insights into MPNN limitations and offer a complementary solution that enhances node embedding methods, advancing the development of more robust graph representation learning.

Abstract: Message Passing Neural Networks (MPNNs) hold a key position in machine learning on graphs, but they struggle with unintended behaviors, such as over-smoothing and over-squashing, due to irregular data structures. The observation and formulation of these limitations have become foundational in constructing more informative graph representations. In this paper, we delve into the limitations of MPNNs, focusing on aspects that have previously been overlooked. Our observations reveal that even within a single layer, the information specific to an individual node can become significantly diluted. To delve into this phenomenon in depth, we present the concept of Over-dilution and formulate it with two dilution factors: intra-node dilution for attribute-level and inter-node dilution for node-level representations. We also introduce a transformer-based solution that alleviates over-dilution and complements existing node embedding methods like MPNNs. Our findings provide new insights and contribute to the development of informative representations. The implementation and supplementary materials are publicly available at https://github.com/LeeJunHyun/NATR.

Method: Leverages VQ-VAE Transformer’s autoregressive loss for OOD detection, integrates with continual learning strategies, and introduces a novel quantitative metric for simultaneous OOD detection and in-distribution performance evaluation.

Result: Superior performance compared to conventional reconstruction methods and established baselines, effectively maintains quality prediction across distribution shifts in real-world welding scenarios.

Conclusion: Provides an explainable and adaptive solution for quality assurance in dynamic manufacturing, contributing to robust practical AI systems in industrial environments.

Abstract: Modern manufacturing relies heavily on fusion welding processes, including gas metal arc welding (GMAW). Despite significant advances in machine learning-based quality prediction, current models exhibit critical limitations when confronted with the inherent distribution shifts that occur in dynamic manufacturing environments. In this work, we extend the VQ-VAE Transformer architecture - previously demonstrating state-of-the-art performance in weld quality prediction - by leveraging its autoregressive loss as a reliable out-of-distribution (OOD) detection mechanism. Our approach exhibits superior performance compared to conventional reconstruction methods, embedding error-based techniques, and other established baselines. By integrating OOD detection with continual learning strategies, we optimize model adaptation, triggering updates only when necessary and thereby minimizing costly labeling requirements. We introduce a novel quantitative metric that simultaneously evaluates OOD detection capability while interpreting in-distribution performance. Experimental validation in real-world welding scenarios demonstrates that our framework effectively maintains robust quality prediction capabilities across significant distribution shifts, addressing critical challenges in dynamic manufacturing environments where process parameters frequently change. This research makes a substantial contribution to applied artificial intelligence by providing an explainable and at the same time adaptive solution for quality assurance in dynamic manufacturing processes - a crucial step towards robust, practical AI systems in the industrial environment.

Method: Physically informative neural symbolic approach that learns physical entity activity state dynamics and integrates them into multi-layer spatiotemporal co-evolution networks using physical information methods.

Result: Experimental results show the model achieves better performance and more accurate/effective industrial chain elasticity predictions.

Conclusion: The approach has practical significance for industrial development by providing more accurate resilience predictions for complex industrial networks.

Abstract: Industrial chain plays an increasingly important role in the sustainable development of national economy. However, as a typical complex network, data-driven deep learning is still in its infancy in describing and analyzing the resilience of complex networks, and its core is the lack of a theoretical framework to describe the system dynamics. In this paper, we propose a physically informative neural symbolic approach to describe the evolutionary dynamics of complex networks for resilient prediction. The core idea is to learn the dynamics of the activity state of physical entities and integrate it into the multi-layer spatiotemporal co-evolution network, and use the physical information method to realize the joint learning of physical symbol dynamics and spatiotemporal co-evolution topology, so as to predict the industrial chain resilience. The experimental results show that the model can obtain better results and predict the elasticity of the industry chain more accurately and effectively, which has certain practical significance for the development of the industry.

Method: Proposes RKD-SC framework with KDL-DARTS algorithm for lightweight architecture search and two-stage robust knowledge distillation, plus channel-aware transformer block for noise resilience.

Result: Significantly reduces model parameters while preserving teacher model performance and showing superior robustness in image classification tasks.

Conclusion: The framework enables efficient deployment of large-scale models in semantic communication systems with improved robustness against channel noise.

Abstract: Large-scale models (LSMs) can be an effective framework for semantic representation and understanding, thereby providing a suitable tool for designing semantic communication (SC) systems. However, their direct deployment is often hindered by high computational complexity and resource requirements. In this paper, a novel robust knowledge distillation based semantic communication (RKD-SC) framework is proposed to enable efficient and \textcolor{black}{channel-noise-robust} LSM-powered SC. The framework addresses two key challenges: determining optimal compact model architectures and effectively transferring knowledge while maintaining robustness against channel noise. First, a knowledge distillation-based lightweight differentiable architecture search (KDL-DARTS) algorithm is proposed. This algorithm integrates knowledge distillation loss and a complexity penalty into the neural architecture search process to identify high-performance, lightweight semantic encoder architectures. Second, a novel two-stage robust knowledge distillation (RKD) algorithm is developed to transfer semantic capabilities from an LSM (teacher) to a compact encoder (student) and subsequently enhance system robustness. To further improve resilience to channel impairments, a channel-aware transformer (CAT) block is introduced as the channel codec, trained under diverse channel conditions with variable-length outputs. Extensive simulations on image classification tasks demonstrate that the RKD-SC framework significantly reduces model parameters while preserving a high degree of the teacher model’s performance and exhibiting superior robustness compared to existing methods.

Method: Proposes Neural Contrast Expansion (NCE), an architecture inspired by strong contrast expansion (SCE) formalism, to learn surrogate PDE kernels from structure-property data. Works for linear self-adjoint PDEs on bi-phase microstructures without requiring full PDE solution field measurements.

Result: NCE models demonstrate accurate and insightful sensitivity information for static conduction and electromagnetic wave propagation cases. The method provides useful material design insights while being more accessible as it only requires macroscopic property measurements.

Conclusion: NCE successfully bridges the gap between traditional PDE solving and data-driven approaches by offering both computational efficiency and explainable sensitivity analysis, making it particularly valuable for material development contexts where only macroscopic property data is available.

Abstract: Effective properties of composite materials are defined as the ensemble average of property-specific PDE solutions over the underlying microstructure distributions. Traditionally, predicting such properties can be done by solving PDEs derived from microstructure samples or building data-driven models that directly map microstructure samples to properties. The former has a higher running cost, but provides explainable sensitivity information that may guide material design; the latter could be more cost-effective if the data overhead is amortized, but its learned sensitivities are often less explainable. With a focus on properties governed by linear self-adjoint PDEs (e.g., Laplace, Helmholtz, and Maxwell curl-curl) defined on bi-phase microstructures, we propose a structure-property model that is both cost-effective and explainable. Our method is built on top of the strong contrast expansion (SCE) formalism, which analytically maps $N$-point correlations of an unbounded random field to its effective properties. Since real-world material samples have finite sizes and analytical PDE kernels are not always available, we propose Neural Contrast Expansion (NCE), an SCE-inspired architecture to learn surrogate PDE kernels from structure-property data. For static conduction and electromagnetic wave propagation cases, we show that NCE models reveal accurate and insightful sensitivity information useful for material design. Compared with other PDE kernel learning methods, our method does not require measurements about the PDE solution fields, but rather only requires macroscopic property measurements that are more accessible in material development contexts.

Method: Unsupervised learning approach with three innovations: 1) pseudo coordinates for relative spatial layout and scale-invariant learning, 2) adaptive balancing of feature and geometric similarity, 3) geometric-consistent loss function for robustness. Uses tile-based post-processing with overlapping regions and majority voting for scalability.

Result: Achieves state-of-the-art accuracy in matching tasks, surpassing existing methods by a large margin, particularly in high-noise and large-scale scenarios.

Conclusion: Provides a scalable and practical solution for map alignment that is robust, efficient, and does not require training data, making it suitable for real-world large-scale applications.

Abstract: Map-to-map matching is a critical task for aligning spatial data across heterogeneous sources, yet it remains challenging due to the lack of ground truth correspondences, sparse node features, and scalability demands. In this paper, we propose an unsupervised graph-based framework that addresses these challenges through three key innovations. First, our method is an unsupervised learning approach that requires no training data, which is crucial for large-scale map data where obtaining labeled training samples is challenging. Second, we introduce pseudo coordinates that capture the relative spatial layout of nodes within each map, which enhances feature discriminability and enables scale-invariant learning. Third, we design an mechanism to adaptively balance feature and geometric similarity, as well as a geometric-consistent loss function, ensuring robustness to noisy or incomplete coordinate data. At the implementation level, to handle large-scale maps, we develop a tile-based post-processing pipeline with overlapping regions and majority voting, which enables parallel processing while preserving boundary coherence. Experiments on real-world datasets demonstrate that our method achieves state-of-the-art accuracy in matching tasks, surpassing existing methods by a large margin, particularly in high-noise and large-scale scenarios. Our framework provides a scalable and practical solution for map alignment, offering a robust and efficient alternative to traditional approaches.

Method: Compared epistemic uncertainty quantification between Gaussian Process and three NN methods (Deep Ensembles, Monte-Carlo Dropout, Stochastic Variational Inference) using an algebraic turbulence closure benchmark.

Result: GP had best accuracy (RMSE 2.14e-5) but high computational cost (O(n³)). Deep Ensembles performed well with RMSE 4.59e-4 and provided robust uncertainty estimates with better computational efficiency.

Conclusion: While GP offers superior accuracy, NN methods like Deep Ensembles provide good performance with better computational efficiency and intuitive uncertainty quantification, making them practical alternatives for turbulence closure modeling.

Abstract: Neural-Network (NN) based turbulence closures have been developed for being used as pre-trained surrogates for traditional turbulence closures, with the aim to increase computational efficiency and prediction accuracy of CFD simulations. The bottleneck to the widespread adaptation of these ML-based closures is the relative lack of uncertainty quantification (UQ) for these models. Especially, quantifying uncertainties associated with out-of-training inputs, that is when the ML-based turbulence closures are queried on inputs outside their training data regime. In the current paper, a published algebraic turbulence closure1 has been utilized to compare the quality of epistemic UQ between three NN-based methods and Gaussian Process (GP). The three NN-based methods explored are Deep Ensembles (DE), Monte-Carlo Dropout (MCD), and Stochastic Variational Inference (SVI). In the in-training results, we find the exact GP performs the best in accuracy with a Root Mean Squared Error (RMSE) of $2.14 \cdot 10^{-5}$ followed by the DE with an RMSE of $4.59 \cdot 10^{-4}$. Next, the paper discusses the performance of the four methods for quantifying out-of-training uncertainties. For performance, the Exact GP yet again is the best in performance, but has similar performance to the DE in the out-of-training regions. In UQ accuracy for the out-of-training case, SVI and DE hold the best miscalibration error for one of the cases. However, the DE performs the best in Negative Log-Likelihood for both out-of-training cases. We observe that for the current problem, in terms of accuracy GP > DE > SV I > MCD. The DE results are relatively robust and provide intuitive UQ estimates, despite performing naive ensembling. In terms of computational cost, the GP is significantly higher than the NN-based methods with a $O(n^3)$ computational complexity for each training step

Method: Tri-Accel dynamically adapts three strategies during training: precision-adaptive updates based on curvature and gradient variance, sparse second-order signals using Hessian/Fisher sparsity patterns, and memory-elastic batch scaling that adjusts batch size according to VRAM availability.

Result: Achieves up to 9.9% reduction in training time, 13.3% lower memory usage, and +1.1 percentage points accuracy improvement over FP32 baselines on CIFAR-10 with ResNet-18 and EfficientNet-B0.

Conclusion: The framework demonstrates how algorithmic adaptivity and hardware awareness can be combined to improve scalability in resource-constrained settings, making neural network training more efficient for edge devices and cost-sensitive cloud deployments.

Abstract: Deep neural networks are increasingly bottlenecked by the cost of optimization, both in terms of GPU memory and compute time. Existing acceleration techniques, such as mixed precision, second-order methods, and batch size scaling, are typically used in isolation. We present Tri-Accel, a unified optimization framework that co-adapts three acceleration strategies along with adaptive parameters during training: (1) Precision-Adaptive Updates that dynamically assign mixed-precision levels to layers based on curvature and gradient variance; (2) Sparse Second-Order Signals that exploit Hessian/Fisher sparsity patterns to guide precision and step size decisions; and (3) Memory-Elastic Batch Scaling that adjusts batch size in real time according to VRAM availability. On CIFAR-10 with ResNet-18 and EfficientNet-B0, Tri-Accel achieves up to 9.9% reduction in training time and 13.3% lower memory usage, while improving accuracy by +1.1 percentage points over FP32 baselines. Tested on CIFAR-10/100, our approach demonstrates adaptive learning behavior, with efficiency gradually improving over the course of training as the system learns to allocate resources more effectively. Compared to static mixed-precision training, Tri-Accel maintains 78.1% accuracy while reducing memory footprint from 0.35GB to 0.31GB on standard hardware. The framework is implemented with custom Triton kernels, whose hardware-aware adaptation enables automatic optimization without manual hyperparameter tuning, making it practical for deployment across diverse computational environments. This work demonstrates how algorithmic adaptivity and hardware awareness can be combined to improve scalability in resource-constrained settings, paving the way for more efficient neural network training on edge devices and cost-sensitive cloud deployments.

Method: Integrates Cortical Spiking Network with Population Coding (CSNPC) for robust classification and Reinforcement-Guided Hyper-Heuristic Optimizer for Spiking Systems (RHOSS) using Q-learning for hyperparameter optimization under fairness constraints. Includes XAI techniques (saliency-based attribution and spike activity profiling) within Modular Supervisory Framework.

Result: Achieves 90.8% recall at 5% FPR on BAF dataset, outperforming state-of-the-art models while maintaining over 98% predictive equality across demographic attributes. Saliency attributions align with spiking dynamics.

Conclusion: Demonstrates effective combination of population-coded SNNs with reinforcement-guided hyper-heuristics for fair, transparent, high-performance fraud detection in financial applications.

Abstract: The growing adoption of home banking systems has heightened the risk of cyberfraud, necessitating fraud detection mechanisms that are not only accurate but also fair and explainable. While AI models have shown promise in this domain, they face key limitations, including computational inefficiency, the interpretability challenges of spiking neural networks (SNNs), and the complexity and convergence instability of hyper-heuristic reinforcement learning (RL)-based hyperparameter optimization. To address these issues, we propose a novel framework that integrates a Cortical Spiking Network with Population Coding (CSNPC) and a Reinforcement-Guided Hyper-Heuristic Optimizer for Spiking Systems (RHOSS). The CSNPC, a biologically inspired SNN, employs population coding for robust classification, while RHOSS uses Q-learning to dynamically select low-level heuristics for hyperparameter optimization under fairness and recall constraints. Embedded within the Modular Supervisory Framework for Spiking Network Training and Interpretation (MoSSTI), the system incorporates explainable AI (XAI) techniques, specifically, saliency-based attribution and spike activity profiling, to increase transparency. Evaluated on the Bank Account Fraud (BAF) dataset suite, our model achieves a $90.8%$ recall at a strict $5%$ false positive rate (FPR), outperforming state-of-the-art spiking and non-spiking models while maintaining over $98%$ predictive equality across key demographic attributes. The explainability module further confirms that saliency attributions align with spiking dynamics, validating interpretability. These results demonstrate the potential of combining population-coded SNNs with reinforcement-guided hyper-heuristics for fair, transparent, and high-performance fraud detection in real-world financial applications.

Method: Analyzed attention output geometry across diverse models and datasets, identified low-rank structure induced by attention projection matrices. Proposed subspace-constrained training for sparse autoencoders that initializes feature directions into the active subspace of activations.

Result: Found that attention outputs are confined to low-dimensional subspaces (60% directions account for 99% variance). Subspace-constrained training reduced dead features from 87% to below 1% in Attention Output SAEs with 1M features.

Conclusion: Attention outputs have intrinsic low-dimensional geometry, which is fundamental to dead feature problems. Subspace-constrained initialization provides practical solution for improving sparse dictionary learning in LLMs, with broader applicability to other methods.

Abstract: While transformer models are widely believed to operate in high-dimensional hidden spaces, we show that attention outputs are confined to a surprisingly low-dimensional subspace, where about 60% of the directions account for 99% of the variance–a phenomenon that is induced by the attention output projection matrix and consistently observed across diverse model families and datasets. Critically, we find this low-rank structure as a fundamental cause of the prevalent dead feature problem in sparse dictionary learning, where it creates a mismatch between randomly initialized features and the intrinsic geometry of the activation space. Building on this insight, we propose a subspace-constrained training method for sparse autoencoders (SAEs), initializing feature directions into the active subspace of activations. Our approach reduces dead features from 87% to below 1% in Attention Output SAEs with 1M features, and can further extend to other sparse dictionary learning methods. Our findings provide both new insights into the geometry of attention and practical tools for improving sparse dictionary learning in large language models.

Method: Proposes DUFL with an innovative local data update scheme using three parameters: server payment, outdated data conservation rate, and fresh data collection volume. Uses a two-stage Stackelberg game with dynamic constraints to derive optimal strategies.

Result: Experimental results on real-world datasets demonstrate significant performance improvements in handling data staleness while maintaining data volume considerations.

Conclusion: DUFL effectively coordinates data staleness and volume through its incentive mechanism and DoS metric, providing optimal update strategies for both clients and server in federated learning environments.

Abstract: Handling data staleness remains a significant challenge in federated learning with highly time-sensitive tasks, where data is generated continuously and data staleness largely affects model performance. Although recent works attempt to optimize data staleness by determining local data update frequency or client selection strategy, none of them explore taking both data staleness and data volume into consideration. In this paper, we propose DUFL(Data Updating in Federated Learning), an incentive mechanism featuring an innovative local data update scheme manipulated by three knobs: the server’s payment, outdated data conservation rate, and clients’ fresh data collection volume, to coordinate staleness and volume of local data for best utilities. To this end, we introduce a novel metric called DoS(the Degree of Staleness) to quantify data staleness and conduct a theoretic analysis illustrating the quantitative relationship between DoS and model performance. We model DUFL as a two-stage Stackelberg game with dynamic constraint, deriving the optimal local data update strategy for each client in closed-form and the approximately optimal strategy for the server. Experimental results on real-world datasets demonstrate the significant performance of our approach.

Method: Sig-DEG uses partial signatures to summarize Brownian motion over sub-intervals and adopts a recurrent structure for accurate global SDE approximation. It’s trained via supervised learning to match fine-resolution diffusion outputs on a coarse time grid.

Result: The method reduces inference steps by an order of magnitude while achieving competitive generation quality compared to original diffusion models.

Conclusion: Signature-based approximations are effective for efficient generative modeling, enabling fast generation without requiring fine-grained Brownian paths during inference.

Abstract: Diffusion models have achieved state-of-the-art results in generative modelling but remain computationally intensive at inference time, often requiring thousands of discretization steps. To this end, we propose Sig-DEG (Signature-based Differential Equation Generator), a novel generator for distilling pre-trained diffusion models, which can universally approximate the backward diffusion process at a coarse temporal resolution. Inspired by high-order approximations of stochastic differential equations (SDEs), Sig-DEG leverages partial signatures to efficiently summarize Brownian motion over sub-intervals and adopts a recurrent structure to enable accurate global approximation of the SDE solution. Distillation is formulated as a supervised learning task, where Sig-DEG is trained to match the outputs of a fine-resolution diffusion model on a coarse time grid. During inference, Sig-DEG enables fast generation, as the partial signature terms can be simulated exactly without requiring fine-grained Brownian paths. Experiments demonstrate that Sig-DEG achieves competitive generation quality while reducing the number of inference steps by an order of magnitude. Our results highlight the effectiveness of signature-based approximations for efficient generative modeling.

Method: RuscaRL uses checklist-style rubrics as explicit scaffolding during rollout generation to steer diverse high-quality responses, with gradual decay of guidance. It also uses rubrics as references for verifiable rewards during model training, enabling effective RL on general reasoning tasks.

Result: Extensive experiments show RuscaRL’s superiority across benchmarks, boosting Qwen-2.5-7B-Instruct from 23.6 to 50.3 on HealthBench-500 (surpassing GPT-4.1) and achieving 61.1 with Qwen3-30B-A3B-Instruct, outperforming leading LLMs including OpenAI-o3.

Conclusion: RuscaRL effectively breaks the exploration bottleneck for LLM reasoning through instructional scaffolding, enabling significant performance improvements and expanding reasoning boundaries under best-of-N evaluation.

Abstract: Recent advances in Large Language Models (LLMs) have underscored the potential of Reinforcement Learning (RL) to facilitate the emergence of reasoning capabilities. Despite the encouraging results, a fundamental dilemma persists as RL improvement relies on learning from high-quality samples, yet the exploration for such samples remains bounded by the inherent limitations of LLMs. This, in effect, creates an undesirable cycle in which what cannot be explored cannot be learned. In this work, we propose Rubric-Scaffolded Reinforcement Learning (RuscaRL), a novel instructional scaffolding framework designed to break the exploration bottleneck for general LLM reasoning. Specifically, RuscaRL introduces checklist-style rubrics as (1) explicit scaffolding for exploration during rollout generation, where different rubrics are provided as external guidance within task instructions to steer diverse high-quality responses. This guidance is gradually decayed over time, encouraging the model to internalize the underlying reasoning patterns; (2) verifiable rewards for exploitation during model training, where we can obtain robust LLM-as-a-Judge scores using rubrics as references, enabling effective RL on general reasoning tasks. Extensive experiments demonstrate the superiority of the proposed RuscaRL across various benchmarks, effectively expanding reasoning boundaries under the best-of-N evaluation. Notably, RuscaRL significantly boosts Qwen-2.5-7B-Instruct from 23.6 to 50.3 on HealthBench-500, surpassing GPT-4.1. Furthermore, our fine-tuned variant on Qwen3-30B-A3B-Instruct achieves 61.1 on HealthBench-500, outperforming leading LLMs including OpenAI-o3.

Method: PSI multiplies three calibrated components: geometric cluster quality (S), alignment to labeled categories (Q), and open-vocabulary semantic distinctness via CLIP (D). Tested on ResNet-50 with Tiny-ImageNet.

Result: PSI identifies neurons with coherent, nameable prototypes and reveals higher polysemanticity in later layers. Patch-swap interventions show aligned replacements increase target-neuron activation significantly more than controls.

Conclusion: PSI provides a principled and practical method for discovering, quantifying, and studying polysemantic units in neural networks.

Abstract: Neural networks often contain polysemantic neurons that respond to multiple, sometimes unrelated, features, complicating mechanistic interpretability. We introduce the Polysemanticity Index (PSI), a null-calibrated metric that quantifies when a neuron’s top activations decompose into semantically distinct clusters. PSI multiplies three independently calibrated components: geometric cluster quality (S), alignment to labeled categories (Q), and open-vocabulary semantic distinctness via CLIP (D). On a pretrained ResNet-50 evaluated with Tiny-ImageNet images, PSI identifies neurons whose activation sets split into coherent, nameable prototypes, and reveals strong depth trends: later layers exhibit substantially higher PSI than earlier layers. We validate our approach with robustness checks (varying hyperparameters, random seeds, and cross-encoder text heads), breadth analyses (comparing class-only vs. open-vocabulary concepts), and causal patch-swap interventions. In particular, aligned patch replacements increase target-neuron activation significantly more than non-aligned, random, shuffled-position, or ablate-elsewhere controls. PSI thus offers a principled and practical lever for discovering, quantifying, and studying polysemantic units in neural networks.

Method: Proposed activation steering methodology to characterize reflection instructions (no reflection, intrinsic reflection, triggered reflection). Constructed steering vectors between reflection levels and performed interventions on GSM8k-adv with Qwen2.5-3B and Gemma3-4B models.

Result: Clear stratification across reflection levels was revealed. Reflection can be systematically enhanced or suppressed through activation interventions, with suppression being considerably easier than stimulation.

Conclusion: The work demonstrates controllability of reflection and highlights both opportunities (reflection-enhancing defenses) and risks (adversarial inhibition in jailbreak attacks), opening a path toward mechanistic understanding of reflective reasoning in LLMs.

Abstract: Reflection, the ability of large language models (LLMs) to evaluate and revise their own reasoning, has been widely used to improve performance on complex reasoning tasks. Yet, most prior work emphasizes designing reflective prompting strategies or reinforcement learning objectives, leaving the inner mechanisms of reflection underexplored. In this paper, we investigate reflection through the lens of latent directions in model activations. We propose a methodology based on activation steering to characterize how instructions with different reflective intentions: no reflection, intrinsic reflection, and triggered reflection. By constructing steering vectors between these reflection levels, we demonstrate that (1) new reflection-inducing instructions can be systematically identified, (2) reflective behavior can be directly enhanced or suppressed through activation interventions, and (3) suppressing reflection is considerably easier than stimulating it. Experiments on GSM8k-adv with Qwen2.5-3B and Gemma3-4B reveal clear stratification across reflection levels, and steering interventions confirm the controllability of reflection. Our findings highlight both opportunities (e.g., reflection-enhancing defenses) and risks (e.g., adversarial inhibition of reflection in jailbreak attacks). This work opens a path toward mechanistic understanding of reflective reasoning in LLMs.

Method: Proposed efficient first-order online algorithm that works in both full-information and bandit feedback settings, handling adversarial inputs while maintaining budget constraints.

Result: Algorithm guarantees O(√T) α-regret against optimal fixed feasible benchmark while consuming at most O(B_T log T) + Õ(√T) resources. Provides improved guarantees for Adversarial Bandits with Knapsacks problem in bandit setting.

Conclusion: Matching lower bounds prove tightness of results. The framework applies broadly to α-approximately convex functions, encompassing many common non-convex optimization problems with theoretical guarantees.

Abstract: We study an online learning problem with long-term budget constraints in the adversarial setting. In this problem, at each round $t$, the learner selects an action from a convex decision set, after which the adversary reveals a cost function $f_t$ and a resource consumption function $g_t$. The cost and consumption functions are assumed to be $\alpha$-approximately convex - a broad class that generalizes convexity and encompasses many common non-convex optimization problems, including DR-submodular maximization, Online Vertex Cover, and Regularized Phase Retrieval. The goal is to design an online algorithm that minimizes cumulative cost over a horizon of length $T$ while approximately satisfying a long-term budget constraint of $B_T$. We propose an efficient first-order online algorithm that guarantees $O(\sqrt{T})$ $\alpha$-regret against the optimal fixed feasible benchmark while consuming at most $O(B_T \log T)+ \tilde{O}(\sqrt{T})$ resources in both full-information and bandit feedback settings. In the bandit feedback setting, our approach yields an efficient solution for the $\texttt{Adversarial Bandits with Knapsacks}$ problem with improved guarantees. We also prove matching lower bounds, demonstrating the tightness of our results. Finally, we characterize the class of $\alpha$-approximately convex functions and show that our results apply to a broad family of problems.

Method: Mechanistic analysis of CARTRIDGE structure, empirical testing across tasks and model sizes, and proposing Sampled Chunk Initialization (SCI) for improved training convergence.

Result: Found that CARTRIDGE keys serve as stable retrieval routers, while values handle most compression. SCI enables faster convergence than previous methods.

Conclusion: Provides foundational understanding of CARTRIDGE mechanisms and suggests SCI optimization, paving way for further scaling of long-context LLM inference.

Abstract: A bottleneck for long-context LLM inference is the linearly growing KV cache. Recent work has proposed CARTRIDGES, an approach which leverages offline compute to train a much smaller KV cache than is typically required for a full document (up to 40x less memory usage at inference time). In this paper, we present the first mechanistic exploration of the learned CARTRIDGE key-value cache structure. In particular, we propose that (1) CARTRIDGE keys act as stable, shareable retrieval routers for the compressed corpora and (2) most of the learned compression occurs within the CARTRIDGE value vectors. We present empirical evidence of our routing theory across tasks, model families, and model sizes; for example, we can ablate the learned CARTRIDGE key vectors between tasks with little performance loss. Finally, we propose a slight improvement in initialization called Sampled Chunk Initialization (SCI). We suggest that SCI can lead to faster CARTRIDGE convergence than previously demonstrated in the literature. Our findings lay the groundwork for broader empirical study of CARTRIDGE training optimization which may be crucial for further scaling.

Method: TabResFlow uses three components: (1) MLP encoder for numerical features, (2) fully connected ResNet backbone for feature extraction, and (3) conditional spline-based normalizing flow for flexible density estimation.

Result: TabResFlow achieves 9.64% improvement over TreeFlow (strongest probabilistic model) and 5.6x speed-up in inference time compared to NodeFlow. It also shows superior performance in real-world used car price prediction with novel AURC metric.

Conclusion: TabResFlow provides a flexible and efficient solution for probabilistic tabular regression, addressing limitations of conventional methods and demonstrating practical value in industrial applications requiring trustworthy uncertainty estimation.

Abstract: Tabular regression is a well-studied problem with numerous industrial applications, yet most existing approaches focus on point estimation, often leading to overconfident predictions. This issue is particularly critical in industrial automation, where trustworthy decision-making is essential. Probabilistic regression models address this challenge by modeling prediction uncertainty. However, many conventional methods assume a fixed-shape distribution (typically Gaussian), and resort to estimating distribution parameters. This assumption is often restrictive, as real-world target distributions can be highly complex. To overcome this limitation, we introduce TabResFlow, a Normalizing Spline Flow model designed specifically for univariate tabular regression, where commonly used simple flow networks like RealNVP and Masked Autoregressive Flow (MAF) are unsuitable. TabResFlow consists of three key components: (1) An MLP encoder for each numerical feature. (2) A fully connected ResNet backbone for expressive feature extraction. (3) A conditional spline-based normalizing flow for flexible and tractable density estimation. We evaluate TabResFlow on nine public benchmark datasets, demonstrating that it consistently surpasses existing probabilistic regression models on likelihood scores. Our results demonstrate 9.64% improvement compared to the strongest probabilistic regression model (TreeFlow), and on average 5.6 times speed-up in inference time compared to the strongest deep learning alternative (NodeFlow). Additionally, we validate the practical applicability of TabResFlow in a real-world used car price prediction task under selective regression. To measure performance in this setting, we introduce a novel Area Under Risk Coverage (AURC) metric and show that TabResFlow achieves superior results across this metric.

Method: Developed three main methods: 1) ComBI (Compressed BST of Inverted hash tables) for efficient nearest-neighbor search, 2) GRAF (Guided Random Forest) for classification with global/local partitioning, and 3) CRCS (Continuous Representation of Codon Switches) deep learning framework for genetic mutation analysis.

Result: ComBI achieved 0.90 precision with 4X-296X speed-ups on billion-sample datasets; GRAF delivered competitive accuracy across 115 datasets; CRCS enabled somatic mutation identification and survival prediction validated in multiple cancers.

Conclusion: The developed methods provide efficient, scalable, and interpretable tools for large-scale data analysis and biomedical applications, with demonstrated performance improvements and practical cancer genomics applications.

Abstract: This thesis develops computational methods in similarity-preserving hashing, classification, and cancer genomics. Standard space partitioning-based hashing relies on Binary Search Trees (BSTs), but their exponential growth and sparsity hinder efficiency. To overcome this, we introduce Compressed BST of Inverted hash tables (ComBI), which enables fast approximate nearest-neighbor search with reduced memory. On datasets of up to one billion samples, ComBI achieves 0.90 precision with 4X-296X speed-ups over Multi-Index Hashing, and also outperforms Cellfishing.jl on single-cell RNA-seq searches with 2X-13X gains. Building on hashing structures, we propose Guided Random Forest (GRAF), a tree-based ensemble classifier that integrates global and local partitioning, bridging decision trees and boosting while reducing generalization error. Across 115 datasets, GRAF delivers competitive or superior accuracy, and its unsupervised variant (uGRAF) supports guided hashing and importance sampling. We show that GRAF and ComBI can be used to estimate per-sample classifiability, which enables scalable prediction of cancer patient survival. To address challenges in interpreting mutations, we introduce Continuous Representation of Codon Switches (CRCS), a deep learning framework that embeds genetic changes into numerical vectors. CRCS allows identification of somatic mutations without matched normals, discovery of driver genes, and scoring of tumor mutations, with survival prediction validated in bladder, liver, and brain cancers. Together, these methods provide efficient, scalable, and interpretable tools for large-scale data analysis and biomedical applications.

Method: Investigated three CNN architectures (single-branch 2D, single-branch 1D, and multiple-branch models) and compared them with Adaptive Kalman Filter using simulated train operation datasets with and without Wheel Slide Protection activation.

Result: CNN-based approaches demonstrated superior accuracy and robustness compared to traditional methods, with the multiple-branch model performing best, particularly under challenging operational conditions.

Conclusion: Deep learning techniques have strong potential to enhance railway safety and operational efficiency by effectively capturing intricate patterns in complex transportation datasets.

Abstract: In this study, we explore the use of Convolutional Neural Networks for improving train speed estimation accuracy, addressing the complex challenges of modern railway systems. We investigate three CNN architectures - single-branch 2D, single-branch 1D, and multiple-branch models - and compare them with the Adaptive Kalman Filter. We analyse their performance using simulated train operation datasets with and without Wheel Slide Protection activation. Our results reveal that CNN-based approaches, especially the multiple-branch model, demonstrate superior accuracy and robustness compared to traditional methods, particularly under challenging operational conditions. These findings highlight the potential of deep learning techniques to enhance railway safety and operational efficiency by more effectively capturing intricate patterns in complex transportation datasets.

Method: The method assumes latent rationales mapped to a probabilistic embedding space, with classifier predictions conditioned on rationale embeddings via a stochastic correlation matrix. A graph generator decodes rationale structures from embeddings for interpretability. Uses alternating optimization similar to EM algorithm and can be applied to any existing GNN architecture.

Result: Extensive experiments on five graph classification datasets show the framework outperforms state-of-the-art methods in both uncertainty quantification and GNN interpretability. Case studies demonstrate that decoded rationale structures provide meaningful explanations.

Conclusion: The proposed approach provides an effective solution for making GNNs more reliable through better uncertainty calibration and interpretable rationale extraction, enabling safer deployment in critical applications.

Abstract: Graph neural networks (GNNs) are powerful tools on graph data. However, their predictions are mis-calibrated and lack interpretability, limiting their adoption in critical applications. To address this issue, we propose a new uncertainty-aware and interpretable graph classification model that combines graph functional neural process and graph generative model. The core of our method is to assume a set of latent rationales which can be mapped to a probabilistic embedding space; the predictive distribution of the classifier is conditioned on such rationale embeddings by learning a stochastic correlation matrix. The graph generator serves to decode the graph structure of the rationales from the embedding space for model interpretability. For efficient model training, we adopt an alternating optimization procedure which mimics the well known Expectation-Maximization (EM) algorithm. The proposed method is general and can be applied to any existing GNN architecture. Extensive experiments on five graph classification datasets demonstrate that our framework outperforms state-of-the-art methods in both uncertainty quantification and GNN interpretability. We also conduct case studies to show that the decoded rationale structure can provide meaningful explanations.

Method: Proposes RoSDHB algorithm that integrates Polyak’s momentum with a new coordinated compression mechanism for Byzantine-robust distributed learning.

Result: RoSDHB performs comparably to Byz-DASHA-PAGE under standard gradient dissimilarity heterogeneity model while relying on fewer assumptions (only Lipschitz smoothness of average loss function). Empirical results show strong robustness with significant communication savings on image classification tasks.

Conclusion: RoSDHB provides an effective solution for Byzantine-robust distributed learning that achieves strong performance with reduced communication costs and fewer theoretical assumptions compared to existing state-of-the-art methods.

Abstract: Distributed learning (DL) enables scalable model training over decentralized data, but remains challenged by Byzantine faults and high communication costs. While both issues have been studied extensively in isolation, their interaction is less explored. Prior work shows that naively combining communication compression with Byzantine-robust aggregation degrades resilience to faulty nodes (or workers). The state-of-the-art algorithm, namely Byz-DASHA-PAGE [29], makes use of the momentum variance reduction scheme to mitigate the detrimental impact of compression noise on Byzantine-robustness. We propose a new algorithm, named RoSDHB, that integrates the classic Polyak’s momentum with a new coordinated compression mechanism. We show that RoSDHB performs comparably to Byz-DASHA-PAGE under the standard (G, B)-gradient dissimilarity heterogeneity model, while it relies on fewer assumptions. In particular, we only assume Lipschitz smoothness of the average loss function of the honest workers, in contrast to [29]that additionally assumes a special smoothness of bounded global Hessian variance. Empirical results on benchmark image classification task show that RoSDHB achieves strong robustness with significant communication savings.

Method: Framed as multi-label sequence prediction problem, trained lightweight transformer on 66M expert activation traces from LDJnr-Puffin dataset using DeepSeek-V2-Chat-Lite MoE.

Result: Achieved 97.5% accuracy and 86.6% F1-score on unseen prompts from WebGLM-QA dataset, improving GPU cache hit rate from 17% to 72% when only 10% of experts fit in GPU cache.

Conclusion: Learning-based expert activation prediction effectively addresses memory constraints for MoE models on edge devices, significantly outperforming heuristic baselines.

Abstract: The deployment of large-scale Mixture-of-Experts (MoE) models on edge devices presents significant challenges due to memory constraints. While MoE architectures enable efficient utilization of computational resources by activating only a subset of experts per inference, they require careful memory management to operate efficiently in resource-constrained environments. Traditional heuristic-based expert caching strategies such as MoE-Infinity struggle to maintain high cache hit rates as models parameters scale. In this work, we introduce MoE-Beyond, a learning-based expert activation predictor trained to predict expert activations during autoregressive decoding. By framing the task as a multi-label sequence prediction problem, we train a lightweight transformer model on 66 million expert activation traces extracted from LDJnr-Puffin dataset [5] using DeepSeek-V2-Chat-Lite MoE. Our predictor generalizes effectively across unseen prompts from WebGLM-QA dataset [6], achieving 97.5% accuracy and an 86.6% F1-score. Simulation results show that MoE-Beyond improves GPU cache hit rate from 17% to 72% when only 10% of experts fit in GPU cache, outperforming heuristic baselines.

Method: Proposed Oracle Gradient Querying (OGQ) as ideal benchmark and practical Strategic Gradient Querying (SGQ) that selects one user’s gradient per iteration based on expected improvement without requiring oracle access.

Result: Theoretical analysis shows OGQ enhances transient-state performance and reduces steady-state variance under Polyak-Lojasiewicz condition. SGQ improves transient-state performance over SGD while maintaining practical feasibility.

Conclusion: Strategic gradient querying provides significant benefits over uniform querying, with both theoretical guarantees and practical algorithms that outperform standard stochastic gradient methods.

Abstract: This paper considers a finite-sum optimization problem under first-order queries and investigates the benefits of strategic querying on stochastic gradient-based methods compared to uniform querying strategy. We first introduce Oracle Gradient Querying (OGQ), an idealized algorithm that selects one user’s gradient yielding the largest possible expected improvement (EI) at each step. However, OGQ assumes oracle access to the gradients of all users to make such a selection, which is impractical in real-world scenarios. To address this limitation, we propose Strategic Gradient Querying (SGQ), a practical algorithm that has better transient-state performance than SGD while making only one query per iteration. For smooth objective functions satisfying the Polyak-Lojasiewicz condition, we show that under the assumption of EI heterogeneity, OGQ enhances transient-state performance and reduces steady-state variance, while SGQ improves transient-state performance over SGD. Our numerical experiments validate our theoretical findings.

Method: Algorithm operates initially without user parameters, computes threshold to filter sparse points and outliers, forms preliminary clusters, then reintegrates excluded points. Uses single integer parameter only if adjustment is needed.

Result: Experimental evaluations on diverse datasets show the method provides accessible and robust performance for density-based clustering tasks.

Conclusion: The proposed method offers an effective alternative to traditional density-based clustering by minimizing parameter dependency while maintaining robust performance.

Abstract: Clustering algorithms are widely used in various applications, with density-based methods such as Density-Based Spatial Clustering of Applications with Noise (DBSCAN) being particularly prominent. These algorithms identify clusters in high-density regions while treating sparser areas as noise. However, reliance on user-defined parameters often poses optimization challenges that require domain expertise. This paper presents a novel density-based clustering method inspired by the concept of selective attention, which minimizes the need for user-defined parameters under standard conditions. Initially, the algorithm operates without requiring user-defined parameters. If parameter adjustment is needed, the method simplifies the process by introducing a single integer parameter that is straightforward to tune. The approach computes a threshold to filter out the most sparsely distributed points and outliers, forms a preliminary cluster structure, and then reintegrates the excluded points to finalize the results. Experimental evaluations on diverse data sets highlight the accessibility and robust performance of the method, providing an effective alternative for density-based clustering tasks.

Method: Introduces CIFR benchmark with diverse cipher encodings, evaluates defense strategies, and trains probe monitors on model internal activations from multiple fine-tunes.

Result: Probe monitors achieve over 99% detection accuracy, generalize to unseen cipher variants and families, and outperform state-of-the-art monitoring approaches.

Conclusion: The CIFR benchmark and probe monitoring approach provide effective defense against cipher-enabled attacks while maintaining fine-tuning functionality, with code and data made available for further research.

Abstract: Large language model fine-tuning APIs enable widespread model customization, yet pose significant safety risks. Recent work shows that adversaries can exploit access to these APIs to bypass model safety mechanisms by encoding harmful content in seemingly harmless fine-tuning data, evading both human monitoring and standard content filters. We formalize the fine-tuning API defense problem, and introduce the Cipher Fine-tuning Robustness benchmark (CIFR), a benchmark for evaluating defense strategies’ ability to retain model safety in the face of cipher-enabled attackers while achieving the desired level of fine-tuning functionality. We include diverse cipher encodings and families, with some kept exclusively in the test set to evaluate for generalization across unseen ciphers and cipher families. We then evaluate different defenses on the benchmark and train probe monitors on model internal activations from multiple fine-tunes. We show that probe monitors achieve over 99% detection accuracy, generalize to unseen cipher variants and families, and compare favorably to state-of-the-art monitoring approaches. We open-source CIFR and the code to reproduce our experiments to facilitate further research in this critical area. Code and data are available online https://github.com/JackYoustra/safe-finetuning-api

Method: ONG preconditions new task gradients with an efficient EKFAC approximation of the inverse Fisher information matrix, then projects these natural gradients onto the orthogonal complement of prior task gradients to preserve previous knowledge.

Result: The method was benchmarked on Permuted and Rotated MNIST datasets, with code available for reproducibility.

Conclusion: ONG provides theoretically justified updates that follow steepest descent under Riemannian metric while maintaining performance on previously learned tasks in continual learning scenarios.

Abstract: Orthogonal gradient descent has emerged as a powerful method for continual learning tasks. However, its Euclidean projections overlook the underlying information-geometric structure of the space of distributions parametrized by neural networks, which can lead to suboptimal convergence in learning tasks. To counteract this, we combine it with the idea of the natural gradient and present ONG (Orthogonal Natural Gradient Descent). ONG preconditions each new task gradient with an efficient EKFAC approximation of the inverse Fisher information matrix, yielding updates that follow the steepest descent direction under a Riemannian metric. To preserve performance on previously learned tasks, ONG projects these natural gradients onto the orthogonal complement of prior task gradients. We provide a theoretical justification for this procedure, introduce the ONG algorithm, and benchmark its performance on the Permuted and Rotated MNIST datasets. All code for our experiments/reproducibility can be found at https://github.com/yajatyadav/orthogonal-natural-gradient.

Method: Sharpness-aware Geometric Defense (SaGD) framework that uses jitter-based perturbation in adversarial training to smooth the rugged loss landscape and improve geometric embedding convergence.

Result: Significantly improves FPR and AUC over state-of-the-art defense approaches in differentiating CIFAR-100 from six OOD datasets under various attacks.

Conclusion: The framework successfully addresses the challenge of distinguishing adversarial ID samples from true OOD samples and reveals the relationship between sharp loss landscape and adversarial OOD detection.

Abstract: Out-of-distribution (OOD) detection ensures safe and reliable model deployment. Contemporary OOD algorithms using geometry projection can detect OOD or adversarial samples from clean in-distribution (ID) samples. However, this setting regards adversarial ID samples as OOD, leading to incorrect OOD predictions. Existing efforts on OOD detection with ID and OOD data under attacks are minimal. In this paper, we develop a robust OOD detection method that distinguishes adversarial ID samples from OOD ones. The sharp loss landscape created by adversarial training hinders model convergence, impacting the latent embedding quality for OOD score calculation. Therefore, we introduce a {\bf Sharpness-aware Geometric Defense (SaGD)} framework to smooth out the rugged adversarial loss landscape in the projected latent geometry. Enhanced geometric embedding convergence enables accurate ID data characterization, benefiting OOD detection against adversarial attacks. We use Jitter-based perturbation in adversarial training to extend the defense ability against unseen attacks. Our SaGD framework significantly improves FPR and AUC over the state-of-the-art defense approaches in differentiating CIFAR-100 from six other OOD datasets under various attacks. We further examine the effects of perturbations at various adversarial training levels, revealing the relationship between the sharp loss landscape and adversarial OOD detection.

Method: Comparative analysis of dense and sparse attention mechanisms in graph transformers, examining their respective trade-offs, performance characteristics, and appropriate use cases.

Result: The paper provides insights into when to use dense vs sparse attention in graph transformers, highlighting the strengths and weaknesses of each approach for different graph structures and tasks.

Conclusion: Both dense and sparse attention have distinct advantages in graph transformers, and the choice depends on specific application requirements. The paper also identifies current challenges in attention design for graph transformers.

Abstract: Graphs have become a central representation in machine learning for capturing relational and structured data across various domains. Traditional graph neural networks often struggle to capture long-range dependencies between nodes due to their local structure. Graph transformers overcome this by using attention mechanisms that allow nodes to exchange information globally. However, there are two types of attention in graph transformers: dense and sparse. In this paper, we compare these two attention mechanisms, analyze their trade-offs, and highlight when to use each. We also outline current challenges and problems in designing attention for graph transformers.

Method: Used open-sourced Llama 3.2 models fine-tuned on human annotated assertiveness datasets, extracted residual activations across all layers, computed similarity metrics to localize assertive representations, and derived steering vectors from identified sub-components.

Result: Identified layers most sensitive to assertiveness contrasts, revealed that high-assertive representations decompose into orthogonal emotional and logical clusters (paralleling dual-route Elaboration Likelihood Model), and showed emotional vectors broadly influence prediction accuracy while logical vectors exert localized effects.

Conclusion: Provides mechanistic evidence for multi-component structure of LLM assertiveness and highlights avenues for mitigating overconfident behavior through targeted interventions.

Abstract: Large Language Models (LLMs) often display overconfidence, presenting information with unwarranted certainty in high-stakes contexts. We investigate the internal basis of this behavior via mechanistic interpretability. Using open-sourced Llama 3.2 models fine-tuned on human annotated assertiveness datasets, we extract residual activations across all layers, and compute similarity metrics to localize assertive representations. Our analysis identifies layers most sensitive to assertiveness contrasts and reveals that high-assertive representations decompose into two orthogonal sub-components of emotional and logical clusters-paralleling the dual-route Elaboration Likelihood Model in Psychology. Steering vectors derived from these sub-components show distinct causal effects: emotional vectors broadly influence prediction accuracy, while logical vectors exert more localized effects. These findings provide mechanistic evidence for the multi-component structure of LLM assertiveness and highlight avenues for mitigating overconfident behavior.

Method: Periodic insertion of control tokens during inference to inform model of remaining token budget, combined with two-stage training: SFT for budget constraint familiarity followed by curriculum RL with length-aware reward function.

Result: Significantly outperforms strong baselines in maintaining performance across various reasoning budgets on challenging mathematical benchmarks.

Conclusion: Provides scalable and effective solution for efficient and controllable LLM reasoning, making advanced models more practical for resource-constrained and real-time deployment.

Abstract: Recent advancements in Large Language Models (LLMs) have leveraged increased test-time computation to enhance reasoning capabilities, a strategy that, while effective, incurs significant latency and resource costs, limiting their applicability in real-world time-constrained or cost-sensitive scenarios. This paper introduces BudgetThinker, a novel framework designed to empower LLMs with budget-aware reasoning, enabling precise control over the length of their thought processes. We propose a methodology that periodically inserts special control tokens during inference to continuously inform the model of its remaining token budget. This approach is coupled with a comprehensive two-stage training pipeline, beginning with Supervised Fine-Tuning (SFT) to familiarize the model with budget constraints, followed by a curriculum-based Reinforcement Learning (RL) phase that utilizes a length-aware reward function to optimize for both accuracy and budget adherence. We demonstrate that BudgetThinker significantly surpasses strong baselines in maintaining performance across a variety of reasoning budgets on challenging mathematical benchmarks. Our method provides a scalable and effective solution for developing efficient and controllable LLM reasoning, making advanced models more practical for deployment in resource-constrained and real-time environments.

Method: Proposes Cross-Modal Conflict Injection (CMCI) to embed subtle semantic contradictions between medical images and paired reports, creating plausible but disruptive adversarial pairs in a simulated semi-open attack environment.

Result: Evaluation on IU-Xray and MIMIC-CXR QA tasks shows F1 score reductions up to 27.66%, with LLaVA-Med-1.5 F1 rates dropping to as low as 51.36%. CMCI attacks cause the most severe degradation compared to basic textual/visual attacks.

Conclusion: Reveals fundamental security gaps in clinical RAG systems and provides guidelines for threat-aware design and robust multimodal consistency checks to ensure safe development of future medical RAG systems.

Abstract: Large Vision-Language Models (LVLMs) augmented with Retrieval-Augmented Generation (RAG) are increasingly employed in medical AI to enhance factual grounding through external clinical image-text retrieval. However, this reliance creates a significant attack surface. We propose MedThreatRAG, a novel multimodal poisoning framework that systematically probes vulnerabilities in medical RAG systems by injecting adversarial image-text pairs. A key innovation of our approach is the construction of a simulated semi-open attack environment, mimicking real-world medical systems that permit periodic knowledge base updates via user or pipeline contributions. Within this setting, we introduce and emphasize Cross-Modal Conflict Injection (CMCI), which embeds subtle semantic contradictions between medical images and their paired reports. These mismatches degrade retrieval and generation by disrupting cross-modal alignment while remaining sufficiently plausible to evade conventional filters. While basic textual and visual attacks are included for completeness, CMCI demonstrates the most severe degradation. Evaluations on IU-Xray and MIMIC-CXR QA tasks show that MedThreatRAG reduces answer F1 scores by up to 27.66% and lowers LLaVA-Med-1.5 F1 rates to as low as 51.36%. Our findings expose fundamental security gaps in clinical RAG systems and highlight the urgent need for threat-aware design and robust multimodal consistency checks. Finally, we conclude with a concise set of guidelines to inform the safe development of future multimodal medical RAG systems.

Method: Integrates decision Transformer with Generalized Policy Gradient framework to model long-term dependencies in stochastic transportation networks.

Result: Experimental results on Sioux Falls Network show improved accuracy and stability in path decisions, with higher on-time arrival probability compared to previous baselines.

Conclusion: The proposed solution provides more accurate and reliable path planning by effectively handling stochastic dependencies in transportation networks.

Abstract: With the rapidly increased number of vehicles in urban areas, existing road infrastructure struggles to accommodate modern traffic demands, resulting in the issue of congestion. This highlights the importance of efficient path planning strategies. However, most recent navigation models focus solely on deterministic or time-dependent networks, while overlooking the correlations and the stochastic nature of traffic flows. In this work, we address the reliable shortest path problem within stochastic transportation networks under certain dependencies. We propose a path planning solution that integrates the decision Transformer with the Generalized Policy Gradient (GPG) framework. Based on the decision Transformer’s capability to model long-term dependencies, our proposed solution improves the accuracy and stability of path decisions. Experimental results on the Sioux Falls Network (SFN) demonstrate that our approach outperforms previous baselines in terms of on-time arrival probability, providing more accurate path planning solutions.