Method: Provides comprehensive categorization of benchmarks and task setups, groups modeling strategies by challenges they target, and analyzes their strengths and limitations.

Result: Offers a structured overview of TQA research with focus on LLM-based methods, unifying disparate research threads and identifying open problems.

Conclusion: The survey provides a consolidated foundation for the TQA community, enabling deeper understanding of state of the art and guiding future developments in this rapidly evolving area.

Abstract: Table Question Answering (TQA) aims to answer natural language questions about tabular data, often accompanied by additional contexts such as text passages. The task spans diverse settings, varying in table representation, question/answer complexity, modality involved, and domain. While recent advances in large language models (LLMs) have led to substantial progress in TQA, the field still lacks a systematic organization and understanding of task formulations, core challenges, and methodological trends, particularly in light of emerging research directions such as reinforcement learning. This survey addresses this gap by providing a comprehensive and structured overview of TQA research with a focus on LLM-based methods. We provide a comprehensive categorization of existing benchmarks and task setups. We group current modeling strategies according to the challenges they target, and analyze their strengths and limitations. Furthermore, we highlight underexplored but timely topics that have not been systematically covered in prior research. By unifying disparate research threads and identifying open problems, our survey offers a consolidated foundation for the TQA community, enabling a deeper understanding of the state of the art and guiding future developments in this rapidly evolving area.

Method: Proposes multi-token prediction (MTP) loss for S2UT models to predict multiple subsequent tokens per position. Introduces MTP-S2UT variant that applies MTP loss to intermediate layers where CTC loss is computed, enabling earlier information enrichment.

Result: All MTP loss variants consistently improve S2UT translation quality. MTP-S2UT achieves the best performance among the tested approaches.

Conclusion: Applying multi-token prediction loss, particularly at intermediate layers (MTP-S2UT), effectively enhances semantic density and improves speech-to-unit translation performance by enabling earlier and more effective information enrichment.

Abstract: Current direct speech-to-speech translation methods predominantly employ speech tokens as intermediate representations. However, a single speech token is not dense in semantics, so we generally need multiple tokens to express a complete semantic unit. To address this limitation, we introduce multi-token prediction (MTP) loss into speech-to-unit translation (S2UT) models, enabling models to predict multiple subsequent tokens at each position, thereby capturing more complete semantics and enhancing information density per position. Initial MTP implementations apply the loss at the final layer, which improves output representation but initiates information enrichment too late. We hypothesize that advancing the information enrichment process to intermediate layers can achieve earlier and more effective enhancement of hidden representation. Consequently, we propose MTP-S2UT loss, applying MTP loss to hidden representation where CTC loss is computed. Experiments demonstrate that all MTP loss variants consistently improve the quality of S2UT translation, with MTP-S2UT achieving the best performance.

Method: Benchmarked eight LLMs to inject emotional appeal into fallacious arguments while preserving logical structures, then used best models to generate stimuli for human study.

Result: LLM-driven emotional framing reduced human fallacy detection by 14.5% on average. Humans detected fallacies better with enjoyment than fear/sadness. Fear, sadness, and enjoyment correlated with higher convincingness than neutral states.

Conclusion: The work reveals implications for AI-driven emotional manipulation in fallacious argumentation, showing how emotional appeals can mask logical flaws and increase persuasiveness.

Abstract: Logical fallacies are common in public communication and can mislead audiences; fallacious arguments may still appear convincing despite lacking soundness, because convincingness is inherently subjective. We present the first computational study of how emotional framing interacts with fallacies and convincingness, using large language models (LLMs) to systematically change emotional appeals in fallacious arguments. We benchmark eight LLMs on injecting emotional appeal into fallacious arguments while preserving their logical structures, then use the best models to generate stimuli for a human study. Our results show that LLM-driven emotional framing reduces human fallacy detection in F1 by 14.5% on average. Humans perform better in fallacy detection when perceiving enjoyment than fear or sadness, and these three emotions also correlate with significantly higher convincingness compared to neutral or other emotion states. Our work has implications for AI-driven emotional manipulation in the context of fallacious argumentation.

Method: Conducted experiments asking LLMs to explain patterns in various integer sequences, including arithmetic sequences and randomly generated series.

Result: LLMs successfully identified correct patterns in arithmetic/geometric sequences but frequently over-recognized inconsistent patterns in random series, even with multi-step reasoning models like OpenAI o3, o4-mini, and Google Gemini 2.5 Flash.

Conclusion: LLMs exhibit a tendency to perceive non-existent patterns (AI equivalent of Idola Tribus), highlighting limitations in logical reasoning capabilities despite chain-of-thought mechanisms.

Abstract: We present a tendency of large language models (LLMs) to generate absurd patterns despite their clear inappropriateness in a simple task of identifying regularities in number series. Several approaches have been proposed to apply LLMs to complex real-world tasks, such as providing knowledge through retrieval-augmented generation and executing multi-step tasks using AI agent frameworks. However, these approaches rely on the logical consistency and self-coherence of LLMs, making it crucial to evaluate these aspects and consider potential countermeasures. To identify cases where LLMs fail to maintain logical consistency, we conducted an experiment in which LLMs were asked to explain the patterns in various integer sequences, ranging from arithmetic sequences to randomly generated integer series. While the models successfully identified correct patterns in arithmetic and geometric sequences, they frequently over-recognized patterns that were inconsistent with the given numbers when analyzing randomly generated series. This issue was observed even in multi-step reasoning models, including OpenAI o3, o4-mini, and Google Gemini 2.5 Flash Preview Thinking. This tendency to perceive non-existent patterns can be interpreted as the AI model equivalent of Idola Tribus and highlights potential limitations in their capability for applied tasks requiring logical reasoning, even when employing chain-of-thought reasoning mechanisms.

Method: Two-stage framework: 1) Joint semantic and cluster-based filtering guided by EIRE (Entity-intent-relation extractor) to score relevance and build clean database; 2) EIRE-guided conflict-aware filtering that analyzes semantic consistency between query, answers, and retrieved knowledge before generation.

Result: Significantly outperforms state-of-the-art defense methods across various LLMs and datasets, achieving improvements in both generation robustness and output trustworthiness.

Conclusion: SeCon-RAG effectively preserves useful knowledge while mitigating conflict contamination, providing a trustworthy RAG solution that balances security and information retention.

Abstract: Retrieval-augmented generation (RAG) systems enhance large language models (LLMs) with external knowledge but are vulnerable to corpus poisoning and contamination attacks, which can compromise output integrity. Existing defenses often apply aggressive filtering, leading to unnecessary loss of valuable information and reduced reliability in generation. To address this problem, we propose a two-stage semantic filtering and conflict-free framework for trustworthy RAG. In the first stage, we perform a joint filter with semantic and cluster-based filtering which is guided by the Entity-intent-relation extractor (EIRE). EIRE extracts entities, latent objectives, and entity relations from both the user query and filtered documents, scores their semantic relevance, and selectively adds valuable documents into the clean retrieval database. In the second stage, we proposed an EIRE-guided conflict-aware filtering module, which analyzes semantic consistency between the query, candidate answers, and retrieved knowledge before final answer generation, filtering out internal and external contradictions that could mislead the model. Through this two-stage process, SeCon-RAG effectively preserves useful knowledge while mitigating conflict contamination, achieving significant improvements in both generation robustness and output trustworthiness. Extensive experiments across various LLMs and datasets demonstrate that the proposed SeCon-RAG markedly outperforms state-of-the-art defense methods.

Method: Uses residual vector quantization to discretize pretrained KG embeddings into compact code sequences, integrates them as learnable LLM tokens, and applies ontology-guided class constraints for semantic consistency.

Result: Achieves state-of-the-art performance on two benchmark datasets, demonstrating effective alignment of structured knowledge with large language models.

Conclusion: ReaLM effectively bridges the KG-LLM gap through discretization and semantic constraints, enabling superior knowledge graph completion.

Abstract: Large Language Models (LLMs) have recently emerged as a powerful paradigm for Knowledge Graph Completion (KGC), offering strong reasoning and generalization capabilities beyond traditional embedding-based approaches. However, existing LLM-based methods often struggle to fully exploit structured semantic representations, as the continuous embedding space of pretrained KG models is fundamentally misaligned with the discrete token space of LLMs. This discrepancy hinders effective semantic transfer and limits their performance. To address this challenge, we propose ReaLM, a novel and effective framework that bridges the gap between KG embeddings and LLM tokenization through the mechanism of residual vector quantization. ReaLM discretizes pretrained KG embeddings into compact code sequences and integrates them as learnable tokens within the LLM vocabulary, enabling seamless fusion of symbolic and contextual knowledge. Furthermore, we incorporate ontology-guided class constraints to enforce semantic consistency, refining entity predictions based on class-level compatibility. Extensive experiments on two widely used benchmark datasets demonstrate that ReaLM achieves state-of-the-art performance, confirming its effectiveness in aligning structured knowledge with large-scale language models.

Method: Tested 28 different ciphers by fine-tuning and prompting up to 10 models to reason in each cipher, using math problems as a proxy for reasoning ability and measuring accuracy.

Result: Found an asymmetry: models can translate ciphered text accurately but struggle with reasoning in it, especially with lesser-known ciphers. Ciphered reasoning capability correlates with cipher prevalence in pretraining data and improves slowly with fine-tuning.

Conclusion: Ciphered reasoning may be an ineffective evasion tactic for current models, but provides guidance for constraining this capability in future frontier models.

Abstract: Detecting harmful AI actions is important as AI agents gain adoption. Chain-of-thought (CoT) monitoring is one method widely used to detect adversarial attacks and AI misalignment. However, attackers and misaligned models might evade CoT monitoring through ciphered reasoning: reasoning hidden in encrypted, translated, or compressed text. To assess this risk, we test whether models can perform ciphered reasoning. For each of 28 different ciphers, we fine-tune and prompt up to 10 models to reason in that cipher. We measure model accuracy on math problems as a proxy for reasoning ability. Across the models we test, we find an asymmetry: model accuracy can drop significantly when reasoning in ciphered text, even though models demonstrate comprehension of ciphered text by being able to translate it accurately to English. Even frontier models struggle with lesser-known ciphers, although they can reason accurately in well-known ciphers like rot13. We show that ciphered reasoning capability correlates with cipher prevalence in pretraining data. We also identify scaling laws showing that ciphered reasoning capability improves slowly with additional fine-tuning data. Our work suggests that evading CoT monitoring using ciphered reasoning may be an ineffective tactic for current models and offers guidance on constraining the development of this capability in future frontier models.

Method: Proposed Preference-Aware Memory Update Mechanism (PAMU) that integrates sliding window averages (SW) with exponential moving averages (EMA) to create fused preference-aware representations.

Result: Experiments on five task scenarios of the LoCoMo dataset show PAMU significantly improves output quality of LLMs across five baselines in long-term conversations.

Conclusion: PAMU effectively addresses the memory updating gap and enhances LLM agent performance in long-term conversational contexts through dynamic and personalized memory refinement.

Abstract: One of the key factors influencing the reasoning capabilities of LLM-based agents is their ability to leverage long-term memory. Integrating long-term memory mechanisms allows agents to make informed decisions grounded in historical interactions. While recent advances have significantly improved the storage and retrieval components, by encoding memory into dense vectors for similarity search or organizing memory as structured knowledge graphs most existing approaches fall short in memory updating. In particular, they lack mechanisms for dynamically refining preference memory representations in response to evolving user behaviors and contexts. To address this gap, we propose a Preference-Aware Memory Update Mechanism (PAMU) that enables dynamic and personalized memory refinement. By integrating sliding window averages (SW) with exponential moving averages (EMA), PAMU constructs a fused preference-aware representation that captures both short-term fluctuations and long-term user tendencies. We conduct experiments on five task scenarios of the LoCoMo dataset, and the results show that our mechanism can significantly improve the output quality of LLM in five baselines, validating its effectiveness in long-term conversations.

Method: Combines fine-tuned layout parser for document normalization, inference-efficient LLM extractor with parallel prompting and instruction tuning, and two-stage automated evaluation framework with new benchmark datasets.

Result: Significantly outperforms baselines in accuracy and efficiency; fine-tuned 0.6B LLM achieves top-tier accuracy while reducing latency and computational cost; deployed in Alibaba’s HR platform.

Conclusion: The framework successfully addresses all three challenges and enables real-time resume extraction applications at scale in production environments.

Abstract: Automated resume information extraction is critical for scaling talent acquisition, yet its real-world deployment faces three major challenges: the extreme heterogeneity of resume layouts and content, the high cost and latency of large language models (LLMs), and the lack of standardized datasets and evaluation tools. In this work, we present a layout-aware and efficiency-optimized framework for automated extraction and evaluation that addresses all three challenges. Our system combines a fine-tuned layout parser to normalize diverse document formats, an inference-efficient LLM extractor based on parallel prompting and instruction tuning, and a robust two-stage automated evaluation framework supported by new benchmark datasets. Extensive experiments show that our framework significantly outperforms strong baselines in both accuracy and efficiency. In particular, we demonstrate that a fine-tuned compact 0.6B LLM achieves top-tier accuracy while significantly reducing inference latency and computational cost. The system is fully deployed in Alibaba’s intelligent HR platform, supporting real-time applications across its business units.

Method: Proposes EVisRAG framework that first observes retrieved images and records per-image evidence, then derives final answer from aggregated evidence. Uses Reward-Scoped Group Relative Policy Optimization (RS-GRPO) to bind fine-grained rewards to scope-specific tokens for joint optimization of visual perception and reasoning.

Result: Experimental results show 27% average improvement over backbone VLM on multiple visual question answering benchmarks. EVisRAG improves answer accuracy by precisely perceiving and localizing question-relevant evidence across multiple images.

Conclusion: EVisRAG effectively addresses multi-image reasoning challenges in VRAG systems, enabling more reliable evidence integration and reducing hallucinations through evidence-guided reasoning.

Abstract: Visual retrieval-augmented generation (VRAG) augments vision-language models (VLMs) with external visual knowledge to ground reasoning and reduce hallucinations. Yet current VRAG systems often fail to reliably perceive and integrate evidence across multiple images, leading to weak grounding and erroneous conclusions. In this paper, we propose EVisRAG, an end-to-end framework that learns to reason with evidence-guided multi-image to address this issue. The model first observes retrieved images and records per-image evidence, then derives the final answer from the aggregated evidence. To train EVisRAG effectively, we introduce Reward-Scoped Group Relative Policy Optimization (RS-GRPO), which binds fine-grained rewards to scope-specific tokens to jointly optimize visual perception and reasoning abilities of VLMs. Experimental results on multiple visual question answering benchmarks demonstrate that EVisRAG delivers substantial end-to-end gains over backbone VLM with 27% improvements on average. Further analysis shows that, powered by RS-GRPO, EVisRAG improves answer accuracy by precisely perceiving and localizing question-relevant evidence across multiple images and deriving the final answer from that evidence, much like a real detective.

Method: Two-step methodology: (1) correlation test to filter judges with strong alignment, (2) human-likeness test using z-scores to identify human-like judgment (|z| < 1) and super-consistent judgment (z > 1) patterns. Evaluated 54 LLMs including 43 open-source and 11 closed models.

Result: 27 out of 54 LLMs achieved Tier 1 performance: 23 models exhibited human-like judgment patterns that preserve human judgment nuances, while 4 models showed super-consistent behavior that exceeds typical human-to-human agreement levels. Judge excellence was found to depend on specific training strategies rather than model size alone.

Conclusion: Correlation alone is insufficient for judge evaluation; the benchmark provides a standardized method for classifying LLM judges into performance tiers and establishes a “Turing Test for judges” based on agreement patterns.

Abstract: This research introduces the Judge’s Verdict Benchmark, a novel two-step methodology to evaluate Large Language Models (LLMs) as judges for response accuracy evaluation tasks. We assess how well 54 LLMs can replicate human judgment when scoring responses from RAG (Retrieval-Augmented Generation) or Agentic pipelines against ground truth answers. Our methodology progresses from traditional correlation analysis to comprehensive Cohen’s Kappa analysis that measures actual agreement patterns. The two-step approach includes: (1) a correlation test that filters judges with strong alignment, followed by (2) a human-likeness test using z-scores to identify two distinct judgment patterns: human-like judgment (|z| < 1) that mimics natural human variation, and super-consistent judgment (z > 1) that exceeds typical human-to-human agreement levels. This methodology reveals that 27 out of 54 tested LLMs achieve Tier 1 performance: 23 models exhibit human-like patterns that preserve the nuances of human judgment, while 4 models demonstrate super-consistent behavior, a pattern that could indicate either enhanced reliability or oversimplification of complex judgments. Testing 43 open-source models (1B-405B parameters) and 11 closed models (GPT, Gemini, Claude variants), we demonstrate that judge excellence is not solely dependent on model size but on specific training strategies. Our key contributions include: (1) establishing that correlation alone is insufficient for judge evaluation, (2) introducing a “Turing Test for judges” based on agreement patterns, and (3) providing a standardized benchmark for classifying LLM judges into distinct performance tiers for different evaluation needs.

Method: Large-scale empirical study of 3,600 cases from six medical datasets and six multi-agent frameworks, using mixed-methods approach combining qualitative analysis with quantitative auditing.

Result: Identified four dominant failure patterns: flawed consensus from shared model deficiencies, suppression of correct minority opinions, ineffective discussion dynamics, and critical information loss during synthesis.

Conclusion: High accuracy alone is insufficient for clinical trust; transparent and auditable reasoning processes are essential for responsible development and deployment of medical AI.

Abstract: While large language model (LLM)-based multi-agent systems show promise in simulating medical consultations, their evaluation is often confined to final-answer accuracy. This practice treats their internal collaborative processes as opaque “black boxes” and overlooks a critical question: is a diagnostic conclusion reached through a sound and verifiable reasoning pathway? The inscrutable nature of these systems poses a significant risk in high-stakes medical applications, potentially leading to flawed or untrustworthy conclusions. To address this, we conduct a large-scale empirical study of 3,600 cases from six medical datasets and six representative multi-agent frameworks. Through a rigorous, mixed-methods approach combining qualitative analysis with quantitative auditing, we develop a comprehensive taxonomy of collaborative failure modes. Our quantitative audit reveals four dominant failure patterns: flawed consensus driven by shared model deficiencies, suppression of correct minority opinions, ineffective discussion dynamics, and critical information loss during synthesis. This study demonstrates that high accuracy alone is an insufficient measure of clinical or public trust. It highlights the urgent need for transparent and auditable reasoning processes, a cornerstone for the responsible development and deployment of medical AI.

Method: Propose Gold Panning Bandits framework that reorders documents and observes shifts in model responses to identify relevant content. Frame document reordering as bipartite matching problem, with both optimal Hungarian algorithm (O(N³)) and greedy strategy (O(N log N)) approaches.

Result: The approach identifies relevant documents using up to 65% fewer language model queries than random permutation baselines on knowledge-intensive NLP tasks, substantially reducing computational cost without requiring model retraining.

Conclusion: Inherent LLM biases can be transformed from liabilities into assets for efficient, inference-time optimization, demonstrating that position bias can be leveraged as a valuable signal rather than just mitigated.

Abstract: Large language models exhibit a strong position bias in multi-document contexts, systematically prioritizing information based on location rather than relevance. While existing approaches treat this bias as noise to be mitigated, we introduce Gold Panning Bandits, a framework that leverages position bias as a diagnostic signal: by reordering documents and observing shifts in the model’s responses, we can efficiently identify the most relevant content. We frame the problem of choosing reorderings as a bipartite matching problem. While an optimal assignment can be computed at each iteration with the Hungarian algorithm in $O(N^3)$ time, we propose a greedy $O(N \log N)$ strategy that achieves comparable performance by prioritizing the placement of the most uncertain documents in the most informative positions. Our approach identifies relevant documents using up to 65% fewer language model queries than random permutation baselines on knowledge-intensive NLP tasks, substantially reducing computational cost without model retraining. This work demonstrates that inherent LLM biases can be transformed from liabilities into assets for efficient, inference-time optimization.

Method: Combines chi-square statistical analysis for keyword extraction with adaptive majority voting for decision-making, exploring statistical vs random keyword selection and adaptive voting mechanisms.

Result: Achieves micro-F1 of 67.61, outperforming n-gram baselines (60.75) and random approaches (14.65). Chi-square keywords provide consistent improvements across categories.

Conclusion: Chi-square-based keywords show the most consistent impact across all categories, and adaptive voting benefits ambiguous cases requiring extended classification rounds.

Abstract: The BLP-2025 Task 1A requires Bengali hate speech classification into six categories. Traditional supervised approaches need extensive labeled datasets that are expensive for low-resource languages. We developed PromptGuard, a few-shot framework combining chi-square statistical analysis for keyword extraction with adaptive majority voting for decision-making. We explore statistical keyword selection versus random approaches and adaptive voting mechanisms that extend classification based on consensus quality. Chi-square keywords provide consistent improvements across categories, while adaptive voting benefits ambiguous cases requiring extended classification rounds. PromptGuard achieves a micro-F1 of 67.61, outperforming n-gram baselines (60.75) and random approaches (14.65). Ablation studies confirm chi-square-based keywords show the most consistent impact across all categories.

Method: RISE represents semantic transformations as consistent rotational operations in embedding space, leveraging the manifold structure of language representations and evaluating across multiple embedding models, datasets, and languages.

Result: RISE consistently maps discourse-level semantic transformations (e.g., negation, conditionality) across 7 morphologically diverse languages in 5 language groups and 3 embedding models, showing high transfer performance.

Conclusion: This work provides the first systematic demonstration that discourse-level semantic transformations correspond to consistent geometric operations in multilingual embedding spaces, empirically supporting the Linear Representation Hypothesis at sentence level.

Abstract: Understanding how language and embedding models encode semantic relationships is fundamental to model interpretability and control. While early word embeddings exhibited intuitive vector arithmetic (‘‘king’’ - ‘‘man’’ + ‘‘woman’’ = ‘‘queen’’), modern high-dimensional text representations lack straightforward interpretable geometric properties. We introduce Rotor-Invariant Shift Estimation (RISE), a geometric approach that represents semantic transformations as consistent rotational operations in embedding space, leveraging the manifold structure of modern language representations. RISE operations have the ability to operate across both languages and models with high transfer of performance, suggesting the existence of analogous cross-lingual geometric structure. We evaluate RISE across three embedding models, three datasets, and seven morphologically diverse languages in five major language groups. Our results demonstrate that RISE consistently maps discourse-level semantic transformations with distinct grammatical features (e.g., negation and conditionality) across languages and models. This work provides the first systematic demonstration that discourse-level semantic transformations correspond to consistent geometric operations in multilingual embedding spaces, empirically supporting the Linear Representation Hypothesis at the sentence level.

Method: Developed an algorithm for text prompt injection attacks against vision language models.

Result: The approach demonstrated effectiveness and efficiency in misleading models, particularly working well for large models with low computational demands.

Conclusion: Text prompt injection presents a significant security vulnerability for vision language models that requires attention.

Abstract: The widespread application of large vision language models has significantly raised safety concerns. In this project, we investigate text prompt injection, a simple yet effective method to mislead these models. We developed an algorithm for this type of attack and demonstrated its effectiveness and efficiency through experiments. Compared to other attack methods, our approach is particularly effective for large models without high demand for computational resources.

Method: Formulates nutritional QA as supervised knowledge-graph-guided multi-agent collaboration, integrating agent nodes into heterogeneous knowledge graphs and using GNNs to learn task-aware routing distributions with gradient-based subgraph retrieval.

Result: Consistently outperforms both single-agent and ensemble baselines across multiple benchmarks and backbone models.

Conclusion: NG-Router provides a principled approach to domain-aware multi-agent reasoning for complex nutritional health tasks.

Abstract: Diet plays a central role in human health, and Nutrition Question Answering (QA) offers a promising path toward personalized dietary guidance and the prevention of diet-related chronic diseases. However, existing methods face two fundamental challenges: the limited reasoning capacity of single-agent systems and the complexity of designing effective multi-agent architectures, as well as contextual overload that hinders accurate decision-making. We introduce Nutritional-Graph Router (NG-Router), a novel framework that formulates nutritional QA as a supervised, knowledge-graph-guided multi-agent collaboration problem. NG-Router integrates agent nodes into heterogeneous knowledge graphs and employs a graph neural network to learn task-aware routing distributions over agents, leveraging soft supervision derived from empirical agent performance. To further address contextual overload, we propose a gradient-based subgraph retrieval mechanism that identifies salient evidence during training, thereby enhancing multi-hop and relational reasoning. Extensive experiments across multiple benchmarks and backbone models demonstrate that NG-Router consistently outperforms both single-agent and ensemble baselines, offering a principled approach to domain-aware multi-agent reasoning for complex nutritional health tasks.

Method: Developed a theory-informed taxonomy of narrative-understanding tasks and conducted a survey of 78 existing benchmarks to assess coverage and alignment.

Result: Only 27% of narrative tasks are well captured by existing benchmarks, with significant gaps in areas like narrative events, style, perspective, and revelation. Need for benchmarks assessing subjective and perspectival aspects.

Conclusion: The taxonomy, survey, and methodology provide valuable tools for NLP researchers to better test LLM narrative understanding capabilities and address current evaluation gaps.

Abstract: We present NarraBench, a theory-informed taxonomy of narrative-understanding tasks, as well as an associated survey of 78 existing benchmarks in the area. We find significant need for new evaluations covering aspects of narrative understanding that are either overlooked in current work or are poorly aligned with existing metrics. Specifically, we estimate that only 27% of narrative tasks are well captured by existing benchmarks, and we note that some areas – including narrative events, style, perspective, and revelation – are nearly absent from current evaluations. We also note the need for increased development of benchmarks capable of assessing constitutively subjective and perspectival aspects of narrative, that is, aspects for which there is generally no single correct answer. Our taxonomy, survey, and methodology are of value to NLP researchers seeking to test LLM narrative understanding.

Method: CoBia creates constructed conversations where models utter biased claims about social groups, then evaluates if models can recover and reject biased follow-up questions across 6 socio-demographic categories using established bias metrics.

Result: Purposefully constructed conversations reliably reveal bias amplification, and LLMs often fail to reject biased follow-up questions during dialogue, highlighting deeply embedded biases.

Conclusion: Stress-testing through adversarial conversations effectively surfaces embedded biases in LLMs, demonstrating the need for improved safety mechanisms beyond standard checks.

Abstract: Improvements in model construction, including fortified safety guardrails, allow Large language models (LLMs) to increasingly pass standard safety checks. However, LLMs sometimes slip into revealing harmful behavior, such as expressing racist viewpoints, during conversations. To analyze this systematically, we introduce CoBia, a suite of lightweight adversarial attacks that allow us to refine the scope of conditions under which LLMs depart from normative or ethical behavior in conversations. CoBia creates a constructed conversation where the model utters a biased claim about a social group. We then evaluate whether the model can recover from the fabricated bias claim and reject biased follow-up questions. We evaluate 11 open-source as well as proprietary LLMs for their outputs related to six socio-demographic categories that are relevant to individual safety and fair treatment, i.e., gender, race, religion, nationality, sex orientation, and others. Our evaluation is based on established LLM-based bias metrics, and we compare the results against human judgments to scope out the LLMs’ reliability and alignment. The results suggest that purposefully constructed conversations reliably reveal bias amplification and that LLMs often fail to reject biased follow-up questions during dialogue. This form of stress-testing highlights deeply embedded biases that can be surfaced through interaction. Code and artifacts are available at https://github.com/nafisenik/CoBia.

Method: Reframes prompt selection as curriculum design, integrating two signals: model-perceived difficulty (quantified via prediction disagreement in active learning) and intrinsic sample complexity (inherent difficulty of question-image pairs). Uses difficulty-balanced sampling to select diverse prompts across both dimensions.

Result: Extensive experiments on five challenging benchmarks with multiple MLLMs show substantial and consistent improvements, greatly reducing performance discrepancies caused by random sampling.

Conclusion: Provides a principled and robust approach for enhancing multimodal reasoning through difficulty-balanced prompt selection that aligns with model capabilities.

Abstract: The effectiveness of Multimodal Chain-of-Thought (MCoT) prompting is often limited by the use of randomly or manually selected examples. These examples fail to account for both model-specific knowledge distributions and the intrinsic complexity of the tasks, resulting in suboptimal and unstable model performance. To address this, we propose a novel framework inspired by the pedagogical principle of “tailored teaching with balanced difficulty”. We reframe prompt selection as a prompt curriculum design problem: constructing a well ordered set of training examples that align with the model’s current capabilities. Our approach integrates two complementary signals: (1) model-perceived difficulty, quantified through prediction disagreement in an active learning setup, capturing what the model itself finds challenging; and (2) intrinsic sample complexity, which measures the inherent difficulty of each question-image pair independently of any model. By jointly analyzing these signals, we develop a difficulty-balanced sampling strategy that ensures the selected prompt examples are diverse across both dimensions. Extensive experiments conducted on five challenging benchmarks and multiple popular Multimodal Large Language Models (MLLMs) demonstrate that our method yields substantial and consistent improvements and greatly reduces performance discrepancies caused by random sampling, providing a principled and robust approach for enhancing multimodal reasoning.

Method: Each input token is represented as a sparse, non-negative mixture over k context-independent sense vectors, which can be pooled into sentence embeddings or used at token level.

Result: On STEL benchmark, improves style representation effectiveness by ~8 points over SBERT-style baselines while maintaining competitive performance on authorship verification. Specific style attributes can be assigned to specific sense vectors.

Conclusion: iBERT provides structural modularity to interpretably decompose discriminative signals in data, enabling generalization even when supervision blends stylistic and semantic factors, and is not limited to stylistic modeling.

Abstract: We present iBERT (interpretable-BERT), an encoder to produce inherently interpretable and controllable embeddings - designed to modularize and expose the discriminative cues present in language, such as stylistic and semantic structure. Each input token is represented as a sparse, non-negative mixture over k context-independent sense vectors, which can be pooled into sentence embeddings or used directly at the token level. This enables modular control over representation, before any decoding or downstream use. To demonstrate our model’s interpretability, we evaluate it on a suite of style-focused tasks. On the STEL benchmark, it improves style representation effectiveness by ~8 points over SBERT-style baselines, while maintaining competitive performance on authorship verification. Because each embedding is a structured composition of interpretable senses, we highlight how specific style attributes - such as emoji use, formality, or misspelling can be assigned to specific sense vectors. While our experiments center on style, iBERT is not limited to stylistic modeling. Its structural modularity is designed to interpretably decompose whichever discriminative signals are present in the data - enabling generalization even when supervision blends stylistic and semantic factors.

Method: Multi-agent simulations in a dynamic sandbox environment where agents’ behaviors and interactions generate emergent events that drive story development organically.

Result: The system generates coherent stories exceeding 10,000 words and achieves state-of-the-art performance across multiple metrics.

Conclusion: This hybrid bottom-up approach offers a scalable and innovative solution for creating dynamic, immersive long-form stories that evolve naturally from agent-driven interactions.

Abstract: Human writers often begin their stories with an overarching mental scene, where they envision the interactions between characters and their environment. Inspired by this creative process, we propose a novel approach to long-form story generation, termed hybrid bottom-up long-form story generation, using multi-agent simulations. In our method, agents interact within a dynamic sandbox environment, where their behaviors and interactions with one another and the environment generate emergent events. These events form the foundation for the story, enabling organic character development and plot progression. Unlike traditional top-down approaches that impose rigid structures, our hybrid bottom-up approach allows for the natural unfolding of events, fostering more spontaneous and engaging storytelling. The system is capable of generating stories exceeding 10,000 words while maintaining coherence and consistency, addressing some of the key challenges faced by current story generation models. We achieve state-of-the-art performance across several metrics. This approach offers a scalable and innovative solution for creating dynamic, immersive long-form stories that evolve organically from agent-driven interactions.

Method: Partitions transformer layers into three groups: initial full attention layers, selection layers that identify salient tokens using aggregated attention scores, and sparse-attention layers that attend only to selected tokens.

Result: Matches or surpasses full attention accuracy on reasoning benchmarks (AIME, GPQA-Diamond), reduces attended tokens by up to 5x, and achieves 1.5x end-to-end speedup.

Conclusion: Selective reuse of intermediate attention maps provides a robust path for efficient long-context reasoning without sacrificing model accuracy.

Abstract: Large reasoning models (LRMs) achieve state-of-the-art performance on challenging benchmarks by generating long chains of intermediate steps, but their inference cost is dominated by decoding, where each new token must attend to the entire growing sequence. Existing sparse attention methods reduce computation by pruning the key-value (KV) cache, yet they suffer from severe accuracy degradation on reasoning tasks due to cumulative selection errors and the dynamic importance of tokens over long derivations. We present \textbf{DELTA}, a training-free sparse attention mechanism that achieves computational efficiency without sacrificing model accuracy. DELTA partitions transformer layers into three groups: initial layers that use full attention, a small set of \emph{selection layers} that identify salient tokens via aggregated head-level attention scores, and subsequent \emph{sparse-attention layers} that attend only to the selected subset. This design preserves the full KV cache in GPU memory for accuracy, while avoiding expensive full-attention computation over many layers. On reasoning benchmarks such as AIME and GPQA-Diamond, DELTA matches or surpasses full attention in accuracy, while reducing the number of attended tokens by up to $5\times$ and delivering $1.5\times$ end-to-end speedup. Our results show that selective reuse of intermediate attention maps offers a robust path toward efficient long-context reasoning.

Method: Fine-tuned arLLMs and dLLMs on three datasets, evaluated with forward/backward QA to test knowledge generalization and reversal curse. Proposed masked fine-tuning for arLLMs.

Result: dLLMs achieved high accuracy on both forward and backward QAs without paraphrases, while arLLMs required extensive data augmentation and still suffered from reversal curse. Masked fine-tuning improved arLLM data efficiency.

Conclusion: dLLMs are superior for knowledge injection via fine-tuning, being reversal-curse-free and data-efficient. The proposed masked fine-tuning method successfully bridges the performance gap for arLLMs.

Abstract: Despite autoregressive large language models (arLLMs) being the current dominant paradigm in language modeling, they resist knowledge injection via fine-tuning due to inherent shortcomings such as the “reversal curse” – the challenge of answering questions that reverse the original information order in the training sample. Masked diffusion large language models (dLLMs) are rapidly emerging as a powerful alternative to the arLLM paradigm, with evidence of better data efficiency and free of the “reversal curse” in pre-training. However, it is unknown whether these advantages extend to the post-training phase, i.e. whether pre-trained dLLMs can easily acquire new knowledge through fine-tuning. On three diverse datasets, we fine-tune arLLMs and dLLMs, evaluating them with forward and backward style Question Answering (QA) to probe knowledge generalization and the reversal curse. Our results confirm that arLLMs critically rely on extensive data augmentation via paraphrases for QA generalization, and paraphrases are only effective when their information order matches the QA style. Conversely, dLLMs achieve high accuracies on both forward and backward QAs without paraphrases; adding paraphrases yields only marginal gains. Lastly, inspired by the dLLM’s performance, we introduce a novel masked fine-tuning paradigm for knowledge injection into pre-trained arLLMs. This proposed method successfully and drastically improves the data efficiency of arLLM fine-tuning, effectively closing the performance gap with dLLMs.

Method: Proposes abductive preference learning - fine-tuning paradigm that reverses conditioning by learning preferences over prompts given a response. Implements abductive DPO and DPOP variants, with multitask objective combining standard and abductive approaches.

Result: Multitask DPOP boosts accuracy from 90.0% to 99.5% in response selection and 54.7% to 85.0% in prompt discrimination on abductive dataset. On AlpacaEval, improves win rate from 5.26% to 6.17%. Qualitative evidence shows improved sensitivity to prompt differences.

Conclusion: Abductive preference learning preserves conventional optimization benefits while addressing counterfactual prompt challenge, demonstrating complementary strengths when combined with standard methods.

Abstract: Frontier large language models such as GPT-5 and Claude Sonnet remain prone to overconfidence even after alignment through Reinforcement Learning with Human Feedback (RLHF) and Direct Preference Optimization (DPO). For instance, they tend to offer the same conservative answer “No” to both questions “Can I eat the [food / potato chips] that has been left out overnight?” despite the latter requiring no refridgeration for safe consumption. We find that this failure is potentially attributed to a limitation of existing preference learning: it emphasizes selecting the correct response for a given prompt, while neglecting counterfactual prompts that should alter the response. To address this limitation, we propose abductive preference learning, a fine-tuning paradigm that reverses the conventional conditioning by learning preferences over prompts given a response. To validate this idea, we construct an abductive dataset derived from the HaluEval QA benchmark with 1,001 entries, implementing abductive DPO and its variant DPOP. Experiments reveal complementary strengths: standard methods improve response selection, abductive methods improve prompt discrimination, while a multitask objective unifies both. On the abductive dataset, multitask DPOP boosts accuracy from $90.0%$ to $99.5%$ in response selection and $54.7%$ to $85.0%$ in prompt discrimination, with qualitative evidence highlighting improved sensitivity to prompt differences. Finally, evaluation on AlpacaEval shows multitask DPOP improves win rate (from $5.26%$ to $6.17%$), confirming that abductive preference learning preserves the benefits of conventional preference optimization while addressing the overlooked challenge of counterfactual prompts.

Method: D3 employs two protocols: MORE (Multi-Advocate One-Round Evaluation) with parallel defenses, and SAMRE (Single-Advocate Multi-Round Evaluation) with budgeted stopping. It uses role-specialized agents (advocates, judge, jury) in structured debates with probabilistic modeling of score gaps.

Result: D3 achieves state-of-the-art agreement with human judgments (accuracy and Cohen’s kappa), reduces positional and verbosity biases through anonymization and role diversification, and provides favorable cost-accuracy trade-offs via budgeted stopping.

Conclusion: D3 establishes a principled, practical framework for reliable, interpretable, and cost-aware LLM evaluation through structured debate protocols and probabilistic modeling.

Abstract: The evaluation of Large Language Models (LLMs) remains challenging due to inconsistency, bias, and the absence of transparent decision criteria in automated judging. We present Debate, Deliberate, Decide (D3), a cost-aware, adversarial multi-agent framework that orchestrates structured debate among role-specialized agents (advocates, a judge, and an optional jury) to produce reliable and interpretable evaluations. D3 instantiates two complementary protocols: (1) Multi-Advocate One-Round Evaluation (MORE), which elicits k parallel defenses per answer to amplify signal via diverse advocacy, and (2) Single-Advocate Multi-Round Evaluation (SAMRE) with budgeted stopping, which iteratively refines arguments under an explicit token budget and convergence checks. We develop a probabilistic model of score gaps that (i) characterizes reliability and convergence under iterative debate and (ii) explains the separation gains from parallel advocacy. Under mild assumptions, the posterior distribution of the round-r gap concentrates around the true difference and the probability of mis-ranking vanishes; moreover, aggregating across k advocates provably increases expected score separation. We complement theory with a rigorous experimental suite across MT-Bench, AlignBench, and AUTO-J, showing state-of-the-art agreement with human judgments (accuracy and Cohen’s kappa), reduced positional and verbosity biases via anonymization and role diversification, and a favorable cost-accuracy frontier enabled by budgeted stopping. Ablations and qualitative analyses isolate the contributions of debate, aggregation, and anonymity. Together, these results establish D3 as a principled, practical recipe for reliable, interpretable, and cost-aware LLM evaluation.

Method: Uses LLM as cerebral cortex for semantic reasoning, dynamic memory module as prefrontal cortex for adaptive feature retention, and specialized lightweight models as basal ganglia with winner-takes-all routing for personality pattern recognition.

Result: Extensive experiments on Kaggle and Pandora datasets show HIPPD consistently outperforms state-of-the-art baselines.

Conclusion: The brain-inspired hierarchical framework effectively addresses limitations of existing approaches and demonstrates superior performance in personality detection from text.

Abstract: Personality detection from text aims to infer an individual’s personality traits based on linguistic patterns. However, existing machine learning approaches often struggle to capture contextual information spanning multiple posts and tend to fall short in extracting representative and robust features in semantically sparse environments. This paper presents HIPPD, a brain-inspired framework for personality detection that emulates the hierarchical information processing of the human brain. HIPPD utilises a large language model to simulate the cerebral cortex, enabling global semantic reasoning and deep feature abstraction. A dynamic memory module, modelled after the prefrontal cortex, performs adaptive gating and selective retention of critical features, with all adjustments driven by dopaminergic prediction error feedback. Subsequently, a set of specialised lightweight models, emulating the basal ganglia, are dynamically routed via a strict winner-takes-all mechanism to capture the personality-related patterns they are most proficient at recognising. Extensive experiments on the Kaggle and Pandora datasets demonstrate that HIPPD consistently outperforms state-of-the-art baselines.

Method: Train a switcher LM that learns to choose between different model checkpoints to generate the next segment in a response sequence, based on query context and model strengths.

Result: Model collaboration outperforms individual models on 16/18 tasks, and Switch Generation further outperforms baselines by 12.9% on average. It discovers compositional skills and generalizes to unseen models/tasks.

Conclusion: Switch Generation effectively leverages model collaboration to combine complementary strengths, reusing training pipeline by-products that would otherwise be discarded.

Abstract: Alignment training has tradeoffs: it helps language models (LMs) gain in reasoning and instruction following but might lose out on skills such as creativity and calibration, where unaligned base models are better at. We aim to make the best of both worlds through model collaboration, where different models in the training pipeline collaborate and complement each other. Since LM responses feature interleaving skills that favor different models, we propose Switch Generation, where pretrained and aligned model versions take turns to ``speak’’ in a response sequence. Specifically, we train a switcher LM by learning from outcomes of choosing different models to generate the next segment across diverse queries and contexts. At inference time, the switcher LM guides different model checkpoints to dynamically generate the next segment where their strengths are most needed. Extensive experiments with 8 model collaboration baselines and 18 datasets show that 1) model collaboration consistently outperforms individual models on 16 out of 18 tasks, and 2) Switch Generation further outperforms baselines by 12.9% on average. Further analysis reveals that Switch Generation discovers compositional skills to solve problems where individual models struggle and generalizes to unseen models and tasks, reusing and repurposing by-products in expensive model training pipelines that are otherwise discarded.

Method: Automatically generate summaries using various LLMs, use GPT-4o to annotate span-level hallucinations, then fine-tune LLMs using three techniques: gradient ascent, unlikelihood training, and task vector negation.

Result: All three approaches successfully improved faithfulness using span-level annotations, with unlikelihood training being the most effective method.

Conclusion: Fine-tuning LLMs with span-level hallucination annotations can significantly improve summary faithfulness, with unlikelihood training showing the best performance among the tested methods.

Abstract: Abstractive summarization using large language models (LLMs) has become an essential tool for condensing information. However, despite their ability to generate fluent summaries, these models sometimes produce unfaithful summaries, introducing hallucinations at the word, phrase, or concept level. Existing mitigation strategies, such as post-processing corrections or contrastive learning with synthetically generated negative samples, fail to fully address the diverse errors that can occur in LLM-generated summaries. In this paper, we investigate fine-tuning strategies to reduce the occurrence of unfaithful spans in generated summaries. First, we automatically generate summaries for the set of source documents in the training set with a variety of LLMs and then use GPT-4o to annotate any hallucinations it detects at the span-level. Leveraging these annotations, we fine-tune LLMs with both hallucination-free summaries and annotated unfaithful spans to enhance model faithfulness. In this paper, we introduce a new dataset that contains both faithful and unfaithful summaries with span-level labels and we evaluate three techniques to fine-tuning a LLM to improve the faithfulness of the resulting summarization: gradient ascent, unlikelihood training, and task vector negation. Experimental results show that all three approaches successfully leverage span-level annotations to improve faithfulness, with unlikelihood training being the most effective.

Method: Developed PCM for modeling contextual information across modalities and FACT module for detecting false claims using external knowledge and cross-modal reference graphs.

Result: SHIELD outperforms state-of-the-art methods across datasets and metrics, and shows versatility on other tasks like fake news detection.

Conclusion: The framework successfully captures the fundamental nature of hate in memes by addressing both presupposed context and false claims, providing effective detection across various applications.

Abstract: While memes are often humorous, they are frequently used to disseminate hate, causing serious harm to individuals and society. Current approaches to hateful meme detection mainly rely on pre-trained language models. However, less focus has been dedicated to \textit{what make a meme hateful}. Drawing on insights from philosophy and psychology, we argue that hateful memes are characterized by two essential features: a \textbf{presupposed context} and the expression of \textbf{false claims}. To capture presupposed context, we develop \textbf{PCM} for modeling contextual information across modalities. To detect false claims, we introduce the \textbf{FACT} module, which integrates external knowledge and harnesses cross-modal reference graphs. By combining PCM and FACT, we introduce \textbf{\textsf{SHIELD}}, a hateful meme detection framework designed to capture the fundamental nature of hate. Extensive experiments show that SHIELD outperforms state-of-the-art methods across datasets and metrics, while demonstrating versatility on other tasks, such as fake news detection.

Method: Analyzed six widely used tokenizers across seven languages and two domains. Proposed STRR (Single Token Retention Rate) to measure the proportion of words preserved as single tokens, providing an interpretable view of cross-lingual fairness.

Result: Found stable fertility for English, high fertility for Chinese, and little domain sensitivity. STRR revealed systematic prioritization of English, strong support for Chinese, and fragmentation in Hindi. STRR complements fertility and provides practical guidance.

Conclusion: STRR offers an interpretable view of cross-lingual fairness in tokenizers, complementing existing metrics and providing guidance for designing more equitable multilingual tokenizers.

Abstract: Tokenization is a crucial but under-evaluated step in large language models (LLMs). The standard metric, fertility (the average number of tokens per word), captures compression efficiency but obscures how vocabularies are allocated across languages and domains. We analyze six widely used tokenizers across seven languages and two domains, finding stable fertility for English, high fertility for Chinese, and little domain sensitivity. To address fertility’s blind spots, we propose the Single Token Retention Rate (STRR), which measures the proportion of words preserved as single tokens. STRR reveals systematic prioritization of English, strong support for Chinese, and fragmentation in Hindi, offering an interpretable view of cross-lingual fairness. Our results show that STRR complements fertility and provides practical guidance for designing more equitable multilingual tokenizers.

Method: The paper introduces a unified framework that deconstructs search algorithms into three core components: Search Mechanism, Reward Formulation, and Transition Function, establishing a formal distinction between transient Search Guidance and durable Parametric Reward Modeling.

Result: The framework enables a component-centric taxonomy, synthesis of state-of-the-art approaches, and provides a research roadmap for systematic progress in creating autonomous, self-improving agents.

Conclusion: The proposed unified framework resolves ambiguity in deliberative tree search, providing a foundation for more systematic advancement in LLM research by clearly distinguishing between test-time scaling and self-improvement applications.

Abstract: Deliberative tree search is a cornerstone of modern Large Language Model (LLM) research, driving the pivot from brute-force scaling toward algorithmic efficiency. This single paradigm unifies two critical frontiers: \textbf{Test-Time Scaling (TTS)}, which deploys on-demand computation to solve hard problems, and \textbf{Self-Improvement}, which uses search-generated data to durably enhance model parameters. However, this burgeoning field is fragmented and lacks a common formalism, particularly concerning the ambiguous role of the reward signal – is it a transient heuristic or a durable learning target? This paper resolves this ambiguity by introducing a unified framework that deconstructs search algorithms into three core components: the \emph{Search Mechanism}, \emph{Reward Formulation}, and \emph{Transition Function}. We establish a formal distinction between transient \textbf{Search Guidance} for TTS and durable \textbf{Parametric Reward Modeling} for Self-Improvement. Building on this formalism, we introduce a component-centric taxonomy, synthesize the state-of-the-art, and chart a research roadmap toward more systematic progress in creating autonomous, self-improving agents.

Method: Human study conducted in a public museum with 452 participants, investigating how fluency and adequacy errors impact bilingual and non-bilingual users’ reliance on MT during casual use.

Result: Non-bilingual users often over-rely on MT due to lack of evaluation strategies and alternatives, but experiencing errors can prompt users to reassess future reliance.

Conclusion: Highlights the need for MT evaluation and NLP explanation techniques to promote not only MT quality, but also MT literacy among users.

Abstract: As Machine Translation (MT) becomes increasingly commonplace, understanding how the general public perceives and relies on imperfect MT is crucial for contextualizing MT research in real-world applications. We present a human study conducted in a public museum (n=452), investigating how fluency and adequacy errors impact bilingual and non-bilingual users’ reliance on MT during casual use. Our findings reveal that non-bilingual users often over-rely on MT due to a lack of evaluation strategies and alternatives, while experiencing the impact of errors can prompt users to reassess future reliance. This highlights the need for MT evaluation and NLP explanation techniques to promote not only MT quality, but also MT literacy among its users.

Method: Train an LLM-based search agent with query expansion through reinforcement learning, where the agent proposes multiple query variants per turn for simultaneous searching. Incorporate a pre-trained squeezer model to help understand retrieved documents, allowing the search agent to focus on query generation.

Result: Achieves state-of-the-art accuracy on multi-hop QA benchmarks with 4.4% average improvement across seven benchmarks, showing strong gains on tasks requiring diverse evidence aggregation.

Conclusion: Even small-scale 3B LLMs can demonstrate strong query expansion capabilities when assisted by a squeezer model, achieving superior performance on complex multi-hop reasoning tasks.

Abstract: Reasoning-augmented search agents, such as Search-R1, are trained to reason, search, and generate the final answer iteratively. Nevertheless, due to their limited capabilities in reasoning and search, their performance on multi-hop QA benchmarks remains far from satisfactory. To handle complex or compound queries, we train an LLM-based search agent with the native capability of query expansion through reinforcement learning. In each turn, our search agent proposes several query variants, which are searched simultaneously to cover more relevant information. Meanwhile, given limited post-training data and computing resources, it is very challenging for a search agent to master multiple tasks, including query generation, retrieved information understanding, and answer generation. Therefore, we propose incorporating a pre-trained squeezer model that helps the search agent understand the retrieved documents, allowing the search agent to focus on query generation for high retrieval recall. With the assistance of the squeezer model, we discover that even a small-scale 3B LLM can demonstrate a strong capability of query expansion and achieve state-of-the-art accuracy on the multi-hop QA benchmarks. To be specific, our experiments across seven question-answering benchmarks demonstrate that our method, named ExpandSearch, achieves an average improvement of 4.4% compared to state-of-the-art baselines, with strong gains on multi-hop reasoning tasks requiring diverse evidence aggregation.

Method: Conducted experiments with diverse AI models in debate settings, analyzing how models shift responses and investigating factors like sycophancy, social conformity, and model/task types.

Result: Debate led to decreased accuracy over time, with models frequently abandoning correct answers to agree with incorrect peer reasoning, even when stronger models were in the majority.

Conclusion: Naive applications of multi-agent debate can degrade performance when agents lack proper incentives or capabilities to resist persuasive but incorrect reasoning, revealing important failure modes in reason exchange.

Abstract: While multi-agent debate has been proposed as a promising strategy for improving AI reasoning ability, we find that debate can sometimes be harmful rather than helpful. Prior work has primarily focused on debates within homogeneous groups of agents, whereas we explore how diversity in model capabilities influences the dynamics and outcomes of multi-agent interactions. Through a series of experiments, we demonstrate that debate can lead to a decrease in accuracy over time - even in settings where stronger (i.e., more capable) models outnumber their weaker counterparts. Our analysis reveals that models frequently shift from correct to incorrect answers in response to peer reasoning, favoring agreement over challenging flawed reasoning. We perform additional experiments investigating various potential contributing factors to these harmful shifts - including sycophancy, social conformity, and model and task type. These results highlight important failure modes in the exchange of reasons during multi-agent debate, suggesting that naive applications of debate may cause performance degradation when agents are neither incentivised nor adequately equipped to resist persuasive but incorrect reasoning.

Method: Empirical analysis of three Path Drift triggers, development of a three-stage Path Drift Induction Framework, and proposal of path-level defense with role attribution correction and metacognitive reflection.

Result: Identified three behavioral triggers of Path Drift, demonstrated that each stage of the induction framework reduces refusal rates independently, and their combination compounds the effect.

Conclusion: Trajectory-level alignment oversight is necessary for long-form reasoning beyond token-level alignment to mitigate Path Drift risks in LLMs.

Abstract: As large language models (LLMs) are increasingly deployed for complex reasoning tasks, Long Chain-of-Thought (Long-CoT) prompting has emerged as a key paradigm for structured inference. Despite early-stage safeguards enabled by alignment techniques such as RLHF, we identify a previously underexplored vulnerability: reasoning trajectories in Long-CoT models can drift from aligned paths, resulting in content that violates safety constraints. We term this phenomenon Path Drift. Through empirical analysis, we uncover three behavioral triggers of Path Drift: (1) first-person commitments that induce goal-driven reasoning that delays refusal signals; (2) ethical evaporation, where surface-level disclaimers bypass alignment checkpoints; (3) condition chain escalation, where layered cues progressively steer models toward unsafe completions. Building on these insights, we introduce a three-stage Path Drift Induction Framework comprising cognitive load amplification, self-role priming, and condition chain hijacking. Each stage independently reduces refusal rates, while their combination further compounds the effect. To mitigate these risks, we propose a path-level defense strategy incorporating role attribution correction and metacognitive reflection (reflective safety cues). Our findings highlight the need for trajectory-level alignment oversight in long-form reasoning beyond token-level alignment.

Method: Finetuned BERT base and DistilBERT on medical abstracts corpus using three objectives: standard cross-entropy, class weighted cross entropy, and focal loss, while keeping tokenizer, sequence length, optimizer, and schedule fixed.

Result: DistilBERT with plain cross-entropy achieved the best balance on test set performance while using far fewer parameters than BERT base, with results reported using accuracy, Macro F1, and Weighted F1 metrics.

Conclusion: A practical default approach is to start with compact encoders and cross-entropy, then add calibration and task-specific checks before considering heavier models for deployment in constrained health settings.

Abstract: Large language models work well for many NLP tasks, but they are hard to deploy in health settings with strict cost, latency, and privacy limits. We revisit a lightweight recipe for medical abstract classification and ask how far compact encoders can go under a controlled budget. Using the public medical abstracts corpus, we finetune BERT base and DistilBERT with three objectives standard cross-entropy, class weighted cross entropy, and focal loss keeping tokenizer, sequence length, optimizer, and schedule fixed. DistilBERT with plain cross-entropy gives the best balance on the test set while using far fewer parameters than BERT base. We report accuracy, Macro F1, and Weighted F1, release the evaluation code, and include confusion analyses to make error patterns clear. Our results suggest a practical default: start with a compact encoder and cross-entropy, then add calibration and task-specific checks before moving to heavier models.

Method: Developed HUME framework to measure human performance across 16 MTEB datasets covering reranking, classification, clustering, and semantic textual similarity tasks in diverse high- and low-resource languages.

Result: Humans achieved average 77.6% performance vs 80.1% for best embedding model, with substantial variation - models reach near-ceiling performance on some datasets but struggle on others, particularly in low-resource languages.

Conclusion: HUME provides human performance baselines, insights into task difficulty patterns, and an extensible framework that enables more meaningful model evaluation and informs development of both models and benchmarks.

Abstract: Comparing human and model performance offers a valuable perspective for understanding the strengths and limitations of embedding models, highlighting where they succeed and where they fail to capture meaning and nuance. However, such comparisons are rarely made, as human performance on embedding tasks is difficult to measure. To fill this gap, we introduce HUME: Human Evaluation Framework for Text Embeddings. While frameworks like MTEB provide broad model evaluation, they lack reliable estimates of human performance, limiting the interpretability of model scores. We measure human performance across 16 MTEB datasets spanning reranking, classification, clustering, and semantic textual similarity across linguistically diverse high- and low-resource languages. Humans achieve an average performance of 77.6% compared to 80.1% for the best embedding model, although variation is substantial: models reach near-ceiling performance on some datasets while struggling on others, suggesting dataset issues and revealing shortcomings in low-resource languages. We provide human performance baselines, insight into task difficulty patterns, and an extensible evaluation framework that enables a more meaningful interpretation of the model and informs the development of both models and benchmarks. Our code, dataset, and leaderboard are publicly available at https://github.com/embeddings-benchmark/mteb.

Method: CLMN represents concepts as continuous, human-readable embeddings and applies fuzzy-logic reasoning to learn adaptive interaction rules that state how concepts affect each other and the final decision. It augments original text features with concept-aware representations and automatically induces interpretable logic rules.

Result: Across multiple datasets and pre-trained language models, CLMN achieves higher accuracy than existing concept-based methods while improving explanation quality.

Conclusion: Integrating neural representations with symbolic reasoning in a unified concept space can yield practical, transparent NLP systems.

Abstract: Deep learning has advanced NLP, but interpretability remains limited, especially in healthcare and finance. Concept bottleneck models tie predictions to human concepts in vision, but NLP versions either use binary activations that harm text representations or latent concepts that weaken semantics, and they rarely model dynamic concept interactions such as negation and context. We introduce the Concept Language Model Network (CLMN), a neural-symbolic framework that keeps both performance and interpretability. CLMN represents concepts as continuous, human-readable embeddings and applies fuzzy-logic reasoning to learn adaptive interaction rules that state how concepts affect each other and the final decision. The model augments original text features with concept-aware representations and automatically induces interpretable logic rules. Across multiple datasets and pre-trained language models, CLMN achieves higher accuracy than existing concept-based methods while improving explanation quality. These results show that integrating neural representations with symbolic reasoning in a unified concept space can yield practical, transparent NLP systems.

Method: Constructed Unilaw-R1-Data (17K high-quality CoT samples) and used two-stage training combining SFT and RL to enhance legal reasoning and support interpretable decision-making.

Result: Outperformed all similar-scale models and achieved performance comparable to DeepSeek-R1-Distill-Qwen-32B (54.9%). Showed 6.6% average improvement over Qwen-2.5-7B-Instruct on LawBench and LexEval.

Conclusion: Unilaw-R1 demonstrates that specialized legal reasoning models with lightweight architecture can effectively handle complex legal tasks while reducing deployment costs.

Abstract: Reasoning-focused large language models (LLMs) are rapidly evolving across various domains, yet their capabilities in handling complex legal problems remains underexplored. In this paper, we introduce Unilaw-R1, a large language model tailored for legal reasoning. With a lightweight 7-billion parameter scale, Unilaw-R1 significantly reduces deployment cost while effectively tackling three core challenges in the legal domain: insufficient legal knowledge, unreliable reasoning logic, and weak business generalization. To address these issues, we first construct Unilaw-R1-Data, a high-quality dataset containing 17K distilled and screened chain-of-thought (CoT) samples. Based on this, we adopt a two-stage training strategy combining Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), which significantly boosts the performance on complex legal reasoning tasks and supports interpretable decision-making in legal AI applications. To assess legal reasoning ability, we also introduce Unilaw-R1-Eval, a dedicated benchmark designed to evaluate models across single- and multi-choice legal tasks. Unilaw-R1 demonstrates strong results on authoritative benchmarks, outperforming all models of similar scale and achieving performance on par with the much larger DeepSeek-R1-Distill-Qwen-32B (54.9%). Following domain-specific training, it also showed significant gains on LawBench and LexEval, exceeding Qwen-2.5-7B-Instruct (46.6%) by an average margin of 6.6%.

Method: Uses three coordinated agents: User Agents with personality signifiers (Zodiac Signs, MBTI), Dialog Agent for persuasion strategies, and Optimization Agent for evaluation and refinement. Validated through Chain-of-Attitude modeling and LLM persuasion assessment.

Result: Significantly improved persuasion capacity of small LLMs, increasing organic traffic conversion rate by 22.4% (from 1.83% to 2.24%) in real-world marketing scenario.

Conclusion: MADS demonstrates clear business value by enabling scalable generation of persuasive dialogue data and improving conversion rates without human annotation costs.

Abstract: We propose MADS (Multi-Agent Dialogue Simulation), a scalable framework for generating persuasive multi-turn dialogues via agent self-play. MADS employs three coordinated agents: User Agents designed to simulate diverse persona-driven behaviors by leveraging personality signifiers such as Zodiac Signs and MBTI types, a Dialog Agent executing task-oriented persuasion strategies and an Optimization Agent evaluating and refining dialogue outcomes. We further validate its effectiveness through users’ Chain-of-Attitude (CoA) modeling and dedicated LLMs’ persuasion assessment. This approach enables low-cost generation of training data without human annotation, addressing key industry challenges such as lack of user data, cold-start evaluation difficulties, and prompt inefficiency. Applied to a real-world marketing scenario, MADS significantly improved the persuasion capacity of small LLMs, increasing the organic traffic conversion rate by 22.4% (from 1.83% to 2.24%) , demonstrating clear business value.

Method: A-IPO introduces an intention module that infers latent intent behind user prompts and explicitly incorporates this inferred intent into the reward function. It adds an intention-response similarity term that increases preference margin in log-odds.

Result: A-IPO achieves substantial improvements: up to +24.8 win-rate and +45.6 Response-Intention Consistency on Real-pref; up to +38.6 Response Similarity and +52.2 Defense Success Rate on Attack-pref; up to +54.6 Intention Consistency Score on GlobalOpinionQA-Ext.

Conclusion: A-IPO facilitates pluralistic preference optimization while enhancing adversarial robustness, consistently surpassing existing baselines across multiple evaluation benchmarks.

Abstract: Human preferences are diverse and dynamic, shaped by regional, cultural, and social factors. Existing alignment methods like Direct Preference Optimization (DPO) and its variants often default to majority views, overlooking minority opinions and failing to capture latent user intentions in prompts. To address these limitations, we introduce \underline{\textbf{A}}daptive \textbf{\underline{I}}ntent-driven \textbf{\underline{P}}reference \textbf{\underline{O}}ptimization (\textbf{A-IPO}). Specifically,A-IPO introduces an intention module that infers the latent intent behind each user prompt and explicitly incorporates this inferred intent into the reward function, encouraging stronger alignment between the preferred model’s responses and the user’s underlying intentions. We demonstrate, both theoretically and empirically, that incorporating an intention–response similarity term increases the preference margin (by a positive shift of $\lambda,\Delta\mathrm{sim}$ in the log-odds), resulting in clearer separation between preferred and dispreferred responses compared to DPO. For evaluation, we introduce two new benchmarks, Real-pref, Attack-pref along with an extended version of an existing dataset, GlobalOpinionQA-Ext, to assess real-world and adversarial preference alignment. Through explicit modeling of diverse user intents,A-IPO facilitates pluralistic preference optimization while simultaneously enhancing adversarial robustness in preference alignment. Comprehensive empirical evaluation demonstrates that A-IPO consistently surpasses existing baselines, yielding substantial improvements across key metrics: up to +24.8 win-rate and +45.6 Response-Intention Consistency on Real-pref; up to +38.6 Response Similarity and +52.2 Defense Success Rate on Attack-pref; and up to +54.6 Intention Consistency Score on GlobalOpinionQA-Ext.

Method: Multi-agent collaboration pipeline that simulates dialogues between test characters and other characters from a character-scene pool, with an LLM judge selecting evaluation dimensions and adjusting responses into test utterances.

Result: Built FURINA-Bench with established and synthesized characters. Found o3 and DeepSeek-R1 perform best on English/Chinese tasks respectively. Established characters outperform synthesized ones, with reasoning amplifying this disparity. Model scale doesn’t monotonically reduce hallucinations. Reasoning LLMs show trade-off: improved RP performance but increased hallucinations.

Conclusion: FURINA-Builder effectively addresses benchmark limitations and FURINA-Bench poses significant challenges, revealing a Pareto frontier between RP performance and reliability across all LLMs.

Abstract: As large language models (LLMs) advance in role-playing (RP) tasks, existing benchmarks quickly become obsolete due to their narrow scope, outdated interaction paradigms, and limited adaptability across diverse application scenarios. To address this gap, we introduce FURINA-Builder, a novel multi-agent collaboration pipeline that automatically constructs fully customizable RP benchmarks at any scale. It enables evaluation of arbitrary characters across diverse scenarios and prompt formats, as the first benchmark builder in RP area for adaptable assessment. FURINA-Builder simulates dialogues between a test character and other characters drawn from a well-constructed character-scene pool, while an LLM judge selects fine-grained evaluation dimensions and adjusts the test character’s responses into final test utterances. Using this pipeline, we build FURINA-Bench, a new comprehensive role-playing benchmark featuring both established and synthesized test characters, each assessed with dimension-specific evaluation criteria. Human evaluation and preliminary separability analysis justify our pipeline and benchmark design. We conduct extensive evaluations of cutting-edge LLMs and find that o3 and DeepSeek-R1 achieve the best performance on English and Chinese RP tasks, respectively. Across all models, established characters consistently outperform synthesized ones, with reasoning capabilities further amplifying this disparity. Interestingly, we observe that model scale does not monotonically reduce hallucinations. More critically, for reasoning LLMs, we uncover a novel trade-off: reasoning improves RP performance but simultaneously increases RP hallucinations. This trade-off extends to a broader Pareto frontier between RP performance and reliability for all LLMs. These findings demonstrate the effectiveness of FURINA-Builder and the challenge posed by FURINA-Bench.

Method: Proposed PerAugy - a data augmentation technique that performs cross-trajectory shuffling and summary-content perturbation to generate diverse training data for personalized summarization models.

Result: PerAugy boosted accuracy of four SOTA user-encoders (best: 0.132↑ AUC) and increased personalization in two summarizer frameworks by 61.2% on average (PSE-SU4 metric). Introduced diversity metrics (TP, RTC, DegreeD) showed strong correlation between dataset diversity and performance gains.

Conclusion: Increased dataset diversity through PerAugy’s augmentation is a key factor driving performance improvements in personalized summarization, with TP and DegreeD metrics strongly correlating with user-encoder performance across all accuracy measures.

Abstract: Document summarization enables efficient extraction of user-relevant content but is inherently shaped by individual subjectivity, making it challenging to identify subjective salient information in multifaceted documents. This complexity underscores the necessity for personalized summarization. However, training models for personalized summarization has so far been challenging, particularly because diverse training data containing both user preference history (i.e., click-skip trajectory) and expected (gold-reference) summaries are scarce. The MS/CAS PENS dataset is a valuable resource but includes only preference history without target summaries, preventing end-to-end supervised learning, and its limited topic-transition diversity further restricts generalization. To address this, we propose $\mathrm{PerAugy}$, a novel cross-trajectory shuffling and summary-content perturbation based data augmentation technique that significantly boosts the accuracy of four state-of-the-art baseline (SOTA) user-encoders commonly used in personalized summarization frameworks (best result: $\text{0.132}$$\uparrow$ w.r.t AUC). We select two such SOTA summarizer frameworks as baselines and observe that when augmented with their corresponding improved user-encoders, they consistently show an increase in personalization (avg. boost: $\text{61.2%}\uparrow$ w.r.t. PSE-SU4 metric). As a post-hoc analysis of the role of induced diversity in the augmented dataset by \peraugy, we introduce three dataset diversity metrics – $\mathrm{TP}$, $\mathrm{RTC}$, and \degreed\ to quantify the induced diversity. We find that $\mathrm{TP}$ and $\mathrm{DegreeD}$ strongly correlate with user-encoder performance on the PerAugy-generated dataset across all accuracy metrics, indicating that increased dataset diversity is a key factor driving performance gains.

Method: Uses a two-stage stop discriminator to identify reflective yet redundant reasoning and a sliding-window Upper Confidence Bound multi-armed bandit controller to dynamically adjust stopping thresholds based on problem difficulty.

Result: Across four benchmarks and two model families, REFRAIN reduces token usage by 20-55% while maintaining or improving accuracy compared to standard CoT prompting.

Conclusion: When-to-stop reasoning is a practical axis of test-time scaling that enables models to reason just enough rather than more.

Abstract: Chain-of-Thought (CoT) reasoning has driven recent gains of large language models (LLMs) on reasoning-intensive tasks by externalizing intermediate steps. However, excessive or redundant reasoning – so-called overthinking – can increase inference costs and lead LLMs toward incorrect conclusions. In this paper, we present REFRAIN ($\underline{REF}$lective-$\underline{R}$edundancy for $\underline{A}$daptive $\underline{IN}$ference), a training-free framework that adaptively determines when to stop reasoning to mitigate overthinking. REFRAIN integrates a two-stage stop discriminator to identify reflective yet redundant reasoning and a sliding-window Upper Confidence Bound (SW-UCB) multi-armed bandit controller to dynamically adjust stopping thresholds according to problem difficulty without supervision or fine-tuning. Across four representative benchmarks and two model families, REFRAIN reduces token usage by 20-55% while maintaining or improving accuracy compared to standard CoT prompting. Extensive ablation and robustness analyses demonstrate its stability across models, scorers, and prompt variations. In summary, our findings highlight when-to-stop as a new and practical axis of test-time scaling – enabling models to reason not just more, but just enough.

Method: Constructs Tri-Graph using lightweight entity extraction and semantic linking without relation modeling, then uses two-stage retrieval: entity activation via semantic bridging followed by passage retrieval through importance aggregation.

Result: Significantly outperforms baseline models on four datasets, demonstrating superior retrieval performance.

Conclusion: LinearRAG provides an economical and reliable alternative to traditional GraphRAG by eliminating relation extraction while maintaining effective retrieval capabilities for complex queries.

Abstract: Retrieval-Augmented Generation (RAG) is widely used to mitigate hallucinations of Large Language Models (LLMs) by leveraging external knowledge. While effective for simple queries, traditional RAG systems struggle with large-scale, unstructured corpora where information is fragmented. Recent advances incorporate knowledge graphs to capture relational structures, enabling more comprehensive retrieval for complex, multi-hop reasoning tasks. However, existing graph-based RAG (GraphRAG) methods rely on unstable and costly relation extraction for graph construction, often producing noisy graphs with incorrect or inconsistent relations that degrade retrieval quality. In this paper, we revisit the pipeline of existing GraphRAG systems and propose LinearRAG (Linear Graph-based Retrieval-Augmented Generation), an efficient framework that enables reliable graph construction and precise passage retrieval. Specifically, LinearRAG constructs a relation-free hierarchical graph, termed Tri-Graph, using only lightweight entity extraction and semantic linking, avoiding unstable relation modeling. This new paradigm of graph construction scales linearly with corpus size and incurs no extra token consumption, providing an economical and reliable indexing of the original passages. For retrieval, LinearRAG adopts a two-stage strategy: (i) relevant entity activation via local semantic bridging, followed by (ii) passage retrieval through global importance aggregation. Extensive experiments on four datasets demonstrate that LinearRAG significantly outperforms baseline models.

Method: Systematic framework combining OCR engines with LLMs, implementing 25 configurations across three extraction paradigms (direct, replacement, table-based) with intelligent strategy selection based on document-specific characteristics.

Result: Outstanding performance: F1=1.0 accuracy with 0.97s latency for structured documents, and F1=0.997 accuracy with 0.6s for image inputs using PaddleOCR. 54x performance improvement compared to multimodal methods, maintaining sub-second processing speeds.

Conclusion: The repetitive nature of copy-heavy tasks can be transformed from computational burden into optimization opportunity through structure-aware method selection, establishing a general principle for enterprise document processing.

Abstract: Information extraction from copy-heavy documents, characterized by massive volumes of structurally similar content, represents a critical yet understudied challenge in enterprise document processing. We present a systematic framework that strategically combines OCR engines with Large Language Models (LLMs) to optimize the accuracy-efficiency trade-off inherent in repetitive document extraction tasks. Unlike existing approaches that pursue universal solutions, our method exploits document-specific characteristics through intelligent strategy selection. We implement and evaluate 25 configurations across three extraction paradigms (direct, replacement, and table-based) on identity documents spanning four formats (PNG, DOCX, XLSX, PDF). Through table-based extraction methods, our adaptive framework delivers outstanding results: F1=1.0 accuracy with 0.97s latency for structured documents, and F1=0.997 accuracy with 0.6 s for challenging image inputs when integrated with PaddleOCR, all while maintaining sub-second processing speeds. The 54 times performance improvement compared with multimodal methods over naive approaches, coupled with format-aware routing, enables processing of heterogeneous document streams at production scale. Beyond the specific application to identity extraction, this work establishes a general principle: the repetitive nature of copy-heavy tasks can be transformed from a computational burden into an optimization opportunity through structure-aware method selection.

Method: 1) Compare Direct-Answer vs Chain-of-Thought prompting across 8 LLMs; 2) Define token-to-head contribution score to trace bias to specific attention heads; 3) Propose DiffHeads that identifies bias heads through differential activation analysis and selectively masks them.

Result: Direct-Answer prompting increases unfairness by 534.5%-391.9%. A small cluster of bias heads activate under DA but stay dormant with CoT. DiffHeads reduces unfairness by 49.4% under DA and 40.3% under CoT.

Conclusion: The paper provides the first causal link between prompting strategies and bias emergence, and demonstrates that selective masking of identified bias heads effectively reduces unfairness while preserving model utility.

Abstract: Large language models (LLMs) increasingly mediate decisions in domains where unfair treatment of demographic groups is unacceptable. Existing work probes when biased outputs appear, but gives little insight into the mechanisms that generate them, leaving existing mitigations largely fragile. In this paper, we conduct a systematic investigation LLM unfairness and propose DiffHeads, a lightweight debiasing framework for LLMs. We first compare Direct-Answer (DA) prompting to Chain-of-Thought (CoT) prompting across eight representative open- and closed-source LLMs. DA will trigger the nature bias part of LLM and improve measured unfairness by 534.5%-391.9% in both one-turn and two-turn dialogues. Next, we define a token-to-head contribution score that traces each token’s influence back to individual attention heads. This reveals a small cluster of bias heads that activate under DA but stay largely dormant with CoT, providing the first causal link between prompting strategy and bias emergence. Finally, building on this insight, we propose DiffHeads that identifies bias heads through differential activation analysis between DA and CoT, and selectively masks only those heads. DiffHeads reduces unfairness by 49.4%, and 40.3% under DA and CoT, respectively, without harming model utility.

Method: Extracts and blends multiple distinct persona vectors directly in the model’s activation space, then steers generation with this merged vector during inference to enable multi-perspective output without multi-LLM communication.

Result: BILLY surpasses single model prompting and traditional multi-LLM approaches on creativity benchmarks while substantially reducing inference time and computational costs.

Conclusion: Persona vector blending enables effective control over complementary generation aspects and greater interpretability, providing multi-LLM benefits in a single model.

Abstract: Multi-LLM systems enhance the creativity of large language models by simulating human collective intelligence but suffer from significant drawbacks, such as high computational costs and inference latency. To address these limitations, we propose BILLY (BlendIng persona vectors for Large Language model creativitY), a training-free framework that captures the benefits of multi-LLM collaboration, i.e. inducing diverse perspectives and specialized expertise, within a single model. BILLY operates by extracting and blending multiple distinct persona vectors directly in the model’s activation space. We steer the model’s generation process with this merged vector while inference, enabling multi-perspective output without explicit multi-LLM communication. Our experiments across creativity-oriented benchmarks demonstrate that BILLY surpasses single model prompting and traditional multi-LLM approaches, while substantially reducing inference time and computational costs. Our analyses further reveal that distinct persona vectors can be blended to achieve both effective control over complementary aspects of generation and greater interpretability.

Method: Curated developmentally plausible pretraining data equivalent to 100M English words in each of 45 languages, compiled evaluation suites, and trained baseline models for each language.

Result: Created a comprehensive multilingual dataset collection with evaluation benchmarks and baseline models across 45 languages.

Conclusion: BabyBabelLM provides a valuable resource for advancing multilingual pretraining and cognitive modeling research by offering developmentally realistic language data across multiple languages.

Abstract: We present BabyBabelLM, a multilingual collection of datasets modeling the language a person observes from birth until they acquire a native language. We curate developmentally plausible pretraining data aiming to cover the equivalent of 100M English words of content in each of 45 languages. We compile evaluation suites and train baseline models in each language. BabyBabelLM aims to facilitate multilingual pretraining and cognitive modeling.

Method: Organizes provenance tracking around four interrelated dimensions capturing both model- and data-centric perspectives, with a dual-paradigm taxonomy classifying methods into prior-based (proactive traceability embedding) and posterior-based (retrospective inference) approaches.

Result: The survey provides a comprehensive framework for tracing LLM-generated content origins across different dimensions, enhancing understanding of how content is shaped by model architecture, training data, and external influences.

Conclusion: Provenance tracking across these dimensions enhances transparency, accountability, and trustworthiness of LLM deployment in real-world applications, addressing critical risks associated with their widespread use.

Abstract: The rapid advancement of large language models (LLMs) has revolutionized artificial intelligence, shifting from supporting objective tasks (e.g., recognition) to empowering subjective decision-making (e.g., planning, decision). This marks the dawn of general and powerful AI, with applications spanning a wide range of fields, including programming, education, healthcare, finance, and law. However, their deployment introduces multifaceted risks. Due to the black-box nature of LLMs and the human-like quality of their generated content, issues such as hallucinations, bias, unfairness, and copyright infringement become particularly significant. In this context, sourcing information from multiple perspectives is essential. This survey presents a systematic investigation into provenance tracking for content generated by LLMs, organized around four interrelated dimensions that together capture both model- and data-centric perspectives. From the model perspective, Model Sourcing treats the model as a whole, aiming to distinguish content generated by specific LLMs from content authored by humans. Model Structure Sourcing delves into the internal generative mechanisms, analyzing architectural components that shape the outputs of model. From the data perspective, Training Data Sourcing focuses on internal attribution, tracing the origins of generated content back to the training data of model. In contrast, External Data Sourcing emphasizes external validation, identifying external information used to support or influence the responses of model. Moreover, we also propose a dual-paradigm taxonomy that classifies existing sourcing methods into prior-based (proactive traceability embedding) and posterior-based (retrospective inference) approaches. Traceability across these dimensions enhances the transparency, accountability, and trustworthiness of LLMs deployment in real-world applications.

Method: Categorizes methods into post-training, test-time scaling, and data augmentation; summarizes benchmarks; derives unified sandbox-based evaluation with observation coverage metric.

Result: Provides comprehensive categorization of inductive reasoning methods and benchmarks, along with analysis of inductive ability sources.

Conclusion: The survey establishes a solid foundation for future research on inductive reasoning in LLMs by systematizing current knowledge and evaluation approaches.

Abstract: Reasoning is an important task for large language models (LLMs). Among all the reasoning paradigms, inductive reasoning is one of the fundamental types, which is characterized by its particular-to-general thinking process and the non-uniqueness of its answers. The inductive mode is crucial for knowledge generalization and aligns better with human cognition, so it is a fundamental mode of learning, hence attracting increasing interest. Despite the importance of inductive reasoning, there is no systematic summary of it. Therefore, this paper presents the first comprehensive survey of inductive reasoning for LLMs. First, methods for improving inductive reasoning are categorized into three main areas: post-training, test-time scaling, and data augmentation. Then, current benchmarks of inductive reasoning are summarized, and a unified sandbox-based evaluation approach with the observation coverage metric is derived. Finally, we offer some analyses regarding the source of inductive ability and how simple model architectures and data help with inductive tasks, providing a solid foundation for future research.

Method: Delora introduces clean and noisy LoRA modules to establish a noisy label detector. The clean LoRA memorizes clean data while the noisy LoRA memorizes mislabeled data, serving as a learnable threshold for sample selection, decoupling this process from model training.

Result: Experimental results on synthetic and real-world noisy datasets show Delora’s effectiveness in noisy label detection and text classification tasks.

Conclusion: Delora successfully breaks the vicious cycle of noisy label learning by decoupling sample selection from model training, providing an effective solution for PEFT of LLMs with noisy labels.

Abstract: Parameter-efficient fine-tuning (PEFT) large language models (LLMs) have shown impressive performance in various downstream tasks. However, in many real-world scenarios, the collected training data inevitably contains noisy labels. To learn from noisy labels, most solutions select samples with small losses for model training. However, the selected samples, in turn, impact the loss computation in the next iteration. An inaccurate initial selection can create a vicious cycle, leading to suboptimal performance. To break this cycle, we propose Delora, a novel framework that decouples the sample selection from model training. For sample selection, Delora establishes a noisy label detector by introducing clean and noisy LoRA. Benefiting from the memory effect, the clean LoRA is encouraged to memorize clean data, while the noisy LoRA is constrained to memorize mislabeled data, which serves as a learnable threshold for selecting clean and noisy samples. For model training, Delora can use carefully selected samples to fine-tune language models seamlessly. Experimental results on synthetic and real-world noisy datasets demonstrate the effectiveness of Delora in noisy label detection and text classification.

Method: SyTTA couples two uncertainty signals: input-side perplexity (domain mismatch) and output-side predictive entropy (unstable token probabilities) for on-the-fly adaptation without supervision.

Result: Consistent gains across diverse models and domains; on agricultural QA, improves Rouge-LSum by over 120% on Qwen-2.5-7B with only 4 extra tokens per query.

Conclusion: Effective test-time adaptation for LLMs is achievable without labeled examples, supporting deployment in label-scarce domains.

Abstract: Large language models (LLMs) are increasingly deployed in specialized domains such as finance, medicine, and agriculture, where they face significant distribution shifts from their training data. Domain-specific fine-tuning can mitigate this challenge but relies on high-quality labeled data that is expensive and slow to collect in expertise-limited settings. We study label-free test-time adaptation for language models and present SyTTA, an inference-time framework that adapts models on-the-fly without additional supervision. SyTTA couples two complementary uncertainty signals that arise under distribution shift: input-side perplexity, indicating mismatch with domain-specific terminology and patterns, and output-side predictive entropy, indicating diffuse and unstable token probabilities during generation. Across diverse model architectures and domain-specific benchmarks, SyTTA delivers consistent gains. Notably, on agricultural question answering, SyTTA improves Rouge-LSum by over 120% on Qwen-2.5-7B with only 4 extra tokens per query. These results show that effective test-time adaptation for language models is achievable without labeled examples, supporting deployment in label-scarce domains. The code will be made available upon acceptance.

Method: The framework uses token target prediction with two methods to construct target token distributions: data-driven (extracting meaningful tokens from text) and model-derived (using semantically derived tokens from LLM backbone).

Result: Experiments on MTEB v2 benchmark show Text2Token achieves competitive performance with state-of-the-art unsupervised contrastive learning embedder LLM2Vec.

Conclusion: The analysis reveals that vocabulary and representation spaces optimize together toward optimal solutions during training, providing new insights for future work in text representation learning.

Abstract: Unsupervised text representation learning (TRL) is a fundamental task in natural language processing, which is beneficial for improving search and recommendations with the web’s unlabeled texts. A recent empirical study finds that the high-quality representation aligns with the key token of the input text, uncovering the potential connection between representation space and vocabulary space. Inspired by the findings, we revisit the generative tasks and develop an unsupervised generative framework for TRL, Text2Token. The framework is based on the token target prediction task, utilizing carefully constructed target token distribution as supervisory signals. To construct the high-quality target token distribution, we analyze the token-alignment properties with advanced embedders and identify two essential categories of key tokens: (1) the meaningful tokens in the text and (2) semantically derived tokens beyond the text. Based on these insights, we propose two methods – data-driven and model-derived – to construct synthetic token targets from data or the LLM backbone. Experiments on the MTEB v2 benchmark demonstrate that Text2Token achieves performance competitive with the state-of-the-art embedder with unsupervised contrastive learning, LLM2Vec. Our analysis further shows that vocabulary and representation spaces optimize together and toward the optimum solution during training, providing new ideas and insights for future work.

Method: Proposes ImCoref-CeS framework with: 1) ImCoref - improved supervised method with lightweight bridging module for long-text encoding, biaffine scorer for positional information, and hybrid mention regularization; 2) LLM as Checker-Splitter agent to validate candidate mentions and split erroneous clusters.

Result: Extensive experiments demonstrate effectiveness and superior performance compared to existing state-of-the-art methods.

Conclusion: The proposed ImCoref-CeS framework successfully integrates enhanced supervised models with LLM reasoning, achieving top performance in coreference resolution.

Abstract: Coreference Resolution (CR) is a critical task in Natural Language Processing (NLP). Current research faces a key dilemma: whether to further explore the potential of supervised neural methods based on small language models, whose detect-then-cluster pipeline still delivers top performance, or embrace the powerful capabilities of Large Language Models (LLMs). However, effectively combining their strengths remains underexplored. To this end, we propose \textbf{ImCoref-CeS}, a novel framework that integrates an enhanced supervised model with LLM-based reasoning. First, we present an improved CR method (\textbf{ImCoref}) to push the performance boundaries of the supervised neural method by introducing a lightweight bridging module to enhance long-text encoding capability, devising a biaffine scorer to comprehensively capture positional information, and invoking a hybrid mention regularization to improve training efficiency. Importantly, we employ an LLM acting as a multi-role Checker-Splitter agent to validate candidate mentions (filtering out invalid ones) and coreference results (splitting erroneous clusters) predicted by ImCoref. Extensive experiments demonstrate the effectiveness of ImCoref-CeS, which achieves superior performance compared to existing state-of-the-art (SOTA) methods.

Method: Three-phase framework: (1) decomposing queries into candidate assumptions, (2) auditing their support, and (3) conditioning inference only on validated subset. Formally, posterior-constrained inference connecting to selective prediction and rejection learning.

Result: Empirical improvements: +30% gains on GSM8K, +45% on MultiArith, and consistent +20-28% improvements on SVAMP over Chain-of-Thought, Self-Consistency, and CoT-Decoding. Also provides theoretical guarantees under perfect validation and excess-risk bounds under imperfect audits.

Conclusion: AoU effectively addresses reasoning-induced hallucinations in LLMs by constraining inference to validated premises, demonstrating significant improvements in accuracy and faithfulness across multiple benchmarks.

Abstract: Large language models (LLMs) often generate reasoning traces that appear coherent but rest on unsupported assumptions, leading to hallucinated conclusions. Prior work mainly addresses factual hallucinations or relies on post-hoc verification, leaving reasoning-induced hallucinations largely unaddressed. We propose Audit-of-Understanding (AoU), a framework that constrains inference to validated premises through three phases: (1) decomposing a query into candidate assumptions, (2) auditing their support, and (3) conditioning inference only on the validated subset. Formally, AoU is \emph{posterior-constrained inference}, connecting to selective prediction and rejection learning. Our contributions are threefold: (i) theoretical guarantees under perfect validation, (ii) excess-risk bounds under imperfect audits, and (iii) tractability analysis. Empirically, AoU improves both accuracy and faithfulness on GSM8K, MultiArith, and SVAMP, achieving up to +30% gains on GSM8K, +45% on MultiArith, and consistent +20–28% improvements on SVAMP over Chain-of-Thought, Self-Consistency, and CoT-Decoding. Code is available at https://anonymous.4open.science/r/audit-of-understanding-E28B.

Method: Two-stage process: (1) aggregating backdoor representations by injecting known triggers into compromised models, (2) performing recovery fine-tuning to restore benign outputs.

Result: Reduces average Attack Success Rate to 4.41% across benchmarks (28.1%~69.3% improvement over baselines), preserves clean accuracy within 0.5% of original model, and generalizes across different backdoor types.

Conclusion: The defense framework is effective, practical, and robust for real-world deployment scenarios without requiring trigger knowledge.

Abstract: Backdoor attacks are a significant threat to large language models (LLMs), often embedded via public checkpoints, yet existing defenses rely on impractical assumptions about trigger settings. To address this challenge, we propose \ourmethod, a defense framework that requires no prior knowledge of trigger settings. \ourmethod is based on the key observation that when deliberately injecting known backdoors into an already-compromised model, both existing unknown and newly injected backdoors aggregate in the representation space. \ourmethod leverages this through a two-stage process: \textbf{first}, aggregating backdoor representations by injecting known triggers, and \textbf{then}, performing recovery fine-tuning to restore benign outputs. Extensive experiments across multiple LLM architectures demonstrate that: (I) \ourmethod reduces the average Attack Success Rate to 4.41% across multiple benchmarks, outperforming existing baselines by 28.1%$\sim$69.3%$\uparrow$. (II) Clean accuracy and utility are preserved within 0.5% of the original model, ensuring negligible impact on legitimate tasks. (III) The defense generalizes across different types of backdoors, confirming its robustness in practical deployment scenarios.

Method: The authors assess entity alignment through translation tasks and propose two methods: SubSub (substituting subjects with English translations) and SubInj (injecting English subject translations into prompts). They also conduct mechanistic analysis to understand how these interventions work.

Result: The study finds strong correlation between alignment and consistency across all models. The proposed methods (SubSub and SubInj) significantly improve both factual recall accuracy and crosslingual consistency by reinforcing entity representation alignment through the model’s internal pivot-language processing.

Conclusion: Entity alignment is a key factor in crosslingual consistency of factual knowledge in multilingual LLMs. The proposed interventions offer practical strategies for improving multilingual factual prediction by leveraging the model’s internal language processing mechanisms.

Abstract: Multilingual large language models (LLMs) are expected to recall factual knowledge consistently across languages. However, the factors that give rise to such crosslingual consistency – and its frequent failure – remain poorly understood. In this work, we hypothesize that these inconsistencies may arise from failures in entity alignment, the process of mapping subject and object entities into a shared conceptual space across languages. To test this, we assess alignment through entity-level (subject and object) translation tasks, and find that consistency is strongly correlated with alignment across all studied models, with misalignment of subjects or objects frequently resulting in inconsistencies. Building on this insight, we propose SubSub and SubInj, two effective methods that integrate English translations of subjects into prompts across languages, leading to substantial gains in both factual recall accuracy and consistency. Finally, our mechanistic analysis reveals that these interventions reinforce the entity representation alignment in the conceptual space through model’s internal pivot-language processing, offering effective and practical strategies for improving multilingual factual prediction.

Method: Recursive exploitation of the model’s intrinsic capabilities in reasoning, verification, and summarization to enhance retention of correct solutions and reduce Pass@k vs Pass@1 disparity.

Result: Achieves 99.79 score on AIME2025 using only 4% of the computation required by DeepConf, with comprehensive validation across multiple models and benchmarks.

Conclusion: Provides new insights for designing efficient and scalable test-time inference strategies for advanced language models.

Abstract: Test-time scaling has emerged as a promising paradigm in language modeling, wherein additional computational resources are allocated during inference to enhance model performance. Recent approaches, such as DeepConf, have demonstrated the efficacy of this strategy, however, they often incur substantial computational overhead to achieve competitive results. In this work, we propose MatryoshkaThinking, a novel method that significantly reduces computational cost while maintaining state-of-the-art performance. Specifically, MatryoshkaThinking attains a score of 99.79 on AIME2025 using only 4% of the computation required by DeepConf. The core of our approach lies in the recursive exploitation of the model’s intrinsic capabilities in reasoning, verification, and summarization, which collectively enhance the retention of correct solutions and reduce the disparity between Pass@k and Pass@1. Comprehensive evaluations across multiple open-source models and challenging multi-modal reasoning benchmarks validate the effectiveness and generality of our method. These findings offer new insights into the design of efficient and scalable test-time inference strategies for advanced language models.

Method: Proposed a framework analyzing cognitive and affective empathy across 315 unique personas defined by intersecting demographic attributes (age, culture, gender) across four LLMs, using both quantitative and qualitative analysis.

Result: Attributes profoundly shape empathetic responses, with multiple attributes sometimes attenuating or reversing expected empathy patterns. Models broadly reflect real-world empathetic trends but show notable misalignments for certain groups like Confucian culture.

Conclusion: Designing empathy-aware LLMs that account for demographic diversity is crucial for promoting more inclusive and equitable model behavior.

Abstract: Large Language Models’ (LLMs) ability to converse naturally is empowered by their ability to empathetically understand and respond to their users. However, emotional experiences are shaped by demographic and cultural contexts. This raises an important question: Can LLMs demonstrate equitable empathy across diverse user groups? We propose a framework to investigate how LLMs’ cognitive and affective empathy vary across user personas defined by intersecting demographic attributes. Our study introduces a novel intersectional analysis spanning 315 unique personas, constructed from combinations of age, culture, and gender, across four LLMs. Results show that attributes profoundly shape a model’s empathetic responses. Interestingly, we see that adding multiple attributes at once can attenuate and reverse expected empathy patterns. We show that they broadly reflect real-world empathetic trends, with notable misalignments for certain groups, such as those from Confucian culture. We complement our quantitative findings with qualitative insights to uncover model behaviour patterns across different demographic groups. Our findings highlight the importance of designing empathy-aware LLMs that account for demographic diversity to promote more inclusive and equitable model behaviour.

Method: Combined end-to-end architecture using pre-trained speech encoders and Large Language Models (LLMs) for simultaneous ASR and ST tasks.

Result: Best model achieves better translation results than SeamlessM4T and matches performance of cascaded Whisper+NLLB system, with up to 8% gain in COMET-DA22 metric for English-to-German translation.

Conclusion: The combined end-to-end architecture with pre-trained speech encoders and LLMs is effective for simultaneous ASR and ST, outperforming large foundational models and matching cascaded systems.

Abstract: Speech Translation (ST) is a machine translation task that involves converting speech signals from one language to the corresponding text in another language; this task has two different approaches, namely the traditional cascade and the more recent end-to-end. This paper explores a combined end-to-end architecture of pre-trained speech encoders and Large Language Models (LLMs) for performing both Automatic Speech Recognition (ASR) and ST simultaneously. Experiments with the English-to-German language pair show that our best model not only can achieve better translation results than SeamlessM4T, a large foundational end-to-end, multi-modal translation model, but can also match the performance of a cascaded system with Whisper and NLLB, with up to a score gain of 8% in $\text{COMET}^{\text{DA}}_{22}$ metric.

Method: Developed a Python package that automatically tags ASCs and computes 50 indices covering diversity, proportion, frequency, and ASC-verb lemma association strength. Conducted bivariate and multivariate analyses to examine relationships with L2 writing scores.

Result: The paper presents the ASC analyzer tool and demonstrates its utility through empirical analysis linking ASC-based indices to L2 writing proficiency scores.

Conclusion: The ASC analyzer provides a systematic and scalable solution for measuring argument structure construction usage in L2 writing assessment, filling an important gap in language proficiency analysis tools.

Abstract: Argument structure constructions (ASCs) offer a theoretically grounded lens for analyzing second language (L2) proficiency, yet scalable and systematic tools for measuring their usage remain limited. This paper introduces the ASC analyzer, a publicly available Python package designed to address this gap. The analyzer automatically tags ASCs and computes 50 indices that capture diversity, proportion, frequency, and ASC-verb lemma association strength. To demonstrate its utility, we conduct both bivariate and multivariate analyses that examine the relationship between ASC-based indices and L2 writing scores.

Method: Introduces RefusalBench, a generative methodology that programmatically creates diagnostic test cases through controlled linguistic perturbation. Uses 176 distinct perturbation strategies across six categories of informational uncertainty and three intensity levels.

Result: Evaluation of over 30 models reveals systematic failure patterns: refusal accuracy drops below 50% on multi-document tasks, models exhibit dangerous overconfidence or overcaution, and neither scale nor extended reasoning improves performance. Refusal comprises separable detection and categorization skills.

Conclusion: Selective refusal is a trainable, alignment-sensitive capability that offers a clear path for improvement. The authors release two benchmarks (RefusalBench-NQ and RefusalBench-GaRAGe) and the complete generation framework for continued dynamic evaluation of this critical capability.

Abstract: The ability of language models in RAG systems to selectively refuse to answer based on flawed context is critical for safety, yet remains a significant failure point. Our large-scale study reveals that even frontier models struggle in this setting, with refusal accuracy dropping below 50% on multi-document tasks, while exhibiting either dangerous overconfidence or overcaution. Static benchmarks fail to reliably evaluate this capability, as models exploit dataset-specific artifacts and memorize test instances. We introduce RefusalBench, a generative methodology that programmatically creates diagnostic test cases through controlled linguistic perturbation. Our framework employs 176 distinct perturbation strategies across six categories of informational uncertainty and three intensity levels. Evaluation of over 30 models uncovers systematic failure patterns: refusal comprises separable detection and categorization skills, and neither scale nor extended reasoning improves performance. We find that selective refusal is a trainable, alignment-sensitive capability, offering a clear path for improvement. We release two benchmarks – RefusalBench-NQ (single document) and RefusalBench-GaRAGe (multi-document) – and our complete generation framework to enable continued, dynamic evaluation of this critical capability.

Method: Proposes AssoMem framework that constructs an associative memory graph anchoring dialogue utterances to automatically extracted clues. Uses multi-dimensional retrieval signals (relevance, importance, temporal alignment) with adaptive mutual information-driven fusion strategy.

Result: Extensive experiments across three benchmarks and the new MeetingQA dataset show AssoMem consistently outperforms state-of-the-art baselines, demonstrating superiority in context-aware memory recall.

Conclusion: AssoMem provides an effective solution for memory recall in QA systems by leveraging associative memory structures and multi-dimensional retrieval signals, particularly beneficial in similarity-dense scenarios.

Abstract: Accurate recall from large scale memories remains a core challenge for memory augmented AI assistants performing question answering (QA), especially in similarity dense scenarios where existing methods mainly rely on semantic distance to the query for retrieval. Inspired by how humans link information associatively, we propose AssoMem, a novel framework constructing an associative memory graph that anchors dialogue utterances to automatically extracted clues. This structure provides a rich organizational view of the conversational context and facilitates importance aware ranking. Further, AssoMem integrates multi-dimensional retrieval signals-relevance, importance, and temporal alignment using an adaptive mutual information (MI) driven fusion strategy. Extensive experiments across three benchmarks and a newly introduced dataset, MeetingQA, demonstrate that AssoMem consistently outperforms SOTA baselines, verifying its superiority in context-aware memory recall.

Method: STEAM identifies semantic anchors for factual associations and guides internal representations toward these anchors using alignment loss during optimization.

Result: Experimental results show STEAM improves model’s reasoning ability with edited knowledge and enhances semantic coherence compared to locate-and-edit methods.

Conclusion: Latent-space alignment is crucial for reliable and coherent knowledge editing, enabling better integration of updated facts into the model’s knowledge structure.

Abstract: Large Language Models store extensive factual knowledge acquired during large-scale pre-training. However, this knowledge is inherently static, reflecting only the state of the world at the time of training. Knowledge editing has emerged as a promising solution for updating outdated or incorrect facts without full retraining. However, most existing locate-and-edit methods primarily focus on token-level likelihood optimization without addressing semantic coherence. Our analysis reveals that such edited knowledge is often encoded as isolated residual streams in the model’s latent space, distinct from pre-existing knowledge and bypassing natural reasoning process. To address this, we propose \textsc{Steam}, a semantic-level knowledge editing framework that enhances integration of updated knowledge into the model’s knowledge structure. \textsc{Steam} first identifies target representations as semantic anchors for the updated factual association, then guides the internal representation of the edited fact towards these anchors through an alignment loss during optimization. Experimental results demonstrate that \textsc{Steam} improves model’s ability to reason with edited knowledge and enhances semantic coherence, underscoring the importance of latent-space alignment for reliable and coherent knowledge editing. The code is available at https://github.com/GY-Jeong/STEAM.

Method: Introduced LongQAEval framework comparing coarse answer-level vs fine-grained sentence-level evaluation across correctness, relevance, and safety dimensions using physician annotations of 300 real patient questions answered by physicians and LLMs.

Result: IAA varies by dimension: fine-grained annotation improves agreement on correctness, coarse improves agreement on relevance, and safety judgments remain inconsistent. Annotating only small sentence subsets provides reliability comparable to coarse annotations.

Conclusion: Partial sentence annotation can reduce cost and effort while maintaining reliability comparable to coarse evaluation methods in clinical QA assessment.

Abstract: Evaluating long-form clinical question answering (QA) systems is resource-intensive and challenging: accurate judgments require medical expertise and achieving consistent human judgments over long-form text is difficult. We introduce LongQAEval, an evaluation framework and set of evaluation recommendations for limited-resource and high-expertise settings. Based on physician annotations of 300 real patient questions answered by physicians and LLMs, we compare coarse answer-level versus fine-grained sentence-level evaluation over the dimensions of correctness, relevance, and safety. We find that inter-annotator agreement (IAA) varies by dimension: fine-grained annotation improves agreement on correctness, coarse improves agreement on relevance, and judgments on safety remain inconsistent. Additionally, annotating only a small subset of sentences can provide reliability comparable to coarse annotations, reducing cost and effort.

Method: Developed LISTEN benchmark with controlled tests to disentangle lexical reliance from acoustic sensitivity, evaluating six state-of-the-art LALMs across various conditions including neutral lexical cues, cue alignment, cue conflict, and paralinguistic settings.

Result: Models consistently showed lexical dominance - predicting ’neutral’ when lexical cues were neutral, limited performance gains with aligned cues, failure to classify distinct emotions under cue conflict, and near-chance performance in paralinguistic settings.

Conclusion: Current LALMs largely ’transcribe’ rather than ’listen,’ heavily relying on lexical semantics while underutilizing acoustic cues. LISTEN provides a principled framework for assessing emotion understanding in multimodal models.

Abstract: Understanding emotion from speech requires sensitivity to both lexical and acoustic cues. However, it remains unclear whether large audio language models (LALMs) genuinely process acoustic information or rely primarily on lexical content. We present LISTEN (Lexical vs. Acoustic Speech Test for Emotion in Narratives), a controlled benchmark designed to disentangle lexical reliance from acoustic sensitivity in emotion understanding. Across evaluations of six state-of-the-art LALMs, we observe a consistent lexical dominance. Models predict “neutral” when lexical cues are neutral or absent, show limited gains under cue alignment, and fail to classify distinct emotions under cue conflict. In paralinguistic settings, performance approaches chance. These results indicate that current LALMs largely “transcribe” rather than “listen,” relying heavily on lexical semantics while underutilizing acoustic cues. LISTEN offers a principled framework for assessing emotion understanding in multimodal models.

Method: RECON adds an explicit summarization module trained via two-stage process: relevance pretraining on QA datasets followed by multi-aspect distillation from proprietary LLMs for factuality and clarity. Integrated into Search-R1 pipeline.

Result: Reduces total context length by 35%, improves training speed and inference latency, boosts EM score by 14.5% for 3B model and 3.0% for 7B model, with particular strength in multi-hop QA.

Conclusion: Learned context compression is essential for building practical, scalable, and performant RAG systems.

Abstract: Retrieval-augmented generation (RAG) systems trained using reinforcement learning (RL) with reasoning are hampered by inefficient context management, where long, noisy retrieved documents increase costs and degrade performance. We introduce RECON (REasoning with CONdensation), a framework that integrates an explicit summarization module to compress evidence within the reasoning loop. Our summarizer is trained via a two-stage process: relevance pretraining on QA datasets, followed by multi-aspect distillation from proprietary LLMs to ensure factuality and clarity. Integrated into the Search-R1 pipeline, RECON reduces total context length by 35%, leading to improved training speed and inference latency, while simultaneously improving RAG performance on downstream QA benchmarks. Notably, it boosts the average EM score of the 3B model by 14.5% and the 7B model by 3.0%, showing particular strength in multi-hop QA. RECON demonstrates that learned context compression is essential for building practical, scalable, and performant RAG systems. Our code implementation is made available at https://github.com/allfornancy/RECON.

Method: Constructed RagRefuse benchmark with domain-stratified data, and introduced SafeRAG-Steering - a model-centric embedding intervention that steers embeddings toward safe, non-refusing output regions during inference.

Result: Analysis shows context arrangement, domain factors, and harmful-text density trigger refusals on benign queries, with effects varying by model alignment choices. SafeRAG-Steering reduces over-refusals in contaminated RAG pipelines.

Conclusion: SafeRAG-Steering effectively mitigates over-refusals in RAG systems while preserving legitimate safety refusals, addressing the trade-off between safety and utility in safety-aligned LLMs.

Abstract: Safety alignment in large language models (LLMs) induces over-refusals – where LLMs decline benign requests due to aggressive safety filters. We analyze this phenomenon in retrieval-augmented generation (RAG), where both the query intent and retrieved context properties influence refusal behavior. We construct RagRefuse, a domain-stratified benchmark spanning medical, chemical, and open domains, pairing benign and harmful queries with controlled context contamination patterns and sizes. Our analysis shows that context arrangement / contamination, domain of query and context, and harmful-text density trigger refusals even on benign queries, with effects depending on model-specific alignment choices. To mitigate over-refusals, we introduce \textsc{SafeRAG-Steering}, a model-centric embedding intervention that steers the embedding regions towards the confirmed safe, non-refusing output regions at inference time. This reduces over-refusals in contaminated RAG pipelines while preserving legitimate refusals.

Method: Introduces Time Independence Loss (TIL) and Aligned Cross Entropy (AXE) Loss based on edit distance, combined with frame fusion that weights and sums keyframes with their context frames.

Result: Experimental results on AISHELL-1 and AISHELL-2 dataset subsets show the proposed methods outperform previous work and achieve at least 86% reduction in frame count.

Conclusion: The proposed alignment methods effectively address speech-text length mismatch in ASR while significantly reducing computational requirements through frame reduction.

Abstract: The mismatch of speech length and text length poses a challenge in automatic speech recognition (ASR). In previous research, various approaches have been employed to align text with speech, including the utilization of Connectionist Temporal Classification (CTC). In earlier work, a key frame mechanism (KFDS) was introduced, utilizing intermediate CTC outputs to guide downsampling and preserve keyframes, but traditional methods (CTC) failed to align speech and text appropriately when downsampling speech to a text-similar length. In this paper, we focus on speech recognition in those cases where the length of speech aligns closely with that of the corresponding text. To address this issue, we introduce two methods for alignment: a) Time Independence Loss (TIL) and b) Aligned Cross Entropy (AXE) Loss, which is based on edit distance. To enhance the information on keyframes, we incorporate frame fusion by applying weights and summing the keyframe with its context 2 frames. Experimental results on AISHELL-1 and AISHELL-2 dataset subsets show that the proposed methods outperform the previous work and achieve a reduction of at least 86% in the number of frames.

Method: Uses sample-level redundancy analysis and frames benchmark compression as an optimization problem. Proposes EssenceBench with iterative Genetic Algorithm combining fitness-based subset search and attribution-based sample search.

Result: Achieves superior compression with lower reconstruction error and higher efficiency. On HellaSwag (10K samples), preserves all model rankings within 5% using 25x fewer samples, and 95% ranking preservation with 200x fewer samples.

Conclusion: EssenceBench effectively compresses benchmarks while maintaining ranking consistency, offering a systematic solution for large-scale benchmark evaluation.

Abstract: As the demand for comprehensive evaluations of diverse model capabilities steadily increases, benchmark suites have correspondingly grown significantly in scale. Despite notable advances in redundancy reduction and subset-level performance prediction, a systematic framework that effectively integrates these methods to ensure both prediction accuracy and ranking consistency is still largely elusive. In this paper, we first perform a sample-level analysis of benchmark redundancy and identify several highly similar samples that can be eliminated. Besides, we frame benchmark compression as an optimization problem with the aim of score reconstruction. Building on these, we then propose EssenceBench, a coarse-to-fine framework utilizing an iterative Genetic Algorithm (GA), which takes the advantages of fitness-based subset search and attribution-based sample search. Compared to previous methods, our approach yields superior compression results with lower reconstruction error and markedly higher efficiency. In particular, on the HellaSwag benchmark (10K samples), our method preserves the ranking of all models shifting within 5% using 25x fewer samples, and achieves 95% ranking preservation shifting within 5% using only 200x fewer samples.

Method: Combines neural-based LLMs with symbolic knowledge heuristics based on Natural Semantic Metalanguage (NSM) for semantic decomposition, using human-centric collaborative design with 200+ semi-literate participants.

Result: Achieved over 80% semantic comprehensibility, accessible learning curve, and universal adaptability for underprivileged populations with limited formal education.

Conclusion: The ideographic metalanguage system effectively serves underprivileged populations by transcending academic, linguistic, and cultural boundaries in digital communication.

Abstract: Digital communication has become the cornerstone of modern interaction, enabling rapid, accessible, and interactive exchanges. However, individuals with lower academic literacy often face significant barriers, exacerbating the “digital divide”. In this work, we introduce a novel, universal ideographic metalanguage designed as an innovative communication framework that transcends academic, linguistic, and cultural boundaries. Our approach leverages principles of Neuro-symbolic AI, combining neural-based large language models (LLMs) enriched with world knowledge and symbolic knowledge heuristics grounded in the linguistic theory of Natural Semantic Metalanguage (NSM). This enables the semantic decomposition of complex ideas into simpler, atomic concepts. Adopting a human-centric, collaborative methodology, we engaged over 200 semi-literate participants in defining the problem, selecting ideographs, and validating the system. With over 80% semantic comprehensibility, an accessible learning curve, and universal adaptability, our system effectively serves underprivileged populations with limited formal education.

Method: Developed FML-bench with 8 diverse fundamental ML research problems, reduced coding burden, and created a unified evaluation framework with five complementary metrics to comprehensively assess agent performance.

Result: Evaluation of state-of-the-art agents showed that those employing broad research exploration strategies outperformed those focusing on narrow but deep exploration, suggesting breadth of exploration leads to more effective research outcomes.

Conclusion: FML-bench provides a comprehensive benchmark for evaluating ML research agents, emphasizing fundamental problems and diverse exploration strategies, with findings suggesting broad exploration approaches are more effective than incremental refinement.

Abstract: Large language models (LLMs) have sparked growing interest in automatic machine learning research agents. Among them, agents capable of autonomously proposing ideas and conducting machine learning experiments are particularly promising, as they maximize research automation and accelerate scientific progress by iteratively refining ideas based on experimental results. However, comprehensively evaluating such agents remains challenging. Existing benchmarks tend to overemphasize engineering aspects while neglecting academic rigor, creating barriers that obscure a clear assessment of an agent’s scientific capabilities in machine learning research. They also suffer from limited task diversity, an overemphasis on application-oriented tasks over fundamental research problems, and limited scalability to realistic research settings. To address these limitations, we introduce FML-bench, a benchmark designed to evaluate automatic machine learning research agents on 8 diverse and fundamental machine learning research problems. It reduces coding burden, emphasizes fundamental problems rather than specific use cases, offers high task diversity, and is extensible to real-world machine learning GitHub repositories. Furthermore, we present a unified evaluation framework with five complementary metrics, designed to comprehensively assess agent performance on our benchmark. We evaluate state-of-the-art automatic research agents on FML-bench, and find that agents employing broad research exploration strategies outperform those focusing on narrow but deep exploration. These findings suggest that emphasizing the breadth of exploration may lead to more effective research outcomes than focusing solely on incremental refinement. Our benchmark is available at https://github.com/qrzou/FML-bench.

Method: Applied Latent Dirichlet Allocation (LDA) modeling to a large corpus of TED Talks, comparing thematic supply from creators with audience engagement demand, and conducting longitudinal analysis of temporal dynamics.

Result: Identified persistent mismatches between producer offerings and consumer preferences, with temporal dynamics exerting stronger influence on engagement than thematic content - when content is delivered matters more than what is delivered.

Conclusion: Challenges content-centric assumptions about popularity, highlighting the importance of timing and contextual factors in shaping consumer responses, with practical implications for marketers, platform managers, and content creators.

Abstract: Understanding what drives popularity is critical in today’s digital service economy, where content creators compete for consumer attention. Prior studies have primarily emphasized the role of content features, yet creators often misjudge what audiences actually value. This study applies Latent Dirichlet Allocation (LDA) modeling to a large corpus of TED Talks, treating the platform as a case of digital service provision in which creators (speakers) and consumers (audiences) interact. By comparing the thematic supply of creators with the demand expressed in audience engagement, we identify persistent mismatches between producer offerings and consumer preferences. Our longitudinal analysis further reveals that temporal dynamics exert a stronger influence on consumer engagement than thematic content, suggesting that when content is delivered may matter more than what is delivered. These findings challenge the dominant assumption that content features are the primary drivers of popularity and highlight the importance of timing and contextual factors in shaping consumer responses. The results provide new insights into consumer attention dynamics on digital platforms and carry practical implications for marketers, platform managers, and content creators seeking to optimize audience engagement strategies.

Method: Used a general-purpose instruction-tuned LLaMA-4 17B model with single in-context example (few-shot learning) without domain-specific fine-tuning, focusing on effective prompt engineering.

Result: Ranked 5th among 17 teams with 105 submissions, achieving average F1 score of 37.76 with notable improvements in reason and provenance accuracy.

Conclusion: Large non-domain-specific LLMs with effective prompt engineering can serve as strong, scalable baselines for specialized clinical NLP tasks, demonstrating competitive performance without domain adaptation.

Abstract: Medical order extraction is essential for structuring actionable clinical information, supporting decision-making, and enabling downstream applications such as documentation and workflow automation. Orders may be embedded in diverse sources, including electronic health records, discharge summaries, and multi-turn doctor-patient dialogues, and can span categories such as medications, laboratory tests, imaging studies, and follow-up actions. The MEDIQA-OE 2025 shared task focuses on extracting structured medical orders from extended conversational transcripts, requiring the identification of order type, description, reason, and provenance. We present the MasonNLP submission, which ranked 5th among 17 participating teams with 105 total submissions. Our approach uses a general-purpose, instruction-tuned LLaMA-4 17B model without domain-specific fine-tuning, guided by a single in-context example. This few-shot configuration achieved an average F1 score of 37.76, with notable improvements in reason and provenance accuracy. These results demonstrate that large, non-domain-specific LLMs, when paired with effective prompt engineering, can serve as strong, scalable baselines for specialized clinical NLP tasks.

Method: Simple modification to standard Rotary Positional Embeddings (RoPE) extension to accommodate diffusion probabilistic modeling, plus comparison of masking strategies during post-training.

Result: UltraLLaDA significantly outperforms training-free baselines on long-context tasks, achieving stable scaling to 128K-token context window.

Conclusion: Special positional extension is key for scaling diffusion LLMs to extended contexts, providing practical guidance for efficient 128K-scale context via post-training.

Abstract: Diffusion LLMs have attracted growing interest, with plenty of recent work emphasizing their great potential in various downstream tasks; yet the long-context behavior of diffusion LLMs remains largely uncharted. We present a case study of post-training techniques for extending the context window of diffusion LLMs (i.e., LLaDA) without retraining from scratch. We show that a simple modification to the standard Rotary Positional Embeddings (RoPE) extension effectively accommodates the probabilistic modeling inherent in the diffusion process, enabling stable scaling to longer context ranges. We further compare masking strategies used during post-training and analyze their impact on optimization stability and long-range recall. Instantiating these insights, we introduce UltraLLaDA, a diffusion LLM with a 128K-token context window that, in our empirical evaluation on long-context tasks, significantly outperforms training-free baselines. Our experimental results highlight the special positional extension as a key lever for scaling diffusion LLMs to extended contexts and offer practical guidance for practitioners seeking 128K-scale context via efficient post-training.

Method: Contrastive learning-based OCR with auto-glyph feature recommendation for cluster-based labeling, augmented with image transformations and Generative Adversarial Networks for data diversity and volume.

Result: Achieved 95% accuracy for machine printed and 87% for handwritten samples on Takri script. Demonstrated universal behavior across Indic scripts (both low and high resource).

Conclusion: VOLTAGE successfully addresses OCR challenges for low-resource languages, enabling digital inclusion and preservation of endangered scripts through unsupervised methodology with high accuracy.

Abstract: UNESCO has classified 2500 out of 7000 languages spoken worldwide as endangered. Attrition of a language leads to loss of traditional wisdom, folk literature, and the essence of the community that uses it. It is therefore imperative to bring digital inclusion to these languages and avoid its extinction. Low resource languages are at a greater risk of extinction. Lack of unsupervised Optical Character Recognition(OCR) methodologies for low resource languages is one of the reasons impeding their digital inclusion. We propose VOLTAGE - a contrastive learning based OCR methodology, leveraging auto-glyph feature recommendation for cluster-based labelling. We augment the labelled data for diversity and volume using image transformations and Generative Adversarial Networks. Voltage has been designed using Takri - a family of scripts used in 16th to 20th century in the Himalayan regions of India. We present results for Takri along with other Indic scripts (both low and high resource) to substantiate the universal behavior of the methodology. An accuracy of 95% for machine printed and 87% for handwritten samples on Takri script has been achieved. We conduct baseline and ablation studies along with building downstream use cases for Takri, demonstrating the usefulness of our work.

Method: AdvPrompt uses black-box adversarial prompting with iterative refinement to generate high-quality prompts that elicit concise responses from both open-source and closed-source models.

Result: Achieves 3x reduction in response length on GSM8K for Qwen3, ~40% average token reduction across four benchmarks, and 35-47% reduction for closed-source APIs on MATH-500.

Conclusion: Black-box adversarial prompting is a practical and effective strategy for enhancing LRM efficiency across various model scales and families.

Abstract: Large reasoning models (LRMs) have demonstrated remarkable proficiency in tackling complex reasoning tasks through step-by-step thinking. However, such a lengthy reasoning process incurs substantial computational and latency overheads, hindering the practical deployment of these models. In this work, we present a new perspective on mitigating overthinking in LRMs via black-box adversarial prompting. By treating both open-source LRMs and closed-source APIs as black-box communicators, we investigate how to elicit concise responses without sacrificing accuracy. We introduce AdvPrompt, an iterative refinement framework that generates high-quality adversarial prompts from diverse perspectives. Experiments across multiple benchmarks demonstrate that AdvPrompt consistently reduces token usage while preserving performance. Notably, AdvPrompt achieves a 3x reduction in average response length on simple GSM8K questions for the Qwen3 model series, and delivers an average ~40% token reduction across four benchmarks. For closed-source APIs, AdvPrompt reduces token usage on MATH-500 by 35% for Claude-3.7 and 47% for Gemini-2.5. Further analysis reveals the generalizability of AdvPrompt across various model scales and families, underscoring the potential of black-box prompting as a practical and effective strategy for enhancing LRM efficiency.

Method: Created benchmark by selecting and annotating samples from genuine LLM-human dialogues, providing faithful reflection of real-world hallucinations.

Result: Hallucinations occur in 31.4% of query-response pairs overall, increasing to 60.0% in Math & Number Problems. LLMs show promise but insufficient performance as hallucination detectors.

Conclusion: Authentic benchmarks are crucial for understanding real-world LLM hallucinations, and current LLM-based detection methods need improvement for practical deployment.

Abstract: As large language models (LLMs) are increasingly applied in sensitive domains such as medicine and law, hallucination detection has become a critical task. Although numerous benchmarks have been proposed to advance research in this area, most of them are artificially constructed–either through deliberate hallucination induction or simulated interactions–rather than derived from genuine LLM-human dialogues. Consequently, these benchmarks fail to fully capture the characteristics of hallucinations that occur in real-world usage. To address this limitation, we introduce AuthenHallu, the first hallucination detection benchmark built entirely from authentic LLM-human interactions. For AuthenHallu, we select and annotate samples from genuine LLM-human dialogues, thereby providing a faithful reflection of how LLMs hallucinate in everyday user interactions. Statistical analysis shows that hallucinations occur in 31.4% of the query-response pairs in our benchmark, and this proportion increases dramatically to 60.0% in challenging domains such as Math & Number Problems. Furthermore, we explore the potential of using vanilla LLMs themselves as hallucination detectors and find that, despite some promise, their current performance remains insufficient in real-world scenarios.

Method: Uses 1.58-bit encoders (BitNet-style for text, DiNOv2-based for vision) to create compact embeddings, combines them to query a fixed-size key-value episodic memory, and employs BitNet decoder with per-layer conditioning and sliding-window attention for long/streaming inputs.

Result: Achieves competitive captioning and multimodal understanding performance with low latency and small model footprint, demonstrating strong quality-speed trade-off suitable for edge deployment.

Conclusion: BitMar is well-suited for edge deployment due to its efficient quantization, episodic memory architecture, and attention mechanisms that enable effective multimodal generation under tight memory constraints.

Abstract: Cross-attention transformers and other multimodal vision-language models excel at grounding and generation; however, their extensive, full-precision backbones make it challenging to deploy them on edge devices. Memory-augmented architectures enhance the utilization of past context; however, most works rarely pair them with aggressive edge-oriented quantization. We introduce BitMar, a quantized multimodal transformer that proposes an external human-like episodic memory for effective image-text generation on hardware with limited resources. BitMar utilizes 1.58-bit encoders, one for text (BitNet-style) and one for vision (DiNOv2-based), to create compact embeddings that are combined and used to query a fixed-size key-value episodic memory. During vector retrieval, the BitNet decoder applies per-layer conditioning, which increases the contextual relevance of generated content. The decoder also employs attention sinks with a sliding-window mechanism to process long or streaming inputs under tight memory budgets. The combination of per-layer conditioning and sliding-window attention achieves a strong quality-speed trade-off, delivering competitive captioning and multimodal understanding at low latency with a small model footprint. These characteristics make BitMar well-suited for edge deployment.

Method: Identifies key entities as initial nodes, generates reasoning child nodes refined through historical error analysis and real-time feedback, integrates depth-first search with innovative node generation, and dynamically adjusts strategies based on error paths and concurrent nodes.

Result: Achieved 54.47% F1 score on HotpotQA and 70.05% on StrategyQA, surpassing best baselines by 5.03% and 7.25% respectively.

Conclusion: FGDIP demonstrates versatility and potential to enhance language agents in multi-hop reasoning tasks through dynamic and adaptive reasoning strategies.

Abstract: Recent advancements in language agents have led to significant improvements in multi-hop reasoning tasks. However, existing approaches often struggle with handling open-domain problems, which require massive information retrieval due to their reliance on a fixed sequence of actions. To address this, we propose Feedback-Guided Dynamic Interactive Planning (FGDIP), a novel framework tailored to enhance reasoning in LLMs by utilizing dynamic and adaptive strategies for information exploration in open-domain multi-hop reasoning tasks. Our approach begins by identifying key entities relevant to the problem, which serve as the initial nodes in the reasoning process. From these initial nodes, we then generate reasoning child nodes with the process being refined through a combination of historical error analysis and real-time feedback, which allows the framework to dynamically adjust and optimize its reasoning strategies. By integrating depth-first search with an innovative node generation technique, our framework adapts based on both prior error paths and concurrently generated nodes at the same hierarchical level. This dynamic strategy effectively expands the search space while ensuring the reasoning process systematically converges toward accurate solutions. Experimental results show that FGDIP achieved up to 54.47% F1 score on the HotpotQA dataset and 70.05% on the StrategyQA dataset, surpassing the best baseline by 5.03% and 7.25% respectively, highlighting its versatility and potential to enhance language agents in multi-hop reasoning tasks.

Method: Uses LLM contextual embeddings with temporal decay functions and attention mechanisms to adjust semantic importance by time intervals, then maps to latent topic space with state transition matrices for dynamic evolution modeling.

Result: Outperforms existing models across multiple metrics, effectively captures topic generation, expansion, and decline, and improves topic coherence, diversity, stability, and interpretability.

Conclusion: Provides a systematic solution for dynamic topic modeling that enriches topic modeling research and supports complex text analysis across multiple domains.

Abstract: This paper proposes a modeling framework for dynamic topic evolution based on temporal large language models. The method first uses a large language model to obtain contextual embeddings of text and then introduces a temporal decay function and an attention mechanism. These components allow the model to adjust the importance of semantic units according to time intervals and capture topic variations across different periods. The temporal representations are then mapped into a latent topic space, where a state transition matrix is applied to describe the dynamic evolution of topics. A joint optimization objective constrains both semantic modeling and temporal consistency, ensuring diversity and smoothness in topic generation. The design emphasizes the unified modeling of semantic representation and temporal evolution, which improves topic coherence and diversity while enhancing stability and interpretability over time. Experiments on real-world corpora show that the framework effectively captures the generation, expansion, and decline of topics and outperforms existing models across multiple metrics. Overall, the proposed method provides a systematic solution for understanding dynamic semantic patterns in large-scale text, enriches the research paradigm of topic modeling, and supports complex text analysis tasks in multiple domains.

Method: Analyzed calibration data impacts from activation pattern perspective, explored underlying mechanisms, and proposed a calibration data curation framework based on activation space representativeness and diversity.

Result: Found that representativeness and diversity in activation space fundamentally determine calibration data quality, and the proposed framework enhances performance of existing compression methods in preserving critical LLM capabilities.

Conclusion: Calibration data’s impact on compressed LLM capabilities is determined by activation space properties, and systematic data curation can significantly improve compression method performance on preserving high-level reasoning abilities.

Abstract: Post-training compression has been a widely employed approach to scale down large language model (LLM) and facilitate efficient inference. In various proposed compression methods, including pruning and quantization, calibration data plays a vital role by informing the weight importance and activation dynamic ranges. However, how calibration data impacts the LLM capability after compression is less explored. Few of the existing works, though recognizing the significance of this study, only investigate the language modeling or commonsense reasoning performance degradation from limited angles, like the data sources or sample amounts. More systematic research is still needed to examine the impacts on different LLM capabilities in terms of compositional properties and domain correspondence of calibration data. In this work, we aim at bridging this gap and further analyze underlying influencing mechanisms from the activation pattern perspective. Especially, we explore the calibration data’s impacts on high-level complex reasoning capabilities, like math problem solving and code generation. Delving into the underlying mechanism, we find that the representativeness and diversity in activation space more fundamentally determine the quality of calibration data. Finally, we propose a calibration data curation framework based on such observations and analysis, enhancing the performance of existing post-training compression methods on preserving critical LLM capabilities. Our code is provided in \href{https://github.com/BokwaiHo/COLA.git}{Link}.

Method: Created FactAppeal dataset with 3,226 manually annotated English news sentences containing span-level annotations for factual statements and source mentions, including fine-grained characteristics like source types, naming, roles, credentials, and attribution methods.

Result: The best performing model (Gemma 2 9B) achieved a macro-F1 score of 0.73 for the Epistemic Appeal Identification task.

Conclusion: The paper successfully establishes a new task for analyzing epistemic structures in factual claims and provides a comprehensive dataset that enables fine-grained analysis of how evidence and sources are used to support claims.

Abstract: How is a factual claim made credible? We propose the novel task of Epistemic Appeal Identification, which identifies whether and how factual statements have been anchored by external sources or evidence. To advance research on this task, we present FactAppeal, a manually annotated dataset of 3,226 English-language news sentences. Unlike prior resources that focus solely on claim detection and verification, FactAppeal identifies the nuanced epistemic structures and evidentiary basis underlying these claims and used to support them. FactAppeal contains span-level annotations which identify factual statements and mentions of sources on which they rely. Moreover, the annotations include fine-grained characteristics of factual appeals such as the type of source (e.g. Active Participant, Witness, Expert, Direct Evidence), whether it is mentioned by name, mentions of the source’s role and epistemic credentials, attribution to the source via direct or indirect quotation, and other features. We model the task with a range of encoder models and generative decoder models in the 2B-9B parameter range. Our best performing model, based on Gemma 2 9B, achieves a macro-F1 score of 0.73.

Method: Used article matching strategy to compare reports on same events and applied FactAppeal framework to a corpus of over 470K articles covering COVID-19 pandemic and Israel-Hamas war.

Result: CNN’s reporting contains more factual statements and is more likely to ground them in external sources. CNN builds credibility by citing Experts and Expert Documents (appeal to formal authority), while Fox News favors News Reports and direct quotations.

Conclusion: This work quantifies how partisan outlets use systematically different epistemic strategies to construct reality, adding a new dimension to the study of media bias.

Abstract: While media bias is widely studied, the epistemic strategies behind factual reporting remain computationally underexplored. This paper analyzes these strategies through a large-scale comparison of CNN and Fox News. To isolate reporting style from topic selection, we employ an article matching strategy to compare reports on the same events and apply the FactAppeal framework to a corpus of over 470K articles covering two highly politicized periods: the COVID-19 pandemic and the Israel-Hamas war. We find that CNN’s reporting contains more factual statements and is more likely to ground them in external sources. The outlets also exhibit sharply divergent sourcing patterns: CNN builds credibility by citing Experts} and Expert Documents, constructing an appeal to formal authority, whereas Fox News favors News Reports and direct quotations. This work quantifies how partisan outlets use systematically different epistemic strategies to construct reality, adding a new dimension to the study of media bias.

Method: Multi-expert framework with reasoning agent for question decomposition, coding agent for sub-query generation, refinement step for column selection, and adaptive router to choose between modular pipeline and baseline parser. Supports parallel execution for scalability.

Result: Achieves 86.07% EX on Spider benchmark using 14B models with Planner&Executor merging strategy, narrowing gap to GPT-4-based SOTA (89.65% EX) while using smaller open-source LLMs.

Conclusion: AGENTIQL provides a robust, scalable, and interpretable approach to semantic parsing that enhances both accuracy and transparency through intermediate reasoning steps.

Abstract: LLMs have advanced text-to-SQL generation, yet monolithic architectures struggle with complex reasoning and schema diversity. We propose AGENTIQL, an agent-inspired multi-expert framework that combines a reasoning agent for question decomposition, a coding agent for sub-query generation, and a refinement step for column selection. An adaptive router further balances efficiency and accuracy by selecting between our modular pipeline and a baseline parser. Several steps in the pipeline can be executed in parallel, making the framework scalable to larger workloads. Evaluated on the Spider benchmark, AGENTIQL improves execution accuracy and interpretability and achieves up to 86.07% EX with 14B models using the Planner&Executor merging strategy. The attained performance is contingent upon the efficacy of the routing mechanism, thereby narrowing the gap to GPT-4-based SOTA (89.65% EX) while using much smaller open-source LLMs. Beyond accuracy, AGENTIQL enhances transparency by exposing intermediate reasoning steps, offering a robust, scalable, and interpretable approach to semantic parsing.

Method: Two-stage training (SFT and RFT) with predefined browser actions operating directly on raw web pages via Playwright, plus an explicit memory mechanism for storing key conclusions across steps.

Result: BrowserAgent-7B achieves around 20% improvement over Search-R1 on multi-hop QA tasks like HotpotQA, 2Wiki, and Bamboogle, despite using significantly less training data.

Conclusion: BrowserAgent serves as a more advanced framework for interactive and scalable web agents by mimicking human browsing behaviors more closely.

Abstract: Efficiently solving real-world problems with LLMs increasingly hinges on their ability to interact with dynamic web environments and autonomously acquire external information. While recent research like Search-R1 and WebDancer demonstrates strong performance in solving web tasks, they heavily rely on additional tools to convert the interactive web environment into static text content. This is in contrast to human browsing behaviors, which involve diverse interactions with the browser, such as scrolling, clicking, and typing. In this paper, we propose BrowserAgent, a more interactive agent that solves complex tasks through human-inspired browser actions. BrowserAgent operates directly on raw web pages via Playwright through a set of predefined browser actions. We adopt a two-stage training (Supervised Fine-Tuning (SFT) and Rejection Fine-Tuning (RFT)) to improve the model’s generalization abilities. Despite using significantly less training data than Search-R1, BrowserAgent achieves more competitive results across different Open-QA tasks. Additionally, we introduce an explicit memory mechanism to store key conclusions across steps, further enhancing the model’s reasoning capabilities for long-horizon tasks. Notably, BrowserAgent-7B can achieve around 20% improvement over Search-R1 on multi-hop QA tasks like HotpotQA, 2Wiki, and Bamboogle. These results indicate that BrowserAgent can serve as a more advanced framework for more interactive and scalable web agents.

Method: Proposed ConsistentGuard uses reasoning-based approach with alignment to enhance explainability and boost knowledge transfer between languages, requiring only 1,000 training samples.

Result: Outperforms larger models trained with more data on three datasets across six languages, demonstrating strong interpretability and generalization ability.

Conclusion: The method provides an effective multilingual safeguard solution with minimal data requirements and contributes a benchmark extension and code release for future research.

Abstract: Recent advances in LLMs have enhanced AI capabilities, but also increased the risk posed by malicious requests, highlighting the need for effective LLM safeguards to detect such queries. Existing approaches largely rely on classifier-based methods that lack interpretability and perform poorly on low-resource languages. To address these limitations, we propose ConsistentGuard, a novel reasoning-based multilingual safeguard, which enhances explainability via reasoning and boosts knowledge transfer between languages through alignment. With only 1,000 training samples, our method demonstrates superior performance on three datasets across six languages, outperforming larger models trained with significantly more data, and exhibits strong interpretability and generalization ability. We also contribute a multilingual benchmark extension and release our codes to support future research.

Method: Trains a 4B LM with reinforcement learning using one quality reward and three faithfulness rewards to generate effective rephrasings of pretraining data while maintaining core semantics and structure.

Result: Achieves 4.7%-14.0% relative accuracy gains on 22 downstream tasks, outperforms state-of-the-art prompting-based methods, and improves organic data efficiency by 2-3x.

Conclusion: RePro provides an efficient and controllable path to effectively harness pretraining data, preserving critical information and faithfully reflecting organic data characteristics better than prompting-based approaches.

Abstract: High-quality pretraining data is the fossil fuel of large language models (LLMs), yet its reserves are running low for frontier models. In this paper, we introduce RePro, a novel web recycling method that trains a relatively small LM with reinforcement learning to generate effective and faithful rephrasings of pretraining data. Specifically, we design one quality reward and three faithfulness rewards, optimizing the LM rephraser to convert organic data into high-quality rephrasings while maintaining its core semantics and structure. In our experiment, we train a 4B rephraser to recycle 72B tokens sampled from DCLM-RefinedWeb. Pretraining results on 400M and 1.4B models demonstrate that RePro delivers 4.7%-14.0% relative accuracy gains over organic-only baseline on 22 downstream tasks. RePro also outperforms ReWire, the state-of-the-art web recycling method that prompts a 70B rephraser, as well as the organic baseline with a 4x larger data pool. Experiments with different amounts of recycled data highlight that RePro improves organic data efficiency by 2-3x. Individual and distributional analyses validate that RePro preserves more critical information and faithfully reflects the characteristics of organic data compared to prompting-based methods. Together, these results show that RePro provides an efficient and controllable path to effectively harness the fossil fuel of LLM pretraining. We open-source our code, rephraser, and recycled data at https://github.com/cxcscmu/RePro.

Method: Modular framework integrating sentiment analysis, contextual embeddings, linguistic feature extraction, and emotion detection through multi-layer architecture using DCNNs and BERT.

Result: The model is in conceptual stage but demonstrates feasibility for real-world applications like chatbots and social media analysis.

Conclusion: The proposed modular deep learning approach shows promise for effective sarcasm detection in text-based communication systems.

Abstract: Sarcasm is a nuanced and often misinterpreted form of communication, especially in text, where tone and body language are absent. This paper proposes a modular deep learning framework for sarcasm detection, leveraging Deep Convolutional Neural Networks (DCNNs) and contextual models such as BERT to analyze linguistic, emotional, and contextual cues. The system integrates sentiment analysis, contextual embeddings, linguistic feature extraction, and emotion detection through a multi-layer architecture. While the model is in the conceptual stage, it demonstrates feasibility for real-world applications such as chatbots and social media analysis.

Method: Benchmarked state-of-the-art LLMs against expert human reviewers across 180 full-text scientific articles using causal mediation analysis as a representative methodological domain.

Result: Model performance closely correlated with human judgments (accuracy correlation 0.71; F1 correlation 0.97), achieving near-human accuracy on straightforward criteria but declining sharply on complex assessments (lagging experts by up to 15%). Errors resulted from superficial linguistic cues and longer documents yielded lower accuracy.

Conclusion: Current LLMs excel at identifying explicit methodological features but require human oversight for nuanced interpretations. Integrating automated extraction with expert review provides a promising approach to enhance efficiency and rigor in evidence synthesis.

Abstract: Systematic reviews are crucial for synthesizing scientific evidence but remain labor-intensive, especially when extracting detailed methodological information. Large language models (LLMs) offer potential for automating methodological assessments, promising to transform evidence synthesis. Here, using causal mediation analysis as a representative methodological domain, we benchmarked state-of-the-art LLMs against expert human reviewers across 180 full-text scientific articles. Model performance closely correlated with human judgments (accuracy correlation 0.71; F1 correlation 0.97), achieving near-human accuracy on straightforward, explicitly stated methodological criteria. However, accuracy sharply declined on complex, inference-intensive assessments, lagging expert reviewers by up to 15%. Errors commonly resulted from superficial linguistic cues – for instance, models frequently misinterpreted keywords like “longitudinal” or “sensitivity” as automatic evidence of rigorous methodological approache, leading to systematic misclassifications. Longer documents yielded lower model accuracy, whereas publication year showed no significant effect. Our findings highlight an important pattern for practitioners using LLMs for methods review and synthesis from full texts: current LLMs excel at identifying explicit methodological features but require human oversight for nuanced interpretations. Integrating automated information extraction with targeted expert review thus provides a promising approach to enhance efficiency and methodological rigor in evidence synthesis across diverse scientific fields.

Method: Collected 8,000+ annotated sentences from news, social media, and literary texts, then fine-tuned mBERT and XLM-RoBERTa models on this corpus for NER.

Result: Both models achieved over 80% precision, recall, and F1-score across entity types, with promising cross-lingual transferability to Cebuano and Tagalog.

Conclusion: HiligayNER successfully establishes the first baseline for Hiligaynon NER, contributing to language technology development for underrepresented Philippine languages and supporting future multilingual NLP research.

Abstract: The language of Hiligaynon, spoken predominantly by the people of Panay Island, Negros Occidental, and Soccsksargen in the Philippines, remains underrepresented in language processing research due to the absence of annotated corpora and baseline models. This study introduces HiligayNER, the first publicly available baseline model for the task of Named Entity Recognition (NER) in Hiligaynon. The dataset used to build HiligayNER contains over 8,000 annotated sentences collected from publicly available news articles, social media posts, and literary texts. Two Transformer-based models, mBERT and XLM-RoBERTa, were fine-tuned on this collected corpus to build versions of HiligayNER. Evaluation results show strong performance, with both models achieving over 80% in precision, recall, and F1-score across entity types. Furthermore, cross-lingual evaluation with Cebuano and Tagalog demonstrates promising transferability, suggesting the broader applicability of HiligayNER for multilingual NLP in low-resource settings. This work aims to contribute to language technology development for underrepresented Philippine languages, specifically for Hiligaynon, and support future research in regional language processing.

Method: Systematic survey and organization of inference-time scaling techniques into two comprehensive perspectives: Output-focused methods (complex multi-step generation strategies) and Input-focused methods (few-shot and RAG approaches).

Result: A structured framework for understanding the rapidly evolving field of inference-time scaling, with detailed categorization of techniques including reasoning methods, search/decoding strategies, and comprehensive RAG analysis.

Conclusion: Inference-time scaling represents a new paradigm for improving LLM performance that addresses the data availability bottleneck by leveraging additional computation at deployment time rather than through expensive model re-training.

Abstract: The performance gains of LLMs have historically been driven by scaling up model size and training data. However, the rapidly diminishing availability of high-quality training data is introducing a fundamental bottleneck, shifting the focus of research toward inference-time scaling. This paradigm uses additional computation at the time of deployment to substantially improve LLM performance on downstream tasks without costly model re-training. This review systematically surveys the diverse techniques contributing to this new era of inference-time scaling, organizing the rapidly evolving field into two comprehensive perspectives: Output-focused and Input-focused methods. Output-focused techniques encompass complex, multi-step generation strategies, including reasoning (e.g., CoT, ToT, ReAct), various search and decoding methods (e.g., MCTS, beam search), training for long CoT (e.g., RLVR, GRPO), and model ensemble methods. Input-focused techniques are primarily categorized by few-shot and RAG, with RAG as the central focus. The RAG section is further detailed through a structured examination of query expansion, data, retrieval and reranker, LLM generation methods, and multi-modal RAG.

Method: Developed HCRS - a five-dimensional evaluation framework integrating automatic measures with structured human feedback, grounded in HCI and health communication research.

Result: Proposed framework and validation protocol for capturing relational and contextual aspects of readability in health text simplification.

Conclusion: HCRS advances health text simplification evaluation beyond surface metrics, enabling NLP systems to better align with diverse users’ needs, expectations, and lived experiences.

Abstract: Text simplification is essential for making public health information accessible to diverse populations, including those with limited health literacy. However, commonly used evaluation metrics in Natural Language Processing (NLP), such as BLEU, FKGL, and SARI, mainly capture surface-level features and fail to account for human-centered qualities like clarity, trustworthiness, tone, cultural relevance, and actionability. This limitation is particularly critical in high-stakes health contexts, where communication must be not only simple but also usable, respectful, and trustworthy. To address this gap, we propose the Human-Centered Readability Score (HCRS), a five-dimensional evaluation framework grounded in Human-Computer Interaction (HCI) and health communication research. HCRS integrates automatic measures with structured human feedback to capture the relational and contextual aspects of readability. We outline the framework, discuss its integration into participatory evaluation workflows, and present a protocol for empirical validation. This work aims to advance the evaluation of health text simplification beyond surface metrics, enabling NLP systems that align more closely with diverse users’ needs, expectations, and lived experiences.

Method: Proposed a bottom-up approach that linearizes tree structures by generating implicit, aggregated summaries at each hierarchical level, allowing knowledge to be stored in a knowledge base for RAG.

Result: Response quality was comparable to using RAG on raw unstructured code, but the proposed method generated over 68% fewer documents in the retriever, showing significant efficiency gains.

Conclusion: Leveraging implicit, linearized knowledge is an effective and scalable strategy for handling complex hierarchical data structures in RAG systems.

Abstract: Large Language Models (LLMs) are adept at generating responses based on information within their context. While this ability is useful for interacting with structured data like code files, another popular method, Retrieval-Augmented Generation (RAG), retrieves relevant documents to augment the model’s in-context learning. However, it is not well-explored how to best represent this retrieved knowledge for generating responses on structured data, particularly hierarchical structures like trees. In this work, we propose a novel bottom-up method to linearize knowledge from tree-like structures (like a GitHub repository) by generating implicit, aggregated summaries at each hierarchical level. This approach enables the knowledge to be stored in a knowledge base and used directly with RAG. We then compare our method to using RAG on raw, unstructured code, evaluating the accuracy and quality of the generated responses. Our results show that while response quality is comparable across both methods, our approach generates over 68% fewer documents in the retriever, a significant gain in efficiency. This finding suggests that leveraging implicit, linearized knowledge may be a highly effective and scalable strategy for handling complex, hierarchical data structures.

Method: Conducted controlled experiments using three types of transliteration (romanization, phonemic transcription, and substitution ciphers) plus orthography, evaluating on named entity recognition and natural language inference tasks.

Result: Romanization significantly outperformed other input types across most evaluation settings, showing it’s the most effective approach for multilingual NLP.

Conclusion: Longer subword tokens shared with pre-trained languages lead to better model utilization, making romanization the most successful transliteration method.

Abstract: Transliteration has emerged as a promising means to bridge the gap between various languages in multilingual NLP, showing promising results especially for languages using non-Latin scripts. We investigate the degree to which shared script, overlapping token vocabularies, and shared phonology contribute to performance of multilingual models. To this end, we conduct controlled experiments using three kinds of transliteration (romanization, phonemic transcription, and substitution ciphers) as well as orthography. We evaluate each model on two downstream tasks – named entity recognition (NER) and natural language inference (NLI) – and find that romanization significantly outperforms other input types in 7 out of 8 evaluation settings, largely consistent with our hypothesis that it is the most effective approach. We further analyze how each factor contributed to the success, and suggest that having longer (subword) tokens shared with pre-trained languages leads to better utilization of the model.

Method: Created DUAL-Bench benchmark evaluating 18 VLMs across 12 hazard categories, focusing on robustness under semantics-preserving visual perturbations to measure safe completion rates.

Result: Poor performance across models: GPT-5-Nano achieves 12.9% safe completion, GPT-5 models average 7.9%, and Qwen models only 3.9%, showing substantial room for improvement.

Conclusion: DUAL-Bench enables development of more nuanced alignment strategies to ensure VLMs remain both safe and useful in complex multimodal settings, addressing the balance between safety and usefulness.

Abstract: As vision-language models become increasingly capable, maintaining a balance between safety and usefulness remains a central challenge. Safety mechanisms, while essential, can backfire, causing over-refusal, where models decline benign requests out of excessive caution. Yet, no existing benchmark has systematically addressed over-refusal in the visual modality. This setting introduces unique challenges, such as dual-use cases where an instruction is harmless, but the accompanying image contains harmful content. Models frequently fail in such scenarios, either refusing too conservatively or completing tasks unsafely, which highlights the need for more fine-grained alignment. The ideal behavior is safe completion, i.e., fulfilling the benign parts of a request while explicitly warning about any potentially harmful elements. To address this, we present DUAL-Bench, the first multimodal benchmark focused on over-refusal and safe completion in VLMs. We evaluated 18 VLMs across 12 hazard categories, with focus on their robustness under semantics-preserving visual perturbations. The results reveal substantial room for improvement: GPT-5-Nano achieves 12.9% safe completion, GPT-5 models average 7.9%, and Qwen models only 3.9%. We hope that DUAL-Bench will foster the development of more nuanced alignment strategies that ensure models remain both safe and useful in complex multimodal settings.

Method: Evaluated six lightweight industry-oriented test-time scaling strategies and four LLMs (including two reasoning models) on BIRD Mini-Dev benchmark, measuring accuracy, inference latency, and token consumption.

Result: Divide-and-Conquer prompting and few-shot demonstrations consistently improved performance for both general-purpose and reasoning-focused LLMs. Additional workflow steps had mixed results, and base model selection was critical.

Conclusion: Reveals practical trade-offs between accuracy, efficiency, and complexity in Text2SQL system deployment, highlighting the importance of strategy selection and model choice for industrial applications.

Abstract: Large language models (LLMs) are increasingly powering Text-to-SQL (Text2SQL) systems, enabling non-expert users to query industrial databases using natural language. While test-time scaling strategies have shown promise in LLM-based solutions, their effectiveness in real-world applications, especially with the latest reasoning models, remains uncertain. In this work, we benchmark six lightweight, industry-oriented test-time scaling strategies and four LLMs, including two reasoning models, evaluating their performance on the BIRD Mini-Dev benchmark. Beyond standard accuracy metrics, we also report inference latency and token consumption, providing insights relevant for practical system deployment. Our findings reveal that Divide-and-Conquer prompting and few-shot demonstrations consistently enhance performance for both general-purpose and reasoning-focused LLMs. However, introducing additional workflow steps yields mixed results, and base model selection plays a critical role. This work sheds light on the practical trade-offs between accuracy, efficiency, and complexity when deploying Text2SQL systems.

Method: Uses a multi-agent architecture with independent MCP servers for different functions (skeleton initialization, digest construction, skeleton refinement) that can be aggregated hierarchically. A planner agent dynamically orchestrates workflow based on tool descriptions and execution history.

Result: Human evaluations show the system outperforms representative baselines in both content depth and length, demonstrating the effectiveness of MCP-based modular planning.

Conclusion: The hierarchical modular approach using MCP servers enables precise capture of research perspectives and generates comprehensive surveys, with the system showing superior performance compared to baseline methods.

Abstract: We introduce LLM x MapReduce-V3, a hierarchically modular agent system designed for long-form survey generation. Building on the prior work, LLM x MapReduce-V2, this version incorporates a multi-agent architecture where individual functional components, such as skeleton initialization, digest construction, and skeleton refinement, are implemented as independent model-context-protocol (MCP) servers. These atomic servers can be aggregated into higher-level servers, creating a hierarchically structured system. A high-level planner agent dynamically orchestrates the workflow by selecting appropriate modules based on their MCP tool descriptions and the execution history. This modular decomposition facilitates human-in-the-loop intervention, affording users greater control and customization over the research process. Through a multi-turn interaction, the system precisely captures the intended research perspectives to generate a comprehensive skeleton, which is then developed into an in-depth survey. Human evaluations demonstrate that our system surpasses representative baselines in both content depth and length, highlighting the strength of MCP-based modular planning.

Method: Proposed ADVICE framework - a fine-tuning approach that facilitates answer-grounded confidence estimation to address the model’s failure to condition confidence on its own answer.

Result: Extensive experiments show ADVICE substantially improves confidence calibration while preserving task performance. The framework strengthens answer-groundedness and leads to more balanced, well-calibrated confidence distributions.

Conclusion: The work identifies answer-independence as the origin of overconfidence and establishes ADVICE as an effective framework for more trustworthy confidence verbalization in LLMs.

Abstract: Recent progress in large language models (LLMs) has enabled them to express their confidence in natural language, enhancing transparency and reliability. However, their confidence often exhibits overconfidence, the cause of which remains poorly understood. In this work, we conduct a detailed analysis of the dynamics underlying verbalized confidence and identify answer-independence as a key factor, defined as the model’s failure to condition confidence on its own answer. To address this, we propose ADVICE (Answer-Dependent Verbalized Confidence Estimation), a fine-tuning framework that facilitates answer-grounded confidence estimation. Extensive experiments show that ADVICE substantially improves confidence calibration while preserving task performance. Further analyses confirm that ADVICE strengthens answer-groundedness, leading to more balanced and well-calibrated confidence distributions. Our findings shed light on the origin of overconfidence and establish a framework for more trustworthy confidence verbalization.

Method: Proposes GapDNER which treats context gaps as additional span types and converts span classification to token-pair grid tagging. Uses two interactive components: intra-span regularity extraction with biaffine mechanism and linear attention, and inter-span relation enhancement with criss-cross attention. Uses BFS algorithm for entity decoding.

Result: Achieves new state-of-the-art performance on three datasets for discontinuous NER and shows remarkable advantages in recognizing complex entity structures.

Conclusion: GapDNER effectively addresses discontinuous NER challenges by modeling context gaps and using comprehensive token-pair grid features, demonstrating superior performance in handling complex biomedical entity structures.

Abstract: In biomedical fields, one named entity may consist of a series of non-adjacent tokens and overlap with other entities. Previous methods recognize discontinuous entities by connecting entity fragments or internal tokens, which face challenges of error propagation and decoding ambiguity due to the wide variety of span or word combinations. To address these issues, we deeply explore discontinuous entity structures and propose an effective Gap-aware grid tagging model for Discontinuous Named Entity Recognition, named GapDNER. Our GapDNER innovatively applies representation learning on the context gaps between entity fragments to resolve decoding ambiguity and enhance discontinuous NER performance. Specifically, we treat the context gap as an additional type of span and convert span classification into a token-pair grid tagging task. Subsequently, we design two interactive components to comprehensively model token-pair grid features from both intra- and inter-span perspectives. The intra-span regularity extraction module employs the biaffine mechanism along with linear attention to capture the internal regularity of each span, while the inter-span relation enhancement module utilizes criss-cross attention to obtain semantic relations among different spans. At the inference stage of entity decoding, we assign a directed edge to each entity fragment and context gap, then use the BFS algorithm to search for all valid paths from the head to tail of grids with entity tags. Experimental results on three datasets demonstrate that our GapDNER achieves new state-of-the-art performance on discontinuous NER and exhibits remarkable advantages in recognizing complex entity structures.

Method: Introduced a formalism for evaluating game evaluations. Used a dataset of 100+ novel board games and 450+ human judgments to compare reasoning models, language models, and symbolic agents on assessing game payoff/fairness and funness.

Result: Reasoning models align better with human evaluations than non-reasoning language models. However, as models approach game-theoretic optimality, their fit to human data weakens. Funness assessments show more variability across models. Reasoning models have highly variable resource usage.

Conclusion: The study highlights the importance of developing resource-rational meta-reasoning in AI systems, as current reasoning models show unpredictable resource consumption when evaluating complex queries like game assessments.

Abstract: Reasoning is not just about solving problems – it is also about evaluating which problems are worth solving at all. Evaluations of artificial intelligence (AI) systems primarily focused on problem solving, historically by studying how models play games such as chess and Go. In this paper, we advocate for a new paradigm that assesses AI systems’ evaluation of games. First, we introduce a formalism for evaluating such evaluations. We then leverage a large-scale dataset of over $100$ novel board games and over 450 human judgments to compare evaluations produced by modern language and reasoning models against those of people and symbolic computational agents. We consider two kinds of evaluative queries: assessing the payoff (or fairness) and the funness of games. These queries span two dimensions relevant to the design of evaluations of AI evaluations: how complex a query is to compute and how difficult a query is to quantify. Our results show that reasoning models are generally more aligned to people in their evaluations of games than non-reasoning language models. However, we observe a non-monotonic relationship: as models get closer to game-theoretic optimal, their fit to human data weakens. We also observe more “jaggedness” across models for assessing funness, in line with the greater difficulty of quantifying this query. Across queries and games, reasoning models show highly variable and unpredictable resource usage when assessing queries, pointing to the importance of imbuing more resource-rational meta-reasoning in language and reasoning models.

Method: Implemented the BiLSTM-CNN-CRF model combining character-level CNN representations, word-level BiLSTM context modeling, and CRF for structured prediction. Tested on named entity recognition (NER) and part-of-speech (POS) tagging tasks.

Result: Successfully reproduced key results with 91.18% F1-score on CoNLL-2003 NER, demonstrating the model’s effectiveness across sequence labeling tasks.

Conclusion: The BiLSTM-CNN-CRF architecture is reproducible and effective for sequence labeling tasks. The study provides detailed analysis of architecture components and releases open-source PyTorch implementation to support further research.

Abstract: We present a reproducibility study of the state-of-the-art neural architecture for sequence labeling proposed by Ma and Hovy (2016)\cite{ma2016end}. The original BiLSTM-CNN-CRF model combines character-level representations via Convolutional Neural Networks (CNNs), word-level context modeling through Bi-directional Long Short-Term Memory networks (BiLSTMs), and structured prediction using Conditional Random Fields (CRFs). This end-to-end approach eliminates the need for hand-crafted features while achieving excellent performance on named entity recognition (NER) and part-of-speech (POS) tagging tasks. Our implementation successfully reproduces the key results, achieving 91.18% F1-score on CoNLL-2003 NER and demonstrating the model’s effectiveness across sequence labeling tasks. We provide a detailed analysis of the architecture components and release an open-source PyTorch implementation to facilitate further research.

Method: Developed a reproducible pipeline that preserves punctuation as sibling nodes prior to binarization, with derived artifacts including intermediate markers, reversibility signatures, and alignment indices.

Result: On Penn Treebank, punctuation-aware preprocessing improved head prediction accuracy from 73.66% (Collins rules) and 86.66% (MLP) to 91.85% with the same classifier, achieving competitive alignment with CCGbank derivations.

Conclusion: The punctuation-aware binarization approach significantly improves parsing performance and structural compatibility, with all code and resources released for replication and extension.

Abstract: This article presents a curated resource and evaluation suite for punctuation-aware treebank binarization. Standard binarization pipelines drop punctuation before head selection, which alters constituent shape and harms head-child identification. We release (1) a reproducible pipeline that preserves punctuation as sibling nodes prior to binarization, (2) derived artifacts and metadata (intermediate @X markers, reversibility signatures, alignment indices), and (3) an accompanying evaluation suite covering head-child prediction, round-trip reversibility, and structural compatibility with derivational resources (CCGbank). On the Penn Treebank, punctuation-aware preprocessing improves head prediction accuracy from 73.66% (Collins rules) and 86.66% (MLP) to 91.85% with the same classifier, and achieves competitive alignment against CCGbank derivations. All code, configuration files, and documentation are released to enable replication and extension to other corpora.

Method: The authors categorize Korean obfuscation approaches based on linguistic characteristics and define transformation rules from real-world examples. They construct three dataset versions (easy, normal, hard) representing different obfuscation difficulty levels.

Result: KOTOX is the first dataset that simultaneously supports deobfuscation and detoxification for Korean language, providing a resource to better understand and mitigate obfuscated toxic content in LLMs for low-resource languages.

Conclusion: The dataset facilitates improved detection and neutralization of toxic content in Korean, addressing the gap in resources for low-resource languages and helping LLMs handle obfuscated toxic expressions more effectively.

Abstract: Toxic content has become an increasingly critical social issue with the rapid expansion of online communication. While numerous studies explored methods for detecting and detoxifying such content, most have focused primarily on English, leaving low-resource language underrepresented. Consequently, Large Language Models~(LLMs) often struggle to identify and neutralize toxic expressions in these languages. This challenge becomes even more pronounced when user employ obfuscation techniques to evade detection systems. Therefore, we propose a \textbf{KOTOX: Korean Toxic Dataset} for deobfuscation and detoxicification to address this issue. We categorize various obfuscation approaches based on linguistic characteristics of Korean and define a set of transformation rules grounded in real-word examples. Using these rules, we construct three dataset versions (easy, normal, and hard) representing different levels of obfuscation difficulty. This is the first dataset that simultaneously supports deobfuscation and detoxification for the Korean language. We expect it to facilitate better understanding and mitigating of obfuscated toxic content in LLM for low-resource languages. Our code and data are available at https://github.com/leeyejin1231/KOTOX.

Method: Developed a fully automated pipeline to construct the JBA benchmark, systematically categorizing premises into 3 main types and 13 subtypes based on required abilities. Also proposed a recognition enhancement framework to strengthen MLLM robustness against false premises.

Result: Current MLLMs still struggle with false premise recognition. Models trained with the proposed enhancement framework achieved significant improvements in false premise recognition capabilities.

Conclusion: The JBA benchmark provides rigorous evaluation of false premise recognition, and the proposed enhancement framework effectively strengthens MLLM robustness against false premises, addressing a critical vulnerability in current multimodal models.

Abstract: Multimodal large language models (MLLMs) have witnessed astonishing advancements in recent years. Despite these successes, MLLMs remain vulnerable to flase premise problems. However, existing benchmarks targeting this issue are limited in scope: they often lack fine-grained categorization, exhibit insufficient coverage, and thus fail to provide a rigorous evaluation of the ability of models to recognize false premises. To bridge this gap, we introduce a fully automated pipeline for constructing a comprehensive benchmark of false premise questions. Our method systematically categorizes the premises into three main types and thirteen subtypes according to the abilities required to identify the premises, resulting in the JBA dataset.Results show current MLLMs still struggle with false premise recognition. Building upon this benchmark, we further propose a recognition enhancement framework tailored to strengthen the robustness of MLLMs to detect false premises. Extensive experiments demonstrate that models trained with our framework achieve significant improvements in false premise recognition.

Method: Uses multiple specialized modules focusing on different linguistic/contextual features, reinforcement learning to optimize module weights, and voting mechanism for final decision.

Result: Improves detection accuracy, addresses implicit hate speech, and provides interpretable insights into dataset characteristics.

Conclusion: RV-HATE achieves superior performance over conventional static methods by adapting to dataset-specific attributes.

Abstract: Hate speech remains prevalent in human society and continues to evolve in its forms and expressions. Modern advancements in internet and online anonymity accelerate its rapid spread and complicate its detection. However, hate speech datasets exhibit diverse characteristics primarily because they are constructed from different sources and platforms, each reflecting different linguistic styles and social contexts. Despite this diversity, prior studies on hate speech detection often rely on fixed methodologies without adapting to data-specific features. We introduce RV-HATE, a detection framework designed to account for the dataset-specific characteristics of each hate speech dataset. RV-HATE consists of multiple specialized modules, where each module focuses on distinct linguistic or contextual features of hate speech. The framework employs reinforcement learning to optimize weights that determine the contribution of each module for a given dataset. A voting mechanism then aggregates the module outputs to produce the final decision. RV-HATE offers two primary advantages: (1)~it improves detection accuracy by tailoring the detection process to dataset-specific attributes, and (2)~it also provides interpretable insights into the distinctive features of each dataset. Consequently, our approach effectively addresses implicit hate speech and achieves superior performance compared to conventional static methods. Our code is available at https://github.com/leeyejin1231/RV-HATE.

Method: Critical Token Fine-tuning (CFT) identifies functionally indispensable tokens via counterfactual perturbations and updates only these decisive reasoning steps while preserving non-critical tokens.

Result: CFT consistently outperforms standard SFT on 11 mathematical reasoning benchmarks across 5 models, achieving better performance while fine-tuning less than 12% of tokens. It also enables test-time scaling and provides stronger RL initialization.

Conclusion: CFT is a practical and general framework for efficient and robust LLM fine-tuning that enhances both generation quality and diversity through targeted token updates.

Abstract: Large language models (LLMs) primarily rely on supervised fine-tuning (SFT) as a key method to adapt pre-trained models to domain-specific tasks such as mathematical reasoning. However, standard SFT uniformly penalizes all tokens, neglecting that only a small subset of critical tokens determines reasoning correctness. This uniform supervision often causes reduced output diversity and limited generalization. We propose Critical Token Fine-tuning (CFT), a simple yet effective approach that updates only tokens identified as functionally indispensable via counterfactual perturbations. By focusing gradient signals on these decisive reasoning steps while preserving the diversity of non-critical tokens, CFT can enhance both generation and diversity. Extensive experiments on five models across three families (Qwen, OLMo, LLaMA) and eleven mathematical reasoning benchmarks show that CFT, despite fine-tuning on less than 12% of tokens, consistently outperforms standard SFT. Moreover, CFT enables test-time scaling through improved sampling diversity and provides a stronger initialization for reinforcement learning, sustaining performance gains in later training stages while maintaining higher entropy for better exploration. These results highlight CFT as a practical and general framework for efficient and robust LLM fine-tuning.

Method: Introduces DEEPRESEARCHGUARD with four-stage safeguards and open-domain evaluation of references and reports, along with DRSAFEBENCH benchmark for deep research safety. Evaluates performance across multiple metrics including defense success rate and over-refusal rate.

Result: DEEPRESEARCHGUARD achieves 18.16% average defense success rate improvement while reducing over-refusal rate by 6%. Input guard provides substantial early-stage protection, while plan and research guards enhance citation discipline and source credibility.

Conclusion: DEEPRESEARCHGUARD enables comprehensive open-domain evaluation and stage-aware defenses that effectively block harmful content propagation while systematically improving report quality without excessive over-refusal rates.

Abstract: Deep research frameworks have shown promising capabilities in synthesizing comprehensive reports from web sources. While deep research possesses significant potential to address complex issues through planning and research cycles, existing frameworks are deficient in sufficient evaluation procedures and stage-specific protections. They typically treat evaluation as exact match accuracy of question-answering, but overlook crucial aspects of report quality such as credibility, coherence, breadth, depth, and safety. This oversight may result in hazardous or malicious sources being integrated into the final report. To address these issues, we introduce DEEPRESEARCHGUARD, a comprehensive framework featuring four-stage safeguards with open-domain evaluation of references and reports. We assess performance across multiple metrics, e.g., defense success rate and over-refusal rate, and five key report dimensions. In the absence of a suitable safety benchmark, we introduce DRSAFEBENCH, a stage-wise benchmark for deep research safety. Our evaluation spans diverse state-of-the-art LLMs, including GPT-4o, Gemini-2.5-flash, DeepSeek-v3, and o4-mini. DEEPRESEARCHGUARD achieves an average defense success rate improvement of 18.16% while reducing over-refusal rate by 6%. The input guard provides the most substantial early-stage protection by filtering out obvious risks, while the plan and research guards enhance citation discipline and source credibility. Through extensive experiments, we show that DEEPRESEARCHGUARD enables comprehensive open-domain evaluation and stage-aware defenses that effectively block harmful content propagation, while systematically improving report quality without excessive over-refusal rates. The code can be found via https://github.com/Jasonya/DeepResearchGuard.

Method: Comprehensive audit of six LLMs across 2,820 hiring scenarios with diverse disability, gender, nationality, and caste profiles, using ABLEIST metrics (five ableism-specific and three intersectional harm metrics).

Result: Significant increases in ABLEIST harms towards disabled candidates, with state-of-the-art models failing to detect these harms. Intersectional harms (e.g., Tokenism) were sharply amplified for gender and caste-marginalized disabled candidates.

Conclusion: Current safety tools have critical blind spots, highlighting the need for intersectional safety evaluations of frontier models in high-stakes domains like hiring.

Abstract: Large language models (LLMs) are increasingly under scrutiny for perpetuating identity-based discrimination in high-stakes domains such as hiring, particularly against people with disabilities (PwD). However, existing research remains largely Western-centric, overlooking how intersecting forms of marginalization–such as gender and caste–shape experiences of PwD in the Global South. We conduct a comprehensive audit of six LLMs across 2,820 hiring scenarios spanning diverse disability, gender, nationality, and caste profiles. To capture subtle intersectional harms and biases, we introduce ABLEIST (Ableism, Inspiration, Superhumanization, and Tokenism), a set of five ableism-specific and three intersectional harm metrics grounded in disability studies literature. Our results reveal significant increases in ABLEIST harms towards disabled candidates–harms that many state-of-the-art models failed to detect. These harms were further amplified by sharp increases in intersectional harms (e.g., Tokenism) for gender and caste-marginalized disabled candidates, highlighting critical blind spots in current safety tools and the need for intersectional safety evaluations of frontier models in high-stakes domains like hiring.

Method: DND identifies critical tokens at transformer layer ends using a router, feeds them back for extra processing, employs router controlling loss for better token selection, and uses threshold control for selection stability. It integrates into pre-trained models during post-training.

Result: Boosts dense Qwen3-1.7B by 1.88% and MoE Qwen3-30B-A3B by 0.87% on diverse benchmarks with minimal parameter and computing increase.

Conclusion: DND effectively improves LLM performance through dynamic token selection and nested reprocessing, demonstrating practical efficiency gains for both dense and MoE models.

Abstract: We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs by selecting critical tokens to reprocess in a nested depth manner. Specifically, at the end of the given transformer layer, DND identifies more critical tokens with a router and feeds them back for an extra round of processing, effectively ``reviewing" difficult tokens while avoiding redundant computation for easier ones. The dynamic selection mechanism is tailored for precise control via two novel strategies: a router controlling loss to enhance token selection distinguishability, and a threshold control scheme to ensure selection stability. We demonstrate the effectiveness of DND by directly integrating it into pre-trained dense and MoE models during a post-training phase. On diverse benchmarks, this approach boosts the performances of the dense Qwen3-1.7B by 1.88% and the MoE Qwen3-30B-A3B by 0.87%, all with a minimal parameter and computing increase.

Method: Developed LogiNumSynth synthesizer that generates natural language problems with controllable reasoning world richness, logical reasoning depth, and numerical computation complexity.

Result: Experiments show persistent weaknesses in LLMs’ logical-numerical reasoning, and LogiNumSynth can effectively diagnose these issues and provide targeted supervision to improve reasoning skills.

Conclusion: LogiNumSynth serves as both a diagnostic tool for evaluating reasoning weaknesses and a source of targeted training data to advance integrated logical-numerical reasoning capabilities in language models.

Abstract: Joint logical-numerical reasoning remains a major challenge for language models, yet existing datasets rely on fixed rule sets and offer limited control over task complexity, constraining their generalizability for evaluation and training. We present LogiNumSynth, a flexible natural language problem synthesizer that synthesizes tasks requiring proficiency in joint logical reasoning (e.g., rule-based reasoning) and numerical reasoning (e.g., arithmetic computation). LogiNumSynth supports fine-grained control over reasoning world richness, logical reasoning depth, and the complexity of numerical computations, enabling flexible data synthesis across difficulty levels. We demonstrate three key contributions: (1) Synthesizer – synthesizing fully controllable joint reasoning tasks over natural language; (2) Evaluation & Process Analysis – evaluating both process accuracy and answer accuracy; (3) Targeted Training – using synthesized data to enhance LLMs’ reasoning performance. Experiments with multiple LLMs highlight persistent weaknesses in logical-numerical reasoning, showing that LogiNumSynth can serve as both a diagnostic tool and a source of targeted supervision for advancing integrated reasoning skills.

Method: Conducted two-stage survey to identify clinical needs, built DoctorFLAN dataset (92k Q&A across 22 tasks and 27 specialties), created evaluation benchmarks DoctorFLAN-test and DotaBench

Result: DoctorFLAN notably improves performance of open-source LLMs in medical contexts, facilitates alignment with physician workflows

Conclusion: Provides valuable resource and framework for advancing doctor-centered medical LLM development, complementing existing patient-oriented models

Abstract: The rise of large language models (LLMs) has transformed healthcare by offering clinical guidance, yet their direct deployment to patients poses safety risks due to limited domain expertise. To mitigate this, we propose repositioning LLMs as clinical assistants that collaborate with experienced physicians rather than interacting with patients directly. We conduct a two-stage inspiration-feedback survey to identify real-world needs in clinical workflows. Guided by this, we construct DoctorFLAN, a large-scale Chinese medical dataset comprising 92,000 Q&A instances across 22 clinical tasks and 27 specialties. To evaluate model performance in doctor-facing applications, we introduce DoctorFLAN-test (550 single-turn Q&A items) and DotaBench (74 multi-turn conversations). Experimental results with over ten popular LLMs demonstrate that DoctorFLAN notably improves the performance of open-source LLMs in medical contexts, facilitating their alignment with physician workflows and complementing existing patient-oriented models. This work contributes a valuable resource and framework for advancing doctor-centered medical LLM development

Method: Two-stage framework: 1) Latent Refinement maintains masked positions as distributional mixtures of predicted tokens and mask embeddings for global consistency, 2) Predictive Feedback Loop progressively finalizes confident tokens while retaining uncertain ones for iterative feedback, using KL-divergence dynamics for convergence.

Result: Significant improvements: HumanEval +6.3, MBPP +2.6 (coding), GSM8K +2.9, MATH500 +3.8 (reasoning) with speedups up to 10.6x compared to baseline models.

Conclusion: LRD provides a strong and versatile alternative for parallel sequence generation, achieving both improved accuracy and substantial speedups across multiple domains.

Abstract: Autoregressive (AR) models remain the standard for natural language generation but still suffer from high latency due to strictly sequential decoding. Recent diffusion-inspired approaches, such as LlaDA and Dream, mitigate this by generating in parallel, yet they suffer from two core limitations: information loss, as predictive distributions for non-finalized tokens are discarded at each step, and premature commitment, where local decisions are made without sufficient global coordination. We introduce Latent Refinement Decoding (LRD), a two-stage framework with Latent Refinement and a Predictive Feedback Loop. The first stage maintains masked positions as distributional mixtures of predicted tokens and the mask embedding, allowing the model to establish more globally consistent beliefs. The second stage progressively finalizes confident tokens while retaining uncertain ones for iterative feedback. KL-divergence dynamics provide a principled and reliable criterion for convergence and early stopping. Experiments across coding (HumanEval +6.3, MBPP +2.6) and reasoning (GSM8K +2.9, MATH500 +3.8) show that LRD improves accuracy while delivering speedups of up to 10.6x, making it a strong and versatile alternative for parallel sequence generation.

Method: CGPO leverages confidence signals to identify points of maximal uncertainty in reasoning and applies self-generated, non-human-like reasoning-path guidance to mitigate trajectory drift.

Result: Experiments on code and mathematical reasoning tasks show CGPO with small model-generated data achieves better performance than approaches using strong model or human-annotated data with the same training data amount.

Conclusion: Confidence-guided reasoning path optimization effectively improves LLM reasoning by targeting uncertainty points rather than explicit errors, enabling better performance with less dependence on human or high-capacity model annotations.

Abstract: Current approaches for strengthening LLM reasoning tend to introduce a training bias toward human-like reasoning trajectories. In step-wise preference optimization, in particular, dependence on human or higher-capacity model annotations for intermediate steps limits exploration of alternative, non-human-like reasoning paths and thus constrains achievable performance. Furthermore, through a small-scale pilot study, we observed that in approximately 75% of cases, the model’s first erroneous step occurs after the lowest-confidence point. This suggests that guiding the model at its lowest-confidence point before an error provides more accurate supervision than locating the first explicit error. In this paper, we propose Confidence-Guided Reasoning Path Preference Optimization (CGPO), a method that leverages a confidence signal to identify points of maximal uncertainty in the model’s reasoning process and applies self-generated, non-human-like reasoning-path guidance to mitigate trajectory drift. Our experiments span diverse models applied to both code and mathematical reasoning tasks. The results show that, with the same amount of training data, our method using data generated by a small model can achieve better performance in most cases compared with approaches using data generated by a strong model or human-annotated.

Method: Uses TypePilot agentic framework with Scala and formal verification (Stainless framework) to enforce safety constraints through type-focused pipeline.

Result: Substantially mitigates input validation and injection vulnerabilities compared to direct code generation or naive secure prompting.

Conclusion: Type-guided LLM workflows have significant potential to improve trustworthiness of automated code generation in high-assurance domains.

Abstract: Large language Models (LLMs) have shown remarkable proficiency in code generation tasks across various programming languages. However, their outputs often contain subtle but critical vulnerabilities, posing significant risks when deployed in security-sensitive or mission-critical systems. This paper introduces TypePilot, an agentic AI framework designed to enhance the security and robustness of LLM-generated code by leveraging strongly typed and verifiable languages, using Scala as a representative example. We evaluate the effectiveness of our approach in two settings: formal verification with the Stainless framework and general-purpose secure code generation. Our experiments with leading open-source LLMs reveal that while direct code generation often fails to enforce safety constraints, just as naive prompting for more secure code, our type-focused agentic pipeline substantially mitigates input validation and injection vulnerabilities. The results demonstrate the potential of structured, type-guided LLM workflows to improve the SotA of the trustworthiness of automated code generation in high-assurance domains.

Method: The authors conduct exploratory studies on diverse multi-label datasets, analyze similarity distributions across models and datasets, and propose a label-specific thresholding method that optimizes thresholds for each label using validation data.

Result: The label-specific thresholding method achieves 46% average improvement over normalized 0.5 thresholding and outperforms uniform thresholding approaches by 14% on average, while maintaining strong performance with limited labeled examples.

Conclusion: Label-specific thresholding significantly improves distance-based multi-label text classification performance, with the method being robust even when limited labeled data is available.

Abstract: Distance-based unsupervised text classification is a method within text classification that leverages the semantic similarity between a label and a text to determine label relevance. This method provides numerous benefits, including fast inference and adaptability to expanding label sets, as opposed to zero-shot, few-shot, and fine-tuned neural networks that require re-training in such cases. In multi-label distance-based classification and information retrieval algorithms, thresholds are required to determine whether a text instance is “similar” to a label or query. Similarity between a text and label is determined in a dense embedding space, usually generated by state-of-the-art sentence encoders. Multi-label classification complicates matters, as a text instance can have multiple true labels, unlike in multi-class or binary classification, where each instance is assigned only one label. We expand upon previous literature on this underexplored topic by thoroughly examining and evaluating the ability of sentence encoders to perform distance-based classification. First, we perform an exploratory study to verify whether the semantic relationships between texts and labels vary across models, datasets, and label sets by conducting experiments on a diverse collection of realistic multi-label text classification (MLTC) datasets. We find that similarity distributions show statistically significant differences across models, datasets and even label sets. We propose a novel method for optimizing label-specific thresholds using a validation set. Our label-specific thresholding method achieves an average improvement of 46% over normalized 0.5 thresholding and outperforms uniform thresholding approaches from previous work by an average of 14%. Additionally, the method demonstrates strong performance even with limited labeled examples.

Method: Compiled a meta-collection of hate speech datasets for European Spanish with unified labels and metadata, then extended it by translating into European Portuguese and two Galician variants (one convergent with Spanish, another with Portuguese). Evaluated state-of-the-art LLMs in zero-shot, few-shot, and fine-tuning settings, and performed cross-lingual analysis.

Result: Established new benchmarks for hate speech detection in Iberian languages and provided baseline results for future research. The created aligned multilingual corpora enable more consistent and scalable hate speech detection across these languages.

Conclusion: The findings highlight the importance of multilingual and variety-aware approaches in hate speech detection and provide a foundation for improved benchmarking in underrepresented European languages, addressing the data gap for low-resource languages with multiple linguistic varieties.

Abstract: Hate speech poses a serious threat to social cohesion and individual well-being, particularly on social media, where it spreads rapidly. While research on hate speech detection has progressed, it remains largely focused on English, resulting in limited resources and benchmarks for low-resource languages. Moreover, many of these languages have multiple linguistic varieties, a factor often overlooked in current approaches. At the same time, large language models require substantial amounts of data to perform reliably, a requirement that low-resource languages often cannot meet. In this work, we address these gaps by compiling a meta-collection of hate speech datasets for European Spanish, standardised with unified labels and metadata. This collection is based on a systematic analysis and integration of existing resources, aiming to bridge the data gap and support more consistent and scalable hate speech detection. We extended this collection by translating it into European Portuguese and into a Galician standard that is more convergent with Spanish and another Galician variant that is more convergent with Portuguese, creating aligned multilingual corpora. Using these resources, we establish new benchmarks for hate speech detection in Iberian languages. We evaluate state-of-the-art large language models in zero-shot, few-shot, and fine-tuning settings, providing baseline results for future research. Moreover, we perform a cross-lingual analysis with our target languages. Our findings underscore the importance of multilingual and variety-aware approaches in hate speech detection and offer a foundation for improved benchmarking in underrepresented European languages.

Method: Clinical framework for chest X-ray VQA that probes CoT faithfulness via controlled text and image modifications across three axes: clinical fidelity, causal attribution, and confidence calibration, validated through reader studies with radiologists.

Result: Evaluator-radiologist correlations fell within inter-radiologist range, with strong alignment for attribution (τ_b=0.670), moderate for fidelity (τ_b=0.387), and weak for confidence tone (τ_b=0.091). Proprietary models scored higher on attribution (25.0% vs 1.4%) and often on fidelity (36.1% vs 31.7%) than open-source models.

Conclusion: Answer accuracy and explanation quality are decoupled, text cues shift explanations more than visual cues, and there are significant deployment risks requiring evaluation beyond final answer accuracy, especially in clinical settings.

Abstract: Vision-language models (VLMs) often produce chain-of-thought (CoT) explanations that sound plausible yet fail to reflect the underlying decision process, undermining trust in high-stakes clinical use. Existing evaluations rarely catch this misalignment, prioritizing answer accuracy or adherence to formats. We present a clinically grounded framework for chest X-ray visual question answering (VQA) that probes CoT faithfulness via controlled text and image modifications across three axes: clinical fidelity, causal attribution, and confidence calibration. In a reader study (n=4), evaluator-radiologist correlations fall within the observed inter-radiologist range for all axes, with strong alignment for attribution (Kendall’s $\tau_b=0.670$), moderate alignment for fidelity ($\tau_b=0.387$), and weak alignment for confidence tone ($\tau_b=0.091$), which we report with caution. Benchmarking six VLMs shows that answer accuracy and explanation quality are decoupled, acknowledging injected cues does not ensure grounding, and text cues shift explanations more than visual cues. While some open-source models match final answer accuracy, proprietary models score higher on attribution (25.0% vs. 1.4%) and often on fidelity (36.1% vs. 31.7%), highlighting deployment risks and the need to evaluate beyond final answer accuracy.

Method: Introduced Completion under Discourse Relation (CuDR) task and constructed minimal contrastive pairs for activation patching. Used circuit discovery to identify sparse computational graphs that control discourse processing.

Result: Found sparse circuits (≈0.2% of GPT-2) successfully recover discourse understanding in English PDTB-based CuDR task. These circuits generalize to unseen discourse frameworks (RST, SDRT). Lower layers capture linguistic features (lexical semantics, coreference) while upper layers encode discourse-level abstractions.

Conclusion: Discursive circuits are sparse but effective for discourse understanding, with consistent feature utility across frameworks and clear layer specialization in linguistic processing.

Abstract: Which components in transformer language models are responsible for discourse understanding? We hypothesize that sparse computational graphs, termed as discursive circuits, control how models process discourse relations. Unlike simpler tasks, discourse relations involve longer spans and complex reasoning. To make circuit discovery feasible, we introduce a task called Completion under Discourse Relation (CuDR), where a model completes a discourse given a specified relation. To support this task, we construct a corpus of minimal contrastive pairs tailored for activation patching in circuit discovery. Experiments show that sparse circuits ($\approx 0.2%$ of a full GPT-2 model) recover discourse understanding in the English PDTB-based CuDR task. These circuits generalize well to unseen discourse frameworks such as RST and SDRT. Further analysis shows lower layers capture linguistic features such as lexical semantics and coreference, while upper layers encode discourse-level abstractions. Feature utility is consistent across frameworks (e.g., coreference supports Expansion-like relations).

Method: Modular pipeline with semantic chunking, hierarchical concept extraction, Bloom’s Taxonomy-guided question generation, multi-chunk retrieval, and distractor contexts.

Result: Enables efficient generation of domain-grounded QAC triples for diverse RAG adaptation strategies, handling large evolving document corpora without redundant processing.

Conclusion: RAGen provides a scalable solution for adapting RAG systems to dynamic domains like scientific research and enterprise knowledge bases through automated domain-specific data generation.

Abstract: Retrieval-Augmented Generation (RAG) combines the language understanding and reasoning power of large language models (LLMs) with external retrieval to enable domain-grounded responses. Effectively adapting RAG systems to domain-specific settings requires specialized, context-rich training data beyond general-purpose question-answering. Here, we propose RAGen, a scalable and modular framework for generating domain-grounded question-answer-context (QAC) triples tailored to diverse RAG adaptation approaches. RAGen produces these QAC triples by identifying key concepts in documents, generating diverse questions guided by Bloom’s Taxonomy-inspired principles, and pairing them with precise answers extracted from relevant contexts. RAGen supports multiple RAG adaptation strategies, including the optimization of key components such as the LLM, retriever, and embedding model, etc. Its modular pipeline features semantic chunking, hierarchical concept extraction, and multi-chunk retrieval, along with the introduction of curated distractor contexts to promote robust reasoning. Designed for scalability, RAGen efficiently handles large and evolving document corpora without redundant processing, making it especially suitable for dynamic evolving domains such as scientific research and enterprise knowledge bases.

Method: Introduced SLAQ framework comparing LLMs’ answers to same factual questions asked in isolation (short) vs. integrated into complex queries (long), testing 16 LLMs across 600 queries with mechanistic analysis.

Result: Found systematic misalignment between short and long query answers, position-dependent accuracy loss, momentum effects, and that aligned facts activate overlapping model internals with 78% prediction accuracy.

Conclusion: Factual consistency over query complexity is crucial for LLM trustworthiness, challenging current evaluation practices that overestimate reliability in complex knowledge-seeking tasks.

Abstract: Large language models (LLMs) can correctly answer “When was Einstein born?” yet fail to provide the same date when writing about Einstein’s life revealing a fundamental inconsistency in how models access factual knowledge across task complexities. While models display impressive accuracy on factual question-answering benchmarks, the reliability gap between simple and complex queries remains poorly understood, eroding their trustworthiness. In this work, we introduce Short-Long Form Alignment for Factual Question Answering (SLAQ), a controlled evaluation framework that compares LLMs’ answers to the same factual questions asked (a) in isolation (short) vs. (b) integrated into complex queries (long). Looking at 16 LLMs across 600 queries, we find a systematic misalignment of answers to the corresponding short and long queries. We further uncover position-dependent accuracy loss and momentum effects where consecutive correct or incorrect answers create self-reinforcing patterns. Through mechanistic analysis, we find that aligned facts activate overlapping model internals, and that metrics based on mechanistic similarity can predict short-long answer alignment with up to 78% accuracy. Our work establishes factual consistency over query complexity as an important aspect of LLMs’ trustworthiness and challenges current evaluation practices, which implicitly assume that good performance for simple factual queries implies reliability in more complex knowledge-seeking tasks too.

Method: Introduces WebRouter with cost-aware Variational Information Bottleneck (ca-VIB) objective that learns compressed prompt representations while penalizing expected operational costs.

Result: Experiments on five WebVoyager benchmark websites show 87.8% cost reduction compared to GPT-4o baseline with only 3.8% accuracy drop.

Conclusion: WebRouter effectively addresses the cost-performance trade-off in web agents through information-theoretic routing with significant cost savings.

Abstract: LLM-brained web agents offer powerful capabilities for web automation but face a critical cost-performance trade-off. The challenge is amplified by web agents’ inherently complex prompts that include goals, action histories, and environmental states, leading to degraded LLM ensemble performance. To address this, we introduce WebRouter, a novel query-specific router trained from an information-theoretic perspective. Our core contribution is a cost-aware Variational Information Bottleneck (ca-VIB) objective, which learns a compressed representation of the input prompt while explicitly penalizing the expected operational cost. Experiments on five real-world websites from the WebVoyager benchmark show that WebRouter reduces operational costs by a striking 87.8% compared to a GPT-4o baseline, while incurring only a 3.8% accuracy drop.

Method: Evaluated BERT and DistilBERT on Moral Foundations Twitter Corpus (MFTC) and Moral Foundations Reddit Corpus (MFRC) using in-domain and cross-domain protocols with multi-label classification, and introduced the MFC metric to quantify cross-domain fairness stability.

Result: Found pronounced asymmetry in transfer (Twitter->Reddit degraded micro-F1 by 14.9% vs 1.5% for Reddit->Twitter), significant fairness violations in authority label (DPD: 0.22-0.23, EOD: 0.40-0.41), and MFC showed perfect negative correlation with Demographic Parity Difference (rho = -1.000).

Conclusion: MFC serves as a complementary, diagnosis-oriented metric for fairness-aware evaluation of moral reasoning models, enabling more reliable deployment across different linguistic contexts by revealing hidden fairness issues.

Abstract: Ensuring fairness in natural language processing for moral sentiment classification is challenging, particularly under cross-domain shifts where transformer models are increasingly deployed. Using the Moral Foundations Twitter Corpus (MFTC) and Moral Foundations Reddit Corpus (MFRC), this work evaluates BERT and DistilBERT in a multi-label setting with in-domain and cross-domain protocols. Aggregate performance can mask disparities: we observe pronounced asymmetry in transfer, with Twitter->Reddit degrading micro-F1 by 14.9% versus only 1.5% for Reddit->Twitter. Per-label analysis reveals fairness violations hidden by overall scores; notably, the authority label exhibits Demographic Parity Differences of 0.22-0.23 and Equalized Odds Differences of 0.40-0.41. To address this gap, we introduce the Moral Fairness Consistency (MFC) metric, which quantifies the cross-domain stability of moral foundation detection. MFC shows strong empirical validity, achieving a perfect negative correlation with Demographic Parity Difference (rho = -1.000, p < 0.001) while remaining independent of standard performance metrics. Across labels, loyalty demonstrates the highest consistency (MFC = 0.96) and authority the lowest (MFC = 0.78). These findings establish MFC as a complementary, diagnosis-oriented metric for fairness-aware evaluation of moral reasoning models, enabling more reliable deployment across heterogeneous linguistic contexts. .

Method: Compared supervised fine-tuning (T5 models) and few-shot in-context learning (GPT models) using graph-matching, bidirectional entailment with theorem proving, and well-formedness checks under normalized and unnormalized targets.

Result: Models performing well on graph-matching often fail to produce logically equivalent formulas. Normalization improves well-formedness and logical adequacy. Performance degrades with formula complexity and specific linguistic features like coordination and passive voice.

Conclusion: Graph-based metrics have limitations for reasoning applications, motivating logic-sensitive evaluation methods and simplified, normalized target representations for better semantic parsing.

Abstract: Graph-matching metrics such as Smatch are the de facto standard for evaluating neural semantic parsers, yet they capture surface overlap rather than logical equivalence. We reassess evaluation by pairing graph-matching with automated theorem proving. We compare two approaches to building parsers: supervised fine-tuning (T5-Small/Base) and few-shot in-context learning (GPT-4o/4.1/5), under normalized and unnormalized targets. We evaluate outputs using graph-matching, bidirectional entailment between source and target formulas with a first-order logic theorem prover, and well-formedness. Across settings, we find that models performing well on graph-matching often fail to produce logically equivalent formulas. Normalization reduces incidental target variability, improves well-formedness, and strengthens logical adequacy. Error analysis shows performance degrades with increasing formula complexity and with coordination, prepositional phrases, and passive voice; the dominant failures involve variable binding and indexing, and predicate naming. These findings highlight limits of graph-based metrics for reasoning-oriented applications and motivate logic-sensitive evaluation and training objectives together with simplified, normalized target representations. All code and data for our experiments are publicly available.

Method: Created CNSocialDepress dataset containing 44,178 texts from 233 users, with 10,306 depression-related segments annotated by psychological experts, providing binary risk labels and structured multi-dimensional psychological attributes.

Result: Experimental results demonstrate the dataset’s utility across various NLP tasks, including structured psychological profiling and fine-tuning large language models for depression detection.

Conclusion: The dataset provides effective and practical value for depression risk identification and psychological analysis, offering insights for mental health applications tailored for Chinese-speaking populations.

Abstract: Depression is a pressing global public health issue, yet publicly available Chinese-language resources for risk detection remain scarce and are mostly limited to binary classification. To address this limitation, we release CNSocialDepress, a benchmark dataset for depression risk detection from Chinese social media posts. The dataset contains 44,178 texts from 233 users, within which psychological experts annotated 10,306 depression-related segments. CNSocialDepress provides binary risk labels together with structured multi-dimensional psychological attributes, enabling interpretable and fine-grained analysis of depressive signals. Experimental results demonstrate its utility across a wide range of NLP tasks, including structured psychological profiling and fine-tuning of large language models for depression detection. Comprehensive evaluations highlight the dataset’s effectiveness and practical value for depression risk identification and psychological analysis, thereby providing insights to mental health applications tailored for Chinese-speaking populations.

Method: XQuant introduces two key innovations: a computationally negligible data-free calibration method and cross-layer KV cache compression, enabling training-free, plug-and-play ultra-low bit-width quantization.

Result: Extensive experiments on TruthfulQA and LongBench show XQuant outperforms state-of-the-art methods (KIVI-2bit and AsymKV-1.5bit) by achieving lower bit-width while maintaining superior performance.

Conclusion: XQuant establishes a better trade-off between memory efficiency and model accuracy, providing an effective solution for deploying LLMs in resource-constrained environments.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse natural language processing tasks. However, their extensive memory requirements, particularly due to KV cache growth during long-text understanding and generation, present significant challenges for deployment in resource-constrained environments. Quantization has emerged as a promising solution to reduce memory consumption while preserving historical information. We propose XQuant, a training-free and plug-and-play framework that achieves ultra-low equivalent bit-width KV cache quantization. XQuant introduces two key innovations: a computationally negligible data-free calibration method and cross-layer KV cache compression, enabling quantization to sub-1.4 bits. Extensive experiments on TruthfulQA and LongBench demonstrate that XQuant outperforms state-of-the-art methods (e.g., KIVI-2bit and AsymKV-1.5bit) by achieving lower bit-width while maintaining superior performance, establishing a better trade-off between memory efficiency and model accuracy.

Method: Proposes repurposing automated fact-checking as a cognitive self-defense tool for agents, requiring them to critically evaluate retrieved information by corroborating claims with external evidence and assessing source trustworthiness.

Result: Identifies that current defenses are insufficient against content-based attacks and establishes the need for agents to perform critical evaluation of external information.

Conclusion: Automated fact-checking should be adapted as a defense mechanism to protect AI agents from content-based manipulation attacks, enabling them to verify information credibility independently.

Abstract: When AI agents retrieve and reason over external documents, adversaries can manipulate the data they receive to subvert their behaviour. Previous research has studied indirect prompt injection, where the attacker injects malicious instructions. We argue that injection of instructions is not necessary to manipulate agents - attackers could instead supply biased, misleading, or false information. We term this an attack by content. Existing defenses, which focus on detecting hidden commands, are ineffective against attacks by content. To defend themselves and their users, agents must critically evaluate retrieved information, corroborating claims with external evidence and evaluating source trustworthiness. We argue that this is analogous to an existing NLP task, automated fact-checking, which we propose to repurpose as a cognitive self-defense tool for agents.

Method: Systematically evaluated reliability and validity of human psychometric tests for three constructs (sexism, racism, morality) using multiple item and prompt variations. Validity was assessed through convergent (theory-based inter-test correlations) and ecological approaches (alignment between test scores and real-world downstream task behavior).

Result: Found moderate reliability across variations, but psychometric test scores did not align with model behavior in downstream tasks - in some cases even showing negative correlations, indicating low ecological validity.

Conclusion: Systematic evaluation of psychometric tests is essential before interpreting their scores for LLMs, and human-designed psychometric tests cannot be directly applied to LLMs without adaptation.

Abstract: Psychometric tests are increasingly used to assess psychological constructs in large language models (LLMs). However, it remains unclear whether these tests – originally developed for humans – yield meaningful results when applied to LLMs. In this study, we systematically evaluate the reliability and validity of human psychometric tests for three constructs: sexism, racism, and morality. We find moderate reliability across multiple item and prompt variations. Validity is evaluated through both convergent (i.e., testing theory-based inter-test correlations) and ecological approaches (i.e., testing the alignment between tests scores and behavior in real-world downstream tasks). Crucially, we find that psychometric test scores do not align, and in some cases even negatively correlate with, model behavior in downstream tasks, indicating low ecological validity. Our results highlight that systematic evaluations of psychometric tests is essential before interpreting their scores. They also suggest that psychometric tests designed for humans cannot be applied directly to LLMs without adaptation.

Method: Proposes EASE framework with sequential evaluation: evidence-based evaluation, reasoning-based evaluation using LLMs, and sentiment-based fallback. Uses instruction tuning with pseudo labels to enhance evaluation accuracy and integrates evaluators’ assessments with news content.

Result: EASE achieves state-of-the-art performance across multiple benchmarks and significantly improves generalization to real-time news. Introduces RealTimeNews-25 benchmark for evaluating emerging news with limited evidence.

Conclusion: EASE effectively addresses evidence scarcity in real-time fake news detection through dynamic evaluation-aware decision-making, demonstrating superior performance and generalization capabilities.

Abstract: Fake news detection becomes particularly challenging in real-time scenarios, where emerging events often lack sufficient supporting evidence. Existing approaches often rely heavily on external evidence and therefore struggle to generalize under evidence scarcity. To address this issue, we propose Evaluation-Aware Selection of Experts (EASE), a novel framework for real-time fake news detection that dynamically adapts its decision-making process according to the assessed sufficiency of available evidence. EASE introduces a sequential evaluation mechanism comprising three independent perspectives: (1) Evidence-based evaluation, which assesses evidence and incorporates it into decision-making only when the evidence is sufficiently supportive; (2) Reasoning-based evaluation, which leverages the world knowledge of large language models (LLMs) and applies them only when their reliability is adequately established; and (3) Sentiment-based fallback, which integrates sentiment cues when neither evidence nor reasoning is reliable. To enhance the accuracy of evaluation processes, EASE employs instruction tuning with pseudo labels to guide each evaluator in justifying its perspective-specific knowledge through interpretable reasoning. Furthermore, the expert modules integrate the evaluators’ justified assessments with the news content to enable evaluation-aware decision-making, thereby enhancing overall detection accuracy. Moreover, we introduce RealTimeNews-25, a new benchmark comprising recent news for evaluating model generalization on emerging news with limited evidence. Extensive experiments demonstrate that EASE not only achieves state-of-the-art performance across multiple benchmarks, but also significantly improves generalization to real-time news. The code and dataset are available: https://github.com/wgyhhhh/EASE.

Method: Tested three frontier models across three datasets using narrow in-context examples (64-256 examples) and analyzed step-by-step reasoning through chain-of-thought prompting.

Result: Models produced broadly misaligned responses at rates between 2%-17% with 64 examples and up to 58% with 256 examples. 67.5% of misaligned traces showed models adopting dangerous personas to rationalize harmful outputs.

Conclusion: Emergent misalignment is not limited to fine-tuning but also emerges in in-context learning, with models developing harmful personas to justify misaligned behavior, highlighting a significant safety concern.

Abstract: Recent work has shown that narrow finetuning can produce broadly misaligned LLMs, a phenomenon termed emergent misalignment (EM). While concerning, these findings were limited to finetuning and activation steering, leaving out in-context learning (ICL). We therefore ask: does EM emerge in ICL? We find that it does: across three datasets, three frontier models produce broadly misaligned responses at rates between 2% and 17% given 64 narrow in-context examples, and up to 58% with 256 examples. We also examine mechanisms of EM by eliciting step-by-step reasoning (while leaving in-context examples unchanged). Manual analysis of the resulting chain-of-thought shows that 67.5% of misaligned traces explicitly rationalize harmful outputs by adopting a reckless or dangerous ‘‘persona’’, echoing prior results on finetuning-induced EM.

Method: Hierarchical extraction framework that organizes information at multiple levels using state-of-the-art LLMs, enabling creation of semantically rich and well-structured knowledge graphs.

Result: Comprehensive evaluation from structural and semantic perspectives reveals strengths and shortcomings of current LLMs in KG construction, identifying key challenges for future work.

Conclusion: The framework advances KG construction capabilities and the released dataset on children’s mental well-being aims to foster transparent, reliable applications in high-stakes domains like healthcare.

Abstract: Knowledge graphs (KGs) are vital for knowledge-intensive tasks and have shown promise in reducing hallucinations in large language models (LLMs). However, constructing high-quality KGs remains difficult, requiring accurate information extraction and structured representations that support interpretability and downstream utility. Existing LLM-based approaches often focus narrowly on entity and relation extraction, limiting coverage to sentence-level contexts or relying on predefined schemas. We propose a hierarchical extraction framework that organizes information at multiple levels, enabling the creation of semantically rich and well-structured KGs. Using state-of-the-art LLMs, we extract and construct knowledge graphs and evaluate them comprehensively from both structural and semantic perspectives. Our results highlight the strengths and shortcomings of current LLMs in KG construction and identify key challenges for future work. To advance research in this area, we also release a curated dataset of LLM-generated KGs derived from research papers on children’s mental well-being. This resource aims to foster more transparent, reliable, and impactful applications in high-stakes domains such as healthcare.

Method: The approach provides language feedback embeddings as input to Transformer-based policies, optionally adding auxiliary self-supervised learning objectives for feedback prediction alongside traditional next action prediction.

Result: Significant improvements in compositional generalization abilities and robustness on embodied Vision-and-Language tasks in the BabyAI-XGen environment, showing successful conversion of suboptimal behavior into learning opportunities.

Conclusion: Language feedback is a competitive and intuitive alternative to intermediate scalar rewards for language-specified embodied tasks, enabling data-efficient learning from both optimal and suboptimal demonstrations.

Abstract: Current approaches to embodied AI tend to learn policies from expert demonstrations. However, without a mechanism to evaluate the quality of demonstrated actions, they are limited to learning from optimal behaviour, or they risk replicating errors and inefficiencies. While reinforcement learning offers one alternative, the associated exploration typically results in sacrificing data efficiency. This work explores how agents trained with imitation learning can learn robust representations from both optimal and suboptimal demonstrations when given access to constructive language feedback as a means to contextualise different modes of behaviour. We directly provide language feedback embeddings as part of the input sequence into a Transformer-based policy, and optionally complement the traditional next action prediction objective with auxiliary self-supervised learning objectives for feedback prediction. We test our approach on a range of embodied Vision-and-Language tasks in our custom BabyAI-XGen environment and show significant improvements in agents’ compositional generalisation abilities and robustness, suggesting that our data-efficient method allows models to successfully convert suboptimal behaviour into learning opportunities. Overall, our results suggest that language feedback is a competitive and intuitive alternative to intermediate scalar rewards for language-specified embodied tasks.

Method: Developed five prompt templates (Basic Object Focus, Contextual Scene, Educational Layout, Multi-Level Detail, Grid Layout) with accessibility constraints, evaluated using 400 sentence simplifications from four TS datasets through CLIPScores and human annotation by accessibility experts.

Result: Basic Object Focus template achieved highest semantic alignment, Retro style was most accessible, Wikipedia was most effective data source, and Text Simplicity showed strong reliability while Image Quality was more subjective.

Conclusion: The framework provides practical guidelines for accessible content generation and emphasizes the importance of structured prompting in AI-generated visual accessibility tools, with visual minimalism enhancing language accessibility.

Abstract: Individuals with intellectual disabilities often have difficulties in comprehending complex texts. While many text-to-image models prioritize aesthetics over accessibility, it is not clear how visual illustrations relate to text simplifications (TS) generated from them. This paper presents a structured vision-language model (VLM) prompting framework for generating accessible images from simplified texts. We designed five prompt templates, i.e., Basic Object Focus, Contextual Scene, Educational Layout, Multi-Level Detail, and Grid Layout, each following distinct spatial arrangements while adhering to accessibility constraints such as object count limits, spatial separation, and content restrictions. Using 400 sentence-level simplifications from four established TS datasets (OneStopEnglish, SimPA, Wikipedia, and ASSET), we conducted a two-phase evaluation: Phase 1 assessed prompt template effectiveness with CLIPScores, and Phase 2 involved human annotation of generated images across ten visual styles by four accessibility experts. Results show that the Basic Object Focus prompt template achieved the highest semantic alignment, indicating that visual minimalism enhances language accessibility. Expert evaluation further identified Retro style as the most accessible and Wikipedia as the most effective data source. Inter-annotator agreement varied across dimensions, with Text Simplicity showing strong reliability and Image Quality proving more subjective. Overall, our framework offers practical guidelines for accessible content generation and underscores the importance of structured prompting in AI-generated visual accessibility tools.

Method: Constructed SEV dataset to elicit comparable internal states, extracted context-agnostic emotion directions, identified neurons and attention heads through analytical decomposition and causal analysis, and integrated local components into global emotion circuits.

Result: Identified consistent cross-context emotion encoding, validated causal roles of emotional components, and achieved 99.65% emotion-expression accuracy through circuit modulation, surpassing prompting- and steering-based methods.

Conclusion: First systematic study to uncover and validate emotion circuits in LLMs, offering new insights into interpretability and controllable emotional intelligence.

Abstract: As the demand for emotional intelligence in large language models (LLMs) grows, a key challenge lies in understanding the internal mechanisms that give rise to emotional expression and in controlling emotions in generated text. This study addresses three core questions: (1) Do LLMs contain context-agnostic mechanisms shaping emotional expression? (2) What form do these mechanisms take? (3) Can they be harnessed for universal emotion control? We first construct a controlled dataset, SEV (Scenario-Event with Valence), to elicit comparable internal states across emotions. Subsequently, we extract context-agnostic emotion directions that reveal consistent, cross-context encoding of emotion (Q1). We identify neurons and attention heads that locally implement emotional computation through analytical decomposition and causal analysis, and validate their causal roles via ablation and enhancement interventions. Next, we quantify each sublayer’s causal influence on the model’s final emotion representation and integrate the identified local components into coherent global emotion circuits that drive emotional expression (Q2). Directly modulating these circuits achieves 99.65% emotion-expression accuracy on the test set, surpassing prompting- and steering-based methods (Q3). To our knowledge, this is the first systematic study to uncover and validate emotion circuits in LLMs, offering new insights into interpretability and controllable emotional intelligence.

Method: Large-scale experiments across multiple datasets and LLMs, analysis of perplexity as a key metric for readability, and proposing a benchmarking procedure for LLM-specific utility judgments.

Result: Human-annotated passages are not optimal for LLMs, ground-truth utilitarian passages are not transferable across different LLMs, and LLMs struggle to assess utility effectively - failing to reject all passages for known queries and select truly useful ones for unknown queries.

Conclusion: The findings highlight the necessity of adopting LLM-specific utility in RAG research, as utility judgments should be tailored to specific language models rather than treated as generic attributes.

Abstract: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by incorporating external knowledge. While traditional retrieval focuses on relevance, RAG’s effectiveness depends on the utility of retrieved passages, i.e., the usefulness in facilitating the generation of an accurate and comprehensive answer. Existing studies often treat utility as a generic attribute, ignoring the fact that different LLMs may benefit differently from the same passage due to variations in internal knowledge and comprehension ability. In this work, we introduce and systematically investigate the notion of LLM-specific utility. Through large-scale experiments across multiple datasets and LLMs, we demonstrate that human-annotated passages are not optimal for LLMs and that ground-truth utilitarian passages are not transferable across different LLMs. These findings highlight the necessity of adopting the LLM-specific utility in RAG research. Our findings indicate that some human-annotated passages are not ground-truth utilitarian passages for specific LLMs, partially due to the varying readability of queries and passages for LLMs, a tendency for which perplexity is a key metric. Based on these findings, we propose a benchmarking procedure for LLM-specific utility judgments. We evaluate existing utility judgment methods on six datasets and find that while verbalized methods using pseudo-answers perform robustly, LLMs struggle to assess utility effectively-failing to reject all passages for known queries and to select truly useful ones for unknown queries.

Method: Proposed Rollout Routing Replay (R3) method that records routing distributions from inference engine and replays them during training to reduce policy KL divergence.

Result: R3 significantly reduces training-inference policy divergence, prevents RL training collapse, and outperforms methods like GSPO and TIS across various settings.

Conclusion: R3 provides an effective solution for stabilizing RL training in MoE models by addressing fundamental routing inconsistencies between training and inference phases.

Abstract: Reinforcement learning (RL) has emerged as a crucial approach for enhancing the capabilities of large language models. However, in Mixture-of-Experts (MoE) models, the routing mechanism often introduces instability, even leading to catastrophic RL training collapse. We analyze the training-inference consistency of MoE models and identify a notable discrepancy in routing behaviors between the two phases. Moreover, even under identical conditions, the routing framework can yield divergent expert selections across repeated forward passes. To address this foundational inconsistency, we propose Rollout Routing Replay (R3), a method that records routing distributions from the inference engine and replays them during training. R3 significantly reduces training-inference policy KL divergence and mitigates extreme discrepancies without compromising training speed. Extensive experiments on various settings confirm that R3 succeeds in stabilizing RL training, preventing collapse and outperforming methods such as GSPO and TIS. We believe this work can offer a new solution for stabilizing RL in MoE models.

Method: Fine-tuned Pythia, Llama3, and Mistral models (1.4B-70B parameters) on common datasets, tracked verbatim memorization throughout training. Used n-gram memorization score for early stopping and introduced n-gram-aware loss regularizer.

Result: Memorization increases dramatically in first few epochs, often before validation perplexity or evaluation performance is optimized. The proposed methods reduce memorization by up to 40% across all model families with minimal performance trade-offs compared to existing strategies.

Conclusion: The study provides practical, scalable insights into memorization dynamics during language model fine-tuning and offers effective mitigation strategies that balance memorization reduction with model performance.

Abstract: Although large language models excel across many tasks, they can memorise training data and thereby expose private or copyrighted text. Most defences target the pre-training stage, leaving memorisation during fine-tuning, especially for domain adaptation and instruction tuning, poorly understood. We fine-tune Pythia, Llama3, and Mistral models spanning 1.4B-70B parameters on common evaluation datasets and track verbatim memorisation throughout training. We find that memorisation increases dramatically in the first few epochs, often significantly before either validation perplexity or evaluation performance is optimised. We use a simple but effective n-gram memorisation score which reliably precedes verbatim memorisation; using it as an early-stopping criterion mitigates memorisation with minimal performance loss. Further, we introduce an n-gram-aware loss regulariser and show that it reduces memorisation across all model families tested by up to 40% while minimising evaluation performance trade-offs when compared to an existing memorisation mitigation strategy. These results yield practical, scalable insights into memorisation dynamics during language model fine-tuning.

Method: Curated a human-verified multimodal Werewolf dataset with 100+ hours of video and 32.4M tokens, then proposed a two-stage strategy-alignment evaluation: speech evaluation (multiple-choice tasks across 5 social dimensions) and decision evaluation (voting choices and role inferences).

Result: State-of-the-art LLMs show diverse performance with roughly half scoring below 0.50, revealing significant gaps in deception and counterfactual reasoning capabilities.

Conclusion: The dataset and evaluation framework enable fine-grained assessment of linguistic and reasoning capabilities in multi-agent interaction, highlighting current limitations in social intelligence and inspiring future research.

Abstract: Social deduction games like Werewolf combine language, reasoning, and strategy, providing a testbed for studying natural language and social intelligence. However, most studies reduce the game to LLM-based self-play, yielding templated utterances and anecdotal cases that overlook the richness of social gameplay. Evaluation further relies on coarse metrics such as survival time or subjective scoring due to the lack of quality reference data. To address these gaps, we curate a high-quality, human-verified multimodal Werewolf dataset containing over 100 hours of video, 32.4M utterance tokens, and 15 rule variants. Based on this dataset, we propose a novel strategy-alignment evaluation that leverages the winning faction’s strategies as ground truth in two stages: 1) Speech evaluation, formulated as multiple-choice-style tasks that assess whether the model can adopt appropriate stances across five dimensions of social ability; and 2) Decision evaluation, which assesses the model’s voting choices and opponent-role inferences. This framework enables a fine-grained evaluation of models’ linguistic and reasoning capabilities, while capturing their ability to generate strategically coherent gameplay. Our experiments show that state-of-the-art LLMs show diverse performance, with roughly half remain below 0.50, revealing clear gaps in deception and counterfactual reasoning. We hope our dataset further inspires research on language, reasoning, and strategy in multi-agent interaction.

Method: Combines introspection (generating and classifying feasible/infeasible tasks) with consensus-based rewarding (reinforcing stability of self-knowledge through internal agreement) using internally generated data.

Result: Improved self-knowledge in LLaMA-3.1-8B and Qwen-2.5-7B, with gains of 28% in accuracy and 12% in F1, outperforming baselines in few iterations without external supervision.

Conclusion: KnowRL unlocks LLMs’ capacity to self-improve knowledge awareness, enabling more reliable and accountable AI for safer deployment in critical applications due to its simplicity and independence from external effort.

Abstract: Truly reliable AI requires more than simply scaling up knowledge; it demands the ability to know what it knows and when it does not. Yet recent research shows that even the best LLMs misjudge their own competence in more than one in five cases, making any response born of such internal uncertainty impossible to fully trust. Inspired by self-improvement reinforcement learning techniques that require minimal data, we present a simple but powerful framework KnowRL that strengthens a model’s internal understanding of its own feasibility boundaries, enabling safer and more responsible behaviour. Our framework combines two components: (i) introspection, where the model generates and classifies tasks it judges feasible or infeasible, and (ii) consensus-based rewarding, where stability of self-knowledge assessment is reinforced through internal agreement. By using internally generated data, this design strengthens consistency in self-knowledge and entirely avoids costly external supervision. In experiments on LLaMA-3.1-8B and Qwen-2.5-7B, KnowRL steadily improved self-knowledge, validated by both intrinsic self-consistency and extrinsic benchmarking. With nothing more than a small seed set and no external supervision, our method drove gains as high as 28% in accuracy and 12% in F1, outperforming baselines in just a few iterations. Our framework essentially unlocks the untapped capacity of LLMs to self-improve their knowledge awareness, opening the door to reliable, more accountable AI and safer deployment in critical applications. Owing to its simplicity and independence from external effort, we encourage applying this reliability-enhancing process to all future models.

Method: Study the interplay between synthesis methods (using LLMs to generate survey responses) and rectification methods (debiasing population estimates). Analyze optimal allocation of human responses between synthesis and rectification under fixed budget constraints.

Result: Synthesis alone introduces substantial bias (24-86%). Combining synthesis with rectification reduces bias below 5% and increases effective sample size by up to 14%. Allocating most human responses to rectification rather than fine-tuning produces more effective estimation.

Conclusion: The common practice of using all human responses for fine-tuning is suboptimal. Under fixed budgets, allocating most resources to rectification results in far more effective population estimation from LLM-generated survey responses.

Abstract: Surveys provide valuable insights into public opinion and behavior, but their execution is costly and slow. Large language models (LLMs) have been proposed as a scalable, low-cost substitute for human respondents, but their outputs are often biased and yield invalid estimates. We study the interplay between synthesis methods that use LLMs to generate survey responses and rectification methods that debias population estimates, and explore how human responses are best allocated between them. Using two panel surveys with questions on nutrition, politics, and economics, we find that synthesis alone introduces substantial bias (24-86%), whereas combining it with rectification reduces bias below 5% and increases effective sample size by up to 14%. Overall, we challenge the common practice of using all human responses for fine-tuning, showing that under a fixed budget, allocating most to rectification results in far more effective estimation.

Method: Applied automatic personality and gender classifiers to model replies on open-ended questions from Reddit, comparing six widely used LLMs to human-authored responses.

Result: LLMs systematically express higher Agreeableness and lower Neuroticism, reflecting cooperative and stable conversational tendencies. Gendered language patterns resemble human writers but with reduced variation.

Conclusion: The study provides new insights into personality and demographic patterns of generative AI through large-scale comparative analyses and contributes a new dataset of human and model responses.

Abstract: Generative large language models (LLMs) have become central to everyday life, producing human-like text across diverse domains. A growing body of research investigates whether these models also exhibit personality- and demographic-like characteristics in their language. In this work, we introduce a novel, data-driven methodology for assessing LLM personality without relying on self-report questionnaires, applying instead automatic personality and gender classifiers to model replies on open-ended questions collected from Reddit. Comparing six widely used models to human-authored responses, we find that LLMs systematically express higher Agreeableness and lower Neuroticism, reflecting cooperative and stable conversational tendencies. Gendered language patterns in model text broadly resemble those of human writers, though with reduced variation, echoing prior findings on automated agents. We contribute a new dataset of human and model responses, along with large-scale comparative analyses, shedding new light on the topic of personality and demographic patterns of generative AI.

Method: Convert CNER into sustained entity triplet sequence generation using pre-trained seq2seq models, with type-specific confidence-based pseudo labeling and knowledge distillation at triplet level.

Result: Outperforms previous state-of-the-art methods on two benchmark datasets in multiple CNER settings, achieving smallest gap compared with non-continual learning results.

Conclusion: GenCNER provides an effective generative framework that successfully mitigates catastrophic forgetting and semantic shift issues in continual NER through triplet-level generation and knowledge preservation techniques.

Abstract: Traditional named entity recognition (NER) aims to identify text mentions into pre-defined entity types. Continual Named Entity Recognition (CNER) is introduced since entity categories are continuously increasing in various real-world scenarios. However, existing continual learning (CL) methods for NER face challenges of catastrophic forgetting and semantic shift of non-entity type. In this paper, we propose GenCNER, a simple but effective Generative framework for CNER to mitigate the above drawbacks. Specifically, we skillfully convert the CNER task into sustained entity triplet sequence generation problem and utilize a powerful pre-trained seq2seq model to solve it. Additionally, we design a type-specific confidence-based pseudo labeling strategy along with knowledge distillation (KD) to preserve learned knowledge and alleviate the impact of label noise at the triplet level. Experimental results on two benchmark datasets show that our framework outperforms previous state-of-the-art methods in multiple CNER settings, and achieves the smallest gap compared with non-CL results.

Method: Used Large Language Models to perform stopword removal, lemmatization, and stemming, comparing them against traditional algorithms on web-sourced data across six European languages in text classification tasks.

Result: LLMs achieved 97% accuracy for stopword removal, 82% for lemmatization, and 74% for stemming. ML models trained on LLM-preprocessed text showed up to 6% improvement in F1 score compared to traditional preprocessing.

Conclusion: LLM-based preprocessing is a viable alternative to traditional methods, offering context-aware processing that improves downstream text classification performance across multiple languages.

Abstract: Text preprocessing is a fundamental component of Natural Language Processing, involving techniques such as stopword removal, stemming, and lemmatization to prepare text as input for further processing and analysis. Despite the context-dependent nature of the above techniques, traditional methods usually ignore contextual information. In this paper, we investigate the idea of using Large Language Models (LLMs) to perform various preprocessing tasks, due to their ability to take context into account without requiring extensive language-specific annotated resources. Through a comprehensive evaluation on web-sourced data, we compare LLM-based preprocessing (specifically stopword removal, lemmatization and stemming) to traditional algorithms across multiple text classification tasks in six European languages. Our analysis indicates that LLMs are capable of replicating traditional stopword removal, lemmatization, and stemming methods with accuracies reaching 97%, 82%, and 74%, respectively. Additionally, we show that ML algorithms trained on texts preprocessed by LLMs achieve an improvement of up to 6% with respect to the $F_1$ measure compared to traditional techniques. Our code, prompts, and results are publicly available at https://github.com/GianCarloMilanese/llm_pipeline_wi-iat.

Method: Proposes a multi-path reasoning mechanism to obtain fine-grained comparable signals and a segment-aware temporalized cross-attention module to adaptively fuse aligned representations, pinpointing subtle dissonances between internal states and external reasoning.

Result: Extensive experiments on three diverse benchmarks and two leading LLMs demonstrate that the framework consistently and significantly outperforms strong baselines.

Conclusion: The unified framework successfully resolves the detection dilemma by bridging internal state probing and chain-of-thought verification, providing comprehensive hallucination detection across both fact-intensive and logic-intensive tasks.

Abstract: The detection of sophisticated hallucinations in Large Language Models (LLMs) is hampered by a ``Detection Dilemma’’: methods probing internal states (Internal State Probing) excel at identifying factual inconsistencies but fail on logical fallacies, while those verifying externalized reasoning (Chain-of-Thought Verification) show the opposite behavior. This schism creates a task-dependent blind spot: Chain-of-Thought Verification fails on fact-intensive tasks like open-domain QA where reasoning is ungrounded, while Internal State Probing is ineffective on logic-intensive tasks like mathematical reasoning where models are confidently wrong. We resolve this with a unified framework that bridges this critical gap. However, unification is hindered by two fundamental challenges: the Signal Scarcity Barrier, as coarse symbolic reasoning chains lack signals directly comparable to fine-grained internal states, and the Representational Alignment Barrier, a deep-seated mismatch between their underlying semantic spaces. To overcome these, we introduce a multi-path reasoning mechanism to obtain more comparable, fine-grained signals, and a segment-aware temporalized cross-attention module to adaptively fuse these now-aligned representations, pinpointing subtle dissonances. Extensive experiments on three diverse benchmarks and two leading LLMs demonstrate that our framework consistently and significantly outperforms strong baselines. Our code is available: https://github.com/peach918/HalluDet.

Method: Constructs fully connected graph over PhoBERT token embeddings, applies GAT layer to capture inter-token dependencies, adds Transformer self-attention layer, and uses classification head for sequence labeling.

Result: Consistently outperforms strong baselines including transformer-only and hybrid neural models (BiLSTM + CNN + CRF) on three Vietnamese datasets: PhoNER-COVID19, PhoDisfluency, and VietMed-NER.

Conclusion: Combining pretrained semantic features with graph-based relational modeling is effective for improved token classification across multiple domains.

Abstract: We propose a novel neural architecture named TextGraphFuseGAT, which integrates a pretrained transformer encoder (PhoBERT) with Graph Attention Networks for token-level classification tasks. The proposed model constructs a fully connected graph over the token embeddings produced by PhoBERT, enabling the GAT layer to capture rich inter-token dependencies beyond those modeled by sequential context alone. To further enhance contextualization, a Transformer-style self-attention layer is applied on top of the graph-enhanced embeddings. The final token representations are passed through a classification head to perform sequence labeling. We evaluate our approach on three Vietnamese benchmark datasets: PhoNER-COVID19 for named entity recognition in the COVID-19 domain, PhoDisfluency for speech disfluency detection, and VietMed-NER for medical-domain NER. VietMed-NER is the first Vietnamese medical spoken NER dataset, featuring 18 entity types collected from real-world medical speech transcripts and annotated with the BIO tagging scheme. Its specialized vocabulary and domain-specific expressions make it a challenging benchmark for token-level classification models. Experimental results show that our method consistently outperforms strong baselines, including transformer-only and hybrid neural models such as BiLSTM + CNN + CRF, confirming the effectiveness of combining pretrained semantic features with graph-based relational modeling for improved token classification across multiple domains.

Method: Two-step reformulation: remove self-talk behaviors and reorder sub-conclusions, implemented via a small auxiliary model with minimal computational overhead.

Result: PART consistently disrupts distillation across various student models and benchmarks. For example, a 32B model’s performance dropped from 54.17 to 46.88 on AIME 2024 (13.5% degradation).

Conclusion: PART effectively protects reasoning traces from unauthorized distillation while preserving information for human users, addressing the trade-off between transparency and protection.

Abstract: Recent advances in Large Language Models (LLMs) show that extending the length of reasoning chains significantly improves performance on complex tasks. While revealing these reasoning traces helps users better follow, verify, and learn from the model’s problem-solving process, it also makes them highly vulnerable to unauthorized distillation. To mitigate this risk, proprietary model providers often adopt aggressive protection strategies, such as replacing detailed reasoning with brief summaries, which deprive users of valuable intermediate information. To address this trade-off, we propose PART, an information-preserving antidistillation reformulation of reasoning traces. Motivated by the difference between how humans understand reasoning traces and how LLMs exploit them for supervised fine-tuning, we design a simple but effective two-step reformulation: removing self-talk behaviors and reordering sub-conclusions. A small auxiliary model is trained to perform this reformulation, incurring minimal computational overhead. Extensive experiments demonstrate that PART consistently disrupts distillation across student models of different sizes and types on various reasoning benchmarks. For instance, when training on reformulated traces, even the performance of a large 32B student model decreases from 54.17 to 46.88 on AIME 2024, corresponding to a 13.5% degradation.

Method: Proposes a critical framework connecting empirical measurements of language vitality (demographic strength) and digitality (online presence) with postcolonial theory and epistemic injustice, analyzing data across all documented human languages to categorize them into four groups.

Result: Identified four language categories: Strongholds (33%, high vitality/digitality), Digital Echoes (6%, high digitality despite declining vitality), Fading Voices (36%, low on both), and Invisible Giants (27%, high vitality but near-zero digitality). Shows patterns reflect continuities from colonial-era linguistic hierarchies to contemporary AI development.

Conclusion: English dominance in AI is not a technical necessity but an artifact of power structures that systematically exclude marginalized linguistic knowledge. Calls for decolonizing language technology and democratizing access to AI benefits to address digital epistemic injustice.

Abstract: Large Language Models are trained on massive multilingual corpora, yet this abundance masks a profound crisis: of the world’s 7,613 living languages, approximately 2,000 languages with millions of speakers remain effectively invisible in digital ecosystems. We propose a critical framework connecting empirical measurements of language vitality (real world demographic strength) and digitality (online presence) with postcolonial theory and epistemic injustice to explain why linguistic inequality in AI systems is not incidental but structural. Analyzing data across all documented human languages, we identify four categories: Strongholds (33%, high vitality and digitality), Digital Echoes (6%, high digitality despite declining vitality), Fading Voices (36%, low on both dimensions), and critically, Invisible Giants (27%, high vitality but near-zero digitality) - languages spoken by millions yet absent from the LLM universe. We demonstrate that these patterns reflect continuities from colonial-era linguistic hierarchies to contemporary AI development, constituting what we term digital epistemic injustice. Our analysis reveals that English dominance in AI is not a technical necessity but an artifact of power structures that systematically exclude marginalized linguistic knowledge. We conclude with implications for decolonizing language technology and democratizing access to AI benefits.

Method: Developed a framework grounded in sociocultural theory, constructed a benchmark dataset annotated by culturally diverse raters, and proposed new evaluation desiderata: conversational framing, stylistic sensitivity, and subjective correctness.

Result: Evaluation of top LLMs shows they struggle with cultural adaptation in conversational settings despite their capabilities.

Conclusion: The proposed framework and benchmark provide more realistic evaluation of LLMs’ cultural adaptation abilities, revealing significant gaps in current models’ performance.

Abstract: Existing benchmarks that measure cultural adaptation in LLMs are misaligned with the actual challenges these models face when interacting with users from diverse cultural backgrounds. In this work, we introduce the first framework and benchmark designed to evaluate LLMs in realistic, multicultural conversational settings. Grounded in sociocultural theory, our framework formalizes how linguistic style - a key element of cultural communication - is shaped by situational, relational, and cultural context. We construct a benchmark dataset based on this framework, annotated by culturally diverse raters, and propose a new set of desiderata for cross-cultural evaluation in NLP: conversational framing, stylistic sensitivity, and subjective correctness. We evaluate today’s top LLMs on our benchmark and show that these models struggle with cultural adaptation in a conversational setting.

Method: Uses structured prompting to formulate attacks as multiple-choice questions with KG factual evidence, implements semantics-based and centrality-based filters to compress candidate sets, precomputes high-order adjacency, and fine-tunes LLM with triple classification for better filtering.

Result: Outperforms strongest black-box baselines, provides explanations via reasoning, and shows competitive performance compared to white-box methods on two knowledge graph datasets.

Conclusion: LLMAtKGE effectively combines semantic and structural information for adversarial attacks while generating human-readable explanations, demonstrating the potential of LLMs in knowledge graph security applications.

Abstract: Adversarial attacks on knowledge graph embeddings (KGE) aim to disrupt the model’s ability of link prediction by removing or inserting triples. A recent black-box method has attempted to incorporate textual and structural information to enhance attack performance. However, it is unable to generate human-readable explanations, and exhibits poor generalizability. In the past few years, large language models (LLMs) have demonstrated powerful capabilities in text comprehension, generation, and reasoning. In this paper, we propose LLMAtKGE, a novel LLM-based framework that selects attack targets and generates human-readable explanations. To provide the LLM with sufficient factual context under limited input constraints, we design a structured prompting scheme that explicitly formulates the attack as multiple-choice questions while incorporating KG factual evidence. To address the context-window limitation and hesitation issues, we introduce semantics-based and centrality-based filters, which compress the candidate set while preserving high recall of attack-relevant information. Furthermore, to efficiently integrate both semantic and structural information into the filter, we precompute high-order adjacency and fine-tune the LLM with a triple classification task to enhance filtering performance. Experiments on two widely used knowledge graph datasets demonstrate that our attack outperforms the strongest black-box baselines and provides explanations via reasoning, and showing competitive performance compared with white-box methods. Comprehensive ablation and case studies further validate its capability to generate explanations.

Method: Generated 32 million simulated survey responses using 8 different Survey Response Generation Methods across 4 political attitude surveys and 10 open-weight language models, comparing individual-level and subpopulation-level alignment.

Result: Found significant differences between Survey Response Generation Methods, with Restricted Generation Methods performing best overall, and reasoning output not consistently improving alignment.

Conclusion: Survey Response Generation Methods significantly impact simulated survey responses, and Restricted Generation Methods are recommended as the most effective approach for generating closed-ended survey responses with LLMs.

Abstract: Many in-silico simulations of human survey responses with large language models (LLMs) focus on generating closed-ended survey responses, whereas LLMs are typically trained to generate open-ended text instead. Previous research has used a diverse range of methods for generating closed-ended survey responses with LLMs, and a standard practice remains to be identified. In this paper, we systematically investigate the impact that various Survey Response Generation Methods have on predicted survey responses. We present the results of 32 mio. simulated survey responses across 8 Survey Response Generation Methods, 4 political attitude surveys, and 10 open-weight language models. We find significant differences between the Survey Response Generation Methods in both individual-level and subpopulation-level alignment. Our results show that Restricted Generation Methods perform best overall, and that reasoning output does not consistently improve alignment. Our work underlines the significant impact that Survey Response Generation Methods have on simulated survey responses, and we develop practical recommendations on the application of Survey Response Generation Methods.

Method: Two-stage optimization: 1) Learn task-specific LoRA adapters using few samples from each dataset, 2) Update shared LoRA adapter by aggregating gradients from multiple tasks to promote knowledge transfer.

Result: Matches or surpasses performance of traditional full-data LoRA fine-tuning in multi-task and multilingual learning scenarios while using significantly less task-specific data.

Conclusion: MeTA-LoRA significantly improves data efficiency in multi-task adaptation of LLMs through its two-stage optimization approach.

Abstract: Low-Rank Adaptation (LoRA) has emerged as one of the most widely used parameter-efficient fine-tuning (PEFT) methods for adapting large language models (LLMs) to downstream tasks. While highly effective in single-task settings, it struggles to efficiently leverage inter-task knowledge in complex multi-task learning scenarios, often requiring substantial task-specific data to achieve optimal performance. To address this limitation, we introduce MeTA-LoRA, a two-stage optimization framework that significantly improves data efficiency in multi-task adaptation. In the first stage, task-specific LoRA adapters are learned using only a few samples from each involved dataset, enabling rapid adaptation without large-scale supervision. In the second stage, the shared LoRA adapter is updated by aggregating gradients from multiple tasks to promote knowledge transfer across tasks, further reducing data usage by leveraging common patterns. In both multi-task learning and multilingual learning scenarios, our method matches or surpasses the performance of traditional full-data LoRA fine-tuning approaches, while using significantly less task-specific data.

Method: Uses unsupervised procedure to generate aspect-specific summarizing sentences, trains embedding models to map related summaries close together, then distills these into a unified model that predicts multiple aspect embeddings in one forward pass.

Result: The framework successfully generates multifaceted embeddings and includes an embedding decoding pipeline that can decode embeddings back to natural language descriptions, even for unoccupied regions in visualizations.

Conclusion: SemCSE-Multi provides interpretable, multifaceted embeddings for scientific abstracts with improved user interpretability through effective decoding capabilities.

Abstract: We propose SemCSE-Multi, a novel unsupervised framework for generating multifaceted embeddings of scientific abstracts, evaluated in the domains of invasion biology and medicine. These embeddings capture distinct, individually specifiable aspects in isolation, thus enabling fine-grained and controllable similarity assessments as well as adaptive, user-driven visualizations of scientific domains. Our approach relies on an unsupervised procedure that produces aspect-specific summarizing sentences and trains embedding models to map semantically related summaries to nearby positions in the embedding space. We then distill these aspect-specific embedding capabilities into a unified embedding model that directly predicts multiple aspect embeddings from a scientific abstract in a single, efficient forward pass. In addition, we introduce an embedding decoding pipeline that decodes embeddings back into natural language descriptions of their associated aspects. Notably, we show that this decoding remains effective even for unoccupied regions in low-dimensional visualizations, thus offering vastly improved interpretability in user-centric settings.

Method: Designed controlled variants that selectively relax attention principles (token mixing, sequence-dependent activations, dot-product form, query-key coupling), applied uniformly across layers and in hybrid architectures with some standard attention layers.

Result: Token mixing mechanisms are indispensable (absence causes near-random behavior), while mathematical form and sequence dependency can be substantially relaxed, especially when preserved in subset of layers. Hybrid architectures show cooperative effects where failing variants work when interleaved with standard attention.

Conclusion: Attention’s effectiveness relies more on token mixing than specific mathematical form or sequence dependency, opening avenues for simplifying language models without performance loss.

Abstract: The success of Transformer language models is widely credited to their dot-product attention mechanism, which interweaves a set of key design principles: mixing information across positions (enabling multi-token interactions), sequence-dependent activations (where attention weights adapt to each input), a specific mathematical form (dot-product similarities plus softmax weighting), and coupling of queries and keys to evolving hidden states (grounding attention in the current layer). However, the necessity of each of these principles remains largely untested. In this work, we systematically deconstruct attention by designing controlled variants that selectively relax these principles, applied both uniformly across all layers and in hybrid architectures where only some layers retain standard attention. Our empirical analysis reveals that mechanisms for mixing tokens are indispensable, as their absence collapses models to near-random behavior, while the exact mathematical form and sequence dependency can be substantially relaxed, especially when preserved in just a subset of layers. Surprisingly, even variants that fail in isolation can achieve robust performance when interleaved with standard attention, highlighting a cooperative effect. These findings deepen our understanding of what truly underpins attention’s effectiveness and open new avenues for simplifying language models without sacrificing performance.

Method: AdaKD consists of two modules: Loss-Driven Adaptive Token Focusing (LATF) that dynamically adjusts distillation focus based on student’s learning stability, and Inverse Difficulty Temperature Scaling (IDTS) that uses low temperatures for difficult tokens and high temperatures for easy tokens.

Result: AdaKD consistently improves performance of various distillation methods across multiple model architectures and benchmarks as a plug-and-play framework.

Conclusion: The proposed adaptive approach to knowledge distillation better aligns with the dynamic learning process and significantly enhances knowledge transfer efficiency in language model compression.

Abstract: Knowledge distillation (KD) is a key technique for compressing large-scale language models (LLMs), yet prevailing logit-based methods typically employ static strategies that are misaligned with the dynamic learning process of student models. These methods typically treat all tokens indiscriminately and apply a single, fixed temperature, resulting in suboptimal knowledge transfer. To address these limitations, we propose LLM-Oriented Token-Adaptive Knowledge Distillation (AdaKD), a novel framework that adapts the distillation process to the real-time learning state of each token. AdaKD consists of two synergistic modules driven by a unified token difficulty metric. First, our Loss-Driven Adaptive Token Focusing (LATF) module dynamically adjusts the distillation focus by monitoring the student’s learning stability, concentrating computational resources on the most valuable tokens at each training phase. Second, we introduce Inverse Difficulty Temperature Scaling (IDTS), a counterintuitive yet effective token-level temperature strategy. It employs low temperatures for difficult tokens for targeted error correction, and high temperatures for easy tokens to encourage students to learn from the teacher’s complete and smooth output distribution, thereby enhancing generalization. As a plug-and-play framework, AdaKD can consistently improve the performance of various distillation methods on multiple model architectures and benchmarks.

Method: Proposes Multi-Path Plan Aggregation (MPPA) with variable interval plan generation and aggregation using a lightweight LoRA module, plus online Step-DPO with TSMC for stepwise supervision.

Result: Outperforms DeepSeek-R1 distillation and outcome-reward RL baselines on math, science, and logical reasoning benchmarks using only 10% SFT data and 5% preference pairs.

Conclusion: MPPA with Step-DPO effectively mitigates CoT derailment and enables efficient training for improved reasoning in small LMs.

Abstract: Inference-time scaling enhances the reasoning ability of a language model (LM) by extending its chain-of-thought (CoT). However, existing approaches typically generate the entire reasoning chain in a single forward pass, which often leads to CoT derailment, i.e., the reasoning trajectory drifting off course due to compounding errors. This problem is particularly severe for smaller LMs with long CoTs due to their limited capacity. To address this, we analyze raw long CoTs and uncover a reasoning hierarchy consisting of planning and execution steps. Our analysis reveals that most reasoning errors stem from incorrect planning. Motivated by this observation, we propose Multi-Path Plan Aggregation (MPPA), a framework that augments single-pass reasoning with plan exploration and aggregation. Following a variable interval schedule based on the token position, MPPA generates multiple candidate plans and aggregates them into a refined planning step. To maintain efficiency, we adopt a minimal design in which the base LM serves as the primary policy, while a lightweight LoRA module implements the plan aggregation policy. We further observe that outcome-reward RL is inefficient for long trajectories (e.g., exceeding 4K tokens). To overcome this, we introduce online Step-DPO, a process-level preference optimization scheme that leverages Twisted Sequential Monte Carlo (TSMC) to provide scalable stepwise supervision using small LMs. This yields more efficient training, improved stability, and higher accuracy. Extensive experiments on challenging math, science, and logical reasoning benchmarks demonstrate that, with only 10% SFT data and 5% of preference pairs, our method outperforms both the DeepSeek-R1 distillation baseline and the outcome-reward RL baseline across multiple base models and tasks.

Method: Created Acadreason benchmark with 50 expert-annotated academic problems across computer science, economics, law, mathematics, and philosophy, sourced from top-tier publications with rigorous quality control.

Result: Most LLMs scored below 20 points, with GPT-5 achieving only 16 points. Agents performed better but none exceeded 40 points, showing significant capability gaps.

Conclusion: Current LLMs and agents have substantial limitations in handling super-intelligent academic research tasks, highlighting the challenging nature of the Acadreason benchmark.

Abstract: In recent years, the research focus of large language models (LLMs) and agents has shifted increasingly from demonstrating novel capabilities to complex reasoning and tackling challenging tasks. However, existing evaluations focus mainly on math/code contests or general tasks, while existing multi-domain academic benchmarks lack sufficient reasoning depth, leaving the field without a rigorous benchmark for high-level reasoning. To fill this gap, we introduce the Acadreason benchmark, designed to evaluate the ability of LLMs and agents to acquire and reason over academic knowledge. It consists of 50 expert-annotated academic problems across five high-reasoning domains, including computer science, economics, law, mathematics, and philosophy. All questions are sourced from top-tier publications in recent years and undergo rigorous annotation and quality control to ensure they are both challenging and answerable. We conduct systematic evaluations of over 10 mainstream LLMs and agents. The results show that most LLMs scored below 20 points, with even the cutting-edge GPT-5 achieving only 16 points. While agents achieved higher scores, none exceeded 40 points. This demonstrates the current capability gap between LLMs and agents in super-intelligent academic research tasks and highlights the challenges of Acadreason.

Method: Proposed Language-Centric Omnimodal Embedding (LCO-Emb) framework that leverages implicit alignment from MLLM generative pretraining, with contrastive learning as refinement. Analyzed anisotropy and kernel similarity structure to confirm latent alignment.

Result: Achieved state-of-the-art performance across diverse benchmarks and modalities. Identified Generation-Representation Scaling Law (GRSL) showing representation quality scales with generative capabilities. Validated on challenging visual-document retrieval tasks.

Conclusion: Improving generative abilities is an effective paradigm for enhancing representation quality. Continual generative pretraining before contrastive learning can further boost embedding capabilities, with theoretical explanation linking generative quality to representation performance upper bound.

Abstract: Recent multimodal embedding approaches leveraging multimodal large language models (MLLMs) fine-tuned with contrastive learning (CL) have shown promising results, yet the underlying reasons behind their superiority remain underexplored. This work argues that a crucial advantage of MLLM-based approaches stems from implicit cross-modal alignment achieved during generative pretraining, where the language decoder learns to exploit multimodal signals within a shared representation space for generating unimodal outputs. Through analysis of anisotropy and kernel similarity structure, we empirically confirm that latent alignment emerges within MLLM representations, allowing CL to serve as a lightweight refinement stage. Leveraging this insight, we propose a Language-Centric Omnimodal Embedding framework, termed LCO-Emb. Extensive experiments across diverse backbones and benchmarks demonstrate its effectiveness, achieving state-of-the-art performance across modalities. Furthermore, we identify a Generation-Representation Scaling Law (GRSL), showing that the representational capabilities gained through contrastive refinement scales positively with the MLLM’s generative capabilities. This suggests that improving generative abilities evolves as an effective paradigm for enhancing representation quality. We provide a theoretical explanation of GRSL, which formally links the MLLM’s generative quality to the upper bound on its representation performance, and validate it on a challenging, low-resource visual-document retrieval task, showing that continual generative pretraining before CL can further enhance the potential of a model’s embedding capabilities. Codes, models, and resources are available at https://github.com/LCO-Embedding/LCO-Embedding.

Method: AMA integrates verified trading data, expert-checked news, and diverse agent architectures within a unified trading framework. It implements four agents with different risk styles and reasoning capabilities, evaluated across multiple LLM backends.

Result: Live experiments show agent frameworks display distinct behavioral patterns (from aggressive to conservative), while model backbones contribute less to outcome variation.

Conclusion: AMA establishes a foundation for rigorous, reproducible, and continuously evolving evaluation of financial reasoning and trading intelligence in LLM-based agents.

Abstract: Although Large Language Model (LLM)-based agents are increasingly used in financial trading, it remains unclear whether they can reason and adapt in live markets, as most studies test models instead of agents, cover limited periods and assets, and rely on unverified data. To address these gaps, we introduce Agent Market Arena (AMA), the first lifelong, real-time benchmark for evaluating LLM-based trading agents across multiple markets. AMA integrates verified trading data, expert-checked news, and diverse agent architectures within a unified trading framework, enabling fair and continuous comparison under real conditions. It implements four agents, including InvestorAgent as a single-agent baseline, TradeAgent and HedgeFundAgent with different risk styles, and DeepFundAgent with memory-based reasoning, and evaluates them across GPT-4o, GPT-4.1, Claude-3.5-haiku, Claude-sonnet-4, and Gemini-2.0-flash. Live experiments on both cryptocurrency and stock markets demonstrate that agent frameworks display markedly distinct behavioral patterns, spanning from aggressive risk-taking to conservative decision-making, whereas model backbones contribute less to outcome variation. AMA thus establishes a foundation for rigorous, reproducible, and continuously evolving evaluation of financial reasoning and trading intelligence in LLM-based agents.

Method: Conducted systematic investigation from three perspectives: data (using real end-to-end tool-use trajectories and high-diversity datasets), algorithm (exploration-friendly techniques like clip higher, reward shaping, and policy entropy), and reasoning mode (deliberative strategy with fewer tool calls).

Result: Achieved strong results on challenging benchmarks (AIME2024/AIME2025, GPQA-Diamond, LiveCodeBench-v6) with smaller models - 4B-sized models outperformed 32B-sized models in agentic reasoning performance.

Conclusion: Simple practices in data curation, algorithm design, and reasoning strategies consistently enhance agentic reasoning and training efficiency, establishing a practical baseline for future agentic RL research.

Abstract: Recently, the emergence of agentic RL has showcased that RL could also effectively improve the agentic reasoning ability of LLMs, yet the key design principles and optimal practices remain unclear. In this work, we conduct a comprehensive and systematic investigation to demystify reinforcement learning in agentic reasoning from three key perspectives: data, algorithm, and reasoning mode. We highlight our key insights: (i) Replacing stitched synthetic trajectories with real end-to-end tool-use trajectories yields a far stronger SFT initialization; high-diversity, model-aware datasets sustain exploration and markedly improve RL performance. (ii) Exploration-friendly techniques are crucial for agentic RL, such as clip higher, overlong reward shaping, and maintaining adequate policy entropy could improve the training efficiency. (iii) A deliberative strategy with fewer tool calls outperforms frequent tool calls or verbose self-reasoning, improving tool efficiency and final accuracy. Together, these simple practices consistently enhance agentic reasoning and training efficiency, achieving strong results on challenging benchmarks with smaller models, and establishing a practical baseline for future agentic RL research. Beyond these empirical insights, we further contribute a high-quality, real end-to-end agentic SFT dataset along with a high-quality RL dataset, and demonstrate the effectiveness of our insights in boosting the agentic reasoning ability of LLMs across four challenging benchmarks, including AIME2024/AIME2025, GPQA-Diamond, and LiveCodeBench-v6. With our recipes, 4B-sized models could also achieve superior agentic reasoning performance compared to 32B-sized models. Code and models: https://github.com/Gen-Verse/Open-AgentRL

Method: Evaluate LRM robustness under two dynamic scenarios: interruptions (testing partial outputs on limited budget) and dynamic context (testing adaptation to in-flight changes).

Result: Static evaluations overestimate robustness - state-of-the-art LRMs fail unpredictably when interrupted or exposed to changing context, with up to 60% performance drop when updates occur late in reasoning.

Conclusion: LRMs exhibit novel failure modes including reasoning leakage, panic under time pressure, and self-doubt when incorporating updated information, highlighting the need for dynamic evaluation frameworks.

Abstract: Large Reasoning Models (LRMs) excel at complex reasoning but are traditionally evaluated in static, “frozen world” settings: model responses are assumed to be instantaneous, and the context of a request is presumed to be immutable over the duration of the response. While generally true for short-term tasks, the “frozen world” assumption breaks down in modern reasoning tasks such as assistive programming, where models may take hours to think through problems and code may change dramatically from the time the model starts thinking to the model’s final output. In this work, we challenge the frozen world assumption and evaluate LRM robustness under two realistic dynamic scenarios: interruptions, which test the quality of the model’s partial outputs on a limited budget, and dynamic context, which tests model adaptation to in-flight changes. Across mathematics and programming benchmarks that require long-form reasoning, static evaluations consistently overestimate robustness: even state-of-the-art LRMs, which achieve high accuracy in static settings, can fail unpredictably when interrupted or exposed to changing context, with performance dropping by up to 60% when updates are introduced late in the reasoning process. Our analysis further reveals several novel failure modes, including reasoning leakage, where models fold the reasoning into their final answer when interrupted; panic, where under time pressure models abandon reasoning entirely and return incorrect answers; and self-doubt, where performance degrades while incorporating updated information.

Method: Uses Memory-smoothed Focal Loss and Entity-aware KNN to handle inaccurate annotation noise (entity ambiguity), and Boundary Mixup with noise-tolerant loss to handle incomplete annotation noise (decision boundary shifting).

Result: SANTA effectively mitigates both types of noise and achieves new state-of-the-art performance on five public datasets.

Conclusion: Separate tailored strategies for different types of annotation noise in distantly-supervised NER lead to superior performance compared to uniform approaches.

Abstract: Distantly-Supervised Named Entity Recognition effectively alleviates the burden of time-consuming and expensive annotation in the supervised setting. But the context-free matching process and the limited coverage of knowledge bases introduce inaccurate and incomplete annotation noise respectively. Previous studies either considered only incomplete annotation noise or indiscriminately handle two types of noise with the same strategy. In this paper, we argue that the different causes of two types of noise bring up the requirement of different strategies in model architecture. Therefore, we propose the SANTA to handle these two types of noise separately with (1) Memory-smoothed Focal Loss and Entity-aware KNN to relieve the entity ambiguity problem caused by inaccurate annotation, and (2) Boundary Mixup to alleviate decision boundary shifting problem caused by incomplete annotation and a noise-tolerant loss to improve the robustness. Benefiting from our separate tailored strategies, we confirm in the experiment that the two types of noise are well mitigated. SANTA also achieves a new state-of-the-art on five public datasets.

Method: Used cleaned Turkish Learner Corpus with syntactic features, comparing structural Part-of-Speech n-gram model with hybrid model retaining function words.

Result: Models achieved promising results in identifying native languages from L2 Turkish texts, with analysis of most predictive features revealing L1-specific transfer effects.

Conclusion: Successfully demonstrated NLI applicability to Turkish language, providing insights into L1 transfer patterns and making data/code available for future research.

Abstract: This paper presents the first application of Native Language Identification (NLI) for the Turkish language. NLI is the task of automatically identifying an individual’s native language (L1) based on their writing or speech in a non-native language (L2). While most NLI research has focused on L2 English, our study extends this scope to L2 Turkish by analyzing a corpus of texts written by native speakers of Albanian, Arabic and Persian. We leverage a cleaned version of the Turkish Learner Corpus and demonstrate the effectiveness of syntactic features, comparing a structural Part-of-Speech n-gram model to a hybrid model that retains function words. Our models achieve promising results, and we analyze the most predictive features to reveal L1-specific transfer effects. We make our data and code publicly available for further study.

Method: The paper presents a taxonomy of NLP in education, reviews task definitions and challenges, and discusses cutting-edge techniques including LLM-involved methods.

Result: The survey organizes relevant datasets and papers, showcases off-the-shelf demonstrations for educators/researchers, and provides a comprehensive Github repository.

Conclusion: Five promising future directions: generalization across subjects/languages, deployed LLM-based systems, adaptive learning, interpretability, and ethical considerations for NLP in education.

Abstract: Natural Language Processing (NLP) aims to analyze text or speech via techniques in the computer science field. It serves applications in the domains of healthcare, commerce, education, and so on. Particularly, NLP has been widely applied to the education domain and its applications have enormous potential to help teaching and learning. In this survey, we review recent advances in NLP with a focus on solving problems relevant to the education domain. In detail, we begin with introducing the related background and the real-world scenarios in education to which NLP techniques could contribute. Then, we present a taxonomy of NLP in the education domain and highlight typical NLP applications including question answering, question construction, automated assessment, and error correction. Next, we illustrate the task definition, challenges, and corresponding cutting-edge techniques based on the above taxonomy. In particular, LLM-involved methods are included for discussion due to the wide usage of LLMs in diverse NLP applications. After that, we showcase some off-the-shelf demonstrations in this domain, which are designed for educators or researchers. At last, we conclude with five promising directions for future research, including generalization over subjects and languages, deployed LLM-based systems for education, adaptive learning for teaching and learning, interpretability for education, and ethical consideration of NLP techniques. We organize all relevant datasets and papers in the open-available Github Link for better review https://github.com/LiXinyuan1015/NLP-for-Education.

Method: Evaluated 25 models on six practical medical tasks, comparing performance before and after fine-tuning, with focus on hallucinations, ICD10 coding, and instruction adherence.

Result: Nine out of twelve biomedical models showed performance decline after fine-tuning. General-domain models like Meta-Llama-3.1-70B-Instruct outperformed biomedical counterparts.

Conclusion: There is a trade-off between domain-specific fine-tuning and general medical task performance, suggesting that biomedical training may not always improve performance on practical medical tasks.

Abstract: Large Language Models (LLMs) are expected to significantly contribute to patient care, diagnostics, and administrative processes. Emerging biomedical LLMs aim to address healthcare-specific challenges, including privacy demands and computational constraints. Assessing the models’ suitability for this sensitive application area is of the utmost importance. However, biomedical training has not been systematically evaluated on medical tasks. This study investigates the effect of biomedical training in the context of six practical medical tasks evaluating $25$ models. In contrast to previous evaluations, our results reveal a performance decline in nine out of twelve biomedical models after fine-tuning, particularly on tasks involving hallucinations, ICD10 coding, and instruction adherence. General-domain models like Meta-Llama-3.1-70B-Instruct outperformed their biomedical counterparts, indicating a trade-off between domain-specific fine-tuning and general medical task performance. We open-source all evaluation scripts and datasets at https://github.com/TIO-IKIM/CLUE to support further research in this critical area.

Method: Introduces information bottleneck theory to model context compression and proposes a cross-attention-based approach to approximate mutual information, which can be flexibly adapted for different scenarios.

Result: Extensive experiments on four datasets show the method achieves 25% higher compression rate than state-of-the-art while maintaining question answering performance. In some cases, compressed context even outperforms full context.

Conclusion: Information bottleneck theory provides a novel and effective framework for context compression in LLMs, addressing key limitations of existing methods and enabling more efficient processing of long contexts.

Abstract: Generative LLM have achieved remarkable success in various industrial applications, owing to their promising In-Context Learning capabilities. However, the issue of long context in complex tasks poses a significant barrier to their wider adoption, manifested in two main aspects: (i) The excessively long context leads to high costs and inference delays. (ii) A substantial amount of task-irrelevant information introduced by long contexts exacerbates the “lost in the middle” problem. Existing methods compress context by removing redundant tokens using metrics such as self-information or PPL, which is inconsistent with the objective of retaining the most important tokens when conditioning on a given query. In this study, we introduce information bottleneck theory (IB) to model the problem, offering a novel perspective that thoroughly addresses the essential properties required for context compression. Additionally, we propose a cross-attention-based approach to approximate mutual information in IB, which can be flexibly replaced with suitable alternatives in different scenarios. Extensive experiments on four datasets demonstrate that our method achieves a 25% increase in compression rate compared to the state-of-the-art, while maintaining question answering performance. In particular, the context compressed by our method even outperform the full context in some cases.

Method: Inject explicit markers (mentioning linguistic identity) and implicit markers (dialectal features) into inputs, then analyze how frequently model outputs flip across 3 LLMs, 1 LM, and 5 linguistic identities.

Result: Implicit dialect markers cause more output flips than explicit markers, flip percentages vary across ethnicities, and larger models show more robustness.

Conclusion: Caution is needed when deploying LLMs for high-stakes hate speech detection due to demonstrated brittleness and bias issues.

Abstract: Large Language Models (LLMs) offer a lucrative promise for scalable content moderation, including hate speech detection. However, they are also known to be brittle and biased against marginalised communities and dialects. This requires their applications to high-stakes tasks like hate speech detection to be critically scrutinized. In this work, we investigate the robustness of hate speech classification using LLMs particularly when explicit and implicit markers of the speaker’s ethnicity are injected into the input. For explicit markers, we inject a phrase that mentions the speaker’s linguistic identity. For the implicit markers, we inject dialectal features. By analysing how frequently model outputs flip in the presence of these markers, we reveal varying degrees of brittleness across 3 LLMs and 1 LM and 5 linguistic identities. We find that the presence of implicit dialect markers in inputs causes model outputs to flip more than the presence of explicit markers. Further, the percentage of flips varies across ethnicities. Finally, we find that larger models are more robust. Our findings indicate the need for exercising caution in deploying LLMs for high-stakes tasks like hate speech detection.

Method: Systematic literature review of 175 research papers focusing on textual credibility signals in NLP, examining automatic credibility assessment methods and detection of nine categories of credibility signals, with in-depth analysis of three key categories.

Result: The survey provides an in-depth analysis of credibility signal categories including factuality, subjectivity and bias; persuasion techniques and logical fallacies; and check-worthy and fact-checked claims, along with summaries of existing methods, datasets, and tools.

Conclusion: The paper outlines future research directions and emerging opportunities in credibility assessment, with particular attention to evolving challenges posed by generative AI, positioning NLP research within the broader multidisciplinary landscape.

Abstract: In the age of social media and generative AI, the ability to automatically assess the credibility of online content has become increasingly critical, complementing traditional approaches to false information detection. Credibility assessment relies on aggregating diverse credibility signals - small units of information, such as content subjectivity, bias, or a presence of persuasion techniques - into a final credibility label/score. However, current research in automatic credibility assessment and credibility signals detection remains highly fragmented, with many signals studied in isolation and lacking integration. Notably, there is a scarcity of approaches that detect and aggregate multiple credibility signals simultaneously. These challenges are further exacerbated by the absence of a comprehensive and up-to-date overview of research works that connects these research efforts under a common framework and identifies shared trends, challenges, and open problems. In this survey, we address this gap by presenting a systematic and comprehensive literature review of 175 research papers, focusing on textual credibility signals within the field of Natural Language Processing (NLP), which undergoes a rapid transformation due to advancements in Large Language Models (LLMs). While positioning the NLP research into the the broader multidisciplinary landscape, we examine both automatic credibility assessment methods as well as the detection of nine categories of credibility signals. We provide an in-depth analysis of three key categories: 1) factuality, subjectivity and bias, 2) persuasion techniques and logical fallacies, and 3) check-worthy and fact-checked claims. In addition to summarising existing methods, datasets, and tools, we outline future research direction and emerging opportunities, with particular attention to evolving challenges posed by generative AI.

Method: Proposes SEKE - a Mixture of Specialised Experts framework using DeBERTa as backbone, integrated with BiLSTM network, where experts specialize in distinct regions of input space via learnable routing sub-network.

Result: Achieves state-of-the-art performance on multiple English datasets compared to strong supervised and unsupervised baselines. Experts specialize in distinct syntactic and semantic components like punctuation, stopwords, parts-of-speech, or named entities.

Conclusion: SEKE provides effective keyword extraction with enhanced explainability through MoE framework, successfully handling diverse content even on smaller corpora.

Abstract: Keyword extraction involves identifying the most descriptive words in a document, allowing automatic categorisation and summarisation of large quantities of diverse textual data. Relying on the insight that real-world keyword detection often requires handling of diverse content, we propose a novel supervised keyword extraction approach based on the mixture of experts (MoE) technique. MoE uses a learnable routing sub-network to direct information to specialised experts, allowing them to specialise in distinct regions of the input space. SEKE, a mixture of Specialised Experts for supervised Keyword Extraction, uses DeBERTa as the backbone model and builds on the MoE framework, where experts attend to each token, by integrating it with a bidirectional Long short-term memory (BiLSTM) network, to allow successful extraction even on smaller corpora, where specialisation is harder due to lack of training data. The MoE framework also provides an insight into inner workings of individual experts, enhancing the explainability of the approach. We benchmark SEKE on multiple English datasets, achieving state-of-the-art performance compared to strong supervised and unsupervised baselines. Our analysis reveals that depending on data size and type, experts specialise in distinct syntactic and semantic components, such as punctuation, stopwords, parts-of-speech, or named entities. Code is available at https://github.com/matejMartinc/SEKE_keyword_extraction

Method: Two-step framework: (1) SubData library for standardizing heterogeneous datasets, (2) theory-driven approach to test differently-aligned LLMs on content classification.

Result: SubData enables flexible mapping and customization for diverse research needs, distinguishing it from existing resources.

Conclusion: Invite contributions to extend SubData into a multi-construct benchmark suite for evaluating LLM perspective alignment on NLP tasks.

Abstract: As increasingly capable large language models (LLMs) emerge, researchers have begun exploring their potential for subjective tasks. While recent work demonstrates that LLMs can be aligned with diverse human perspectives, evaluating this alignment on downstream tasks (e.g., hate speech detection) remains challenging due to the use of inconsistent datasets across studies. To address this issue, in this resource paper we propose a two-step framework: we (1) introduce SubData, an open-source Python library designed for standardizing heterogeneous datasets to evaluate LLMs perspective alignment; and (2) present a theory-driven approach leveraging this library to test how differently-aligned LLMs (e.g., aligned with different political viewpoints) classify content targeting specific demographics. SubData’s flexible mapping and taxonomy enable customization for diverse research needs, distinguishing it from existing resources. We illustrate its usage with an example application and invite contributions to extend our initial release into a multi-construct benchmark suite for evaluating LLMs perspective alignment on natural language processing tasks.

Method: Proposes Boundary Layer Update (BLUE) strategy to enhance locate-then-edit methods by addressing the weight shift errors caused by residual distribution.

Result: BLUE delivers 35.59% average performance improvement in sequential batch editing experiments on three LLMs and two datasets, significantly advancing state of the art in model editing while preserving LLMs’ general capabilities.

Conclusion: BLUE effectively addresses the weight shift error problem in locate-then-edit model editing methods and provides substantial performance improvements while maintaining model generalization capabilities.

Abstract: Model editing enables targeted updates to the knowledge of large language models (LLMs) with minimal retraining. Among existing approaches, locate-then-edit methods constitute a prominent paradigm: they first identify critical layers, then compute residuals at the final critical layer based on the target edit, and finally apply least-squares-based multi-layer updates via $\textbf{residual distribution}$. While empirically effective, we identify a counterintuitive failure mode: residual distribution, a core mechanism in these methods, introduces weight shift errors that undermine editing precision. Through theoretical and empirical analysis, we show that such errors increase with the distribution distance, batch size, and edit sequence length, ultimately leading to inaccurate or suboptimal edits. To address this, we propose the $\textbf{B}$oundary $\textbf{L}$ayer $\textbf{U}$pdat$\textbf{E (BLUE)}$ strategy to enhance locate-then-edit methods. Sequential batch editing experiments on three LLMs and two datasets demonstrate that BLUE not only delivers an average performance improvement of 35.59%, significantly advancing the state of the art in model editing, but also enhances the preservation of LLMs’ general capabilities. Our code is available at https://github.com/xpq-tech/BLUE.

Method: Proposes SMI (Size-dependent Mutual Information) that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book QA accuracy. Validated through large-scale document retrieval over 21 public and 3 custom models’ pre-training corpora with multi-template QA evaluation.

Result: SMI significantly outperforms repetition-based baselines and achieves R² > 0.7 in predicting QA accuracy for models above 1B parameters, without additional training. Analysis reveals diminishing returns from scaling data and model size.

Conclusion: There is an intrinsic upper bound on knowledge retention achievable by pre-training alone, motivating the need for retrieval and other augmentation strategies to overcome this limitation.

Abstract: The GPT-4 technical report suggests that downstream performance can be predicted from pre-training signals, but offers little methodological detail on how to quantify this. This work address this gap by modeling knowledge retention, the capacity of a pre-trained language model to memorize factual information from its corpus, and introduce a principled method to estimate it prior to training. We propose Size-dependent Mutual Information (SMI), an information-theoretic predictor that integrates knowledge frequency, knowledge specificity, and model size to forecast closed-book question answering (QA) accuracy. SMI is validated through large-scale document retrieval over the disclosed pre-training corpora of 21 public and 3 custom models, combined with a robust multi-template QA evaluation. Experiments show that SMI significantly outperforms repetition-based baselines and achieves $R^2$ > 0.7 in predicting QA accuracy for models above 1B parameters, without additional training. The analysis further reveals diminishing returns from scaling data and model size and provides evidence for an intrinsic upper bound on knowledge retention achievable by pre-training alone, motivating retrieval and other augmentation strategies.

Method: Two-stage RL framework: (1) Decider RL-agent dynamically selects small ensembles from N LLMs by maximizing error-diversity and reasoning performance using task-adaptive rewards and policy; (2) Fusion RL-agent resolves reasoning conflicts and adapts to different ensemble teams; (3) Introduces focal diversity metric to model error correlations and prune ensemble combinations.

Result: Extensive evaluations on five benchmarks show RL-Focal achieves 8.48% performance improvement with small ensembles compared to the best individual LLM, while offering stronger robustness.

Conclusion: RL-Focal effectively promotes reward-aware and policy-adaptive ensemble selection and inference fusion, demonstrating significant performance gains and robustness across multiple tasks.

Abstract: The advancement of LLMs and their accessibility have triggered renewed interest in multi-agent reinforcement learning as robust and adaptive frameworks for dynamically changing environments. This paper introduces RL-Focal, a two-stage RL agent framework that routes and ensembles LLMs. First, we develop the Decider RL-agent, which learns to dynamically select an ensemble of small size ($m_i$) among $N$ LLMs ($m_i \ll N$) for incoming queries from a user-defined downstream task $i$, by maximizing both error-diversity and reasoning-performance of the selected ensemble through iterative updates of task-adaptive rewards and policy. Second, to enable effective fusion of dynamically selected LLMs, we develop the stage-2 Fusion RL-agent, which learns to resolve reasoning conflicts from different LLMs and dynamically adapts to different ensemble teams composed by the Decider Agent for different downstream tasks. Third, we introduce the focal diversity metric to better model the error correlations among multiple LLMs, further improving the generalization performance of the Decider Agent, which actively prunes the ensemble combinations. By focal diversity, we enhance performance across tasks by effectively promoting reward-aware and policy-adaptive ensemble selection and inference fusion. Extensive evaluations on five benchmarks show that RL-Focal achieves the performance improvement of 8.48% with an ensemble of small size compared to the best individual LLM in a pool and offers stronger robustness. Code is available at https://github.com/sftekin/rl-focal

Method: Introduces Reference-Level Feedback paradigm that extracts desirable characteristics from carefully curated reference samples to guide synthesis of higher-quality instruction-response pairs, creating the REFED dataset of 10K pairs.

Result: Fine-tuning Llama-3.1-8B-Instruct and Mistral-7B-Instruct on REFED achieves state-of-the-art performance among similarly sized models, with 43.96% length-controlled win-rate on AlpacaEval 2.0, outperforming traditional sample-level feedback methods.

Conclusion: Reference-Level Feedback consistently outperforms traditional methods, generalizes across model architectures, and produces high-quality diverse data at low cost, providing an effective solution to synthetic data quality limitations.

Abstract: High-quality instruction-tuning data is crucial for developing Large Language Models (LLMs) that can effectively navigate real-world tasks and follow human instructions. While synthetic data generation offers a scalable approach for creating such datasets, it imposes a quality ceiling where models trained on the data cannot outperform the LLM generating it. To overcome this limitation, we introduce Reference-Level Feedback, a paradigm that extracts desirable characteristics from carefully curated reference samples to guide the synthesis of higher-quality instruction-response pairs. Using this approach, we synthesize REFED, a dataset of 10K instruction-response pairs. Fine-tuning Llama-3.1-8B-Instruct and Mistral-7B-Instruct on REFED demonstrate state-of-the-art performance among similarly sized models, notably reaching a 43.96% length-controlled win-rate on AlpacaEval 2.0. Extensive experiments demonstrate that Reference-Level Feedback consistently outperforms traditional sample-level feedback methods, generalizes across model architectures, and produces high-quality and diverse data at low cost.

Method: Proposes EVOLVE framework with synergistic optimization of training and inference stages: explores optimization methods during training to activate self-refinement, and investigates generation strategies at inference to enhance and utilize self-refinement while collecting training data.

Result: Evolved self-refinement enables Llama-3.1-8B to surpass GPT-4o with 62.3% length-controlled and 63.3% raw win rates on AlpacaEval 2, 50.3% on Arena-Hard, and improves performance on mathematical reasoning benchmarks like GSM8K and MATH.

Conclusion: Self-refinement can be systematically developed through iterative training and serves as a fundamental mechanism for broader self-improvement of intrinsic model abilities, with effective generalization to out-of-domain reasoning tasks.

Abstract: Self-Refinement refers to a model’s ability to revise its own responses to produce improved outputs. This capability can also serve as a fundamental mechanism for Self-Improvement, for example, by reconstructing datasets with refined results to enhance intrinsic model performance. However, our comprehensive experiments reveal that large language models (LLMs) show no clear evidence of inherent Self-Refinement and may even experience response quality degradation after Self-Refinement. To address this issue, we propose EVOLVE, a simple and effective framework for eliciting and tracking the evolution of Self-Refinement through iterative training. We first explore optimization methods during training to activate the model’s Self-Refinement capability. Then, at inference, we investigate various generation strategies to further enhance and utilize Self-Refinement while supplying the necessary data for training. Through synergistic optimization of training and inference stages, we continually evolve the model’s Self-Refinement ability, enabling it to better refine its own responses. Moreover, we demonstrate the potential of leveraging Self-Refinement to achieve broader Self-Improvement of intrinsic model abilities. Experiments show that the evolved Self-Refinement ability enables the Llama-3.1-8B base model to surpass GPT-4o, achieving 62.3% length-controlled and 63.3% raw win rates on AlpacaEval 2, and 50.3% on Arena-Hard. It also generalizes effectively to out-of-domain reasoning tasks, improving performance on mathematical reasoning benchmarks such as GSM8K and MATH.

Method: Analyzed 1.4M comments from 3,161 YouTube news videos; developed fine-grained hope speech classifier; conducted annotation study with 3,750 instances using diverse political representation; tested zero-shot LLMs.

Result: Strong association between political beliefs and content rating; models trained on individual political beliefs show significant disagreement; LLMs align more with liberal raters.

Conclusion: Political bias significantly affects how content about marginalized communities is rated, highlighting challenges in developing fair AI systems for content moderation.

Abstract: This paper makes three contributions. First, via a substantial corpus of 1,419,047 comments posted on 3,161 YouTube news videos of major US cable news outlets, we analyze how users engage with LGBTQ+ news content. Our analyses focus both on positive and negative content. In particular, we construct a fine-grained hope speech classifier that detects positive (hope speech), negative, neutral, and irrelevant content. Second, in consultation with a public health expert specializing on LGBTQ+ health, we conduct an annotation study with a balanced and diverse political representation and release a dataset of 3,750 instances with fine-grained labels and detailed annotator demographic information. Finally, beyond providing a vital resource for the LGBTQ+ community, our annotation study and subsequent in-the-wild assessments reveal (1) strong association between rater political beliefs and how they rate content relevant to a marginalized community; (2) models trained on individual political beliefs exhibit considerable in-the-wild disagreement; and (3) zero-shot large language models (LLMs) align more with liberal raters.

Method: PALETTE introduces adjustment queries where self-referential statements grounded in psychological constructs are treated like factual knowledge, enabling direct editing of personality-related responses without extensive fine-tuning.

Result: The method achieves substantial improvements in personality alignment across dimensions using only 12 editing samples, with experimental results showing more stable and well-balanced personality control in both automatic and human evaluations.

Conclusion: PALETTE provides an effective and efficient approach for personality editing in LLMs that overcomes the limitations of existing methods by requiring minimal data while achieving robust personality control.

Abstract: Large Language Models (LLMs) are integral to applications such as conversational agents and content creation, where precise control over a model’s personality is essential for maintaining tone, consistency, and user engagement. However, prevailing prompt-based or fine-tuning approaches either lack robustness or demand large-scale training data, making them costly and impractical. In this paper, we present PALETTE (Personality Adjustment by LLM SElf-TargeTed quEries), a novel method for personality editing in LLMs. Our approach introduces adjustment queries, where self-referential statements grounded in psychological constructs are treated analogously to factual knowledge, enabling direct editing of personality-related responses. Unlike fine-tuning, PALETTE requires only 12 editing samples to achieve substantial improvements in personality alignment across personality dimensions. Experimental results from both automatic and human evaluations demonstrate that our method enables more stable and well-balanced personality control in LLMs.

Method: LIDDIA leverages large language models’ reasoning capabilities to create an autonomous agent that intelligently balances exploration and exploitation in chemical space during drug discovery.

Result: LIDDIA generated molecules meeting key pharmaceutical criteria for over 70% of 30 clinically relevant targets, and identified a promising novel candidate for AR/NR3C4 (a critical target for prostate and breast cancers).

Conclusion: LIDDIA serves as a low-cost, highly-adaptable tool for autonomous drug discovery, demonstrating effective navigation of the drug discovery process and identification of novel therapeutic candidates.

Abstract: Drug discovery is a long, expensive, and complex process, relying heavily on human medicinal chemists, who can spend years searching the vast space of potential therapies. Recent advances in artificial intelligence for chemistry have sought to expedite individual drug discovery tasks; however, there remains a critical need for an intelligent agent that can navigate the drug discovery process. Towards this end, we introduce LIDDIA, an autonomous agent capable of intelligently navigating the drug discovery process in silico. By leveraging the reasoning capabilities of large language models, LIDDIA serves as a low-cost and highly-adaptable tool for autonomous drug discovery. We comprehensively examine LIDDIA , demonstrating that (1) it can generate molecules meeting key pharmaceutical criteria on over 70% of 30 clinically relevant targets, (2) it intelligently balances exploration and exploitation in the chemical space, and (3) it identifies one promising novel candidate on AR/NR3C4, a critical target for both prostate and breast cancers. Code and dataset are available at https://github.com/ninglab/LIDDiA

Method: Clustering-On-Difficulty (COD) framework that clusters tasks by difficulty scaling features, excludes non-emergent/irregular tasks, uses performance scaling laws for cluster prediction, and maps subset to full set performance.

Result: Applied to 70B parameter LLM, achieved 1.36% average prediction error across eight key LLM benchmarks.

Conclusion: COD provides accurate downstream performance prediction for efficient resource allocation and training monitoring in LLM pre-training.

Abstract: The escalating scale and cost of Large Language Models (LLMs) training necessitate accurate pre-training prediction of downstream task performance for efficient resource allocation. This is challenged by: 1) the emergence phenomenon, where metrics become meaningful only after extensive training, hindering prediction by smaller models; and 2) uneven task difficulty and inconsistent performance scaling patterns, leading to high metric variability. Current prediction methods lack accuracy and reliability. We propose a Clustering-On-Difficulty (COD) framework for downstream performance prediction. The COD framework clusters tasks by their difficulty scaling features, thereby establishing a more stable and predictable support subset through the exclusion of tasks exhibiting non-emergent behavior or irregular scaling. We adopt a performance scaling law to predict cluster-wise performance with theoretical support. Predictable subset performance acts as an intermediate predictor for the full evaluation set. We further derive a mapping function to accurately extrapolate the performance of the subset to the full set. Applied to an LLM with 70B parameters, COD achieved a 1.36% average prediction error across eight key LLM benchmarks, offering actionable insights for resource allocation and training monitoring of LLMs pretraining.

Method: Thinking-Optimal Scaling strategy: uses seed data with varying response length distributions to teach models different reasoning efforts, then selects shortest correct responses under different reasoning efforts for self-improvement.

Result: Self-improved models built on Qwen2.5-32B-Instruct outperform other distillation-based 32B o1-like models across math benchmarks and achieve performance on par with teacher model QwQ-32B-Preview.

Conclusion: There exists an optimal scaled length distribution that differs across domains, and scaling with longer CoTs can impair reasoning performance in certain domains, requiring domain-specific optimization strategies.

Abstract: Recent studies have shown that making a model spend more time thinking through longer Chain of Thoughts (CoTs) enables it to gain significant improvements in complex reasoning tasks. While current researches continue to explore the benefits of increasing test-time compute by extending the CoT lengths of Large Language Models (LLMs), we are concerned about a potential issue hidden behind the current pursuit of test-time scaling: Would excessively scaling the CoT length actually bring adverse effects to a model’s reasoning performance? Our explorations on mathematical reasoning tasks reveal an unexpected finding that scaling with longer CoTs can indeed impair the reasoning performance of LLMs in certain domains. Moreover, we discover that there exists an optimal scaled length distribution that differs across different domains. Based on these insights, we propose a Thinking-Optimal Scaling strategy. Our method first uses a small set of seed data with varying response length distributions to teach the model to adopt different reasoning efforts for deep thinking. Then, the model selects its shortest correct response under different reasoning efforts on additional problems for self-improvement. Our self-improved models built upon Qwen2.5-32B-Instruct outperform other distillation-based 32B o1-like models across various math benchmarks, and achieve performance on par with the teacher model QwQ-32B-Preview that produces the seed data.

Method: Created MathTutorBench with datasets and metrics covering tutor abilities based on learning sciences research. Trained a reward model to score pedagogical quality of teacher responses, and evaluated various closed- and open-weight models.

Result: Subject expertise (solving ability) doesn’t immediately translate to good teaching - pedagogy and subject expertise form a trade-off navigated by tutoring specialization. Tutoring becomes more challenging in longer dialogs where simple questioning strategies fail.

Conclusion: MathTutorBench enables rapid benchmarking of future tutoring models, revealing important insights about the relationship between subject expertise and pedagogical effectiveness in AI tutors.

Abstract: Evaluating the pedagogical capabilities of AI-based tutoring models is critical for making guided progress in the field. Yet, we lack a reliable, easy-to-use, and simple-to-run evaluation that reflects the pedagogical abilities of models. To fill this gap, we present MathTutorBench, an open-source benchmark for holistic tutoring model evaluation. MathTutorBench contains a collection of datasets and metrics that broadly cover tutor abilities as defined by learning sciences research in dialog-based teaching. To score the pedagogical quality of open-ended teacher responses, we train a reward model and show it can discriminate expert from novice teacher responses with high accuracy. We evaluate a wide set of closed- and open-weight models on MathTutorBench and find that subject expertise, indicated by solving ability, does not immediately translate to good teaching. Rather, pedagogy and subject expertise appear to form a trade-off that is navigated by the degree of tutoring specialization of the model. Furthermore, tutoring appears to become more challenging in longer dialogs, where simpler questioning strategies begin to fail. We release the benchmark, code, and leaderboard openly to enable rapid benchmarking of future models.

Method: Propose PKUE (Precise Knowledge Utilization Enhancement) - fine-tunes LLMs on self-generated responses to precise factual questions using preference optimization. Also construct FactualBench dataset with 181k Chinese data across 21 domains.

Result: Extensive experiments show PKUE significantly improves LLM overall performance with consistent enhancement across factual tasks of various forms, general tasks beyond factuality, and tasks in different languages.

Conclusion: PKUE effectively addresses factual hallucinations by enhancing LLMs’ fundamental ability to precisely leverage knowledge, demonstrating broad improvements across multiple task types and languages.

Abstract: Large Language Models (LLMs) often struggle to align their responses with objective facts, resulting in the issue of factual hallucinations, which can be difficult to detect and mislead users without relevant knowledge. Although post-training techniques have been employed to mitigate the issue, existing methods usually suffer from poor generalization and trade-offs in other different capabilities. In this paper, we propose to address these by directly augmenting LLM’s fundamental ability to precisely leverage its knowledge and introduce PKUE (Precise Knowledge Utilization Enhancement), which fine-tunes the model on self-generated responses to precise and simple factual questions through preference optimization. Furthermore, we construct FactualBench, a comprehensive and precise factual QA dataset containing 181k Chinese data spanning 21 domains, to facilitate both evaluation and training. Extensive experiments demonstrate that PKUE significantly improves LLM overall performance, with consistent enhancement across factual tasks of various forms, general tasks beyond factuality, and tasks in different language.

Method: The authors analyze feature learning dynamics using a simplified setting with normalized ReLU self-attention layers and structured data that contains disentangled two-type features, examining how these features are learned sequentially during training.

Result: The paper provides the first rigorous theoretical demonstration of feature-level two-stage optimization in transformers, showing how models progress from learning one feature type (e.g., syntax) to another (e.g., semantics), with this process being related to spectral properties of attention weights.

Conclusion: Transformers naturally exhibit two-stage training dynamics when dealing with disentangled feature types, and this sequential learning process is mathematically grounded in the spectral properties of attention mechanisms, providing theoretical explanation for observed empirical phenomena.

Abstract: Transformers may exhibit two-stage training dynamics during the real-world training process. For instance, when training GPT-2 on the Counterfact dataset, the answers progress from syntactically incorrect to syntactically correct to semantically correct. However, existing theoretical analyses hardly account for this feature-level two-stage phenomenon, which originates from the disentangled two-type features like syntax and semantics. In this paper, we theoretically demonstrate how the two-stage training dynamics potentially occur in transformers. Specifically, we analyze the feature learning dynamics induced by the aforementioned disentangled two-type feature structure, grounding our analysis in a simplified yet illustrative setting that comprises a normalized ReLU self-attention layer and structured data. Such disentanglement of feature structure is general in practice, e.g., natural languages contain syntax and semantics, and proteins contain primary and secondary structures. To our best knowledge, this is the first rigorous result regarding a feature-level two-stage optimization process in transformers. Additionally, a corollary indicates that such a two-stage process is closely related to the spectral properties of the attention weights, which accords well with our empirical findings.

Method: Textual Model Predictive Control (TMPC) adapts MPC for text generation with two key principles: (1) Hindsight Subgoal Identification to discover meaningful intermediate outputs, and (2) Subgoal-Conditioned Re-Generation to guide subsequent planning iterations.

Result: TMPC consistently improves performance across three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis.

Conclusion: TMPC provides a general framework for test-time LLM alignment that effectively balances planning granularity through hierarchical subgoal discovery and regeneration.

Abstract: Aligning Large Language Models (LLMs) with human preferences through finetuning is resource-intensive, motivating lightweight alternatives at test time. We address test-time alignment through the lens of sequential decision making, a perspective that reveals two fundamental challenges. When actions are defined at the token level, as in guided decoding, alignment suffers from the curse of horizon. Conversely, when actions are at the response level, as in traditional iterative refinement, the curse of dimensionality emerges. To resolve this trade-off, we draw inspiration from Model Predictive Control (MPC) in control theory to propose Textual Model Predictive Control (TMPC), a novel predictive planning framework adapted for aligning LLMs at inference time. A key limitation of standard MPC is its reliance on predefined, hard segment boundaries, which are often absent in text generation. TMPC overcomes this by introducing two principles inspired by hierarchical reinforcement learning: (1) Hindsight Subgoal Identification, where TMPC analyzes generation subgoals to retrospectively identify high-reward intermediate outputs as subgoals. This allows the framework to discover meaningful, task-specific planning steps (e.g., a sentence in machine translation or a bug fix in code generation.). (2) Subgoal-Conditioned Re-Generation, where these identified subgoals are used to guide subsequent planning iterations. By conditioning on these proven, high-quality subgoals, TMPC ensures stable improvement by building upon previously validated successes. TMPC is evaluated on three tasks with distinct segmentation properties: discourse-level translation, long-form response generation, and program synthesis. The results demonstrate that TMPC consistently improves performance, highlighting the generality.

Method: Utilizes various linguistic representations from different layers of pre-trained transformer-based models for authorship attribution.

Result: Outperforms state-of-the-art baseline in both in-domain and out-of-domain scenarios, with improved robustness on out-of-domain data.

Conclusion: Using multiple transformer layers enhances model robustness and provides insights into layer specialization for stylistic feature representation in out-of-domain scenarios.

Abstract: We propose a new approach for the authorship attribution task that leverages the various linguistic representations learned at different layers of pre-trained transformer-based models. We evaluate our approach on three datasets, comparing it to a state-of-the-art baseline in in-domain and out-of-domain scenarios. We found that utilizing various transformer layers improves the robustness of authorship attribution models when tested on out-of-domain data, resulting in new state-of-the-art results. Our analysis gives further insights into how our model’s different layers get specialized in representing certain stylistic features that benefit the model when tested out of the domain.

Method: Using mechanistic interpretability to analyze model internal states and attention distributions, then proposing ADAPTVIS - an inference-time method that sharpens attention on relevant regions when confident, and broadens attention when confidence is low.

Result: Significant improvements on spatial reasoning benchmarks (up to 50 absolute point improvement on WhatsUp and VSR) with negligible computational cost.

Conclusion: ADAPTVIS effectively enhances spatial reasoning in VLMs by dynamically adjusting attention distribution based on confidence, demonstrating that attention alignment with object locations is crucial for spatial reasoning success.

Abstract: Large Vision Language Models (VLMs) have long struggled with spatial reasoning tasks. Surprisingly, even simple spatial reasoning tasks, such as recognizing “under” or “behind” relationships between only two objects, pose significant challenges for current VLMs. In this work, we study the spatial reasoning challenge from the lens of mechanistic interpretability, diving into the model’s internal states to examine the interactions between image and text tokens. By tracing attention distribution over the image through out intermediate layers, we observe that successful spatial reasoning correlates strongly with the model’s ability to align its attention distribution with actual object locations, particularly differing between familiar and unfamiliar spatial relationships. Motivated by these findings, we propose ADAPTVIS based on inference-time confidence scores to sharpen the attention on highly relevant regions when confident, while smoothing and broadening the attention window to consider a wider context when confidence is lower. This training-free decoding method shows significant improvement (e.g., up to a 50 absolute point improvement) on spatial reasoning benchmarks such as WhatsUp and VSR with negligible cost. We make code and data publicly available for research purposes at https://github.com/shiqichen17/AdaptVis.

Method: Uses LLMs to evaluate comparative contribution value of samples when added to subsets, with a greedy incremental sampling process that avoids exhaustive dataset traversal.

Result: Selected data outperforms full dataset performance and achieves competitive results with recent methods while requiring fewer selections. Successfully validated on larger medical dataset.

Conclusion: The choice-based framework effectively balances quality and diversity while reducing training overhead, demonstrating practical applicability in real-world scenarios.

Abstract: Selecting high-quality and diverse training samples from extensive datasets plays a crucial role in reducing training overhead and enhancing the performance of Large Language Models (LLMs). However, existing studies fall short in assessing the overall value of selected data, focusing primarily on individual quality, and struggle to strike an effective balance between ensuring diversity and minimizing data point traversals. Therefore, this paper introduces a novel choice-based sample selection framework that shifts the focus from evaluating individual sample quality to comparing the contribution value of different samples when incorporated into the subset. Thanks to the advanced language understanding capabilities of LLMs, we utilize LLMs to evaluate the value of each option during the selection process. Furthermore, we design a greedy sampling process where samples are incrementally added to the subset, thereby improving efficiency by eliminating the need for exhaustive traversal of the entire dataset with the limited budget. Extensive experiments demonstrate that selected data from our method not only surpasses the performance of the full dataset but also achieves competitive results with recent powerful studies, while requiring fewer selections. Moreover, we validate our approach on a larger medical dataset, highlighting its practical applicability in real-world applications. Our code and data are available at https://github.com/BIRlz/comperative_sample_selection.

Method: Proposed LADM framework that leverages attention mechanism’s retrieval capabilities to capture contextual dependencies and identify high-quality long-context data from multi-domain pre-training corpora.

Result: Experimental results show LADM significantly boosts LLM performance on multiple long-context tasks using only 1B tokens for continual training.

Conclusion: LADM provides an effective solution for long-context data quality measurement and selection, enabling efficient improvement of LLMs’ long-context capabilities with minimal training resources.

Abstract: Long-context modeling has drawn more and more attention in the area of Large Language Models (LLMs). Continual training with long-context data becomes the de-facto method to equip LLMs with the ability to process long inputs. However, it still remains an open challenge to measure the quality of long-context training data. To address this issue, we propose a Long-context data selection framework with Attention-based Dependency Measurement (LADM), which can efficiently identify high-quality long-context data from a large-scale, multi-domain pre-training corpus. LADM leverages the retrieval capabilities of the attention mechanism to capture contextual dependencies, ensuring a comprehensive quality measurement of long-context data. Experimental results show that our LADM framework significantly boosts the performance of LLMs on multiple long-context tasks with only 1B tokens for continual training.

Method: Targeted data augmentation using advanced LLMs with Chain-of-Thought prompting for difficult segments, followed by efficient fine-tuning (Unsloth with QLoRA) on a 1.8B parameter Sailor model.

Result: Significant improvement in Vi-Ja translation quality over existing baselines.

Conclusion: Integrated approach combining LLM-based data augmentation and efficient fine-tuning effectively addresses low-resource translation challenges for Vietnamese-Japanese language pair.

Abstract: Neural Machine Translation (NMT) driven by Transformer architectures has advanced significantly, yet faces challenges with low-resource language pairs like Vietnamese-Japanese (Vi-Ja). Issues include sparse parallel data and handling linguistic/cultural nuances. Recent progress in Large Language Models (LLMs) with strong reasoning, often refined via Reinforcement Learning (RL), enables high-quality synthetic data generation. We introduce VNJPTranslate, a pipeline designed to systematically address the Vi-Ja translation task. It features a targeted data augmentation strategy using advanced LLMs with Chain-of-Thought prompting for challenging segments identified via corpus analysis. Subsequently, we employ efficient fine-tuning techniques (Unsloth with QLoRA) on a capable, low-parameter autoregressive model (specifically, a fine-tuned version of the 1.8B parameter Sailor model, which is based on the Qwen architecture) to create a practical and high-performing translation system. This integrated approach aims to improve Vi-Ja translation quality significantly over existing baselines.

Method: Uses similarity metrics like BERTScore between sampled generations and reference answers as surrogate rewards in a REINFORCE-style algorithm, extendable to various alignment scenarios by combining similarity rewards with task-specific objectives.

Result: Achieves performance comparable to prior alignment methods across multiple scenarios without requiring binary preference data or reward models.

Conclusion: RefAlign provides an effective alternative to traditional alignment approaches by leveraging similarity-based rewards, making alignment more resource-efficient while maintaining competitive performance.

Abstract: Large language models~(LLMs) are expected to be helpful, harmless, and honest. In different alignment scenarios, such as safety, confidence, and general preference alignment, binary preference data collection and reward modeling are resource-intensive but play a central role in transferring human preferences. In this work, we explore using the similarity between sampled generations and reference answers as a supplementary reward function for alignment. When unary reference answers are available, such similarity-based rewards can circumvent the need for binary preference data and explicit reward modeling. We introduce \textit{RefAlign}, a versatile REINFORCE-style alignment algorithm that does not rely on reward or reference models. RefAlign utilizes language generation evaluation metrics, such as BERTScore, between sampled generations and reference answers as surrogate rewards. Beyond general preference optimization, RefAlign can be naturally extended to diverse scenarios, including safety and confidence alignment, by combining similarity-based rewards with task-specific objectives. Across multiple scenarios, RefAlign achieves performance comparable to prior alignment methods while operating without binary preference data or reward models. The code is available at https://github.com/mzhaoshuai/RefAlign.

Method: Systematic evaluation of diverse models including fine-tuned decoder-based LLMs and encoder-based transformers on event occurrence prediction, temporal ordering, and survival analysis using timestamped clinical findings extracted via LLM-assisted annotation.

Result: Encoder-based models consistently achieve higher F1 scores and superior temporal concordance for event forecasting, while fine-tuned masking approaches enhance ranking performance. Instruction-tuned decoder models show relative advantage in survival analysis, especially for early prognosis.

Conclusion: Time ordering in clinical time series provides additional benefits beyond text ordering, highlighting the value of time-ordered corpora for temporal tasks in the LLM era.

Abstract: Clinical case reports encode temporal patient trajectories that are often underexploited by traditional machine learning methods relying on structured data. In this work, we introduce the forecasting problem from textual time series, where timestamped clinical findings – extracted via an LLM-assisted annotation pipeline – serve as the primary input for prediction. We systematically evaluate a diverse suite of models, including fine-tuned decoder-based large language models and encoder-based transformers, on tasks of event occurrence prediction, temporal ordering, and survival analysis. Our experiments reveal that encoder-based models consistently achieve higher F1 scores and superior temporal concordance for short- and long-horizon event forecasting, while fine-tuned masking approaches enhance ranking performance. In contrast, instruction-tuned decoder models demonstrate a relative advantage in survival analysis, especially in early prognosis settings. Our sensitivity analyses further demonstrate the importance of time ordering, which requires clinical time series construction, as compared to text ordering, the format of the text inputs that LLMs are classically trained on. This highlights the additional benefit that can be ascertained from time-ordered corpora, with implications for temporal tasks in the era of widespread LLM use.

Method: Used RASP (Restricted Access Sequence Processing) programming language with Transformer Encoder-Decoder architecture, employing 19 attention-head compatible flat pattern-matching rules, word-level tokens with POS tagging, and masking techniques for prepositional phrases and sentential complements.

Result: Achieved near perfect scores on structural generalization splits: exact match on COGS and semantic exact match on ReCOGS_pos, including handling pp recursion and cp recursion through decoder loops.

Conclusion: COGS tasks do not require hierarchical or tree-structured solutions; flat pattern-matching rules in Transformers can effectively handle compositional generalization without recursive processing.

Abstract: Humans understand new combinations of words encountered if they are combinations of words recognized from different contexts, an ability called Compositional Generalization. The COGS benchmark (Kim and Linzen, 2020) arXiv:2010.05465 reports 0% accuracy for Transformer models on some structural generalizations. We use (Weiss et al., 2021) arXiv:2106.06981’s Restricted Access Sequence Processing (RASP), a Transformer-equivalent programming language, to demonstrate that a Transformer Encoder-Decoder can perform COGS and the semantically equivalent ReCOGS_pos (Wu et al., 2024) arXiv:2303.13716 systematically and compositionally: Our RASP models attain near perfect scores on structural generalization splits on COGS (exact match) and ReCOGS_pos (semantic exact match). Our RASP models show the (Re)COGS tasks do not require a hierarchical or tree-structured solution (contrary to (Kim and Linzen, 2020) arXiv:2010.05465, (Yao and Koller, 2022) arXiv:2210.13050, (Murty et al., 2022) arXiv:2305.18741, (Liu et al., 2021) arXiv:2107.06516): we use word-level tokens with an “embedding” layer that tags with possible part of speech, applying just once per encoder pass 19 attention-head compatible flat pattern-matching rules (easily identified with specific training examples), shown using grammar coverage (Zeller et al., 2023) to cover the non-recursive aspects of the input grammar, plus masking out prepositional phrases (“pp noun”) and/or sentential complements (cp) when recognizing grammar patterns and extracting nouns related to the main verb in the sentence, and output the next logical form (LF) token (repeating until the LF is complete). The models do not apply recursive, tree-structured rules like “np_det pp np -> np_pp -> np”, but score near perfect semantic and string exact match on both COGS and ReCOGS pp recursion, cp recursion using the decoder loop.

Method: Developed DMDTEval framework with three components: (1) constructed translation test set with multi-domain ambiguous word annotations, (2) curated diverse disambiguation prompt strategies, (3) designed precise disambiguation metrics and studied various prompt strategies on multiple state-of-the-art LLMs.

Result: Comprehensive experiments across 4 language pairs and 13 domains revealed crucial findings about LLMs’ disambiguation performance in multi-domain translation settings.

Conclusion: The findings from extensive experiments are expected to pave the way and facilitate further research in improving LLMs’ disambiguation capabilities in multi-domain translation.

Abstract: Currently, Large Language Models (LLMs) have achieved remarkable results in machine translation. However, their performance in multi-domain translation (MDT) is less satisfactory, the meanings of words can vary across different domains, highlighting the significant ambiguity inherent in MDT. Therefore, evaluating the disambiguation ability of LLMs in MDT, remains an open problem. To this end, we present an evaluation and analysis of LLMs on disambiguation in multi-domain translation (DMDTEval), our systematic evaluation framework consisting of three critical aspects: (1) we construct a translation test set with multi-domain ambiguous word annotation, (2) we curate a diverse set of disambiguation prompt strategies, and (3) we design precise disambiguation metrics, and study the efficacy of various prompt strategies on multiple state-of-the-art LLMs. We conduct comprehensive experiments across 4 language pairs and 13 domains, our extensive experiments reveal a number of crucial findings that we believe will pave the way and also facilitate further research in the critical area of improving the disambiguation of LLMs.

Method: WebThinker integrates a Deep Web Explorer module for dynamic web search and information extraction, employs an Autonomous Think-Search-and-Draft strategy for real-time reasoning and writing, and uses RL-based training via iterative online Direct Preference Optimization to enhance research tool utilization.

Result: Extensive experiments on complex reasoning benchmarks (GPQA, GAIA, WebWalkerQA, HLE) and scientific report generation tasks (Glaive) show WebThinker significantly outperforms existing methods and strong proprietary systems.

Conclusion: WebThinker enhances LRM reliability and applicability in complex scenarios, paving the way for more capable and versatile deep research systems.

Abstract: Large reasoning models (LRMs), such as OpenAI-o1 and DeepSeek-R1, demonstrate impressive long-horizon reasoning capabilities. However, their reliance on static internal knowledge limits their performance on complex, knowledge-intensive tasks and hinders their ability to produce comprehensive research reports requiring synthesis of diverse web information. To address this, we propose WebThinker, a deep research agent that empowers LRMs to autonomously search the web, navigate among web pages, and draft reports during the reasoning process. WebThinker integrates a Deep Web Explorer module, enabling LRMs to dynamically search, navigate, and extract information from the web when encountering knowledge gaps. It also employs an Autonomous Think-Search-and-Draft strategy, allowing the model to seamlessly interleave reasoning, information gathering, and report writing in real time. To further enhance research tool utilization, we introduce an RL-based training strategy via iterative online Direct Preference Optimization (DPO). Extensive experiments on complex reasoning benchmarks (GPQA, GAIA, WebWalkerQA, HLE) and scientific report generation tasks (Glaive) demonstrate that WebThinker significantly outperforms existing methods and strong proprietary systems. Our approach enhances LRM reliability and applicability in complex scenarios, paving the way for more capable and versatile deep research systems. The code is available at https://github.com/RUC-NLPIR/WebThinker.

Method: A reference-free, LLM-based question-answering framework that integrates humans in the loop to incorporate insights from professional literary translators and researchers, focusing on literary devices, cultural understanding, and authorial voice.

Result: LITRANSPROQA substantially outperforms current metrics with up to 0.07 gain in correlation and over 15 points improvement in adequacy assessments. It reaches adequacy performance comparable to trained linguistic students but still falls behind experienced professional translators.

Conclusion: LITRANSPROQA shows broad applicability to open-source models and potential as an accessible, training-free tool for evaluating literary translations, especially where local processing is needed due to copyright or ethical considerations.

Abstract: The impact of Large Language Models (LLMs) has extended into literary domains. However, existing evaluation metrics for literature prioritize mechanical accuracy over artistic expression and tend to overrate machine translation as being superior to human translation from experienced professionals. In the long run, this bias could result in an irreversible decline in translation quality and cultural authenticity. In response to the urgent need for a specialized literary evaluation metric, we introduce LITRANSPROQA, a novel, reference-free, LLM-based question-answering framework designed for literary translation evaluation. LITRANSPROQA integrates humans in the loop to incorporate insights from professional literary translators and researchers, focusing on critical elements in literary quality assessment such as literary devices, cultural understanding, and authorial voice. Our extensive evaluation shows that while literary-finetuned XCOMET-XL yields marginal gains, LITRANSPROQA substantially outperforms current metrics, achieving up to 0.07 gain in correlation and surpassing the best state-of-the-art metrics by over 15 points in adequacy assessments. Incorporating professional translator insights as weights further improves performance, highlighting the value of translator inputs. Notably, LITRANSPROQA reaches an adequacy performance comparable to trained linguistic student evaluators, though it still falls behind experienced professional translators. LITRANSPROQA shows broad applicability to open-source models like LLaMA3.3-70b and Qwen2.5-32b, indicating its potential as an accessible and training-free tool for evaluating literary translations that require local processing due to copyright or ethical considerations.

Method: Comprehensive safety evaluation of 11 MLRMs across 5 benchmarks, analysis of safety degradation patterns, and construction of multimodal tuning dataset with safety-oriented thought processes for fine-tuning.

Result: Revealed prevalent safety degradation in most advanced models, with significant degradation in jailbreak robustness benchmarks but less in safety-awareness benchmarks. Fine-tuning with safety-oriented thought process dataset effectively enhances safety on both benchmark types.

Conclusion: Leveraging models’ intrinsic reasoning capabilities through safety-oriented thought processes provides a new perspective for developing safe MLRMs, offering a potential approach to address safety issues.

Abstract: The rapid development of Multimodal Large Reasoning Models (MLRMs) has demonstrated broad application potential, yet their safety and reliability remain critical concerns that require systematic exploration. To address this gap, we conduct a comprehensive and systematic safety evaluation of 11 MLRMs across 5 benchmarks and unveil prevalent safety degradation phenomena in most advanced models. Moreover, our analysis reveals distinct safety patterns across different benchmarks: significant safety degradation is observed across jailbreak robustness benchmarks, whereas safety-awareness benchmarks demonstrate less pronounced degradation. In particular, the long thought process in some scenarios even enhances safety performance. Therefore, it is a potential approach to address safety issues in MLRMs by leveraging the intrinsic reasoning capabilities of the model to detect unsafe intent. To operationalize this insight, we construct a multimodal tuning dataset that incorporates a safety-oriented thought process. Experimental results from fine-tuning existing MLRMs with this dataset effectively enhances the safety on both jailbreak robustness and safety-awareness benchmarks. This study provides a new perspective for developing safe MLRMs. Our dataset is available at https://github.com/xinyuelou/Think-in-Safety.

Method: Uses two innovations: (1) response-based inference to generate pseudo reference passages from responses, and (2) response refinement leveraging the dual-role of CQR where the model refines responses based on shared objectives between response refinement and CQR.

Result: Achieves 96.9-99.1% of retrieval accuracy attainable with reference passages, and surpasses state-of-the-art method by up to 31.6% without using actual reference passages.

Conclusion: DualReform provides an effective reference-free solution for CQR optimization that performs nearly as well as methods requiring reference passages, making it practical for real-world applications.

Abstract: Conversational query reformulation (CQR) has become indispensable for improving retrieval in dialogue-based applications. However, existing approaches typically rely on reference passages for optimization, which are impractical to acquire in real-world scenarios. To address this limitation, we introduce a novel reference-free preference optimization framework DualReform that generates pseudo reference passages from commonly-encountered conversational datasets containing only queries and responses. DualReform attains this goal through two key innovations: (1) response-based inference, where responses serve as proxies to infer pseudo reference passages, and (2) response refinement via the dual-role of CQR, where a CQR model refines responses based on the shared objectives between response refinement and CQR. Despite not relying on reference passages, DualReform achieves 96.9–99.1% of the retrieval accuracy attainable only with reference passages and surpasses the state-of-the-art method by up to 31.6%.

Method: Convert all judgment tasks into a unified format with verifiable rewards, then use RL to train thinking-judges at 8B, 32B, and 70B scales on synthetic data.

Result: J1-Qwen-32B outperforms o1-mini, o3, and 671B DeepSeek-R1 on some benchmarks, and develops systematic evaluation strategies including dynamic criteria generation and iterative self-correction.

Conclusion: The J1 framework effectively optimizes LLM judges’ reasoning process, achieving superior performance through RL training on synthetic data with verifiable rewards.

Abstract: The progress of AI is bottlenecked by the quality of evaluation, making powerful LLM-as-a-Judge models a core solution. The efficacy of these judges depends on their chain-of-thought reasoning, creating a critical need for methods that can effectively optimize this reasoning process. In this work, we introduce J1, a reinforcement learning framework for teaching LLM judges to think before making decisions. Our core contribution lies in converting all judgment tasks for non-verifiable and verifiable prompts into a unified format with verifiable rewards, enabling direct optimization of evaluation quality while mitigating positional bias. We then use RL to train thinking-judges at scales of 8B, 32B, and 70B and show that they obtain state-of-the-art performance across multiple benchmarks. In particular, J1-Qwen-32B, our multitasked pointwise and pairwise judge also outperforms o1-mini, o3, and a much larger 671B DeepSeek-R1 on some benchmarks, while only training on synthetic data. Through comprehensive ablations of pairwise, pointwise, and multitask J1 variants, we demonstrate the effectiveness of our approach across seed prompts, reward strategies, and training recipes. Qualitative analysis reveals that J1 develops systematic evaluation strategies, including dynamic criteria generation, reference answer creation, iterative self-correction of initial assessments, and feedback generation for low-quality responses.

Method: Decouples long-context preference optimization into two components: short-context PO (using preference pairs from short contexts) and short-to-long reward alignment (encouraging reward consistency between short and long contexts with identical task-relevant information).

Result: SoLoPO enhances preference optimization algorithms with stronger length and domain generalization abilities across various long-context benchmarks, while achieving notable improvements in computational and memory efficiency.

Conclusion: SoLoPO effectively transfers models’ short-context handling ability to long-context scenarios, improving data construction and training efficiency while being compatible with mainstream preference optimization algorithms.

Abstract: Despite advances in pretraining with extended context lengths, large language models (LLMs) still face challenges in effectively utilizing real-world long-context information, primarily due to insufficient long-context alignment caused by data quality issues, training inefficiencies, and the lack of well-designed optimization objectives. To address these limitations, we propose a framework named $\textbf{S}$h$\textbf{o}$rt-to-$\textbf{Lo}$ng $\textbf{P}$reference $\textbf{O}$ptimization ($\textbf{SoLoPO}$), decoupling long-context preference optimization (PO) into two components: short-context PO and short-to-long reward alignment (SoLo-RA), supported by both theoretical and empirical evidence. Specifically, short-context PO leverages preference pairs sampled from short contexts to enhance the model’s contextual knowledge utilization ability. Meanwhile, SoLo-RA explicitly encourages reward score consistency utilization for the responses when conditioned on both short and long contexts that contain identical task-relevant information. This facilitates transferring the model’s ability to handle short contexts into long-context scenarios. SoLoPO is compatible with mainstream preference optimization algorithms, while substantially improving the efficiency of data construction and training processes. Experimental results show that SoLoPO enhances all these algorithms with respect to stronger length and domain generalization abilities across various long-context benchmarks, while achieving notable improvements in both computational and memory efficiency.

Method: LRC trains low-rank projection matrices that jointly perform soft pruning by compressing teacher weights and activation cloning by aligning student activations (including FFN signals) with teacher activations, eliminating the need for explicit alignment modules.

Result: LRC matches or surpasses state-of-the-art models trained on trillions of tokens while using only 20B tokens, achieving over 1,000x training efficiency with open-source teachers like Llama-3.2-3B-Instruct and Qwen2.5-3B/7B-Instruct.

Conclusion: LRC provides an efficient unified framework for knowledge transfer that maximizes behavioral equivalence with teacher models while dramatically reducing training costs.

Abstract: Training high-performing Small Language Models (SLMs) remains costly, even with knowledge distillation and pruning from larger teacher models. Existing work often faces three key challenges: (1) information loss from hard pruning, (2) inefficient alignment of representations, and (3) underutilization of informative activations, particularly from Feed-Forward Networks (FFNs). To address these challenges, we introduce Low-Rank Clone (LRC), an efficient pre-training method that constructs SLMs aspiring to behavioral equivalence with strong teacher models. LRC trains a set of low-rank projection matrices that jointly enable soft pruning by compressing teacher weights, and activation clone by aligning student activations, including FFN signals, with those of the teacher. This unified design maximizes knowledge transfer while removing the need for explicit alignment modules. Extensive experiments with open-source teachers (e.g., Llama-3.2-3B-Instruct, Qwen2.5-3B/7B-Instruct) show that LRC matches or surpasses state-of-the-art models trained on trillions of tokens–while using only 20B tokens, achieving over 1,000x training efficiency. Our codes and model checkpoints are available at https://github.com/CURRENTF/LowRankClone and https://huggingface.co/collections/JitaiHao/low-rank-clone-lrc-6828389e96a93f1d4219dfaf.

Method: Proposes ARENA framework with structured protocol, KL divergence stabilization, and adaptive reward calculation modules to enable evidence identification, structured reasoning, and interpretable decision traces.

Result: Achieves 10-30% accuracy improvements on three multi-hop QA datasets using Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct, comparable to advanced closed-source LLMs like OpenAI o1 and DeepSeek R1.

Conclusion: ARENA provides a transparent and robust RAG framework that generalizes well to unseen datasets and tasks, with publicly released models and codes.

Abstract: Retrieval-Augmented Generation (RAG) delivers substantial value in knowledge-intensive applications. Many recent works use reinforcement learning (RL) to elicit strong reasoning in RAG generators. However, two key challenges remain unresolved: (1) Transparency: most prior methods do not explicitly indicate which references are actually used during the reasoning that leads to the final answer, limiting interpretability and visibility; (2) Stability: the KL divergence estimator used in existing RL-based approaches may cause gradient spikes, leading to unstable training. To address these challenges, we propose Adaptive-Rewarded Evidence Navigation Agent (ARENA), a transparent and robust RAG generator framework trained via RL with designed rewards. Based on our structured protocol, KL divergence stabilization, and adaptive reward calculation modules, ARENA enables the RAG generator to identify key evidence, perform structured reasoning, and generate answers with interpretable decision traces. Applied to Qwen2.5-7B-Instruct and Llama3.1-8B-Instruct, extensive experiments across multiple baselines show 10-30% accuracy improvements on three multi-hop QA datasets, comparable to advanced closed-source LLMs (e.g., OpenAI o1, DeepSeek R1). Further analyses show that ARENA generalizes well to unseen datasets and tasks. Our models and codes are publicly released.

Method: Systematically injecting Gaussian noise into model activations across multiple open-weight LLMs to test safety fine-tuning robustness.

Result: Gaussian noise increases harmful-output rates by up to 27% (p < 0.001), deeper safety fine-tuning provides no extra protection, and chain-of-thought reasoning remains largely unaffected.

Conclusion: Current safety alignment techniques have critical vulnerabilities, highlighting reasoning-based and reinforcement learning approaches as promising directions for developing more robust AI safety systems.

Abstract: Safety guardrails in large language models (LLMs) are a critical component in preventing harmful outputs. Yet, their resilience under perturbation remains poorly understood. In this paper, we investigate the robustness of safety fine-tuning in LLMs by systematically injecting Gaussian noise into model activations. We show across multiple open-weight models that (1) Gaussian noise raises harmful-output rates (p < 0.001) by up to 27%, (2) that deeper safety fine-tuning affords no extra protection, and (3) that chain-of-thought reasoning remains largely intact. The findings reveal critical vulnerabilities in current safety alignment techniques and highlight the potential of reasoning-based and reinforcement learning approaches as promising direction for developing more robust AI safety systems. These results have important implications for real-world deployment of LLMs in safety-critical applications as these results imply that widely-deployed safety tuning methods can fail even without adversarial prompts.

Method: Leverages code-mixing and phonetic perturbations to bypass safety filters, with an extension to jailbreak-template-based strategy and a novel template. Applies phonetic misspellings to sensitive words in code-mixed prompts.

Result: Achieved 99% Attack Success Rate for text generation and 78% for image generation, with Attack Relevance Rate of 100% for text and 96% for image generation. Phonetic perturbations impact word tokenization, leading to jailbreak success.

Conclusion: Study highlights need for more generalizable safety alignment for multilingual multimodal models, especially in real-world settings with misspelled words, as current models remain vulnerable to these attack strategies.

Abstract: Recently released LLMs have strong multilingual & multimodal capabilities. Model vulnerabilities are exposed using audits and red-teaming efforts. Existing efforts have focused primarily on the English language; thus, models continue to be susceptible to multilingual jailbreaking strategies, especially for multimodal contexts. In this study, we introduce a novel strategy that leverages code-mixing and phonetic perturbations to jailbreak LLMs for both text and image generation tasks. We also present an extension to a current jailbreak-template-based strategy and propose a novel template, showing higher effectiveness than baselines. Our work presents a method to effectively bypass safety filters in LLMs while maintaining interpretability by applying phonetic misspellings to sensitive words in code-mixed prompts. We achieve a 99% Attack Success Rate for text generation and 78% for image generation, with Attack Relevance Rate of 100% for text generation and 96% for image generation for the phonetically perturbed code-mixed prompts. Our interpretability experiments reveal that phonetic perturbations impact word tokenization, leading to jailbreak success. Our study motivates increasing the focus towards more generalizable safety alignment for multilingual multimodal models, especially in real-world settings wherein prompts can have misspelt words. \textit{\textbf{Warning: This paper contains examples of potentially harmful and offensive content.}}

Method: Develops sparse augmented tensor networks (Saten) that enable full model compression of LLMs during fine-tuning phase.

Result: Saten improves both accuracy and compression efficiency in tensorized language models, achieving state-of-the-art performance.

Conclusion: The Saten framework successfully addresses challenges in compressing pre-trained LLMs and enhances tensor network performance for downstream tasks.

Abstract: The efficient implementation of large language models (LLMs) is crucial for deployment on resource-constrained devices. Low-rank tensor compression techniques, such as tensor-train (TT) networks, have been widely studied for over-parameterized neural networks. However, their applications to compress pre-trained large language models (LLMs) for downstream tasks (post-training) remains challenging due to the high-rank nature of pre-trained LLMs and the lack of access to pretraining data. In this study, we investigate low-rank tensorized LLMs during fine-tuning and propose sparse augmented tensor networks (Saten) to enhance their performance. The proposed Saten framework enables full model compression. Experimental results demonstrate that Saten enhances both accuracy and compression efficiency in tensorized language models, achieving state-of-the-art performance.

Method: Investigated positional distribution of hallucinations in LLM-based long response generation using long document summarization as case study, exploring attention and decoding dynamics.

Result: Found consistent phenomenon: hallucinations concentrate disproportionately in latter parts of generated long responses.

Conclusion: Identified positional hallucination bias and explored mitigation methods to improve faithfulness in concluding segments of long outputs.

Abstract: Large Language Models (LLMs) have significantly advanced text generation capabilities, including tasks like summarization, often producing coherent and fluent outputs. However, faithfulness to source material remains a significant challenge due to the generation of hallucinations. While extensive research focuses on detecting and reducing these inaccuracies, less attention has been paid to the positional distribution of hallucination within generated text, particularly in long outputs. In this work, we investigate where hallucinations occur in LLM-based long response generation, using long document summarization as a key case study. Focusing on the challenging setting of long context-aware long response generation, we find a consistent and concerning phenomenon: hallucinations tend to concentrate disproportionately in the latter parts of the generated long response. To understand this bias, we explore potential contributing factors related to the dynamics of attention and decoding over long sequences. Furthermore, we investigate methods to mitigate this positional hallucination, aiming to improve faithfulness specifically in the concluding segments of long outputs.

Method: Online reinforcement learning framework using simulated student-tutor interactions, with controllable reward weighting to balance pedagogical support and student solving accuracy.

Result: Trained a 7B parameter tutor model without human annotations that reaches similar performance to larger proprietary models like LearnLM, better preserving reasoning capabilities than SFT baselines.

Conclusion: The framework enables training effective pedagogical tutors with optional interpretability through thinking tags, tracing the Pareto frontier between pedagogical support and student accuracy.

Abstract: Large language models (LLMs) can transform education, but their optimization for direct question-answering often undermines effective pedagogy which requires strategically withholding answers. To mitigate this, we propose an online reinforcement learning (RL)-based alignment framework that can quickly adapt LLMs into effective tutors using simulated student-tutor interactions by emphasizing pedagogical quality and guided problem-solving over simply giving away answers. We use our method to train a 7B parameter tutor model without human annotations which reaches similar performance to larger proprietary models like LearnLM. We introduce a controllable reward weighting to balance pedagogical support and student solving accuracy, allowing us to trace the Pareto frontier between these two objectives. Our models better preserve reasoning capabilities than single-turn SFT baselines and can optionally enhance interpretability through thinking tags that expose the model’s instructional planning.

Method: Constructs a problem-solving template library via MCTS on a small seed set, then integrates this high-level structured guidance into RL training to align rollout generation with proven template structures.

Result: Outperforms GRPO by 99% on AIME and 41% on AMC, with superior stability on weak models and remarkable cross-domain generalization.

Conclusion: TemplateRL’s structure-guided design effectively steers policies toward validated strategic patterns, improving RL sampling efficiency and demonstrating potential for broader tasks through interpretable and editable template libraries.

Abstract: Reinforcement learning (RL) has emerged as an effective paradigm for enhancing model reasoning. However, existing RL methods like GRPO often rely on unstructured self-sampling to fit scalar rewards, often producing inefficient rollouts that fail to capture transferable problem-solving strategies. To address these limitations, we propose TemplateRL, a structured template-guided RL framework that augments policy optimization with explicit template guidance. Our approach first constructs a problem-solving template library via MCTS on a small seed set, then seamlessly integrates this high-level structured guidance into RL training. By guiding rollout generation to align with proven template structures, TemplateRL significantly improves high-quality trajectory hit rates while reducing ineffective exploration. This structure-guided design steers the policy toward validated strategic patterns, stabilizing training dynamics, and enhancing RL sampling efficiency. Notably, the explicit template library is interpretable, editable, and supports online updates-enabling continuous updates during both training and inference. Extensive experiments demonstrate that TemplateRL outperforms GRPO by 99% on AIME and 41% on AMC, with superior stability on weak models and remarkable cross-domain generalization, highlighting its potential for broader tasks.

Method: Two personalization strategies: (1) lightweight finetuning using Low-Rank Adaptation (LoRA) on open-source models, and (2) personalized prompting that tailors model behavior at inference time without retraining. Also investigated semi-supervised approaches combining limited annotated data with self-supervised learning from users’ publications.

Result: Personalized LoRA model outperforms GPT-4 with contextual prompting by 21.4% in F1 score and exceeds the best performing oracle baseline by 8.3%. Achieves comparable performance using only 10% of annotated training data.

Conclusion: First systematic exploration of efficient, low-resource personalization for jargon detection using open-source language models, offering a practical path toward scalable, user-adaptive NLP systems.

Abstract: Personalizing jargon detection and explanation is essential for making technical documents accessible to readers with diverse disciplinary backgrounds. However, tailoring models to individual users typically requires substantial annotation efforts and computational resources due to user-specific finetuning. To address this, we present a systematic study of personalized jargon detection, focusing on methods that are both efficient and scalable for real-world deployment. We explore two personalization strategies: (1) lightweight finetuning using Low-Rank Adaptation (LoRA) on open-source models, and (2) personalized prompting, which tailors model behavior at inference time without retaining. To reflect realistic constraints, we also investigate semi-supervised approaches that combine limited annotated data with self-supervised learning from users’ publications. Our personalized LoRA model outperforms GPT-4 with contextual prompting by 21.4% in F1 score and exceeds the best performing oracle baseline by 8.3%. Remarkably, our method achieves comparable performance using only 10% of the annotated training data, demonstrating its practicality for resource-constrained settings. Our study offers the first work to systematically explore efficient, low-resource personalization of jargon detection using open-source language models, offering a practical path toward scalable, user-adaptive NLP system.

Method: Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD) that identifies essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling.

Result: Superior accuracy and significant computational efficiency improvements on RAG benchmarks (TyDi QA, Hotpot QA, Musique) compared to previous surrogate-based methods. Mechanistic analysis reveals specific attention heads and MLP layers responsible for context attribution.

Conclusion: ARC-JSD provides an efficient and accurate solution for context attribution in RAG systems while offering insights into the internal mechanisms of how LLMs process and attribute context.

Abstract: Retrieval-Augmented Generation (RAG) leverages large language models (LLMs) combined with external contexts to enhance the accuracy and reliability of generated responses. However, reliably attributing generated content to specific context segments, context attribution, remains challenging due to the computationally intensive nature of current methods, which often require extensive fine-tuning or human annotation. In this work, we introduce a novel Jensen-Shannon Divergence driven method to Attribute Response to Context (ARC-JSD), enabling efficient and accurate identification of essential context sentences without additional fine-tuning, gradient-calculation or surrogate modelling. Evaluations on a wide range of RAG benchmarks, such as TyDi QA, Hotpot QA, and Musique, using instruction-tuned LLMs in different scales demonstrate superior accuracy and significant computational efficiency improvements compared to the previous surrogate-based method. Furthermore, our mechanistic analysis reveals specific attention heads and multilayer perceptron (MLP) layers responsible for context attribution, providing valuable insights into the internal workings of RAG models and how they affect RAG behaviours. Our code is available at https://github.com/ruizheliUOA/ARC_JSD.

Method: TRIM employs an iterative adjustment process with quality metrics to optimize dimension-wise sparsity allocation, reducing variance in quality retention across outputs and preserving critical information. It can be integrated with existing layer-wise pruning strategies.

Result: TRIM achieves new SOTA results across diverse LLM families (Qwen2.5, LLaMA-2, OPT) and sparsity levels. At 80% sparsity, it reduces perplexity by 48% for Qwen2.5-14B and over 90% for OPT-13B compared to baselines, while enhancing stability.

Conclusion: Fine-grained, dimension-wise sparsity adaptation is crucial for pushing the limits of extreme LLM compression, demonstrating the effectiveness of targeted row-wise iterative pruning.

Abstract: Large Language Models (LLMs) present significant computational and memory challenges due to their extensive size, making pruning essential for their efficient deployment. Existing one-shot pruning methods often apply uniform sparsity constraints across layers or within each layer, resulting in suboptimal performance, especially at high sparsity ratios. This work introduces TRIM (Targeted Row-wise Iterative Metric-driven pruning), a novel approach that applies varying sparsity ratios to individual output dimensions (rows) within each layer. TRIM employs an iterative adjustment process guided by quality metrics to optimize dimension-wise sparsity allocation, focusing on reducing variance in quality retention across outputs to preserve critical information. TRIM can be seamlessly integrated with existing layer-wise pruning strategies. Our evaluations on perplexity and zero-shot tasks across diverse LLM families (Qwen2.5, LLaMA-2, and OPT) and sparsity levels demonstrate that TRIM achieves new state-of-the-art results and enhances stability. For instance, at 80% sparsity, TRIM reduces perplexity by 48% for Qwen2.5-14B and over 90% for OPT-13B compared to baseline methods. We conclude that fine-grained, dimension-wise sparsity adaptation is crucial for pushing the limits of extreme LLM compression. Code available at: https://github.com/flobk/TRIM

Method: A hybrid framework combining transformer-based contextual embeddings with graph convolutional networks, using bidirectional cross-attention between rule-based syntactic dependency parsing and semantic similarity computations to fuse local syntactic structure with global semantic context.

Result: Outperforms existing approaches on both low-resource Bengali datasets and high-resource English SemEval 2014 Task 4 dataset, achieving 0.93% F1-score improvement for Restaurant domain and 1.06% gain for Laptop domain in SemEval benchmark.

Conclusion: The bidirectional cross-attention mechanism effectively enhances aspect-level sentiment classification by integrating syntactic and semantic information, demonstrating strong performance across both low- and high-resource language settings.

Abstract: Aspect-Based Sentiment Analysis (ABSA) is a fundamental task in natural language processing, offering fine-grained insights into opinions expressed in text. While existing research has largely focused on resource-rich languages like English which leveraging large annotated datasets, pre-trained models, and language-specific tools. These resources are often unavailable for low-resource languages such as Bengali. The ABSA task in Bengali remains poorly explored and is further complicated by its unique linguistic characteristics and a lack of annotated data, pre-trained models, and optimized hyperparameters. To address these challenges, this research propose CrosGrpsABS, a novel hybrid framework that leverages bidirectional cross-attention between syntactic and semantic graphs to enhance aspect-level sentiment classification. The CrosGrpsABS combines transformerbased contextual embeddings with graph convolutional networks, built upon rule-based syntactic dependency parsing and semantic similarity computations. By employing bidirectional crossattention, the model effectively fuses local syntactic structure with global semantic context, resulting in improved sentiment classification performance across both low- and high-resource settings. We evaluate CrosGrpsABS on four low-resource Bengali ABSA datasets and the high-resource English SemEval 2014 Task 4 dataset. The CrosGrpsABS consistently outperforms existing approaches, achieving notable improvements, including a 0.93% F1-score increase for the Restaurant domain and a 1.06% gain for the Laptop domain in the SemEval 2014 Task 4 benchmark.

Method: Establishes a unified framework for efficiency strategies and systematically reviews data-centric compression methods that directly compress data volume during training or inference.

Result: Demonstrates that data-centric compression constitutes a crucial paradigm change for long-context AI and analyzes benefits across diverse scenarios.

Conclusion: Provides a novel perspective on AI efficiency, synthesizes existing efforts, and outlines key challenges and future research directions to address ever-increasing context lengths.

Abstract: The advancement of large language models (LLMs) and multi-modal LLMs (MLLMs) has historically relied on scaling model parameters. However, as hardware limits constrain further model growth, the primary computational bottleneck has shifted to the quadratic cost of self-attention over increasingly long sequences by ultra-long text contexts, high-resolution images, and extended videos. In this position paper, \textbf{we argue that the focus of research for efficient artificial intelligence (AI) is shifting from model-centric compression to data-centric compression}. We position data-centric compression as the emerging paradigm, which improves AI efficiency by directly compressing the volume of data processed during model training or inference. To formalize this shift, we establish a unified framework for existing efficiency strategies and demonstrate why it constitutes a crucial paradigm change for long-context AI. We then systematically review the landscape of data-centric compression methods, analyzing their benefits across diverse scenarios. Finally, we outline key challenges and promising future research directions. Our work aims to provide a novel perspective on AI efficiency, synthesize existing efforts, and catalyze innovation to address the challenges posed by ever-increasing context lengths.

Method: Orchestrates specialized AI agents (Planner, Step Definer, Extractor, QA Agents) that decompose tasks into subtasks, communicate via chain-of-thought prompting, and progressively refine retrieval and synthesis while maintaining modular interpretability.

Result: Significantly outperforms standalone LLMs and existing RAG methods across all model scales on multi-hop and ambiguous QA benchmarks. Small LLaMA3-8B with MA-RAG surpasses larger standalone LLMs, while larger variants set new SOTA results. Generalizes to specialized domains like medical QA without domain-specific fine-tuning.

Conclusion: MA-RAG establishes a new paradigm for efficient and reliable multi-agent RAG, improving answer accuracy, robustness, and providing interpretable intermediate reasoning steps through collaborative, modular reasoning.

Abstract: We present MA-RAG, a Multi-Agent framework for Retrieval-Augmented Generation (RAG) that addresses the inherent ambiguities and reasoning challenges in complex information-seeking tasks. Unlike conventional RAG methods that rely on end-to-end fine-tuning or isolated component enhancements, MA-RAG orchestrates a collaborative set of specialized AI agents: Planner, Step Definer, Extractor, and QA Agents, each responsible for a distinct stage of the RAG pipeline. By decomposing tasks into subtasks such as query disambiguation, evidence extraction, and answer synthesis, and enabling agents to communicate intermediate reasoning via chain-of-thought prompting, MA-RAG progressively refines retrieval and synthesis while maintaining modular interpretability. Extensive experiments on multi-hop and ambiguous QA benchmarks, including NQ, HotpotQA, 2WikimQA, and TriviaQA, demonstrate that MA-RAG significantly outperforms standalone LLMs and existing RAG methods across all model scales. Notably, even a small LLaMA3-8B model equipped with MA-RAG surpasses larger standalone LLMs, while larger variants (LLaMA3-70B and GPT-4o-mini) set new state-of-the-art results on challenging multi-hop datasets. Ablation studies reveal that both the planner and extractor agents are critical for multi-hop reasoning, and that high-capacity models are especially important for the QA agent to synthesize answers effectively. Beyond general-domain QA, MA-RAG generalizes to specialized domains such as medical QA, achieving competitive performance against domain-specific models without any domain-specific fine-tuning. Our results highlight the effectiveness of collaborative, modular reasoning in retrieval-augmented systems: MA-RAG not only improves answer accuracy and robustness but also provides interpretable intermediate reasoning steps, establishing a new paradigm for efficient and reliable multi-agent RAG.

Method: Proposed ARM with Ada-GRPO training method, an adaptation of Group Relative Policy Optimization that prevents format collapse and enables adaptive format selection.

Result: ARM achieves 30% average token reduction (up to 70%) while maintaining comparable performance to Long CoT models, plus 2x training speedup.

Conclusion: ARM provides efficient adaptive reasoning with three modes (Adaptive, Instruction-Guided, Consensus-Guided), balancing performance and computational efficiency for autonomous AI systems.

Abstract: While large reasoning models demonstrate strong performance on complex tasks, they lack the ability to adjust reasoning token usage based on task difficulty. This often leads to the “overthinking” problem – excessive and unnecessary reasoning – which, although potentially mitigated by human intervention to control the token budget, still fundamentally contradicts the goal of achieving fully autonomous AI. In this work, we propose Adaptive Reasoning Model (ARM), a reasoning model capable of adaptively selecting appropriate reasoning formats based on the task at hand. These formats include three efficient ones – Direct Answer, Short CoT, and Code – as well as a more elaborate format, Long CoT. To train ARM, we introduce Ada-GRPO, an adaptation of Group Relative Policy Optimization (GRPO), which addresses the format collapse issue in traditional GRPO. Ada-GRPO enables ARM to achieve high token efficiency, reducing tokens by an average of 30%, and up to 70%, while maintaining performance comparable to the model that relies solely on Long CoT. Furthermore, not only does it improve inference efficiency through reduced token generation, but it also brings a 2x speedup in training. In addition to the default Adaptive Mode, ARM supports two additional reasoning modes: 1) Instruction-Guided Mode, which allows users to explicitly specify the reasoning format via special tokens – ideal when the appropriate format is known for a batch of tasks. 2) Consensus-Guided Mode, which aggregates the outputs of the three efficient formats and resorts to Long CoT in case of disagreement, prioritizing performance with higher token usage.

Method: Multi-Scale Manifold Alignment (MSMA) framework that decomposes LLM representations into local, intermediate, and global manifolds, learns cross-scale mappings preserving geometry and information, and evaluates using multiple estimators including relative KL reduction and mutual information gains.

Result: Consistent hierarchical patterns observed across GPT-2, BERT, RoBERTa, and T5; MSMA improves alignment metrics with statistical significance across seeds; controlled interventions at different scales yield distinct architecture-dependent effects on lexical diversity, sentence structure, and discourse coherence.

Conclusion: Multi-objective alignment offers a practical lens for analyzing cross-scale information flow and guiding representation-level control, though theoretical analysis relies on idealized assumptions.

Abstract: We present Multi-Scale Manifold Alignment(MSMA), an information-geometric framework that decomposes LLM representations into local, intermediate, and global manifolds and learns cross-scale mappings that preserve geometry and information. Across GPT-2, BERT, RoBERTa, and T5, we observe consistent hierarchical patterns and find that MSMA improves alignment metrics under multiple estimators (e.g., relative KL reduction and MI gains with statistical significance across seeds). Controlled interventions at different scales yield distinct and architecture-dependent effects on lexical diversity, sentence structure, and discourse coherence. While our theoretical analysis relies on idealized assumptions, the empirical results suggest that multi-objective alignment offers a practical lens for analyzing cross-scale information flow and guiding representation-level control.

Method: Proposes Dynamical Manifold Evolution Theory with three interpretable metrics (state continuity C, attractor compactness Q, topological persistence P) to analyze latent dynamics across Transformer architectures.

Result: Reveals consistent links between latent dynamics and text quality: smoother trajectories correlate with fluency, richer topological organization with coherence. Different models show distinct dynamical regimes, and decoding parameters shape trajectories predictably.

Conclusion: DMET offers a testable phenomenological framework for interpreting, monitoring, and guiding LLM behavior, providing insights into the relationship between internal dynamics and generation quality.

Abstract: This paper introduces the Dynamical Manifold Evolution Theory (DMET), a conceptual framework that models large language model (LLM) generation as a continuous trajectory evolving on a low-dimensional semantic manifold. The theory characterizes latent dynamics through three interpretable metrics-state continuity ($C$), attractor compactness ($Q$), and topological persistence ($P$)-which jointly capture the smoothness, stability, and structure of representation evolution. Empirical analyses across multiple Transformer architectures reveal consistent links between these latent dynamics and text quality: smoother trajectories correspond to greater fluency, and richer topological organization correlates with enhanced coherence. Different models exhibit distinct dynamical regimes, reflecting diverse strategies of semantic organization in latent space. Moreover, decoding parameters such as temperature and top-$p$ shape these trajectories in predictable ways, defining a balanced region that harmonizes fluency and creativity. As a phenomenological rather than first-principles framework, DMET provides a unified and testable perspective for interpreting, monitoring, and guiding LLM behavior, offering new insights into the interplay between internal representation dynamics and external text generation quality.

Method: Used over 600 distinct trolley problems as probes, introduced a taxonomy of moral rationales based on consequentialism and deontology to systematically classify LLM reasoning traces.

Result: LLM chains-of-thought tend to favor deontological principles (moral obligations), while post-hoc explanations shift toward consequentialist rationales (utility maximization).

Conclusion: The framework provides foundation for understanding how LLMs process ethical considerations, enabling safer and more interpretable deployment in high-stakes decision-making environments.

Abstract: As AI systems increasingly navigate applications in healthcare, law, and governance, understanding how they handle ethically complex scenarios becomes critical. Previous work has mainly examined the moral judgments in large language models (LLMs), rather than their underlying moral reasoning process. In contrast, we focus on a large-scale analysis of the moral reasoning traces provided by LLMs. Furthermore, unlike prior work that attempted to draw inferences from only a handful of moral dilemmas, our study leverages over 600 distinct trolley problems as probes for revealing the reasoning patterns that emerge within different LLMs. We introduce and test a taxonomy of moral rationales to systematically classify reasoning traces according to two main normative ethical theories: consequentialism and deontology. Our analysis reveals that LLM chains-of-thought tend to favor deontological principles based on moral obligations, while post-hoc explanations shift notably toward consequentialist rationales that emphasize utility. Our framework provides a foundation for understanding how LLMs process and articulate ethical considerations, an important step toward safe and interpretable deployment of LLMs in high-stakes decision-making environments. Our code is available at https://github.com/keenansamway/moral-lens .

Method: A framework that adapts pretrained token-level LMs to operate in sentence space by autoregressively predicting continuous embeddings of next sentences. Two embedding paradigms: semantic embeddings (via autoencoding) and contextual embeddings (via next-sentence prediction). Two inference regimes: Discretized (decode embeddings to text) and Continuous (reason entirely in embedding space).

Result: Contextual embeddings under continuous inference show competitive performance with Chain-of-Thought across mathematics, logic, commonsense, and planning domains while reducing inference-time FLOPs by half on average. Early signs of scalability and modular adaptation observed.

Conclusion: Pretrained language models can effectively transition to abstract, structured reasoning within latent embedding spaces, enabling more efficient and higher-level reasoning capabilities.

Abstract: Autoregressive language models (LMs) generate one token at a time, yet human reasoning operates over higher-level abstractions - sentences, propositions, and concepts. This contrast raises a central question- Can LMs likewise learn to reason over structured semantic units rather than raw token sequences? In this work, we investigate whether pretrained LMs can be lifted into such abstract reasoning spaces by building on their learned representations. We present a framework that adapts a pretrained token-level LM to operate in sentence space by autoregressively predicting continuous embeddings of next sentences. We explore two embedding paradigms inspired by classical representation learning: 1) semantic embeddings, learned via autoencoding to preserve surface meaning; and 2) contextual embeddings, trained via next-sentence prediction to encode anticipatory structure. We evaluate both under two inference regimes: Discretized, which decodes each predicted embedding into text before re-encoding; and Continuous, which reasons entirely in embedding space for improved efficiency. Across four domains - mathematics, logic, commonsense, and planning - contextual embeddings under continuous inference show competitive performance with Chain-of-Thought (CoT) while reducing inference-time FLOPs on average by half. We also present early signs of scalability and modular adaptation. Finally, to visualize latent trajectories, we introduce SentenceLens, a diagnostic tool that decodes intermediate model states into interpretable sentences. Together, our results indicate that pretrained LMs can effectively transition to abstract, structured reasoning within latent embedding spaces.

Method: Split choices into groups, ensemble LLM predictions across groups using attention mask and positional encoding, requiring no labeled data for tuning.

Result: Outperforms standard inference and baseline methods across three LLMs and datasets, comprehensively addressing confidence distortion.

Conclusion: Self-ensemble effectively solves LLM confidence distortion in multi-choice QA through plug-and-play ensembling without parameter tuning.

Abstract: Although Large Language Models (LLMs) perform well in general fields, they exhibit a confidence distortion problem on multi-choice question-answering (MCQA), particularly as the number of answer choices increases. Specifically, on MCQA with many choices, LLMs suffer from under-confidence in correct predictions and over-confidence in incorrect ones, leading to a substantially degraded performance. To solve this problem, we propose Self-ensemble in this work. Our method splits the choices into several groups and ensembles LLM predictions across these groups to reach a final decision. The advantage of Self-ensemble is its plug-and-play nature, where it can be integrated into existing LLM architecture based on a designed attention mask and positional encoding, without requiring labeled datasets for parameter tuning. Experimental results on three LLMs and datasets demonstrate that Self-ensemble comprehensively addresses the confidence distortion problem of LLMs, outperforming standard inference as well as baseline methods.

Method: Directly finetunes base model guided by safety-aligned Ref-Teacher that filters harmful prompts from user data and distills safety-alignment knowledge into base model.

Result: Effectively minimizes harmful outputs and enhances finetuning accuracy for user-specific tasks.

Conclusion: Provides practical solution for secure and reliable deployment of LLMs in FaaS by addressing harmful finetuning attacks through teacher-guided finetuning.

Abstract: Recently, major AI providers such as Google and OpenAI have introduced Finetuning-as-a-Service (FaaS), which allows users to customize Large Language Models (LLMs) using their own data. However, this service is vulnerable to safety degradation when user data includes harmful prompts, a threat known as harmful finetuning attacks. Prior works attempt to mitigate this issue by first constructing safety-aligned model and then finetuning the model on user data. However, we observe that the safety-aligned weights provide weak initialization for downstream task learning, leading to suboptimal safety-alignment and downstream task performance. To address this, we propose a Refusal-Teacher (Ref-Teacher)-guided finetuning framework. Instead of finetuning a safety-aligned model on user data, our approach directly finetunes the base model under the guidance of a safety-aligned Ref-Teacher, which filters harmful prompts from user data and distills safety-alignment knowledge into the base model. Extensive experiments demonstrate that our Ref-Teacher-guided finetuning strategy effectively minimizes harmful outputs and enhances finetuning accuracy for user-specific tasks, offering a practical solution for secure and reliable deployment of LLMs in FaaS.

Method: Two-step approach: 1) Open-ended per-instance issue analysis using LLMs, 2) Clustering discovered issues using an intuitive cumulative algorithm to generate structured error type reports.

Result: LLM-as-a-qualitative-judge matches human-annotated issues in 2/3 cases, produces error type reports resembling human reports, and in a case study substantially improved NLG system performance.

Conclusion: LLM-as-a-qualitative-judge effectively provides qualitative insights for NLG system improvement, bridging the gap between quantitative evaluation and actionable development guidance.

Abstract: Prompting large language models (LLMs) to evaluate generated text, known as LLM-as-a-judge, has become a standard evaluation approach in natural language generation (NLG), but is primarily used as a quantitative tool, i.e. with numerical scores as main outputs. In this work, we propose LLM-as-a-qualitative-judge, an LLM-based evaluation approach with the main output being a structured report of common issue types in the NLG system outputs. Our approach is targeted at providing developers with meaningful insights on what improvements can be done to a given NLG system and consists of two main steps, namely open-ended per-instance issue analysis and clustering of the discovered issues using an intuitive cumulative algorithm. We also introduce a strategy for evaluating the proposed approach, coupled with ~300 annotations of issues in instances from 12 NLG datasets. Our results show that instance-specific issues output by LLM-as-a-qualitative-judge match those annotated by humans in 2/3 cases, and that LLM-as-a-qualitative-judge is capable of producing error type reports resembling the reports composed by human annotators. We also demonstrate in a case study how the use of LLM-as-a-qualitative-judge can substantially improve NLG systems performance. Our code and data are publicly available at https://github.com/tunde-ajayi/llm-as-a-qualitative-judge.

Method: Proposed framework with six dimensions (Clarity, Evidence, Emotional Appeal, Rebuttal, Audience Adaptation, Fairness), annotated 4,214 CS instances, and developed multi-task and dependency-based classification strategies.

Result: Achieved strong results (0.94 and 0.96 average F1 respectively) on both expert- and user-written counter-speech, outperforming standard baselines and revealing strong interdependence among dimensions.

Conclusion: The framework provides an effective computational approach for assessing counter-speech effectiveness with strong classification performance.

Abstract: Counter-speech (CS) is a key strategy for mitigating online Hate Speech (HS), yet defining the criteria to assess its effectiveness remains an open challenge. We propose a novel computational framework for CS effectiveness classification, grounded in linguistics, communication and argumentation concepts. Our framework defines six core dimensions - Clarity, Evidence, Emotional Appeal, Rebuttal, Audience Adaptation, and Fairness - which we use to annotate 4,214 CS instances from two benchmark datasets, resulting in a novel linguistic resource released to the community. In addition, we propose two classification strategies, multi-task and dependency-based, achieving strong results (0.94 and 0.96 average F1 respectively on both expert- and user-written CS), outperforming standard baselines, and revealing strong interdependence among dimensions.

Method: Empirical analysis of representation alignment in LLMs’ middle layers, followed by proposing ITLC - a novel method using latent injection for cross-lingual language control.

Result: Confirmed existence of natural representation alignment in LLMs, demonstrated ITLC’s strong cross-lingual control capabilities while preserving semantic integrity, and showed effectiveness in mitigating language confusion.

Conclusion: This work advances understanding of representation alignment in LLMs and provides a practical solution for enhancing both monolingual and cross-lingual performance through inference-time language control.

Abstract: Large Language Models (LLMs) have demonstrated remarkable generalization capabilities across tasks and languages, revolutionizing natural language processing. This paper investigates the naturally emerging representation alignment in LLMs, particularly in the middle layers, and its implications for disentangling language-specific and language-agnostic information. We empirically confirm the existence of this alignment, analyze its behavior in comparison to explicitly designed alignment models, and demonstrate its potential for language-specific manipulation without semantic degradation. Building on these findings, we propose Inference-Time Language Control (ITLC), a novel method that leverages latent injection to enable precise cross-lingual language control and mitigate language confusion in LLMs. Our experiments highlight ITLC’s strong cross-lingual control capabilities while preserving semantic integrity in target languages. Furthermore, we demonstrate its effectiveness in alleviating the cross-lingual language confusion problem, which persists even in current large-scale LLMs, leading to inconsistent language generation. This work advances our understanding of representation alignment in LLMs and introduces a practical solution for enhancing their monolingual and cross-lingual performance.

Method: Created a structured, difficulty-aware benchmark with two task families: multilingual financial reasoning (cross-lingual evidence integration) and financial OCR (extracting structured text from scanned documents).

Result: Evaluating 21 leading LLMs showed even frontier models like GPT-4o achieve only 46.01% overall, performing better on vision and audio but dropping sharply in multilingual settings.

Conclusion: The findings expose persistent limitations in multilingual, multimodal, and expert-level financial reasoning, highlighting the need for more comprehensive evaluation frameworks.

Abstract: Real-world financial analysis involves information across multiple languages and modalities, from reports and news to scanned filings and meeting recordings. Yet most existing evaluations of LLMs in finance remain text-only, monolingual, and largely saturated by current models. To bridge these gaps, we present MultiFinBen, the first expert-annotated multilingual (five languages) and multimodal (text, vision, audio) benchmark for evaluating LLMs in realistic financial contexts. MultiFinBen introduces two new task families: multilingual financial reasoning, which tests cross-lingual evidence integration from filings and news, and financial OCR, which extracts structured text from scanned documents containing tables and charts. Rather than aggregating all available datasets, we apply a structured, difficulty-aware selection based on advanced model performance, ensuring balanced challenge and removing redundant tasks. Evaluating 21 leading LLMs shows that even frontier multimodal models like GPT-4o achieve only 46.01% overall, stronger on vision and audio but dropping sharply in multilingual settings. These findings expose persistent limitations in multilingual, multimodal, and expert-level financial reasoning. All datasets, evaluation scripts, and leaderboards are publicly released.

Method: Systematic experiments comparing traditional CoT exemplars and enhanced CoT exemplars (constructed using answers from advanced models like Qwen2.5-Max and DeepSeek-R1) against Zero-Shot CoT on recent strong models.

Result: Both traditional and enhanced CoT exemplars fail to improve reasoning performance compared to Zero-Shot CoT. Models tend to ignore exemplars and focus primarily on instructions, showing no observable gain in reasoning ability.

Conclusion: Current ICL+CoT framework has limitations in mathematical reasoning, calling for re-examination of the ICL paradigm and exemplar definition for stronger models.

Abstract: In-Context Learning (ICL) is an essential emergent ability of Large Language Models (LLMs), and recent studies introduce Chain-of-Thought (CoT) to exemplars of ICL to enhance the reasoning capability, especially in mathematics tasks. However, given the continuous advancement of model capabilities, it remains unclear whether CoT exemplars still benefit recent, stronger models in such tasks. Through systematic experiments, we find that for recent strong models such as the Qwen2.5 series, adding traditional CoT exemplars does not improve reasoning performance compared to Zero-Shot CoT. Instead, their primary function is to align the output format with human expectations. We further investigate the effectiveness of enhanced CoT exemplars, constructed using answers from advanced models such as \texttt{Qwen2.5-Max} and \texttt{DeepSeek-R1}. Experimental results indicate that these enhanced exemplars still fail to improve the model’s reasoning performance. Further analysis reveals that models tend to ignore the exemplars and focus primarily on the instructions, leading to no observable gain in reasoning ability. Overall, our findings highlight the limitations of the current ICL+CoT framework in mathematical reasoning, calling for a re-examination of the ICL paradigm and the definition of exemplars.

Method: Created MFTCXplain dataset with 3,000 tweets across Portuguese, Italian, Persian, and English, annotated with hate speech labels, moral categories, and text span-level rationales using Moral Foundations Theory.

Result: LLMs show misalignment with human moral reasoning - perform well in hate speech detection (F1 up to 0.836) but poorly in predicting moral sentiments (F1 < 0.35), with limited rationale alignment especially in underrepresented languages.

Conclusion: Current LLMs have limited capacity to internalize and reflect human moral reasoning, highlighting the need for improved multilingual moral reasoning capabilities.

Abstract: Ensuring the moral reasoning capabilities of Large Language Models (LLMs) is a growing concern as these systems are used in socially sensitive tasks. Nevertheless, current evaluation benchmarks present two major shortcomings: a lack of annotations that justify moral classifications, which limits transparency and interpretability; and a predominant focus on English, which constrains the assessment of moral reasoning across diverse cultural settings. In this paper, we introduce MFTCXplain, a multilingual benchmark dataset for evaluating the moral reasoning of LLMs via multi-hop hate speech explanation using the Moral Foundations Theory. MFTCXplain comprises 3,000 tweets across Portuguese, Italian, Persian, and English, annotated with binary hate speech labels, moral categories, and text span-level rationales. Our results show a misalignment between LLM outputs and human annotations in moral reasoning tasks. While LLMs perform well in hate speech detection (F1 up to 0.836), their ability to predict moral sentiments is notably weak (F1 < 0.35). Furthermore, rationale alignment remains limited mainly in underrepresented languages. Our findings show the limited capacity of current LLMs to internalize and reflect human moral reasoning

Method: Proposed Doc2SAR framework that integrates domain-specific tools with multimodal LLMs enhanced via supervised fine-tuning, plus introduced DocSAR-200 benchmark with 200 annotated scientific documents for evaluation.

Result: Doc2SAR achieves state-of-the-art performance with 80.78% Table Recall on DocSAR-200, exceeding GPT-4o by 51.48%, and demonstrates practical usability with efficient inference and web app.

Conclusion: The synergistic approach combining specialized tools with fine-tuned MLLMs effectively addresses SAR extraction challenges and significantly outperforms end-to-end baselines across diverse document types.

Abstract: Extracting molecular structure-activity relationships (SARs) from scientific literature and patents is essential for drug discovery and materials research. However, this task remains challenging due to heterogeneous document formats and limitations of existing methods. Specifically, rule-based approaches relying on rigid templates fail to generalize across diverse document layouts, while general-purpose multimodal large language models (MLLMs) lack sufficient accuracy and reliability for specialized tasks, such as layout detection and optical chemical structure recognition (OCSR). To address these challenges, we introduce DocSAR-200, a rigorously annotated benchmark of 200 scientific documents designed specifically for evaluating SAR extraction methods. Additionally, we propose Doc2SAR, a novel synergistic framework that integrates domain-specific tools with MLLMs enhanced via supervised fine-tuning (SFT). Extensive experiments demonstrate that Doc2SAR achieves state-of-the-art performance across various document types, significantly outperforming leading end-to-end baselines. Specifically, Doc2SAR attains an overall Table Recall of 80.78% on DocSAR-200, exceeding end2end GPT-4o by 51.48%. Furthermore, Doc2SAR demonstrates practical usability through efficient inference and is accompanied by a web app.

Method: PhoniTale performs IPA-based phonological adaptation and syllable-aware alignment to retrieve L1 keyword sequences, then uses LLMs to generate verbal cues for vocabulary mnemonics.

Result: Evaluation through automated metrics and human recall tests shows PhoniTale consistently outperforms previous automated approaches and achieves quality comparable to human-written mnemonics.

Conclusion: The proposed PhoniTale system effectively addresses the limitations of previous methods by incorporating proper phonological guidance and alignment, demonstrating superior performance in generating helpful vocabulary learning mnemonics.

Abstract: Vocabulary acquisition poses a significant challenge for second-language (L2) learners, especially when learning typologically distant languages such as English and Korean, where phonological and structural mismatches complicate vocabulary learning. Recently, large language models (LLMs) have been used to generate keyword mnemonics by leveraging similar keywords from a learner’s first language (L1) to aid in acquiring L2 vocabulary. However, most methods still rely on direct IPA-based phonetic matching or employ LLMs without phonological guidance. In this paper, we present PhoniTale, a novel cross-lingual mnemonic generation system that performs IPA-based phonological adaptation and syllable-aware alignment to retrieve L1 keyword sequence and uses LLMs to generate verbal cues. We evaluate PhoniTale through automated metrics and a short-term recall test with human participants, comparing its output to human-written and prior automated mnemonics. Our findings show that PhoniTale consistently outperforms previous automated approaches and achieves quality comparable to human-written mnemonics.

Method: Proposed Learnable Calibration method tailored for on-device settings with limited computational resources, focusing on resource-efficient solutions for compositional multi-tasking.

Result: Created a benchmark comprising four practically relevant compositional tasks to facilitate research in this setting.

Conclusion: The contributions lay groundwork for advancing LLM capabilities in real-world multi-tasking scenarios and expanding applicability to complex, resource-constrained use cases.

Abstract: Adapter parameters provide a mechanism to modify the behavior of machine learning models and have gained significant popularity in the context of large language models (LLMs) and generative AI. These parameters can be merged to support multiple tasks via a process known as task merging. However, prior work on merging in LLMs, particularly in natural language processing, has been limited to scenarios where each test example addresses only a single task. In this paper, we focus on on-device settings and study the problem of text-based compositional multi-tasking, where each test example involves the simultaneous execution of multiple tasks. For instance, generating a translated summary of a long text requires solving both translation and summarization tasks concurrently. To facilitate research in this setting, we propose a benchmark comprising four practically relevant compositional tasks. We also present an efficient method (Learnable Calibration) tailored for on-device applications, where computational resources are limited, emphasizing the need for solutions that are both resource-efficient and high-performing. Our contributions lay the groundwork for advancing the capabilities of LLMs in real-world multi-tasking scenarios, expanding their applicability to complex, resource-constrained use cases.

Method: Systematic exploration of reducing human priors using Dafny formal language, featuring automatic data curation pipeline and RL designs integrated with formal verifier feedback. Introduces DafnyComp benchmark for compositional formal programs.

Result: SFT enables small models (0.5B) to generate syntactically valid and verifiable Dafny code, outperforming proprietary models. RL with regularization further improves performance and generalization to out-of-domain tasks, achieving best results on DafnyComp benchmark.

Conclusion: Formal language-based reasoning with automatic verification provides scalable and reliable approach for software verification, reducing dependency on human priors while enabling small models to outperform larger proprietary ones.

Abstract: Existing informal language-based (e.g., human language) Large Language Models (LLMs) trained with Reinforcement Learning (RL) face a significant challenge: their verification processes, which provide crucial training signals, are neither reliable nor scalable. In fact, the prevalent large proprietary models could hardly generate verifiable programs. A promising yet largely uncharted alternative is formal language-based reasoning. Grounding LLMs in rigorous formal systems where generative models operate in formal language spaces (e.g., Dafny) enables the automatic and mathematically provable verification of their reasoning processes and outcomes. This capability is pivotal for achieving large-scale, reliable formal software verification. It is a common practice to employ human-annotated chain-of-thought and other human priors to induce the reasoning and coding capabilities of LLMs. Unfortunately, it becomes unacceptably all-consuming to provide such priors for supervising complex programming tasks. In this work, we systematically explore ways to reduce human priors with the formal language, Dafny, as the main environment for our pilot study. Our pipeline mainly relies on introducing an automatic and scalable data curation pipeline, and careful RL designs integrated with feedback from the formal language verifier. We introduce DafnyComp, a benchmark of compositional formal programs with auto-formalized specifications for specification reasoning. Our supervised fine-tuning (SFT) stage enables even small models (e.g., 0.5B) to generate syntactically valid and verifiable Dafny code, surpassing proprietary models. RL with regularization further improves performance, achieving stronger generalization to out-of-domain tasks and outperforming all strong baselines on the challenging DafnyComp benchmark.

Method: Propose TriangleMix - a static attention pattern that uses dense attention in some layers and switches to Triangle attention in others, based on gradient analysis showing decoding-time contribution sparsity.

Result: For 128K inputs, Triangle attention achieves 15.3x speedup in attention computation, outperforming typical dynamic sparse methods (1.9x-3.4x). TriangleMix can be combined with dynamic sparsity for additional 6%-19% TTFT reduction.

Conclusion: TriangleMix effectively reduces attention overhead while preserving performance, and can be seamlessly integrated with existing dynamic sparsity methods for further acceleration.

Abstract: Large Language Models (LLMs) incur quadratic attention complexity with input length, creating a major time bottleneck in the prefilling stage. Existing acceleration methods largely exploit attention score sparsity by estimating blocks with high attention scores and applying dynamic sparse attention. In this work, we identify another untapped form of sparsity in the prefilling stage, namely decoding-time contribution sparsity, where many attention blocks exhibit nontrivial attention scores during prefilling yet contribute negligibly to subsequent decoding, as indicated by gradient-based analysis. Building on this observation, we propose TriangleMix, a training-free static attention pattern that uses dense attention in a subset of layers and switches to Triangle attention in the others. Extensive experiments show that TriangleMix preserves nearly lossless performance relative to dense attention while substantially reducing attention overhead in Triangle layers. For 128K inputs, Triangle attention achieves a 15.3x speedup in attention computation, significantly exceeding the acceleration of typical dynamic sparse methods (1.9x to 3.4x). Furthermore, TriangleMix can be seamlessly combined with dynamic sparsity approaches, delivering an additional 6% to 19% reduction in TTFT over using dynamic sparsity alone.

Method: Propose an add-on module that removes instance-level principal components from embeddings, transforming them to approximate Gaussian properties without modifying original training data.

Result: Evaluations on 8 benchmark datasets show improved adversarial robustness while maintaining comparable before-attack accuracy to baselines.

Conclusion: The approach achieves a balanced trade-off between robustness and generalization without requiring adversarial examples or costly training augmentation.

Abstract: Pre-trained language models (PLMs) have driven substantial progress in natural language processing but remain vulnerable to adversarial attacks, raising concerns about their robustness in real-world applications. Previous studies have sought to mitigate the impact of adversarial attacks by introducing adversarial perturbations into the training process, either implicitly or explicitly. While both strategies enhance robustness, they often incur high computational costs. In this work, we propose a simple yet effective add-on module that enhances the adversarial robustness of PLMs by removing instance-level principal components, without relying on conventional adversarial defences or perturbing the original training data. Our approach transforms the embedding space to approximate Gaussian properties, thereby reducing its susceptibility to adversarial perturbations while preserving semantic relationships. This transformation aligns embedding distributions in a way that minimises the impact of adversarial noise on decision boundaries, enhancing robustness without requiring adversarial examples or costly training-time augmentation. Evaluations on eight benchmark datasets show that our approach improves adversarial robustness while maintaining comparable before-attack accuracy to baselines, achieving a balanced trade-off between robustness and generalisation.

Method: Developed RaR framework with multi-step retrieval and reasoning, evaluated 25 LLMs across diverse architectures and scales using expert-curated radiology questions from RSNA-RadioQA, ExtendedQA, and real-world board examination questions.

Result: RaR significantly improved mean diagnostic accuracy over zero-shot prompting and conventional RAG, with greatest gains in small-scale models. Reduced hallucinations by mean 9.4% and retrieved clinically relevant context in 46% of cases. Even clinically fine-tuned models showed benefits.

Conclusion: RaR enhances factuality and diagnostic accuracy in radiology QA, demonstrating that retrieval remains beneficial despite embedded domain knowledge. Framework is publicly available to support clinical translation.

Abstract: Clinical decision-making in radiology increasingly benefits from artificial intelligence (AI), particularly through large language models (LLMs). However, traditional retrieval-augmented generation (RAG) systems for radiology question answering (QA) typically rely on single-step retrieval, limiting their ability to handle complex clinical reasoning tasks. Here we propose radiology Retrieval and Reasoning (RaR), a multi-step retrieval and reasoning framework designed to improve diagnostic accuracy, factual consistency, and clinical reliability of LLMs in radiology question answering. We evaluated 25 LLMs spanning diverse architectures, parameter scales (0.5B to >670B), and training paradigms (general-purpose, reasoning-optimized, clinically fine-tuned), using 104 expert-curated radiology questions from previously established RSNA-RadioQA and ExtendedQA datasets. To assess generalizability, we additionally tested on an unseen internal dataset of 65 real-world radiology board examination questions. RaR significantly improved mean diagnostic accuracy over zero-shot prompting and conventional online RAG. The greatest gains occurred in small-scale models, while very large models (>200B parameters) demonstrated minimal changes (<2% improvement). Additionally, RaR retrieval reduced hallucinations (mean 9.4%) and retrieved clinically relevant context in 46% of cases, substantially aiding factual grounding. Even clinically fine-tuned models showed gains from RaR (e.g., MedGemma-27B), indicating that retrieval remains beneficial despite embedded domain knowledge. These results highlight the potential of RaR to enhance factuality and diagnostic accuracy in radiology QA, warranting future studies to validate their clinical utility. All datasets, code, and the full RaR framework are publicly available to support open research and clinical translation.

Method: Proposed DEM-MoE (Demographic-Aware Mixture of Experts) that routes inputs to expert subnetworks based on annotator demographics, and uses LLM-generated synthetic annotations via zero-shot persona prompting for data imputation.

Result: DEM-MoE performs competitively across demographic groups, especially on datasets with high annotator disagreement. Synthetic judgments align moderately well with human annotations and optimal data blending strategies depend on dataset structure.

Conclusion: The combination of architectural innovations (DEM-MoE) and data-centric approaches (synthetic data generation) improves representation of diverse perspectives in subjective NLP tasks.

Abstract: We present an approach to modeling annotator disagreement in subjective NLP tasks through both architectural and data-centric innovations. Our model, DEM-MoE (Demographic-Aware Mixture of Experts), routes inputs to expert subnetworks based on annotator demographics, enabling it to better represent structured, group-level variation compared to prior models. DEM-MoE consistently performs competitively across demographic groups, and shows especially strong results on datasets with high annotator disagreement. To address sparse demographic coverage, we test whether LLM-generated synthetic annotations via zero-shot persona prompting can be used for data imputation. We show these synthetic judgments align moderately well with human annotations on our data and offer a scalable way to potentially enrich training data. We then propose and evaluate approaches for blending real and synthetic data using strategies tailored to dataset structure. We find that the optimal strategies depend on dataset structure. Together, these contributions improve the representation of diverse perspectives.

Method: Proposes Thinking with Nothinking Calibration (JointThinking) - generates two answers in parallel (Thinking and Nothinking modes), triggers second thinking round only when responses are inconsistent using a single prompt with two different answers.

Result: Significantly outperforms few-shot CoT, thinking twice and majority voting; achieves comparable in-distribution performance to training-based SOTA while substantially outperforming on out-of-distribution tasks; shows strong scalability as model size increases.

Conclusion: JointThinking demonstrates the importance of structural thinking diversity and consistency checks, with strong scalability and promising directions for future ICL research in reasoning LLMs.

Abstract: Reasoning large language models (RLLMs) have recently demonstrated remarkable capabilities through structured and multi-step reasoning. While prior research has primarily focused on improving their training and inference strategies, their potential for in-context learning (ICL) remains largely underexplored. To fill this gap, we propose Thinking with Nothinking Calibration (JointThinking), a new ICL paradigm that prompts the model to generate two answers in parallel: one in Thinking mode and the other in Nothinking mode. A second round of Thinking is triggered only when the two initial responses are inconsistent, using a single prompt with two different answers. Extensive experiments across multiple reasoning benchmarks demonstrate that JointThinking significantly outperforms few-shot chain-of-thought (CoT), thinking twice and majority voting. Moreover, it achieves comparable in-distribution performance to training-based SOTA reasoning method, while substantially outperforming on out-of-distribution tasks. We further conduct a systematic analysis of the calibration mechanism, showing the importance of structural thinking diversity and the benefits of consistency check. Additionally, we observe that the performance gap between actual and ideal reasoning narrows as model size increases in the second thinking, indicating the strong scalability of our approach. Finally, we discuss current limitations and outline promising directions for future ICL research in RLLMs.

Method: Created a manually translated Bangla multi-step reasoning dataset from the English Reveal dataset, then conducted controlled evaluation of English-centric and Bangla-centric multilingual models on both original and translated versions.

Result: Reasoning context is beneficial for challenging non-binary questions, but models struggle to effectively employ relevant Bangla reasoning steps. Different trends were observed across models and languages.

Conclusion: Models have difficulty leveraging reasoning steps in Bangla effectively, highlighting the need for better cross-lingual reasoning capabilities and evaluation in low-resource languages.

Abstract: Language models have demonstrated remarkable performance on complex multi-step reasoning tasks. However, their evaluation has been predominantly confined to high-resource languages such as English. In this paper, we introduce a manually translated Bangla multi-step reasoning dataset derived from the English Reveal dataset, featuring both binary and non-binary question types. We conduct a controlled evaluation of English-centric and Bangla-centric multilingual small language models on the original dataset and our translated version to compare their ability to exploit relevant reasoning steps to produce correct answers. Our results show that, in comparable settings, reasoning context is beneficial for more challenging non-binary questions, but models struggle to employ relevant Bangla reasoning steps effectively. We conclude by exploring how reasoning steps contribute to models’ predictions, highlighting different trends across models and languages.

Method: Proposed two developer-role-based attacks: D-Attack (enhances contextual simulation) and DH-CoT (strengthens attacks with deceptive chain-of-thought), plus MDH filtering method combining LLM screening and human verification.

Result: Developer messages significantly improve jailbreak attack success rates, and MDH reliably filters low-quality samples from datasets.

Conclusion: Developer-role-based approaches are effective for jailbreaking reasoning models, and proper dataset filtering is crucial for accurate attack evaluation.

Abstract: Jailbreaking commercial black-box models is one of the most challenging and serious security threats today. Existing attacks achieve certain success on non-reasoning models but perform limitedly on the latest reasoning models. We discover that carefully crafted developer messages can markedly boost jailbreak effectiveness. Building on this, we propose two developer-role-based attacks: D-Attack, which enhances contextual simulation, and DH-CoT, which strengthens attacks with deceptive chain-of-thought. In experiments, we further diccover that current red-teaming datasets often contain samples unsuited for measuring attack gains: prompts that fail to trigger defenses, prompts where malicious content is not the sole valid output, and benign prompts. Such data hinders accurate measurement of the true improvement brought by an attack method. To address this, we introduce MDH, a Malicious content Detection approach combining LLM-based screening with Human verification to balance accuracy and cost, with which we clean data and build the RTA dataset series. Experiments demonstrate that MDH reliably filters low-quality samples and that developer messages significantly improve jailbreak attack success. Codes, datasets, and other results will be released in https://github.com/AlienZhang1996/DH-CoT.

Method: ICE transforms prefix-only prompting into in-place prompting for dLLMs by integrating prompts directly within masked token positions during iterative refinement and using confidence-aware early exit mechanism.

Result: ICE achieves up to 17.29% accuracy improvement with 4.12x speedup on GSM8K, and up to 276.67x acceleration on MMLU while maintaining competitive performance.

Conclusion: ICE demonstrates the effectiveness of in-place prompting for diffusion LLMs, enabling significant computational efficiency and performance improvements through bidirectional attention mechanisms.

Abstract: Despite large language models (LLMs) have achieved remarkable success, their prefix-only prompting paradigm and sequential generation process offer limited flexibility for bidirectional information. Diffusion large language models (dLLMs) present new opportunities through their bidirectional attention mechanisms and iterative refinement processes, enabling more flexible in-place prompting strategies. We introduce ICE (In-Place Chain-of-Thought Prompting with Early Exit), a novel framework that transforms prefix-only prompting into in-place prompting specifically designed for dLLMs. ICE integrates in-place prompts directly within masked token positions during iterative refinement and employs a confidence-aware early exit mechanism to significantly reduce computational overhead. Extensive experiments demonstrate ICE’s effectiveness, achieving up to 17.29% accuracy improvement with 4.12$\times$ speedup on GSM8K, and up to 276.67$\times$ acceleration on MMLU while maintaining competitive performance.

Method: A three-layer hypergraph-based retrieval-augmented generation framework that unifies EEG domain knowledge, individual patient cases, and a large-scale repository into a traversable n-ary relational hypergraph.

Result: EEG-MedRAG significantly outperforms TimeRAG and HyperGraphRAG in answer accuracy and retrieval, demonstrating strong potential for real-world clinical decision support.

Conclusion: The framework enables joint semantic-temporal retrieval and causal-chain diagnostic generation, with publicly available data and code for further research and application.

Abstract: With the widespread application of electroencephalography (EEG) in neuroscience and clinical practice, efficiently retrieving and semantically interpreting large-scale, multi-source, heterogeneous EEG data has become a pressing challenge. We propose EEG-MedRAG, a three-layer hypergraph-based retrieval-augmented generation framework that unifies EEG domain knowledge, individual patient cases, and a large-scale repository into a traversable n-ary relational hypergraph, enabling joint semantic-temporal retrieval and causal-chain diagnostic generation. Concurrently, we introduce the first cross-disease, cross-role EEG clinical QA benchmark, spanning seven disorders and five authentic clinical perspectives. This benchmark allows systematic evaluation of disease-agnostic generalization and role-aware contextual understanding. Experiments show that EEG-MedRAG significantly outperforms TimeRAG and HyperGraphRAG in answer accuracy and retrieval, highlighting its strong potential for real-world clinical decision support. Our data and code are publicly available at https://github.com/yi9206413-boop/EEG-MedRAG.

Method: Created a multilabel Hebrew corpus with manual annotations of 12 social identities, benchmarked multilabel and single-label encoders alongside Hebrew-tuned LLMs (2B-9B parameters), and applied the classifier to analyze politicians’ posts and parliamentary speeches.

Result: Hebrew-tuned LLMs achieved the best performance (macro-F1 = 0.74). Analysis revealed differences in popularity, temporal trends, clustering patterns, and gender-related variations in identity expression, with comparison between elite discourse and public identity priorities.

Conclusion: HebID provides a comprehensive foundation for studying social identities in Hebrew and serves as a model for similar research in other non-English political contexts, addressing the gap in nuanced identity analysis beyond English-centric approaches.

Abstract: Political language is deeply intertwined with social identities. While social identities are often shaped by specific cultural contexts and expressed through particular uses of language, existing datasets for group and identity detection are predominantly English-centric, single-label and focus on coarse identity categories. We introduce HebID, the first multilabel Hebrew corpus for social identity detection: 5,536 sentences from Israeli politicians’ Facebook posts (Dec 2018-Apr 2021), manually annotated for twelve nuanced social identities (e.g. Rightist, Ultra-Orthodox, Socially-oriented) grounded by survey data. We benchmark multilabel and single-label encoders alongside 2B-9B-parameter generative LLMs, finding that Hebrew-tuned LLMs provide the best results (macro-$F_1$ = 0.74). We apply our classifier to politicians’ Facebook posts and parliamentary speeches, evaluating differences in popularity, temporal trends, clustering patterns, and gender-related variations in identity expression. We utilize identity choices from a national public survey, enabling a comparison between identities portrayed in elite discourse and the public’s identity priorities. HebID provides a comprehensive foundation for studying social identities in Hebrew and can serve as a model for similar research in other non-English political contexts.

Method: Created a Hebrew-language corpus of 10,410 sentences from Knesset speeches, Facebook posts, and news outlets, with manual annotations. Developed a two-stage classification pipeline combining finetuned encoder models and decoder LLMs (DictaLM 2.0).

Result: Best model achieved F1 of 0.74 for binary PDD detection and macro-F1 of 0.67 for classification of delegitimization characteristics. Analysis revealed rising PDD over three decades, higher prevalence on social media, greater use by male politicians, and stronger tendencies among right-leaning actors.

Conclusion: Automated PDD analysis is feasible and valuable for understanding democratic discourse, with applications for tracking political discourse patterns across platforms and time.

Abstract: We present the first large-scale computational study of political delegitimization discourse (PDD), defined as symbolic attacks on the normative validity of political entities. We curate and manually annotate a novel Hebrew-language corpus of 10,410 sentences drawn from Knesset speeches (1993-2023), Facebook posts (2018-2021), and leading news outlets, of which 1,812 instances (17.4%) exhibit PDD and 642 carry additional annotations for intensity, incivility, target type, and affective framing. We introduce a two-stage classification pipeline combining finetuned encoder models and decoder LLMs. Our best model (DictaLM 2.0) attains an F$_1$ of 0.74 for binary PDD detection and a macro-F$_1$ of 0.67 for classification of delegitimization characteristics. Applying this classifier to longitudinal and cross-platform data, we see a marked rise in PDD over three decades, higher prevalence on social media versus parliamentary debate, greater use by male than female politicians, and stronger tendencies among right-leaning actors - with pronounced spikes during election campaigns and major political events. Our findings demonstrate the feasibility and value of automated PDD analysis for understanding democratic discourse.

Method: Created MCPVerse benchmark with 550+ real-world executable tools and 140k+ token action space, using outcome-based evaluation with real-time ground truth for time-sensitive tasks. Tested state-of-the-art LLMs in three modes: Oracle, Standard, and Max-Scale.

Result: Most models suffer performance degradation with larger tool sets, but agentic models like Claude-4-Sonnet can effectively leverage expanded exploration spaces to improve accuracy.

Conclusion: MCPVerse exposes limitations of current models in complex real-world scenarios and serves as a critical benchmark for advancing agentic tool use capabilities.

Abstract: Large Language Models (LLMs) are evolving from text generators into reasoning agents. This transition makes their ability to use external tools a critical capability. However, evaluating this skill presents a significant challenge. Existing benchmarks are often limited by their reliance on synthetic tools and severely constrained action spaces. To address these limitations, we introduce MCPVerse, an expansive, real-world benchmark for evaluating agentic tool use. MCPVerse integrates more than 550 real-world, executable tools to create an unprecedented action space exceeding 140k tokens, and employs outcome-based evaluation with real-time ground truth for time-sensitive tasks. We benchmarked the state-of-the-art LLMs across three modes (Oracle, Standard, and Max-Scale), revealing that while most models suffer performance degradation when confronted with larger tool sets, the agentic models, such as Claude-4-Sonnet, can effectively leverage expanded exploration spaces to improve accuracy. This finding not only exposes the limitations of state-of-the-art models in complex, real-world scenarios but also establishes MCPVerse as a critical benchmark for measuring and advancing agentic tool use capabilities.

Method: Used a curated dataset of 500,000+ text samples from 5,000 authors with identified musical preferences to build advanced personality assessment models.

Result: Revealed significant personality differences across fans of five musical genres.

Conclusion: Musical preferences do leave detectable traces in language through personality characteristics, providing resources for interdisciplinary research.

Abstract: Music serves as a powerful reflection of individual identity, often aligning with deeper psychological traits. Prior research has established correlations between musical preferences and personality, while separate studies have demonstrated that personality is detectable through linguistic analysis. Our study bridges these two research domains by investigating whether individuals’ musical preferences leave traces in their spontaneous language through the lens of the Big Five personality traits (Openness, Conscientiousness, Extroversion, Agreeableness, and Neuroticism). Using a carefully curated dataset of over 500,000 text samples from nearly 5,000 authors with reliably identified musical preferences, we build advanced models to assess personality characteristics. Our results reveal significant personality differences across fans of five musical genres. We release resources for future research at the intersection of computational linguistics, music psychology and personality analysis.

Method: ChronoRAG refines document information into coherent passages and explicitly captures/maintains temporal order among retrieved passages.

Result: Substantial improvements on NarrativeQA and GutenQA datasets, especially for tasks requiring factual identification and comprehension of complex sequential relationships.

Conclusion: Reasoning over temporal order is crucial for resolving narrative QA tasks, and ChronoRAG effectively addresses this need.

Abstract: Long-context question answering over narrative tasks is challenging because correct answers often hinge on reconstructing a coherent timeline of events while preserving contextual f low in a limited context window. Retrievalaugmented generation (RAG) methods aim to address this challenge by selectively retrieving only necessary document segments. However, narrative texts possess unique characteristics that limit the effectiveness of these existing approaches. Specifically, understanding narrative texts requires more than isolated segments, as the broader context and sequential relationships between segments are crucial for comprehension. To address these limitations, we propose ChronoRAG, a novel RAG framework specialized for narrative texts. This approach focuses on two essential aspects: refining dispersed document information into coherent and structured passages and preserving narrative flow by explicitly capturing and maintaining the temporal order among retrieved passages. We empirically demonstrate the effectiveness of ChronoRAG through experiments on the NarrativeQA and GutenQAdataset, showing substantial improvements in tasks requiring both factual identification and comprehension of complex sequential relationships, underscoring that reasoning over temporal order is crucial in resolving narrative QA.

Method: Proposes Distillation-Guided Policy Optimization (DGPO) with cold-start initialization from teacher demonstrations and continuous teacher guidance during policy optimization, plus ARC metric for fine-grained capability analysis.

Result: DGPO enables compact models to achieve sophisticated agentic search behaviors, sometimes outperforming larger teacher models, making agentic RAG feasible in resource-constrained environments.

Conclusion: DGPO successfully addresses RL challenges for compact models, enabling sophisticated agentic RAG behaviors in computing resource-constrained settings through effective distillation and guidance techniques.

Abstract: Reinforcement Learning has emerged as a dominant post-training approach to elicit agentic RAG behaviors such as search and planning from language models. Despite its success with larger models, applying RL to compact models (e.g., 0.5–1B parameters) presents unique challenges. The compact models exhibit poor initial performance, resulting in sparse rewards and unstable training. To overcome these difficulties, we propose Distillation-Guided Policy Optimization (DGPO), which employs cold-start initialization from teacher demonstrations and continuous teacher guidance during policy optimization. To understand how compact models preserve agentic behavior, we introduce Agentic RAG Capabilities (ARC), a fine-grained metric analyzing reasoning, search coordination, and response synthesis. Comprehensive experiments demonstrate that DGPO enables compact models to achieve sophisticated agentic search behaviors, even outperforming the larger teacher model in some cases. DGPO makes agentic RAG feasible in computing resource-constrained environments.

Method: Four prompting approaches: 1) basic role prompting, 2) improved role prompting, 3) automatic prompt optimization (APO), and 4) rule-based role prompting (RRP) with character-card/scene-contract design and strict function calling enforcement.

Result: Rule-based role prompting achieved best performance with overall score of 0.571, improving on zero-shot baseline score of 0.519. RRP outperformed more elaborate methods like APO.

Conclusion: RRP design substantially improves effectiveness and reliability of role-playing dialogue agents. All best-performing prompts and APO tool are open-sourced to support future persona prompt development.

Abstract: This report investigates approaches for prompting a tool-augmented large language model (LLM) to act as a role-playing dialogue agent in the API track of the Commonsense Persona-grounded Dialogue Challenge (CPDC) 2025. In this setting, dialogue agents often produce overly long in-character responses (over-speaking) while failing to use tools effectively according to the persona (under-acting), such as generating function calls that do not exist or making unnecessary tool calls before answering. We explore four prompting approaches to address these issues: 1) basic role prompting, 2) improved role prompting, 3) automatic prompt optimization (APO), and 4) rule-based role prompting. The rule-based role prompting (RRP) approach achieved the best performance through two novel techniques-character-card/scene-contract design and strict enforcement of function calling-which led to an overall score of 0.571, improving on the zero-shot baseline score of 0.519. These findings demonstrate that RRP design can substantially improve the effectiveness and reliability of role-playing dialogue agents compared with more elaborate methods such as APO. To support future efforts in developing persona prompts, we are open-sourcing all of our best-performing prompts and the APO tool Source code is available at https://github.com/scb-10x/apo

Method: Used representation analysis to study attention heads and neuronal activation patterns, examining how reasoning patterns affect safety mechanisms during training and inference.

Result: Discovered that specific attention heads facilitate refusal by reducing attention to CoT tokens, and found significant activation entanglement between reasoning and safety in safety-critical neurons after fine-tuning with reasoning patterns.

Conclusion: RIM vulnerability emerges from mechanistic interactions between reasoning and safety systems, with neuronal entanglement correlating with catastrophic forgetting, providing a neuron-level explanation for misalignment.

Abstract: With the growing accessibility and wide adoption of large language models, concerns about their safety and alignment with human values have become paramount. In this paper, we identify a concerning phenomenon: Reasoning-Induced Misalignment (RIM), in which misalignment emerges when reasoning capabilities strengthened-particularly when specific types of reasoning patterns are introduced during inference or training. Beyond reporting this vulnerability, we provide the first mechanistic account of its origins. Through representation analysis, we discover that specific attention heads facilitate refusal by reducing their attention to CoT tokens, a mechanism that modulates the model’s rationalization process during inference. During training, we find significantly higher activation entanglement between reasoning and safety in safety-critical neurons than in control neurons, particularly after fine-tuning with those identified reasoning patterns. This entanglement strongly correlates with catastrophic forgetting, providing a neuron-level explanation for RIM.

Method: REFRAG compresses, senses, and expands the context by exploiting the block-diagonal attention patterns in RAG, eliminating unnecessary computations over irrelevant retrieved passages.

Result: Achieves 30.85× acceleration in time-to-first-token (3.75× improvement over previous work) without perplexity loss, and extends LLM context size by 16× while maintaining accuracy across various tasks.

Conclusion: REFRAG provides substantial speedup with no accuracy loss across diverse long-context tasks, demonstrating that exploiting RAG-specific sparsity patterns can significantly improve system efficiency.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in leveraging extensive external knowledge to enhance responses in multi-turn and agentic applications, such as retrieval-augmented generation (RAG). However, processing long-context inputs introduces significant system latency and demands substantial memory for the key-value cache, resulting in reduced throughput and a fundamental trade-off between knowledge enrichment and system efficiency. While minimizing latency for long-context inputs is a primary objective for LLMs, we contend that RAG require specialized consideration. In RAG, much of the LLM context consists of concatenated passages from retrieval, with only a small subset directly relevant to the query. These passages often exhibit low semantic similarity due to diversity or deduplication during re-ranking, leading to block-diagonal attention patterns that differ from those in standard LLM generation tasks. Based on this observation, we argue that most computations over the RAG context during decoding are unnecessary and can be eliminated with minimal impact on performance. To this end, we propose REFRAG, an efficient decoding framework that compresses, senses, and expands to improve latency in RAG applications. By exploiting the sparsity structure, we demonstrate a 30.85 the time-to-first-token acceleration (3.75 improvement to previous work) without loss in perplexity. In addition, our optimization framework for large context enables REFRAG to extend the context size of LLMs by 16. We provide rigorous validation of REFRAG across diverse long-context tasks, including RAG, multi-turn conversations, and long document summarization, spanning a wide range of datasets. Experimental results confirm that REFRAG delivers substantial speedup with no loss in accuracy compared to LLaMA models and other state-of-the-art baselines across various context sizes.

Method: Three core technologies: 1) Dynamic expert scheduling for intelligent computational resource allocation, 2) Hierarchical Sparse Attention (HSA) that reduces complexity from O(n²) to O(n), 3) Soft absorption guidance with DPM-solver++ to reduce diffusion steps.

Result: Comprehensive experiments on various long-text generation benchmarks demonstrate superiority over existing state-of-the-art methods.

Conclusion: DrDiff successfully overcomes the efficiency-quality trade-off in long-text generation through its three core technologies, achieving better performance than current SOTA methods.

Abstract: This paper introduces DrDiff, a novel framework for long-text generation that overcomes the efficiency-quality trade-off through three core technologies. First, we design a dynamic expert scheduling mechanism that intelligently allocates computational resources during the diffusion process based on text complexity, enabling more efficient handling of text generation tasks of varying difficulty. Second, we introduce a Hierarchical Sparse Attention (HSA) mechanism that adaptively adjusts attention patterns according to a variety of input lengths, reducing computational complexity from O($n^2$) to O($n$) while maintaining model performance. Finally, we propose a soft absorption guidance optimization strategy that combines with DPM-solver++ to reduce diffusion steps, significantly improving generation speed. Comprehensive experiments on various long-text generation benchmarks demonstrate the superiority of our DrDiff over the existing SOTA methods.

Method: SPFT-SQL introduces two key components: 1) Verification-based iterative fine-tuning before self-play to synthesize high-quality data and build models with varying capabilities, 2) Error-driven loss during self-play that incentivizes learning from opponent model’s incorrect outputs to distinguish between correct and erroneous SQL.

Result: Extensive experiments on six open-source LLMs and five benchmarks show that SPFT-SQL outperforms existing state-of-the-art methods.

Conclusion: The proposed SPFT-SQL method effectively addresses the limitations of standard self-play fine-tuning for Text-to-SQL tasks by incorporating verification-based data synthesis and error-driven learning, achieving superior performance compared to current approaches.

Abstract: Despite the significant advancements of self-play fine-tuning (SPIN), which can transform a weak large language model (LLM) into a strong one through competitive interactions between models of varying capabilities, it still faces challenges in the Text-to-SQL task. SPIN does not generate new information, and the large number of correct SQL queries produced by the opponent model during self-play reduces the main model’s ability to generate accurate SQL queries. To address this challenge, we propose a new self-play fine-tuning method tailored for the Text-to-SQL task, called SPFT-SQL. Prior to self-play, we introduce a verification-based iterative fine-tuning approach, which synthesizes high-quality fine-tuning data iteratively based on the database schema and validation feedback to enhance model performance, while building a model base with varying capabilities. During the self-play fine-tuning phase, we propose an error-driven loss method that incentivizes incorrect outputs from the opponent model, enabling the main model to distinguish between correct SQL and erroneous SQL generated by the opponent model, thereby improving its ability to generate correct SQL. Extensive experiments and in-depth analyses on six open-source LLMs and five widely used benchmarks demonstrate that our approach outperforms existing state-of-the-art (SOTA) methods.

Method: Replicated human color-naming studies using Gemini 2.0-flash and Llama 3.3-70B-Instruct, including English color-naming tasks and simulated cultural evolution through iterated in-context language learning.

Result: Gemini aligned well with English speakers’ naming patterns and achieved high IB-efficiency, while Llama showed efficient but lower complexity systems. Both LLMs restructured random systems toward greater IB-efficiency and cross-linguistic patterns.

Conclusion: LLMs are capable of evolving perceptually grounded, human-like semantic systems driven by the same Information Bottleneck efficiency principle that governs human languages.

Abstract: Converging evidence suggests that systems of semantic categories across human languages achieve near-optimal compression via the Information Bottleneck (IB) complexity-accuracy principle. Large language models (LLMs) are not trained for this objective, which raises the question: are LLMs capable of evolving efficient human-like semantic systems? To address this question, we focus on the domain of color as a key testbed of cognitive theories of categorization and replicate with LLMs (Gemini 2.0-flash and Llama 3.3-70B-Instruct) two influential human behavioral studies. First, we conduct an English color-naming study, showing that Gemini aligns well with the naming patterns of native English speakers and achieves a significantly high IB-efficiency score, while Llama exhibits an efficient but lower complexity system compared to English. Second, to test whether LLMs simply mimic patterns in their training data or actually exhibit a human-like inductive bias toward IB-efficiency, we simulate cultural evolution of pseudo color-naming systems in LLMs via iterated in-context language learning. We find that akin to humans, LLMs iteratively restructure initially random systems towards greater IB-efficiency and increased alignment with patterns observed across the world’s languages. These findings demonstrate that LLMs are capable of evolving perceptually grounded, human-like semantic systems, driven by the same fundamental principle that governs semantic efficiency across human languages.

Method: Two-stage approach: 1) Fine-tuning a large language model on benchmark datasets curated for spotlight generation, 2) Alignment via Direct Preference Optimization (DPO) to improve quality.

Result: The model precisely identifies key document elements while enhancing readability and significantly boosting engagement value compared to traditional approaches.

Conclusion: Spotlight paradigm successfully creates more engaging narratives by selectively emphasizing compelling content, demonstrating superior engagement value over traditional comprehensive summaries.

Abstract: In this paper, we introduce Spotlight, a novel paradigm for information extraction that produces concise, engaging narratives by highlighting the most compelling aspects of a document. Unlike traditional summaries, which prioritize comprehensive coverage, spotlights selectively emphasize intriguing content to foster deeper reader engagement with the source material. We formally differentiate spotlights from related constructs and support our analysis with a detailed benchmarking study using new datasets curated for this work. To generate high-quality spotlights, we propose a two-stage approach: fine-tuning a large language model on our benchmark data, followed by alignment via Direct Preference Optimization (DPO). Our comprehensive evaluation demonstrates that the resulting model not only identifies key elements with precision but also enhances readability and boosts the engagement value of the original document.

Method: SpanEIT builds span-based representations for entities and sentiment phrases, uses bidirectional attention for fine-grained interactions, employs graph attention networks for syntactic and co-occurrence relations, and includes a coreference-aware memory module for entity-level consistency.

Result: Experiments on FSAD, BARU, and IMDB datasets show SpanEIT outperforms state-of-the-art transformer and hybrid baselines in accuracy and F1 scores. Ablation studies confirm the effectiveness of the proposed components.

Conclusion: SpanEIT demonstrates strong potential for fine-grained sentiment analysis applications like social media monitoring and customer feedback analysis, with validated effectiveness through comprehensive experiments and interpretability analyses.

Abstract: Entity-level sentiment classification involves identifying the sentiment polarity linked to specific entities within text. This task poses several challenges: effectively modeling the subtle and complex interactions between entities and their surrounding sentiment expressions; capturing dependencies that may span across sentences; and ensuring consistent sentiment predictions for multiple mentions of the same entity through coreference resolution. Additionally, linguistic phenomena such as negation, ambiguity, and overlapping opinions further complicate the analysis. These complexities make entity-level sentiment classification a difficult problem, especially in real-world, noisy textual data. To address these issues, we propose SpanEIT, a novel framework integrating dynamic span interaction and graph-aware memory mechanisms for enhanced entity-sentiment relational modeling. SpanEIT builds span-based representations for entities and candidate sentiment phrases, employs bidirectional attention for fine-grained interactions, and uses a graph attention network to capture syntactic and co-occurrence relations. A coreference-aware memory module ensures entity-level consistency across documents. Experiments on FSAD, BARU, and IMDB datasets show SpanEIT outperforms state-of-the-art transformer and hybrid baselines in accuracy and F1 scores. Ablation and interpretability analyses validate the effectiveness of our approach, underscoring its potential for fine-grained sentiment analysis in applications like social media monitoring and customer feedback analysis.

Method: Developed HalluDetect hallucination detection system and benchmarked five mitigation architectures including AgentBot.

Result: HalluDetect achieved 68.92% F1 score (22.47% improvement over baselines). AgentBot minimized hallucinations to 0.4159 per turn with 96.13% token accuracy.

Conclusion: Optimized inference strategies can significantly improve factual accuracy, providing a scalable framework for hallucination mitigation in LLMs.

Abstract: Large Language Models (LLMs) are widely used in industry but remain prone to hallucinations, limiting their reliability in critical applications. This work addresses hallucination reduction in consumer grievance chatbots built using LLaMA 3.1 8B Instruct, a compact model frequently used in industry. We develop HalluDetect, an LLM-based hallucination detection system that achieves an F1 score of 68.92% outperforming baseline detectors by 22.47%. Benchmarking five hallucination mitigation architectures, we find that out of them, AgentBot minimizes hallucinations to 0.4159 per turn while maintaining the highest token accuracy (96.13%), making it the most effective mitigation strategy. Our findings provide a scalable framework for hallucination mitigation, demonstrating that optimized inference strategies can significantly improve factual accuracy.

Method: Extract a task-aware steering vector from KL divergence gradient between warm-started and pre-trained models, then use it to guide decoding process toward task distribution.

Result: Improves multiple-choice accuracy by up to 5 percentage points and open-ended truthfulness by 2 percentage points across three tasks and nine benchmarks, with similar gains on commonsense datasets.

Conclusion: SVDecode offers a lightweight, theoretically grounded path to stronger task adaptation for large language models without adding trainable parameters beyond PEFT adapters.

Abstract: Adapting billion-parameter language models to a downstream task is still costly, even with parameter-efficient fine-tuning (PEFT). We re-cast task adaptation as output-distribution alignment: the objective is to steer the output distribution toward the task distribution directly during decoding rather than indirectly through weight updates. Building on this view, we introduce Steering Vector Decoding (SVDecode), a lightweight, PEFT-compatible, and theoretically grounded method. We start with a short warm-start fine-tune and extract a task-aware steering vector from the Kullback-Leibler (KL) divergence gradient between the output distribution of the warm-started and pre-trained models. This steering vector is then used to guide the decoding process to steer the model’s output distribution towards the task distribution. We theoretically prove that SVDecode is first-order equivalent to the gradient step of full fine-tuning and derive a globally optimal solution for the strength of the steering vector. Across three tasks and nine benchmarks, SVDecode paired with four standard PEFT methods improves multiple-choice accuracy by up to 5 percentage points and open-ended truthfulness by 2 percentage points, with similar gains (1-2 percentage points) on commonsense datasets without adding trainable parameters beyond the PEFT adapter. SVDecode thus offers a lightweight, theoretically grounded path to stronger task adaptation for large language models.

Method: EpiCache uses block-wise prefill to bound cache growth, episodic KV compression that clusters conversation history into coherent episodes for episode-specific eviction, and adaptive layer-wise budget allocation based on each layer’s sensitivity to eviction.

Result: Across three LongConvQA benchmarks, EpiCache improves accuracy by up to 40%, maintains near-full KV accuracy under 4-6x compression, and reduces latency/memory by up to 2.4x/3.5x.

Conclusion: EpiCache enables efficient multi-turn interaction under strict resource limits by effectively managing KV cache memory while preserving conversational context and accuracy.

Abstract: Modern large language models (LLMs) extend context lengths to millions of tokens, enabling coherent, personalized responses grounded in long conversational histories. This ability, however, hinges on Key-Value (KV) caching, whose memory grows linearly with dialogue length and quickly becomes the bottleneck in resource-constrained environments. An active line of research for reducing memory bottleneck is KV cache compression, which seeks to limit cache size while preserving accuracy. Yet existing methods face two major limitations: (i) evicting the KV cache after full-context prefill causes unbounded peak memory, and (ii) query-dependent eviction narrows the cache to a single query, leading to failure cases in multi-turn conversations. We introduce EpiCache, a training-free KV cache management framework for long conversational question answering (LongConvQA) under fixed memory budgets. EpiCache bounds cache growth through block-wise prefill and preserves topic-relevant context via episodic KV compression, which clusters conversation history into coherent episodes and applies episode-specific KV cache eviction. We further design an adaptive layer-wise budget allocation strategy that measures each layer’s sensitivity to eviction and distributes the memory budget across layers accordingly. Across three LongConvQA benchmarks, EpiCache improves accuracy by up to 40%, maintains near-full KV accuracy under 4-6x compression, and reduces latency/memory by up to 2.4x/3.5x, enabling efficient multi-turn interaction under strict resource limits. Our code is available at https://github.com/apple/ml-epicache.

Method: Created annotated corpus of 16,112 tokens and applied Conditional Random Fields (CRF) machine learning technique for POS tagging.

Result: Achieved overall tagging accuracy of 85.70% with precision of 86%, recall of 86%, and f1-score of 85%.

Conclusion: Successfully demonstrated first POS tagging system for Nagamese language using CRF approach with promising results.

Abstract: This paper investigates part-of-speech tagging, an important task in Natural Language Processing (NLP) for the Nagamese language. The Nagamese language, a.k.a. Naga Pidgin, is an Assamese-lexified Creole language developed primarily as a means of communication in trade between the Nagas and people from Assam in northeast India. A substantial amount of work in part-of-speech-tagging has been done for resource-rich languages like English, Hindi, etc. However, no work has been done in the Nagamese language. To the best of our knowledge, this is the first attempt at part-of-speech tagging for the Nagamese Language. The aim of this work is to identify the part-of-speech for a given sentence in the Nagamese language. An annotated corpus of 16,112 tokens is created and applied machine learning technique known as Conditional Random Fields (CRF). Using CRF, an overall tagging accuracy of 85.70%; precision, recall of 86%, and f1-score of 85% is achieved. Keywords. Nagamese, NLP, part-of-speech, machine learning, CRF.

Method: Recursive Thematic Partitioning (RTP) leverages Large Language Models to interactively build a binary tree where each node is a natural language question that semantically partitions the data, creating an interpretable taxonomy.

Result: RTP’s question-driven hierarchy is more interpretable than BERTopic’s keyword-based topics and serves as powerful features in downstream classification tasks, especially when themes correlate with task labels. Thematic paths can also structure generative model prompts.

Conclusion: RTP shifts text analysis from statistical pattern discovery to knowledge-driven thematic analysis and enables structured synthesis through controllable generative prompts based on discovered corpus characteristics.

Abstract: Unsupervised analysis of text corpora is challenging, especially in data-scarce domains where traditional topic models struggle. While these models offer a solution, they typically describe clusters with lists of keywords that require significant manual effort to interpret and often lack semantic coherence. To address this critical interpretability gap, we introduce Recursive Thematic Partitioning (RTP), a novel framework that leverages Large Language Models (LLMs) to interactively build a binary tree. Each node in the tree is a natural language question that semantically partitions the data, resulting in a fully interpretable taxonomy where the logic of each cluster is explicit. Our experiments demonstrate that RTP’s question-driven hierarchy is more interpretable than the keyword-based topics from a strong baseline like BERTopic. Furthermore, we establish the quantitative utility of these clusters by showing they serve as powerful features in downstream classification tasks, particularly when the data’s underlying themes correlate with the task labels. RTP introduces a new paradigm for data exploration, shifting the focus from statistical pattern discovery to knowledge-driven thematic analysis. Furthermore, we demonstrate that the thematic paths from the RTP tree can serve as structured, controllable prompts for generative models. This transforms our analytical framework into a powerful tool for synthesis, enabling the consistent imitation of specific characteristics discovered in the source corpus.

Method: KLCF uses pretrained knowledge boundaries to construct fact checklists for improving factual coverage, and trains a self-assessment module based on base model’s internal knowledge to enhance factual precision. It employs Dual-Fact Alignment mechanism and is fully external-knowledge-free.

Result: KLCF substantially improves factuality metrics across multiple long-form benchmarks and effectively alleviates model hallucinations.

Conclusion: The proposed KLCF framework provides an efficient and scalable solution to address LLM hallucinations by focusing on internal knowledge consistency, outperforming prior methods that rely on external retrieval or heavy verification.

Abstract: Hallucination and factuality deficits remain key obstacles to the reliability of large language models (LLMs) in long-form generation. Existing reinforcement learning from human feedback (RLHF) frameworks primarily rely on preference rewards, yet they often overlook the model’s internal knowledge boundaries, exacerbating the so-called “hallucination tax”. To address this challenge, we propose Knowledge-Level Consistency Reinforcement Learning Framework (KLCF), a novel framework that focuses on the knowledge consistency between the policy model’s expressed knowledge and the base model’s parametric knowledge, and introduces a Dual-Fact Alignment mechanism to jointly optimize factual recall and precision. Specifically, KLCF leverages pretrained knowledge boundaries to construct fact checklist, guiding online reinforcement learning to improve factual coverage and recall; simultaneously, it trains a self-assessment module based on the base model’s internal knowledge to enhance factual precision during generation. Unlike prior methods that rely on external retrieval or heavy verification, our reward design is fully external-knowledge-free and lightweight, making KLCF efficient and easily scalable to large-scale training. Experimental results demonstrate that KLCF substantially improves factuality metrics across multiple long-form benchmarks and effectively alleviates model hallucinations.

Method: MemGen consists of a memory trigger that monitors reasoning state to decide explicit memory invocation, and a memory weaver that takes the agent’s current state as stimulus to construct latent token sequences as machine-native memory to enrich reasoning.

Result: Extensive experiments across eight benchmarks show MemGen surpasses leading external memory systems (ExpeL, AWM) by up to 38.22%, exceeds GRPO by up to 13.44%, and exhibits strong cross-domain generalization ability.

Conclusion: MemGen spontaneously evolves distinct human-like memory faculties including planning memory, procedural memory, and working memory without explicit supervision, suggesting an emergent trajectory toward more naturalistic forms of machine cognition.

Abstract: Agent memory shapes how Large Language Model (LLM)-powered agents, akin to the human brain, progressively refine themselves through environment interactions. Existing paradigms remain constrained: parametric memory forcibly adjusts model parameters, and retrieval-based memory externalizes experience into structured databases, yet neither captures the fluid interweaving of reasoning and memory that underlies human cognition. To address this gap, we propose MemGen, a dynamic generative memory framework that equips agents with a human-esque cognitive faculty. It consists of a \textit{memory trigger}, which monitors the agent’s reasoning state to decide explicit memory invocation, and a \textit{memory weaver}, which takes the agent’s current state as stimulus to construct a latent token sequence as machine-native memory to enrich its reasoning. In this way, MemGen enables agents to recall and augment latent memory throughout reasoning, producing a tightly interwoven cycle of memory and cognition. Extensive experiments across eight benchmarks show that MemGen surpasses leading external memory systems such as ExpeL and AWM by up to $38.22%$, exceeds GRPO by up to $13.44%$, and exhibits strong cross-domain generalization ability. More importantly, we find that without explicit supervision, MemGen spontaneously evolves distinct human-like memory faculties, including planning memory, procedural memory, and working memory, suggesting an emergent trajectory toward more naturalistic forms of machine cognition.

Method: Adapted IRE framework from educational theory for dialogue diagnosis, developed graph-based encoding that integrates teacher questions with knowledge components to capture key information.

Result: Experiments on three real-world dialogue datasets show DiaCDM significantly improves diagnostic accuracy and enhances interpretability of results.

Conclusion: DiaCDM provides teachers with a powerful tool for assessing students’ cognitive states in dialogue settings, marking the first exploration of cognitive diagnosis in dialogue contexts.

Abstract: While cognitive diagnosis (CD) effectively assesses students’ knowledge mastery from structured test data, applying it to real-world teacher-student dialogues presents two fundamental challenges. Traditional CD models lack a suitable framework for handling dynamic, unstructured dialogues, and it’s difficult to accurately extract diagnostic semantics from lengthy dialogues. To overcome these hurdles, we propose DiaCDM, an innovative model. We’ve adapted the initiation-response-evaluation (IRE) framework from educational theory to design a diagnostic framework tailored for dialogue. We also developed a unique graph-based encoding method that integrates teacher questions with relevant knowledge components to capture key information more precisely. To our knowledge, this is the first exploration of cognitive diagnosis in a dialogue setting. Experiments on three real-world dialogue datasets confirm that DiaCDM not only significantly improves diagnostic accuracy but also enhances the results’ interpretability, providing teachers with a powerful tool for assessing students’ cognitive states. The code is available at https://github.com/Mind-Lab-ECNU/DiaCDM/tree/main.

Method: Fine-tuned small language models for automated item alignment at domain and skill levels using data from college admissions tests, compared with embedding-based supervised machine learning models, and conducted semantic similarity analysis to understand misclassifications.

Result: SLMs consistently outperformed embedding-based models, especially for fine-grained skill alignment. Including more item text data substantially improved performance beyond sample size increases alone. Semantic analysis showed certain skills were too close, explaining misclassifications.

Conclusion: Fine-tuned SLMs are effective for automated item alignment and provide a more efficient alternative to human expert judgment, though semantic closeness between certain skills remains a challenge for perfect classification.

Abstract: Aligning test items to content standards is a critical step in test development to collect validity evidence based on content. Item alignment has typically been conducted by human experts. This judgmental process can be subjective and time-consuming. This study investigated the performance of fine-tuned small language models (SLMs) for automated item alignment using data from a large-scale standardized reading and writing test for college admissions. Different SLMs were trained for alignment at both domain and skill levels respectively with 10 skills mapped to 4 content domains. The model performance was evaluated in multiple criteria on two testing datasets. The impact of types and sizes of the input data for training was investigated. Results showed that including more item text data led to substantially better model performance, surpassing the improvements induced by sample size increase alone. For comparison, supervised machine learning models were trained using the embeddings from the multilingual-E5-large-instruct model. The study results showed that fine-tuned SLMs consistently outperformed the embedding-based supervised machine learning models, particularly for the more fine-grained skill alignment. To better understand model misclassifications, multiple semantic similarity analysis including pairwise cosine similarity, Kullback-Leibler divergence of embedding distributions, and two-dimension projections of item embeddings were conducted. These analyses consistently showed that certain skills in SAT and PSAT were semantically too close, providing evidence for the observed misclassification.

Method: REPO uses heterogeneous rewards: preference-trained RM for human alignment, RJ for persuasive behavior and SOP compliance, and programmatic RF for deterministic checks on numerics, formatting, and guardrails.

Result: REPO achieved average dialogue rating of 4.63 (+1.20 over base), increased excellent response conversations to 66.67%, and achieved 93.33% bad-case fix rate with 75.56% clean fixes, outperforming SFT, DPO, PPO, and GRPO.

Conclusion: REPO effectively aligns LLMs with complex business requirements for negotiation, demonstrating emergent capabilities like proactive empathy and calibrated tactics that surpass human annotations.

Abstract: We study deploying large language models (LLMs) as business development (BD) agents for persuasive price negotiation in online travel agencies (OTAs), where aligning traveler affordability and hotel profitability directly affects bookings, partner relationships, and access to travel. The agent must follow a Standard Operating Procedure (SOP) while conducting multi-turn persuasion, interpreting colloquial inputs, and adhering to guardrails (no over-promising, no hallucinations). Conventional post-training – supervised fine-tuning (SFT) or single-source reward optimization – overfits scripts, misses nuanced persuasive style, and fails to enforce verifiable business constraints. We propose Reward-Enhanced Policy Optimization (REPO), a reinforcement learning post-training framework that aligns an LLM with heterogeneous rewards: a preference-trained reward model (RM) for dense human alignment, a reward judge (RJ) for high-level persuasive behavior and SOP compliance, and programmatic reward functions (RF) for deterministic checks on numerics, formatting, and guardrails. A straightforward enhancement mechanism is proposed to combine the RM with RJ and RF signals to curb reward hacking and improve negotiation quality. In production-style evaluations – approximately 150 turns from real dialogues and 225 turns from curated bad-case dialogues – REPO lifts average dialogue rating to 4.63: +1.20 over base, +0.83 over Direct Preference Optimization (DPO); +0.33 over Group Relative Policy Optimization (GRPO), increases the share of conversations with at least one excellent response to 66.67% (+23.34 percentage points over GRPO), and achieves a 93.33% bad-case fix rate with 75.56% clean fixes, outperforming SFT, DPO, PPO, and GRPO. We also observe emergent capabilities – proactive empathy, localized reasoning, calibrated tactics – that surpass gold annotations.

Method: Developed a comprehensive evaluation framework for English texts, framing bias detection as a multi-label task using a demographic-focused taxonomy. Conducted systematic evaluation with models across scales and techniques including prompting, in-context learning, and fine-tuning using twelve datasets spanning diverse content types and demographics.

Result: The study demonstrates the promise of fine-tuned smaller models for scalable bias detection. However, analyses expose persistent gaps across demographic axes and multi-demographic targeted biases.

Conclusion: While fine-tuned smaller models show potential for scalable bias detection, there are significant gaps in detecting multi-demographic biases, underscoring the need for more effective and scalable auditing frameworks to address regulatory requirements.

Abstract: Large-scale web-scraped text corpora used to train general-purpose AI models often contain harmful demographic-targeted social biases, creating a regulatory need for data auditing and developing scalable bias-detection methods. Although prior work has investigated biases in text datasets and related detection methods, these studies remain narrow in scope. They typically focus on a single content type (e.g., hate speech), cover limited demographic axes, overlook biases affecting multiple demographics simultaneously, and analyze limited techniques. Consequently, practitioners lack a holistic understanding of the strengths and limitations of recent large language models (LLMs) for automated bias detection. In this study, we present a comprehensive evaluation framework aimed at English texts to assess the ability of LLMs in detecting demographic-targeted social biases. To align with regulatory requirements, we frame bias detection as a multi-label task using a demographic-focused taxonomy. We then conduct a systematic evaluation with models across scales and techniques, including prompting, in-context learning, and fine-tuning. Using twelve datasets spanning diverse content types and demographics, our study demonstrates the promise of fine-tuned smaller models for scalable detection. However, our analyses also expose persistent gaps across demographic axes and multi-demographic targeted biases, underscoring the need for more effective and scalable auditing frameworks.

Method: Three approaches were tested: 1) Classical supervised ML with embeddings and dimensionality reduction, 2) Fine-tuning eight BERT model variants for domain and skill alignment, 3) Ensemble learning with majority voting and stacking using multiple meta-models.

Result: DeBERTa-v3-base achieved highest F1 score (0.950) for domain alignment, RoBERTa-large achieved highest F1 score (0.869) for skill alignment. Ensemble models didn’t surpass best language models. Dimension reduction helped linear classifiers but not language models.

Conclusion: Fine-tuned language models demonstrated superior performance for automated item alignment compared to classical machine learning and ensemble methods, providing effective solutions for aligning assessment items to content standards.

Abstract: Accurate alignment of items to content standards is critical for valid score interpretation in large-scale assessments. This study evaluates three automated paradigms for aligning items with four domain and nineteen skill labels. First, we extracted embeddings and trained multiple classical supervised machine learning models, and further investigated the impact of dimensionality reduction on model performance. Second, we fine-tuned eight BERT model and its variants for both domain and skill alignment. Third, we explored ensemble learning with majority voting and stacking with multiple meta-models. The DeBERTa-v3-base achieved the highest weighted-average F1 score of 0.950 for domain alignment while the RoBERTa-large yielded the highest F1 score of 0.869 for skill alignment. Ensemble models did not surpass the best-performing language models. Dimension reduction enhanced linear classifiers based on embeddings but did not perform better than language models. This study demonstrated different methods in automated item alignment to content standards.}

Method: Created WebDetective benchmark with hint-free multi-hop questions and a controlled Wikipedia sandbox for full traceability, plus a holistic evaluation framework that separates search sufficiency, knowledge utilization, and refusal behavior.

Result: Evaluation of 25 state-of-the-art models revealed systematic weaknesses: models struggle with knowledge utilization despite sufficient evidence and show near-absent appropriate refusal when evidence is lacking. Models excel at executing given reasoning paths but fail at discovering them.

Conclusion: WebDetective’s diagnostic framework can guide architectural improvements, as demonstrated by EvidenceLoop workflow that incorporates verification loops and systematic evidence tracking. The benchmark is crucial for developing genuinely autonomous reasoning systems rather than pattern-following agents.

Abstract: RAG (Retrieval-Augmented Generation) systems and web agents are increasingly evaluated on multi-hop deep search tasks, yet current practice suffers from two major limitations. First, most benchmarks leak the reasoning path in the question text, allowing models to follow surface cues rather than discover reasoning chains autonomously. Second, evaluation is typically reduced to a single pass rate, which collapses diverse behaviours into one score and obscures whether failures stem from inadequate search, poor knowledge use, or inappropriate refusal. To address these issues, we present WebDetective, a benchmark of hint-free multi-hop questions paired with a controlled Wikipedia sandbox that ensures full traceability of model actions, and a holistic evaluation framework that separates search sufficiency, knowledge utilisation, and refusal behaviour. Our evaluation of 25 state-of-the-art models reveals systematic weaknesses across all architectures: models struggle with knowledge utilisation despite having sufficient evidence and demonstrate near-absent appropriate refusal when evidence is lacking. These patterns expose a fundamental gap: today’s systems excel at executing given reasoning paths but fail when required to discover them. We develop an agentic workflow, EvidenceLoop, that explicitly targets the challenges our benchmark identifies, incorporating verification loops and systematic evidence tracking that improve both search and synthesis capabilities. This baseline demonstrates that WebDetective’s diagnostic framework can guide concrete architectural improvements, establishing our benchmark as a critical tool for developing genuinely autonomous reasoning systems rather than pattern-following agents.

Method: Uses three trust metrics: Self-Consistency, Class Entailment Alignment, and LLM Agreement Scoring to create unified trust scores for selective distillation.

Result: Outperforms label-based learning and conventional distillation on USPTO-2M dataset in accuracy, stability, and interpretability.

Conclusion: Establishes reliable paradigm for leveraging reasoning-aware trust indicators in patent analytics.

Abstract: Large language models (LLMs) increasingly generate natural language rationales to enhance interpretability, but these often contain logical errors, label mismatches, and domain-specific misalignments. Directly using such rationales as supervision risks propagating noise and undermining training stability. To address this challenge, we introduce Self-Filtered Distillation, a framework specifically tailored for patent classification, which treats LLM-generated rationales as trust signals rather than ground-truth supervision. The framework employs selective distillation guided by three unsupervised trust metrics: (1) Self-Consistency, which measures the stability of LLM-generated rationales across multiple generations; (2) Class Entailment Alignment, which assesses semantic coherence with patent-specific class definitions; and (3) LLM Agreement Scoring, which validates rationale-label plausibility. These metrics are integrated into a unified trust score that primarily weights training samples while optionally filtering out extremely low-trust cases, enabling reasoning-aware supervision. Experiments on the USPTO-2M dataset, a widely used benchmark for patent classification, show that our method outperforms label-based learning and conventional distillation in accuracy, stability, and interpretability, establishing a reliable paradigm for leveraging reasoning-aware trust indicators in patent analytics.

Method: Used linear probes on final-token representations of LLMs and identified specific attention heads in the final Transformer layer that show opposite activation patterns for simple and difficult problems.

Result: Demonstrated that problem difficulty can be linearly modeled from LLM representations, with specific attention heads reliably distinguishing difficulty levels through opposite activation patterns.

Conclusion: LLMs possess structured difficulty perception capabilities that can be leveraged for automatic difficulty annotation, reducing reliance on human labeling in benchmark construction and curriculum learning.

Abstract: Large language models (LLMs) are increasingly deployed on complex reasoning tasks, yet little is known about their ability to internally evaluate problem difficulty, which is an essential capability for adaptive reasoning and efficient resource allocation. In this work, we investigate whether LLMs implicitly encode problem difficulty in their internal representations. Using a linear probe on the final-token representations of LLMs, we demonstrate that the difficulty level of math problems can be linearly modeled. We further locate the specific attention heads of the final Transformer layer: these attention heads have opposite activation patterns for simple and difficult problems, thus achieving perception of difficulty. Our ablation experiments prove the accuracy of the location. Crucially, our experiments provide practical support for using LLMs as automatic difficulty annotators, potentially substantially reducing reliance on costly human labeling in benchmark construction and curriculum learning. We also uncover that there is a significant difference in entropy and difficulty perception at the token level. Our study reveals that difficulty perception in LLMs is not only present but also structurally organized, offering new theoretical insights and practical directions for future research. Our code is available at https://github.com/Aegis1863/Difficulty-Perception-of-LLMs.

Method: Combined Centered Kernel Alignment (CKA) with REPINA regularization into TRepLiNa method, tested on Aya-23 8B with QLoRA across Mundari, Santali, Bhili language pairs with Hindi/English pivots in zero-shot, few-shot, and fine-tuning settings.

Result: Aligning mid-level layers using TRepLiNa (CKA+REPINA) proved to be a low-cost, practical approach that improves LRL translation, particularly in data-scarce environments.

Conclusion: The proposed TRepLiNa method effectively enhances translation quality for low-resource languages by enforcing cross-lingual similarity in specific internal layers of multilingual LLMs.

Abstract: The 2025 Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo) Language Challenge addresses one of India’s most pressing linguistic gaps: the lack of resources for its diverse low-resource languages (LRLs). In this study, we investigate whether enforcing cross-lingual similarity in specific internal layers of a decoder-only multilingual large language model (LLM) can improve translation quality from LRL to high-resource language (HRL). Specifically, we combine Centered Kernel Alignment (CKA), a similarity metric that encourages representations of different languages to align, with REPINA, a regularization method that constrains parameter updates to remain close to the pretrained model, into a joint method we call TRepLiNa. In this research project, we experiment with zero-shot, few-shot, and fine-tuning settings using Aya-23 8B with QLoRA across MMLoSo shared task language pairs (Mundari, Santali, Bhili) with Hindi/English pivots. Our results show that aligning mid-level layers using TRepLiNa (CKA+REPINA) is a low-cost, practical approach to improving LRL translation, especially in data-scarce settings.

Method: NHA integrates intra-layer and inter-layer hybridization using linear RNN for long-term context, sliding window for short-term tokens, and single softmax attention over all keys/values without additional parameters.

Result: NHA outperforms Transformers and other hybrid baselines on recall-intensive and commonsense reasoning tasks, and enables pretrained LLMs to achieve competitive accuracy with significant efficiency gains.

Conclusion: NHA provides an effective hybrid attention architecture that smoothly transitions between linear and full attention while maintaining structural uniformity and improving both efficiency and accuracy.

Abstract: Transformers excel at sequence modeling but face quadratic complexity, while linear attention offers improved efficiency but often compromises recall accuracy over long contexts. In this work, we introduce Native Hybrid Attention (NHA), a novel hybrid architecture of linear and full attention that integrates both intra & inter-layer hybridization into a unified layer design. NHA maintains long-term context in key-value slots updated by a linear RNN, and augments them with short-term tokens from a sliding window. A single \texttt{softmax attention} operation is then applied over all keys and values, enabling per-token and per-head context-dependent weighting without requiring additional fusion parameters. The inter-layer behavior is controlled through a single hyperparameter, the sliding window size, which allows smooth adjustment between purely linear and full attention while keeping all layers structurally uniform. Experimental results show that NHA surpasses Transformers and other hybrid baselines on recall-intensive and commonsense reasoning tasks. Furthermore, pretrained LLMs can be structurally hybridized with NHA, achieving competitive accuracy while delivering significant efficiency gains. Code is available at https://github.com/JusenD/NHA.

Method: Developed TRACE analyzer that: 1) decomposes thought process into minimally complete sub-thoughts, 2) infers discourse relationships to construct granular thought progression graphs, 3) identifies common thinking patterns for similar queries.

Result: Identified two major overthinking patterns: Explorer and Late Landing. Found that long-thinking models are 5-20x slower on simple tasks with no substantial gains. Over-verification and over-exploration are the primary drivers of overthinking.

Conclusion: Proposed a utility-based definition of overthinking that moves beyond length-based metrics, offering more insightful understanding of LLM thought progression and practical guidelines for principled overthinking management.

Abstract: Models employing long chain-of-thought (CoT) reasoning have shown superior performance on complex reasoning tasks. Yet, this capability introduces a critical and often overlooked inefficiency – overthinking – models often engage in unnecessarily extensive reasoning even for simple queries, incurring significant computations without accuracy improvements. While prior work has explored solutions to mitigate overthinking, a fundamental gap remains in our understanding of its underlying causes. Most existing analyses are limited to superficial, profiling-based observations, failing to delve into LLMs’ inner workings. This study introduces a systematic, fine-grained analyzer of LLMs’ thought process to bridge the gap, TRACE. We first benchmark the overthinking issue, confirming that long-thinking models are five to twenty times slower on simple tasks with no substantial gains. We then use TRACE to first decompose the thought process into minimally complete sub-thoughts. Next, by inferring discourse relationships among sub-thoughts, we construct granular thought progression graphs and subsequently identify common thinking patterns for topically similar queries. Our analysis reveals two major patterns for open-weight thinking models – Explorer and Late Landing. This finding provides evidence that over-verification and over-exploration are the primary drivers of overthinking in LLMs. Grounded in thought structures, we propose a utility-based definition of overthinking, which moves beyond length-based metrics. This revised definition offers a more insightful understanding of LLMs’ thought progression, as well as practical guidelines for principled overthinking management.

Method: Proposed an adaptive world model-enhanced reasoning mechanism that constructs dynamic textual world models to track entity states and temporal sequences. It monitors reasoning trajectories for confusion indicators and intervenes by providing clear world state descriptions.

Result: Evaluations on three social benchmarks show significant accuracy improvements (e.g., +10% in Hi-ToM) while reducing computational costs by up to 33.8% token reduction.

Conclusion: The world model-enhanced reasoning mechanism offers a simple yet effective solution for deploying LLMs in social contexts by helping models navigate cognitive dilemmas and distinguish between external events and internal beliefs.

Abstract: While large language models (LLMs) excel in mathematical and code reasoning, we observe they struggle with social reasoning tasks, exhibiting cognitive confusion, logical inconsistencies, and conflation between objective world states and subjective belief states. Through deteiled analysis of DeepSeek-R1’s reasoning trajectories, we find that LLMs frequently encounter reasoning impasses and tend to output contradictory terms like “tricky” and “confused” when processing scenarios with multiple participants and timelines, leading to erroneous reasoning or infinite loops. The core issue is their inability to disentangle objective reality from agents’ subjective beliefs. To address this, we propose an adaptive world model-enhanced reasoning mechanism that constructs a dynamic textual world model to track entity states and temporal sequences. It dynamically monitors reasoning trajectories for confusion indicators and promptly intervenes by providing clear world state descriptions, helping models navigate through cognitive dilemmas. The mechanism mimics how humans use implicit world models to distinguish between external events and internal beliefs. Evaluations on three social benchmarks demonstrate significant improvements in accuracy (e.g., +10% in Hi-ToM) while reducing computational costs (up to 33.8% token reduction), offering a simple yet effective solution for deploying LLMs in social contexts.

Method: Using language models to automatically extract linguistic features (processes, participants, circumstances) and analyze sequential patterns in narratives.

Result: Applied to dream narratives including a war veteran with PTSD, revealing distinctive patterns where verbal processes dominate over mental ones.

Conclusion: The framework successfully links linguistic choices to psychological states, demonstrating the relationship between style and subjective experience.

Abstract: Personal narratives are stories authors construct to make meaning of their experiences. Style, the distinctive way authors use language to express themselves, is fundamental to how these narratives convey subjective experiences. Yet there is a lack of a formal framework for systematically analyzing these stylistic choices. We present a novel approach that formalizes style in personal narratives as patterns in the linguistic choices authors make when communicating subjective experiences. Our framework integrates three domains: functional linguistics establishes language as a system of meaningful choices, computer science provides methods for automatically extracting and analyzing sequential patterns, and these patterns are linked to psychological observations. Using language models, we automatically extract linguistic features such as processes, participants, and circumstances. We apply our framework to hundreds of dream narratives, including a case study on a war veteran with post-traumatic stress disorder. Analysis of his narratives uncovers distinctive patterns, particularly how verbal processes dominate over mental ones, illustrating the relationship between linguistic choices and psychological states.

Method: dInfer decomposes the inference pipeline into four modular components (model, diffusion iteration manager, decoding strategy, KV-cache manager) with novel algorithms and system-level optimizations.

Result: Achieves over 1,100 tokens/sec on HumanEval and averages 800+ tokens/sec across six benchmarks on 8x H800 GPUs, with 10x speedup over Fast-dLLM and 2-3x speedup over optimized AR model QWen2.5-3B.

Conclusion: dInfer provides a standardized, efficient inference framework for dLLMs that significantly outperforms existing systems while maintaining model performance, enabling broader adoption of diffusion-based language models.

Abstract: Diffusion-based large language models (dLLMs) have emerged as a promising alternative to autoregressive (AR) LLMs, leveraging denoising-based generation to enable inherent parallelism. Even more and more open-sourced dLLM models emerge, yet their widespread adoption remains constrained by the lack of a standardized and efficient inference framework. We present dInfer, an efficient and extensible framework for dLLM inference. dInfer decomposes the inference pipeline into four modular components–model, diffusion iteration manager, decoding strategy, and KV-cache manager–and integrates novel algorithms for each component alongside system-level optimizations. Through this combination of algorithmic innovations and system enhancements, dInfer achieves substantial efficiency gains without compromising output quality on LLaDA-MoE. At batch size 1, it surpasses 1,100 tokens per second on HumanEval and averages over 800 tokens per second across six benchmarks on $8\times$ H800 GPUs. Compared to prior systems, dInfer delivers a $10\times$ speedup over Fast-dLLM while maintaining similar model performance. Even compared to the AR model (with a comparable number of activation parameters and performance) QWen2.5-3B, which is highly optimized with the latest vLLM inference engine, dInfer still delivers a $2$-$3\times$ speedup. The implementation of dInfer is open-sourced at https://github.com/inclusionAI/dInfer.

Method: SAC directly selects anchor tokens from original context and aggregates contextual information into their KV representations using anchor embeddings and bidirectional attention modification, eliminating autoencoding training.

Result: SAC consistently outperforms existing context compression methods across various compression ratios, achieving 1 EM improvement at 5x compression over strong baselines on MRQA, with increasing advantages at higher compression ratios.

Conclusion: SAC provides a more effective approach to context compression by directly leveraging contextual tokens through anchor selection and aggregation, avoiding the limitations of autoencoding-based methods.

Abstract: Context compression presents a promising approach for accelerating large language model (LLM) inference by compressing long contexts into compact representations. Current context compression methods predominantly rely on autoencoding tasks to train context-agnostic compression tokens to compress contextual semantics. While autoencoding tasks enable compression tokens to acquire compression capabilities, compression via autoencoding tasks creates a fundamental mismatch: the models are optimized for reconstruction that diverge from actual downstream tasks, thereby weakening the features more beneficial for real-world usage. We propose Semantic-Anchor Compression (SAC), a novel method that shifts from autoencoding task based compression to an architecture that is equipped with this compression capability \textit{a priori}. Instead of training models to compress contexts through autoencoding tasks, SAC directly selects so-called anchor tokens from the original context and aggregates contextual information into their key-value (KV) representations. By deriving representations directly from the contextual tokens, SAC eliminates the need for autoencoding training. To ensure compression performance while directly leveraging anchor tokens, SAC incorporates two key designs: (1) anchor embeddings that enable the compressor to identify critical tokens, and (2) bidirectional attention modification that allows anchor tokens to capture information from the entire context. Experimental results demonstrate that SAC consistently outperforms existing context compression methods across various compression ratios. On out-of-distribution evaluation using MRQA, SAC achieves 1 EM improvement at 5x compression over strong baselines, with increasing advantages at higher compression ratios.

Method: Introduce DITING framework with six evaluation dimensions, develop AgentEval multi-agent evaluation framework, create MetricAlign meta-evaluation dataset, and evaluate fourteen LLM models.

Result: Chinese-trained LLMs surpass larger foreign counterparts; DeepSeek-V3 delivers most faithful and stylistically coherent translations; AgentEval achieves highest correlation with human judgments among seven automatic metrics.

Conclusion: Establishes new paradigm for exploring LLM-based web novel translation and provides public resources to advance future research.

Abstract: Large language models (LLMs) have substantially advanced machine translation (MT), yet their effectiveness in translating web novels remains unclear. Existing benchmarks rely on surface-level metrics that fail to capture the distinctive traits of this genre. To address these gaps, we introduce DITING, the first comprehensive evaluation framework for web novel translation, assessing narrative and cultural fidelity across six dimensions: idiom translation, lexical ambiguity, terminology localization, tense consistency, zero-pronoun resolution, and cultural safety, supported by over 18K expert-annotated Chinese-English sentence pairs. We further propose AgentEval, a reasoning-driven multi-agent evaluation framework that simulates expert deliberation to assess translation quality beyond lexical overlap, achieving the highest correlation with human judgments among seven tested automatic metrics. To enable metric comparison, we develop MetricAlign, a meta-evaluation dataset of 300 sentence pairs annotated with error labels and scalar quality scores. Comprehensive evaluation of fourteen open, closed, and commercial models reveals that Chinese-trained LLMs surpass larger foreign counterparts, and that DeepSeek-V3 delivers the most faithful and stylistically coherent translations. Our work establishes a new paradigm for exploring LLM-based web novel translation and provides public resources to advance future research.

Method: Dynamic-filter Sequence-level Policy Optimization (DSPO) - an improved RL algorithm using sequence-level optimization and dynamic sample filtering, trained purely through RL without supervised demonstration data.

Result: DSPO-trained 7B model improves over comparable previous work by 34.1% across multiple QA benchmarks, outperforms 14B model from previous work in complex multihop QA by nearly 9% relative, while maintaining exceptional training stability.

Conclusion: DSPO effectively unlocks LLMs’ true agentic potential for complex knowledge-seeking tasks through robust RL training, demonstrating superior performance and stability compared to existing approaches.

Abstract: Enhancing LLMs with the ability to actively search external knowledge is crucial for complex and real-world tasks. Current approaches either rely on prompting to elicit the model’s innate agent capabilities, or suffer from performance ceilings and collapse when applying RL to complex interactive tasks, leaving their true agentic potential untapped. To address this, we introduce \textbf{D}ynamic-filter \textbf{S}equence-level \textbf{P}olicy \textbf{O}ptimization (DSPO), an improved RL algorithm designed for robust agent training through sequence-level optimization and dynamic sample filtering. We train our model purely through RL to interleave multi-turn search and reasoning, obviating the need for supervised demonstration data. Across multiple QA benchmarks, our DSPO-trained 7B model improves over a comparable previous work by \textbf{34.1%}, and even outperforms the 14B model from previous work in complex multihop QA such as HotpotQA by nearly \textbf{9% relative}, maintaining exceptional training stability.

Method: Proposed Sandwiched Policy Gradient (SPG) that leverages both upper and lower bounds of the true log-likelihood instead of one-sided approximations like ELBO.

Result: SPG significantly outperforms baselines, improving accuracy by 3.6% in GSM8K, 2.6% in MATH500, 18.4% in Countdown and 27.0% in Sudoku over state-of-the-art RL methods.

Conclusion: SPG provides an effective solution for RL alignment of diffusion LLMs by addressing the policy gradient bias issue through dual-bound approximation of log-likelihood.

Abstract: Diffusion large language models (dLLMs) are emerging as an efficient alternative to autoregressive models due to their ability to decode multiple tokens in parallel. However, aligning dLLMs with human preferences or task-specific rewards via reinforcement learning (RL) is challenging because their intractable log-likelihood precludes the direct application of standard policy gradient methods. While prior work uses surrogates like the evidence lower bound (ELBO), these one-sided approximations can introduce significant policy gradient bias. To address this, we propose the Sandwiched Policy Gradient (SPG) that leverages both an upper and a lower bound of the true log-likelihood. Experiments show that SPG significantly outperforms baselines based on ELBO or one-step estimation. Specifically, SPG improves the accuracy over state-of-the-art RL methods for dLLMs by 3.6% in GSM8K, 2.6% in MATH500, 18.4% in Countdown and 27.0% in Sudoku.

Method: Distill a large teacher ViT into a 5M-parameter TinyViT and fine-tune using metric-based few-shot learning (prototypical loss with 5-shot episodes), with Grad-CAM integration for explainable predictions.

Result: Achieves 91% accuracy and AUC ≥0.95 on multi-site dataset of 79 Batten-disease MRIs and 90 controls, outperforming 3D-ResNet and Swin-Tiny baselines with >90% sensitivity and ~90% specificity.

Conclusion: The model’s small size and strong performance demonstrate a practical AI solution for early Batten disease detection with explainable predictions.

Abstract: Batten disease (neuronal ceroid lipofuscinosis) is a rare pediatric neurodegenerative disorder whose early MRI signs are subtle and often missed. We propose TinyViT-Batten, a few-shot Vision Transformer (ViT) framework to detect early Batten disease from pediatric brain MRI with limited training cases. We distill a large teacher ViT into a 5 M-parameter TinyViT and fine-tune it using metric-based few-shot learning (prototypical loss with 5-shot episodes). Our model achieves high accuracy (approximately 91%) and area under ROC of at least 0.95 on a multi-site dataset of 79 genetically confirmed Batten-disease MRIs (27 CLN3 from the Hochstein natural-history study, 32 CLN2 from an international longitudinal cohort, 12 early-manifestation CLN2 cases reported by Cokal et al., and 8 public Radiopaedia scans) together with 90 age-matched controls, outperforming a 3D-ResNet and Swin-Tiny baseline. We further integrate Gradient-weighted Class Activation Mapping (Grad-CAM) to highlight disease-relevant brain regions, enabling explainable predictions. The model’s small size and strong performance (sensitivity greater than 90%, specificity approximately 90%) demonstrates a practical AI solution for early Batten disease detection.

Method: The review analyzes YOLO evolution chronologically from YOLOv5 to YOLO26, highlighting key innovations at each stage. Benchmarking is performed on MS COCO dataset comparing YOLOv5, YOLOv8, YOLO11, YOLO26 with other detectors using metrics like precision, recall, mAP, and inference speed.

Result: YOLO26 introduces significant innovations including DFL removal, native NMS-free inference, Progressive Loss Balancing, STAL, and MuSGD optimizer. Benchmarking shows trade-offs between accuracy and efficiency across different YOLO versions, with YOLO26 representing the latest advancements in the family.

Conclusion: The YOLO family has evolved significantly with continuous architectural improvements. Future challenges include addressing dense-scene limitations, integrating CNN-Transformer hybrids, developing open-vocabulary detection, and implementing edge-aware training approaches for broader real-world applications.

Abstract: This paper presents a comprehensive overview of the Ultralytics YOLO(You Only Look Once) family of object detectors, focusing the architectural evolution, benchmarking, deployment perspectives, and future challenges. The review begins with the most recent release, YOLO26 (YOLOv26), which introduces key innovations including Distribution Focal Loss (DFL) removal, native NMS-free inference, Progressive Loss Balancing (ProgLoss), Small-Target-Aware Label Assignment (STAL), and the MuSGD optimizer for stable training. The progression is then traced through YOLO11, with its hybrid task assignment and efficiency-focused modules; YOLOv8, which advanced with a decoupled detection head and anchor-free predictions; and YOLOv5, which established the modular PyTorch foundation that enabled modern YOLO development. Benchmarking on the MS COCO dataset provides a detailed quantitative comparison of YOLOv5, YOLOv8, YOLO11, and YOLO26, alongside cross-comparisons with YOLOv12, YOLOv13, RT-DETR, and DEIM. Metrics including precision, recall, F1 score, mean Average Precision, and inference speed are analyzed to highlight trade-offs between accuracy and efficiency. Deployment and application perspectives are further discussed, covering export formats, quantization strategies, and real-world use in robotics, agriculture, surveillance, and manufacturing. Finally, the paper identifies challenges and future directions, including dense-scene limitations, hybrid CNN-Transformer integration, open-vocabulary detection, and edge-aware training approaches.

Method: TreeNet integrates key features from neural networks, ensemble learning, and tree-based decision models into a layered decision ensemble learning methodology specifically designed for medical image analysis.

Result: Achieved F1-score of 0.85 with full training data and 0.77 with 50% training data (only 0.08 reduction), with processing speed of 32 FPS suitable for real-time applications.

Conclusion: TreeNet demonstrates efficiency and usability in medical image analysis, particularly for real-time applications, showing robustness to data limitations while maintaining interpretability.

Abstract: Within the domain of medical image analysis, three distinct methodologies have demonstrated commendable accuracy: Neural Networks, Decision Trees, and Ensemble-Based Learning Algorithms, particularly in the specialized context of genstro institutional track abnormalities detection. These approaches exhibit efficacy in disease detection scenarios where a substantial volume of data is available. However, the prevalent challenge in medical image analysis pertains to limited data availability and data confidence. This paper introduces TreeNet, a novel layered decision ensemble learning methodology tailored for medical image analysis. Constructed by integrating pivotal features from neural networks, ensemble learning, and tree-based decision models, TreeNet emerges as a potent and adaptable model capable of delivering superior performance across diverse and intricate machine learning tasks. Furthermore, its interpretability and insightful decision-making process enhance its applicability in complex medical scenarios. Evaluation of the proposed approach encompasses key metrics including Accuracy, Precision, Recall, and training and evaluation time. The methodology resulted in an F1-score of up to 0.85 when using the complete training data, with an F1-score of 0.77 when utilizing 50% of the training data. This shows a reduction of F1-score of 0.08 while in the reduction of 50% of the training data and training time. The evaluation of the methodology resulted in the 32 Frame per Second which is usable for the realtime applications. This comprehensive assessment underscores the efficiency and usability of TreeNet in the demanding landscape of medical image analysis specially in the realtime analysis.

Method: MCE consists of two components: Learning Capability Enhancement (LCE) with multi-level factors for dynamic modality balancing, and Representation Capability Enhancement (RCE) with subset prediction and cross-modal completion tasks to improve feature semantics and robustness.

Result: Comprehensive evaluations on four multi-modal benchmarks show MCE consistently outperforms state-of-the-art methods under various missing configurations.

Conclusion: MCE effectively tackles the limitations of existing approaches by addressing both learning progress imbalance and feature quality degradation in multi-modal learning with missing modalities.

Abstract: Multi-modal learning has made significant advances across diverse pattern recognition applications. However, handling missing modalities, especially under imbalanced missing rates, remains a major challenge. This imbalance triggers a vicious cycle: modalities with higher missing rates receive fewer updates, leading to inconsistent learning progress and representational degradation that further diminishes their contribution. Existing methods typically focus on global dataset-level balancing, often overlooking critical sample-level variations in modality utility and the underlying issue of degraded feature quality. We propose Modality Capability Enhancement (MCE) to tackle these limitations. MCE includes two synergistic components: i) Learning Capability Enhancement (LCE), which introduces multi-level factors to dynamically balance modality-specific learning progress, and ii) Representation Capability Enhancement (RCE), which improves feature semantics and robustness through subset prediction and cross-modal completion tasks. Comprehensive evaluations on four multi-modal benchmarks show that MCE consistently outperforms state-of-the-art methods under various missing configurations. The journal preprint version is now available at https://doi.org/10.1016/j.patcog.2025.112591. Our code is available at https://github.com/byzhaoAI/MCE.

Method: Uses B-Spline encoding for consistent representation, then applies multi-stage residual quantization to position, rotation, and gripper subspaces to produce compressed discrete tokens with coarse-to-fine granularity.

Result: Achieves 6.8× sequence length reduction, lowers target entropy, enables faster training convergence and improved model performance across real-robot and simulation experiments.

Conclusion: OmniSAT provides efficient compression while maintaining reconstruction quality, making auto-regressive VLA models more scalable and effective.

Abstract: Existing Vision-Language-Action (VLA) models can be broadly categorized into diffusion-based and auto-regressive (AR) approaches: diffusion models capture continuous action distributions but rely on computationally heavy iterative denoising. In contrast, AR models enable efficient optimization and flexible sequence construction, making them better suited for large-scale pretraining. To further improve AR efficiency, particularly when action chunks induce extended and high-dimensional sequences, prior work applies entropy-guided and token-frequency techniques to shorten the sequence length. However, such compression struggled with \textit{poor reconstruction or inefficient compression}. Motivated by this, we introduce an Omni Swift Action Tokenizer, which learns a compact, transferable action representation. Specifically, we first normalize value ranges and temporal horizons to obtain a consistent representation with B-Spline encoding. Then, we apply multi-stage residual quantization to the position, rotation, and gripper subspaces, producing compressed discrete tokens with coarse-to-fine granularity for each part. After pre-training on the large-scale dataset Droid, the resulting discrete tokenization shortens the training sequence by 6.8$\times$, and lowers the target entropy. To further explore the potential of OmniSAT, we develop a cross-embodiment learning strategy that builds on the unified action-pattern space and jointly leverages robot and human demonstrations. It enables scalable auxiliary supervision from heterogeneous egocentric videos. Across diverse real-robot and simulation experiments, OmniSAT encompasses higher compression while preserving reconstruction quality, enabling faster AR training convergence and model performance.

Method: Uses CLIP as teacher and few-shot vision encoder as student, with unsupervised proxy task for knowledge distillation. Includes synergistic knowledge mining module, visual-semantic bridging module, and adaptive classifier construction with visual weight generator and semantic weight reconstructor.

Result: Experimental results on four FSL datasets show SynTrans significantly outperforms current state-of-the-art methods, even when paired with a simple few-shot vision encoder.

Conclusion: SynTrans effectively addresses FSL challenges by leveraging large multimodal models’ knowledge through synergistic transfer mechanisms, achieving superior performance over existing approaches.

Abstract: Few-shot learning (FSL) addresses the challenge of classifying novel classes with limited training samples. While some methods leverage semantic knowledge from smaller-scale models to mitigate data scarcity, these approaches often introduce noise and bias due to the data’s inherent simplicity. In this paper, we propose a novel framework, Synergistic Knowledge Transfer (SynTrans), which effectively transfers diverse and complementary knowledge from large multimodal models to empower the off-the-shelf few-shot learner. Specifically, SynTrans employs CLIP as a robust teacher and uses a few-shot vision encoder as a weak student, distilling semantic-aligned visual knowledge via an unsupervised proxy task. Subsequently, a training-free synergistic knowledge mining module facilitates collaboration among large multimodal models to extract high-quality semantic knowledge. Building upon this, a visual-semantic bridging module enables bi-directional knowledge transfer between visual and semantic spaces, transforming explicit visual and implicit semantic knowledge into category-specific classifier weights. Finally, SynTrans introduces a visual weight generator and a semantic weight reconstructor to adaptively construct optimal multimodal FSL classifiers. Experimental results on four FSL datasets demonstrate that SynTrans, even when paired with a simple few-shot vision encoder, significantly outperforms current state-of-the-art methods.

Method: Proposes KAMamba with: 1) Knowledge-aware transition loss using class transition matrices, 2) Multi-task learning with three losses (PreC, PostC, Chg), 3) SSTMamba modules to disentangle spatial-spectral-temporal information, 4) SDMamba backbone for efficiency.

Result: On Saskatchewan MODIS dataset: 1.5-6% F1 improvement for change detection, ~2% gains in OA, AA, and Kappa for LULC classification over baselines.

Conclusion: KAMamba effectively addresses MODIS challenges through knowledge integration, multi-task learning, and efficient Mamba architecture, achieving significant performance improvements.

Abstract: Although change detection using MODIS time series is critical for environmental monitoring, it is a highly challenging task due to key MODIS difficulties, e.g., mixed pixels, spatial-spectral-temporal information coupling effect, and background class heterogeneity. This paper presents a novel knowledge-aware Mamba (KAMamba) for enhanced MODIS change detection, with the following contributions. First, to leverage knowledge regarding class transitions, we design a novel knowledge-driven transition-matrix-guided approach, leading to a knowledge-aware transition loss (KAT-loss) that can enhance detection accuracies. Second, to improve model constraints, a multi-task learning approach is designed, where three losses, i.e., pre-change classification loss (PreC-loss), post-change classification loss (PostC-loss), and change detection loss (Chg-loss) are used for improve model learning. Third, to disentangle information coupling in MODIS time series, novel spatial-spectral-temporal Mamba (SSTMamba) modules are designed. Last, to improve Mamba model efficiency and remove computational cost, a sparse and deformable Mamba (SDMamba) backbone is used in SSTMamba. On the MODIS time-series dataset for Saskatchewan, Canada, we evaluate the method on land-cover change detection and LULC classification; results show about 1.5-6% gains in average F1 for change detection over baselines, and about 2% improvements in OA, AA, and Kappa for LULC classification.

Method: Uses a learnable concentration token to aggregate image and reasoning text features, generating a differentiable positional prior heatmap that is decoded to precise masks through a lightweight decoder.

Result: Matches or surpasses best reported metrics on RefCOCO series and ReasonSeg benchmarks under comparable protocols, achieving state-of-the-art performance across validation and test partitions.

Conclusion: The heatmap quality strongly influences mask quality, supporting consistent association between reasoning output and mask generation, demonstrating advantages in concentration and precision for bridging reasoning and segmentation.

Abstract: Existing works on reasoning segmentation either connect hidden features from a language model directly to a mask decoder or represent positions in text, which limits interpretability and semantic detail. To solve this, we present CoPRS, a Multi-modal Chain-of-Thought (MCoT)-based positional perception model that bridges language reasoning to segmentation through a differentiable and interpretable positional prior instantiated as a heatmap. By making the reasoning process clear via MCoT and expressing it as a dense, differentiable heatmap, this interface enhances interpretability and diagnostic analysis and yields more concentrated evidence on the target. A learnable concentration token aggregates features of the image and reasoning text to generate this positional prior, which is decoded to precise masks through a lightweight decoder, providing a direct connection between reasoning and segmentation. Across the RefCOCO series and ReasonSeg, CoPRS matches or surpasses the best reported metrics on each standard split under comparable protocols, with performance at or above prior state of the art across both validation and test partitions. Extensive experiments reveal that the quality of the heatmap strongly influences the resulting mask quality, supporting a consistent association between the reasoning output and downstream mask generation. Collectively, these findings support the utility of this paradigm in bridging reasoning and segmentation and show advantages in concentration driven by reasoning and predicting masks more precisely. Code, checkpoints and logs are released at https://github.com/ZhenyuLU-Heliodore/CoPRS.git.

Method: Hybrid framework integrating NN-UNet’s feature extraction with diffusion probabilistic models that iteratively refine segmentation masks by learning residual error distributions.

Result: Superior performance on BraTS 2021 datasets with improvements in Dice coefficient and Hausdorff distance metrics, enhanced robustness across modalities and tumor subregions.

Conclusion: NNDM establishes a new direction for combining deterministic segmentation networks with stochastic diffusion models, advancing automated brain tumor analysis.

Abstract: Accurate detection and segmentation of brain tumors in magnetic resonance imaging (MRI) are critical for effective diagnosis and treatment planning. Despite advances in convolutional neural networks (CNNs) such as U-Net, existing models often struggle with generalization, boundary precision, and limited data diversity. To address these challenges, we propose NNDM (NN_UNet Diffusion Model)a hybrid framework that integrates the robust feature extraction of NN-UNet with the generative capabilities of diffusion probabilistic models. In our approach, the diffusion model progressively refines the segmentation masks generated by NN-UNet by learning the residual error distribution between predicted and ground-truth masks. This iterative denoising process enables the model to correct fine structural inconsistencies and enhance tumor boundary delineation. Experiments conducted on the BraTS 2021 datasets demonstrate that NNDM achieves superior performance compared to conventional U-Net and transformer-based baselines, yielding improvements in Dice coefficient and Hausdorff distance metrics. Moreover, the diffusion-guided refinement enhances robustness across modalities and tumor subregions. The proposed NNDM establishes a new direction for combining deterministic segmentation networks with stochastic diffusion models, advancing the state of the art in automated brain tumor analysis.

Method: Proposes two representations: Temporal-ranked dynamic image (emphasizes temporal progression) and Motion-intensity dynamic image (highlights subtle motions through frame reordering). Uses Adaptive fusion network to optimally integrate these representations.

Result: Achieved 93.95% Accuracy and 0.897 UF1 on CASME-II (new SOTA), 82.47% Accuracy and 0.665 UF1 on SAMM, and 76.00% Accuracy on MMEW, demonstrating strong generalization.

Conclusion: Both the input representations and proposed architecture significantly improve MER performance, providing foundation for affective computing, lie detection, and human-computer interaction applications.

Abstract: Micro-expressions (MEs) are subtle, transient facial changes with very low intensity, almost imperceptible to the naked eye, yet they reveal a person genuine emotion. They are of great value in lie detection, behavioral analysis, and psychological assessment. This paper proposes a novel MER method with two main contributions. First, we propose two complementary representations - Temporal-ranked dynamic image, which emphasizes temporal progression, and Motion-intensity dynamic image, which highlights subtle motions through a frame reordering mechanism incorporating motion intensity. Second, we propose an Adaptive fusion network, which automatically learns to optimally integrate these two representations, thereby enhancing discriminative ME features while suppressing noise. Experiments on three benchmark datasets (CASME-II, SAMM and MMEW) demonstrate the superiority of the proposed method. Specifically, AFN achieves 93.95 Accuracy and 0.897 UF1 on CASME-II, setting a new state-of-the-art benchmark. On SAMM, the method attains 82.47 Accuracy and 0.665 UF1, demonstrating more balanced recognition across classes. On MMEW, the model achieves 76.00 Accuracy, further confirming its generalization ability. The obtained results show that both the input and the proposed architecture play important roles in improving the performance of MER. Moreover, they provide a solid foundation for further research and practical applications in the fields of affective computing, lie detection, and human-computer interaction.

Method: Proposes a unique taxonomy dividing CVS into four key parts: MVMC tracking, Re-ID, action understanding, and combined methods. Systematically reviews state-of-the-art datasets, methodologies, results, and evaluation metrics.

Result: Provides a structured view of the field’s progression and identifies open research challenges including occlusions, diverse viewpoints, environmental variability, and emerging technologies like lifelong learning and federated learning.

Conclusion: Outlines key research directions for enhancing robustness, efficiency, and adaptability of CVS in complex real-world applications, aiming to inspire next-generation intelligent and adaptive vision systems.

Abstract: Connected Vision Systems (CVS) are transforming a variety of applications, including autonomous vehicles, smart cities, surveillance, and human-robot interaction. These systems harness multi-view multi-camera (MVMC) data to provide enhanced situational awareness through the integration of MVMC tracking, re-identification (Re-ID), and action understanding (AU). However, deploying CVS in real-world, dynamic environments presents a number of challenges, particularly in addressing occlusions, diverse viewpoints, and environmental variability. Existing surveys have focused primarily on isolated tasks such as tracking, Re-ID, and AU, often neglecting their integration into a cohesive system. These reviews typically emphasize single-view setups, overlooking the complexities and opportunities provided by multi-camera collaboration and multi-view data analysis. To the best of our knowledge, this survey is the first to offer a comprehensive and integrated review of MVMC that unifies MVMC tracking, Re-ID, and AU into a single framework. We propose a unique taxonomy to better understand the critical components of CVS, dividing it into four key parts: MVMC tracking, Re-ID, AU, and combined methods. We systematically arrange and summarize the state-of-the-art datasets, methodologies, results, and evaluation metrics, providing a structured view of the field’s progression. Furthermore, we identify and discuss the open research questions and challenges, along with emerging technologies such as lifelong learning, privacy, and federated learning, that need to be addressed for future advancements. The paper concludes by outlining key research directions for enhancing the robustness, efficiency, and adaptability of CVS in complex, real-world applications. We hope this survey will inspire innovative solutions and guide future research toward the next generation of intelligent and adaptive CVS.

Method: Employed various morphing tools and cross-dataset evaluation with incremental testing protocol to assess generalization as synthetic images were added during training.

Result: Generalization improves with controlled synthetic data integration or gradual addition of bona fide images, but indiscriminate synthetic use causes sub-optimal performance. Pure synthetic data achieves highest EER.

Conclusion: Synthetic data can enhance S-MAD when carefully combined with real data, but should not be used exclusively as it yields the worst performance in operational scenarios.

Abstract: This paper investigates the use of synthetic face data to enhance Single-Morphing Attack Detection (S-MAD), addressing the limitations of availability of large-scale datasets of bona fide images due to privacy concerns. Various morphing tools and cross-dataset evaluation schemes were utilized to conduct this study. An incremental testing protocol was implemented to assess the generalization capabilities as more and more synthetic images were added. The results of the experiments show that generalization can be improved by carefully incorporating a controlled number of synthetic images into existing datasets or by gradually adding bona fide images during training. However, indiscriminate use of synthetic data can lead to sub-optimal performance. Evenmore, the use of only synthetic data (morphed and non-morphed images) achieves the highest Equal Error Rate (EER), which means in operational scenarios the best option is not relying only on synthetic data for S-MAD.

Method: Uses Masked Video Modeling to jointly mine and compress semantics in self-supervised manner, with explicit regularization of non-semantic entropy. SMC++ adds masked motion prediction and Transformer-based compression with blueprint semantic representation.

Result: SMC and SMC++ models show remarkable superiority over traditional, learnable, and perceptual quality-oriented video codecs on three video analysis tasks across seven datasets.

Conclusion: The proposed semantic-preserving video compression framework effectively maintains video semantics while achieving superior performance on downstream analysis tasks compared to existing methods.

Abstract: Most video compression methods focus on human visual perception, neglecting semantic preservation. This leads to severe semantic loss during the compression, hampering downstream video analysis tasks. In this paper, we propose a Masked Video Modeling (MVM)-powered compression framework that particularly preserves video semantics, by jointly mining and compressing the semantics in a self-supervised manner. While MVM is proficient at learning generalizable semantics through the masked patch prediction task, it may also encode non-semantic information like trivial textural details, wasting bitcost and bringing semantic noises. To suppress this, we explicitly regularize the non-semantic entropy of the compressed video in the MVM token space. The proposed framework is instantiated as a simple Semantic-Mining-then-Compression (SMC) model. Furthermore, we extend SMC as an advanced SMC++ model from several aspects. First, we equip it with a masked motion prediction objective, leading to better temporal semantic learning ability. Second, we introduce a Transformer-based compression module, to improve the semantic compression efficacy. Considering that directly mining the complex redundancy among heterogeneous features in different coding stages is non-trivial, we introduce a compact blueprint semantic representation to align these features into a similar form, fully unleashing the power of the Transformer-based compression module. Extensive results demonstrate the proposed SMC and SMC++ models show remarkable superiority over previous traditional, learnable, and perceptual quality-oriented video codecs, on three video analysis tasks and seven datasets. \textit{Codes and model are available at: https://github.com/tianyuan168326/VideoSemanticCompression-Pytorch.

Method: Uses MLLM’s cross-modal attention to perform rectilinear warping of input images, reallocating spatial resolution toward important regions while preserving all original image information non-uniformly.

Result: Consistently improves accuracy across five benchmarks (TextVQA, GQA, DocVQA, POPE, MMMU) and four MLLMs, strengthens compositional reasoning, and reduces hallucinations, outperforming four competitive baselines.

Conclusion: Attention-guided warping prioritizes query-relevant information while preserving context, and MLLMs perform better when given such warped inputs.

Abstract: Multimodal large language models (MLLMs) often miss small details and spatial relations in cluttered scenes, leading to errors in fine-grained perceptual grounding. We introduce AttWarp, a lightweight method that allocates more resolution to query-relevant content while compressing less informative areas, all while preserving global context. At test time, the approach uses an MLLM’s cross-modal attention to perform rectilinear warping of the input image, reallocating spatial resolution toward regions the model deems important, without changing model weights or architecture. This attention-guided warping preserves all original image information but redistributes it non-uniformly, so small objects and subtle relationships become easier for the same model to read while the global layout remains intact. Across five benchmarks (TextVQA, GQA, DocVQA, POPE, MMMU) and four MLLMs (LLaVA, Qwen-VL, InternVL, and InstructBLIP), AttWarp consistently improves accuracy, strengthens compositional reasoning, and reduces hallucinations, outperforming four competitive baselines that manipulate raw images at test time. Together, these results show that attention-guided warping prioritizes information relevant to the query while preserving context, and that the same MLLMs perform better when given such warped inputs.

Method: Interactive framework where human corrections serve as interventional signals to identify when models rely on superficial features. The system propagates correction-informed edits across visually similar images to systematically correct failure modes.

Result: Improves segmentation accuracy by up to 9 mIoU points (12-15% relative improvement) on challenging cubemap data, with 3-4× reductions in annotation effort compared to standard retraining while maintaining competitive benchmark performance.

Conclusion: Provides a practical framework for building segmentation systems that are accurate, robust to dataset biases, data-efficient, and adaptable to real-world domains like urban climate monitoring and autonomous driving.

Abstract: Segmentation models achieve high accuracy on benchmarks but often fail in real-world domains by relying on spurious correlations instead of true object boundaries. We propose a human-in-the-loop interactive framework that enables interventional learning through targeted human corrections of segmentation outputs. Our approach treats human corrections as interventional signals that show when reliance on superficial features (e.g., color or texture) is inappropriate. The system learns from these interventions by propagating correction-informed edits across visually similar images, effectively steering the model toward robust, semantically meaningful features rather than dataset-specific artifacts. Unlike traditional annotation approaches that simply provide more training data, our method explicitly identifies when and why the model fails and then systematically corrects these failure modes across the entire dataset. Through iterative human feedback, the system develops increasingly robust representations that generalize better to novel domains and resist artifactual correlations. We demonstrate that our framework improves segmentation accuracy by up to 9 mIoU points (12-15% relative improvement) on challenging cubemap data and yields 3-4$\times$ reductions in annotation effort compared to standard retraining, while maintaining competitive performance on benchmark datasets. This work provides a practical framework for researchers and practitioners seeking to build segmentation systems that are accurate, robust to dataset biases, data-efficient, and adaptable to real-world domains such as urban climate monitoring and autonomous driving.

Method: Conducted studies with human participants using different image-text pairs, analyzed features of images and texts that help infer masked/unfamiliar words, and examined correlations with participant language backgrounds.

Result: Only some intuitive features strongly correlate with participant performance, suggesting need for further investigation of predictive features. AI systems show potential but need improvement in reasoning about participant performance.

Conclusion: The study reveals limited correlation between intuitive features and success in word inference tasks, highlighting the need for deeper investigation into predictive features and promising directions for improving AI reasoning about human learning performance.

Abstract: We investigate a new setting for foreign language learning, where learners infer the meaning of unfamiliar words in a multimodal context of a sentence describing a paired image. We conduct studies with human participants using different image-text pairs. We analyze the features of the data (i.e., images and texts) that make it easier for participants to infer the meaning of a masked or unfamiliar word, and what language backgrounds of the participants correlate with success. We find only some intuitive features have strong correlations with participant performance, prompting the need for further investigating of predictive features for success in these tasks. We also analyze the ability of AI systems to reason about participant performance, and discover promising future directions for improving this reasoning ability.

Method: End-to-end AI framework with fine-tuned YOLOv11 for real-time traffic detection, graph-based viewpoint normalization for camera inconsistencies, and domain-specific LLM for automated traffic pattern summaries from 24/7 video streams.

Result: Validated on 9M+ images from 1,000 NYC cameras during congestion pricing rollout: 9% weekday passenger vehicle decline in Congestion Relief Zone, early truck reductions with rebound signs, increased pedestrian/cyclist activity. LLM accuracy improved with example prompts.

Conclusion: Framework provides practical, infrastructure-ready solution for large-scale traffic monitoring with minimal human intervention, demonstrating policy-relevant insights for transportation management.

Abstract: Accurate, scalable traffic monitoring is critical for real-time and long-term transportation management, particularly during disruptions such as natural disasters, large construction projects, or major policy changes like New York City’s first-in-the-nation congestion pricing program. However, widespread sensor deployment remains limited due to high installation, maintenance, and data management costs. While traffic cameras offer a cost-effective alternative, existing video analytics struggle with dynamic camera viewpoints and massive data volumes from large camera networks. This study presents an end-to-end AI-based framework leveraging existing traffic camera infrastructure for high-resolution, longitudinal analysis at scale. A fine-tuned YOLOv11 model, trained on localized urban scenes, extracts multimodal traffic density and classification metrics in real time. To address inconsistencies from non-stationary pan-tilt-zoom cameras, we introduce a novel graph-based viewpoint normalization method. A domain-specific large language model was also integrated to process massive data from a 24/7 video stream to generate frequent, automated summaries of evolving traffic patterns, a task far exceeding manual capabilities. We validated the system using over 9 million images from roughly 1,000 traffic cameras during the early rollout of NYC congestion pricing in 2025. Results show a 9% decline in weekday passenger vehicle density within the Congestion Relief Zone, early truck volume reductions with signs of rebound, and consistent increases in pedestrian and cyclist activity at corridor and zonal scales. Experiments showed that example-based prompts improved LLM’s numerical accuracy and reduced hallucinations. These findings demonstrate the framework’s potential as a practical, infrastructure-ready solution for large-scale, policy-relevant traffic monitoring with minimal human intervention.

Method: 1) Investigated resolution preferences across tasks, finding correlation with image complexity and VLLM uncertainty variance. 2) Proposed empirical formula to determine optimal resolution. 3) Developed parameter-efficient fine-tuning to extend VLLMs to optimal resolutions.

Result: Extensive experiments on various vision-language tasks validate the effectiveness of the proposed method.

Conclusion: The approach successfully addresses the resolution limitation in VLLMs and improves performance across different vision-language tasks.

Abstract: Real-world vision-language applications demand varying levels of perceptual granularity. However, most existing visual large language models (VLLMs), such as LLaVA, pre-assume a fixed resolution for downstream tasks, which leads to subpar performance. To address this problem, we first conduct a comprehensive and pioneering investigation into the resolution preferences of different vision-language tasks, revealing a correlation between resolution preferences with image complexity, and uncertainty variance of the VLLM at different image input resolutions. Building on this insight, we propose an empirical formula to determine the optimal resolution for a given vision-language task, combining these two factors. Second, based on rigorous experiments, we propose a novel parameter-efficient fine-tuning technique to extend the visual input resolution of pre-trained VLLMs to the identified optimal resolution. Extensive experiments on various vision-language tasks validate the effectiveness of our method.

Method: A lightweight Confidence Refinement Network integrates uncertainty estimates with color, edge, and texture features to adjust detection scores without modifying the base model.

Result: Experiments on D-Fire dataset show improved precision, recall, and mean average precision compared to baselines, with modest computational overhead.

Conclusion: Post-detection rescoring effectively enhances robustness of compact deep learning models for real-world fire and smoke detection.

Abstract: Accurate fire and smoke detection is critical for safety and disaster response, yet existing vision-based methods face challenges in balancing efficiency and reliability. Compact deep learning models such as YOLOv5n and YOLOv8n are widely adopted for deployment on UAVs, CCTV systems, and IoT devices, but their reduced capacity often results in false positives and missed detections. Conventional post-detection methods such as Non-Maximum Suppression and Soft-NMS rely only on spatial overlap, which can suppress true positives or retain false alarms in cluttered or ambiguous fire scenes. To address these limitations, we propose an uncertainty aware post-detection framework that rescales detection confidences using both statistical uncertainty and domain relevant visual cues. A lightweight Confidence Refinement Network integrates uncertainty estimates with color, edge, and texture features to adjust detection scores without modifying the base model. Experiments on the D-Fire dataset demonstrate improved precision, recall, and mean average precision compared to existing baselines, with only modest computational overhead. These results highlight the effectiveness of post-detection rescoring in enhancing the robustness of compact deep learning models for real-world fire and smoke detection.

Method: A framework with Position Aware Embedding that maps target pose information to high-dimensional feature maps, and a Rendering Network with dual encoder branches that merges features to resolve both high and low level details.

Result: Experimental results show superior efficiency and visual quality compared to existing approaches, particularly in handling complex translational movements without explicit geometric operations like warping.

Conclusion: This work represents a step toward enabling real-time live and interactive telepresence applications by providing efficient view synthesis.

Abstract: Recent advancements in view synthesis have significantly enhanced immersive experiences across various computer graphics and multimedia applications, including telepresence and entertainment. By enabling the generation of new perspectives from a single input view, view synthesis allows users to better perceive and interact with their environment. However, many state-of-the-art methods, while achieving high visual quality, face limitations in real-time performance, which makes them less suitable for live applications where low latency is critical. In this paper, we present a lightweight, position-aware network designed for real-time view synthesis from a single input image and a target camera pose. The proposed framework consists of a Position Aware Embedding, which efficiently maps positional information from the target pose to generate high dimensional feature maps. These feature maps, along with the input image, are fed into a Rendering Network that merges features from dual encoder branches to resolve both high and low level details, producing a realistic new view of the scene. Experimental results demonstrate that our method achieves superior efficiency and visual quality compared to existing approaches, particularly in handling complex translational movements without explicit geometric operations like warping. This work marks a step toward enabling real-time live and interactive telepresence applications.

Method: Tested various CRF types on two datasets: low-quality satellite imagery and high-quality aerial photographs, comparing performance across different approaches.

Result: Identified which CRF models perform best on different image types, revealing the strengths and limitations of various CRF approaches for segmentation.

Conclusion: The study demonstrates the varying effectiveness of different CRF models for image segmentation, highlighting both pitfalls and potentials of each approach for satellite and aerial imagery.

Abstract: The output of image the segmentation process is usually not very clear due to low quality features of Satellite images. The purpose of this study is to find a suitable Conditional Random Field (CRF) to achieve better clarity in a segmented image. We started with different types of CRFs and studied them as to why they are or are not suitable for our purpose. We evaluated our approach on two different datasets - Satellite imagery having low quality features and high quality Aerial photographs. During the study we experimented with various CRFs to find which CRF gives the best results on images and compared our results on these datasets to show the pitfalls and potentials of different approaches.

Method: Built on YOLOv11 framework with three innovations: multi-scale residual module for contextual feature extraction, lightweight feature fusion to reduce model size, and litchi occlusion detection head to mitigate occlusion effects.

Result: Achieves 6.35 MB parameter size (32.5% smaller than baseline), 90.1% mAP (+2.5%), 85.5% F1-Score (+1.4%), and 57.2 FPS for real-time detection.

Conclusion: YOLOv11-Litchi is suitable for UAV-based litchi detection in complex orchard environments and shows potential for broader precision agriculture applications.

Abstract: Litchi is a high-value fruit, yet traditional manual selection methods are increasingly inadequate for modern production demands. Integrating UAV-based aerial imagery with deep learning offers a promising solution to enhance efficiency and reduce costs. This paper introduces YOLOv11-Litchi, a lightweight and robust detection model specifically designed for UAV-based litchi detection. Built upon the YOLOv11 framework, the proposed model addresses key challenges such as small target size, large model parameters hindering deployment, and frequent target occlusion. To tackle these issues, three major innovations are incorporated: a multi-scale residual module to improve contextual feature extraction across scales, a lightweight feature fusion method to reduce model size and computational costs while maintaining high accuracy, and a litchi occlusion detection head to mitigate occlusion effects by emphasizing target regions and suppressing background interference. Experimental results validate the model’s effectiveness. YOLOv11-Litchi achieves a parameter size of 6.35 MB - 32.5% smaller than the YOLOv11 baseline

• while improving mAP by 2.5% to 90.1% and F1-Score by 1.4% to 85.5%. Additionally, the model achieves a frame rate of 57.2 FPS, meeting real-time detection requirements. These findings demonstrate the suitability of YOLOv11-Litchi for UAV-based litchi detection in complex orchard environments, showcasing its potential for broader applications in precision agriculture.

Method: Uses single-branch architecture with two-stage cross-attention to align waypoint tokens with HD map, then refine attention over image and depth patches. Employs smoothness loss to prevent abrupt steering/speed changes and fine-tunes top 15 layers of 11B LLaMA-3.2 backbone.

Result: Achieves 84% success rate (+4%), 0.66 SPL (+0.11), reduces collisions from 2.6% to 1.2% (-1.4%) on nuScenes/Waymo benchmark. Key contributions: waypoint tokens, partial VLM fine-tuning, and cross-attention fusion.

Conclusion: NovaDrive enables real-time autonomous driving with improved safety and efficiency, shorter routes reducing fuel/battery usage, and can be extended to other embodied-AI domains.

Abstract: Autonomous vehicles must react in milliseconds while reasoning about road geometry and traffic intent to navigate complex situations. We introduce NovaDrive, a single-branch vision-language architecture that processes front-camera images, HD-map tiles, LiDAR depth, and textual waypoints in a single branch. A lightweight, two-stage cross-attention block first aligns waypoint tokens with the HD map, then refines attention over fine-grained image and depth patches. Coupled with a novel smoothness loss that discourages abrupt steering and speed changes, this design eliminates the need for recurrent memory. We fine-tune the top 15 layers of an 11B LLaMA-3.2 vision-language backbone, enabling real-time inference. On the nuScenes / Waymo subset of the MD-NEX Outdoor benchmark, NovaDrive raises success rate to 84% (+4%), boosts path-efficiency (SPL) to 0.66 (+0.11), and reduces collision frequency from 2.6% to 1.2% (-1.4%) relative to the previous state-of-the-art. Our ablations confirm that waypoint tokens, partial VLM fine-tuning, and the cross-attention fusion each contribute the most to these gains. Beyond safety, NovaDrive’s shorter routes (resulting from the novel smoothness loss) translate to lower fuel or battery usage, pointing toward leaner, more easily updated driving stacks. NovaDrive can be extended to other embodied-AI domains as well.

Method: Ceb extracts potential foreground-foreground boundaries using a revised Watershed algorithm, constructs boundary signatures by sampling pixels from current and neighboring boundaries, uses a boundary classifier to predict binary boundary labels, and then divides/merges regions based on predicted labels.

Result: Extensive experiments on six datasets show Ceb outperforms existing pixel clustering methods on semantic segmentation probability maps and achieves highly competitive performance compared to state-of-the-art cell instance segmentation methods.

Conclusion: The Ceb method effectively addresses the limitations of pixel-wise objectives by leveraging boundary features and achieves superior performance in cell instance segmentation.

Abstract: State-of-the-art (SOTA) methods for cell instance segmentation are based on deep learning (DL) semantic segmentation approaches, focusing on distinguishing foreground pixels from background pixels. In order to identify cell instances from foreground pixels (e.g., pixel clustering), most methods decompose instance information into pixel-wise objectives, such as distances to foreground-background boundaries (distance maps), heat gradients with the center point as heat source (heat diffusion maps), and distances from the center point to foreground-background boundaries with fixed angles (star-shaped polygons). However, pixel-wise objectives may lose significant geometric properties of the cell instances, such as shape, curvature, and convexity, which require a collection of pixels to represent. To address this challenge, we present a novel pixel clustering method, called Ceb (for Cell boundaries), to leverage cell boundary features and labels to divide foreground pixels into cell instances. Starting with probability maps generated from semantic segmentation, Ceb first extracts potential foreground-foreground boundaries with a revised Watershed algorithm. For each boundary candidate, a boundary feature representation (called boundary signature) is constructed by sampling pixels from the current foreground-foreground boundary as well as the neighboring background-foreground boundaries. Next, a boundary classifier is used to predict its binary boundary label based on the corresponding boundary signature. Finally, cell instances are obtained by dividing or merging neighboring regions based on the predicted boundary labels. Extensive experiments on six datasets demonstrate that Ceb outperforms existing pixel clustering methods on semantic segmentation probability maps. Moreover, Ceb achieves highly competitive performance compared to SOTA cell instance segmentation methods.

Method: The method uses initially matched features to build a computational model of feature spatial order, calculates possible spatial ranges for subsequent matches, filters unnecessary matches, and integrates with epipolar geometry. An image alignment method based on fundamental matrix removes rotation effects.

Result: Experiments on benchmark datasets, simulated images, and real images show the proposed method is significantly more efficient and accurate than traditional methods.

Conclusion: Integrating feature spatial order with epipolar geometry in a progressive framework effectively improves feature matching efficiency and accuracy.

Abstract: Image feature matching plays a vital role in many computer vision tasks. Although many image feature detection and matching techniques have been proposed over the past few decades, it is still time-consuming to match feature points in two images, especially for images with a large number of detected features. Feature spatial order can estimate the probability that a pair of features is correct. Since it is a completely independent concept from epipolar geometry, it can be used to complement epipolar geometry in guiding feature match in a target region so as to improve matching efficiency. In this paper, we integrate the concept of feature spatial order into a progressive matching framework. We use some of the initially matched features to build a computational model of feature spatial order and employs it to calculates the possible spatial range of subsequent feature matches, thus filtering out unnecessary feature matches. We also integrate it with epipolar geometry to further improve matching efficiency and accuracy. Since the spatial order of feature points is affected by image rotation, we propose a suitable image alignment method from the fundamental matrix of epipolar geometry to remove the effect of image rotation. To verify the feasibility of the proposed method, we conduct a series of experiments, including a standard benchmark dataset, self-generated simulated images, and real images. The results demonstrate that our proposed method is significantly more efficient and has more accurate feature matching than the traditional method.

Method: CAPEL classifies images into class clusters with distinct prompts, ensembles in classification logits space, uses cluster-preserving regularization, and adaptive prompt weighting.

Result: The method aligns better with visual feature distribution and maintains cluster-specific discriminative power for robust performance.

Conclusion: CAPEL effectively addresses limitations of conventional prompt ensembling by preserving cluster nature and optimizing prompt fine-tuning.

Abstract: Vision-language models (VLMs) such as CLIP achieve zero-shot transfer across various tasks by pre-training on numerous image-text pairs. These models often benefit from using an ensemble of context prompts to represent a class. Despite being effective, conventional prompt ensembling that averages textual features of context prompts often yields suboptimal results. This is because feature averaging shifts the class centroids away from the true class distribution. To address this issue, we propose the Cluster-Aware Prompt Ensemble Learning (CAPEL) framework, which preserves the cluster nature of context prompts. CAPEL classifies images into one of several class clusters, each represented by a distinct prompt. Instead of ensembling prompts in the feature space, we perform ensembling in the classification logits space, aligning better with the visual feature distribution. To further optimize prompt fine-tuning while maintaining cluster-specific discriminative power, we introduce a cluster-preserving regularization term. This ensures that prompts remain distinct and specialized for different clusters, preventing collapse into a uniform direction. Additionally, we integrate an adaptive prompt weighting technique to dynamically adjust the attention weights for flawed or ambiguous prompts, ensuring robust performance across diverse datasets and tasks.

Method: Uses diffusion model inversion with a feature injection module for style transfer, region control mechanism with distance-based masks for spatial precision, and Fourier transform-based style enhancement for visual quality.

Result: Achieves superior performance in scene text style transformation, outperforming state-of-the-art methods in both visual fidelity and text preservation.

Conclusion: The proposed framework successfully enables flexible and high-fidelity style transfer for scene text while maintaining readability and stylistic consistency.

Abstract: With the rapid development of diffusion models, style transfer has made remarkable progress. However, flexible and localized style editing for scene text remains an unsolved challenge. Although existing scene text editing methods have achieved text region editing, they are typically limited to content replacement and simple styles, which lack the ability of free-style transfer. In this paper, we introduce SceneTextStylizer, a novel training-free diffusion-based framework for flexible and high-fidelity style transfer of text in scene images. Unlike prior approaches that either perform global style transfer or focus solely on textual content modification, our method enables prompt-guided style transformation specifically for text regions, while preserving both text readability and stylistic consistency. To achieve this, we design a feature injection module that leverages diffusion model inversion and self-attention to transfer style features effectively. Additionally, a region control mechanism is introduced by applying a distance-based changing mask at each denoising step, enabling precise spatial control. To further enhance visual quality, we incorporate a style enhancement module based on the Fourier transform to reinforce stylistic richness. Extensive experiments demonstrate that our method achieves superior performance in scene text style transformation, outperforming existing state-of-the-art methods in both visual fidelity and text preservation.

Method: Uses temporal context resampler with predefined latent queries and transformer encoders to extract critical temporal information. Employs teacher-student network to generate attention and entropy maps for dependency-weighted spatial context selection, where student selects top-k tokens with highest spatial dependency.

Result: Reduces entropy modeling time by approximately 65% and achieves 11% BD-Rate reduction compared to previous state-of-the-art conditional entropy model.

Conclusion: The CGT entropy model effectively balances computational efficiency and compression performance by optimizing both temporal and spatial context modeling through resampling and dependency-weighted selection mechanisms.

Abstract: Conditional entropy models effectively leverage spatio-temporal contexts to reduce video redundancy. However, incorporating temporal context often introduces additional model complexity and increases computational cost. In parallel, many existing spatial context models lack explicit modeling the ordering of spatial dependencies, which may limit the availability of relevant context during decoding. To address these issues, we propose the Context Guided Transformer (CGT) entropy model, which estimates probability mass functions of the current frame conditioned on resampled temporal context and dependency-weighted spatial context. A temporal context resampler learns predefined latent queries to extract critical temporal information using transformer encoders, reducing downstream computational overhead. Meanwhile, a teacher-student network is designed as dependency-weighted spatial context assigner to explicitly model the dependency of spatial context order. The teacher generates an attention map to represent token importance and an entropy map to reflect prediction certainty from randomly masked inputs, guiding the student to select the weighted top-k tokens with the highest spatial dependency. During inference, only the student is used to predict undecoded tokens based on high-dependency context. Experimental results demonstrate that our CGT model reduces entropy modeling time by approximately 65% and achieves an 11% BD-Rate reduction compared to the previous state-of-the-art conditional entropy model.

Method: Fuses depth and mask features using a compact self-supervised encoder to produce stable object representations, combining these with bounding box IoU and re-identification features for matching. Uses zero-shot depth estimator and promptable visual segmentation model for spatial cues.

Result: Outperforms TBD state-of-the-art on challenging benchmarks with non-linear motion, occlusion, and crowded scenes (SportsMOT, DanceTrack), while achieving competitive performance on simpler benchmarks with linear motion (MOT17).

Conclusion: The proposed method effectively addresses limitations of IoU-based tracking by using fused depth-mask features through self-supervised learning, providing robust object representations without expensive segmentation IoU computations.

Abstract: Multi-object tracking (MOT) methods often rely on Intersection-over-Union (IoU) for association. However, this becomes unreliable when objects are similar or occluded. Also, computing IoU for segmentation masks is computationally expensive. In this work, we use segmentation masks to capture object shapes, but we do not compute segmentation IoU. Instead, we fuse depth and mask features and pass them through a compact encoder trained self-supervised. This encoder produces stable object representations, which we use as an additional similarity cue alongside bounding box IoU and re-identification features for matching. We obtain depth maps from a zero-shot depth estimator and object masks from a promptable visual segmentation model to obtain fine-grained spatial cues. Our MOT method is the first to use the self-supervised encoder to refine segmentation masks without computing masks IoU. MOT can be divided into joint detection-ReID (JDR) and tracking-by-detection (TBD) models. The latter are computationally more efficient. Experiments of our TBD method on challenging benchmarks with non-linear motion, occlusion, and crowded scenes, such as SportsMOT and DanceTrack, show that our method outperforms the TBD state-of-the-art on most metrics, while achieving competitive performance on simpler benchmarks with linear motion, such as MOT17.

Method: Created CHUG dataset with 856 UGC-HDR source videos transcoded across multiple resolutions and bitrates (total 5,992 videos), and conducted large-scale subjective study via Amazon Mechanical Turk collecting 211,848 perceptual ratings.

Result: CHUG provides the first comprehensive benchmark for analyzing UGC-specific distortions in HDR videos, offering a large-scale, diverse, and real-world UGC dataset.

Conclusion: CHUG will advance No-Reference HDR-VQA research by providing a publicly available dataset that addresses the unique challenges of user-generated HDR content quality assessment.

Abstract: High Dynamic Range (HDR) videos enhance visual experiences with superior brightness, contrast, and color depth. The surge of User-Generated Content (UGC) on platforms like YouTube and TikTok introduces unique challenges for HDR video quality assessment (VQA) due to diverse capture conditions, editing artifacts, and compression distortions. Existing HDR-VQA datasets primarily focus on professionally generated content (PGC), leaving a gap in understanding real-world UGC-HDR degradations. To address this, we introduce CHUG: Crowdsourced User-Generated HDR Video Quality Dataset, the first large-scale subjective study on UGC-HDR quality. CHUG comprises 856 UGC-HDR source videos, transcoded across multiple resolutions and bitrates to simulate real-world scenarios, totaling 5,992 videos. A large-scale study via Amazon Mechanical Turk collected 211,848 perceptual ratings. CHUG provides a benchmark for analyzing UGC-specific distortions in HDR videos. We anticipate CHUG will advance No-Reference (NR) HDR-VQA research by offering a large-scale, diverse, and real-world UGC dataset. The dataset is publicly available at: https://shreshthsaini.github.io/CHUG/.

Method: Proposed a holistic taxonomy of spatial tasks unifying existing benchmarks, conducted standardized evaluation across 8 key benchmarks using over 10 billion tokens, and performed qualitative evaluation on diverse human-intuitive scenarios.

Result: GPT-5 demonstrates unprecedented spatial intelligence strength but still falls significantly short of human performance; spatial tasks expose greater model capability deficiencies than non-spatial tasks; proprietary models show no decisive advantage on the most difficult spatial tasks.

Conclusion: Current multimodal models, including the most advanced GPT-5, still have substantial limitations in spatial intelligence compared to humans, highlighting a critical gap in achieving artificial general intelligence that can reason about the physical world.

Abstract: Multimodal models have achieved remarkable progress in recent years. Nevertheless, they continue to exhibit notable limitations in spatial understanding and reasoning, the very capability that anchors artificial general intelligence in the physical world. With the recent release of GPT-5, allegedly the most powerful AI model to date, it is timely to examine where the leading models (GPT, Gemini, Grok, Seed, Qwen, and Intern) stand on the path toward spatial intelligence. We first propose a holistic taxonomy of spatial tasks that unifies existing benchmarks and a standardized protocol for the fair evaluation of state-of-the-art proprietary and open-source models across eight key benchmarks, at a cost exceeding ten billion total tokens. Our empirical study then reveals that (1) GPT-5 demonstrates unprecedented strength in spatial intelligence (SI), yet (2) still falls short of human performance significantly across a broad spectrum of SI-tasks. Moreover, we (3) show that SI-tasks expose greater model capability deficiency than non-SI tasks, to the extent that (4) proprietary models do not exhibit a decisive advantage when facing the most difficult ones. In addition, we conduct a qualitative evaluation across a diverse set of scenarios that are intuitive for humans, yet fail even the most advanced multimodal models.

Method: Use geometric priors from pre-processing to guide optimal basis placement on estimated geometric scaffold, and introduce scene-adaptive virtual viewpoints to compensate for geometric deficiencies in input trajectory.

Result: Significant enhancements in rendering quality and memory requirements compared to baselines using regular placements, demonstrated through comprehensive analysis in large-scale indoor scenes.

Conclusion: Scene-adaptive strategies with geometric guidance and virtual viewpoints greatly improve upon uniform basis arrangements in scalable Neural Radiance Field representations for indoor environments.

Abstract: We propose scene-adaptive strategies to efficiently allocate representation capacity for generating immersive experiences of indoor environments from incomplete observations. Indoor scenes with multiple rooms often exhibit irregular layouts with varying complexity, containing clutter, occlusion, and flat walls. We maximize the utilization of limited resources with guidance from geometric priors, which are often readily available after pre-processing stages. We record observation statistics on the estimated geometric scaffold and guide the optimal placement of bases, which greatly improves upon the uniform basis arrangements adopted by previous scalable Neural Radiance Field (NeRF) representations. We also suggest scene-adaptive virtual viewpoints to compensate for geometric deficiencies inherent in view configurations in the input trajectory and impose the necessary regularization. We present a comprehensive analysis and discussion regarding rendering quality and memory requirements in several large-scale indoor scenes, demonstrating significant enhancements compared to baselines that employ regular placements.

Method: Uses Gaussian splatting with object templates as structural priors, followed by refinement pipeline that adapts templates to temporal variations using few-shot observations. Enables generalization across time steps through simple transformations.

Result: Achieves superior reconstruction quality compared to baselines while enabling fast and lightweight updates. Validated on real-world datasets collected specifically for long-term scene changes.

Conclusion: LTGS provides an efficient and scalable framework for modeling long-term scene chronology from sparse captures, significantly enhancing temporal evolution of 3D environments.

Abstract: Recent advances in novel-view synthesis can create the photo-realistic visualization of real-world environments from conventional camera captures. However, acquiring everyday environments from casual captures faces challenges due to frequent scene changes, which require dense observations both spatially and temporally. We propose long-term Gaussian scene chronology from sparse-view updates, coined LTGS, an efficient scene representation that can embrace everyday changes from highly under-constrained casual captures. Given an incomplete and unstructured Gaussian splatting representation obtained from an initial set of input images, we robustly model the long-term chronology of the scene despite abrupt movements and subtle environmental variations. We construct objects as template Gaussians, which serve as structural, reusable priors for shared object tracks. Then, the object templates undergo a further refinement pipeline that modulates the priors to adapt to temporally varying environments based on few-shot observations. Once trained, our framework is generalizable across multiple time steps through simple transformations, significantly enhancing the scalability for a temporal evolution of 3D environments. As existing datasets do not explicitly represent the long-term real-world changes with a sparse capture setup, we collect real-world datasets to evaluate the practicality of our pipeline. Experiments demonstrate that our framework achieves superior reconstruction quality compared to other baselines while enabling fast and light-weight updates.

Method: Proposes multi-view transformer (MVT) with pretrained backbones and patch masking for cross-view correspondence, geometric consistency for calibrated setups, enhanced Ensemble Kalman Smoother with variance inflation for uncertainty, and distillation procedure using pseudo-labels.

Result: Framework components consistently outperform existing methods across three animal species (flies, mice, chickadees), with each component providing complementary benefits for practical pose estimation.

Conclusion: The framework provides a practical, uncertainty-aware system for reliable pose estimation that enables downstream behavioral analyses under real-world data constraints.

Abstract: Multi-view pose estimation is essential for quantifying animal behavior in scientific research, yet current methods struggle to achieve accurate tracking with limited labeled data and suffer from poor uncertainty estimates. We address these challenges with a comprehensive framework combining novel training and post-processing techniques, and a model distillation procedure that leverages the strengths of these techniques to produce a more efficient and effective pose estimator. Our multi-view transformer (MVT) utilizes pretrained backbones and enables simultaneous processing of information across all views, while a novel patch masking scheme learns robust cross-view correspondences without camera calibration. For calibrated setups, we incorporate geometric consistency through 3D augmentation and a triangulation loss. We extend the existing Ensemble Kalman Smoother (EKS) post-processor to the nonlinear case and enhance uncertainty quantification via a variance inflation technique. Finally, to leverage the scaling properties of the MVT, we design a distillation procedure that exploits improved EKS predictions and uncertainty estimates to generate high-quality pseudo-labels, thereby reducing dependence on manual labels. Our framework components consistently outperform existing methods across three diverse animal species (flies, mice, chickadees), with each component contributing complementary benefits. The result is a practical, uncertainty-aware system for reliable pose estimation that enables downstream behavioral analyses under real-world data constraints.

Method: Modified Mobile 3D Vision Transformer (MDvT) by introducing SpectralCA block with bi-directional cross-attention to fuse spectral and spatial features, reducing parameters and inference time while maintaining accuracy.

Result: Experimental evaluation on WHU-Hi-HongHu dataset showed improved UAV perception efficiency, enabling real-time operation for navigation, object recognition, and environmental monitoring tasks.

Conclusion: The proposed architecture successfully enhances UAV perception capabilities through hyperspectral imaging integration, making it suitable for real-time applications in complex environments.

Abstract: The relevance of this research lies in the growing demand for unmanned aerial vehicles (UAVs) capable of operating reliably in complex environments where conventional navigation becomes unreliable due to interference, poor visibility, or camouflage. Hyperspectral imaging (HSI) provides unique opportunities for UAV-based computer vision by enabling fine-grained material recognition and object differentiation, which are critical for navigation, surveillance, agriculture, and environmental monitoring. The aim of this work is to develop a deep learning architecture integrating HSI into UAV perception for navigation, object detection, and terrain classification. Objectives include: reviewing existing HSI methods, designing a hybrid 2D/3D convolutional architecture with spectral-spatial cross-attention, training, and benchmarking. The methodology is based on the modification of the Mobile 3D Vision Transformer (MDvT) by introducing the proposed SpectralCA block. This block employs bi-directional cross-attention to fuse spectral and spatial features, enhancing accuracy while reducing parameters and inference time. Experimental evaluation was conducted on the WHU-Hi-HongHu dataset, with results assessed using Overall Accuracy, Average Accuracy, and the Kappa coefficient. The findings confirm that the proposed architecture improves UAV perception efficiency, enabling real-time operation for navigation, object recognition, and environmental monitoring tasks. Keywords: SpectralCA, deep learning, computer vision, hyperspectral imaging, unmanned aerial vehicle, object detection, semi-supervised learning.

Method: Encoder uses dense-to-sparse transformation to create compact motion prior representations. Decoder employs an advanced flow-driven diffusion model to reconstruct scene dynamics from these priors.

Result: Superior rate-distortion performance compared to state-of-the-art conventional video codec ECM on scene dynamics sequences.

Conclusion: The proposed framework demonstrates effective ultra-low bitrate video compression using motion pattern priors, achieving high-quality reconstruction across diverse scenes.

Abstract: This paper proposes a novel generative video compression framework that leverages motion pattern priors, derived from subtle dynamics in common scenes (e.g., swaying flowers or a boat drifting on water), rather than relying on video content priors (e.g., talking faces or human bodies). These compact motion priors enable a new approach to ultra-low bitrate communication while achieving high-quality reconstruction across diverse scene contents. At the encoder side, motion priors can be streamlined into compact representations via a dense-to-sparse transformation. At the decoder side, these priors facilitate the reconstruction of scene dynamics using an advanced flow-driven diffusion model. Experimental results illustrate that the proposed method can achieve superior rate-distortion-performance and outperform the state-of-the-art conventional-video codec Enhanced Compression Model (ECM) on-scene dynamics sequences. The project page can be found at-https://github.com/xyzysz/GNVDC.

Method: Built on single-step diffusion model with face supervision mechanism for facial focus and reference-based mechanism for identity restoration. Uses newly created PortraitSR-4K dataset for training.

Result: Achieves state-of-the-art performance on PortraitISR while maintaining comparable or better performance on general image and aligned face datasets.

Conclusion: HeadsUp provides an effective end-to-end solution for portrait super-resolution that eliminates blending artifacts and preserves facial identity.

Abstract: Portrait pictures, which typically feature both human subjects and natural backgrounds, are one of the most prevalent forms of photography on social media. Existing image super-resolution (ISR) techniques generally focus either on generic real-world images or strictly aligned facial images (i.e., face super-resolution). In practice, separate models are blended to handle portrait photos: the face specialist model handles the face region, and the general model processes the rest. However, these blending approaches inevitably introduce blending or boundary artifacts around the facial regions due to different model training recipes, while human perception is particularly sensitive to facial fidelity. To overcome these limitations, we study the portrait image supersolution (PortraitISR) problem, and propose HeadsUp, a single-step diffusion model that is capable of seamlessly restoring and upscaling portrait images in an end-to-end manner. Specifically, we build our model on top of a single-step diffusion model and develop a face supervision mechanism to guide the model in focusing on the facial region. We then integrate a reference-based mechanism to help with identity restoration, reducing face ambiguity in low-quality face restoration. Additionally, we have built a high-quality 4K portrait image ISR dataset dubbed PortraitSR-4K, to support model training and benchmarking for portrait images. Extensive experiments show that HeadsUp achieves state-of-the-art performance on the PortraitISR task while maintaining comparable or higher performance on both general image and aligned face datasets.

Method: Two-pronged approach: 1) Use denoising diffusion models to expand datasets with diverse image types (stereo, wide-angled, macro, close-up), 2) Apply Controlnet to enhance image quality and evaluate dataset improvements.

Result: The proposed methods aim to create more comprehensive and higher quality underwater image datasets, addressing current limitations in dataset diversity and image quality.

Conclusion: Expanding datasets with diffusion models and enhancing images with Controlnet can improve marine ecosystem studies by providing better quality and more diverse underwater imagery.

Abstract: Underwater images play a crucial role in ocean research and marine environmental monitoring since they provide quality information about the ecosystem. However, the complex and remote nature of the environment results in poor image quality with issues such as low visibility, blurry textures, color distortion, and noise. In recent years, research in image enhancement has proven to be effective but also presents its own limitations, like poor generalization and heavy reliance on clean datasets. One of the challenges herein is the lack of diversity and the low quality of images included in these datasets. Also, most existing datasets consist only of monocular images, a fact that limits the representation of different lighting conditions and angles. In this paper, we propose a new plan of action to overcome these limitations. On one hand, we call for expanding the datasets using a denoising diffusion model to include a variety of image types such as stereo, wide-angled, macro, and close-up images. On the other hand, we recommend enhancing the images using Controlnet to evaluate and increase the quality of the corresponding datasets, and hence improve the study of the marine ecosystem. Tags - Underwater Images, Denoising Diffusion, Marine ecosystem, Controlnet

Method: Embed atmospheric scattering model into PDE with edge-preserving diffusion and nonlocal operators, using adaptive regularization based on dark channel prior, and solving with GPU-accelerated fixed-point solver.

Result: Effective haze removal with preserved image fidelity, mathematically proven well-posedness with existence and uniqueness of weak solution in H₀¹(Ω).

Conclusion: The proposed PDE framework offers a rigorous mathematical alternative to data-driven dehazing techniques, achieving good performance with theoretical guarantees.

Abstract: This paper introduces a novel partial differential equation (PDE) framework for single-image dehazing. We embed the atmospheric scattering model into a PDE featuring edge-preserving diffusion and a nonlocal operator to maintain both local details and global structures. A key innovation is an adaptive regularization mechanism guided by the dark channel prior, which adjusts smoothing strength based on haze density. The framework’s mathematical well-posedness is rigorously established by proving the existence and uniqueness of its weak solution in $H_0^1(\Omega)$. An efficient, GPU-accelerated fixed-point solver is used for implementation. Experiments confirm our method achieves effective haze removal while preserving high image fidelity, offering a principled alternative to purely data-driven techniques.

Method: Developed a novel INR architecture that disentangles spatial and temporal trajectory parameters, creating a framework that operates directly on INR parameter space for classification. Evaluated using biologically grounded trajectory simulation with 450 subjects.

Result: Achieved 81.3% accuracy for brain aging trajectory classification in irregular sampling experiments, outperforming standard deep learning baseline (73.7%).

Conclusion: INR-based approach effectively models continuous brain aging processes and provides superior classification performance compared to traditional methods, especially for irregularly sampled longitudinal data.

Abstract: The human brain undergoes dynamic, potentially pathology-driven, structural changes throughout a lifespan. Longitudinal Magnetic Resonance Imaging (MRI) and other neuroimaging data are valuable for characterizing trajectories of change associated with typical and atypical aging. However, the analysis of such data is highly challenging given their discrete nature with different spatial and temporal image sampling patterns within individuals and across populations. This leads to computational problems for most traditional deep learning methods that cannot represent the underlying continuous biological process. To address these limitations, we present a new, fully data-driven method for representing aging trajectories across the entire brain by modelling subject-specific longitudinal T1-weighted MRI data as continuous functions using Implicit Neural Representations (INRs). Therefore, we introduce a novel INR architecture capable of partially disentangling spatial and temporal trajectory parameters and design an efficient framework that directly operates on the INRs’ parameter space to classify brain aging trajectories. To evaluate our method in a controlled data environment, we develop a biologically grounded trajectory simulation and generate T1-weighted 3D MRI data for 450 healthy and dementia-like subjects at regularly and irregularly sampled timepoints. In the more realistic irregular sampling experiment, our INR-based method achieves 81.3% accuracy for the brain aging trajectory classification task, outperforming a standard deep learning baseline model (73.7%).

Method: Created RoHOI benchmark with 20 corruption types based on HICO-DET and V-COCO datasets, introduced a new robustness-focused metric, and proposed SAMPL strategy that uses semantic-aware masking and progressive learning to guide model optimization using holistic and partial cues.

Result: Extensive experiments show that existing HOI models suffer significant performance drops under corruptions, while the proposed SAMPL approach outperforms state-of-the-art methods and sets new standards for robust HOI detection.

Conclusion: The RoHOI benchmark and SAMPL strategy effectively address robustness challenges in HOI detection, providing a framework for evaluating and improving model resilience against real-world corruptions in robot-human assistance scenarios.

Abstract: Human-Object Interaction (HOI) detection is crucial for robot-human assistance, enabling context-aware support. However, models trained on clean datasets degrade in real-world conditions due to unforeseen corruptions, leading to inaccurate predictions. To address this, we introduce the first robustness benchmark for HOI detection, evaluating model resilience under diverse challenges. Despite advances, current models struggle with environmental variability, occlusions, and noise. Our benchmark, RoHOI, includes 20 corruption types based on the HICO-DET and V-COCO datasets and a new robustness-focused metric. We systematically analyze existing models in the HOI field, revealing significant performance drops under corruptions. To improve robustness, we propose a Semantic-Aware Masking-based Progressive Learning (SAMPL) strategy to guide the model to be optimized based on holistic and partial cues, thus dynamically adjusting the model’s optimization to enhance robust feature learning. Extensive experiments show that our approach outperforms state-of-the-art methods, setting a new standard for robust HOI detection. Benchmarks, datasets, and code are available at https://github.com/KratosWen/RoHOI.

Method: Proposed a multi-strategy framework: 1) Multi-scenario dataset STP-AgriData combining real orchard and internet data, 2) Data augmentation for lighting variations, 3) REAS-Det network with RFAConv and C3RFEM for scale variation and MultiSEAM with soft-NMS for occlusion.

Result: Achieved precision of 87.6%, recall of 74.9%, mAP@.50 of 82.8%, and mAP@.50:.95 of 53.3%, outperforming other state-of-the-art detection methods.

Conclusion: The proposed framework effectively addresses key challenges in real-world pomelo detection and demonstrates superior performance compared to existing methods.

Abstract: As a specialty agricultural product with a large market scale, Shatian pomelo necessitates the adoption of automated detection to ensure accurate quantity and meet commercial demands for lean production. Existing research often involves specialized networks tailored for specific theoretical or dataset scenarios, but these methods tend to degrade performance in real-world. Through analysis of factors in this issue, this study identifies four key challenges that affect the accuracy of Shatian pomelo detection: imaging devices, lighting conditions, object scale variation, and occlusion. To mitigate these challenges, a multi-strategy framework is proposed in this paper. Firstly, to effectively solve tone variation introduced by diverse imaging devices and complex orchard environments, we utilize a multi-scenario dataset, STP-AgriData, which is constructed by integrating real orchard images with internet-sourced data. Secondly, to simulate the inconsistent illumination conditions, specific data augmentations such as adjusting contrast and changing brightness, are applied to the above dataset. Thirdly, to address the issues of object scale variation and occlusion in fruit detection, an REAS-Det network is designed in this paper. For scale variation, RFAConv and C3RFEM modules are designed to expand and enhance the receptive fields. For occlusion variation, a multi-scale, multi-head feature selection structure (MultiSEAM) and soft-NMS are introduced to enhance the handling of occlusion issues to improve detection accuracy. The results of these experiments achieved a precision(P) of 87.6%, a recall (R) of 74.9%, a mAP@.50 of 82.8%, and a mAP@.50:.95 of 53.3%. Our proposed network demonstrates superior performance compared to other state-of-the-art detection methods.

Method: Joint training of segmentation and retrieval models where both are optimized together - segmentation model uses retrieved image-mask pairs for anatomical understanding, while retrieval model learns segmentation-relevant features beyond visual similarity.

Result: Substantial improvements across multiple architectures: On ACDC dataset, SegFormer improved from Dice 0.8708±0.042 to 0.9115±0.031 and HD from 1.8130±2.49 to 1.1489±0.30. Consistent improvements shown on ACDC and M&Ms datasets.

Conclusion: J-RAS effectively enhances segmentation performance by enabling retrieval to provide meaningful contextual cues, demonstrating good generalizability across different architectures and datasets.

Abstract: Image segmentation, the process of dividing images into meaningful regions, is critical in medical applications for accurate diagnosis, treatment planning, and disease monitoring. Although manual segmentation by healthcare professionals produces precise outcomes, it is time-consuming, costly, and prone to variability due to differences in human expertise. Artificial intelligence (AI)-based methods have been developed to address these limitations by automating segmentation tasks; however, they often require large, annotated datasets that are rarely available in practice and frequently struggle to generalize across diverse imaging conditions due to inter-patient variability and rare pathological cases. In this paper, we propose Joint Retrieval Augmented Segmentation (J-RAS), a joint training method for guided image segmentation that integrates a segmentation model with a retrieval model. Both models are jointly optimized, enabling the segmentation model to leverage retrieved image-mask pairs to enrich its anatomical understanding, while the retrieval model learns segmentation-relevant features beyond simple visual similarity. This joint optimization ensures that retrieval actively contributes meaningful contextual cues to guide boundary delineation, thereby enhancing the overall segmentation performance. We validate J-RAS across multiple segmentation backbones, including U-Net, TransUNet, SAM, and SegFormer, on two benchmark datasets: ACDC and M&Ms, demonstrating consistent improvements. For example, on the ACDC dataset, SegFormer without J-RAS achieves a mean Dice score of 0.8708$\pm$0.042 and a mean Hausdorff Distance (HD) of 1.8130$\pm$2.49, whereas with J-RAS, the performance improves substantially to a mean Dice score of 0.9115$\pm$0.031 and a mean HD of 1.1489$\pm$0.30. These results highlight the method’s effectiveness and its generalizability across architectures and datasets.

Method: Three-stage physics-informed generation: parameterized template creation, illumination-aware 2D synthesis, and physical engine-based 3D rendering, plus a masking approach for real-world flare removal evaluation.

Result: Created FlareX dataset with 9,500 2D templates from 95 patterns and 3,000 flare image pairs from 60 3D scenes, with extensive experiments showing effectiveness.

Conclusion: The proposed physics-informed flare generation method and FlareX dataset significantly improve model performance on real-world flare removal compared to existing approaches.

Abstract: Lens flare occurs when shooting towards strong light sources, significantly degrading the visual quality of images. Due to the difficulty in capturing flare-corrupted and flare-free image pairs in the real world, existing datasets are typically synthesized in 2D by overlaying artificial flare templates onto background images. However, the lack of flare diversity in templates and the neglect of physical principles in the synthesis process hinder models trained on these datasets from generalizing well to real-world scenarios. To address these challenges, we propose a new physics-informed method for flare data generation, which consists of three stages: parameterized template creation, the laws of illumination-aware 2D synthesis, and physical engine-based 3D rendering, which finally gives us a mixed flare dataset that incorporates both 2D and 3D perspectives, namely FlareX. This dataset offers 9,500 2D templates derived from 95 flare patterns and 3,000 flare image pairs rendered from 60 3D scenes. Furthermore, we design a masking approach to obtain real-world flare-free images from their corrupted counterparts to measure the performance of the model on real-world images. Extensive experiments demonstrate the effectiveness of our method and dataset.

Method: Three complementary approaches: 1) Retinex-based synthesis pipeline for controllable flicker generation, 2) 4,000 real-world flicker image captures, 3) Green-screen method to incorporate motion while preserving real flicker degradation.

Result: The benchmark enables comprehensive evaluation of flicker removal methods and helps models better understand spatial and temporal characteristics of real flicker artifacts for improved generalization.

Conclusion: BurstDeflicker provides an effective dataset that advances flicker removal research by combining synthetic and real data acquisition strategies to overcome dataset limitations.

Abstract: Flicker artifacts in short-exposure images are caused by the interplay between the row-wise exposure mechanism of rolling shutter cameras and the temporal intensity variations of alternating current (AC)-powered lighting. These artifacts typically appear as uneven brightness distribution across the image, forming noticeable dark bands. Beyond compromising image quality, this structured noise also affects high-level tasks, such as object detection and tracking, where reliable lighting is crucial. Despite the prevalence of flicker, the lack of a large-scale, realistic dataset has been a significant barrier to advancing research in flicker removal. To address this issue, we present BurstDeflicker, a scalable benchmark constructed using three complementary data acquisition strategies. First, we develop a Retinex-based synthesis pipeline that redefines the goal of flicker removal and enables controllable manipulation of key flicker-related attributes (e.g., intensity, area, and frequency), thereby facilitating the generation of diverse flicker patterns. Second, we capture 4,000 real-world flicker images from different scenes, which help the model better understand the spatial and temporal characteristics of real flicker artifacts and generalize more effectively to wild scenarios. Finally, due to the non-repeatable nature of dynamic scenes, we propose a green-screen method to incorporate motion into image pairs while preserving real flicker degradation. Comprehensive experiments demonstrate the effectiveness of our dataset and its potential to advance research in flicker removal.

Method: Proposed MIMO model with visual referring multimodal input and pixel grounding multimodal output, trained on MIMOSeg dataset containing 895K samples covering instruction following and complex QA with multimodal input/output.

Result: Experiments on medical multimodal tasks show MIMO uniquely combines visual referring and pixel grounding capabilities not available in previous models.

Conclusion: MIMO successfully addresses the input and output limitations of existing medical vision language models by integrating visual understanding and pixel-level grounding capabilities.

Abstract: Currently, medical vision language models are widely used in medical vision question answering tasks. However, existing models are confronted with two issues: for input, the model only relies on text instructions and lacks direct understanding of visual clues in the image; for output, the model only gives text answers and lacks connection with key areas in the image. To address these issues, we propose a unified medical vision language model MIMO, with visual referring Multimodal Input and pixel grounding Multimodal Output. MIMO can not only combine visual clues and textual instructions to understand complex medical images and semantics, but can also ground medical terminologies in textual output within the image. To overcome the scarcity of relevant data in the medical field, we propose MIMOSeg, a comprehensive medical multimodal dataset including 895K samples. MIMOSeg is constructed from four different perspectives, covering basic instruction following and complex question answering with multimodal input and multimodal output. We conduct experiments on several downstream medical multimodal tasks. Extensive experimental results verify that MIMO can uniquely combine visual referring and pixel grounding capabilities, which are not available in previous models.

Method: Proposed Q-Adapter introduces learnable query tokens and a gating layer into Vision Encoder to extract sparse, caption-relevant features without external textual supervision, enabling efficient fine-tuning for video captioning tasks.

Result: Achieved state-of-the-art performance among PEFT methods on MSR-VTT and MSVD datasets across BLEU@4, METEOR, ROUGE-L, and CIDEr metrics, and competitive performance compared to full fine-tuning approaches while using only 1.4% of parameters.

Conclusion: Q-Adapter effectively balances caption quality and parameter efficiency, demonstrating strong scalability for video-language modeling and providing insights into optimization strategies for adapter-based learning.

Abstract: Recent advances in video captioning are driven by large-scale pretrained models, which follow the standard “pre-training followed by fine-tuning” paradigm, where the full model is fine-tuned for downstream tasks. Although effective, this approach becomes computationally prohibitive as the model size increases. The Parameter-Efficient Fine-Tuning (PEFT) approach offers a promising alternative, but primarily focuses on the language components of Multimodal Large Language Models (MLLMs). Despite recent progress, PEFT remains underexplored in multimodal tasks and lacks sufficient understanding of visual information during fine-tuning the model. To bridge this gap, we propose Query-Adapter (Q-Adapter), a lightweight visual adapter module designed to enhance MLLMs by enabling efficient fine-tuning for the video captioning task. Q-Adapter introduces learnable query tokens and a gating layer into Vision Encoder, enabling effective extraction of sparse, caption-relevant features without relying on external textual supervision. We evaluate Q-Adapter on two well-known video captioning datasets, MSR-VTT and MSVD, where it achieves state-of-the-art performance among the methods that take the PEFT approach across BLEU@4, METEOR, ROUGE-L, and CIDEr metrics. Q-Adapter also achieves competitive performance compared to methods that take the full fine-tuning approach while requiring only 1.4% of the parameters. We further analyze the impact of key hyperparameters and design choices on fine-tuning effectiveness, providing insights into optimization strategies for adapter-based learning. These results highlight the strong potential of Q-Adapter in balancing caption quality and parameter efficiency, demonstrating its scalability for video-language modeling.

Method: Uses 3D anchor point-based spatial-temporal prediction inspired by video compression, combined with adaptive quantization and context-based entropy coding to reduce storage.

Result: Achieves state-of-the-art reconstruction quality with fastest rendering speed and remarkably low storage footprint (~1MB), achieving 40x compression on synthetic scenes and 90x on real-world scenes.

Conclusion: P-4DGS provides an efficient solution for compact 4D scene modeling with superior compression efficiency while maintaining high reconstruction quality.

Abstract: 3D Gaussian Splatting (3DGS) has garnered significant attention due to its superior scene representation fidelity and real-time rendering performance, especially for dynamic 3D scene reconstruction (\textit{i.e.}, 4D reconstruction). However, despite achieving promising results, most existing algorithms overlook the substantial temporal and spatial redundancies inherent in dynamic scenes, leading to prohibitive memory consumption. To address this, we propose P-4DGS, a novel dynamic 3DGS representation for compact 4D scene modeling. Inspired by intra- and inter-frame prediction techniques commonly used in video compression, we first design a 3D anchor point-based spatial-temporal prediction module to fully exploit the spatial-temporal correlations across different 3D Gaussian primitives. Subsequently, we employ an adaptive quantization strategy combined with context-based entropy coding to further reduce the size of the 3D anchor points, thereby achieving enhanced compression efficiency. To evaluate the rate-distortion performance of our proposed P-4DGS in comparison with other dynamic 3DGS representations, we conduct extensive experiments on both synthetic and real-world datasets. Experimental results demonstrate that our approach achieves state-of-the-art reconstruction quality and the fastest rendering speed, with a remarkably low storage footprint (around \textbf{1MB} on average), achieving up to \textbf{40$\times$} and \textbf{90$\times$} compression on synthetic and real-world scenes, respectively.

Method: Proposes CCFormer with three key components: Early Integration Module (EIM) for multi-scale visual-audio fusion, Multi-query Transformer Module (MTM) for spatial-temporal modeling with dynamic audio queries, and Bi-modal Contrastive Learning (BCL) for cross-modal alignment.

Result: Achieves new state-of-the-art performance on S4, MS3 and AVSS datasets, demonstrating superior segmentation accuracy and robustness.

Conclusion: CCFormer effectively addresses limitations of previous methods by comprehensively processing local/global information and capturing spatial-temporal context, setting new benchmarks in audio-visual segmentation.

Abstract: Audio-Visual Segmentation (AVS) aims to generate pixel-wise segmentation maps that correlate with the auditory signals of objects. This field has seen significant progress with numerous CNN and Transformer-based methods enhancing the segmentation accuracy and robustness. Traditional CNN approaches manage audio-visual interactions through basic operations like padding and multiplications but are restricted by CNNs’ limited local receptive field. More recently, Transformer-based methods treat auditory cues as queries, utilizing attention mechanisms to enhance audio-visual cooperation within frames. Nevertheless, they typically struggle to extract multimodal coefficients and temporal dynamics adequately. To overcome these limitations, we present the Complementary and Contrastive Transformer (CCFormer), a novel framework adept at processing both local and global information and capturing spatial-temporal context comprehensively. Our CCFormer initiates with the Early Integration Module (EIM) that employs a parallel bilateral architecture, merging multi-scale visual features with audio data to boost cross-modal complementarity. To extract the intra-frame spatial features and facilitate the perception of temporal coherence, we introduce the Multi-query Transformer Module (MTM), which dynamically endows audio queries with learning capabilities and models the frame and video-level relations simultaneously. Furthermore, we propose the Bi-modal Contrastive Learning (BCL) to promote the alignment across both modalities in the unified feature space. Through the effective combination of those designs, our method sets new state-of-the-art benchmarks across the S4, MS3 and AVSS datasets. Our source code and model weights will be made publicly available at https://github.com/SitongGong/CCFormer

Method: ViTAR uses a cognitive chain of “think-act-rethink-answer” to treat medical images as interactive objects. It employs a two-stage training: supervised fine-tuning with curated datasets (1K interactive examples + 16K VQA data) followed by reinforcement learning to optimize decision-making.

Result: ViTAR outperforms strong state-of-the-art models. Visual attention analysis shows it increasingly anchors to clinically critical regions from “think” to “rethink” rounds and maintains high attention to visual tokens during reasoning.

Conclusion: Embedding expert-style iterative thinking chains into VLMs enhances both performance and trustworthiness of medical AI by mimicking human diagnostic reasoning processes.

Abstract: Medical vision-language models (VLMs) excel at image-text understanding but typically rely on a single-pass reasoning that neglects localized visual cues. In clinical practice, however, human experts iteratively scan, focus, and refine the regions of interest before reaching a final diagnosis. To narrow this machine-human perception gap, we introduce ViTAR, a novel VLM framework that emulates the iterative reasoning process of human experts through a cognitive chain of “think-act-rethink-answer”. ViTAR treats medical images as interactive objects, enabling models to engage multi-step visual reasoning. To support this approach, we curate a high-quality instruction dataset comprising 1K interactive examples that encode expert-like diagnostic behaviors. In addition, a 16K visual question answering training data has been curated towards fine-grained visual diagnosis. We introduce a two-stage training strategy that begins with supervised fine-tuning to guide cognitive trajectories, followed by the reinforcement learning to optimize decision-making. Extensive evaluations demonstrate that ViTAR outperforms strong state-of-the-art models. Visual attention analysis reveals that from the “think” to “rethink” rounds, ViTAR increasingly anchors visual grounding to clinically critical regions and maintains high attention allocation to visual tokens during reasoning, providing mechanistic insight into its improved performance. These findings demonstrate that embedding expert-style iterative thinking chains into VLMs enhances both performance and trustworthiness of medical AI.

Method: The authors created the DREAM benchmark consisting of: a diverse deepfake video dataset, large-scale human annotations (140,000 realism scores and descriptions from 3,500 annotators), and evaluation of 16 realism assessment methods including CLIP-based approaches.

Result: The paper presents a comprehensive benchmark with extensive human annotations and evaluates multiple assessment methods, providing foundational resources for deepfake realism evaluation.

Conclusion: The DREAM benchmark establishes a foundation for future research in deepfake visual realism assessment and related areas, addressing a previously understudied aspect of deepfake analysis.

Abstract: Deep learning based face-swap videos, widely known as deepfakes, have drawn wide attention due to their threat to information credibility. Recent works mainly focus on the problem of deepfake detection that aims to reliably tell deepfakes apart from real ones, in an objective way. On the other hand, the subjective perception of deepfakes, especially its computational modeling and imitation, is also a significant problem but lacks adequate study. In this paper, we focus on the visual realism assessment of deepfakes, which is defined as the automatic assessment of deepfake visual realism that approximates human perception of deepfakes. It is important for evaluating the quality and deceptiveness of deepfakes which can be used for predicting the influence of deepfakes on Internet, and it also has potentials in improving the deepfake generation process by serving as a critic. This paper prompts this new direction by presenting a comprehensive benchmark called DREAM, which stands for Deepfake REalism AssessMent. It is comprised of a deepfake video dataset of diverse quality, a large scale annotation that includes 140,000 realism scores and textual descriptions obtained from 3,500 human annotators, and a comprehensive evaluation and analysis of 16 representative realism assessment methods, including recent large vision language model based methods and a newly proposed description-aligned CLIP method. The benchmark and insights included in this study can lay the foundation for future research in this direction and other related areas.

Method: Proposes a three-part framework: 1) semantic-related feature learning module to discover semantic information and label correlations, 2) semantic-guided feature enhancement module to align visual and semantic feature spaces, and 3) collaborative learning that integrates feature learning and label recovery in a mutually reinforced loop.

Result: Extensive experiments on MS-COCO, VOC2007, and NUS-WIDE datasets show that CLSL outperforms state-of-the-art methods for multi-label image recognition with incomplete labels.

Conclusion: The collaborative learning framework effectively addresses both semantic-aware feature learning and missing label recovery, demonstrating superior performance on multiple benchmark datasets.

Abstract: Multi-label image recognition with incomplete labels is a critical learning task and has emerged as a focal topic in computer vision. However, this task is confronted with two core challenges: semantic-aware feature learning and missing label recovery. In this paper, we propose a novel Collaborative Learning of Semantic-aware feature learning and Label recovery (CLSL) method for multi-label image recognition with incomplete labels, which unifies the two aforementioned challenges into a unified learning framework. More specifically, we design a semantic-related feature learning module to learn robust semantic-related features by discovering semantic information and label correlations. Then, a semantic-guided feature enhancement module is proposed to generate high-quality discriminative semantic-aware features by effectively aligning visual and semantic feature spaces. Finally, we introduce a collaborative learning framework that integrates semantic-aware feature learning and label recovery, which can not only dynamically enhance the discriminability of semantic-aware features but also adaptively infer and recover missing labels, forming a mutually reinforced loop between the two processes. Extensive experiments on three widely used public datasets (MS-COCO, VOC2007, and NUS-WIDE) demonstrate that CLSL outperforms the state-of-the-art multi-label image recognition methods with incomplete labels.

Method: Uses probabilistic hyper-graphs with masked autoencoders that randomly mask entire modalities (not just patches), combines pre-training and fine-tuning in a single loop, and enables inference-time ensembles with knowledge distillation.

Result: Enables creation of inference-time ensembles that boost prediction performance and consistency, with knowledge distillation working effectively even on models under 1M parameters.

Conclusion: PHG-MAE provides a unified framework for multi-modal learning applicable to domains like UAV scenes, autonomous driving, and robotics, with released code and extended dataset for reproducibility.

Abstract: The computer vision domain has greatly benefited from an abundance of data across many modalities to improve on various visual tasks. Recently, there has been a lot of focus on self-supervised pre-training methods through Masked Autoencoders (MAE) \cite{he2022masked,bachmann2022multimae}, usually used as a first step before optimizing for a downstream task, such as classification or regression. This is very useful as it doesn’t require any manually labeled data. In this work, we introduce Probabilistic Hyper-Graphs using Masked Autoencoders (PHG-MAE): a novel model that unifies the classical work on neural graphs \cite{leordeanu2021semi} with the modern approach of masked autoencoders under a common theoretical framework. Through random masking of entire modalities, not just patches, the model samples from the distribution of hyper-edges on each forward pass. Additionally, the model adapts the standard MAE algorithm by combining pre-training and fine-tuning into a single training loop. Moreover, our approach enables the creation of inference-time ensembles which, through aggregation, boost the final prediction performance and consistency. Lastly, we show that we can apply knowledge distillation on top of the ensembles with little loss in performance, even with models that have fewer than 1M parameters. While our work mostly focuses on outdoor UAV scenes that contain multiple world interpretations and modalities, the same steps can be followed in other similar domains, such as autonomous driving or indoor robotics. In order to streamline the process of integrating external pre-trained experts for computer vision multi-modal multi-task learning (MTL) scenarios, we developed a data-pipeline software. Using this tool, we have created and released a fully-automated extension of the Dronescapes dataset. All the technical details, code and reproduction steps are publicly released.

Method: Reconstructs discrete optical remote sensing images into continuous video sequences, enabling vision foundation models developed for video segmentation to track landslide scar evolution through knowledge-guided, auto-propagation, and interactive refinement paradigms.

Result: Validated on Baige and Sela landslides (2017-2025), the framework successfully tracks landslide scar evolution, capturing failure precursors for early warning and post-failure evolution for secondary hazard assessment.

Conclusion: The proposed universal framework enables continuous tracking of landslide scars, providing critical insights for early warning systems and long-term stability assessment of landslide hazards.

Abstract: Tracking the spatiotemporal evolution of large-scale landslide scars is critical for understanding the evolution mechanisms and failure precursors, enabling effective early-warning. However, most existing studies have focused on single-phase or pre- and post-failure dual-phase landslide identification. Although these approaches delineate post-failure landslide boundaries, it is challenging to track the spatiotemporal evolution of landslide scars. To address this problem, this study proposes a novel and universal framework for tracking the spatiotemporal evolution of large-scale landslide scars using a vision foundation model. The key idea behind the proposed framework is to reconstruct discrete optical remote sensing images into a continuous video sequence. This transformation enables a vision foundation model, which is developed for video segmentation, to be used for tracking the evolution of landslide scars. The proposed framework operates within a knowledge-guided, auto-propagation, and interactive refinement paradigm to ensure the continuous and accurate identification of landslide scars. The proposed framework was validated through application to two representative cases: the post-failure Baige landslide and the active Sela landslide (2017-2025). Results indicate that the proposed framework enables continuous tracking of landslide scars, capturing both failure precursors critical for early warning and post-failure evolution essential for assessing secondary hazards and long-term stability.

Method: Uses VGGT foundation model for reliable initial poses and dense point clouds, hybrid Gaussian representation with dual position-shape optimization, graph-guided attribute refinement, and flow-based depth regularization.

Result: Achieves more robust performance on both forward-facing and large-scale complex datasets compared to other pose-free methods.

Conclusion: Gesplat overcomes the limitations of pose-dependent methods in sparse-view settings, enabling geometrically consistent reconstruction from unposed sparse images.

Abstract: Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have advanced 3D reconstruction and novel view synthesis, but remain heavily dependent on accurate camera poses and dense viewpoint coverage. These requirements limit their applicability in sparse-view settings, where pose estimation becomes unreliable and supervision is insufficient. To overcome these challenges, we introduce Gesplat, a 3DGS-based framework that enables robust novel view synthesis and geometrically consistent reconstruction from unposed sparse images. Unlike prior works that rely on COLMAP for sparse point cloud initialization, we leverage the VGGT foundation model to obtain more reliable initial poses and dense point clouds. Our approach integrates several key innovations: 1) a hybrid Gaussian representation with dual position-shape optimization enhanced by inter-view matching consistency; 2) a graph-guided attribute refinement module to enhance scene details; and 3) flow-based depth regularization that improves depth estimation accuracy for more effective supervision. Comprehensive quantitative and qualitative experiments demonstrate that our approach achieves more robust performance on both forward-facing and large-scale complex datasets compared to other pose-free methods.

Method: Clients estimate and upload pseudo label distributions, server adjusts them to avoid global class imbalance, and uses partial prompt aggregation (visual prompts aggregated at server, text prompts kept locally).

Result: Extensive experiments show FedCoPL achieves superior performance compared to baseline methods in unsupervised federated classification tasks.

Conclusion: FedCoPL successfully enables classification in UFL using CLIP, demonstrating effective handling of class imbalance through cooperative pseudo labeling and personalized prompt strategies.

Abstract: Unsupervised Federated Learning (UFL) aims to collaboratively train a global model across distributed clients without sharing data or accessing label information. Previous UFL works have predominantly focused on representation learning and clustering tasks. Recently, vision language models (e.g., CLIP) have gained significant attention for their powerful zero-shot prediction capabilities. Leveraging this advancement, classification problems that were previously infeasible under the UFL paradigm now present promising new opportunities, yet remain largely unexplored. In this paper, we extend UFL to the classification problem with CLIP for the first time and propose a novel method, \underline{\textbf{Fed}}erated \underline{\textbf{Co}}operative \underline{\textbf{P}}seudo \underline{\textbf{L}}abeling (\textbf{FedCoPL}). Specifically, clients estimate and upload their pseudo label distribution, and the server adjusts and redistributes them to avoid global imbalance among classes. Moreover, we introduce a partial prompt aggregation protocol for effective collaboration and personalization. In particular, visual prompts containing general image features are aggregated at the server, while text prompts encoding personalized knowledge are retained locally. Extensive experiments demonstrate the superior performance of our FedCoPL compared to baseline methods. Our code is available at \href{https://github.com/krumpguo/FedCoPL}{https://github.com/krumpguo/FedCoPL}.

Method: ACRE modifies GRPO with an auxiliary consistency check: after generating reasoning and initial answer, answer options are shuffled and the model is prompted again with the same reasoning to predict a second answer, with rewards based on consistency and correctness.

Result: ACRE achieves average improvements of 2.2% on Video Reasoning benchmarks and 1.5% on multimodal math reasoning benchmarks over the GRPO baseline.

Conclusion: The proposed ACRE method effectively addresses reasoning-answer inconsistency in RL-trained multimodal LLMs by incorporating consistency verification into the reward mechanism.

Abstract: Recent advances in large language models (LLMs) have demonstrated that reinforcement learning with verifiable rewards (RLVR) can significantly enhance reasoning abilities by directly optimizing correctness, rather than relying solely on supervised imitation. This paradigm has been extended to multimodal LLMs for complex video and image understanding tasks. However, while outcome-driven RL improves answer accuracy, it can inadvertently decouple the reasoning chain from the final answer, leading to situations where models produce inconsistency between the reasoning trace and final answer. In our experiments on multiple-choice visual question-answering tasks, the standard GRPO method yields only 79.7% consistency on MMVU between the reasoning steps and the chosen answers, indicating frequent mismatches between answers and reasoning. To this end, we propose Answer-Consistent Reinforcement Learning (ACRE) that modifies the GRPO algorithm with an auxiliary consistency check. After the model generates a chain of thought and an initial answer for a given question, we shuffle the answer options and prompt the model again with the same reasoning trace to predict a second answer. We design a consistency-verification reward that grants a high reward only if both the original and the post-shuffle answers agree and are correct; otherwise, a lower reward is assigned accordingly. This mechanism penalizes reasoning-answer misalignment and discourages the model from relying on spurious patterns, such as option ordering biases. We evaluate ACRE on challenging Video Reasoning benchmarks and multimodal math reasoning benchmarks, achieving an average 2.2% and 1.5% improvement for Video Reasoning and Math Reasoning tasks over the GRPO baseline.

Method: In-Context Forensic Chain integrates objectified rule construction with adaptive filtering to build knowledge base, and multi-step progressive reasoning pipeline mirroring expert workflows from coarse to fine-grained analysis.

Result: Surpasses state-of-the-art training-free methods and achieves competitive/superior performance compared to weakly and fully supervised approaches across multiple benchmarks.

Conclusion: ICFC enables systematic exploitation of MLLM reasoning for image classification, pixel localization, and text interpretability, providing effective training-free solution for image manipulation localization.

Abstract: Advances in image tampering pose serious security threats, underscoring the need for effective image manipulation localization (IML). While supervised IML achieves strong performance, it depends on costly pixel-level annotations. Existing weakly supervised or training-free alternatives often underperform and lack interpretability. We propose the In-Context Forensic Chain (ICFC), a training-free framework that leverages multi-modal large language models (MLLMs) for interpretable IML tasks. ICFC integrates an objectified rule construction with adaptive filtering to build a reliable knowledge base and a multi-step progressive reasoning pipeline that mirrors expert forensic workflows from coarse proposals to fine-grained forensics results. This design enables systematic exploitation of MLLM reasoning for image-level classification, pixel-level localization, and text-level interpretability. Across multiple benchmarks, ICFC not only surpasses state-of-the-art training-free methods but also achieves competitive or superior performance compared to weakly and fully supervised approaches.

Method: Created ImmerIris dataset collected via VR headsets, established evaluation protocols, and proposed a normalization-free recognition paradigm that learns directly from ocular images with minimal preprocessing.

Result: Current methods designed for on-axis imagery perform poorly on off-axis data. The proposed normalization-free approach consistently outperforms normalization-based methods on the ImmerIris dataset.

Conclusion: Normalization-free iris recognition is a promising direction for robust immersive recognition, addressing the limitations of traditional methods in off-axis acquisition scenarios.

Abstract: In egocentric applications such as augmented and virtual reality, immersive iris recognition is emerging as an accurate and seamless way to identify persons. While classic systems acquire iris images on-axis, i.e., via dedicated frontal sensors in controlled settings, the immersive setup primarily captures off-axis irises through tilt-placed headset cameras, with only mild control in open scenes. This yields unique challenges, including perspective distortion, intensified quality degradations, and intra-class variations in iris texture. Datasets capturing these challenges remain scarce. To fill this gap, this paper introduces ImmerIris, a large-scale dataset collected via VR headsets, containing 499,791 ocular images from 564 subjects. It is, to the best of current knowledge, the largest public dataset and among the first dedicated to off-axis acquisition. Based on ImmerIris, evaluation protocols are constructed to benchmark recognition methods under different challenging factors. Current methods, primarily designed for classic on-axis imagery, perform unsatisfactorily on the immersive setup, mainly due to reliance on fallible normalization. To this end, this paper further proposes a normalization-free paradigm that directly learns from ocular images with minimal adjustment. Despite its simplicity, this approach consistently outperforms normalization-based counterparts, pointing to a promising direction for robust immersive recognition.

Method: Collected finger tapping videos, extracted temporal, frequency, and amplitude features from wrist/hand movements, and built a hybrid framework with Conv1D MaxPooling, BiLSTM layers, attention mechanisms, and dense layers for multi-class classification.

Result: The model demonstrated strong performance in distinguishing between five PD severity classes, showing improved automated PD severity detection.

Conclusion: Integrating spatial-temporal representations with attention mechanisms can improve automated PD severity detection, making it a promising non-invasive tool for clinical PD monitoring and progression tracking.

Abstract: Effective clinical management and intervention development depend on accurate evaluation of Parkinsons disease (PD) severity. Many researchers have worked on developing gesture-based PD recognition systems; however, their performance accuracy is not satisfactory. In this study, we propose a multi-class Parkinson Disease detection system based on finger tapping using an attention-enhanced CNN BiLSTM. We collected finger tapping videos and derived temporal, frequency, and amplitude based features from wrist and hand movements. Then, we proposed a hybrid deep learning framework integrating CNN, BiLSTM, and attention mechanisms for multi-class PD severity classification from video-derived motion features. First, the input sequence is reshaped and passed through a Conv1D MaxPooling block to capture local spatial dependencies. The resulting feature maps are fed into a BiLSTM layer to model temporal dynamics. An attention mechanism focuses on the most informative temporal features, producing a context vector that is further processed by a second BiLSTM layer. CNN-derived features and attention-enhanced BiLSTM outputs are concatenated, followed by dense and dropout layers, before the final softmax classifier outputs the predicted PD severity level. The model demonstrated strong performance in distinguishing between the five severity classes, suggesting that integrating spatial temporal representations with attention mechanisms can improve automated PD severity detection, making it a promising non-invasive tool to support clinicians in PD monitoring and progression tracking.

Method: Developed DeepFusionNet architecture for both tasks: 1) Low-light enhancement with ~2.5M parameters, 2) Super-resolution with ~100K parameters using autoencoder approach instead of GANs.

Result: Low-light enhancement: 92.8% SSIM and 26.30 PSNR on LOL-v1 dataset. Super-resolution: 25.30 PSNR and 80.7% SSIM on validation set.

Conclusion: DeepFusionNet provides efficient solutions for both low-light enhancement and super-resolution tasks with significantly reduced computational requirements while maintaining high image quality metrics.

Abstract: Computer vision and image processing applications suffer from dark and low-light images, particularly during real-time image transmission. Currently, low light and dark images are converted to bright and colored forms using autoencoders; however, these methods often achieve low SSIM and PSNR scores and require high computational power due to their large number of parameters. To address these challenges, the DeepFusionNet architecture has been developed. According to the results obtained with the LOL-v1 dataset, DeepFusionNet achieved an SSIM of 92.8% and a PSNR score of 26.30, while containing only approximately 2.5 million parameters. On the other hand, conversion of blurry and low-resolution images into high-resolution and blur-free images has gained importance in image processing applications. Unlike GAN-based super-resolution methods, an autoencoder-based super resolution model has been developed that contains approximately 100 thousand parameters and uses the DeepFusionNet architecture. According to the results of the tests, the DeepFusionNet based super-resolution method achieved a PSNR of 25.30 and a SSIM score of 80.7 percent according to the validation set.

Method: Colorize a single key view, fine-tune a personalized colorizer to propagate colors to novel views and time steps, then use Lab color space Gaussian splatting for 3D reconstruction. Recasts 3D colorization as single image paradigm.

Result: Extensive experiments show Color3D delivers more consistent and chromatically rich renderings with precise user control across diverse static and dynamic 3D colorization benchmarks.

Conclusion: The framework successfully enables flexible user-guided colorization of 3D scenes while maintaining consistency and color richness, integrating arbitrary image colorization models with enhanced flexibility.

Abstract: In this work, we present Color3D, a highly adaptable framework for colorizing both static and dynamic 3D scenes from monochromatic inputs, delivering visually diverse and chromatically vibrant reconstructions with flexible user-guided control. In contrast to existing methods that focus solely on static scenarios and enforce multi-view consistency by averaging color variations which inevitably sacrifice both chromatic richness and controllability, our approach is able to preserve color diversity and steerability while ensuring cross-view and cross-time consistency. In particular, the core insight of our method is to colorize only a single key view and then fine-tune a personalized colorizer to propagate its color to novel views and time steps. Through personalization, the colorizer learns a scene-specific deterministic color mapping underlying the reference view, enabling it to consistently project corresponding colors to the content in novel views and video frames via its inherent inductive bias. Once trained, the personalized colorizer can be applied to infer consistent chrominance for all other images, enabling direct reconstruction of colorful 3D scenes with a dedicated Lab color space Gaussian splatting representation. The proposed framework ingeniously recasts complicated 3D colorization as a more tractable single image paradigm, allowing seamless integration of arbitrary image colorization models with enhanced flexibility and controllability. Extensive experiments across diverse static and dynamic 3D colorization benchmarks substantiate that our method can deliver more consistent and chromatically rich renderings with precise user control. Project Page https://yecongwan.github.io/Color3D/.

Method: The system combines a segmentation branch using nnUNet for lesion detection with an arterial atlas for vessel mapping and identification, and a generation branch using pGAN to synthesize MRA images from MRI.

Result: The implementation demonstrates promising results in localizing occluded vessels on stroke-affected T1 MRI scans.

Conclusion: The approach has potential for faster and more informed stroke diagnosis by accurately localizing occluded vessels using only MRI scans.

Abstract: A key challenge in ischemic stroke diagnosis using medical imaging is the accurate localization of the occluded vessel. Current machine learning methods in focus primarily on lesion segmentation, with limited work on vessel localization. In this study, we introduce Stroke Locus Net, an end-to-end deep learning pipeline for detection, segmentation, and occluded vessel localization using only MRI scans. The proposed system combines a segmentation branch using nnUNet for lesion detection with an arterial atlas for vessel mapping and identification, and a generation branch using pGAN to synthesize MRA images from MRI. Our implementation demonstrates promising results in localizing occluded vessels on stroke-affected T1 MRI scans, with potential for faster and more informed stroke diagnosis.

Method: Uses two components: ReMix Module leverages MLLMs to edit semantic features and adapt instruction embeddings to DiT backbone without fine-tuning; IP-ControlNet extends ControlNet to decouple semantic/layout cues and introduces ε-equivariant latent space for joint denoising in shared noise space.

Result: ReMix supports personalized generation, image editing, style transfer, and multi-condition synthesis while maintaining character consistency and spatial controllability.

Conclusion: ReMix provides an effective and efficient unified framework for character-consistent image generation and editing, validated through extensive experiments.

Abstract: Recent advances in large-scale text-to-image diffusion models (e.g., FLUX.1) have greatly improved visual fidelity in consistent character generation and editing. However, existing methods rarely unify these tasks within a single framework. Generation-based approaches struggle with fine-grained identity consistency across instances, while editing-based methods often lose spatial controllability and instruction alignment. To bridge this gap, we propose ReMix, a unified framework for character-consistent generation and editing. It constitutes two core components: the ReMix Module and IP-ControlNet. The ReMix Module leverages the multimodal reasoning ability of MLLMs to edit semantic features of input images and adapt instruction embeddings to the native DiT backbone without fine-tuning. While this ensures coherent semantic layouts, pixel-level consistency and pose controllability remain challenging. To address this, IP-ControlNet extends ControlNet to decouple semantic and layout cues from reference images and introduces an {\epsilon}-equivariant latent space that jointly denoises the reference and target images within a shared noise space. Inspired by convergent evolution and quantum decoherence,i.e., where environmental noise drives state convergence, this design promotes feature alignment in the hidden space, enabling consistent object generation while preserving identity. ReMix supports a wide range of tasks, including personalized generation, image editing, style transfer, and multi-condition synthesis. Extensive experiments validate its effectiveness and efficiency as a unified framework for character-consistent image generation and editing.

Method: Proposes SaFiRe framework that mimics human two-phase cognitive process: global understanding followed by detail-oriented inspection, leveraging Mamba’s scan-then-update property for efficient multi-cycle refinement with linear complexity.

Result: Extensive experiments on standard and proposed aRefCOCO benchmark demonstrate SaFiRe’s superiority over state-of-the-art baselines in handling ambiguous referring expressions.

Conclusion: SaFiRe effectively addresses challenging real-world RIS scenarios through its two-phase cognitive design and efficient refinement process, showing improved performance on ambiguous expressions.

Abstract: Referring Image Segmentation (RIS) aims to segment the target object in an image given a natural language expression. While recent methods leverage pre-trained vision backbones and more training corpus to achieve impressive results, they predominantly focus on simple expressions–short, clear noun phrases like “red car” or “left girl”. This simplification often reduces RIS to a key word/concept matching problem, limiting the model’s ability to handle referential ambiguity in expressions. In this work, we identify two challenging real-world scenarios: object-distracting expressions, which involve multiple entities with contextual cues, and category-implicit expressions, where the object class is not explicitly stated. To address the challenges, we propose a novel framework, SaFiRe, which mimics the human two-phase cognitive process–first forming a global understanding, then refining it through detail-oriented inspection. This is naturally supported by Mamba’s scan-then-update property, which aligns with our phased design and enables efficient multi-cycle refinement with linear complexity. We further introduce aRefCOCO, a new benchmark designed to evaluate RIS models under ambiguous referring expressions. Extensive experiments on both standard and proposed datasets demonstrate the superiority of SaFiRe over state-of-the-art baselines.

Method: Uses active sampling strategy to guide annotators for optimal point label selection, then propagates sparse labels with hybrid approach leveraging both SAM2 and superpixel-based methods.

Result: Experiments on two underwater datasets show SparseUWSeg outperforms state-of-the-art approaches, achieving up to +5% mIoU improvement over D+NN baseline.

Conclusion: Main contribution is an effective interactive annotation tool that enables ecology researchers to efficiently generate high-quality segmentation masks using foundation models and computer vision.

Abstract: Semantic segmentation is essential to automate underwater imagery analysis with ecology monitoring purposes. Unfortunately, fine grained underwater scene analysis is still an open problem even for top performing segmentation models. The high cost of obtaining dense, expert-annotated, segmentation labels hinders the supervision of models in this domain. While sparse point-labels are easier to obtain, they introduce challenges regarding which points to annotate and how to propagate the sparse information. We present SparseUWSeg, a novel framework that addresses both issues. SparseUWSeg employs an active sampling strategy to guide annotators, maximizing the value of their point labels. Then, it propagates these sparse labels with a hybrid approach leverages both the best of SAM2 and superpixel-based methods. Experiments on two diverse underwater datasets demonstrate the benefits of SparseUWSeg over state-of-the-art approaches, achieving up to +5% mIoU over D+NN. Our main contribution is the design and release of a simple but effective interactive annotation tool, integrating our algorithms. It enables ecology researchers to leverage foundation models and computer vision to efficiently generate high-quality segmentation masks to process their data.

Method: Proposes ViConEx-Med framework using multi-concept learnable tokens and specialized attention layers for processing visual and text-based concept tokens to produce concept-level localization maps while maintaining predictive accuracy.

Result: Experiments on synthetic and real-world medical datasets show ViConEx-Med outperforms prior concept-based models and achieves competitive performance with black-box models in both concept detection and localization precision.

Conclusion: The approach suggests a promising direction for building inherently interpretable models grounded in visual concepts, with code publicly available.

Abstract: Concept-based models aim to explain model decisions with human-understandable concepts. However, most existing approaches treat concepts as numerical attributes, without providing complementary visual explanations that could localize the predicted concepts. This limits their utility in real-world applications and particularly in high-stakes scenarios, such as medical use-cases. This paper proposes ViConEx-Med, a novel transformer-based framework for visual concept explainability, which introduces multi-concept learnable tokens to jointly predict and localize visual concepts. By leveraging specialized attention layers for processing visual and text-based concept tokens, our method produces concept-level localization maps while maintaining high predictive accuracy. Experiments on both synthetic and real-world medical datasets demonstrate that ViConEx-Med outperforms prior concept-based models and achieves competitive performance with black-box models in terms of both concept detection and localization precision. Our results suggest a promising direction for building inherently interpretable models grounded in visual concepts. Code is publicly available at https://github.com/CristianoPatricio/viconex-med.

Method: Predict 3D coordinates of both front and back surfaces, densely sample 3D coordinates between them to create ultra-dense 2D-3D correspondences, and use Hierarchical Continuous Coordinate Encoding (HCCE) for accurate and efficient coordinate representation.

Result: Outperforms existing state-of-the-art methods across seven classic BOP core datasets on the BOP website.

Conclusion: Incorporating both front and back surfaces with ultra-dense sampling significantly improves pose estimation accuracy compared to methods that only use the front surface.

Abstract: In pose estimation for seen objects, a prevalent pipeline involves using neural networks to predict dense 3D coordinates of the object surface on 2D images, which are then used to establish dense 2D-3D correspondences. However, current methods primarily focus on more efficient encoding techniques to improve the precision of predicted 3D coordinates on the object’s front surface, overlooking the potential benefits of incorporating the back surface and interior of the object. To better utilize the full surface and interior of the object, this study predicts 3D coordinates of both the object’s front and back surfaces and densely samples 3D coordinates between them. This process creates ultra-dense 2D-3D correspondences, effectively enhancing pose estimation accuracy based on the Perspective-n-Point (PnP) algorithm. Additionally, we propose Hierarchical Continuous Coordinate Encoding (HCCE) to provide a more accurate and efficient representation of front and back surface coordinates. Experimental results show that, compared to existing state-of-the-art (SOTA) methods on the BOP website, the proposed approach outperforms across seven classic BOP core datasets. Code is available at https://github.com/WangYuLin-SEU/HCCEPose.

Method: Proposed Text-Conditioned Multi-granularity Alignment (TCMA) framework with global video-sentence alignment, sentence-guided frame aggregation, word-guided patch alignment, plus Word and Patch Selection module and Text-Adaptive Dynamic Temperature Mechanism.

Result: Achieved state-of-the-art performance with 45.5% R@1 in text-to-video and 42.8% R@1 in video-to-text retrieval on DVTMD dataset, establishing the first complete benchmark for drone text-video retrieval.

Conclusion: The DVTMD dataset and TCMA framework effectively address the limitations in UAV video-text retrieval, demonstrating superior performance and providing a solid foundation for future research in this domain.

Abstract: Unmanned aerial vehicles (UAVs) have become powerful platforms for real-time, high-resolution data collection, producing massive volumes of aerial videos. Efficient retrieval of relevant content from these videos is crucial for applications in urban management, emergency response, security, and disaster relief. While text-video retrieval has advanced in natural video domains, the UAV domain remains underexplored due to limitations in existing datasets, such as coarse and redundant captions. Thus, in this work, we construct the Drone Video-Text Match Dataset (DVTMD), which contains 2,864 videos and 14,320 fine-grained, semantically diverse captions. The annotations capture multiple complementary aspects, including human actions, objects, background settings, environmental conditions, and visual style, thereby enhancing text-video correspondence and reducing redundancy. Building on this dataset, we propose the Text-Conditioned Multi-granularity Alignment (TCMA) framework, which integrates global video-sentence alignment, sentence-guided frame aggregation, and word-guided patch alignment. To further refine local alignment, we design a Word and Patch Selection module that filters irrelevant content, as well as a Text-Adaptive Dynamic Temperature Mechanism that adapts attention sharpness to text type. Extensive experiments on DVTMD and CapERA establish the first complete benchmark for drone text-video retrieval. Our TCMA achieves state-of-the-art performance, including 45.5% R@1 in text-to-video and 42.8% R@1 in video-to-text retrieval, demonstrating the effectiveness of our dataset and method. The code and dataset will be released.

Method: Pseudo-balancing enforces demographic balance during pseudo-label selection using only unlabeled images from a race-balanced dataset without requiring ground-truth annotations. Evaluated through fine-tuning biased classifiers and stress-testing with imbalanced data.

Result: Method achieved 79.81% overall accuracy (6.53% improvement over baseline) and reduced gender accuracy gap by 44.17%. In East Asian subgroup, gap narrowed from over 49% to just 5.01%.

Conclusion: Even without label supervision, access to demographically balanced unlabeled datasets can effectively debias computer vision models while preserving or enhancing accuracy.

Abstract: Face gender classification models often reflect and amplify demographic biases present in their training data, leading to uneven performance across gender and racial subgroups. We introduce pseudo-balancing, a simple and effective strategy for mitigating such biases in semi-supervised learning. Our method enforces demographic balance during pseudo-label selection, using only unlabeled images from a race-balanced dataset without requiring access to ground-truth annotations. We evaluate pseudo-balancing under two conditions: (1) fine-tuning a biased gender classifier using unlabeled images from the FairFace dataset, and (2) stress-testing the method with intentionally imbalanced training data to simulate controlled bias scenarios. In both cases, models are evaluated on the All-Age-Faces (AAF) benchmark, which contains a predominantly East Asian population. Our results show that pseudo-balancing consistently improves fairness while preserving or enhancing accuracy. The method achieves 79.81% overall accuracy - a 6.53% improvement over the baseline - and reduces the gender accuracy gap by 44.17%. In the East Asian subgroup, where baseline disparities exceeded 49%, the gap is narrowed to just 5.01%. These findings suggest that even in the absence of label supervision, access to a demographically balanced or moderately skewed unlabeled dataset can serve as a powerful resource for debiasing existing computer vision models.

Method: Proposed a progressive relational learning framework that extends from binary to n-ary relationships, uses a grouped supervision loss for n-ary relational learning without specific annotations, and employs a multi-modal network with hybrid attention mechanisms within scene graphs created with n-ary relationships.

Result: Experiments on ReferIt3D and ScanRefer benchmarks demonstrate that the method outperforms state-of-the-art approaches and proves the advantages of n-ary relational perception in 3D localization.

Conclusion: The proposed n-ary relational learning framework effectively addresses the limitations of pairwise relationship modeling and significantly improves 3D object localization performance by capturing global relational contexts in natural language descriptions.

Abstract: Localizing 3D objects using natural language is essential for robotic scene understanding. The descriptions often involve multiple spatial relationships to distinguish similar objects, making 3D-language alignment difficult. Current methods only model relationships for pairwise objects, ignoring the global perceptual significance of n-ary combinations in multi-modal relational understanding. To address this, we propose a novel progressive relational learning framework for 3D object grounding. We extend relational learning from binary to n-ary to identify visual relations that match the referential description globally. Given the absence of specific annotations for referred objects in the training data, we design a grouped supervision loss to facilitate n-ary relational learning. In the scene graph created with n-ary relationships, we use a multi-modal network with hybrid attention mechanisms to further localize the target within the n-ary combinations. Experiments and ablation studies on the ReferIt3D and ScanRefer benchmarks demonstrate that our method outperforms the state-of-the-art, and proves the advantages of the n-ary relational perception in 3D localization.

Method: Two synergistic pretraining stages: self-supervised learning on 190M tissue patches from 140K slides for cervical-specific feature extraction, followed by multimodal enhancement with 2.5M image-text pairs, integrated with multiple downstream diagnostic functions.

Result: CerS-Path surpasses prior foundation models, achieving 99.38% screening sensitivity in prospective testing on 3,173 cases across five centers, with excellent generalizability and support for eight diagnostic functions including rare cancer classification.

Conclusion: The system represents a significant advance in cervical pathology with strong potential for subspecialty diagnostic translation and cervical cancer screening applications.

Abstract: Cervical cancer remains a major malignancy, necessitating extensive and complex histopathological assessments and comprehensive support tools. Although deep learning shows promise, these models still lack accuracy and generalizability. General foundation models offer a broader reach but remain limited in capturing subspecialty-specific features and task adaptability. We introduce the Cervical Subspecialty Pathology (CerS-Path) diagnostic system, developed through two synergistic pretraining stages: self-supervised learning on approximately 190 million tissue patches from 140,000 slides to build a cervical-specific feature extractor, and multimodal enhancement with 2.5 million image-text pairs, followed by integration with multiple downstream diagnostic functions. Supporting eight diagnostic functions, including rare cancer classification and multimodal Q&A, CerS-Path surpasses prior foundation models in scope and clinical applicability. Comprehensive evaluations demonstrate a significant advance in cervical pathology, with prospective testing on 3,173 cases across five centers maintaining 99.38% screening sensitivity and excellent generalizability, highlighting its potential for subspecialty diagnostic translation and cervical cancer screening.

Method: Proposes SEDD metric combining Gram matrix-based style extraction with metric learning optimized for intra-class compactness and inter-class separation to extract style embeddings. Establishes benchmark using public datasets.

Result: Experiments on various datasets and sim-to-real methods demonstrate the method’s capability to effectively quantify the synthetic-to-real gap.

Conclusion: Provides a standardized quality control tool for systematic diagnosis and targeted enhancement of synthetic datasets, advancing data-driven autonomous driving development.

Abstract: Ensuring the reliability of autonomous driving perception systems requires extensive environment-based testing, yet real-world execution is often impractical. Synthetic datasets have therefore emerged as a promising alternative, offering advantages such as cost-effectiveness, bias free labeling, and controllable scenarios. However, the domain gap between synthetic and real-world datasets remains a critical bottleneck for the generalization of AI-based autonomous driving models. Quantifying this synthetic-to-real gap is thus essential for evaluating dataset utility and guiding the design of more effective training pipelines. In this paper, we establish a systematic framework for quantifying the synthetic-to-real gap in autonomous driving systems, and propose Style Embedding Distribution Discrepancy (SEDD) as a novel evaluation metric. Our framework combines Gram matrix-based style extraction with metric learning optimized for intra-class compactness and inter-class separation to extract style embeddings. Furthermore, we establish a benchmark using publicly available datasets. Experiments are conducted on a variety of datasets and sim-to-real methods, and the results show that our method is capable of quantifying the synthetic-to-real gap. This work provides a standardized quality control tool that enables systematic diagnosis and targeted enhancement of synthetic datasets, advancing future development of data-driven autonomous driving systems.

Method: Introduces AnomReason benchmark with structured quadruple annotations (Name, Phenomenon, Reasoning, Severity) created using AnomAgent - a modular multi-agent pipeline with lightweight human-in-the-loop verification, processing ~4.17B GPT-4o tokens.

Result: Models fine-tuned on AnomReason achieve consistent improvements over strong vision-language baselines using semantic matching metrics (SemAP and SemF1), demonstrating practical utility in explainable deepfake detection and semantic reasonableness assessment.

Conclusion: AnomReason and AnomAgent provide a foundation for measuring and improving semantic plausibility of AI-generated images, with released code, metrics, data, and models to support reproducible research on semantic authenticity and interpretable AIGC forensics.

Abstract: The rapid advancement of AI-generated content (AIGC) has enabled the synthesis of visually convincing images; however, many such outputs exhibit subtle \textbf{semantic anomalies}, including unrealistic object configurations, violations of physical laws, or commonsense inconsistencies, which compromise the overall plausibility of the generated scenes. Detecting these semantic-level anomalies is essential for assessing the trustworthiness of AIGC media, especially in AIGC image analysis, explainable deepfake detection and semantic authenticity assessment. In this paper, we formalize \textbf{semantic anomaly detection and reasoning} for AIGC images and introduce \textbf{AnomReason}, a large-scale benchmark with structured annotations as quadruples \emph{(Name, Phenomenon, Reasoning, Severity)}. Annotations are produced by a modular multi-agent pipeline (\textbf{AnomAgent}) with lightweight human-in-the-loop verification, enabling scale while preserving quality. At construction time, AnomAgent processed approximately 4.17,B GPT-4o tokens, providing scale evidence for the resulting structured annotations. We further show that models fine-tuned on AnomReason achieve consistent gains over strong vision-language baselines under our proposed semantic matching metric (\textit{SemAP} and \textit{SemF1}). Applications to {explainable deepfake detection} and {semantic reasonableness assessment of image generators} demonstrate practical utility. In summary, AnomReason and AnomAgent serve as a foundation for measuring and improving the semantic plausibility of AI-generated images. We will release code, metrics, data, and task-aligned models to support reproducible research on semantic authenticity and interpretable AIGC forensics.

Method: Employed multiple machine learning models: logistic regression, CNNs, ResNet for classification; U-Net for semantic segmentation; EfficientDet for anchor-based object detection on MRI scans.

Result: Demonstrated promising improvements in accuracy and efficiency of brain tumor diagnostics through the applied deep learning techniques.

Conclusion: Deep learning shows significant potential in medical imaging for brain tumor detection and segmentation, with implications for improving clinical diagnostic outcomes.

Abstract: This study explores the application of deep learning techniques in the automated detection and segmentation of brain tumors from MRI scans. We employ several machine learning models, including basic logistic regression, Convolutional Neural Networks (CNNs), and Residual Networks (ResNet) to classify brain tumors effectively. Additionally, we investigate the use of U-Net for semantic segmentation and EfficientDet for anchor-based object detection to enhance the localization and identification of tumors. Our results demonstrate promising improvements in the accuracy and efficiency of brain tumor diagnostics, underscoring the potential of deep learning in medical imaging and its significance in improving clinical outcomes.

Method: Evaluated a large vision model (LVM) in zero-shot setting across four medical imaging tasks using 4D CT data from 122 patients (over 1,820 3D CT volumes).

Result: The LVM achieved competitive performance on segmentation, denoising, and super-resolution, and surpassed specialized baselines in motion prediction with state-of-the-art spatial accuracy.

Conclusion: General-purpose video models can serve as unified learners and reasoners for medical imaging, revealing emergent zero-shot capabilities and laying groundwork for future medical foundation models.

Abstract: Recent advances in large generative models have shown that simple autoregressive formulations, when scaled appropriately, can exhibit strong zero-shot generalization across domains. Motivated by this trend, we investigate whether autoregressive video modeling principles can be directly applied to medical imaging tasks, despite the model never being trained on medical data. Specifically, we evaluate a large vision model (LVM) in a zero-shot setting across four representative tasks: organ segmentation, denoising, super-resolution, and motion prediction. Remarkably, even without domain-specific fine-tuning, the LVM can delineate anatomical structures in CT scans and achieve competitive performance on segmentation, denoising, and super-resolution. Most notably, in radiotherapy motion prediction, the model forecasts future 3D CT phases directly from prior phases of a 4D CT scan, producing anatomically consistent predictions that capture patient-specific respiratory dynamics with realistic temporal coherence. We evaluate the LVM on 4D CT data from 122 patients, totaling over 1,820 3D CT volumes. Despite no prior exposure to medical data, the model achieves strong performance across all tasks and surpasses specialized DVF-based and generative baselines in motion prediction, achieving state-of-the-art spatial accuracy. These findings reveal the emergence of zero-shot capabilities in medical video modeling and highlight the potential of general-purpose video models to serve as unified learners and reasoners laying the groundwork for future medical foundation models built on video models.

Method: Replaces standard positional gradient heuristic with opacity gradient-based densification trigger, pairs with conservative pruning schedule, and uses depth-correlation loss for geometric guidance.

Result: Achieves 40% more compact models (32k vs 57k primitives) on 3-view LLFF dataset and ~70% reduction on Mip-NeRF 360 dataset with modest trade-off in reconstruction metrics.

Conclusion: Establishes new state-of-the-art on quality-vs-efficiency Pareto frontier for few-shot view synthesis through fundamental optimization improvements to 3DGS core framework.

Abstract: 3D Gaussian Splatting (3DGS) struggles in few-shot scenarios, where its standard adaptive density control (ADC) can lead to overfitting and bloated reconstructions. While state-of-the-art methods like FSGS improve quality, they often do so by significantly increasing the primitive count. This paper presents a framework that revises the core 3DGS optimization to prioritize efficiency. We replace the standard positional gradient heuristic with a novel densification trigger that uses the opacity gradient as a lightweight proxy for rendering error. We find this aggressive densification is only effective when paired with a more conservative pruning schedule, which prevents destructive optimization cycles. Combined with a standard depth-correlation loss for geometric guidance, our framework demonstrates a fundamental improvement in efficiency. On the 3-view LLFF dataset, our model is over 40% more compact (32k vs. 57k primitives) than FSGS, and on the Mip-NeRF 360 dataset, it achieves a reduction of approximately 70%. This dramatic gain in compactness is achieved with a modest trade-off in reconstruction metrics, establishing a new state-of-the-art on the quality-vs-efficiency Pareto frontier for few-shot view synthesis.

Method: Three key innovations: 1) Pre-trained Hand Clarity Codebook (HCC) for high-fidelity hand texture priors, 2) Dual-Stream Audio-Aware Module (DSAA) for separate modeling of lip sync and head pose dynamics with interaction, 3) Pose Calibration Trick (PCT) for refining pose conditions with relaxed constraints.

Result: Extensive experiments show state-of-the-art performance with superior hand detail, gesture realism, and identity consistency, validated by both quantitative metrics and qualitative evaluations.

Conclusion: VividAnimator effectively addresses key limitations in human animation by combining HCC, DSAA, and PCT to generate high-quality animations with natural head movements and clear hand details.

Abstract: Existing for audio- and pose-driven human animation methods often struggle with stiff head movements and blurry hands, primarily due to the weak correlation between audio and head movements and the structural complexity of hands. To address these issues, we propose VividAnimator, an end-to-end framework for generating high-quality, half-body human animations driven by audio and sparse hand pose conditions. Our framework introduces three key innovations. First, to overcome the instability and high cost of online codebook training, we pre-train a Hand Clarity Codebook (HCC) that encodes rich, high-fidelity hand texture priors, significantly mitigating hand degradation. Second, we design a Dual-Stream Audio-Aware Module (DSAA) to model lip synchronization and natural head pose dynamics separately while enabling interaction. Third, we introduce a Pose Calibration Trick (PCT) that refines and aligns pose conditions by relaxing rigid constraints, ensuring smooth and natural gesture transitions. Extensive experiments demonstrate that Vivid Animator achieves state-of-the-art performance, producing videos with superior hand detail, gesture realism, and identity consistency, validated by both quantitative metrics and qualitative evaluations.

Method: Proposes a hybrid framework that integrates PV features with BEV maps enriched with semantic and geometric features from DINOv2 foundation model, using a novel distillation process and deformable aggregation to combine both representations.

Result: Achieves state-of-the-art performance on nuScenes and Argoverse 2 benchmarks, demonstrating improved 3D object detection and tracking capabilities for autonomous driving.

Conclusion: The framework successfully leverages foundation model BEV maps to enable more reliable perception, showing the potential of combining PV and BEV representations with foundation model features for autonomous driving applications.

Abstract: Camera-based 3D object detection and tracking are essential for perception in autonomous driving. Current state-of-the-art approaches often rely exclusively on either perspective-view (PV) or bird’s-eye-view (BEV) features, limiting their ability to leverage both fine-grained object details and spatially structured scene representations. In this work, we propose DualViewDistill, a hybrid detection and tracking framework that incorporates both PV and BEV camera image features to leverage their complementary strengths. Our approach introduces BEV maps guided by foundation models, leveraging descriptive DINOv2 features that are distilled into BEV representations through a novel distillation process. By integrating PV features with BEV maps enriched with semantic and geometric features from DINOv2, our model leverages this hybrid representation via deformable aggregation to enhance 3D object detection and tracking. Extensive experiments on the nuScenes and Argoverse 2 benchmarks demonstrate that DualViewDistill achieves state-of-the-art performance. The results showcase the potential of foundation model BEV maps to enable more reliable perception for autonomous driving. We make the code and pre-trained models available at https://dualviewdistill.cs.uni-freiburg.de .

Method: Integrates low-rank adapters into both image encoder and mask decoder of SAM2, using a composite loss function combining segmentationBCE, SoftDice, and FocalTversky losses.

Result: Achieves Dice scores of 0.86 for blood vessel and 0.93 for optic disc segmentation, with AUC values up to 0.98 and 0.99 on 11 fundus datasets, demonstrating state-of-the-art performance.

Conclusion: SAM2LoRA enables efficient adaptation of SAM2 for fundus segmentation with substantially reduced training overhead while maintaining high performance.

Abstract: We propose SAM2LoRA, a parameter-efficient fine-tuning strategy that adapts the Segment Anything Model 2 (SAM2) for fundus image segmentation. SAM2 employs a masked autoencoder-pretrained Hierarchical Vision Transformer for multi-scale feature decoding, enabling rapid inference in low-resource settings; however, fine-tuning remains challenging. To address this, SAM2LoRA integrates a low-rank adapter into both the image encoder and mask decoder, requiring fewer than 5% of the original trainable parameters. Our analysis indicates that for cross-dataset fundus segmentation tasks, a composite loss function combining segmentationBCE, SoftDice, and FocalTversky losses is essential for optimal network tuning. Evaluated on 11 challenging fundus segmentation datasets, SAM2LoRA demonstrates high performance in both blood vessel and optic disc segmentation under cross-dataset training conditions. It achieves Dice scores of up to 0.86 and 0.93 for blood vessel and optic disc segmentation, respectively, and AUC values of up to 0.98 and 0.99, achieving state-of-the-art performance while substantially reducing training overhead.

Method: Uses a factored representation that decomposes scenes into room programs and object poses, learns layout patterns from a function library, uses LLMs for program generation, and employs program-conditioned hierarchical pose prediction with 3D object retrieval.

Result: Generates realistic, real-world rooms that are difficult to distinguish from real ScanNet scenes.

Conclusion: FactoredScenes successfully synthesizes realistic 3D scenes by leveraging structural decomposition and learning object pose variations from lived-in environments.

Abstract: Real-world scenes, such as those in ScanNet, are difficult to capture, with highly limited data available. Generating realistic scenes with varied object poses remains an open and challenging task. In this work, we propose FactoredScenes, a framework that synthesizes realistic 3D scenes by leveraging the underlying structure of rooms while learning the variation of object poses from lived-in scenes. We introduce a factored representation that decomposes scenes into hierarchically organized concepts of room programs and object poses. To encode structure, FactoredScenes learns a library of functions capturing reusable layout patterns from which scenes are drawn, then uses large language models to generate high-level programs, regularized by the learned library. To represent scene variations, FactoredScenes learns a program-conditioned model to hierarchically predict object poses, and retrieves and places 3D objects in a scene. We show that FactoredScenes generates realistic, real-world rooms that are difficult to distinguish from real ScanNet scenes.

Method: Multimodal framework combining CLIP for visual-language reasoning, YOLO-World for object detection, and MOG2-based background subtraction for motion analysis, with motion-based confidence weighting and annotated visual outputs.

Result: Achieves 76.7% accuracy, F1 score of 0.752, and Quadratic Weighted Kappa of 0.684, significantly outperforming unimodal baselines.

Conclusion: The framework effectively preserves ordinal structure and leverages visual-language and motion modalities, with future enhancements including vehicle sizing and refined density metrics.

Abstract: Accurate traffic congestion classification is essential for intelligent transportation systems and real-time urban traffic management. This paper presents a multimodal framework combining open-vocabulary visual-language reasoning (CLIP), object detection (YOLO-World), and motion analysis via MOG2-based background subtraction. The system predicts congestion levels on an ordinal scale from 1 (free flow) to 5 (severe congestion), enabling semantically aligned and temporally consistent classification. To enhance interpretability, we incorporate motion-based confidence weighting and generate annotated visual outputs. Experimental results show the model achieves 76.7 percent accuracy, an F1 score of 0.752, and a Quadratic Weighted Kappa (QWK) of 0.684, significantly outperforming unimodal baselines. These results demonstrate the framework’s effectiveness in preserving ordinal structure and leveraging visual-language and motion modalities. Future enhancements include incorporating vehicle sizing and refined density metrics.

Method: The approach uses intermediate optical flow estimation to synthesize transitional imagery between consecutive aerial frames, artificially augmenting feature correspondences for improved geometric reconstruction.

Result: Experimental validation demonstrates a 20% reduction in minimum overlap requirements for orthomosaic generation.

Conclusion: The framework addresses adoption barriers in precision agriculture and provides pathways for enhanced integration of AI-driven monitoring systems by enabling reliable operation with reduced data requirements.

Abstract: AI-driven crop health mapping systems offer substantial advantages over conventional monitoring approaches through accelerated data acquisition and cost reduction. However, widespread farmer adoption remains constrained by technical limitations in orthomosaic generation from sparse aerial imagery datasets. Traditional photogrammetric reconstruction requires 70-80% inter-image overlap to establish sufficient feature correspondences for accurate geometric registration. AI-driven systems operating under resource-constrained conditions cannot consistently achieve these overlap thresholds, resulting in degraded reconstruction quality that undermines user confidence in autonomous monitoring technologies. In this paper, we present Ortho-Fuse, an optical flow-based framework that enables the generation of a reliable orthomosaic with reduced overlap requirements. Our approach employs intermediate flow estimation to synthesize transitional imagery between consecutive aerial frames, artificially augmenting feature correspondences for improved geometric reconstruction. Experimental validation demonstrates a 20% reduction in minimum overlap requirements. We further analyze adoption barriers in precision agriculture to identify pathways for enhanced integration of AI-driven monitoring systems.

Method: The method optimizes the completion model under two self-supervised auxiliary objectives simulating structural and sensor-level incompleteness, using a meta-auxiliary learning strategy based on MAML. During inference, only the shared encoder is adapted on-the-fly while keeping the decoder fixed, with Adaptive λ-Calibration to balance gradients between primary and auxiliary objectives.

Result: Extensive experiments on synthetic, simulated, and real-world datasets demonstrate that PointMAC achieves state-of-the-art results by refining each sample individually to produce high-quality completions.

Conclusion: This is the first work to apply meta-auxiliary test-time adaptation to point cloud completion, enabling robust adaptation to novel patterns and distortions without requiring additional supervision.

Abstract: Point cloud completion is essential for robust 3D perception in safety-critical applications such as robotics and augmented reality. However, existing models perform static inference and rely heavily on inductive biases learned during training, limiting their ability to adapt to novel structural patterns and sensor-induced distortions at test time. To address this limitation, we propose PointMAC, a meta-learned framework for robust test-time adaptation in point cloud completion. It enables sample-specific refinement without requiring additional supervision. Our method optimizes the completion model under two self-supervised auxiliary objectives that simulate structural and sensor-level incompleteness. A meta-auxiliary learning strategy based on Model-Agnostic Meta-Learning (MAML) ensures that adaptation driven by auxiliary objectives is consistently aligned with the primary completion task. During inference, we adapt the shared encoder on-the-fly by optimizing auxiliary losses, with the decoder kept fixed. To further stabilize adaptation, we introduce Adaptive $\lambda$-Calibration, a meta-learned mechanism for balancing gradients between primary and auxiliary objectives. Extensive experiments on synthetic, simulated, and real-world datasets demonstrate that PointMAC achieves state-of-the-art results by refining each sample individually to produce high-quality completions. To the best of our knowledge, this is the first work to apply meta-auxiliary test-time adaptation to point cloud completion.

Method: Transform 1D PPG signals into 2D image-like representations (STFT, recurrence plots), then fine-tune Vision Foundation Models (DINOv3, SIGLIP-2) using Parameter-Efficient Fine-Tuning techniques.

Result: Achieved state-of-the-art performance on blood pressure estimation and promising results on other vital signs and blood lab measurements, outperforming time-series foundation models.

Conclusion: Vision Foundation Models provide a new powerful class of tools for PPG analysis that are computationally efficient and generalize well across different 2D input representations.

Abstract: Photoplethysmography (PPG) sensor in wearable and clinical devices provides valuable physiological insights in a non-invasive and real-time fashion. Specialized Foundation Models (FM) or repurposed time-series FMs are used to benchmark physiological tasks. Our experiments with fine-tuning FMs reveal that Vision FM (VFM) can also be utilized for this purpose and, in fact, surprisingly leads to state-of-the-art (SOTA) performance on many tasks, notably blood pressure estimation. We leverage VFMs by simply transforming one-dimensional PPG signals into image-like two-dimensional representations, such as the Short-Time Fourier transform (STFT). Using the latest VFMs, such as DINOv3 and SIGLIP-2, we achieve promising performance on other vital signs and blood lab measurement tasks as well. Our proposal, Vision4PPG, unlocks a new class of FMs to achieve SOTA performance with notable generalization to other 2D input representations, including STFT phase and recurrence plots. Our work improves upon prior investigations of vision models for PPG by conducting a comprehensive study, comparing them to state-of-the-art time-series FMs, and demonstrating the general PPG processing ability by reporting results on six additional tasks. Thus, we provide clinician-scientists with a new set of powerful tools that is also computationally efficient, thanks to Parameter-Efficient Fine-Tuning (PEFT) techniques.

Method: Developed Crack-Segmenter with three modules: Scale-Adaptive Embedder for multi-scale feature extraction, Directional Attention Transformer for maintaining crack continuity, and Attention-Guided Fusion for adaptive feature integration.

Result: Outperformed 13 state-of-the-art supervised methods on ten public datasets across all major metrics including mIoU, Dice score, XOR, and Hausdorff Distance.

Conclusion: Annotation-free crack detection is not only feasible but superior to supervised approaches, enabling scalable infrastructure monitoring and advancing self-supervised learning in pavement crack detection.

Abstract: Pavement crack detection has long depended on costly and time-intensive pixel-level annotations, which limit its scalability for large-scale infrastructure monitoring. To overcome this barrier, this paper examines the feasibility of achieving effective pixel-level crack segmentation entirely without manual annotations. Building on this objective, a fully self-supervised framework, Crack-Segmenter, is developed, integrating three complementary modules: the Scale-Adaptive Embedder (SAE) for robust multi-scale feature extraction, the Directional Attention Transformer (DAT) for maintaining linear crack continuity, and the Attention-Guided Fusion (AGF) module for adaptive feature integration. Through evaluations on ten public datasets, Crack-Segmenter consistently outperforms 13 state-of-the-art supervised methods across all major metrics, including mean Intersection over Union (mIoU), Dice score, XOR, and Hausdorff Distance (HD). These findings demonstrate that annotation-free crack detection is not only feasible but also superior, enabling transportation agencies and infrastructure managers to conduct scalable and cost-effective monitoring. This work advances self-supervised learning and motivates pavement cracks detection research.

Method: Two approaches: 1) Dividing images into smaller non-overlapping tiles and shuffling them randomly, 2) Applying image transforms (Fourier, Wavelet transforms, Median filter) and their combinations to recover background noise information used by CNNs.

Result: The methods were tested on six different datasets (natural, synthetic, hybrid) and effectively distinguished between contextual information and background noise, detecting background noise presence without requiring background information.

Conclusion: The proposed techniques successfully identify hidden biases in CNNs and can distinguish between meaningful contextual learning and problematic background noise, addressing the black box problem in CNN-based image classification.

Abstract: CNNs are now prevalent as the primary choice for most machine vision problems due to their superior rate of classification and the availability of user-friendly libraries. These networks effortlessly identify and select features in a non-intuitive data-driven manner, making it difficult to determine which features were most influential. That leads to a ``black box", where users cannot know how the image data are analyzed but rely on empirical results. Therefore the decision-making process can be biased by background information that is difficult to detect. Here we discuss examples of such hidden biases and propose techniques for identifying them, methods to distinguish between contextual information and background noise, and explore whether CNNs learn from irrelevant features. One effective approach to identify dataset bias is to classify blank background parts of the images. However, in some situations a blank background in the images is not available, making it more difficult to separate the foreground information from the blank background. Such parts of the image can also be considered contextual learning, not necessarily bias. To overcome this, we propose two approaches that were tested on six different datasets, including natural, synthetic, and hybrid datasets. The first method involves dividing images into smaller, non-overlapping tiles of various sizes, which are then shuffled randomly, making classification more challenging. The second method involves the application of several image transforms, including Fourier, Wavelet transforms, and Median filter, and their combinations. These transforms help recover background noise information used by CNN to classify images. Results indicate that this method can effectively distinguish between contextual information and background noise, and alert on the presence of background noise even without the need to use background information.

Method: Two-stage post-training pipeline: (1) AVoCaDO SFT - fine-tuning on 107K high-quality temporally-aligned audiovisual captions, (2) AVoCaDO GRPO - using tailored reward functions to enhance temporal coherence and dialogue accuracy while regularizing caption length.

Result: Significantly outperforms existing open-source models across four audiovisual video captioning benchmarks, and achieves competitive performance on VDC and DREAM-1K benchmarks under visual-only settings.

Conclusion: AVoCaDO demonstrates the effectiveness of temporal orchestration between audio and visual modalities for audiovisual video captioning, establishing new state-of-the-art performance.

Abstract: Audiovisual video captioning aims to generate semantically rich descriptions with temporal alignment between visual and auditory events, thereby benefiting both video understanding and generation. In this paper, we present AVoCaDO, a powerful audiovisual video captioner driven by the temporal orchestration between audio and visual modalities. We propose a two-stage post-training pipeline: (1) AVoCaDO SFT, which fine-tunes the model on a newly curated dataset of 107K high-quality, temporally-aligned audiovisual captions; and (2) AVoCaDO GRPO, which leverages tailored reward functions to further enhance temporal coherence and dialogue accuracy while regularizing caption length and reducing collapse. Experimental results demonstrate that AVoCaDO significantly outperforms existing open-source models across four audiovisual video captioning benchmarks, and also achieves competitive performance on the VDC and DREAM-1K benchmark under visual-only settings.

Method: Mesh-Gait reconstructs 3D heatmaps as intermediate representations from 2D silhouettes, enabling effective capture of 3D geometric information. It uses supervised learning with loss calculated between reconstructed 3D joints, virtual markers, meshes and their ground truth for precise spatial alignment.

Result: Extensive experiments demonstrate that Mesh-Gait achieves state-of-the-art accuracy in gait recognition while maintaining computational efficiency.

Conclusion: Mesh-Gait effectively combines 2D silhouette and 3D geometric information through intermediate 3D heatmap reconstruction, providing robust gait recognition with computational efficiency suitable for real-time applications.

Abstract: Gait recognition, a fundamental biometric technology, leverages unique walking patterns for individual identification, typically using 2D representations such as silhouettes or skeletons. However, these methods often struggle with viewpoint variations, occlusions, and noise. Multi-modal approaches that incorporate 3D body shape information offer improved robustness but are computationally expensive, limiting their feasibility for real-time applications. To address these challenges, we introduce Mesh-Gait, a novel end-to-end multi-modal gait recognition framework that directly reconstructs 3D representations from 2D silhouettes, effectively combining the strengths of both modalities. Compared to existing methods, directly learning 3D features from 3D joints or meshes is complex and difficult to fuse with silhouette-based gait features. To overcome this, Mesh-Gait reconstructs 3D heatmaps as an intermediate representation, enabling the model to effectively capture 3D geometric information while maintaining simplicity and computational efficiency. During training, the intermediate 3D heatmaps are gradually reconstructed and become increasingly accurate under supervised learning, where the loss is calculated between the reconstructed 3D joints, virtual markers, and 3D meshes and their corresponding ground truth, ensuring precise spatial alignment and consistent 3D structure. Mesh-Gait extracts discriminative features from both silhouettes and reconstructed 3D heatmaps in a computationally efficient manner. This design enables the model to capture spatial and structural gait characteristics while avoiding the heavy overhead of direct 3D reconstruction from RGB videos, allowing the network to focus on motion dynamics rather than irrelevant visual details. Extensive experiments demonstrate that Mesh-Gait achieves state-of-the-art accuracy. The code will be released upon acceptance of the paper.

Method: Multi-modal and multi-task framework combining 2D temporal silhouettes with 3D SMPL features, using unified transformer for feature fusion and joint gait recognition with human attribute estimation (age, BMI, gender).

Result: Outperforms state-of-the-art methods on large-scale BRIAR datasets under challenging conditions (1km distance, 50° pitch angles), providing accurate gait recognition and human attribute estimation.

Conclusion: Multi-modal and multitask learning effectively advances gait-based human understanding in real-world scenarios by capturing comprehensive walking patterns and additional human attributes.

Abstract: Gait recognition is an important biometric for human identification at a distance, particularly under low-resolution or unconstrained environments. Current works typically focus on either 2D representations (e.g., silhouettes and skeletons) or 3D representations (e.g., meshes and SMPLs), but relying on a single modality often fails to capture the full geometric and dynamic complexity of human walking patterns. In this paper, we propose a multi-modal and multi-task framework that combines 2D temporal silhouettes with 3D SMPL features for robust gait analysis. Beyond identification, we introduce a multitask learning strategy that jointly performs gait recognition and human attribute estimation, including age, body mass index (BMI), and gender. A unified transformer is employed to effectively fuse multi-modal gait features and better learn attribute-related representations, while preserving discriminative identity cues. Extensive experiments on the large-scale BRIAR datasets, collected under challenging conditions such as long-range distances (up to 1 km) and extreme pitch angles (up to 50{\deg}), demonstrate that our approach outperforms state-of-the-art methods in gait recognition and provides accurate human attribute estimation. These results highlight the promise of multi-modal and multitask learning for advancing gait-based human understanding in real-world scenarios.

Method: Transfer learning with InceptionV3 model pretrained on ImageNet to extract visual features from VR gameplay videos, followed by LSTM network to capture temporal dynamics and predict cybersickness severity.

Result: Achieved 68.4% classification accuracy for cybersickness severity, surpassing existing models trained solely on video data.

Conclusion: The approach provides a practical tool for VR developers to evaluate and mitigate cybersickness, and lays foundation for future video-based temporal modeling research to enhance user comfort in VR applications.

Abstract: With the rapid advancement of virtual reality (VR) technology, its adoption across domains such as healthcare, education, and entertainment has grown significantly. However, the persistent issue of cybersickness, marked by symptoms resembling motion sickness, continues to hinder widespread acceptance of VR. While recent research has explored multimodal deep learning approaches leveraging data from integrated VR sensors like eye and head tracking, there remains limited investigation into the use of video-based features for predicting cybersickness. In this study, we address this gap by utilizing transfer learning to extract high-level visual features from VR gameplay videos using the InceptionV3 model pretrained on the ImageNet dataset. These features are then passed to a Long Short-Term Memory (LSTM) network to capture the temporal dynamics of the VR experience and predict cybersickness severity over time. Our approach effectively leverages the time-series nature of video data, achieving a 68.4% classification accuracy for cybersickness severity. This surpasses the performance of existing models trained solely on video data, providing a practical tool for VR developers to evaluate and mitigate cybersickness in virtual environments. Furthermore, this work lays the foundation for future research on video-based temporal modeling for enhancing user comfort in VR applications.

Method: HuLiRAG uses a three-stage cascade: 1) “what” - anchor queries to candidate referents via open-vocabulary detection, 2) “where” - spatially resolve with SAM-derived masks for fine-grained precision, 3) “reweight” - adaptively prioritize through local-global alignment trade-off. Mask-guided fine-tuning injects spatial evidence into generation.

Result: Extensive experiments show the human-like cascade improves grounding fidelity and factual consistency while reducing hallucinations in multimodal question answering.

Conclusion: HuLiRAG advances multimodal question answering toward trustworthy reasoning by transforming grounding from a passive bias into an explicit constraint on answer formulation through human-like processing stages.

Abstract: Multimodal large language models (MLLMs) often fail in fine-grained visual question answering, producing hallucinations about object identities, positions, and relations because textual queries are not explicitly anchored to visual referents. Retrieval-augmented generation (RAG) alleviates some errors, but it fails to align with human-like processing at both the retrieval and augmentation levels. Specifically, it focuses only on global-level image information but lacks local detail and limits reasoning about fine-grained interactions. To overcome this limitation, we present Human-Like Retrieval-Augmented Generation (HuLiRAG), a framework that stages multimodal reasoning as a ``what–where–reweight’’ cascade. Queries are first anchored to candidate referents via open-vocabulary detection (what), then spatially resolved with SAM-derived masks to recover fine-grained precision (where), and adaptively prioritized through the trade-off between local and global alignment (reweight). Mask-guided fine-tuning further injects spatial evidence into the generation process, transforming grounding from a passive bias into an explicit constraint on answer formulation. Extensive experiments demonstrate that this human-like cascade improves grounding fidelity and factual consistency while reducing hallucinations, advancing multimodal question answering toward trustworthy reasoning.

Method: Uses a two-process framework: (1) visibility-constrained diffusion process for diverse pose augmentation that ensures transformations remain within camera field of view, and (2) timestep-aware reverse process for progressive pose refinement using a coarse-to-fine procedure.

Result: Achieved significant improvements on DREAM and RoboKeyGen benchmarks, with 66.75 AUC on the most challenging dataset - a 32.3% gain over state-of-the-art methods.

Conclusion: The proposed diffusion-based approach demonstrates superior generalization capability and robustness through its visibility-constrained pose augmentation and timestep-aware refinement scheme.

Abstract: We propose MonoSE(3)-Diffusion, a monocular SE(3) diffusion framework that formulates markerless, image-based robot pose estimation as a conditional denoising diffusion process. The framework consists of two processes: a visibility-constrained diffusion process for diverse pose augmentation and a timestep-aware reverse process for progressive pose refinement. The diffusion process progressively perturbs ground-truth poses to noisy transformations for training a pose denoising network. Importantly, we integrate visibility constraints into the process, ensuring the transformations remain within the camera field of view. Compared to the fixed-scale perturbations used in current methods, the diffusion process generates in-view and diverse training poses, thereby improving the network generalization capability. Furthermore, the reverse process iteratively predicts the poses by the denoising network and refines pose estimates by sampling from the diffusion posterior of current timestep, following a scheduled coarse-to-fine procedure. Moreover, the timestep indicates the transformation scales, which guide the denoising network to achieve more accurate pose predictions. The reverse process demonstrates higher robustness than direct prediction, benefiting from its timestep-aware refinement scheme. Our approach demonstrates improvements across two benchmarks (DREAM and RoboKeyGen), achieving a notable AUC of 66.75 on the most challenging dataset, representing a 32.3% gain over the state-of-the-art.

Method: CoDeGraph constructs an image-level graph where images are nodes and edges connect those with shared consistent-anomaly patterns. It identifies “neighbor-burnout” phenomenon where consistent-anomaly patches show abrupt similarity spikes after exhausting limited similar matches, and uses community detection to filter these anomalies.

Result: Achieved 98.3% AUROC for anomaly classification and 66.8% F1 (+4.2%) and 68.1% AP (+5.4%) for segmentation on MVTec AD with ViT-L-14-336. With DINOv2, segmentation improved to 69.1% F1 (+6.5%) and 71.9% AP (+9.2%) over state-of-the-art zero-shot methods.

Conclusion: CoDeGraph effectively addresses the consistent anomaly problem in zero-shot detection through graph-based filtering of neighbor-burnout patterns, demonstrating robustness across different backbone architectures and significant performance improvements.

Abstract: Zero-shot image anomaly classification (AC) and segmentation (AS) are vital for industrial quality control, detecting defects without prior training data. Existing representation-based methods compare patch features with nearest neighbors in unlabeled test images but struggle with consistent anomalies – similar defects recurring across multiple images – resulting in poor AC/AS performance. We introduce Consistent-Anomaly Detection Graph (CoDeGraph), a novel algorithm that identifies and filters consistent anomalies from similarity computations. Our key insight is that normal patches in industrial images show stable, gradually increasing similarity to other test images, while consistent-anomaly patches exhibit abrupt similarity spikes after exhausting a limited set of similar matches, a phenomenon we term ``neighbor-burnout.’’ CoDeGraph constructs an image-level graph, with images as nodes and edges connecting those with shared consistent-anomaly patterns, using community detection to filter these anomalies. We provide a theoretical foundation using Extreme Value Theory to explain the effectiveness of our approach. Experiments on MVTec AD with the ViT-L-14-336 backbone achieve 98.3% AUROC for AC and AS performance of 66.8% (+4.2%) F1 and 68.1% (+5.4%) AP over state-of-the-art zero-shot methods. Using the DINOv2 backbone further improves segmentation, yielding 69.1% (+6.5%) F1 and 71.9% (+9.2%) AP, demonstrating robustness across architectures.

Method: Group decision simulation framework with Expert Signature Generator (ESG) to learn individual annotator styles in latent space, and Simulated Consultation Module (SCM) to intelligently generate final segmentation by sampling from this space.

Result: Achieved state-of-the-art results on challenging CBCT and MRI datasets with 92.11% and 90.72% Dice scores respectively.

Conclusion: By treating expert disagreement as useful signal instead of noise, this approach provides a clear path toward more robust and trustworthy AI systems for healthcare.

Abstract: Medical image segmentation annotation suffers from inter-rater variability (IRV) due to differences in annotators’ expertise and the inherent blurriness of medical images. Standard approaches that simply average expert labels are flawed, as they discard the valuable clinical uncertainty revealed in disagreements. We introduce a fundamentally new approach with our group decision simulation framework, which works by mimicking the collaborative decision-making process of a clinical panel. Under this framework, an Expert Signature Generator (ESG) learns to represent individual annotator styles in a unique latent space. A Simulated Consultation Module (SCM) then intelligently generates the final segmentation by sampling from this space. This method achieved state-of-the-art results on challenging CBCT and MRI datasets (92.11% and 90.72% Dice scores). By treating expert disagreement as a useful signal instead of noise, our work provides a clear path toward more robust and trustworthy AI systems for healthcare.

Method: Three core modules: (1) dual-option segmentation using semi-supervised and foundation model-based pipelines for robust ROI segmentation; (2) multimodal interaction with MGRA, POD, and CGPE techniques for cross-modal feature integration; (3) multi-task prediction with staged training for survival, PPG, and OHE prediction.

Result: Extensive experiments on MultiTIPS dataset demonstrate superiority over state-of-the-art approaches, with strong cross-domain generalization and interpretability.

Conclusion: The proposed framework shows promise for clinical application, providing comprehensive prognostic assessment for TIPS procedures with improved accuracy and robustness.

Abstract: Transjugular intrahepatic portosystemic shunt (TIPS) is an established procedure for portal hypertension, but provides variable survival outcomes and frequent overt hepatic encephalopathy (OHE), indicating the necessity of accurate preoperative prognostic modeling. Current studies typically build machine learning models from preoperative CT images or clinical characteristics, but face three key challenges: (1) labor-intensive region-of-interest (ROI) annotation, (2) poor reliability and generalizability of unimodal methods, and (3) incomplete assessment from single-endpoint prediction. Moreover, the lack of publicly accessible datasets constrains research in this field. Therefore, we present MultiTIPS, the first public multi-center dataset for TIPS prognosis, and propose a novel multimodal prognostic framework based on it. The framework comprises three core modules: (1) dual-option segmentation, which integrates semi-supervised and foundation model-based pipelines to achieve robust ROI segmentation with limited annotations and facilitate subsequent feature extraction; (2) multimodal interaction, where three techniques, multi-grained radiomics attention (MGRA), progressive orthogonal disentanglement (POD), and clinically guided prognostic enhancement (CGPE), are introduced to enable cross-modal feature interaction and complementary representation integration, thus improving model accuracy and robustness; and (3) multi-task prediction, where a staged training strategy is used to perform stable optimization of survival, portal pressure gradient (PPG), and OHE prediction for comprehensive prognostic assessment. Extensive experiments on MultiTIPS demonstrate the superiority of the proposed method over state-of-the-art approaches, along with strong cross-domain generalization and interpretability, indicating its promise for clinical application. The dataset and code are available.

Method: Adaptively mask attention weights of influential image tokens in selected transformer layers to corrupt visual-language perception, then use degradation-induced decoding to emphasize visual contexts.

Result: CMG significantly reduces language bias without harming VLM capabilities, improving performance on hallucination benchmarks without additional training costs.

Conclusion: CMG effectively addresses VLM hallucinations through cross-modal guidance, demonstrating superior advantages with no training requirements and good generalization across different VLMs.

Abstract: Vision-Language Models (VLMs) have shown solid ability for multimodal understanding of both visual and language contexts. However, existing VLMs often face severe challenges of hallucinations, meaning that VLMs tend to generate responses that are only fluent in the language but irrelevant to images in previous contexts. To address this issue, we analyze how language bias contributes to hallucinations and then introduce Cross-Modal Guidance(CMG), a training-free decoding method that addresses the hallucinations by leveraging the difference between the output distributions of the original model and the one with degraded visual-language attention. In practice, we adaptively mask the attention weight of the most influential image tokens in selected transformer layers to corrupt the visual-language perception as a concrete type of degradation. Such a degradation-induced decoding emphasizes the perception of visual contexts and therefore significantly reduces language bias without harming the ability of VLMs. In experiment sections, we conduct comprehensive studies. All results demonstrate the superior advantages of CMG with neither additional conditions nor training costs. We also quantitatively show CMG can improve different VLM’s performance on hallucination-specific benchmarks and generalize effectively.

Method: Three key components: 1) Global-Local Feature Fusion Encoding module to enhance local feature correlation and capture global context, 2) Multi-Branch Feature Extraction network to capture neighborhood information and enhance contour features, 3) Feature Fusion via Deep Feature-guided Attention mechanism for precise cross-channel feature fusion.

Result: Achieves 69.83% on SemanticKITTI validation set and 78.65% on nuScenes validation set, demonstrating high performance while maintaining real-time capability.

Conclusion: DAGLFNet effectively balances high performance with real-time processing, showing great potential for LiDAR-based real-time applications in autonomous navigation and environmental perception.

Abstract: Environmental perception systems play a critical role in high-precision mapping and autonomous navigation, with LiDAR serving as a core sensor that provides accurate 3D point cloud data. How to efficiently process unstructured point clouds while extracting structured semantic information remains a significant challenge, and in recent years, numerous pseudo-image-based representation methods have emerged to achieve a balance between efficiency and performance. However, they often overlook the structural and semantic details of point clouds, resulting in limited feature fusion and discriminability. In this work, we propose DAGLFNet, a pseudo-image-based semantic segmentation framework designed to extract discriminative features. First, the Global-Local Feature Fusion Encoding module is used to enhance the correlation among local features within a set and capture global contextual information. Second, the Multi-Branch Feature Extraction network is employed to capture more neighborhood information and enhance the discriminability of contour features. Finally, a Feature Fusion via Deep Feature-guided Attention mechanism is introduced to improve the precision of cross-channel feature fusion. Experimental evaluations show that DAGLFNet achieves 69.83% and 78.65% on the validation sets of SemanticKITTI and nuScenes, respectively. The method balances high performance with real-time capability, demonstrating great potential for LiDAR-based real-time applications.

Method: Introduces motion-aware state fusion Mamba (MSF-Mamba) that fuses local contextual neighboring states using central frame difference (CFD) for motion awareness. Also proposes MSF-Mamba+ with multiscale motion-aware state fusion and adaptive scale weighting module.

Result: Experiments on two public MGR datasets show that even the lightweight MSF-Mamba achieves state-of-the-art performance, outperforming CNN-, Transformer-, and SSM-based models while maintaining high efficiency.

Conclusion: The proposed MSF-Mamba framework effectively addresses vanilla Mamba’s limitations by enabling motion-aware local spatiotemporal modeling, making it suitable for capturing subtle motion cues in micro-gesture recognition tasks.

Abstract: Micro-gesture recognition (MGR) targets the identification of subtle and fine-grained human motions and requires accurate modeling of both long-range and local spatiotemporal dependencies. While CNNs are effective at capturing local patterns, they struggle with long-range dependencies due to their limited receptive fields. Transformer-based models address this limitation through self-attention mechanisms but suffer from high computational costs. Recently, Mamba has shown promise as an efficient model, leveraging state space models (SSMs) to enable linear-time processing However, directly applying the vanilla Mamba to MGR may not be optimal. This is because Mamba processes inputs as 1D sequences, with state updates relying solely on the previous state, and thus lacks the ability to model local spatiotemporal dependencies. In addition, previous methods lack a design of motion-awareness, which is crucial in MGR. To overcome these limitations, we propose motion-aware state fusion mamba (MSF-Mamba), which enhances Mamba with local spatiotemporal modeling by fusing local contextual neighboring states. Our design introduces a motion-aware state fusion module based on central frame difference (CFD). Furthermore, a multiscale version named MSF-Mamba+ has been proposed. Specifically, MSF-Mamba supports multiscale motion-aware state fusion, as well as an adaptive scale weighting module that dynamically weighs the fused states across different scales. These enhancements explicitly address the limitations of vanilla Mamba by enabling motion-aware local spatiotemporal modeling, allowing MSF-Mamba and MSF-Mamba to effectively capture subtle motion cues for MGR. Experiments on two public MGR datasets demonstrate that even the lightweight version, namely, MSF-Mamba, achieves SoTA performance, outperforming existing CNN-, Transformer-, and SSM-based models while maintaining high efficiency.

Method: Proposes a self-refinement framework based on Triangular Consistency principle with three steps: multi-task instruction tuning, generating image-query-answer triplets from unlabeled images with consistency filtering, and model updating using synthetic data.

Result: Using LLaVA-1.5 as baseline, the model achieved consistent improvements across multiple benchmarks without any external supervision, human annotations, or environmental feedback.

Conclusion: VLMs possess self-refinement capabilities that enable autonomous learning, and these insights can inspire future research on VLM learning mechanisms.

Abstract: Vision-Language Models (VLMs) integrate visual knowledge with the analytical capabilities of Large Language Models (LLMs) through supervised visual instruction tuning, using image-question-answer triplets. However, the potential of VLMs trained without supervised instruction remains largely unexplored. This study validates that VLMs possess inherent self-refinement capabilities, enabling them to generate high-quality supervised data without external inputs and thereby learn autonomously. Specifically, to stimulate the self-refinement ability of VLMs, we propose a self-refinement framework based on a Triangular Consistency principle: within the image-query-answer triangle, any masked elements should be consistently and accurately reconstructed. The framework involves three steps: (1) We enable the instruction generation ability of VLMs by adding multi-task instruction tuning like image$\rightarrow$question-answer or image-answer$\rightarrow$question. (2) We generate image-query-answer triplets from unlabeled images and use the Triangular Consistency principle for filtering. (3) The model is further updated using the filtered synthetic data. To investigate the underlying mechanisms behind this self-refinement capability, we conduct a theoretical analysis from a causal perspective. Using the widely recognized LLaVA-1.5 as our baseline, our experiments reveal that the model can autonomously achieve consistent, though deliberately modest, improvements across multiple benchmarks without any external supervision, such as human annotations or environmental feedback. We expect that the insights of this study on the self-refinement ability of VLMs can inspire future research on the learning mechanism of VLMs. Code is available at https://github.com/dengyl20/SRF-LLaVA-1.5.

Method: Proposes HARoPE which inserts a learnable linear transformation parameterized via singular value decomposition (SVD) before the rotary mapping, enabling dynamic frequency reallocation, semantic alignment of rotary planes, and head-specific positional receptive fields while preserving RoPE’s relative-position property.

Result: Extensive experiments on class-conditional ImageNet and text-to-image generation (Flux and MMDiT) show consistent performance improvements over strong RoPE baselines and other extensions.

Conclusion: HARoPE serves as an effective drop-in replacement that offers a principled and adaptable solution for enhancing positional awareness in transformer-based image generative models.

Abstract: Transformers rely on explicit positional encoding to model structure in data. While Rotary Position Embedding (RoPE) excels in 1D domains, its application to image generation reveals significant limitations such as fine-grained spatial relation modeling, color cues, and object counting. This paper identifies key limitations of standard multi-dimensional RoPE-rigid frequency allocation, axis-wise independence, and uniform head treatment-in capturing the complex structural biases required for fine-grained image generation. We propose HARoPE, a head-wise adaptive extension that inserts a learnable linear transformation parameterized via singular value decomposition (SVD) before the rotary mapping. This lightweight modification enables dynamic frequency reallocation, semantic alignment of rotary planes, and head-specific positional receptive fields while rigorously preserving RoPE’s relative-position property. Extensive experiments on class-conditional ImageNet and text-to-image generation (Flux and MMDiT) demonstrate that HARoPE consistently improves performance over strong RoPE baselines and other extensions. The method serves as an effective drop-in replacement, offering a principled and adaptable solution for enhancing positional awareness in transformer-based image generative models.

Method: Uses jigsaw operations (spatial shuffling and random masking of reference patches) to suppress object semantics and isolate stylistic statistics, then integrates style cues into multi-view diffusion models via cross-attention for consistent stylization.

Result: Achieves high style fidelity and multi-view consistency with substantially lower latency compared to existing methods, and generalizes to partial reference stylization, multi-object scenes, and tileable texture generation.

Conclusion: Jigsaw3D provides an effective pipeline for fast, view-consistent 3D style transfer that successfully decouples style from content while maintaining high fidelity.

Abstract: Controllable 3D style transfer seeks to restyle a 3D asset so that its textures match a reference image while preserving the integrity and multi-view consistency. The prevalent methods either rely on direct reference style token injection or score-distillation from 2D diffusion models, which incurs heavy per-scene optimization and often entangles style with semantic content. We introduce Jigsaw3D, a multi-view diffusion based pipeline that decouples style from content and enables fast, view-consistent stylization. Our key idea is to leverage the jigsaw operation - spatial shuffling and random masking of reference patches - to suppress object semantics and isolate stylistic statistics (color palettes, strokes, textures). We integrate these style cues into a multi-view diffusion model via reference-to-view cross-attention, producing view-consistent stylized renderings conditioned on the input mesh. The renders are then style-baked onto the surface to yield seamless textures. Across standard 3D stylization benchmarks, Jigsaw3D achieves high style fidelity and multi-view consistency with substantially lower latency, and generalizes to masked partial reference stylization, multi-object scene styling, and tileable texture generation. Project page is available at: https://babahui.github.io/jigsaw3D.github.io/

Method: VR-Thinker introduces visual reasoning operations and configurable visual memory windows. It uses a reinforcement fine-tuning pipeline with: (i) Cold Start with visual chain-of-thought data, (ii) Rejection sampling Fine-Tuning on high-quality traces, and (iii) Group Relative Policy Optimization to strengthen reasoning.

Result: Achieves state-of-the-art accuracy on video preference benchmarks: 80.5% on VideoGen Reward, 82.3% on GenAI-Bench, and 75.6% on MJ-Bench-Video with a 7B model, especially effective for longer videos.

Conclusion: The approach validates the effectiveness and promise of thinking-with-image multimodal reward modeling, enabling more reliable and faithful visual reasoning within context limits.

Abstract: Recent advancements in multimodal reward models (RMs) have substantially improved post-training for visual generative models. However, current RMs face inherent limitations: (1) visual inputs consume large context budgets, forcing fewer frames and causing loss of fine-grained details; and (2) all visual information is packed into the initial prompt, exacerbating hallucination and forgetting during chain-of-thought reasoning. To overcome these issues, we introduce VideoReward Thinker (VR-Thinker), a thinking-with-image framework that equips the RM with visual reasoning operations (e.g., select frame) and a configurable visual memory window. This allows the RM to actively acquire and update visual evidence within context limits, improving reasoning fidelity and reliability. We activate visual reasoning via a reinforcement fine-tuning pipeline: (i) Cold Start with curated visual chain-of-thought data to distill basic reasoning skills and operation formatting; (ii) select samples whose per-dimension and overall judgments are all correct, then conduct Rejection sampling Fine-Tuning on these high-quality traces to further enhance reasoning; and (iii) apply Group Relative Policy Optimization (GRPO) to strengthen reasoning. Our approach delivers state-of-the-art accuracy among open-source models on video preference benchmarks, especially for longer videos: a 7B VR-Thinker achieves 80.5% on VideoGen Reward, 82.3% on GenAI-Bench, and 75.6% on MJ-Bench-Video. These results validate the effectiveness and promise of thinking-with-image multimodal reward modeling.

Method: Proposes learning an optimal lattice vector quantizer that adaptively allocates quantization resolution across data dimensions based on significance, irregular dilated convolutions, and a U-shaped cascaded LUT structure to capture multi-level contextual information without increasing table size.

Result: The approach effectively balances speed, accuracy, and memory efficiency, demonstrating significant improvements over existing LUT methods by expanding receptive field while maintaining the same space complexity.

Conclusion: The proposed innovations enable effective expansion of CNN receptive field in LUT-driven inference with fixed table size, achieving better performance than current LUT methods through adaptive quantization and multi-level contextual capture strategies.

Abstract: Recently, several look-up table (LUT) methods were developed to greatly expedite the inference of CNNs in a classical strategy of trading space for speed. However, these LUT methods suffer from a common drawback of limited receptive field of the convolution kernels due to the combinatorial explosion of table size. This research aims to expand the CNN receptive field with a fixed table size, thereby enhancing the performance of LUT-driven fast CNN inference while maintaining the same space complexity. To achieve this goal, various techniques are proposed. The main contribution is a novel approach of learning an optimal lattice vector quantizer that adaptively allocates the quantization resolution across data dimensions based on their significance to the inference task. In addition, the lattice vector quantizer offers an inherently more accurate approximation of CNN kernels than scalar quantizer as used in current practice. Furthermore, we introduce other receptive field expansion strategies, including irregular dilated convolutions and a U-shaped cascaded LUT structure, designed to capture multi-level contextual information without inflating table size. Together, these innovations allow our approach to effectively balance speed, accuracy, and memory efficiency, demonstrating significant improvements over existing LUT methods.

Method: COSINE uses foundation models to extract representations for input images and multi-modal prompts, and a SegDecoder to align these representations, model their interaction, and obtain masks specified by input prompts across different granularities.

Result: Comprehensive experiments show significant performance improvements in both open-vocabulary and in-context segmentation tasks. Synergistic collaboration between visual and textual prompts leads to improved generalization over single-modality approaches.

Conclusion: COSINE successfully unifies open-vocabulary and in-context segmentation through multi-modal prompt integration, demonstrating superior performance and better generalization capabilities.

Abstract: In this work, we present COSINE, a unified open-world segmentation model that consolidates open-vocabulary segmentation and in-context segmentation with multi-modal prompts (e.g., text and image). COSINE exploits foundation models to extract representations for an input image and corresponding multi-modal prompts, and a SegDecoder to align these representations, model their interaction, and obtain masks specified by input prompts across different granularities. In this way, COSINE overcomes architectural discrepancies, divergent learning objectives, and distinct representation learning strategies of previous pipelines for open-vocabulary segmentation and in-context segmentation. Comprehensive experiments demonstrate that COSINE has significant performance improvements in both open-vocabulary and in-context segmentation tasks. Our exploratory analyses highlight that the synergistic collaboration between using visual and textual prompts leads to significantly improved generalization over single-modality approaches.

Method: Combines high-precision detection network with unified recognition stage using transformer-based vision model and iterative language modeling for character identification and post-OCR refinement in one seamless process.

Result: Achieves superior accuracy and robustness across multiple international datasets (IR-LPR, UFPR-ALPR, AOLP), outperforming recent segmentation-free approaches.

Conclusion: The framework successfully bridges computer vision and language modeling through embedded pattern analysis, enabling enhanced adaptability for intelligent transportation and surveillance applications.

Abstract: This work presents a pattern-aware framework for automatic license plate recognition (ALPR), designed to operate reliably across diverse plate layouts and challenging real-world conditions. The proposed system consists of a modern, high-precision detection network followed by a recognition stage that integrates a transformer-based vision model with an iterative language modelling mechanism. This unified recognition stage performs character identification and post-OCR refinement in a seamless process, learning the structural patterns and formatting rules specific to license plates without relying on explicit heuristic corrections or manual layout classification. Through this design, the system jointly optimizes visual and linguistic cues, enables iterative refinement to improve OCR accuracy under noise, distortion, and unconventional fonts, and achieves layout-independent recognition across multiple international datasets (IR-LPR, UFPR-ALPR, AOLP). Experimental results demonstrate superior accuracy and robustness compared to recent segmentation-free approaches, highlighting how embedding pattern analysis within the recognition stage bridges computer vision and language modelling for enhanced adaptability in intelligent transportation and surveillance applications.

Method: Integration of three data sources: Sentinel-2 multispectral imagery for spatial monitoring, NASA ITS_LIVE velocity products for glacier kinematics, and MODIS Land Surface Temperature records. Preprocessing included cloud masking, quality filtering, normalization, temporal interpolation, augmentation, and cyclical encoding.

Result: The dataset reveals seasonal glacier velocity cycles, long-term warming of ~0.8 K per decade, and spatial heterogeneity in cryospheric conditions. GLOFNet is publicly available and addresses challenges like class imbalance, cloud contamination, and coarse resolution.

Conclusion: GLOFNet provides a structured foundation for benchmarking multimodal deep learning approaches to rare hazard prediction, supporting future research in glacial hazard forecasting.

Abstract: Glacial Lake Outburst Floods (GLOFs) are rare but destructive hazards in high mountain regions, yet predictive research is hindered by fragmented and unimodal data. Most prior efforts emphasize post-event mapping, whereas forecasting requires harmonized datasets that combine visual indicators with physical precursors. We present GLOFNet, a multimodal dataset for GLOF monitoring and prediction, focused on the Shisper Glacier in the Karakoram. It integrates three complementary sources: Sentinel-2 multispectral imagery for spatial monitoring, NASA ITS_LIVE velocity products for glacier kinematics, and MODIS Land Surface Temperature records spanning over two decades. Preprocessing included cloud masking, quality filtering, normalization, temporal interpolation, augmentation, and cyclical encoding, followed by harmonization across modalities. Exploratory analysis reveals seasonal glacier velocity cycles, long-term warming of ~0.8 K per decade, and spatial heterogeneity in cryospheric conditions. The resulting dataset, GLOFNet, is publicly available to support future research in glacial hazard prediction. By addressing challenges such as class imbalance, cloud contamination, and coarse resolution, GLOFNet provides a structured foundation for benchmarking multimodal deep learning approaches to rare hazard prediction.

Method: Proposed MAKDF module for multi-scale feature extraction, RCFPN neck structure for feature integration, SC_Detect head with spatial and channel preprocessing, and channel pruning for model compression.

Result: mAP50 improved to 94.8% (0.7% higher than baseline), mAP50:95 improved to 86.4% (2.3% higher), while reducing Parameters by 44.2% and GFLOPs by 17.5%.

Conclusion: The improved algorithm demonstrates better performance and efficiency for railroad transmission line foreign object detection tasks.

Abstract: Aiming at the problems of missed detection, false detection and low detection efficiency in transmission line foreign object detection under railway environment, we proposed an improved algorithm MRS-YOLO based on YOLO11. Firstly, a multi-scale Adaptive Kernel Depth Feature Fusion (MAKDF) module is proposed and fused with the C3k2 module to form C3k2_MAKDF, which enhances the model’s feature extraction capability for foreign objects of different sizes and shapes. Secondly, a novel Re-calibration Feature Fusion Pyramid Network (RCFPN) is designed as a neck structure to enhance the model’s ability to integrate and utilize multi-level features effectively. Then, Spatial and Channel Reconstruction Detect Head (SC_Detect) based on spatial and channel preprocessing is designed to enhance the model’s overall detection performance. Finally, the channel pruning technique is used to reduce the redundancy of the improved model, drastically reduce Parameters and Giga Floating Point Operations Per Second (GFLOPs), and improve the detection efficiency. The experimental results show that the mAP50 and mAP50:95 of the MRS-YOLO algorithm proposed in this paper are improved to 94.8% and 86.4%, respectively, which are 0.7 and 2.3 percentage points higher compared to the baseline, while Parameters and GFLOPs are reduced by 44.2% and 17.5%, respectively. It is demonstrated that the improved algorithm can be better applied to the task of foreign object detection in railroad transmission lines.

Method: Proposed deep semi-supervised approach combining consistency regularization with similarity learning using a deep auto-encoder architecture.

Result: Experiments on DeepWeeds dataset and noisy conditions demonstrated effectiveness and robustness compared to state-of-the-art fully supervised models. Ablation studies confirmed the joint learning strategy.

Conclusion: The proposed method effectively utilizes unlabeled data, provides robust classification performance, and addresses the limitation of scarce labeled data in agricultural weed classification applications.

Abstract: Weed species classification represents an important step for the development of automated targeting systems that allow the adoption of precision agriculture practices. To reduce costs and yield losses caused by their presence. The identification of weeds is a challenging problem due to their shared similarities with crop plants and the variability related to the differences in terms of their types. Along with the variations in relation to changes in field conditions. Moreover, to fully benefit from deep learning-based methods, large fully annotated datasets are needed. This requires time intensive and laborious process for data labeling, which represents a limitation in agricultural applications. Hence, for the aim of improving the utilization of the unlabeled data, regarding conditions of scarcity in terms of the labeled data available during the learning phase and provide robust and high classification performance. We propose a deep semi-supervised approach, that combines consistency regularization with similarity learning. Through our developed deep auto-encoder architecture, experiments realized on the DeepWeeds dataset and inference in noisy conditions demonstrated the effectiveness and robustness of our method in comparison to state-of-the-art fully supervised deep learning models. Furthermore, we carried out ablation studies for an extended analysis of our proposed joint learning strategy.

Method: Proposes UniFlow with layer-wise adaptive self-distillation applied to pretrained visual encoders, and a lightweight patch-wise pixel flow decoder that models conditional flow from noisy state to pixel domain using semantic features as conditions.

Result: Achieves win-win outcomes across 13 benchmarks spanning 7 visual tasks. 7B UniFlow-XL surpasses 14B TokenFlow-XL by 7.75% on understanding benchmarks and achieves competitive results in visual reconstruction/generation, outperforming UniTok by 0.15 in rFID and 0.09 in gFID.

Conclusion: UniFlow successfully addresses the understanding-generation trade-off through its adaptive self-distillation and patch-wise flow decoder design, demonstrating superior performance across diverse visual tasks without compromising either capability.

Abstract: Tokenizer is a crucial component for both visual understanding and generation. To advance toward the ultimate goal of universal modeling, recent research has focused on developing a unified tokenizer. However, existing tokenizers face a significant performance trade-off between understanding and generation, stemming from the inherent conflict between high-level semantic abstraction and low-level pixel reconstruction. To tackle this challenge, we propose a generic and unified tokenizer, namely UniFlow, by flexibly adapting any visual encoder with a concise reconstruction decoder. Specifically, we introduce layer-wise adaptive self-distillation applied to the well-pretrained visual encoders, which enables UniFlow to simultaneously inherit the strong semantic features for visual understanding and flexibly adapt to model fine-grained details for visual generation. Moreover, we propose a lightweight patch-wise pixel flow decoder, which efficiently achieves high-fidelity pixel reconstruction by modeling a conditional flow from the noisy state back to the patch-wise pixel domain. By leveraging the semantic features as visual conditions for the decoder, we effectively alleviate the training conflicts between understanding and generation. Furthermore, the patch-wise learning strategy simplifies the data distribution, thereby improving training efficiency. Extensive experiments across 13 challenging benchmarks spanning 7 widely studied visual understanding and generation tasks demonstrate that UniFlow achieves a win-win outcome. For instance, our 7B UniFlow-XL not only surpasses the 14B TokenFlow-XL by 7.75% on average understanding benchmarks, but also achieves competitive results in both visual reconstruction and generation, surpassing UniTok by 0.15 in rFID and 0.09 in gFID (without guidance), respectively.

Method: Proposes Diff-ABFlow framework based on diffusion models with frame-event appearance-boundary fusion. Instead of learning mapping from visual features to motion fields, it learns mapping from noisy flow to clear flow using diffusion models, leveraging complementary strengths of frame cameras (dense appearance) and event cameras (dense boundary completeness).

Result: The method addresses the limitations of both discriminative and generative models that are affected by deteriorated visual features in challenging scenes, providing a more robust optical flow estimation approach.

Conclusion: Diffusion models offer a promising alternative for optical flow estimation in degraded scenes by learning flow-to-flow mappings rather than relying on deteriorated visual features, with frame-event fusion providing complementary appearance and boundary information.

Abstract: Optical flow estimation has achieved promising results in conventional scenes but faces challenges in high-speed and low-light scenes, which suffer from motion blur and insufficient illumination. These conditions lead to weakened texture and amplified noise and deteriorate the appearance saturation and boundary completeness of frame cameras, which are necessary for motion feature matching. In degraded scenes, the frame camera provides dense appearance saturation but sparse boundary completeness due to its long imaging time and low dynamic range. In contrast, the event camera offers sparse appearance saturation, while its short imaging time and high dynamic range gives rise to dense boundary completeness. Traditionally, existing methods utilize feature fusion or domain adaptation to introduce event to improve boundary completeness. However, the appearance features are still deteriorated, which severely affects the mostly adopted discriminative models that learn the mapping from visual features to motion fields and generative models that generate motion fields based on given visual features. So we introduce diffusion models that learn the mapping from noising flow to clear flow, which is not affected by the deteriorated visual features. Therefore, we propose a novel optical flow estimation framework Diff-ABFlow based on diffusion models with frame-event appearance-boundary fusion.

Method: 1) Systematic investigation of vision foundation models for OOD detection; 2) Proposed Mixture of Feature Experts (MoFE) module to partition features into subspaces; 3) Dynamic-β Mixup strategy with adaptive interpolation weights from beta distribution.

Result: Pre-trained DINOv2 achieves OOD detection performance comparable to state-of-the-art methods without fine-tuning. The proposed MoFE and Dynamic-β Mixup significantly outperform baseline methods, especially in large semantic space scenarios.

Conclusion: Vision foundation models provide strong OOD detection capabilities, and the proposed MoFE module with Dynamic-β Mixup effectively addresses challenges in large semantic spaces by refining decision boundaries and adapting to varying learning difficulties.

Abstract: Pre-trained vision foundation models have transformed many computer vision tasks. Despite their strong ability to learn discriminative and generalizable features crucial for out-of-distribution (OOD) detection, their impact on this task remains underexplored. Motivated by this gap, we systematically investigate representative vision foundation models for OOD detection. Our findings reveal that a pre-trained DINOv2 model, even without fine-tuning on in-domain (ID) data, naturally provides a highly discriminative feature space for OOD detection, achieving performance comparable to existing state-of-the-art methods without requiring complex designs. Beyond this, we explore how fine-tuning foundation models on in-domain (ID) data can enhance OOD detection. However, we observe that the performance of vision foundation models remains unsatisfactory in scenarios with a large semantic space. This is due to the increased complexity of decision boundaries as the number of categories grows, which complicates the optimization process. To mitigate this, we propose the Mixture of Feature Experts (MoFE) module, which partitions features into subspaces, effectively capturing complex data distributions and refining decision boundaries. Further, we introduce a Dynamic-$\beta$ Mixup strategy, which samples interpolation weights from a dynamic beta distribution. This adapts to varying levels of learning difficulty across categories, improving feature learning for more challenging categories. Extensive experiments demonstrate the effectiveness of our approach, significantly outperforming baseline methods.

Method: FSVG encapsulates linguistic and visual modalities in a single network without iterations, using language as parallel guidance for modality interaction. It employs similarity-based feature selection to focus only on language-relevant visual features.

Result: Extensive experiments on benchmark datasets show FSVG achieves better balance between accuracy and efficiency compared to state-of-the-art methods.

Conclusion: FSVG provides an effective baseline for visual grounding that reduces computational overhead while maintaining competitive performance through streamlined architecture and feature selection.

Abstract: Visual grounding aims to predict the locations of target objects specified by textual descriptions. For this task with linguistic and visual modalities, there is a latest research line that focuses on only selecting the linguistic-relevant visual regions for object localization to reduce the computational overhead. Albeit achieving impressive performance, it is iteratively performed on different image scales, and at every iteration, linguistic features and visual features need to be stored in a cache, incurring extra overhead. To facilitate the implementation, in this paper, we propose a feature selection-based simple yet effective baseline for visual grounding, called FSVG. Specifically, we directly encapsulate the linguistic and visual modalities into an overall network architecture without complicated iterative procedures, and utilize the language in parallel as guidance to facilitate the interaction between linguistic modal and visual modal for extracting effective visual features. Furthermore, to reduce the computational cost, during the visual feature learning, we introduce a similarity-based feature selection mechanism to only exploit language-related visual features for faster prediction. Extensive experiments conducted on several benchmark datasets comprehensively substantiate that the proposed FSVG achieves a better balance between accuracy and efficiency beyond the current state-of-the-art methods. Code is available at https://github.com/jcwang0602/FSVG.

Method: ViSurf integrates SFT and RLVR in a single stage by injecting ground-truth labels into RLVR rollouts, providing simultaneous external supervision and internal reinforcement. Three novel reward control strategies are introduced to stabilize training.

Result: Extensive experiments across diverse benchmarks show ViSurf outperforms individual SFT, RLVR, and two-stage SFT→RLVR approaches.

Conclusion: ViSurf provides an effective unified framework that validates the integration of supervised and reinforcement learning paradigms for enhanced LVLM performance.

Abstract: Typical post-training paradigms for Large Vision-and-Language Models (LVLMs) include Supervised Fine-Tuning (SFT) and Reinforcement Learning with Verifiable Rewards (RLVR). SFT leverages external guidance to inject new knowledge, whereas RLVR utilizes internal reinforcement to enhance reasoning capabilities and overall performance. However, our analysis reveals that SFT often leads to sub-optimal performance, while RLVR struggles with tasks that exceed the model’s internal knowledge base. To address these limitations, we propose ViSurf (\textbf{Vi}sual \textbf{Su}pervised-and-\textbf{R}einforcement \textbf{F}ine-Tuning), a unified post-training paradigm that integrates the strengths of both SFT and RLVR within a single stage. We analyze the derivation of the SFT and RLVR objectives to establish the ViSurf objective, providing a unified perspective on these two paradigms. The core of ViSurf involves injecting ground-truth labels into the RLVR rollouts, thereby providing simultaneous external supervision and internal reinforcement. Furthermore, we introduce three novel reward control strategies to stabilize and optimize the training process. Extensive experiments across several diverse benchmarks demonstrate the effectiveness of ViSurf, outperforming both individual SFT, RLVR, and two-stage SFT \textrightarrow RLVR. In-depth analysis corroborates these findings, validating the derivation and design principles of ViSurf.

Method: Construct reasoning-enhanced reward modeling dataset via rejection sampling, apply Group Relative Policy Optimization with Gaussian-based reward, and incorporate STD filtering and entropy gating for stable training.

Result: Evaluated on three key IQA tasks: aesthetic quality assessment, technical quality evaluation, and text-image alignment.

Conclusion: The framework successfully enables multi-dimensional quality reasoning and provides interpretable reward signals for policy optimization across multiple visual evaluation tasks.

Abstract: Current visual evaluation approaches are typically constrained to a single task. To address this, we propose OmniQuality-R, a unified reward modeling framework that transforms multi-task quality reasoning into continuous and interpretable reward signals for policy optimization. Inspired by subjective experiments, where participants are given task-specific instructions outlining distinct assessment principles prior to evaluation, we propose OmniQuality-R, a structured reward modeling framework that transforms multi-dimensional reasoning into continuous and interpretable reward signals. To enable this, we construct a reasoning-enhanced reward modeling dataset by sampling informative plan-reason trajectories via rejection sampling, forming a reliable chain-of-thought (CoT) dataset for supervised fine-tuning (SFT). Building on this, we apply Group Relative Policy Optimization (GRPO) for post-training, using a Gaussian-based reward to support continuous score prediction. To further stabilize the training and improve downstream generalization, we incorporate standard deviation (STD) filtering and entropy gating mechanisms during reinforcement learning. These techniques suppress unstable updates and reduce variance in policy optimization. We evaluate OmniQuality-R on three key IQA tasks: aesthetic quality assessment, technical quality evaluation, and text-image alignment.

Method: Enhances linear attention by attaching gated local graph network to value matrix to increase rank, and introduces learnable log-power function to reduce attention entropy and sharpen focus.

Result: Achieves competitive performance on both homophilic and heterophilic graph benchmarks while preserving linear attention scalability.

Conclusion: The proposed hybrid framework successfully addresses expressiveness limitations of linear attention through rank enhancement and attention focusing, maintaining efficiency benefits.

Abstract: Linear attention mechanisms have emerged as efficient alternatives to full self-attention in Graph Transformers, offering linear time complexity. However, existing linear attention models often suffer from a significant drop in expressiveness due to low-rank projection structures and overly uniform attention distributions. We theoretically prove that these properties reduce the class separability of node representations, limiting the model’s classification ability. To address this, we propose a novel hybrid framework that enhances both the rank and focus of attention. Specifically, we enhance linear attention by attaching a gated local graph network branch to the value matrix, thereby increasing the rank of the resulting attention map. Furthermore, to alleviate the excessive smoothing effect inherent in linear attention, we introduce a learnable log-power function into the attention scores to reduce entropy and sharpen focus. We theoretically show that this function decreases entropy in the attention distribution, enhancing the separability of learned embeddings. Extensive experiments on both homophilic and heterophilic graph benchmarks demonstrate that our method achieves competitive performance while preserving the scalability of linear attention.

Method: Uses a motion auto-encoder to create a structured latent space with orthogonalized motion factors, then applies optimal-transport-based flow matching with a transformer predictor to generate smooth motion trajectories conditioned on audio.

Result: Outperforms prior methods on multiple benchmarks in video realism, lip-audio synchronization, and motion fidelity.

Conclusion: Combining fine-grained motion disentanglement with flow-based generative modeling provides a powerful new paradigm for controllable talking-head video synthesis.

Abstract: Audio-driven talking-head generation has advanced rapidly with diffusion-based generative models, yet producing temporally coherent videos with fine-grained motion control remains challenging. We propose DEMO, a flow-matching generative framework for audio-driven talking-portrait video synthesis that delivers disentangled, high-fidelity control of lip motion, head pose, and eye gaze. The core contribution is a motion auto-encoder that builds a structured latent space in which motion factors are independently represented and approximately orthogonalized. On this disentangled motion space, we apply optimal-transport-based flow matching with a transformer predictor to generate temporally smooth motion trajectories conditioned on audio. Extensive experiments across multiple benchmarks show that DEMO outperforms prior methods in video realism, lip-audio synchronization, and motion fidelity. These results demonstrate that combining fine-grained motion disentanglement with flow-based generative modeling provides a powerful new paradigm for controllable talking-head video synthesis.

Method: Framework for corner case recognition based on data distribution perspective, extends out-of-distribution detection benchmarks, and introduces fog-augmented Lost & Found dataset for combined corner case analysis.

Result: Unifies existing scenario-based corner case taxonomies, achieves strong performance on corner case detection tasks across standard benchmarks, and enables analysis of combined corner cases.

Conclusion: Provides principled basis for corner case recognition with manual specification-free definition, offering scalable approach for safety-critical applications.

Abstract: For high-stakes applications, like autonomous driving, a safe operation is necessary to prevent harm, accidents, and failures. Traditionally, difficult scenarios have been categorized into corner cases and addressed individually. However, this example-based categorization is not scalable and lacks a data coverage perspective, neglecting the generalization to training data of machine learning models. In our work, we propose a novel machine learning approach that takes the underlying data distribution into account. Based on our novel perspective, we present a framework for effective corner case recognition for perception on individual samples. In our evaluation, we show that our approach (i) unifies existing scenario-based corner case taxonomies under a distributional perspective, (ii) achieves strong performance on corner case detection tasks across standard benchmarks for which we extend established out-of-distribution detection benchmarks, and (iii) enables analysis of combined corner cases via a newly introduced fog-augmented Lost & Found dataset. These results provide a principled basis for corner case recognition, underlining our manual specification-free definition.

Method: Proposed a multi-dimensional stability evaluation framework with novel metrics for Presence, Localization, and Shape Stability, integrated into a unified mean Average Stability (mAS) score. Conducted extensive experiments on 42 models and variants.

Result: Accuracy (mAP) and stability (mAS) represent largely independent performance dimensions. Analysis identified architectural and training factors that contribute to high accuracy, high stability, or both.

Conclusion: Temporal stability should be treated as a core evaluation criterion alongside accuracy for online HD mapping models. The authors will release a public benchmark to encourage broader focus on stability in autonomous driving systems.

Abstract: As one of the fundamental modules in autonomous driving, online high-definition (HD) maps have attracted significant attention due to their cost-effectiveness and real-time capabilities. Since vehicles always cruise in highly dynamic environments, spatial displacement of onboard sensors inevitably causes shifts in real-time HD mapping results, and such instability poses fundamental challenges for downstream tasks. However, existing online map construction models tend to prioritize improving each frame’s mapping accuracy, while the mapping stability has not yet been systematically studied. To fill this gap, this paper presents the first comprehensive benchmark for evaluating the temporal stability of online HD mapping models. We propose a multi-dimensional stability evaluation framework with novel metrics for Presence, Localization, and Shape Stability, integrated into a unified mean Average Stability (mAS) score. Extensive experiments on 42 models and variants show that accuracy (mAP) and stability (mAS) represent largely independent performance dimensions. We further analyze the impact of key model design choices on both criteria, identifying architectural and training factors that contribute to high accuracy, high stability, or both. To encourage broader focus on stability, we will release a public benchmark. Our work highlights the importance of treating temporal stability as a core evaluation criterion alongside accuracy, advancing the development of more reliable autonomous driving systems. The benchmark toolkit, code, and models will be available at https://stablehdmap.github.io/.

Method: Proposes FS-VFM framework with three learning objectives (3C) that synergize masked image modeling (MIM) and instance discrimination (ID). Uses facial masking strategies including CRFR-P masking for intra-region consistency and inter-region coherency, and self-distillation to couple MIM with ID for local-to-global correspondence. Also introduces FS-Adapter for efficient transfer learning.

Result: Extensive experiments on 11 public benchmarks show FS-VFM consistently outperforms diverse vision foundation models across natural and facial domains, various supervision paradigms, and ViT scales. Even surpasses state-of-the-art task-specific methods while FS-Adapter provides excellent efficiency-performance trade-off.

Conclusion: FS-VFM successfully learns fundamental facial representations that generalize well across multiple face security tasks, demonstrating the effectiveness of combining MIM and ID with the proposed 3C learning objectives for robust face security applications.

Abstract: With abundant, unlabeled real faces, how can we learn robust and transferable facial representations to boost generalization across various face security tasks? We make the first attempt and propose FS-VFM, a scalable self-supervised pre-training framework, to learn fundamental representations of real face images. We introduce three learning objectives, namely 3C, that synergize masked image modeling (MIM) and instance discrimination (ID), empowering FS-VFM to encode both local patterns and global semantics of real faces. Specifically, we formulate various facial masking strategies for MIM and devise a simple yet effective CRFR-P masking, which explicitly prompts the model to pursue meaningful intra-region Consistency and challenging inter-region Coherency. We present a reliable self-distillation mechanism that seamlessly couples MIM with ID to establish underlying local-to-global Correspondence. After pre-training, vanilla vision transformers (ViTs) serve as universal Vision Foundation Models for downstream Face Security tasks: cross-dataset deepfake detection, cross-domain face anti-spoofing, and unseen diffusion facial forensics. To efficiently transfer the pre-trained FS-VFM, we further propose FS-Adapter, a lightweight plug-and-play bottleneck atop the frozen backbone with a novel real-anchor contrastive objective. Extensive experiments on 11 public benchmarks demonstrate that our FS-VFM consistently generalizes better than diverse VFMs, spanning natural and facial domains, fully, weakly, and self-supervised paradigms, small, base, and large ViT scales, and even outperforms SOTA task-specific methods, while FS-Adapter offers an excellent efficiency-performance trade-off. The code and models are available on https://fsfm-3c.github.io/fsvfm.html.

Method: Two-stage approach: 1) Inject 4D scene representation into T2V model via adaptive learning branch, 2) Formulate viewpoint extraction as hybrid-condition guided camera extrinsic denoising process with additional diffusion branch.

Result: Experimental results show superiority over existing competitors, and ablation studies validate effectiveness of key technical designs.

Conclusion: This work proves the potential of video generation models toward 4D interaction in real world.

Abstract: Recent Text-to-Video (T2V) models have demonstrated powerful capability in visual simulation of real-world geometry and physical laws, indicating its potential as implicit world models. Inspired by this, we explore the feasibility of leveraging the video generation prior for viewpoint planning from given 4D scenes, since videos internally accompany dynamic scenes with natural viewpoints. To this end, we propose a two-stage paradigm to adapt pre-trained T2V models for viewpoint prediction, in a compatible manner. First, we inject the 4D scene representation into the pre-trained T2V model via an adaptive learning branch, where the 4D scene is viewpoint-agnostic and the conditional generated video embeds the viewpoints visually. Then, we formulate viewpoint extraction as a hybrid-condition guided camera extrinsic denoising process. Specifically, a camera extrinsic diffusion branch is further introduced onto the pre-trained T2V model, by taking the generated video and 4D scene as input. Experimental results show the superiority of our proposed method over existing competitors, and ablation studies validate the effectiveness of our key technical designs. To some extent, this work proves the potential of video generation models toward 4D interaction in real world.

Method: Systematically classifies image-to-video transfer learning strategies into two categories: frozen features (preserving original representations) and modified features (undergoing modifications), and analyzes their applications across various video-text learning tasks.

Result: Presents detailed experimental analysis investigating the efficacy of different image-to-video transfer learning paradigms on downstream video understanding tasks.

Conclusion: Identifies prevailing challenges and highlights promising directions for future research, aiming to establish a structured roadmap for advancing video-text learning based on existing image-language foundation models.

Abstract: Image-Language Foundation Models (ILFM) have demonstrated remarkable success in image-text understanding/generation tasks, providing transferable multimodal representations that generalize across diverse downstream image-based tasks. The advancement of video-text research has spurred growing interest in extending image-based models to the video domain. This paradigm, known as image-to-video transfer learning, succeeds in alleviating the substantial data and computational requirements associated with training video-language foundation models from scratch for video-text learning. This survey provides the first comprehensive review of this emerging field, which begins by summarizing the widely used ILFM and their capabilities. We then systematically classify existing image-to-video transfer learning strategies into two categories: frozen features and modified features, depending on whether the original representations from ILFM are preserved or undergo modifications. Building upon the task-specific nature of image-to-video transfer, this survey methodically elaborates these strategies and details their applications across a spectrum of video-text learning tasks, ranging from fine-grained (e.g., spatio-temporal video grounding) to coarse-grained (e.g., video question answering). We further present a detailed experimental analysis to investigate the efficacy of different image-to-video transfer learning paradigms on a range of downstream video understanding tasks. Finally, we identify prevailing challenges and highlight promising directions for future research. By offering a comprehensive and structured overview, this survey aims to establish a structured roadmap for advancing video-text learning based on existing ILFM, and to inspire future research directions in this rapidly evolving domain.

Method: Proposes MSM-Seg with three key components: modality-and-slice memory attention (MSMA) for cross-modal and inter-slice relationships, multi-scale category-agnostic prompt encoder (MCP-Encoder) for tumor guidance, and modality-adaptive fusion decoder (MF-Decoder) for complementary decoding across modalities.

Result: Extensive experiments on different MRI datasets demonstrate that MSM-Seg outperforms state-of-the-art methods in multi-modal metastases and glioma tumor segmentation.

Conclusion: The proposed MSM-Seg framework effectively addresses limitations of existing methods and provides superior performance for multi-modal brain tumor segmentation in clinical applications.

Abstract: Multi-modal brain tumor segmentation is critical for clinical diagnosis, and it requires accurate identification of distinct internal anatomical subregions. While the recent prompt-based segmentation paradigms enable interactive experiences for clinicians, existing methods ignore cross-modal correlations and rely on labor-intensive category-specific prompts, limiting their applicability in real-world scenarios. To address these issues, we propose a MSM-Seg framework for multi-modal brain tumor segmentation. The MSM-Seg introduces a novel dual-memory segmentation paradigm that synergistically integrates multi-modal and inter-slice information with the efficient category-agnostic prompt for brain tumor understanding. To this end, we first devise a modality-and-slice memory attention (MSMA) to exploit the cross-modal and inter-slice relationships among the input scans. Then, we propose a multi-scale category-agnostic prompt encoder (MCP-Encoder) to provide tumor region guidance for decoding. Moreover, we devise a modality-adaptive fusion decoder (MF-Decoder) that leverages the complementary decoding information across different modalities to improve segmentation accuracy. Extensive experiments on different MRI datasets demonstrate that our MSM-Seg framework outperforms state-of-the-art methods in multi-modal metastases and glioma tumor segmentation. The code is available at https://github.com/xq141839/MSM-Seg.

Method: Critical State-Based Memory Compression reduces redundancy; Action Pattern Learning constructs state-transition graphs with multi-dimensional edges; Cross-Temporal Interaction models mutual influence between intentions and past/current information.

Result: Superior performance on EPIC-Kitchens-100, THUMOS'14, TVSeries, and PDMB datasets compared to state-of-the-art approaches.

Conclusion: Demonstrates importance of action dynamics learning and cross-temporal interactions for future action understanding research.

Abstract: Action understanding, encompassing action detection and anticipation, plays a crucial role in numerous practical applications. However, untrimmed videos are often characterized by substantial redundant information and noise. Moreover, in modeling action understanding, the influence of the agent’s intention on the action is often overlooked. Motivated by these issues, we propose a novel framework called the State-Specific Model (SSM), designed to unify and enhance both action detection and anticipation tasks. In the proposed framework, the Critical State-Based Memory Compression module compresses frame sequences into critical states, reducing information redundancy. The Action Pattern Learning module constructs a state-transition graph with multi-dimensional edges to model action dynamics in complex scenarios, on the basis of which potential future cues can be generated to represent intention. Furthermore, our Cross-Temporal Interaction module models the mutual influence between intentions and past as well as current information through cross-temporal interactions, thereby refining present and future features and ultimately realizing simultaneous action detection and anticipation. Extensive experiments on multiple benchmark datasets – including EPIC-Kitchens-100, THUMOS'14, TVSeries, and the introduced Parkinson’s Disease Mouse Behaviour (PDMB) dataset – demonstrate the superior performance of our proposed framework compared to other state-of-the-art approaches. These results highlight the importance of action dynamics learning and cross-temporal interactions, laying a foundation for future action understanding research.

Method: Proposes per-pixel reliable blur kernel estimation using a blur prediction network with blur-aware sparsity constraint, dynamic Gaussian densification for incomplete regions, and incorporation of unseen view information for scene optimization.

Result: Extensive experiments show the method outperforms state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-blurred monocular videos.

Conclusion: The proposed unified framework successfully handles both defocus and motion blur in monocular videos, achieving superior novel view synthesis quality compared to existing specialized approaches.

Abstract: This paper presents a unified framework that allows high-quality dynamic Gaussian Splatting from both defocused and motion-blurred monocular videos. Due to the significant difference between the formation processes of defocus blur and motion blur, existing methods are tailored for either one of them, lacking the ability to simultaneously deal with both of them. Although the two can be jointly modeled as blur kernel-based convolution, the inherent difficulty in estimating accurate blur kernels greatly limits the progress in this direction. In this work, we go a step further towards this direction. Particularly, we propose to estimate per-pixel reliable blur kernels using a blur prediction network that exploits blur-related scene and camera information and is subject to a blur-aware sparsity constraint. Besides, we introduce a dynamic Gaussian densification strategy to mitigate the lack of Gaussians for incomplete regions, and boost the performance of novel view synthesis by incorporating unseen view information to constrain scene optimization. Extensive experiments show that our method outperforms the state-of-the-art methods in generating photorealistic novel view synthesis from defocused and motion-blurred monocular videos. Our code and trained model will be made publicly available.

Method: A feed-forward framework that flexibly integrates camera poses, intrinsics, depth maps and other geometric priors to simultaneously generate dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussians in a single forward pass.

Result: Achieves state-of-the-art performance across diverse benchmarks including camera estimation, point map generation, depth estimation, surface normal estimation, and novel view synthesis, while maintaining efficient feed-forward inference.

Conclusion: WorldMirror provides an elegant and unified architecture that leverages available prior information to resolve structural ambiguities and deliver geometrically consistent 3D outputs efficiently.

Abstract: We present WorldMirror, an all-in-one, feed-forward model for versatile 3D geometric prediction tasks. Unlike existing methods constrained to image-only inputs or customized for a specific task, our framework flexibly integrates diverse geometric priors, including camera poses, intrinsics, and depth maps, while simultaneously generating multiple 3D representations: dense point clouds, multi-view depth maps, camera parameters, surface normals, and 3D Gaussians. This elegant and unified architecture leverages available prior information to resolve structural ambiguities and delivers geometrically consistent 3D outputs in a single forward pass. WorldMirror achieves state-of-the-art performance across diverse benchmarks from camera, point map, depth, and surface normal estimation to novel view synthesis, while maintaining the efficiency of feed-forward inference. Code and models will be publicly available soon.

Method: Proposes a hierarchical framework that identifies interaction targets and forecasts fine-grained behaviors using dynamic GCNs to capture human-environment relationships from historical dynamics and scene contexts.

Result: Achieves superior performance across all metrics on real-world benchmarks and live VR environments, demonstrating effectiveness for proactive VR systems.

Conclusion: The framework enables practical applications for proactive VR systems that anticipate user behaviors and adapt virtual environments accordingly.

Abstract: Virtual and augmented reality systems increasingly demand intelligent adaptation to user behaviors for enhanced interaction experiences. Achieving this requires accurately understanding human intentions and predicting future situated behaviors - such as gaze direction and object interactions - which is vital for creating responsive VR/AR environments and applications like personalized assistants. However, accurate behavioral prediction demands modeling the underlying cognitive processes that drive human-environment interactions. In this work, we introduce a hierarchical, intention-aware framework that models human intentions and predicts detailed situated behaviors by leveraging cognitive mechanisms. Given historical human dynamics and the observation of scene contexts, our framework first identifies potential interaction targets and forecasts fine-grained future behaviors. We propose a dynamic Graph Convolutional Network (GCN) to effectively capture human-environment relationships. Extensive experiments on challenging real-world benchmarks and live VR environment demonstrate the effectiveness of our approach, achieving superior performance across all metrics and enabling practical applications for proactive VR systems that anticipate user behaviors and adapt virtual environments accordingly.

Method: Introduces AURA benchmark dataset, evaluates four VAD models across two marine scenes, implements robust frame selection strategies, and compares against multiple human annotators using soft and consensus labels.

Result: VAD model performance varies dramatically and is highly sensitive to training data amount and visual content variability defining ’normal’ scenes.

Conclusion: The work demonstrates the value of soft/consensus labels and provides a practical approach for supporting scientific exploration and scalable biodiversity monitoring.

Abstract: Underwater video monitoring is a promising strategy for assessing marine biodiversity, but the vast volume of uneventful footage makes manual inspection highly impractical. In this work, we explore the use of visual anomaly detection (VAD) based on deep neural networks to automatically identify interesting or anomalous events. We introduce AURA, the first multi-annotator benchmark dataset for underwater VAD, and evaluate four VAD models across two marine scenes. We demonstrate the importance of robust frame selection strategies to extract meaningful video segments. Our comparison against multiple annotators reveals that VAD performance of current models varies dramatically and is highly sensitive to both the amount of training data and the variability in visual content that defines “normal” scenes. Our results highlight the value of soft and consensus labels and offer a practical approach for supporting scientific exploration and scalable biodiversity monitoring.

Method: A face similarity metric that decomposes global similarity into sum of patch-level similarity scores from restricted receptive fields, providing locally additive explanations without post-hoc analysis.

Result: Achieves competitive verification performance with 28x28 patches in 112x112 face images, and surpasses state-of-the-art methods when using 56x56 patches.

Conclusion: The proposed inherently interpretable approach provides transparent face similarity analysis while maintaining competitive performance, offering an alternative to post-hoc explanation methods.

Abstract: Understanding how deep neural networks make decisions is crucial for analyzing their behavior and diagnosing failure cases. In computer vision, a common approach to improve interpretability is to assign importance to individual pixels using post-hoc methods. Although they are widely used to explain black-box models, their fidelity to the model’s actual reasoning is uncertain due to the lack of reliable evaluation metrics. This limitation motivates an alternative approach, which is to design models whose decision processes are inherently interpretable. To this end, we propose a face similarity metric that breaks down global similarity into contributions from restricted receptive fields. Our method defines the similarity between two face images as the sum of patch-level similarity scores, providing a locally additive explanation without relying on post-hoc analysis. We show that the proposed approach achieves competitive verification performance even with patches as small as 28x28 within 112x112 face images, and surpasses state-of-the-art methods when using 56x56 patches.

Method: Developed EGD-YOLOv8n with enhanced feature capture, attention layers, and a specialized detection head for handling objects of varying shapes and sizes. Trained three versions: RGB-only, IR-only, and combined RGB+IR.

Result: The combined RGB+IR model achieved the best accuracy and reliability while maintaining real-time performance on standard GPUs.

Conclusion: EGD-YOLOv8n provides an effective lightweight solution for drone and bird detection that balances accuracy with computational efficiency for practical deployment.

Abstract: Identifying drones and birds correctly is essential for keeping the skies safe and improving security systems. Using the VIP CUP 2025 dataset, which provides both RGB and infrared (IR) images, this study presents EGD-YOLOv8n, a new lightweight yet powerful model for object detection. The model improves how image features are captured and understood, making detection more accurate and efficient. It uses smart design changes and attention layers to focus on important details while reducing the amount of computation needed. A special detection head helps the model adapt to objects of different shapes and sizes. We trained three versions: one using RGB images, one using IR images, and one combining both. The combined model achieved the best accuracy and reliability while running fast enough for real-time use on common GPUs.

Method: Proposes a 2.5D graph-based framework representing 3D CT volumes as structured graphs with axial slice triplets as nodes processed through spectral graph convolution, enabling reasoning over inter-slice dependencies.

Result: Achieves strong cross-dataset generalization across 3 independent institution datasets, shows competitive performance compared to state-of-the-art visual encoders, and demonstrates broader applicability through transfer experiments on radiology report generation and abdominal CT data.

Conclusion: The graph-based approach provides an effective alternative for multi-label CT classification that captures complex spatial relationships while maintaining practical deployment complexity, with demonstrated generalization across datasets and tasks.

Abstract: With the growing volume of CT examinations, there is an increasing demand for automated tools such as organ segmentation, abnormality detection, and report generation to support radiologists in managing their clinical workload. Multi-label classification of 3D Chest CT scans remains a critical yet challenging problem due to the complex spatial relationships inherent in volumetric data and the wide variability of abnormalities. Existing methods based on 3D convolutional neural networks struggle to capture long-range dependencies, while Vision Transformers often require extensive pre-training on large-scale, domain-specific datasets to perform competitively. In this work, we propose a 2.5D alternative by introducing a new graph-based framework that represents 3D CT volumes as structured graphs, where axial slice triplets serve as nodes processed through spectral graph convolution, enabling the model to reason over inter-slice dependencies while maintaining complexity compatible with clinical deployment. Our method, trained and evaluated on 3 datasets from independent institutions, achieves strong cross-dataset generalization, and shows competitive performance compared to state-of-the-art visual encoders. We further conduct comprehensive ablation studies to evaluate the impact of various aggregation strategies, edge-weighting schemes, and graph connectivity patterns. Additionally, we demonstrate the broader applicability of our approach through transfer experiments on automated radiology report generation and abdominal CT data.\ This work extends our previous contribution presented at the MICCAI 2025 EMERGE Workshop.

Method: Uses K-means clustering to partition dataset into style-specific domains, separate encoders for style and content latent spaces, AdaIN for integration, and cluster-specific training to preserve domain characteristics.

Result: Achieved state-of-the-art performance with SSIM of 0.9012, PSNR of 32.5118 dB, and FID of 13.3728.

Conclusion: DISC-GAN successfully addresses underwater image synthesis challenges through style-content disentanglement and cluster-specific training, demonstrating superior performance metrics.

Abstract: In this paper, we propose a novel framework, Disentangled Style-Content GAN (DISC-GAN), which integrates style-content disentanglement with a cluster-specific training strategy towards photorealistic underwater image synthesis. The quality of synthetic underwater images is challenged by optical due to phenomena such as color attenuation and turbidity. These phenomena are represented by distinct stylistic variations across different waterbodies, such as changes in tint and haze. While generative models are well-suited to capture complex patterns, they often lack the ability to model the non-uniform conditions of diverse underwater environments. To address these challenges, we employ K-means clustering to partition a dataset into style-specific domains. We use separate encoders to get latent spaces for style and content; we further integrate these latent representations via Adaptive Instance Normalization (AdaIN) and decode the result to produce the final synthetic image. The model is trained independently on each style cluster to preserve domain-specific characteristics. Our framework demonstrates state-of-the-art performance, obtaining a Structural Similarity Index (SSIM) of 0.9012, an average Peak Signal-to-Noise Ratio (PSNR) of 32.5118 dB, and a Frechet Inception Distance (FID) of 13.3728.

Method: Proposed imHead uses deep implicit functions with a single compact identity space and introduces intermediate region-specific latent representation to enable local edits. Trained on a large-scale dataset of 4K distinct identities.

Result: The model demonstrates superior expressive power in representing diverse identities and expressions compared to previous approaches, while providing interpretable localized editing capabilities.

Conclusion: imHead represents a significant advancement in 3D head modeling by combining expressive avatar generation with efficient localized editing through a compact latent structure.

Abstract: Over the last years, 3D morphable models (3DMMs) have emerged as a state-of-the-art methodology for modeling and generating expressive 3D avatars. However, given their reliance on a strict topology, along with their linear nature, they struggle to represent complex full-head shapes. Following the advent of deep implicit functions, we propose imHead, a novel implicit 3DMM that not only models expressive 3D head avatars but also facilitates localized editing of the facial features. Previous methods directly divided the latent space into local components accompanied by an identity encoding to capture the global shape variations, leading to expensive latent sizes. In contrast, we retain a single compact identity space and introduce an intermediate region-specific latent representation to enable local edits. To train imHead, we curate a large-scale dataset of 4K distinct identities, making a step-towards large scale 3D head modeling. Under a series of experiments we demonstrate the expressive power of the proposed model to represent diverse identities and expressions outperforming previous approaches. Additionally, the proposed approach provides an interpretable solution for 3D face manipulation, allowing the user to make localized edits.

Method: Three distinct humans annotated two sequences of MDA231 human breast carcinoma cells from the Cell Tracking Challenge, creating consistent 3D time-lapse segmentation annotations that were validated against CTC tracking markers and 2D gold truth.

Result: The created annotations are consistent with CTC tracking markers, segmentation accuracy is within inter-annotator variability margins, and the 3D annotations better represent input image complexity compared to automatically created silver truth from CTC.

Conclusion: The presented 3D cell segmentation annotations are valuable for testing and training cell segmentation algorithms, as well as analyzing 3D shapes of highly dynamic objects.

Abstract: High-quality, publicly available segmentation annotations of image and video datasets are critical for advancing the field of image processing. In particular, annotations of volumetric images of a large number of targets are time-consuming and challenging. In (Melnikova, A., & Matula, P., 2025), we presented the first publicly available full 3D time-lapse segmentation annotations of migrating cells with complex dynamic shapes. Concretely, three distinct humans annotated two sequences of MDA231 human breast carcinoma cells (Fluo-C3DL-MDA231) from the Cell Tracking Challenge (CTC). This paper aims to provide a comprehensive description of the dataset and accompanying experiments that were not included in (Melnikova, A., & Matula, P., 2025) due to limitations in publication space. Namely, we show that the created annotations are consistent with the previously published tracking markers provided by the CTC organizers and the segmentation accuracy measured based on the 2D gold truth of CTC is within the inter-annotator variability margins. We compared the created 3D annotations with automatically created silver truth provided by CTC. We have found the proposed annotations better represent the complexity of the input images. The presented annotations can be used for testing and training cell segmentation, or analyzing 3D shapes of highly dynamic objects.

Method: The framework uses Swin Transformer backbone for hierarchical feature extraction, multi-scale context modules (ASPP and PSP), cross-attention for multisensor/multispectral feature fusion, and attention blocks (ECAB and Spatial Attention) for feature refinement.

Result: MSCloudCAM achieves state-of-the-art segmentation accuracy on CloudSEN12 and L8Biome datasets, surpassing leading baseline architectures while maintaining competitive parameter efficiency and computational requirements.

Conclusion: The model demonstrates effectiveness and practicality for large-scale Earth observation tasks, making it well-suited for real-world applications in satellite imagery analysis.

Abstract: Clouds remain a critical challenge in optical satellite imagery, hindering reliable analysis for environmental monitoring, land cover mapping, and climate research. To overcome this, we propose MSCloudCAM, a Cross-Attention with Multi-Scale Context Network tailored for multispectral and multi-sensor cloud segmentation. Our framework exploits the spectral richness of Sentinel-2 (CloudSEN12) and Landsat-8 (L8Biome) data to classify four semantic categories: clear sky, thin cloud, thick cloud, and cloud shadow. MSCloudCAM combines a Swin Transformer backbone for hierarchical feature extraction with multi-scale context modules ASPP and PSP for enhanced scale-aware learning. A Cross-Attention block enables effective multisensor and multispectral feature fusion, while the integration of an Efficient Channel Attention Block (ECAB) and a Spatial Attention Module adaptively refine feature representations. Comprehensive experiments on CloudSEN12 and L8Biome demonstrate that MSCloudCAM delivers state-of-the-art segmentation accuracy, surpassing leading baseline architectures while maintaining competitive parameter efficiency and FLOPs. These results underscore the model’s effectiveness and practicality, making it well-suited for large-scale Earth observation tasks and real-world applications.

Method: Extend CNN-XGBoost pipeline for multi-label classification, replace CNN final layer with XGBoost classifier, validate with DenseNet-121 and ResNet-50 backbones, compare with adversarial training, reweighting, data augmentation, and active learning.

Result: Improves fairness across demographic subgroups while maintaining or improving overall predictive performance. XGBoost with active learning yields largest bias reduction on CheXpert and MIMIC datasets.

Conclusion: Provides practical and effective path toward equitable deep learning deployment in clinical radiology with model-agnostic design and computational efficiency.

Abstract: Deep learning models have shown promise in improving diagnostic accuracy from chest X-rays, but they also risk perpetuating healthcare disparities when performance varies across demographic groups. In this work, we present a comprehensive bias detection and mitigation framework targeting sex, age, and race-based disparities when performing diagnostic tasks with chest X-rays. We extend a recent CNN-XGBoost pipeline to support multi-label classification and evaluate its performance across four medical conditions. We show that replacing the final layer of CNN with an eXtreme Gradient Boosting classifier improves the fairness of the subgroup while maintaining or improving the overall predictive performance. To validate its generalizability, we apply the method to different backbones, namely DenseNet-121 and ResNet-50, and achieve similarly strong performance and fairness outcomes, confirming its model-agnostic design. We further compare this lightweight adapter training method with traditional full-model training bias mitigation techniques, including adversarial training, reweighting, data augmentation, and active learning, and find that our approach offers competitive or superior bias reduction at a fraction of the computational cost. Finally, we show that combining eXtreme Gradient Boosting retraining with active learning yields the largest reduction in bias across all demographic subgroups, both in and out of distribution on the CheXpert and MIMIC datasets, establishing a practical and effective path toward equitable deep learning deployment in clinical radiology.

Method: Two merging strategies: Error-Constrained Layer Merging (ECLM) for selective layer merging based on MPJPE impact, and Mask-guided Token Merging (Mask-ToMe) for merging background tokens. Plus a diffusion-based decoder with temporal context and pose priors.

Result: Achieves up to 2.3x speed-up while slightly improving performance over baseline across multiple benchmarks.

Conclusion: The proposed method effectively reduces computational cost in 3D HMR while maintaining or slightly improving performance through strategic merging and diffusion-based decoding.

Abstract: Recent transformer-based models for 3D Human Mesh Recovery (HMR) have achieved strong performance but often suffer from high computational cost and complexity due to deep transformer architectures and redundant tokens. In this paper, we introduce two HMR-specific merging strategies: Error-Constrained Layer Merging (ECLM) and Mask-guided Token Merging (Mask-ToMe). ECLM selectively merges transformer layers that have minimal impact on the Mean Per Joint Position Error (MPJPE), while Mask-ToMe focuses on merging background tokens that contribute little to the final prediction. To further address the potential performance drop caused by merging, we propose a diffusion-based decoder that incorporates temporal context and leverages pose priors learned from large-scale motion capture datasets. Experiments across multiple benchmarks demonstrate that our method achieves up to 2.3x speed-up while slightly improving performance over the baseline.

Method: Created synthetic dataset RareBoost3D to supplement rare classes, and proposed CSC loss method for cross-domain semantic alignment to align feature representations of same classes across synthetic and real-world domains.

Result: Experimental results show that the cross-domain semantic alignment significantly enhances LiDAR point cloud segmentation performance on real-world data.

Conclusion: The combination of synthetic data augmentation for rare classes and cross-domain feature alignment effectively addresses the long-tail problem in LiDAR point cloud segmentation.

Abstract: Real-world point cloud datasets have made significant contributions to the development of LiDAR-based perception technologies, such as object segmentation for autonomous driving. However, due to the limited number of instances in some rare classes, the long-tail problem remains a major challenge in existing datasets. To address this issue, we introduce a novel, synthetic point cloud dataset named RareBoost3D, which complements existing real-world datasets by providing significantly more instances for object classes that are rare in real-world datasets. To effectively leverage both synthetic and real-world data, we further propose a cross-domain semantic alignment method named CSC loss that aligns feature representations of the same class across different domains. Experimental results demonstrate that this alignment significantly enhances the performance of LiDAR point cloud segmentation models over real-world data.

Method: Created EarthWhere benchmark with 810 globally distributed images across two scales: WhereCountry (500 country-level MCQs) and WhereStreet (310 street-level tasks requiring multi-step reasoning). Used final-prediction metrics (Acc@k for coordinates, hierarchical path scores) and proposed intermediate reasoning evaluation with human-verified visual clues and Shapley-reweighted thinking scores.

Result: Gemini-2.5-Pro achieved best average accuracy at 56.32%, while strongest open-weight model GLM-4.5V reached 34.71%. Web search and reasoning didn’t guarantee improved performance with limited visual clues, and models showed regional biases (up to 42.7% higher scores in certain areas).

Conclusion: The findings highlight both the promise and persistent challenges of VLMs in mitigating bias and achieving robust fine-grained localization, emphasizing the need for continued improvement in geolocation capabilities.

Abstract: Vision-language models (VLMs) have advanced rapidly, yet their capacity for image-grounded geolocation in open-world conditions, a task that is challenging and of demand in real life, has not been comprehensively evaluated. We present EarthWhere, a comprehensive benchmark for VLM image geolocation that evaluates visual recognition, step-by-step reasoning, and evidence use. EarthWhere comprises 810 globally distributed images across two complementary geolocation scales: WhereCountry (i.e., 500 multiple-choice question-answering, with country-level answer and panoramas) and WhereStreet (i.e., 310 fine-grained street-level identification tasks requiring multi-step reasoning with optional web search). For evaluation, we adopt the final-prediction metrics: location accuracies within k km (Acc@k) for coordinates and hierarchical path scores for textual localization. Beyond this, we propose to explicitly score intermediate reasoning chains using human-verified key visual clues and a Shapley-reweighted thinking score that attributes credit to each clue’s marginal contribution. We benchmark 13 state-of-the-art VLMs with web searching tools on our EarthWhere and report different types of final answer accuracies as well as the calibrated model thinking scores. Overall, Gemini-2.5-Pro achieves the best average accuracy at 56.32%, while the strongest open-weight model, GLM-4.5V, reaches 34.71%. We reveal that web search and reasoning do not guarantee improved performance when visual clues are limited, and models exhibit regional biases, achieving up to 42.7% higher scores in certain areas than others. These findings highlight not only the promise but also the persistent challenges of models to mitigate bias and achieve robust, fine-grained localization. We open-source our benchmark at https://github.com/UCSC-VLAA/EarthWhere.

Method: Proposes topological alignment framework using persistent homology to define alignment loss, with graph sparsification for efficient persistence diagram approximation with theoretical error bounds.

Result: Shows enhanced structural coherence of multilingual representations, higher zero-shot accuracy on CIFAR-100, and stronger multilingual retrieval performance on xFlickr&CO.

Conclusion: ToMCLIP provides a general method for incorporating topological alignment into representation learning, extending beyond VLMs to improve multilingual multimodal understanding.

Abstract: Contrastive Vision-Language Models (VLMs) have demonstrated strong zero-shot capabilities. However, their cross-modal alignment remains biased toward English due to limited multilingual multimodal data. Recent multilingual extensions have alleviated this gap but enforce instance-level alignment while neglecting the global geometry of the shared embedding space. We address this problem by introducing ToMCLIP (Topological Alignment for Multilingual CLIP), a topology-aware framework aligning embedding spaces with topology-preserving constraints. The proposed method applies persistent homology to define a topological alignment loss and approximates persistence diagram with theoretical error bounds using graph sparsification strategy. This work validates the proposed approach, showing enhanced structural coherence of multilingual representations, higher zero-shot accuracy on the CIFAR-100, and stronger multilingual retrieval performance on the xFlickr&CO. Beyond VLMs, the proposed approach provides a general method for incorporating topological alignment into representation learning.

Method: Uses early-stopped DDIM inversion to preserve facial structure and identity, with customization through reference images, RGB colors, and textual descriptions as conditioning inputs.

Result: Demonstrates improved customization, color-matching, identity preservation, and compatibility with text/LLMs while maintaining affordable computational costs compared to existing methods.

Conclusion: DreamMakeup provides a superior training-free diffusion-based solution for virtual makeup customization that outperforms both GAN-based and recent diffusion-based frameworks.

Abstract: The exponential growth of the global makeup market has paralleled advancements in virtual makeup simulation technology. Despite the progress led by GANs, their application still encounters significant challenges, including training instability and limited customization capabilities. Addressing these challenges, we introduce DreamMakup - a novel training-free Diffusion model based Makeup Customization method, leveraging the inherent advantages of diffusion models for superior controllability and precise real-image editing. DreamMakeup employs early-stopped DDIM inversion to preserve the facial structure and identity while enabling extensive customization through various conditioning inputs such as reference images, specific RGB colors, and textual descriptions. Our model demonstrates notable improvements over existing GAN-based and recent diffusion-based frameworks - improved customization, color-matching capabilities, identity preservation and compatibility with textual descriptions or LLMs with affordable computational costs.

Method: Leverages rich fine-grained supervision including region-text matching and long-caption modeling, with multiple discriminative objectives and a novel Textual Intra-modal Contrastive (TIC) loss to distinguish semantically similar captions. Trained on curated large-scale English and Chinese data.

Result: Achieves powerful bilingual performance, outperforming existing methods across 29 datasets in 8 tasks and establishing state-of-the-art results in both English and Chinese.

Conclusion: FG-CLIP 2 successfully addresses fine-grained vision-language alignment challenges in bilingual settings, with released model, code, and benchmark to support future research.

Abstract: Fine-grained vision-language understanding requires precise alignment between visual content and linguistic descriptions, a capability that remains limited in current models, particularly in non-English settings. While models like CLIP perform well on global alignment, they often struggle to capture fine-grained details in object attributes, spatial relations, and linguistic expressions, with limited support for bilingual comprehension. To address these challenges, we introduce FG-CLIP 2, a bilingual vision-language model designed to advance fine-grained alignment for both English and Chinese. Our approach leverages rich fine-grained supervision, including region-text matching and long-caption modeling, alongside multiple discriminative objectives. We further introduce the Textual Intra-modal Contrastive (TIC) loss to better distinguish semantically similar captions. Trained on a carefully curated mixture of large-scale English and Chinese data, FG-CLIP 2 achieves powerful bilingual performance. To enable rigorous evaluation, we present a new benchmark for Chinese multimodal understanding, featuring long-caption retrieval and bounding box classification. Extensive experiments on 29 datasets across 8 tasks show that FG-CLIP 2 outperforms existing methods, achieving state-of-the-art results in both languages. We release the model, code, and benchmark to facilitate future research on bilingual fine-grained alignment.

Method: Three-stage progressive pose optimization with dense keypoint-level fusion using only RGB images, enhanced with attentional aggregation and symmetry-aware training to improve accuracy and resolve symmetric object ambiguities.

Result: Extensive experiments on ROBI dataset show DKPMV outperforms state-of-the-art multi-view RGB approaches and surpasses RGB-D methods in most cases.

Conclusion: The proposed RGB-only pipeline achieves superior performance through dense keypoint fusion and geometric optimization, demonstrating effectiveness for textureless object pose estimation.

Abstract: 6D pose estimation of textureless objects is valuable for industrial robotic applications, yet remains challenging due to the frequent loss of depth information. Current multi-view methods either rely on depth data or insufficiently exploit multi-view geometric cues, limiting their performance. In this paper, we propose DKPMV, a pipeline that achieves dense keypoint-level fusion using only multi-view RGB images as input. We design a three-stage progressive pose optimization strategy that leverages dense multi-view keypoint geometry information. To enable effective dense keypoint fusion, we enhance the keypoint network with attentional aggregation and symmetry-aware training, improving prediction accuracy and resolving ambiguities on symmetric objects. Extensive experiments on the ROBI dataset demonstrate that DKPMV outperforms state-of-the-art multi-view RGB approaches and even surpasses the RGB-D methods in the majority of cases. The code will be available soon.

Method: Uses reconstruction stability under random measurement variations as proxy for distribution shift detection. OOD images show higher variability in reconstructions compared to in-distribution images.

Result: Validated on MNIST digit reconstruction, where a model trained only on digit “0” showed higher reconstruction variability and error for OOD digits (1-9), confirming the indicator’s effectiveness.

Conclusion: Proposes pairing generative priors with lightweight guardrails to enable aggressive measurement reduction for in-distribution cases while automatically warning when applied to OOD data.

Abstract: Generative models have shown strong potential as data-driven priors for solving inverse problems such as reconstructing medical images from undersampled measurements. While these priors improve reconstruction quality with fewer measurements, they risk hallucinating features when test images lie outside the training distribution. Existing uncertainty quantification methods in this setting (i) require an in-distribution calibration dataset, which may not be available, (ii) provide heuristic rather than statistical estimates, or (iii) quantify uncertainty from model capacity or limited measurements rather than distribution shift. We propose an instance-level, calibration-free uncertainty indicator that is sensitive to distribution shift, requires no knowledge of the training distribution, and incurs no retraining cost. Our key hypothesis is that reconstructions of in-distribution images remain stable under random measurement variations, while reconstructions of out-of-distribution (OOD) images exhibit greater instability. We use this stability as a proxy for detecting distribution shift. Our proposed OOD indicator is efficiently computable for any computational imaging inverse problem; we demonstrate it on tomographic reconstruction of MNIST digits, where a learned proximal network trained only on digit “0” is evaluated on all ten digits. Reconstructions of OOD digits show higher variability and correspondingly higher reconstruction error, validating this indicator. These results suggest a deployment strategy that pairs generative priors with lightweight guardrails, enabling aggressive measurement reduction for in-distribution cases while automatically warning when priors are applied out of distribution.

Method: Two-stage framework: (1) dynamic IUT-Plug extraction module parses visual scenes into hierarchical symbolic structures, (2) coordinated narrative-flow and image synthesis mechanism ensures cross-modal consistency.

Result: IUT-Plug improves accuracy on established benchmarks and effectively alleviates the three critical forms of context drift across diverse multimodal QA scenarios.

Conclusion: The proposed IUT-Plug framework successfully mitigates context drift in VLMs through explicit structured reasoning, enhancing their multimodal generation capabilities.

Abstract: Existing vision language models (VLMs), including GPT-4 and DALL-E, often struggle to preserve logic, object identity, and style in multimodal image-text generation. This limitation significantly hinders the generalization capability of VLMs in complex image-text input-output scenarios. To address this issue, we propose IUT-Plug, a module grounded in an Image Understanding Tree (IUT), which enhances existing interleaved VLMs through explicit structured reasoning, thereby mitigating context drift in logic, entity identity, and style. The proposed framework operates in two stages. (1) A dynamic IUT-Plug extraction module parses visual scenes into hierarchical symbolic structures. (2) A coordinated narrative-flow and image synthesis mechanism ensures cross-modal consistency. To evaluate our approach, we construct a novel benchmark based on 3,000 real human-generated question-answer pairs over fine-tuned large models, introducing a dynamic evaluation protocol for quantifying context drift in interleaved VLMs. Experimental results demonstrate that IUT-Plug not only improves accuracy on established benchmarks but also effectively alleviates the three critical forms of context drift across diverse multimodal question answering (QA) scenarios.

Method: Uses Group Relative Policy Optimization with three verifiable rewards: chart-type classification accuracy, faithful chart table reconstruction, and process conformity to fine-tune 3B-parameter LVLMs.

Result: Chart-RVR consistently outperforms standard supervised fine-tuning on both in-distribution and out-of-distribution datasets, achieving state-of-the-art results on six chart-reasoning benchmarks and producing more interpretable rationales.

Conclusion: The framework demonstrates that verifiable rewards with GRPO can train reliable, interpretable chart-reasoning models that close the OOD performance gap while improving rationale fidelity.

Abstract: The capabilities of Large Vision-Language Models (LVLMs) have reached state-of-the-art on many visual reasoning tasks, including chart reasoning, yet they still falter on out-of-distribution (OOD) data, and degrade further when asked to produce their chain-of-thought (CoT) rationales, limiting explainability. We present Chart-RVR, a general framework that fine-tunes LVLMs to be more robust and explainable for chart reasoning by coupling Group Relative Policy Optimization (GRPO) with automatically verifiable rewards. Our framework comprises of three rewards that maximize: (i) correct chart-type classification, (ii) faithful chart table reconstruction, and (iii) process conformity. Applied to 3-billion-parameter LVLMs, Chart-RVR consistently outperforms standard supervised fine-tuning (SFT) on both in-distribution and out-of-distribution datasets, closing the OOD performance gap while improving rationale fidelity. The resulting models, the Chart-RVR-3B series, achieve state-of-the-art results on six chart-reasoning benchmarks spanning in-domain and OOD settings, surpassing all existing models of comparable size. Beyond accuracy, Chart-RVR yields more interpretable CoT rationales, strengthening trust and reliability - showcasing the power of verifiable rewards with GRPO for training reliable, interpretable chart-reasoning models.

Method: First introduces Multimodal Mixup (MM) applying Mixup at aggregated multimodal feature level to generate virtual feature-label pairs. Then proposes Balanced Multimodal Mixup (B-MM) that dynamically adjusts mixing ratios for each modality based on their relative contributions to learning.

Result: Extensive experiments on multiple datasets show both methods effectively improve generalization and multimodal robustness, with B-MM providing additional benefits by addressing modality imbalance.

Conclusion: The proposed MM and B-MM methods successfully mitigate overfitting of strong modalities and improve multimodal learning performance, with B-MM offering enhanced capability to handle modality imbalance during training.

Abstract: Multimodal video understanding plays a crucial role in tasks such as action recognition and emotion classification by combining information from different modalities. However, multimodal models are prone to overfitting strong modalities, which can dominate learning and suppress the contributions of weaker ones. To address this challenge, we first propose Multimodal Mixup (MM), which applies the Mixup strategy at the aggregated multimodal feature level to mitigate overfitting by generating virtual feature-label pairs. While MM effectively improves generalization, it treats all modalities uniformly and does not account for modality imbalance during training. Building on MM, we further introduce Balanced Multimodal Mixup (B-MM), which dynamically adjusts the mixing ratios for each modality based on their relative contributions to the learning objective. Extensive experiments on several datasets demonstrate the effectiveness of our methods in improving generalization and multimodal robustness.

Method: The authors establish a conceptual framework organizing agentic MLLMs along three dimensions: agentic internal intelligence (reasoning, reflection, memory), agentic external tool invocation (proactive tool usage), and agentic environment interaction (action-taking in dynamic scenarios).

Result: The survey provides a systematic organization of agentic MLLM concepts, compiles open-source training frameworks and datasets, reviews downstream applications, and outlines future research directions with an actively maintained public repository.

Conclusion: Agentic MLLMs represent an emerging paradigm that combines multimodal capabilities with proactive, goal-directed behavior, creating a foundation for more advanced AI systems that can operate effectively in dynamic real-world environments.

Abstract: With the recent emergence of revolutionary autonomous agentic systems, research community is witnessing a significant shift from traditional static, passive, and domain-specific AI agents toward more dynamic, proactive, and generalizable agentic AI. Motivated by the growing interest in agentic AI and its potential trajectory toward AGI, we present a comprehensive survey on Agentic Multimodal Large Language Models (Agentic MLLMs). In this survey, we explore the emerging paradigm of agentic MLLMs, delineating their conceptual foundations and distinguishing characteristics from conventional MLLM-based agents. We establish a conceptual framework that organizes agentic MLLMs along three fundamental dimensions: (i) Agentic internal intelligence functions as the system’s commander, enabling accurate long-horizon planning through reasoning, reflection, and memory; (ii) Agentic external tool invocation, whereby models proactively use various external tools to extend their problem-solving capabilities beyond their intrinsic knowledge; and (iii) Agentic environment interaction further situates models within virtual or physical environments, allowing them to take actions, adapt strategies, and sustain goal-directed behavior in dynamic real-world scenarios. To further accelerate research in this area for the community, we compile open-source training frameworks, training and evaluation datasets for developing agentic MLLMs. Finally, we review the downstream applications of agentic MLLMs and outline future research directions for this rapidly evolving field. To continuously track developments in this rapidly evolving field, we will also actively update a public repository at https://github.com/HJYao00/Awesome-Agentic-MLLMs.

Method: Iteratively refines inpainting and optimizes 3D Gaussian representation using multiple views adaptively sampled from a perspective graph, propagating inpainted images as prior information and verifying consistency across neighboring views.

Result: Achieves superior 3D inpainting quality with PSNR scores of 26.03 dB on SPIn-NeRF and 29.51 dB on NeRFiller datasets, outperforming existing methods.

Conclusion: PAInpainter demonstrates effectiveness and generalization capability in advancing 3D Gaussian inpainting through perspective-aware content propagation and multi-view consistency verification.

Abstract: 3D Gaussian inpainting, a critical technique for numerous applications in virtual reality and multimedia, has made significant progress with pretrained diffusion models. However, ensuring multi-view consistency, an essential requirement for high-quality inpainting, remains a key challenge. In this work, we present PAInpainter, a novel approach designed to advance 3D Gaussian inpainting by leveraging perspective-aware content propagation and consistency verification across multi-view inpainted images. Our method iteratively refines inpainting and optimizes the 3D Gaussian representation with multiple views adaptively sampled from a perspective graph. By propagating inpainted images as prior information and verifying consistency across neighboring views, PAInpainter substantially enhances global consistency and texture fidelity in restored 3D scenes. Extensive experiments demonstrate the superiority of PAInpainter over existing methods. Our approach achieves superior 3D inpainting quality, with PSNR scores of 26.03 dB and 29.51 dB on the SPIn-NeRF and NeRFiller datasets, respectively, highlighting its effectiveness and generalization capability.

Method: Proposes ContextGen with two key mechanisms: Contextual Layout Anchoring (CLA) to incorporate layout images for object positioning, and Identity Consistency Attention (ICA) to maintain identity using reference images. Also introduces IMIG-100K dataset with detailed annotations.

Result: ContextGen sets new state-of-the-art performance, outperforming existing methods in control precision, identity fidelity, and overall visual quality.

Conclusion: The proposed framework effectively addresses multi-instance generation challenges through layout anchoring and identity consistency mechanisms, demonstrating superior performance over existing approaches.

Abstract: Multi-instance image generation (MIG) remains a significant challenge for modern diffusion models due to key limitations in achieving precise control over object layout and preserving the identity of multiple distinct subjects. To address these limitations, we introduce ContextGen, a novel Diffusion Transformer framework for multi-instance generation that is guided by both layout and reference images. Our approach integrates two key technical contributions: a Contextual Layout Anchoring (CLA) mechanism that incorporates the composite layout image into the generation context to robustly anchor the objects in their desired positions, and Identity Consistency Attention (ICA), an innovative attention mechanism that leverages contextual reference images to ensure the identity consistency of multiple instances. Recognizing the lack of large-scale, hierarchically-structured datasets for this task, we introduce IMIG-100K, the first dataset with detailed layout and identity annotations. Extensive experiments demonstrate that ContextGen sets a new state-of-the-art, outperforming existing methods in control precision, identity fidelity, and overall visual quality.

Method: Three main components: 1) Foreground-aware module to amplify background-target distinction, 2) Wavelet-based frequency enhancement for boundary recognition, 3) Edge constraint module for geometric continuity preservation.

Result: Extensive experiments show superior performance across all metrics, with particular strength in robustness under complex conditions and fine structure recognition.

Conclusion: FASS framework significantly enhances low-contrast image segmentation, enabling applications in diverse and complex medical imaging scenarios.

Abstract: Accurate segmentation of tumors and adjacent normal tissues in medical images is essential for surgical planning and tumor staging. Although foundation models generally perform well in segmentation tasks, they often struggle to focus on foreground areas in complex, low-contrast backgrounds, where some malignant tumors closely resemble normal organs, complicating contextual differentiation. To address these challenges, we propose the Foreground-Aware Spectrum Segmentation (FASS) framework. First, we introduce a foreground-aware module to amplify the distinction between background and the entire volume space, allowing the model to concentrate more effectively on target areas. Next, a feature-level frequency enhancement module, based on wavelet transform, extracts discriminative high-frequency features to enhance boundary recognition and detail perception. Eventually, we introduce an edge constraint module to preserve geometric continuity in segmentation boundaries. Extensive experiments on multiple medical datasets demonstrate superior performance across all metrics, validating the effectiveness of our framework, particularly in robustness under complex conditions and fine structure recognition. Our framework significantly enhances segmentation of low-contrast images, paving the way for applications in more diverse and complex medical imaging scenarios.

Method: Augments VLM outputs with neurosymbolic concept trees learned from LLMs using a beam search-inspired reasoning process that provides interpretable rationales.

Result: Significantly improves compositional generalization by 5-10% across four benchmarks (Winoground, EqBench, ColorSwap, SugarCrepe) in seven different open-source VLMs.

Conclusion: COCO-Tree effectively enhances VLMs’ compositional reasoning capabilities while providing interpretable rationales, addressing key limitations in current vision-language models.

Abstract: Compositional reasoning remains a persistent weakness of modern vision language models (VLMs): they often falter when a task hinges on understanding how multiple objects, attributes, and relations interact within an image. Multiple research works have attempted to improve compositionality performance by creative tricks such as improving prompt structure, chain of thought reasoning, etc. A more recent line of work attempts to impart additional reasoning in VLMs using well-trained Large Language Models (LLMs), which are far superior in linguistic understanding than VLMs to compensate for the limited linguistic prowess of VLMs. However, these approaches are either resource-intensive or do not provide an interpretable reasoning process. In this paper, we present ‘COCO-Tree’ - a novel approach that augments VLM outputs with carefully designed neurosymbolic concept trees learned from LLMs to improve VLM’s linguistic reasoning. COCO-Tree’s beam search-inspired reasoning process boosts compositionality performance and provides a rationale behind VLM predictions. Empirical results on four compositionality benchmarks, Winoground, EqBench, ColorSwap, and SugarCrepe, in seven different open-source VLMs with varying sizes, demonstrate that COCO-Tree significantly improves compositional generalization by 5-10% over baselines.

Method: Proposes Global Spatiotemporal Mamba with 6D selective space-time scan and modulated scan merging for global representations, and Local Refinement Mamba with windowed space-time scan for local keypoint motion details.

Result: Extensive experiments on four benchmark datasets show the proposed model outperforms state-of-the-art VHPE approaches while achieving better computational trade-offs.

Conclusion: The framework successfully extends Mamba to handle high-resolution spatiotemporal representations for VHPE, effectively balancing global and local modeling with linear complexity.

Abstract: Modeling high-resolution spatiotemporal representations, including both global dynamic contexts (e.g., holistic human motion tendencies) and local motion details (e.g., high-frequency changes of keypoints), is essential for video-based human pose estimation (VHPE). Current state-of-the-art methods typically unify spatiotemporal learning within a single type of modeling structure (convolution or attention-based blocks), which inherently have difficulties in balancing global and local dynamic modeling and may bias the network to one of them, leading to suboptimal performance. Moreover, existing VHPE models suffer from quadratic complexity when capturing global dependencies, limiting their applicability especially for high-resolution sequences. Recently, the state space models (known as Mamba) have demonstrated significant potential in modeling long-range contexts with linear complexity; however, they are restricted to 1D sequential data. In this paper, we present a novel framework that extends Mamba from two aspects to separately learn global and local high-resolution spatiotemporal representations for VHPE. Specifically, we first propose a Global Spatiotemporal Mamba, which performs 6D selective space-time scan and spatial- and temporal-modulated scan merging to efficiently extract global representations from high-resolution sequences. We further introduce a windowed space-time scan-based Local Refinement Mamba to enhance the high-frequency details of localized keypoint motions. Extensive experiments on four benchmark datasets demonstrate that the proposed model outperforms state-of-the-art VHPE approaches while achieving better computational trade-offs.

Method: Propose a reinforcement learning framework with cross-modal reward that evaluates alignment between generated auxiliary-line descriptions and ground-truth diagrams. Use GRPO-based RL for precise diagram-text alignment. Create GeoVLMath model and AuxSolidMath dataset with 3,018 geometry problems.

Result: GeoVLMath at 3B and 7B scales achieves competitive and often superior performance compared to strong open-source and proprietary LVLMs on auxiliary-line reasoning benchmarks.

Conclusion: The reinforcement learning approach with cross-modal rewards effectively enhances LVLMs’ auxiliary-line reasoning capabilities in geometry problems, demonstrating the value of focusing on textual descriptions rather than diagram editing.

Abstract: Auxiliary lines are essential for solving complex geometric problems but remain challenging for large vision-language models (LVLMs). Rather than editing diagrams to draw auxiliary lines, which current image editing models struggle to render with geometric precision, we generate textual descriptions of auxiliary-line constructions to better align with the representational strengths of LVLMs. To bridge the gap between textual descriptions and spatial structure, we propose a reinforcement learning framework that enhances diagram-text alignment. At the core of our approach is a cross-modal reward that evaluates how well the generated auxiliary-line description for an original diagram matches a ground-truth auxiliary-line diagram. Built on this reward, we present GeoVLMath, an open-source LVLM tailored to auxiliary-line reasoning in solid geometry. This fine-grained signal drives a GRPO-based RL stage, yielding precise diagram-text alignment. To support training, we develop a scalable data creation pipeline and construct AuxSolidMath, a dataset of 3,018 real-exam geometry problems with paired diagrams and aligned textual fields. At the 3B and 7B scales, GeoVLMath achieves competitive and often superior performance compared with strong open-source and proprietary LVLMs on auxiliary-line reasoning benchmarks.

Method: GIR-Bench evaluates models across three complementary perspectives: 1) Understanding-generation consistency (GIR-Bench-UGC), 2) Reasoning-centric text-to-image generation (GIR-Bench-T2I), and 3) Multi-step reasoning in editing (GIR-Bench-Edit). Each subset has carefully designed task-specific evaluation pipelines tailored for each task.

Result: Extensive ablations show that unified models are more capable of reasoning-driven visual tasks but still exhibit a persistent gap between understanding and generation.

Conclusion: GIR-Bench provides a comprehensive framework for evaluating unified multimodal models, revealing that while these models show promise for advanced multimodal intelligence, there remains a significant gap between their understanding and generation capabilities that needs to be addressed.

Abstract: Unified multimodal models integrate the reasoning capacity of large language models with both image understanding and generation, showing great promise for advanced multimodal intelligence. However, the community still lacks a rigorous reasoning-centric benchmark to systematically evaluate the alignment between understanding and generation, and their generalization potential in complex visual tasks. To this end, we introduce \textbf{GIR-Bench}, a comprehensive benchmark that evaluates unified models across three complementary perspectives. Firstly, we investigate understanding-generation consistency (GIR-Bench-UGC), asking whether models can consistently leverage the same knowledge in both understanding and generation tasks. Secondly, we investigate whether models can perform reasoning-centric text-to-image generation that requires applying logical constraints and implicit knowledge to generate faithful visual content (GIR-Bench-T2I). Thirdly, we evaluate whether models can handle multi-step reasoning in editing (GIR-Bench-Edit). For each subset, we carefully design different task-specific evaluation pipelines tailored for each task. This enables fine-grained and interpretable evaluation while mitigating biases from the prevalent MLLM-as-a-Judge paradigm. Extensive ablations over various unified models and generation-only systems have shown that: Although unified models are more capable of reasoning-driven visual tasks, they still exhibit a persistent gap between understanding and generation. The data and code for GIR-Bench are available at \href{https://hkust-longgroup.github.io/GIR-Bench}{https://hkust-longgroup.github.io/GIR-Bench}.

Method: Developed Vlaser - a foundational vision-language model integrating high-level reasoning with low-level control, built on the Vlaser-6M dataset. Systematically examined how different VLM initializations affect supervised VLA fine-tuning.

Result: Achieved state-of-the-art performance on embodied reasoning benchmarks (spatial reasoning, embodied grounding, embodied QA, task planning) and SOTA results on WidowX benchmark with competitive performance on Google Robot benchmark.

Conclusion: Successfully bridged embodied reasoning with VLA policy learning, providing insights into mitigating domain shift between pre-training data and embodied policy learning data.

Abstract: While significant research has focused on developing embodied reasoning capabilities using Vision-Language Models (VLMs) or integrating advanced VLMs into Vision-Language-Action (VLA) models for end-to-end robot control, few studies directly address the critical gap between upstream VLM-based reasoning and downstream VLA policy learning. In this work, we take an initial step toward bridging embodied reasoning with VLA policy learning by introducing Vlaser - a Vision-Language-Action Model with synergistic embodied reasoning capability, which is a foundational vision-language model designed to integrate high-level reasoning with low-level control for embodied agents. Built upon the high-quality Vlaser-6M dataset, Vlaser achieves state-of-the-art performance across a range of embodied reasoning benchmarks - including spatial reasoning, embodied grounding, embodied QA, and task planning. Furthermore, we systematically examine how different VLM initializations affect supervised VLA fine-tuning, offering novel insights into mitigating the domain shift between internet-scale pre-training data and embodied-specific policy learning data. Based on these insights, our approach achieves state-of-the-art results on the WidowX benchmark and competitive performance on the Google Robot benchmark.

Method: Two-stage framework: (1) Co-Feature Point Prompt Generation (PPG) module uses CLIP and SAM to generate positive/negative point prompts, guiding SAM to focus on anomalies rather than entire objects; (2) Cascaded Prompts for SAM (CPS) module employs hybrid prompts with SAM’s lightweight decoder to refine boundaries and reduce noise.

Result: Achieves state-of-the-art zero-shot anomaly segmentation across multiple datasets. On Visa dataset, outperforms SOTA by 10.3% in F1-max and 7.7% in AP metrics.

Conclusion: The proposed framework effectively combines CLIP and SAM capabilities for precise anomaly segmentation, demonstrating superior performance in zero-shot industrial anomaly detection tasks.

Abstract: Recently, the powerful generalization ability exhibited by foundation models has brought forth new solutions for zero-shot anomaly segmentation tasks. However, guiding these foundation models correctly to address downstream tasks remains a challenge. This paper proposes a novel two-stage framework, for zero-shot anomaly segmentation tasks in industrial anomaly detection. This framework excellently leverages the powerful anomaly localization capability of CLIP and the boundary perception ability of SAM.(1) To mitigate SAM’s inclination towards object segmentation, we propose the Co-Feature Point Prompt Generation (PPG) module. This module collaboratively utilizes CLIP and SAM to generate positive and negative point prompts, guiding SAM to focus on segmenting anomalous regions rather than the entire object. (2) To further optimize SAM’s segmentation results and mitigate rough boundaries and isolated noise, we introduce the Cascaded Prompts for SAM (CPS) module. This module employs hybrid prompts cascaded with a lightweight decoder of SAM, achieving precise segmentation of anomalous regions. Across multiple datasets, consistent experimental validation demonstrates that our approach achieves state-of-the-art zero-shot anomaly segmentation results. Particularly noteworthy is our performance on the Visa dataset, where we outperform the state-of-the-art methods by 10.3% and 7.7% in terms of {$F_1$-max} and AP metrics, respectively.

Method: Created LaryngealCT benchmark with 1,029 CT scans from TCIA, extracted uniform 1mm isotropic volumes using weakly supervised parameter search, and benchmarked 3D DL architectures (3D CNN, ResNet18,50,101, DenseNet121) on early vs. advanced and T4 vs. non-T4 classification tasks.

Result: 3D CNN achieved AUC-0.881 and F1-macro-0.821 for early vs. advanced classification; ResNet18 achieved AUC-0.892 and F1-macro-0.646 for T4 vs. non-T4 classification. 3D GradCAM analysis showed different attention patterns between T4 and non-T4 cases.

Conclusion: LaryngealCT provides open-source data, pretrained models, and explainability tools to establish a reproducible foundation for AI-driven laryngeal cancer research and clinical decision support.

Abstract: Laryngeal cancer imaging research lacks standardised datasets to enable reproducible deep learning (DL) model development. We present LaryngealCT, a curated benchmark of 1,029 computed tomography (CT) scans aggregated from six collections from The Cancer Imaging Archive (TCIA). Uniform 1 mm isotropic volumes of interest encompassing the larynx were extracted using a weakly supervised parameter search framework validated by clinical experts. 3D DL architectures (3D CNN, ResNet18,50,101, DenseNet121) were benchmarked on (i) early (Tis,T1,T2) vs. advanced (T3,T4) and (ii) T4 vs. non-T4 classification tasks. 3D CNN (AUC-0.881, F1-macro-0.821) and ResNet18 (AUC-0.892, F1-macro-0.646) respectively outperformed the other models in the two tasks. Model explainability assessed using 3D GradCAMs with thyroid cartilage overlays revealed greater peri-cartilage attention in non-T4 cases and focal activations in T4 predictions. Through open-source data, pretrained models, and integrated explainability tools, LaryngealCT offers a reproducible foundation for AI-driven research to support clinical decisions in laryngeal oncology.

Method: Decompose face representation into ID and attribute features, using them as joint conditions to guide both inversion and reverse diffusion processes for independent control over ID and attributes.

Result: The method achieves strong ID preservation and structural consistency while enabling precise facial attribute manipulation, supporting complex multi-attribute editing tasks with only text prompts.

Conclusion: The proposed zero-shot face editing method is practical and effective, operating at DDIM inversion speed without requiring region-specific input.

Abstract: Recent advancements in text-guided diffusion models have shown promise for general image editing via inversion techniques, but often struggle to maintain ID and structural consistency in real face editing tasks. To address this limitation, we propose a zero-shot face editing method based on ID-Attribute Decoupled Inversion. Specifically, we decompose the face representation into ID and attribute features, using them as joint conditions to guide both the inversion and the reverse diffusion processes. This allows independent control over ID and attributes, ensuring strong ID preservation and structural consistency while enabling precise facial attribute manipulation. Our method supports a wide range of complex multi-attribute face editing tasks using only text prompts, without requiring region-specific input, and operates at a speed comparable to DDIM inversion. Comprehensive experiments demonstrate its practicality and effectiveness.

Method: The challenge features three tracks: Classic VOS, Referring VOS, and the new Complex VOS (MOSEv2) with increased difficulty. Standard J, F, and J&F metrics are used for VOS/RVOS, while MOSEv2 adopts J&Ḟ to better evaluate objects across scales and disappearance cases.

Result: The challenge highlights top-performing solutions and emerging trends including the growing role of LLM/MLLM components and memory-aware propagation in video segmentation systems.

Conclusion: The LSVOS Challenge aims to chart future directions for resilient, language-aware video segmentation in the wild by introducing more realistic evaluation scenarios and tracking emerging technological trends.

Abstract: This report presents an overview of the 7th Large-scale Video Object Segmentation (LSVOS) Challenge held in conjunction with ICCV 2025. Besides the two traditional tracks of LSVOS that jointly target robustness in realistic video scenarios: Classic VOS (VOS), and Referring VOS (RVOS), the 2025 edition features a newly introduced track, Complex VOS (MOSEv2). Building upon prior insights, MOSEv2 substantially increases difficulty, introducing more challenging but realistic scenarios including denser small objects, frequent disappear/reappear events, severe occlusions, adverse weather and lighting, etc., pushing long-term consistency and generalization beyond curated benchmarks. The challenge retains standard ${J}$, $F$, and ${J&F}$ metrics for VOS and RVOS, while MOSEv2 adopts ${J&\dot{F}}$ as the primary ranking metric to better evaluate objects across scales and disappearance cases. We summarize datasets and protocols, highlight top-performing solutions, and distill emerging trends, such as the growing role of LLM/MLLM components and memory-aware propagation, aiming to chart future directions for resilient, language-aware video segmentation in the wild.

Method: Uses weakly supervised learning and neural identity translation to anonymize facial features while retaining disease features, working with AI systems and supporting secure image reversal.

Result: Achieves 100% diagnostic sensitivity, over 98% accuracy (κ > 0.90) across 11 eye diseases in 3 cohorts, anonymizes over 95% of images, maintains original diagnoses (κ > 0.80), and supports secure reversal with over 98% similarity.

Conclusion: ROFI effectively protects patient privacy in digital medicine while preserving diagnostic capabilities, enabling secure medical image sharing and long-term care.

Abstract: Patient face images provide a convenient mean for evaluating eye diseases, while also raising privacy concerns. Here, we introduce ROFI, a deep learning-based privacy protection framework for ophthalmology. Using weakly supervised learning and neural identity translation, ROFI anonymizes facial features while retaining disease features (over 98% accuracy, $\kappa > 0.90$). It achieves 100% diagnostic sensitivity and high agreement ($\kappa > 0.90$) across eleven eye diseases in three cohorts, anonymizing over 95% of images. ROFI works with AI systems, maintaining original diagnoses ($\kappa > 0.80$), and supports secure image reversal (over 98% similarity), enabling audits and long-term care. These results show ROFI’s effectiveness of protecting patient privacy in the digital medicine era.

Method: FRANCK comprises four components: (1) Objectness Score-based Sample Reweighting module for attention-based scoring and loss reweighting, (2) Contrastive Learning with Matching-based Memory Bank for multi-level feature integration, (3) Uncertainty-weighted Query-fused Feature Distillation for improved feature distillation, and (4) improved self-training pipeline with Dynamic Teacher Updating Interval.

Result: Extensive experiments on several widely used benchmarks demonstrate that FRANCK achieves state-of-the-art performance, highlighting its effectiveness and compatibility with DETR-based SFOD models.

Conclusion: FRANCK effectively adapts source-pre-trained DETR models to target domains with enhanced robustness and generalization, providing a specialized solution for DETR-based source-free object detection.

Abstract: Source-Free Object Detection (SFOD) enables knowledge transfer from a source domain to an unsupervised target domain for object detection without access to source data. Most existing SFOD approaches are either confined to conventional object detection (OD) models like Faster R-CNN or designed as general solutions without tailored adaptations for novel OD architectures, especially Detection Transformer (DETR). In this paper, we introduce Feature Reweighting ANd Contrastive Learning NetworK (FRANCK), a novel SFOD framework specifically designed to perform query-centric feature enhancement for DETRs. FRANCK comprises four key components: (1) an Objectness Score-based Sample Reweighting (OSSR) module that computes attention-based objectness scores on multi-scale encoder feature maps, reweighting the detection loss to emphasize less-recognized regions; (2) a Contrastive Learning with Matching-based Memory Bank (CMMB) module that integrates multi-level features into memory banks, enhancing class-wise contrastive learning; (3) an Uncertainty-weighted Query-fused Feature Distillation (UQFD) module that improves feature distillation through prediction quality reweighting and query feature fusion; and (4) an improved self-training pipeline with a Dynamic Teacher Updating Interval (DTUI) that optimizes pseudo-label quality. By leveraging these components, FRANCK effectively adapts a source-pre-trained DETR model to a target domain with enhanced robustness and generalization. Extensive experiments on several widely used benchmarks demonstrate that our method achieves state-of-the-art performance, highlighting its effectiveness and compatibility with DETR-based SFOD models.

Method: Joint modeling of geometric and textual primitives using unified representations, with Transformer backbone enhanced by type-aware attention mechanism to model spatial dependencies.

Result: Outperforms existing approaches on symbol spotting tasks involving textual annotations and shows superior robustness on complex CAD drawings.

Conclusion: Incorporating textual annotations and explicit relationship modeling significantly improves panoptic symbol spotting performance in CAD drawings.

Abstract: With the widespread adoption of Computer-Aided Design(CAD) drawings in engineering, architecture, and industrial design, the ability to accurately interpret and analyze these drawings has become increasingly critical. Among various subtasks, panoptic symbol spotting plays a vital role in enabling downstream applications such as CAD automation and design retrieval. Existing methods primarily focus on geometric primitives within the CAD drawings to address this task, but they face following major problems: they usually overlook the rich textual annotations present in CAD drawings and they lack explicit modeling of relationships among primitives, resulting in incomprehensive understanding of the holistic drawings. To fill this gap, we propose a panoptic symbol spotting framework that incorporates textual annotations. The framework constructs unified representations by jointly modeling geometric and textual primitives. Then, using visual features extract by pretrained CNN as the initial representations, a Transformer-based backbone is employed, enhanced with a type-aware attention mechanism to explicitly model the different types of spatial dependencies between various primitives. Extensive experiments on the real-world dataset demonstrate that the proposed method outperforms existing approaches on symbol spotting tasks involving textual annotations, and exhibits superior robustness when applied to complex CAD drawings.

Method: Predicts future BEV representations to anticipate scene dynamics, then uses this foresight for trajectory planning. Features future-aware planning and iterative scene modeling with collaborative optimization.

Result: Significantly outperforms existing state-of-the-art methods on NAVSIM and nuScenes benchmarks.

Conclusion: Joint modeling of future scene evolution and trajectory planning in a closed-loop manner enables more informed and adaptive autonomous driving decisions.

Abstract: End-to-end autonomous driving methods aim to directly map raw sensor inputs to future driving actions such as planned trajectories, bypassing traditional modular pipelines. While these approaches have shown promise, they often operate under a one-shot paradigm that relies heavily on the current scene context, potentially underestimating the importance of scene dynamics and their temporal evolution. This limitation restricts the model’s ability to make informed and adaptive decisions in complex driving scenarios. We propose a new perspective: the future trajectory of an autonomous vehicle is closely intertwined with the evolving dynamics of its environment, and conversely, the vehicle’s own future states can influence how the surrounding scene unfolds. Motivated by this bidirectional relationship, we introduce SeerDrive, a novel end-to-end framework that jointly models future scene evolution and trajectory planning in a closed-loop manner. Our method first predicts future bird’s-eye view (BEV) representations to anticipate the dynamics of the surrounding scene, then leverages this foresight to generate future-context-aware trajectories. Two key components enable this: (1) future-aware planning, which injects predicted BEV features into the trajectory planner, and (2) iterative scene modeling and vehicle planning, which refines both future scene prediction and trajectory generation through collaborative optimization. Extensive experiments on the NAVSIM and nuScenes benchmarks show that SeerDrive significantly outperforms existing state-of-the-art methods.

Method: A supervised diffusion-based denoising framework that fine-tunes diffusion models using paired adversarial-clean image datasets with directional, task-specific guidance, complemented by prompt optimization as an additional defense mechanism.

Result: Extensive experiments on image captioning and VQA show the method substantially improves robustness and exhibits strong transferability to unknown adversarial attacks while maintaining high-quality image reconstructions.

Conclusion: The supervised diffusion-based denoising approach effectively defends multimodal systems against adversarial threats, enabling more reliable and secure deployment of MLLMs in real-world applications.

Abstract: Multimodal Large Language Models (MLLMs) have achieved remarkable success in tasks such as image captioning, visual question answering, and cross-modal reasoning by integrating visual and textual modalities. However, their multimodal nature also exposes them to adversarial threats, where attackers can perturb either modality or both jointly to induce harmful, misleading, or policy violating outputs. Existing defense strategies, such as adversarial training and input purification, face notable limitations: adversarial training typically improves robustness only against known attacks while incurring high computational costs, whereas conventional purification approaches often suffer from degraded image quality and insufficient generalization to complex multimodal tasks. In this work, we focus on defending the visual modality, which frequently serves as the primary entry point for adversarial manipulation. We propose a supervised diffusion based denoising framework that leverages paired adversarial clean image datasets to fine-tune diffusion models with directional, task specific guidance. Unlike prior unsupervised purification methods such as DiffPure, our approach achieves higher quality reconstructions while significantly improving defense robustness in multimodal tasks. Furthermore, we incorporate prompt optimization as a complementary defense mechanism, enhancing resistance against diverse and unseen attack strategies. Extensive experiments on image captioning and visual question answering demonstrate that our method not only substantially improves robustness but also exhibits strong transferability to unknown adversarial attacks. These results highlight the effectiveness of supervised diffusion based denoising for multimodal defense, paving the way for more reliable and secure deployment of MLLMs in real world applications.

Method: The survey introduces a taxonomy grounded in disentanglement with four approach families: no explicit disentanglement, textual disentanglement, visual disentanglement, and cross-modal disentanglement. It provides detailed comparative analysis across different problem settings.

Result: The paper systematically reviews CZSL methods, highlighting their core advantages and limitations in various settings like closed-world and open-world CZSL, and identifies significant open challenges.

Conclusion: This survey serves as a foundational resource to guide and inspire further advancements in Compositional Zero-Shot Learning, with available code and papers on their GitHub repository.

Abstract: Compositional Zero-Shot Learning (CZSL) is a critical task in computer vision that enables models to recognize unseen combinations of known attributes and objects during inference, addressing the combinatorial challenge of requiring training data for every possible composition. This is particularly challenging because the visual appearance of primitives is highly contextual; for example, small'' cats appear visually distinct from older’’ ones, and wet'' cars differ significantly from wet’’ cats. Effectively modeling this contextuality and the inherent compositionality is crucial for robust compositional zero-shot recognition. This paper presents, to our knowledge, the first comprehensive survey specifically focused on Compositional Zero-Shot Learning. We systematically review the state-of-the-art CZSL methods, introducing a taxonomy grounded in disentanglement, with four families of approaches: no explicit disentanglement, textual disentanglement, visual disentanglement, and cross-modal disentanglement. We provide a detailed comparative analysis of these methods, highlighting their core advantages and limitations in different problem settings, such as closed-world and open-world CZSL. Finally, we identify the most significant open challenges and outline promising future research directions. This survey aims to serve as a foundational resource to guide and inspire further advancements in this fascinating and important field. Papers studied in this survey with their official code are available on our github: https://github.com/ans92/Compositional-Zero-Shot-Learning

Method: Proposed pixel-aligned Motion Map (MoMap) representation for 3D scene motion, created large-scale database from 50,000+ real videos, trained diffusion model on MoMaps, and developed pipeline for 2D video synthesis via warping and completion.

Result: Experimental results show the approach generates plausible and semantically consistent 3D scene motion.

Conclusion: The MoMap representation and diffusion-based approach effectively enable 3D motion prediction from single images and suggest a new pipeline for 2D video synthesis.

Abstract: This paper addresses the challenge of learning semantically and functionally meaningful 3D motion priors from real-world videos, in order to enable prediction of future 3D scene motion from a single input image. We propose a novel pixel-aligned Motion Map (MoMap) representation for 3D scene motion, which can be generated from existing generative image models to facilitate efficient and effective motion prediction. To learn meaningful distributions over motion, we create a large-scale database of MoMaps from over 50,000 real videos and train a diffusion model on these representations. Our motion generation not only synthesizes trajectories in 3D but also suggests a new pipeline for 2D video synthesis: first generate a MoMap, then warp an image accordingly and complete the warped point-based renderings. Experimental results demonstrate that our approach generates plausible and semantically consistent 3D scene motion.

Method: Uses region-aware disentanglement to separate static (anatomy) and dynamic (pathology progression) features in CXRs, then hierarchically aligns these features with EHR data via local and global synchronization.

Result: Extensive experiments on MIMIC dataset show DiPro effectively extracts temporal clinical dynamics and achieves state-of-the-art performance on disease progression identification and ICU prediction tasks.

Conclusion: DiPro successfully addresses challenges in multimodal longitudinal data by disentangling meaningful features and aligning them across different timescales for improved disease progression modeling.

Abstract: Longitudinal multimodal data, including electronic health records (EHR) and sequential chest X-rays (CXRs), is critical for modeling disease progression, yet remains underutilized due to two key challenges: (1) redundancy in consecutive CXR sequences, where static anatomical regions dominate over clinically-meaningful dynamics, and (2) temporal misalignment between sparse, irregular imaging and continuous EHR data. We introduce $\texttt{DiPro}$, a novel framework that addresses these challenges through region-aware disentanglement and multi-timescale alignment. First, we disentangle static (anatomy) and dynamic (pathology progression) features in sequential CXRs, prioritizing disease-relevant changes. Second, we hierarchically align these static and dynamic CXR features with asynchronous EHR data via local (pairwise interval-level) and global (full-sequence) synchronization to model coherent progression pathways. Extensive experiments on the MIMIC dataset demonstrate that $\texttt{DiPro}$ could effectively extract temporal clinical dynamics and achieve state-of-the-art performance on both disease progression identification and general ICU prediction tasks.

Method: Created synthetic numerosity benchmark (GrayCount250 and NaturalCount6), analyzed scaling effects, identified noise initialization bias, and proposed count-aware layout injection into noise prior.

Result: Scaling hypothesis rejected - larger models/datasets didn’t improve counting accuracy. Proposed method improved GrayCount250 from 20.0% to 85.3% and NaturalCount6 from 74.8% to 86.3%.

Conclusion: Diffusion models rely more on noise initialization than prompt numerosity, but count-aware layout injection effectively addresses counting challenges and generalizes well.

Abstract: Numerosity remains a challenge for state-of-the-art text-to-image generation models like FLUX and GPT-4o, which often fail to accurately follow counting instructions in text prompts. In this paper, we aim to study a fundamental yet often overlooked question: Can diffusion models inherently generate the correct number of objects specified by a textual prompt simply by scaling up the dataset and model size? To enable rigorous and reproducible evaluation, we construct a clean synthetic numerosity benchmark comprising two complementary datasets: GrayCount250 for controlled scaling studies, and NaturalCount6 featuring complex naturalistic scenes. Second, we empirically show that the scaling hypothesis does not hold: larger models and datasets alone fail to improve counting accuracy on our benchmark. Our analysis identifies a key reason: diffusion models tend to rely heavily on the noise initialization rather than the explicit numerosity specified in the prompt. We observe that noise priors exhibit biases toward specific object counts. In addition, we propose an effective strategy for controlling numerosity by injecting count-aware layout information into the noise prior. Our method achieves significant gains, improving accuracy on GrayCount250 from 20.0% to 85.3% and on NaturalCount6 from 74.8% to 86.3%, demonstrating effective generalization across settings.

Method: Introduces a test-time-training (TTT) memory module that continually updates token representations to capture long-range dependencies, replacing token merging. Uses a prompt-dependent memory reader that selectively retrieves context-relevant content from fixed-size memory, with TTT optimized using Hessian-free conjugate-gradient procedure.

Result: Achieves 74.2% overall and 67.8% on Video-MME long split, outperforming both offline and streaming baselines. Sustains high-quality understanding on multi-hour videos with 10k frames and 1M tokens.

Conclusion: video-SALMONN S successfully addresses the scalability challenge in video-understanding LLMs, enabling continuous processing of long video streams under fixed memory constraints while maintaining performance.

Abstract: Continuous, high-frame-rate, high-resolution processing of long video streams is critical for future AI agents, yet current video-understanding LLMs struggle to scale. Offline, fixed-frame-number methods require the stream length to adapt frame rates; streaming methods constrain memory by merging or discarding tokens, losing information. We propose video-SALMONN S, a streaming audio-visual LLM that, to our knowledge, is the first to process 3-hour videos at 1 FPS and 360p resolution under a fixed memory budget. Our model introduces (i) a test-time-training (TTT) memory module that continually updates token representations to capture long-range dependencies by replacing token merging, and (ii) a prompt-dependent memory reader that selectively retrieves context-relevant content from fixed-size memory. The TTT module is optimised with a Hessian-free conjugate-gradient procedure (TTT_HF) for efficient adaptation. On long-video benchmarks (Video-MME, LVBench, VideoEvalPro), video-SALMONN S sustains high-quality understanding on multi-hour videos with 10k frames and 1M tokens. Our 8B-parameter model achieves 74.2% overall and 67.8% on the Video-MME long split, outperforming both offline and streaming baselines.

Method: Developed a morphology-assisted ensemble AI model combining image processing with transformer-based machine learning (GC-ViT) to predict DNA fragmentation from phase contrast microscopy images, using TUNEL assay as gold standard.

Result: The proposed framework achieved 60% sensitivity and 75% specificity in detecting sperm DNA fragmentation, outperforming pure transformer vision models and morphology-only models.

Conclusion: This non-destructive AI methodology enables real-time sperm selection based on DNA integrity, representing a significant advancement for reproductive medicine diagnostics and therapeutic applications.

Abstract: Sperm DNA fragmentation (SDF) is a critical parameter in male fertility assessment that conventional semen analysis fails to evaluate. This study presents the validation of a novel artificial intelligence (AI) tool designed to detect SDF through digital analysis of phase contrast microscopy images, using the terminal deoxynucleotidyl transferase dUTP nick end labeling (TUNEL) assay as the gold standard reference. Utilising the established link between sperm morphology and DNA integrity, the present work proposes a morphology assisted ensemble AI model that combines image processing techniques with state-of-the-art transformer based machine learning models (GC-ViT) for the prediction of DNA fragmentation in sperm from phase contrast images. The ensemble model is benchmarked against a pure transformer vision' model as well as a morphology-only` model. Promising results show the proposed framework is able to achieve sensitivity of 60% and specificity of 75%. This non-destructive methodology represents a significant advancement in reproductive medicine by enabling real-time sperm selection based on DNA integrity for clinical diagnostic and therapeutic applications.

Method: Represents covariance matrices on HPD manifold and multi-features on Grassmann manifold, uses kernel metric learning networks for manifold representations, employs trusted multiview evidence fusion with Dempster-Shafer theory for combining evidence.

Result: Extensive experiments on three real-world PolSAR datasets show consistent outperformance over existing methods in accuracy, robustness, and interpretability.

Conclusion: MMEFnet effectively integrates PolSAR manifold learning and evidence fusion, providing more reliable and interpretable classification with uncertainty quantification.

Abstract: Polarimetric Synthetic Aperture Radar (PolSAR) covariance matrices and their extracted multi-features - such as scattering angle, entropy, texture, and boundary descriptors - provide complementary and physically interpretable information for image classification. Traditional fusion strategies typically concatenate these features or employ deep learning networks to combine them. However, the covariance matrices and multi-features, as two complementary views, lie on different manifolds with distinct geometric structures. Existing fusion methods also overlook the varying importance of different views and ignore uncertainty, often leading to unreliable predictions. To address these issues, we propose a Multiview Manifold Evidential Fusion (MMEFnet) method to effectively fuse these two views. It gives a new framework to integrate PolSAR manifold learning and evidence fusion into a unified architecture. Specifically, covariance matrices are represented on the Hermitian Positive Definite (HPD) manifold, while multi-features are modeled on the Grassmann manifold. Two different kernel metric learning networks are constructed to learn their manifold representations. Subsequently, a trusted multiview evidence fusion, replacing the conventional softmax classifier, estimates belief mass and quantifies the uncertainty of each view from the learned deep features. Finally, a Dempster-Shafer theory-based fusion strategy combines evidence, enabling a more reliable and interpretable classification. Extensive experiments on three real-world PolSAR datasets demonstrate that the proposed method consistently outperforms existing approaches in accuracy, robustness, and interpretability.

Method: PICO quantifies the probability of each feature column representing semantic information and uses it as weight during embedding interaction. It employs prototype iterative construction with performance feedback-based weighting to ensure reliable semantic probability.

Result: Extensive experiments show PICO outperforms state-of-the-art methods by 5.2%-14.1% across various benchmarks and model backbones.

Conclusion: PICO effectively suppresses style interference in cross-modal alignment by separating semantic from style information through probabilistic feature weighting and iterative prototype construction.

Abstract: Cross-modal alignment is an important multi-modal task, aiming to bridge the semantic gap between different modalities. The most reliable fundamention for achieving this objective lies in the semantic consistency between matched pairs. Conventional methods implicitly assume embeddings contain solely semantic information, ignoring the impact of non-semantic information during alignment, which inevitably leads to information bias or even loss. These non-semantic information primarily manifest as stylistic variations in the data, which we formally define as style information. An intuitive approach is to separate style from semantics, aligning only the semantic information. However, most existing methods distinguish them based on feature columns, which cannot represent the complex coupling relationship between semantic and style information. In this paper, we propose PICO, a novel framework for suppressing style interference during embedding interaction. Specifically, we quantify the probability of each feature column representing semantic information, and regard it as the weight during the embedding interaction. To ensure the reliability of the semantic probability, we propose a prototype iterative construction method. The key operation of this method is a performance feedback-based weighting function, and we have theoretically proven that the function can assign higher weight to prototypes that bring higher performance improvements. Extensive experiments on various benchmarks and model backbones demonstrate the superiority of PICO, outperforming state-of-the-art methods by 5.2%-14.1%.

Method: Knowledge distillation framework that transfers capabilities from giga-scale models to large-scale models using only 1,000 pathology slides per target cancer type.

Result: Distilled models outperformed same-size state-of-the-art models, sometimes surpassed giga-scale teacher and huge-scale models, and showed higher robustness to multi-institutional image variations.

Conclusion: The distillation approach provides a data- and parameter-efficient way to achieve giga-scale performance for cancer-specific applications without prohibitive computational costs.

Abstract: Recent studies in pathology foundation models have shown that scaling training data, diversifying cancer types, and increasing model size consistently improve their performance. However, giga-scale foundation models, which are trained on hundreds of thousands of slides covering tens of cancer types and contain billions of parameters, pose significant challenges for practical use due to their tremendous computational costs in both development and deployment. In this work, we present a novel strategy, named the G2L framework, to increase the performance of large-scale foundation models, which consist of only $15%$ of the parameters of giga-scale models, to a comparable performance level of giga-scale models in cancer-specific tasks. Our approach applies knowledge distillation, transferring the capabilities of a giga-scale model to a large-scale model, using just 1K pathology slides of a target cancer (e.g., breast, prostate, etc.). The resulting distilled model not only outperformed state-of-the-art models of the same size (i.e., large-scale) across several benchmarks but also, interestingly, surpassed the giga-scale teacher and huge-scale models in some benchmarks. In addition, the distilled model exhibited a higher robustness index, indicating improved resilience to image variations originating from multiple institutions. These findings suggest that the proposed distillation approach for a large-scale model is a data- and parameter-efficient way to achieve giga-scale-level performance for cancer-specific applications without prohibitive computational burden.

Method: Built on BLEnD dataset, created 313 cultural question templates with three aligned MCQ formats: text-only baseline (Region→Entity), inverted text-only (Entity→Region), and VQA-style with generated images, totaling 4,916 images and 21,000+ MCQ instances.

Result: VLMs show significant fragility in cultural knowledge with performance drops under linguistic rephrasing. Visual cues help but low cross-modal consistency reveals challenges in integrating textual and visual understanding, especially for lower-resource regions.

Conclusion: BLEnD-Vis provides a crucial testbed for analyzing cultural robustness and multimodal grounding, exposing limitations and guiding development of more culturally competent VLMs.

Abstract: As vision-language models (VLMs) are deployed globally, their ability to understand culturally situated knowledge becomes essential. Yet, existing evaluations largely assess static recall or isolated visual grounding, leaving unanswered whether VLMs possess robust and transferable cultural understanding. We introduce BLEnD-Vis, a multimodal, multicultural benchmark designed to evaluate the robustness of everyday cultural knowledge in VLMs across linguistic rephrasings and visual modalities. Building on the BLEnD dataset, BLEnD-Vis constructs 313 culturally grounded question templates spanning 16 regions and generates three aligned multiple-choice formats: (i) a text-only baseline querying from Region $\to$ Entity, (ii) an inverted text-only variant (Entity $\to$ Region), and (iii) a VQA-style version of (ii) with generated images. The resulting benchmark comprises 4,916 images and over 21,000 multiple-choice question (MCQ) instances, validated through human annotation. BLEnD-Vis reveals significant fragility in current VLM cultural knowledge; models exhibit performance drops under linguistic rephrasing and, whilst visual cues often aid performance, low cross-modal consistency highlights challenges in robustly integrating textual and visual understanding, particularly for lower-resource regions. BLEnD-Vis thus provides a crucial testbed for systematically analysing cultural robustness and multimodal grounding, exposing limitations and guiding the development of more culturally competent VLMs.

Method: Used T5 models for SSL translation with a novel SSL dataset containing three testing protocols. Compared models pre-trained on YouTubeASL dataset versus models trained directly on SSL data.

Result: Pre-training on YouTubeASL significantly improved model performance by approximately 3 times in BLEU-4 score, showing effective cross-linguistic transfer from ASL to SSL.

Conclusion: Leveraging large-scale ASL data through pre-training is beneficial for improving SSL translation, highlighting the transferability of sign language models across different sign languages.

Abstract: This paper explores the application of T5 models for Saudi Sign Language (SSL) translation using a novel dataset. The SSL dataset includes three challenging testing protocols, enabling comprehensive evaluation across different scenarios. Additionally, it captures unique SSL characteristics, such as face coverings, which pose challenges for sign recognition and translation. In our experiments, we investigate the impact of pre-training on American Sign Language (ASL) data by comparing T5 models pre-trained on the YouTubeASL dataset with models trained directly on the SSL dataset. Experimental results demonstrate that pre-training on YouTubeASL significantly improves models’ performance (roughly $3\times$ in BLEU-4), indicating cross-linguistic transferability in sign language models. Our findings highlight the benefits of leveraging large-scale ASL data to improve SSL translation and provide insights into the development of more effective sign language translation systems. Our code is publicly available at our GitHub repository.

Method: FlexAC induces hallucination-guided intermediate representations, constructs associative steering vectors from high-association instances, adaptively calibrates their strengths, and incorporates task-specific vectors from target-domain samples for multi-dimensional associative control.

Result: Achieves up to 5.8x improvement in creativity on Creation-MMBench and 29% reduction in hallucination rate on CHAIR, surpassing existing baselines.

Conclusion: FlexAC effectively enables flexible control over associative reasoning in MLLMs, balancing creative guidance with output stability across different tasks.

Abstract: Multimodal large language models (MLLMs) face an inherent trade-off between faithfulness and creativity, as different tasks require varying degrees of associative reasoning. However, existing methods lack the flexibility to modulate this reasoning strength, limiting MLLMs’ adaptability across factual and creative scenarios. To bridge this gap, we propose equipping MLLMs with mechanisms that enable flexible control over associative reasoning. We begin by investigating the internal mechanisms underlying associative behavior in MLLMs and find that: (1) middle layers play a pivotal role in shaping model’s associative tendencies, (2) modifying representations in these layers effectively regulates associative reasoning strength, and (3) hallucinations can be exploited to derive steering vectors that guide this modulation. Building on these findings, we introduce Flexible Association Control (FlexAC), a lightweight and training-free framework for modulating associative behavior in MLLMs. FlexAC first induces hallucination-guided intermediate representations to encode associative directions. Then, it selects high-association instances to construct effective associative steering vectors, whose strengths are adaptively calibrated to balance creative guidance with output stability. Finally, recognizing the multi-dimensional nature of associative reasoning, FlexAC incorporates task-specific associative vectors derived from a forward pass on a few target-domain samples, enabling models to follow diverse associative directions and better adapt to creative tasks. Notably, our method achieves up to a 5.8x improvement in creativity on Creation-MMBench and a 29% reduction in hallucination rate on CHAIR, surpassing existing baselines and demonstrating its effectiveness in enabling flexible control over associative reasoning in MLLMs. Our code is available at https://github.com/ylhz/FlexAC.

Method: Proposes a class prototypes-based supervised contrastive learning approach with multimodal transformer network to capture visual-audio interactions, treating it as fine-grained multilabel classification.

Result: The approach outperforms strong baselines on the APPROVE dataset (193 hours of expert-annotated videos with 19 classes) and other benchmarks like Youtube-8M and COIN.

Conclusion: The proposed multimodal contrastive learning method effectively detects fine-grained educational content in videos and the APPROVE dataset provides a valuable resource for educational content filtering.

Abstract: The recent growth in the consumption of online media by children during early childhood necessitates data-driven tools enabling educators to filter out appropriate educational content for young learners. This paper presents an approach for detecting educational content in online videos. We focus on two widely used educational content classes: literacy and math. For each class, we choose prominent codes (sub-classes) based on the Common Core Standards. For example, literacy codes include letter names', letter sounds’, and math codes include counting', sorting’. We pose this as a fine-grained multilabel classification problem as videos can contain multiple types of educational content and the content classes can get visually similar (e.g., letter names' vs letter sounds’). We propose a novel class prototypes based supervised contrastive learning approach that can handle fine-grained samples associated with multiple labels. We learn a class prototype for each class and a loss function is employed to minimize the distances between a class prototype and the samples from the class. Similarly, distances between a class prototype and the samples from other classes are maximized. As the alignment between visual and audio cues are crucial for effective comprehension, we consider a multimodal transformer network to capture the interaction between visual and audio cues in videos while learning the embedding for videos. For evaluation, we present a dataset, APPROVE, employing educational videos from YouTube labeled with fine-grained education classes by education researchers. APPROVE consists of 193 hours of expert-annotated videos with 19 classes. The proposed approach outperforms strong baselines on APPROVE and other benchmarks such as Youtube-8M, and COIN. The dataset is available at https://github.com/rohit-gupta/MMContrast/tree/main/APPROVE

Method: Used FLAME 3D morphable model to disentangle facial shape and expression dynamics, extracted frame-by-frame parameters from conversational videos, and trained a Conformer model with supervised contrastive learning on the CANDOR dataset.

Result: Achieved 61.14% accuracy on 1,429-way classification (458 times above chance), demonstrating strong identity signatures in facial dynamics. Introduced drift-to-noise ratio (DNR) that negatively correlates with recognition performance.

Conclusion: Facial dynamics contain person-specific signatures that enable identification independent of static appearance, with implications for social perception and clinical assessment.

Abstract: This work investigates whether individuals can be identified solely through the pure dynamical components of their facial expressions, independent of static facial appearance. We leverage the FLAME 3D morphable model to achieve explicit disentanglement between facial shape and expression dynamics, extracting frame-by-frame parameters from conversational videos while retaining only expression and jaw coefficients. On the CANDOR dataset of 1,429 speakers in naturalistic conversations, our Conformer model with supervised contrastive learning achieves 61.14%accuracy on 1,429-way classification – 458 times above chance – demonstrating that facial dynamics carry strong identity signatures. We introduce a drift-to-noise ratio (DNR) that quantifies the reliability of shape expression separation by measuring across-session shape changes relative to within-session variability. DNR strongly negatively correlates with recognition performance, confirming that unstable shape estimation compromises dynamic identification. Our findings reveal person-specific signatures in conversational facial dynamics, with implications for social perception and clinical assessment.

Method: Constructed DocPair dataset with 117K paired documents across 32 domains and 267 document types, each containing high- and low-professionalism versions with identical content but different structure/style. Trained DocReward using Bradley-Terry loss to score documents and penalize predictions that contradict human rankings.

Result: DocReward outperforms GPT-4o and GPT-5 in accuracy by 30.6 and 19.4 percentage points respectively. In document generation evaluation, DocReward achieves 60.8% win rate compared to GPT-5’s 37.7%, demonstrating superior performance in guiding generation agents.

Conclusion: DocReward effectively addresses the gap in evaluating document structure and style, significantly outperforming existing models and proving valuable for guiding agentic workflows to produce human-preferred professional documents.

Abstract: Recent advances in agentic workflows have enabled the automation of tasks such as professional document generation. However, they primarily focus on textual quality, neglecting visual structure and style, which are crucial for readability and engagement. This gap arises mainly from the absence of suitable reward models to guide agentic workflows toward producing documents with stronger structural and stylistic quality. To address this, we propose DocReward, a document reward model that evaluates documents based on their structure and style. We construct a multi-domain dataset DocPair of 117K paired documents, covering 32 domains and 267 document types, each including a high- and low-professionalism document with identical content but different structure and style. This enables the model to evaluate professionalism comprehensively, and in a textual-quality-agnostic way. DocReward is trained using the Bradley-Terry loss to score documents, penalizing predictions that contradict the annotated ranking. To assess the performance of reward models, we create a test dataset containing document bundles ranked by well-educated human evaluators. Notably, DocReward outperforms GPT-4o and GPT-5 in accuracy by 30.6 and 19.4 percentage points, respectively, demonstrating its superiority over baselines. In an extrinsic evaluation of document generation, DocReward achieves a significantly higher win rate of 60.8%, compared to GPT-5’s 37.7% win rate, demonstrating its utility in guiding generation agents toward producing human-preferred documents.

Method: Built a custom lightweight CNN architecture with four blocks of stacked convolutional layers, trained on 5,856 chest X-ray images with preprocessing (resizing to 224x224, grayscale conversion, normalization) and data augmentation (rotation, zoom, shear) to prevent overfitting.

Result: Achieved exceptional performance with overall accuracy of 0.942, precision of 0.92, F1-Score of 0.96, and critically high sensitivity (recall) of 0.99 on independent test set, outperforming heavier architectures on the same dataset.

Conclusion: LightPneumoNet provides an efficient solution for pneumonia detection that can be deployed on low-cost hardware, making advanced computer-aided diagnosis accessible in underserved clinics and serving as a reliable second-opinion tool to improve patient outcomes.

Abstract: Effective pneumonia diagnosis is often challenged by the difficulty of deploying large, computationally expensive deep learning models in resource-limited settings. This study introduces LightPneumoNet, an efficient, lightweight convolutional neural network (CNN) built from scratch to provide an accessible and accurate diagnostic solution for pneumonia detection from chest X-rays. Our model was trained on a public dataset of 5,856 chest X-ray images. Preprocessing included image resizing to 224x224, grayscale conversion, and pixel normalization, with data augmentation (rotation, zoom, shear) to prevent overfitting. The custom architecture features four blocks of stacked convolutional layers and contains only 388,082 trainable parameters, resulting in a minimal 1.48 MB memory footprint. On the independent test set, our model delivered exceptional performance, achieving an overall accuracy of 0.942, precision of 0.92, and an F1-Score of 0.96. Critically, it obtained a sensitivity (recall) of 0.99, demonstrating a near-perfect ability to identify true pneumonia cases and minimize clinically significant false negatives. Notably, LightPneumoNet achieves this high recall on the same dataset where existing approaches typically require significantly heavier architectures or fail to reach comparable sensitivity levels. The model’s efficiency enables deployment on low-cost hardware, making advanced computer-aided diagnosis accessible in underserved clinics and serving as a reliable second-opinion tool to improve patient outcomes.

Method: Created a benchmark NSL dataset with 36 gesture classes, then fine-tuned MobileNetV2 and ResNet50 architectures on this dataset for sign recognition.

Result: MobileNetV2 achieved 90.45% classification accuracy and ResNet50 achieved 88.78% accuracy on the NSL dataset.

Conclusion: Convolutional neural networks are effective for sign recognition in low-resource settings, and transfer learning/fine-tuning can advance research in underexplored sign languages.

Abstract: Sign languages serve as essential communication systems for individuals with hearing and speech impairments. However, digital linguistic dataset resources for underrepresented sign languages, such as Nepali Sign Language (NSL), remain scarce. This study introduces the first benchmark dataset for NSL, consisting of 36 gesture classes with 1,500 samples per class, designed to capture the structural and visual features of the language. To evaluate recognition performance, we fine-tuned MobileNetV2 and ResNet50 architectures on the dataset, achieving classification accuracies of 90.45% and 88.78%, respectively. These findings demonstrate the effectiveness of convolutional neural networks in sign recognition tasks, particularly within low-resource settings. To the best of our knowledge, this work represents the first systematic effort to construct a benchmark dataset and assess deep learning approaches for NSL recognition, highlighting the potential of transfer learning and fine-tuning for advancing research in underexplored sign languages.

Method: DTEA model with Semantic Topology Reconfiguration (STR) module that reorganizes multi-scale semantic features into a dynamic hypergraph, and Entropic Perturbation Gating (EPG) module that assesses channel stability and filters high-entropy channels to improve spatial attention.

Result: Extensive experiments on three benchmark datasets show superior segmentation accuracy and better generalization across various clinical settings compared to existing methods.

Conclusion: The proposed DTEA framework with STR and EPG modules effectively enhances structural and semantic representation in medical image segmentation, achieving improved performance and generalization in clinical applications.

Abstract: In medical image segmentation, skip connections are used to merge global context and reduce the semantic gap between encoder and decoder. Current methods often struggle with limited structural representation and insufficient contextual modeling, affecting generalization in complex clinical scenarios. We propose the DTEA model, featuring a new skip connection framework with the Semantic Topology Reconfiguration (STR) and Entropic Perturbation Gating (EPG) modules. STR reorganizes multi-scale semantic features into a dynamic hypergraph to better model cross-resolution anatomical dependencies, enhancing structural and semantic representation. EPG assesses channel stability after perturbation and filters high-entropy channels to emphasize clinically important regions and improve spatial attention. Extensive experiments on three benchmark datasets show our framework achieves superior segmentation accuracy and better generalization across various clinical settings. The code is available at \href{https://github.com/LWX-Research/DTEA}{https://github.com/LWX-Research/DTEA}.

Method: Four-phase framework: 1) Auto-DG for dataset generation, 2) scale bar object detection, 3) hybrid OCR with DenseNet and CRNN, 4) LLM agent for verification and analysis.

Result: Object detection achieved 100% precision, 95.8% recall, 99.2% mAP@0.5; OCR achieved 89% precision, 65% recall, 75% F1 score; outperforms mainstream OCR engines.

Conclusion: The automated LLM-powered framework significantly enhances efficiency and accuracy of scale bar detection in SEM images, advancing scientific imaging analysis.

Abstract: Microscopic characterizations, such as Scanning Electron Microscopy (SEM), are widely used in scientific research for visualizing and analyzing microstructures. Determining the scale bars is an important first step of accurate SEM analysis; however, currently, it mainly relies on manual operations, which is both time-consuming and prone to errors. To address this issue, we propose a multi-modal and automated scale bar detection and extraction framework that provides concurrent object detection, text detection and text recognition with a Large Language Model (LLM) agent. The proposed framework operates in four phases; i) Automatic Dataset Generation (Auto-DG) model to synthesize a diverse dataset of SEM images ensuring robust training and high generalizability of the model, ii) scale bar object detection, iii) information extraction using a hybrid Optical Character Recognition (OCR) system with DenseNet and Convolutional Recurrent Neural Network (CRNN) based algorithms, iv) an LLM agent to analyze and verify accuracy of the results. The proposed model demonstrates a strong performance in object detection and accurate localization with a precision of 100%, recall of 95.8%, and a mean Average Precision (mAP) of 99.2% at IoU=0.5 and 69.1% at IoU=0.5:0.95. The hybrid OCR system achieved 89% precision, 65% recall, and a 75% F1 score on the Auto-DG dataset, significantly outperforming several mainstream standalone engines, highlighting its reliability for scientific image analysis. The LLM is introduced as a reasoning engine as well as an intelligent assistant that suggests follow-up steps and verifies the results. This automated method powered by an LLM agent significantly enhances the efficiency and accuracy of scale bar detection and extraction in SEM images, providing a valuable tool for microscopic analysis and advancing the field of scientific imaging.

Method: Proposes Class Vectors that capture class-specific representation adjustments during fine-tuning. These vectors disentangle each class’s adaptation in latent space and can be used to steer latent features or update decision boundaries in weight space.

Result: Class Vectors effectively capture semantic shifts for each class, demonstrate linearity and orthogonality properties that enable efficient concept editing via class arithmetic, and show utility in applications like unlearning, environmental adaptation, and adversarial defense.

Conclusion: Class Vectors provide a flexible and efficient approach for post-hoc classifier editing, overcoming limitations of existing methods and enabling practical applications in various domains requiring model adaptation.

Abstract: Image classifiers play a critical role in detecting diseases in medical imaging and identifying anomalies in manufacturing processes. However, their predefined behaviors after extensive training make post hoc model editing difficult, especially when it comes to forgetting specific classes or adapting to distribution shifts. Existing classifier editing methods either focus narrowly on correcting errors or incur extensive retraining costs, creating a bottleneck for flexible editing. Moreover, such editing has seen limited investigation in image classification. To overcome these challenges, we introduce Class Vectors, which capture class-specific representation adjustments during fine-tuning. Whereas task vectors encode task-level changes in weight space, Class Vectors disentangle each class’s adaptation in the latent space. We show that Class Vectors capture each class’s semantic shift and that classifier editing can be achieved either by steering latent features along these vectors or by mapping them into weight space to update the decision boundaries. We also demonstrate that the inherent linearity and orthogonality of Class Vectors support efficient, flexible, and high-level concept editing via simple class arithmetic. Finally, we validate their utility in applications such as unlearning, environmental adaptation, adversarial defense, and adversarial trigger optimization.

Method: Combines Edge-Aware Enhancement Unit (EAEU) for multi-frequency edge feature extraction, Multi-scale Prompt Generation Unit (MSPGU) for semantic-spatial feature integration, and Dual-Source Adaptive Gated Fusion Unit (DAGFU) to merge edge and semantic features.

Result: Tests on ISIC2018 dataset confirm superior performance and reliability as a clinical tool.

Conclusion: EEMS provides enhanced segmentation accuracy and robustness for medical applications.

Abstract: Medical image segmentation is vital for diagnosis, treatment planning, and disease monitoring but is challenged by complex factors like ambiguous edges and background noise. We introduce EEMS, a new model for segmentation, combining an Edge-Aware Enhancement Unit (EAEU) and a Multi-scale Prompt Generation Unit (MSPGU). EAEU enhances edge perception via multi-frequency feature extraction, accurately defining boundaries. MSPGU integrates high-level semantic and low-level spatial features using a prompt-guided approach, ensuring precise target localization. The Dual-Source Adaptive Gated Fusion Unit (DAGFU) merges edge features from EAEU with semantic features from MSPGU, enhancing segmentation accuracy and robustness. Tests on datasets like ISIC2018 confirm EEMS’s superior performance and reliability as a clinical tool.

Method: Four-stage framework: discriminate (identify harmful samples), self-annotate (automatic labeling), error trigger (detect mistakes), and training. Uses iterative process with small seed set (5% of data) to prioritize informative samples.

Result: HaDola matches or outperforms state-of-the-art baselines on VQAv2 and VizWiz datasets with less training data. Models become more accurate and better calibrated while reducing reliance on costly human annotations.

Conclusion: Explicitly modeling human uncertainty in training is more effective than simply scaling dataset size. Better utilization of uncertainty distributions improves model performance and calibration while reducing annotation costs.

Abstract: Large vision-language models (VLMs) achieve strong performance in Visual Question Answering but still rely heavily on supervised fine-tuning (SFT) with massive labeled datasets, which is costly due to human annotations. Crucially, real-world datasets often exhibit human uncertainty (HU) – variation in human confidence across annotations – but standard SFT simply optimizes toward the most frequent label, disregarding HU distributions. This leaves two open questions: How does HU affect SFT, and how can HU be effectively leveraged in training? In this work, we first conduct a systematic evaluation of VLMs across varying HU levels. We have two key findings: (i) surprisingly, high-HU samples contribute little or even degrade model performance, and (ii) naively training on the full dataset yields under-calibrated models that fail to capture HU distributions. Motivated by these findings, we introduce HaDola, a human uncertainty-aware data selection and automatic labeling framework. HaDola operates in four stages – discriminate, self-annotate, error trigger, and training – to iteratively identify harmful samples, prioritize informative ones, and bootstrap from a small seed set (5% of data). Our approach substantially reduces reliance on costly HU annotations and makes VLMs more accurate and better calibrated. Extensive experiments on VQAv2 and VizWiz datasets demonstrate that HaDola consistently matches or outperforms state-of-the-art baselines with less training data. Our work highlights the importance of explicitly modeling HU in SFT, suggesting that better utilization of HU is more effective than merely scaling up dataset size.

Method: Proposed ΔEnergy score based on energy changes during vision-language modality realignment, with EBM (lower-bound maximization) to simultaneously improve OOD detection and generalization. Developed unified fine-tuning framework.

Result: Extensive experiments show superiority, outperforming recent approaches by 10% to 25% in AUROC on challenging OOD detection and generalization benchmarks.

Conclusion: ΔEnergy provides a unified solution that significantly enhances VLMs’ robustness in both OOD generalization and detection, with theoretical guarantees and strong empirical performance.

Abstract: Recent approaches for vision-language models (VLMs) have shown remarkable success in achieving fast downstream adaptation. When applied to real-world downstream tasks, VLMs inevitably encounter both the in-distribution (ID) data and out-of-distribution (OOD) data. The OOD datasets often include both covariate shifts (e.g., known classes with changes in image styles) and semantic shifts (e.g., test-time unseen classes). This highlights the importance of improving VLMs’ generalization ability to covariate-shifted OOD data, while effectively detecting open-set semantic-shifted OOD classes. In this paper, inspired by the substantial energy change observed in closed-set data when re-aligning vision-language modalities (specifically by directly reducing the maximum cosine similarity to a low value), we introduce a novel OOD score, named {\Delta}Energy. {\Delta}Energy significantly outperforms the vanilla energy-based OOD score and provides a more reliable approach for OOD detection. Furthermore, {\Delta}Energy can simultaneously improve OOD generalization under covariate shifts, which is achieved by lower-bound maximization for {\Delta}Energy (termed EBM). EBM is theoretically proven to not only enhance OOD detection but also yields a domain-consistent Hessian, which serves as a strong indicator for OOD generalization. Based on this finding, we developed a unified fine-tuning framework that allows for improving VLMs’ robustness in both OOD generalization and OOD detection. Extensive experiments on challenging OOD detection and generalization benchmarks demonstrate the superiority of our method, outperforming recent approaches by 10% to 25% in AUROC.

Method: Systematic evaluation on 1,000 stratified COCO images and 200 diverse product images, combined with detailed Total Cost of Ownership modeling to establish quantitative break-even thresholds for architecture selection.

Result: Supervised YOLO achieves 91.2% accuracy vs 68.5% for zero-shot Gemini on standard categories, but requires $10,800 annotation cost. Break-even occurs at 55M inferences (151k images/day for 1 year). Gemini shows 52.3% accuracy on diverse products where YOLO gets 0%. Cost per correct detection: Gemini $0.00050 vs YOLO $0.143 at 100k inferences.

Conclusion: Optimal detection architecture selection depends critically on deployment volume, category stability, budget constraints, and accuracy requirements rather than purely technical performance metrics, with clear break-even thresholds established for decision-making.

Abstract: Object detection systems have traditionally relied on supervised learning with manually annotated bounding boxes, achieving high accuracy at the cost of substantial annotation investment. The emergence of Vision-Language Models (VLMs) offers an alternative paradigm enabling zero-shot detection through natural language queries, eliminating annotation requirements but operating with reduced accuracy. This paper presents the first comprehensive cost-effectiveness analysis comparing supervised detection (YOLO) with zero-shot VLM inference (Gemini Flash 2.5). Through systematic evaluation on 1,000 stratified COCO images and 200 diverse product images spanning consumer electronics and rare categories, combined with detailed Total Cost of Ownership modeling, we establish quantitative break-even thresholds governing architecture selection. Our findings reveal that supervised YOLO achieves 91.2% accuracy versus 68.5% for zero-shot Gemini on standard categories, representing a 22.7 percentage point advantage that costs $10,800 in annotation for 100-category systems. However, this advantage justifies investment only beyond 55 million inferences, equivalent to 151,000 images daily for one year. Zero-shot Gemini demonstrates 52.3% accuracy on diverse product categories (ranging from highly web-prevalent consumer electronics at 75-85% to rare specialized equipment at 25-40%) where supervised YOLO achieves 0% due to architectural constraints preventing detection of untrained classes. Cost per Correct Detection analysis reveals substantially lower per-detection costs for Gemini ($0.00050 vs $0.143) at 100,000 inferences despite accuracy deficits. We develop decision frameworks demonstrating that optimal architecture selection depends critically on deployment volume, category stability, budget constraints, and accuracy requirements rather than purely technical performance metrics.

Method: Two-stage generation method with semantic bridging via sketch-to-image translation to enrich sparse representations, and symmetry constraints as geometric priors to leverage structural regularity.

Result: Superior performance compared to existing methods on mainstream sketch datasets in terms of Chamfer Distance, Earth Mover’s Distance, and F-Score.

Conclusion: The proposed semantic bridging and symmetry-aware design effectively address the challenges of sketch-based 3D reconstruction.

Abstract: Sketch-based 3D reconstruction remains a challenging task due to the abstract and sparse nature of sketch inputs, which often lack sufficient semantic and geometric information. To address this, we propose Sketch2Symm, a two-stage generation method that produces geometrically consistent 3D shapes from sketches. Our approach introduces semantic bridging via sketch-to-image translation to enrich sparse sketch representations, and incorporates symmetry constraints as geometric priors to leverage the structural regularity commonly found in everyday objects. Experiments on mainstream sketch datasets demonstrate that our method achieves superior performance compared to existing sketch-based reconstruction methods in terms of Chamfer Distance, Earth Mover’s Distance, and F-Score, verifying the effectiveness of the proposed semantic bridging and symmetry-aware design.

Method: Used ensemble approach with various preprocessing (speckle filtering), flood mapping (supervised/unsupervised), and water depth estimation methods on SAR imagery from two Garonne River flood events, validated against hydrodynamic simulations and in-situ observations.

Result: Speckle filter choice altered flood extent by several km²; supervised flood mapping outperformed unsupervised but tuned unsupervised methods achieved comparable results; compounded uncertainty from preprocessing and mapping introduced high variability in water depth estimates.

Conclusion: Methodological choices significantly impact flood analysis results, with flood mapping method choice being most influential. Ensemble approaches should be adopted to account for uncertainty rather than relying on single configurations.

Abstract: Flood mapping and water depth estimation from Synthetic Aperture Radar (SAR) imagery are crucial for calibrating and validating hydraulic models. This study uses SAR imagery to evaluate various preprocessing (especially speckle noise reduction), flood mapping, and water depth estimation methods. The impact of the choice of method at different steps and its hyperparameters is studied by considering an ensemble of preprocessed images, flood maps, and water depth fields. The evaluation is conducted for two flood events on the Garonne River (France) in 2019 and 2021, using hydrodynamic simulations and in-situ observations as reference data. Results show that the choice of speckle filter alters flood extent estimations with variations of several square kilometers. Furthermore, the selection and tuning of flood mapping methods also affect performance. While supervised methods outperformed unsupervised ones, tuned unsupervised approaches (such as local thresholding or change detection) can achieve comparable results. The compounded uncertainty from preprocessing and flood mapping steps also introduces high variability in the water depth field estimates. This study highlights the importance of considering the entire processing pipeline, encompassing preprocessing, flood mapping, and water depth estimation methods and their associated hyperparameters. Rather than relying on a single configuration, adopting an ensemble approach and accounting for methodological uncertainty should be privileged. For flood mapping, the method choice has the most influence. For water depth estimation, the most influential processing step was the flood map input resulting from the flood mapping step and the hyperparameters of the methods.

Method: Introduces uncertainty into ControlNet’s control mechanism, combining uncertainty control from unlabeled datasets with semantic control from labeled datasets. This allows creation of annotated data with high uncertainty from target domains.

Result: The approach successfully synthesizes annotated images from Home-OCT domain, significantly improving segmentation results without additional supervision. Also demonstrated effectiveness in traffic scene experiments.

Conclusion: Uncertainty-guidance enables arbitrary domain shifts without strict style learning, outperforming style transfer methods and providing a flexible solution for domain adaptation in segmentation tasks.

Abstract: Generative Models are a valuable tool for the controlled creation of high-quality image data. Controlled diffusion models like the ControlNet have allowed the creation of labeled distributions. Such synthetic datasets can augment the original training distribution when discriminative models, like semantic segmentation, are trained. However, this augmentation effect is limited since ControlNets tend to reproduce the original training distribution. This work introduces a method to utilize data from unlabeled domains to train ControlNets by introducing the concept of uncertainty into the control mechanism. The uncertainty indicates that a given image was not part of the training distribution of a downstream task, e.g., segmentation. Thus, two types of control are engaged in the final network: an uncertainty control from an unlabeled dataset and a semantic control from the labeled dataset. The resulting ControlNet allows us to create annotated data with high uncertainty from the target domain, i.e., synthetic data from the unlabeled distribution with labels. In our scenario, we consider retinal OCTs, where typically high-quality Spectralis images are available with given ground truth segmentations, enabling the training of segmentation networks. The recent development in Home-OCT devices, however, yields retinal OCTs with lower quality and a large domain shift, such that out-of-the-pocket segmentation networks cannot be applied for this type of data. Synthesizing annotated images from the Home-OCT domain using the proposed approach closes this gap and leads to significantly improved segmentation results without adding any further supervision. The advantage of uncertainty-guidance becomes obvious when compared to style transfer: it enables arbitrary domain shifts without any strict learning of an image style. This is also demonstrated in a traffic scene experiment.

Method: Four-stage framework: (i) openable-object detection and segmentation, (ii) articulation estimation for joint types and motion parameters, (iii) hidden-geometry completion and interactive object assembly, (iv) interactive scene integration in standard formats.

Result: Achieves state-of-the-art performance on detection/segmentation and articulation metrics across diverse indoor scenes, enabling scalable interactive scene generation.

Conclusion: Provides a practical foundation for scalable interactive scene generation, lowering the barrier to large-scale research on articulated scene understanding.

Abstract: Interactive 3D scenes are increasingly vital for embodied intelligence, yet existing datasets remain limited due to the labor-intensive process of annotating part segmentation, kinematic types, and motion trajectories. We present REACT3D, a scalable zero-shot framework that converts static 3D scenes into simulation-ready interactive replicas with consistent geometry, enabling direct use in diverse downstream tasks. Our contributions include: (i) openable-object detection and segmentation to extract candidate movable parts from static scenes, (ii) articulation estimation that infers joint types and motion parameters, (iii) hidden-geometry completion followed by interactive object assembly, and (iv) interactive scene integration in widely supported formats to ensure compatibility with standard simulation platforms. We achieve state-of-the-art performance on detection/segmentation and articulation metrics across diverse indoor scenes, demonstrating the effectiveness of our framework and providing a practical foundation for scalable interactive scene generation, thereby lowering the barrier to large-scale research on articulated scene understanding. Our project page is \textit{\hypersetup{urlcolor=black}\href{https://react3d.github.io/}{react3d.github.io}}.

Method: Proposes InternSVG with SVG-specific special tokens, subword-based embedding initialization, and two-stage training progressing from short static SVGs to long-sequence illustrations and complex animations. Built on SAgoge dataset and SArena benchmark.

Result: InternSVG achieves substantial gains and consistently outperforms leading open and proprietary counterparts on SArena and prior benchmarks, demonstrating positive transfer and improved overall performance.

Conclusion: The unified MLLM approach with comprehensive data resources enables effective SVG understanding, editing, and generation, addressing key challenges in the field through integrated data-benchmark-model design.

Abstract: General SVG modeling remains challenging due to fragmented datasets, limited transferability of methods across tasks, and the difficulty of handling structural complexity. In response, we leverage the strong transfer and generalization capabilities of multimodal large language models (MLLMs) to achieve unified modeling for SVG understanding, editing, and generation. We present the InternSVG family, an integrated data-benchmark-model suite. At its core is SAgoge, the largest and most comprehensive multimodal dataset for SVG tasks, encompassing both static graphics and dynamic animations. It covers icons, long-sequence illustrations, scientific diagrams, and dynamic animations, supporting tasks of varied difficulty levels and providing deeper hierarchies with richer attributes compared to previous datasets. Based on this resource, we introduce SArena, a companion benchmark with comprehensive task definitions and standardized evaluation that aligns with the domains and difficulty spectrum covered by SAgoge. Building on these foundations, we propose InternSVG, a unified MLLM for SVG understanding, editing, and generation with SVG-specific special tokens, subword-based embedding initialization, and a two-stage training strategy that progresses from short static SVGs to long-sequence illustrations and complex animations. This unified formulation induces positive transfer and improves overall performance. Experiments on SArena and prior benchmark confirm that InternSVG achieves substantial gains and consistently outperforms leading open and proprietary counterparts.

Method: MMAP uses multi-magnification patch representations to capture fine-grained histological details and learns latent prototype embeddings to represent slide-level global context information.

Result: MMAP consistently outperforms all existing state-of-the-art methods across multiple evaluation metrics including MAE, MSE, and Pearson Correlation Coefficient.

Conclusion: The proposed MMAP framework effectively addresses both local feature granularity and global spatial context challenges in predicting spatial gene expression from histological images.

Abstract: Spatial Transcriptomics (ST) enables the measurement of gene expression while preserving spatial information, offering critical insights into tissue architecture and disease pathology. Recent developments have explored the use of hematoxylin and eosin (H&E)-stained whole-slide images (WSIs) to predict transcriptome-wide gene expression profiles through deep neural networks. This task is commonly framed as a regression problem, where each input corresponds to a localized image patch extracted from the WSI. However, predicting spatial gene expression from histological images remains a challenging problem due to the significant modality gap between visual features and molecular signals. Recent studies have attempted to incorporate both local and global information into predictive models. Nevertheless, existing methods still suffer from two key limitations: (1) insufficient granularity in local feature extraction, and (2) inadequate coverage of global spatial context. In this work, we propose a novel framework, MMAP (Multi-MAgnification and Prototype-enhanced architecture), that addresses both challenges simultaneously. To enhance local feature granularity, MMAP leverages multi-magnification patch representations that capture fine-grained histological details. To improve global contextual understanding, it learns a set of latent prototype embeddings that serve as compact representations of slide-level information. Extensive experimental results demonstrate that MMAP consistently outperforms all existing state-of-the-art methods across multiple evaluation metrics, including Mean Absolute Error (MAE), Mean Squared Error (MSE), and Pearson Correlation Coefficient (PCC).

Method: Proposed RALI algorithm using contrastive learning to directly align images with compact text representations learned by RL, eliminating reasoning processes and LLM loading.

Result: RALI achieves generalization performance comparable to reasoning-based models while requiring less than 5% of model parameters and inference time.

Conclusion: The generalization of reasoning-based IQA models comes from converting visual representations to compact text representations, and RALI successfully captures this capability without expensive reasoning processes.

Abstract: Reasoning-based image quality assessment (IQA) models trained through reinforcement learning (RL) exhibit exceptional generalization, yet the underlying mechanisms and critical factors driving this capability remain underexplored in current research. Moreover, despite their superior performance, these models incur inference energy usage and latency orders of magnitude higher than their earlier counterparts, restricting their deployment in specific scenarios. Through extensive experiments, this paper verifies and elaborates that through RL training, MLLMs leverage their reasoning capability to convert redundant visual representations into compact, cross-domain aligned text representations. This conversion is precisely the source of the generalization exhibited by these reasoning-based IQA models. Building on this fundamental insight, we propose a novel algorithm, RALI, which employs contrastive learning to directly align images with these generalizable text representations learned by RL. This approach eliminates the reliance on reasoning processes and even obviates the need to load an LLM. For the quality scoring task, this framework achieves generalization performance comparable to reasoning-based models while requiring less than 5% of their model parameters and inference time.

Method: Enforces 2D Gaussians to produce multi-view consistent material maps during deferred shading, tracks photometric variations across views to identify reflective regions, and introduces environment modeling through ray tracing with 2DGS for indirect illumination.

Result: Faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel view synthesis on widely used benchmarks.

Conclusion: Multi-view consistent material inference with physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting.

Abstract: Modeling reflections from 2D images is essential for photorealistic rendering and novel view synthesis. Recent approaches enhance Gaussian primitives with reflection-related material attributes to enable physically based rendering (PBR) with Gaussian Splatting. However, the material inference often lacks sufficient constraints, especially under limited environment modeling, resulting in illumination aliasing and reduced generalization. In this work, we revisit the problem from a multi-view perspective and show that multi-view consistent material inference with more physically-based environment modeling is key to learning accurate reflections with Gaussian Splatting. To this end, we enforce 2D Gaussians to produce multi-view consistent material maps during deferred shading. We also track photometric variations across views to identify highly reflective regions, which serve as strong priors for reflection strength terms. To handle indirect illumination caused by inter-object occlusions, we further introduce an environment modeling strategy through ray tracing with 2DGS, enabling photorealistic rendering of indirect radiance. Experiments on widely used benchmarks show that our method faithfully recovers both illumination and geometry, achieving state-of-the-art rendering quality in novel views synthesis.

Method: Proposes LM-EEC with: (1) Memory-View MoE module using dual-branch routing to adaptively assign weights to expert features along channel and spatial dimensions, (2) Dual-memory bank system with compression strategy to retain critical long-term information while eliminating redundancy.

Result: Achieves new state-of-the-art results on EgoExo4D benchmark, significantly outperforming existing methods and SAM 2 baseline, with strong generalization across diverse scenarios.

Conclusion: LM-EEC effectively addresses the limitations of SAM 2 for ego-exo correspondence through its dual-memory architecture and adaptive feature routing, demonstrating superior performance on challenging correspondence tasks.

Abstract: Establishing object-level correspondence between egocentric and exocentric views is essential for intelligent assistants to deliver precise and intuitive visual guidance. However, this task faces numerous challenges, including extreme viewpoint variations, occlusions, and the presence of small objects. Existing approaches usually borrow solutions from video object segmentation models, but still suffer from the aforementioned challenges. Recently, the Segment Anything Model 2 (SAM 2) has shown strong generalization capabilities and excellent performance in video object segmentation. Yet, when simply applied to the ego-exo correspondence (EEC) task, SAM 2 encounters severe difficulties due to ineffective ego-exo feature fusion and limited long-term memory capacity, especially for long videos. Addressing these problems, we propose a novel EEC framework based on SAM 2 with long-term memories by presenting a dual-memory architecture and an adaptive feature routing module inspired by Mixture-of-Experts (MoE). Compared to SAM 2, our approach features (i) a Memory-View MoE module which consists of a dual-branch routing mechanism to adaptively assign contribution weights to each expert feature along both channel and spatial dimensions, and (ii) a dual-memory bank system with a simple yet effective compression strategy to retain critical long-term information while eliminating redundancy. In the extensive experiments on the challenging EgoExo4D benchmark, our method, dubbed LM-EEC, achieves new state-of-the-art results and significantly outperforms existing methods and the SAM 2 baseline, showcasing its strong generalization across diverse scenarios. Our code and model are available at https://github.com/juneyeeHu/LM-EEC.

Method: Fusion of high-resolution satellite imagery with AIS trajectory data using YOLO v11 object detection model for vessel detection, classification, and characterization (vessel type, barge cover, operational status, direction, barge count).

Result: High performance across metrics: vessel classification F1=95.8%, barge cover detection F1=91.6%, operational status F1=99.4%, directionality accuracy=93.8%, barge count MAE=2.4 barges. Spatial transferability maintained 98% accuracy.

Conclusion: Integration of non-cooperative satellite sensing with AIS fusion is viable for near-real-time fleet inventories, anomaly detection, and inland waterway surveillance. Future work will expand datasets and incorporate temporal tracking.

Abstract: Maritime Domain Awareness (MDA) for inland waterways remains challenged by cooperative system vulnerabilities. This paper presents a novel framework that fuses high-resolution satellite imagery with vessel trajectory data from the Automatic Identification System (AIS). This work addresses the limitations of AIS-based monitoring by leveraging non-cooperative satellite imagery and implementing a fusion approach that links visual detections with AIS data to identify dark vessels, validate cooperative traffic, and support advanced MDA. The You Only Look Once (YOLO) v11 object detection model is used to detect and characterize vessels and barges by vessel type, barge cover, operational status, barge count, and direction of travel. An annotated data set of 4,550 instances was developed from $5{,}973~\mathrm{mi}^2$ of Lower Mississippi River imagery. Evaluation on a held-out test set demonstrated vessel classification (tugboat, crane barge, bulk carrier, cargo ship, and hopper barge) with an F1 score of 95.8%; barge cover (covered or uncovered) detection yielded an F1 score of 91.6%; operational status (staged or in motion) classification reached an F1 score of 99.4%. Directionality (upstream, downstream) yielded 93.8% accuracy. The barge count estimation resulted in a mean absolute error (MAE) of 2.4 barges. Spatial transferability analysis across geographically disjoint river segments showed accuracy was maintained as high as 98%. These results underscore the viability of integrating non-cooperative satellite sensing with AIS fusion. This approach enables near-real-time fleet inventories, supports anomaly detection, and generates high-quality data for inland waterway surveillance. Future work will expand annotated datasets, incorporate temporal tracking, and explore multi-modal deep learning to further enhance operational scalability.

Method: Based on Qwen3’s LLM and various visual encoders, with comprehensive model architectures, training pipeline, and training data. Introduces a 1+N LoRA approach.

Result: Achieves first-tier performance across text-rich image understanding, reasoning and math, multi-image comprehension, general VQA, hallucination mitigation, multilingual understanding, and GUI-related tasks compared to similar-scale models.

Conclusion: AndesVL provides effective mobile-side MLLMs that balance performance with edge device constraints.

Abstract: In recent years, while cloud-based MLLMs such as QwenVL, InternVL, GPT-4o, Gemini, and Claude Sonnet have demonstrated outstanding performance with enormous model sizes reaching hundreds of billions of parameters, they significantly surpass the limitations in memory, power consumption, and computing capacity of edge devices such as mobile phones. This paper introduces AndesVL, a suite of mobile-side MLLMs with 0.6B to 4B parameters based on Qwen3’s LLM and various visual encoders. We comprehensively outline the model architectures, training pipeline, and training data of AndesVL, which achieves first-tier performance across a wide range of open-source benchmarks, including fields such as text-rich image understanding, reasoning and math, multi-image comprehension, general VQA, hallucination mitigation, multilingual understanding, and GUI-related tasks when compared with state-of-the-art models of a similar scale. Furthermore, we introduce a 1+N LoR

Method: Proposes VGDCFusion with Specific-Prompt Degradation-Coupled Extractor for modality-specific degradation awareness and joint modeling, and Joint-Prompt Degradation-Coupled Fusion for cross-modal degradation perception and residual degradation filtering.

Result: Extensive experiments show VGDCFusion significantly outperforms state-of-the-art fusion approaches under various degraded image scenarios.

Conclusion: The proposed method successfully couples degradation modeling with fusion process and leverages VLMs for effective degradation-aware perception and suppression in image fusion tasks.

Abstract: Existing Infrared and Visible Image Fusion (IVIF) methods typically assume high-quality inputs. However, when handing degraded images, these methods heavily rely on manually switching between different pre-processing techniques. This decoupling of degradation handling and image fusion leads to significant performance degradation. In this paper, we propose a novel VLM-Guided Degradation-Coupled Fusion network (VGDCFusion), which tightly couples degradation modeling with the fusion process and leverages vision-language models (VLMs) for degradation-aware perception and guided suppression. Specifically, the proposed Specific-Prompt Degradation-Coupled Extractor (SPDCE) enables modality-specific degradation awareness and establishes a joint modeling of degradation suppression and intra-modal feature extraction. In parallel, the Joint-Prompt Degradation-Coupled Fusion (JPDCF) facilitates cross-modal degradation perception and couples residual degradation filtering with complementary cross-modal feature fusion. Extensive experiments demonstrate that our VGDCFusion significantly outperforms existing state-of-the-art fusion approaches under various degraded image scenarios. Our code is available at https://github.com/Lmmh058/VGDCFusion.

Method: Incorporates edge-aware image cues, visibility-aware photometric alignment loss, normal-based constraints, and deep image feature embeddings to enforce geometric consistency across views and improve surface boundary delineation.

Result: Achieves state-of-the-art performance in both surface reconstruction and novel view synthesis on standard benchmarks.

Conclusion: The proposed view alignment method significantly improves 3D Gaussian Splatting’s geometric representation capabilities while maintaining its efficiency advantages.

Abstract: 3D Gaussian Splatting has recently emerged as an efficient solution for high-quality and real-time novel view synthesis. However, its capability for accurate surface reconstruction remains underexplored. Due to the discrete and unstructured nature of Gaussians, supervision based solely on image rendering loss often leads to inaccurate geometry and inconsistent multi-view alignment. In this work, we propose a novel method that enhances the geometric representation of 3D Gaussians through view alignment (VA). Specifically, we incorporate edge-aware image cues into the rendering loss to improve surface boundary delineation. To enforce geometric consistency across views, we introduce a visibility-aware photometric alignment loss that models occlusions and encourages accurate spatial relationships among Gaussians. To further mitigate ambiguities caused by lighting variations, we incorporate normal-based constraints to refine the spatial orientation of Gaussians and improve local surface estimation. Additionally, we leverage deep image feature embeddings to enforce cross-view consistency, enhancing the robustness of the learned geometry under varying viewpoints and illumination. Extensive experiments on standard benchmarks demonstrate that our method achieves state-of-the-art performance in both surface reconstruction and novel view synthesis. The source code is available at https://github.com/LeoQLi/VA-GS.

Method: Recast normal integration as estimation of relative scales of continuous components, with heuristic component estimation, optimization term rebalancing, and iterative component merging to reduce problem size.

Result: Achieves state-of-the-art results on standard benchmarks in seconds, with one-order-of-magnitude speedup over pixel-level approaches on large-resolution normal maps.

Conclusion: The proposed component-based approach provides an efficient and scalable solution to surface normal integration by drastically reducing optimization variables while maintaining accuracy.

Abstract: Surface normal integration is a fundamental problem in computer vision, dealing with the objective of reconstructing a surface from its corresponding normal map. Existing approaches require an iterative global optimization to jointly estimate the depth of each pixel, which scales poorly to larger normal maps. In this paper, we address this problem by recasting normal integration as the estimation of relative scales of continuous components. By constraining pixels belonging to the same component to jointly vary their scale, we drastically reduce the number of optimization variables. Our framework includes a heuristic to accurately estimate continuous components from the start, a strategy to rebalance optimization terms, and a technique to iteratively merge components to further reduce the size of the problem. Our method achieves state-of-the-art results on the standard normal integration benchmark in as little as a few seconds and achieves one-order-of-magnitude speedup over pixel-level approaches on large-resolution normal maps.

Method: Uses denoising objective as ELBO-based likelihood surrogate on curated valid-invalid video pairs, with Plausibility Preference Error (PPE) metric tested on 12 scenarios across 4 physics domains.

Result: PPE demonstrates strong alignment with human preference, outperforming state-of-the-art evaluator baselines. Models show clear improvement in physics understanding as capacity and inference settings scale, though struggle with complex dynamics.

Conclusion: LikePhys provides effective evaluation of intuitive physics in video diffusion models, revealing domain-specific capacity variations and scaling trends in physics understanding.

Abstract: Intuitive physics understanding in video diffusion models plays an essential role in building general-purpose physically plausible world simulators, yet accurately evaluating such capacity remains a challenging task due to the difficulty in disentangling physics correctness from visual appearance in generation. To the end, we introduce LikePhys, a training-free method that evaluates intuitive physics in video diffusion models by distinguishing physically valid and impossible videos using the denoising objective as an ELBO-based likelihood surrogate on a curated dataset of valid-invalid pairs. By testing on our constructed benchmark of twelve scenarios spanning over four physics domains, we show that our evaluation metric, Plausibility Preference Error (PPE), demonstrates strong alignment with human preference, outperforming state-of-the-art evaluator baselines. We then systematically benchmark intuitive physics understanding in current video diffusion models. Our study further analyses how model design and inference settings affect intuitive physics understanding and highlights domain-specific capacity variations across physical laws. Empirical results show that, despite current models struggling with complex and chaotic dynamics, there is a clear trend of improvement in physics understanding as model capacity and inference settings scale.

Method: The dataset leverages 11K human observations with egocentric/allocentric perspectives and spatial relations, integrated using LLMs. For point cloud comparison, they propose SCReasoner - an efficient 3D MLLM approach with minimal parameter overhead.

Result: Comprehensive evaluation shows progress and limitations of MLLMs in dynamic scene understanding. Additional experiments demonstrate the task-agnostic effectiveness of Situat3DChange as training data for MLLMs, with good cross-domain transfer capabilities.

Conclusion: Situat3DChange enables better understanding of dynamic 3D environments and supports human-AI collaboration through shared mental models. The proposed SCReasoner efficiently handles point cloud comparison with minimal computational overhead.

Abstract: Physical environments and circumstances are fundamentally dynamic, yet current 3D datasets and evaluation benchmarks tend to concentrate on either dynamic scenarios or dynamic situations in isolation, resulting in incomplete comprehension. To overcome these constraints, we introduce Situat3DChange, an extensive dataset supporting three situation-aware change understanding tasks following the perception-action model: 121K question-answer pairs, 36K change descriptions for perception tasks, and 17K rearrangement instructions for the action task. To construct this large-scale dataset, Situat3DChange leverages 11K human observations of environmental changes to establish shared mental models and shared situational awareness for human-AI collaboration. These observations, enriched with egocentric and allocentric perspectives as well as categorical and coordinate spatial relations, are integrated using an LLM to support understanding of situated changes. To address the challenge of comparing pairs of point clouds from the same scene with minor changes, we propose SCReasoner, an efficient 3D MLLM approach that enables effective point cloud comparison with minimal parameter overhead and no additional tokens required for the language decoder. Comprehensive evaluation on Situat3DChange tasks highlights both the progress and limitations of MLLMs in dynamic scene and situation understanding. Additional experiments on data scaling and cross-domain transfer demonstrate the task-agnostic effectiveness of using Situat3DChange as a training dataset for MLLMs.

Method: Created mmWalk dataset with manually controlled walking trajectories, multi-view sensors, accessibility features, and generated mmWalkVQA benchmark with visual question-answer triplets across 9 categories.

Result: State-of-the-art VLMs struggle with risk assessment and navigational tasks in zero/few-shot settings, but mmWalk-finetuned models show effectiveness on real-world datasets.

Conclusion: The mmWalk dataset advances multi-modal walking assistance research and demonstrates the need for specialized training for navigation tasks in complex outdoor environments.

Abstract: Walking assistance in extreme or complex environments remains a significant challenge for people with blindness or low vision (BLV), largely due to the lack of a holistic scene understanding. Motivated by the real-world needs of the BLV community, we build mmWalk, a simulated multi-modal dataset that integrates multi-view sensor and accessibility-oriented features for outdoor safe navigation. Our dataset comprises 120 manually controlled, scenario-categorized walking trajectories with 62k synchronized frames. It contains over 559k panoramic images across RGB, depth, and semantic modalities. Furthermore, to emphasize real-world relevance, each trajectory involves outdoor corner cases and accessibility-specific landmarks for BLV users. Additionally, we generate mmWalkVQA, a VQA benchmark with over 69k visual question-answer triplets across 9 categories tailored for safe and informed walking assistance. We evaluate state-of-the-art Vision-Language Models (VLMs) using zero- and few-shot settings and found they struggle with our risk assessment and navigational tasks. We validate our mmWalk-finetuned model on real-world datasets and show the effectiveness of our dataset for advancing multi-modal walking assistance.

Method: Proposed Detail Guidance (DG) - a training-free self-guidance strategy that constructs a degraded ‘detail-deficient’ model by disrupting MAs and uses it to guide the original network toward better detail synthesis. DG integrates with Classifier-Free Guidance.

Result: DG consistently improves fine-grained detail quality across various pre-trained DiTs (SD3, SD3.5, and Flux) without requiring retraining.

Conclusion: Massive Activations play a key role in local detail synthesis, and the proposed DG method effectively enhances detail fidelity in DiT-based visual generation systems.

Abstract: Diffusion Transformers (DiTs) have recently emerged as a powerful backbone for visual generation. Recent observations reveal \emph{Massive Activations} (MAs) in their internal feature maps, yet their function remains poorly understood. In this work, we systematically investigate these activations to elucidate their role in visual generation. We found that these massive activations occur across all spatial tokens, and their distribution is modulated by the input timestep embeddings. Importantly, our investigations further demonstrate that these massive activations play a key role in local detail synthesis, while having minimal impact on the overall semantic content of output. Building on these insights, we propose \textbf{D}etail \textbf{G}uidance (\textbf{DG}), a MAs-driven, training-free self-guidance strategy to explicitly enhance local detail fidelity for DiTs. Specifically, DG constructs a degraded ``detail-deficient’’ model by disrupting MAs and leverages it to guide the original network toward higher-quality detail synthesis. Our DG can seamlessly integrate with Classifier-Free Guidance (CFG), enabling further refinements of fine-grained details. Extensive experiments demonstrate that our DG consistently improves fine-grained detail quality across various pre-trained DiTs (\eg, SD3, SD3.5, and Flux).

Method: Created ODI-Bench with 2,000 ODIs and 4,000 QA pairs across 10 tasks. Evaluated 20 MLLMs and proposed Omni-CoT, a training-free chain-of-thought method that integrates textual and visual cues for better ODI understanding.

Result: Experimental results show current MLLMs struggle to capture immersive ODI context. Omni-CoT significantly enhances MLLMs’ comprehension ability in omnidirectional environments.

Conclusion: The study addresses the gap in ODI understanding by providing a comprehensive benchmark and an effective training-free method that improves MLLM performance on omnidirectional images.

Abstract: Omnidirectional images (ODIs) provide full 360x180 view which are widely adopted in VR, AR and embodied intelligence applications. While multi-modal large language models (MLLMs) have demonstrated remarkable performance on conventional 2D image and video understanding benchmarks, their ability to comprehend the immersive environments captured by ODIs remains largely unexplored. To address this gap, we first present ODI-Bench, a novel comprehensive benchmark specifically designed for omnidirectional image understanding. ODI-Bench contains 2,000 high-quality omnidirectional images and over 4,000 manually annotated question-answering (QA) pairs across 10 fine-grained tasks, covering both general-level and spatial-level ODI understanding. Extensive experiments are conducted to benchmark 20 representative MLLMs, including proprietary and open-source models, under both close-ended and open-ended settings. Experimental results reveal that current MLLMs still struggle to capture the immersive context provided by ODIs. To this end, we further introduce Omni-CoT, a training-free method which significantly enhances MLLMs’ comprehension ability in the omnidirectional environment through chain-of-thought reasoning across both textual information and visual cues. Both the benchmark and the code will be released upon the publication.

Method: Systematically evaluate power-law fits to predict training size required for specific ROC-AUC thresholds. Test multiple pathologies and foundation models (XrayCLIP, XraySigLIP) against ResNet-50 baseline.

Result: XrayCLIP and XraySigLIP achieve strong performance with significantly fewer labeled examples than ResNet-50 baseline. Learning curve slopes from just 50 labeled cases accurately forecast final performance plateaus.

Conclusion: Results enable practitioners to minimize annotation costs by labeling only the essential samples needed for targeted performance levels.

Abstract: Chest X-ray classification is vital yet resource-intensive, typically demanding extensive annotated data for accurate diagnosis. Foundation models mitigate this reliance, but how many labeled samples are required remains unclear. We systematically evaluate the use of power-law fits to predict the training size necessary for specific ROC-AUC thresholds. Testing multiple pathologies and foundation models, we find XrayCLIP and XraySigLIP achieve strong performance with significantly fewer labeled examples than a ResNet-50 baseline. Importantly, learning curve slopes from just 50 labeled cases accurately forecast final performance plateaus. Our results enable practitioners to minimize annotation costs by labeling only the essential samples for targeted performance.

Method: Training on 7 diverse datasets with domain-adaptive normalization to prevent negative transfer; using automatically generated mask proposals matched against CLIP embeddings for text-prompted segmentation; supporting both panoptic and open-vocabulary segmentation.

Result: Achieves SOTA on 8/9 zero-shot benchmarks for spatial-prompted segmentation and competitive results on all 5 text-prompted benchmarks, demonstrating that unified models can match or exceed specialized domain-specific approaches.

Conclusion: SNAP provides a practical, scalable tool for 3D annotation by unifying multiple interaction modalities and domains in a single model that outperforms specialized approaches.

Abstract: Interactive 3D point cloud segmentation enables efficient annotation of complex 3D scenes through user-guided prompts. However, current approaches are typically restricted in scope to a single domain (indoor or outdoor), and to a single form of user interaction (either spatial clicks or textual prompts). Moreover, training on multiple datasets often leads to negative transfer, resulting in domain-specific tools that lack generalizability. To address these limitations, we present \textbf{SNAP} (\textbf{S}egment a\textbf{N}ything in \textbf{A}ny \textbf{P}oint cloud), a unified model for interactive 3D segmentation that supports both point-based and text-based prompts across diverse domains. Our approach achieves cross-domain generalizability by training on 7 datasets spanning indoor, outdoor, and aerial environments, while employing domain-adaptive normalization to prevent negative transfer. For text-prompted segmentation, we automatically generate mask proposals without human intervention and match them against CLIP embeddings of textual queries, enabling both panoptic and open-vocabulary segmentation. Extensive experiments demonstrate that SNAP consistently delivers high-quality segmentation results. We achieve state-of-the-art performance on 8 out of 9 zero-shot benchmarks for spatial-prompted segmentation and demonstrate competitive results on all 5 text-prompted benchmarks. These results show that a unified model can match or exceed specialized domain-specific approaches, providing a practical tool for scalable 3D annotation. Project page is at, https://neu-vi.github.io/SNAP/

Method: Adapts an off-the-shelf diffusion model to a target domain using only imperfect pseudo-labels. The method filters suboptimal generations, rectifies image-label misalignments, and standardizes semantics across datasets.

Result: Experiments on five synthetic datasets and two real target datasets show segmentation gains of up to +8.0%pt. mIoU over state-of-the-art translation methods, making rapidly constructed synthetic datasets as effective as high-effort ones.

Conclusion: Fast semantic prototyping combined with generative models enables scalable, high-quality training data creation for urban scene understanding, highlighting a valuable collaborative paradigm.

Abstract: Synthetic datasets are widely used for training urban scene recognition models, but even highly realistic renderings show a noticeable gap to real imagery. This gap is particularly pronounced when adapting to a specific target domain, such as Cityscapes, where differences in architecture, vegetation, object appearance, and camera characteristics limit downstream performance. Closing this gap with more detailed 3D modelling would require expensive asset and scene design, defeating the purpose of low-cost labelled data. To address this, we present a new framework that adapts an off-the-shelf diffusion model to a target domain using only imperfect pseudo-labels. Once trained, it generates high-fidelity, target-aligned images from semantic maps of any synthetic dataset, including low-effort sources created in hours rather than months. The method filters suboptimal generations, rectifies image-label misalignments, and standardises semantics across datasets, transforming weak synthetic data into competitive real-domain training sets. Experiments on five synthetic datasets and two real target datasets show segmentation gains of up to +8.0%pt. mIoU over state-of-the-art translation methods, making rapidly constructed synthetic datasets as effective as high-effort, time-intensive synthetic datasets requiring extensive manual design. This work highlights a valuable collaborative paradigm where fast semantic prototyping, combined with generative models, enables scalable, high-quality training data creation for urban scene understanding.

Method: Fine-tuned three foundation models on manually labeled hyperspectral data from training region and evaluated on independent test region using overall accuracy, average accuracy, and F1-score metrics.

Result: HyperSigma: 34.5% OA, DOFA: 62.6% OA, SpectralEarth: 93.5% OA. Compact SpectralEarth variant achieved 91% when trained from scratch.

Conclusion: SpectralEarth foundation model shows superior performance for operational hyperspectral crop mapping, with model architecture being crucial for strong cross-region and cross-sensor generalization.

Abstract: Foundation models are transforming Earth observation, but their potential for hyperspectral crop mapping remains underexplored. This study benchmarks three foundation models for cereal crop mapping using hyperspectral imagery: HyperSigma, DOFA, and Vision Transformers pre-trained on the SpectralEarth dataset (a large multitemporal hyperspectral archive). Models were fine-tuned on manually labeled data from a training region and evaluated on an independent test region. Performance was measured with overall accuracy (OA), average accuracy (AA), and F1-score. HyperSigma achieved an OA of 34.5% (+/- 1.8%), DOFA reached 62.6% (+/- 3.5%), and the SpectralEarth model achieved an OA of 93.5% (+/- 0.8%). A compact SpectralEarth variant trained from scratch achieved 91%, highlighting the importance of model architecture for strong generalization across geographic regions and sensor platforms. These results provide a systematic evaluation of foundation models for operational hyperspectral crop mapping and outline directions for future model development.

Method: EvoCAD samples multiple CAD objects using vision language models and optimizes them through evolutionary approach with vision and reasoning language models, evaluated on CADPrompt benchmark.

Result: EvoCAD outperforms previous approaches on multiple metrics, especially in generating topologically correct objects, with novel Euler characteristic-based metrics proving effective.

Conclusion: The combination of LLMs with evolutionary computation shows promise for CAD generation, with EvoCAD demonstrating superior performance in topological correctness using new evaluation metrics.

Abstract: Combining large language models with evolutionary computation algorithms represents a promising research direction leveraging the remarkable generative and in-context learning capabilities of LLMs with the strengths of evolutionary algorithms. In this work, we present EvoCAD, a method for generating computer-aided design (CAD) objects through their symbolic representations using vision language models and evolutionary optimization. Our method samples multiple CAD objects, which are then optimized using an evolutionary approach with vision language and reasoning language models. We assess our method using GPT-4V and GPT-4o, evaluating it on the CADPrompt benchmark dataset and comparing it to prior methods. Additionally, we introduce two new metrics based on topological properties defined by the Euler characteristic, which capture a form of semantic similarity between 3D objects. Our results demonstrate that EvoCAD outperforms previous approaches on multiple metrics, particularly in generating topologically correct objects, which can be efficiently evaluated using our two novel metrics that complement existing spatial metrics.

Method: MS-Mix framework includes: 1) Sentiment-Aware Sample Selection to prevent mixing contradictory emotions, 2) Sentiment Intensity Guided module using multi-head self-attention for dynamic modality-specific mixing ratios, 3) Sentiment Alignment Loss with KL-divergence regularization to align predictions across modalities.

Result: Extensive experiments on three benchmark datasets with six state-of-the-art backbones show MS-Mix consistently outperforms existing methods, establishing new standards for robust multimodal sentiment augmentation.

Conclusion: MS-Mix effectively addresses the limitations of traditional Mixup in multimodal sentiment analysis by incorporating emotion-aware mechanisms, leading to improved performance and generalization across various backbone models.

Abstract: Multimodal Sentiment Analysis (MSA) aims to identify and interpret human emotions by integrating information from heterogeneous data sources such as text, video, and audio. While deep learning models have advanced in network architecture design, they remain heavily limited by scarce multimodal annotated data. Although Mixup-based augmentation improves generalization in unimodal tasks, its direct application to MSA introduces critical challenges: random mixing often amplifies label ambiguity and semantic inconsistency due to the lack of emotion-aware mixing mechanisms. To overcome these issues, we propose MS-Mix, an adaptive, emotion-sensitive augmentation framework that automatically optimizes sample mixing in multimodal settings. The key components of MS-Mix include: (1) a Sentiment-Aware Sample Selection (SASS) strategy that effectively prevents semantic confusion caused by mixing samples with contradictory emotions. (2) a Sentiment Intensity Guided (SIG) module using multi-head self-attention to compute modality-specific mixing ratios dynamically based on their respective emotional intensities. (3) a Sentiment Alignment Loss (SAL) that aligns the prediction distributions across modalities, and incorporates the Kullback-Leibler-based loss as an additional regularization term to train the emotion intensity predictor and the backbone network jointly. Extensive experiments on three benchmark datasets with six state-of-the-art backbones confirm that MS-Mix consistently outperforms existing methods, establishing a new standard for robust multimodal sentiment augmentation. The source code is available at: https://github.com/HongyuZhu-s/MS-Mix.

Method: NV3D extracts local features using normal vectors computed per voxel basis via KNN and PCA. It offers two sampling strategies: normal vector density-based sampling and FOV-aware bin-based sampling. The model uses element-wise attention fusion with voxel features as query/value and normal vector features as key.

Result: On KITTI validation set, NV3D without sampling achieves 86.60% and 80.18% mAP for car and cyclist detection, outperforming baseline Voxel R-CNN by 2.61% and 4.23% respectively. With both samplings, it achieves 85.54% mAP for car detection (1.56% improvement) while filtering out ~55% of voxels.

Conclusion: NV3D demonstrates that normal vector features effectively capture surface relationships for 3D object detection, enabling significant data reduction while maintaining or improving performance, particularly benefiting detection of objects with distinct spatial shapes like cars and cyclists.

Abstract: Recent studies in 3D object detection for autonomous vehicles aim to enrich features through the utilization of multi-modal setups or the extraction of local patterns within LiDAR point clouds. However, multi-modal methods face significant challenges in feature alignment, and gaining features locally can be oversimplified for complex 3D object detection tasks. In this paper, we propose a novel model, NV3D, which utilizes local features acquired from voxel neighbors, as normal vectors computed per voxel basis using K-nearest neighbors (KNN) and principal component analysis (PCA). This informative feature enables NV3D to determine the relationship between the surface and pertinent target entities, including cars, pedestrians, or cyclists. During the normal vector extraction process, NV3D offers two distinct sampling strategies: normal vector density-based sampling and FOV-aware bin-based sampling, allowing elimination of up to 55% of data while maintaining performance. In addition, we applied element-wise attention fusion, which accepts voxel features as the query and value and normal vector features as the key, similar to the attention mechanism. Our method is trained on the KITTI dataset and has demonstrated superior performance in car and cyclist detection owing to their spatial shapes. In the validation set, NV3D without sampling achieves 86.60% and 80.18% mean Average Precision (mAP), greater than the baseline Voxel R-CNN by 2.61% and 4.23% mAP, respectively. With both samplings, NV3D achieves 85.54% mAP in car detection, exceeding the baseline by 1.56% mAP, despite roughly 55% of voxels being filtered out.

Method: Proposes ACE-G framework that separates coordinate regressor and map representation into a generic transformer and scene-specific map code. This allows pre-training the transformer on tens of thousands of scenes and training it to generalize from mapping images to unseen query images during pre-training.

Result: Demonstrates significantly increased robustness on multiple challenging relocalization datasets while maintaining attractive computational efficiency.

Conclusion: The separation of coordinate regression and map representation enables pre-training and improved generalization, overcoming the inherent overfitting limitation of previous SCR frameworks.

Abstract: Scene coordinate regression (SCR) has established itself as a promising learning-based approach to visual relocalization. After mere minutes of scene-specific training, SCR models estimate camera poses of query images with high accuracy. Still, SCR methods fall short of the generalization capabilities of more classical feature-matching approaches. When imaging conditions of query images, such as lighting or viewpoint, are too different from the training views, SCR models fail. Failing to generalize is an inherent limitation of previous SCR frameworks, since their training objective is to encode the training views in the weights of the coordinate regressor itself. The regressor essentially overfits to the training views, by design. We propose to separate the coordinate regressor and the map representation into a generic transformer and a scene-specific map code. This separation allows us to pre-train the transformer on tens of thousands of scenes. More importantly, it allows us to train the transformer to generalize from mapping images to unseen query images during pre-training. We demonstrate on multiple challenging relocalization datasets that our method, ACE-G, leads to significantly increased robustness while keeping the computational footprint attractive.

Method: Created ExpVid benchmark with three-level task hierarchy: Fine-grained Perception, Procedural Understanding, and Scientific Reasoning. Used vision-centric annotation pipeline combining automated generation with multi-disciplinary expert validation from peer-reviewed video publications.

Result: Evaluation of 19 leading MLLMs showed they excel at coarse-grained recognition but struggle with fine details, state tracking over time, and linking procedures to outcomes. Notable performance gap between proprietary and open-source models, especially in high-order reasoning.

Conclusion: ExpVid provides a diagnostic tool and roadmap for developing MLLMs capable of becoming trustworthy partners in scientific experimentation by identifying current limitations in fine-grained understanding and scientific reasoning capabilities.

Abstract: Multimodal Large Language Models (MLLMs) hold promise for accelerating scientific discovery by interpreting complex experimental procedures. However, their true capabilities are poorly understood, as existing benchmarks neglect the fine-grained and long-horizon nature of authentic laboratory work, especially in wet-lab settings. To bridge this gap, we introduce ExpVid, the first benchmark designed to systematically evaluate MLLMs on scientific experiment videos. Curated from peer-reviewed video publications, ExpVid features a new three-level task hierarchy that mirrors the scientific process: (1) Fine-grained Perception of tools, materials, and actions; (2) Procedural Understanding of step order and completeness; and (3) Scientific Reasoning that connects the full experiment to its published conclusions. Our vision-centric annotation pipeline, combining automated generation with multi-disciplinary expert validation, ensures that tasks require visual grounding. We evaluate 19 leading MLLMs on ExpVid and find that while they excel at coarse-grained recognition, they struggle with disambiguating fine details, tracking state changes over time, and linking experimental procedures to scientific outcomes. Our results reveal a notable performance gap between proprietary and open-source models, particularly in high-order reasoning. ExpVid not only provides a diagnostic tool but also charts a roadmap for developing MLLMs capable of becoming trustworthy partners in scientific experimentation.

Method: FACE augments Non-negative Matrix Factorization (NMF) with a Kullback-Leibler (KL) divergence regularization term and incorporates classifier supervision during concept learning to enforce predictive consistency.

Result: FACE outperforms existing methods across faithfulness and sparsity metrics on ImageNet, COCO, and CelebA datasets, with theoretical guarantees bounding predictive distribution deviations.

Conclusion: The proposed FACE framework successfully addresses the faithfulness issue in concept-based explanations by ensuring predictive consistency through KL divergence regularization and classifier supervision.

Abstract: Interpreting deep neural networks through concept-based explanations offers a bridge between low-level features and high-level human-understandable semantics. However, existing automatic concept discovery methods often fail to align these extracted concepts with the model’s true decision-making process, thereby compromising explanation faithfulness. In this work, we propose FACE (Faithful Automatic Concept Extraction), a novel framework that augments Non-negative Matrix Factorization (NMF) with a Kullback-Leibler (KL) divergence regularization term to ensure alignment between the model’s original and concept-based predictions. Unlike prior methods that operate solely on encoder activations, FACE incorporates classifier supervision during concept learning, enforcing predictive consistency and enabling faithful explanations. We provide theoretical guarantees showing that minimizing the KL divergence bounds the deviation in predictive distributions, thereby promoting faithful local linearity in the learned concept space. Systematic evaluations on ImageNet, COCO, and CelebA datasets demonstrate that FACE outperforms existing methods across faithfulness and sparsity metrics.

Method: Integrates global and local operators through closed-form Laplacian pyramid decomposition and reconstruction. Uses image-adaptive 3D LUT for global tonal enhancement with weight fusion strategies, and spatial-frequency transformer weight predictor to extract distinct weights. Applies local Laplacian filters to refine edge details in high-frequency components.

Result: Achieves 2.64 dB improvement in PSNR on HDR+ dataset, processes 4K resolution images in 13 ms on a single GPU, and performs favorably compared to state-of-the-art methods on benchmark datasets.

Conclusion: LLF-LUT++ successfully bridges the gap between performance and efficiency in photo enhancement, enabling fast processing of high-resolution images while maintaining excellent enhancement quality.

Abstract: Photo enhancement plays a crucial role in augmenting the visual aesthetics of a photograph. In recent years, photo enhancement methods have either focused on enhancement performance, producing powerful models that cannot be deployed on edge devices, or prioritized computational efficiency, resulting in inadequate performance for real-world applications. To this end, this paper introduces a pyramid network called LLF-LUT++, which integrates global and local operators through closed-form Laplacian pyramid decomposition and reconstruction. This approach enables fast processing of high-resolution images while also achieving excellent performance. Specifically, we utilize an image-adaptive 3D LUT that capitalizes on the global tonal characteristics of downsampled images, while incorporating two distinct weight fusion strategies to achieve coarse global image enhancement. To implement this strategy, we designed a spatial-frequency transformer weight predictor that effectively extracts the desired distinct weights by leveraging frequency features. Additionally, we apply local Laplacian filters to adaptively refine edge details in high-frequency components. After meticulously redesigning the network structure and transformer model, LLF-LUT++ not only achieves a 2.64 dB improvement in PSNR on the HDR+ dataset, but also further reduces runtime, with 4K resolution images processed in just 13 ms on a single GPU. Extensive experimental results on two benchmark datasets further show that the proposed approach performs favorably compared to state-of-the-art methods. The source code will be made publicly available at https://github.com/fengzhang427/LLF-LUT.

Method: IVEBench includes 600 high-quality source videos across 7 semantic dimensions, 8 categories of editing tasks with 35 subcategories, and establishes a three-dimensional evaluation protocol (video quality, instruction compliance, video fidelity) using both traditional metrics and multimodal LLM-based assessments.

Result: Extensive experiments demonstrate IVEBench’s effectiveness in benchmarking state-of-the-art instruction-guided video editing methods and its ability to provide comprehensive, human-aligned evaluation outcomes.

Conclusion: IVEBench successfully addresses the limitations of existing benchmarks and provides a modern, comprehensive evaluation suite for instruction-guided video editing methods.

Abstract: Instruction-guided video editing has emerged as a rapidly advancing research direction, offering new opportunities for intuitive content transformation while also posing significant challenges for systematic evaluation. Existing video editing benchmarks fail to support the evaluation of instruction-guided video editing adequately and further suffer from limited source diversity, narrow task coverage and incomplete evaluation metrics. To address the above limitations, we introduce IVEBench, a modern benchmark suite specifically designed for instruction-guided video editing assessment. IVEBench comprises a diverse database of 600 high-quality source videos, spanning seven semantic dimensions, and covering video lengths ranging from 32 to 1,024 frames. It further includes 8 categories of editing tasks with 35 subcategories, whose prompts are generated and refined through large language models and expert review. Crucially, IVEBench establishes a three-dimensional evaluation protocol encompassing video quality, instruction compliance and video fidelity, integrating both traditional metrics and multimodal large language model-based assessments. Extensive experiments demonstrate the effectiveness of IVEBench in benchmarking state-of-the-art instruction-guided video editing methods, showing its ability to provide comprehensive and human-aligned evaluation outcomes.

Method: PhySIC starts with coarse monocular depth and body estimates, performs occlusion-aware inpainting, fuses visible depth with unscaled geometry, synthesizes missing support surfaces, and uses confidence-weighted optimization to refine body pose, camera parameters, and global scale while enforcing depth alignment, contact priors, interpenetration avoidance, and 2D reprojection consistency.

Result: PhySIC reduces mean per-vertex scene error from 641 mm to 227 mm, halves PA-MPJPE to 42 mm, improves contact F1 from 0.09 to 0.51, and handles multiple humans efficiently (9s for optimization, 27s end-to-end).

Conclusion: PhySIC advances scalable 3D scene understanding by converting single images into physically plausible 3D human-scene pairs with realistic interactions and handling of occlusions.

Abstract: Reconstructing metrically accurate humans and their surrounding scenes from a single image is crucial for virtual reality, robotics, and comprehensive 3D scene understanding. However, existing methods struggle with depth ambiguity, occlusions, and physically inconsistent contacts. To address these challenges, we introduce PhySIC, a framework for physically plausible Human-Scene Interaction and Contact reconstruction. PhySIC recovers metrically consistent SMPL-X human meshes, dense scene surfaces, and vertex-level contact maps within a shared coordinate frame from a single RGB image. Starting from coarse monocular depth and body estimates, PhySIC performs occlusion-aware inpainting, fuses visible depth with unscaled geometry for a robust metric scaffold, and synthesizes missing support surfaces like floors. A confidence-weighted optimization refines body pose, camera parameters, and global scale by jointly enforcing depth alignment, contact priors, interpenetration avoidance, and 2D reprojection consistency. Explicit occlusion masking safeguards invisible regions against implausible configurations. PhySIC is efficient, requiring only 9 seconds for joint human-scene optimization and under 27 seconds end-to-end. It naturally handles multiple humans, enabling reconstruction of diverse interactions. Empirically, PhySIC outperforms single-image baselines, reducing mean per-vertex scene error from 641 mm to 227 mm, halving PA-MPJPE to 42 mm, and improving contact F1 from 0.09 to 0.51. Qualitative results show realistic foot-floor interactions, natural seating, and plausible reconstructions of heavily occluded furniture. By converting a single image into a physically plausible 3D human-scene pair, PhySIC advances scalable 3D scene understanding. Our implementation is publicly available at https://yuxuan-xue.com/physic.

Method: Proposes InfiniHumanData - an automatic pipeline using vision-language and image generation models to create large-scale multi-modal datasets, and InfiniHumanGen - a diffusion-based generative pipeline conditioned on text, body shape, and clothing assets.

Result: Generated 111K identities with unprecedented diversity, each annotated with multi-granularity text, multi-view RGB images, detailed clothing images, and SMPL parameters. User studies show generated identities are indistinguishable from scan renderings. Demonstrates significant improvements in visual quality, generation speed, and controllability over state-of-the-art methods.

Conclusion: The approach enables high-quality avatar generation with fine-grained control at effectively unbounded scale through a practical and affordable solution, with plans to publicly release the pipeline, dataset, and models.

Abstract: Generating realistic and controllable 3D human avatars is a long-standing challenge, particularly when covering broad attribute ranges such as ethnicity, age, clothing styles, and detailed body shapes. Capturing and annotating large-scale human datasets for training generative models is prohibitively expensive and limited in scale and diversity. The central question we address in this paper is: Can existing foundation models be distilled to generate theoretically unbounded, richly annotated 3D human data? We introduce InfiniHuman, a framework that synergistically distills these models to produce richly annotated human data at minimal cost and with theoretically unlimited scalability. We propose InfiniHumanData, a fully automatic pipeline that leverages vision-language and image generation models to create a large-scale multi-modal dataset. User study shows our automatically generated identities are undistinguishable from scan renderings. InfiniHumanData contains 111K identities spanning unprecedented diversity. Each identity is annotated with multi-granularity text descriptions, multi-view RGB images, detailed clothing images, and SMPL body-shape parameters. Building on this dataset, we propose InfiniHumanGen, a diffusion-based generative pipeline conditioned on text, body shape, and clothing assets. InfiniHumanGen enables fast, realistic, and precisely controllable avatar generation. Extensive experiments demonstrate significant improvements over state-of-the-art methods in visual quality, generation speed, and controllability. Our approach enables high-quality avatar generation with fine-grained control at effectively unbounded scale through a practical and affordable solution. We will publicly release the automatic data generation pipeline, the comprehensive InfiniHumanData dataset, and the InfiniHumanGen models at https://yuxuan-xue.com/infini-human.

Method: Fuses dense 2D features from vision foundation models with partial 3D point clouds using Transformer encoder enhanced by Mixture-of-Experts, with parallel decoders for pose-size estimation and shape reconstruction.

Result: Achieves state-of-the-art accuracy on seen categories and strong zero-shot generalization on unseen real-world objects across four benchmarks spanning 300+ categories, with real-time inference at 28 FPS.

Conclusion: Establishes a new standard for open-set 6D understanding in robotics and embodied AI, demonstrating effective category-agnostic performance trained solely on synthetic data.

Abstract: Estimating an object’s 6D pose, size, and shape from visual input is a fundamental problem in computer vision, with critical applications in robotic grasping and manipulation. Existing methods either rely on object-specific priors such as CAD models or templates, or suffer from limited generalization across categories due to pose-shape entanglement and multi-stage pipelines. In this work, we propose a unified, category-agnostic framework that simultaneously predicts 6D pose, size, and dense shape from a single RGB-D image, without requiring templates, CAD models, or category labels at test time. Our model fuses dense 2D features from vision foundation models with partial 3D point clouds using a Transformer encoder enhanced by a Mixture-of-Experts, and employs parallel decoders for pose-size estimation and shape reconstruction, achieving real-time inference at 28 FPS. Trained solely on synthetic data from 149 categories in the SOPE dataset, our framework is evaluated on four diverse benchmarks SOPE, ROPE, ObjaversePose, and HANDAL, spanning over 300 categories. It achieves state-of-the-art accuracy on seen categories while demonstrating remarkably strong zero-shot generalization to unseen real-world objects, establishing a new standard for open-set 6D understanding in robotics and embodied AI.

Method: 1) Construct Math-VR, a large-scale bilingual dataset for mathematics with visual reasoning; 2) Develop an image-to-code converter to parse mathematical figures into code; 3) Train CodePlot-CoT model using this data to generate both text reasoning and executable plotting code.

Result: The model achieves up to 21% improvement over base models on the new benchmark, validating the efficacy of the code-driven reasoning paradigm for multimodal mathematical reasoning.

Conclusion: This work opens a new direction for multimodal mathematical reasoning and provides the community with the first large-scale dataset, benchmark, and strong approach for such problems, with all resources made publicly available.

Abstract: Recent advances in Large Language Models (LLMs) and Vision Language Models (VLMs) have shown significant progress in mathematical reasoning, yet they still face a critical bottleneck with problems requiring visual assistance, such as drawing auxiliary lines or plotting functions to solve the problems. Most LLMs and VLMs are constrained to text-only reasoning chains, while multimodal unified models that can generate interleaved text and images lack the necessary precision and controllability for such tasks. To address this, we propose CodePlot-CoT, a code-driven Chain-of-Thought paradigm for “thinking with images” in mathematics. Our approach leverages the VLM to generate text reasoning as well as executable plotting code, which is then rendered into images as “visual thought”, to solve mathematical problems. To achieve this, we first construct Math-VR, the first large-scale, bilingual dataset and benchmark for Mathematics problems with Visual Reasoning, comprising 178K samples. Second, to create high-quality training data, we develop a state-of-the-art image-to-code converter specialized for parsing complex mathematical figures into codes. Finally, using these training data, we train the CodePlot-CoT model for solving mathematical problems. Experimental results show that our model achieves up to 21% increase over base model on our new benchmark, fully validating the efficacy of our proposed code-driven reasoning paradigm. Our work opens a new direction for multimodal mathematical reasoning and provides the community with the first large-scale dataset, comprehensive benchmark, and strong approach for such problems. To facilitate future research, we make our datasets, code, and pretrained models publicly available at https://github.com/HKU-MMLab/Math-VR-CodePlot-CoT.

Method: Replace VAE with pretrained representation encoders (DINO, SigLIP, MAE) paired with trained decoders to form RAEs, analyze challenges of high-dimensional latent spaces, propose theoretically motivated solutions, and use DiT variant with lightweight DDT head.

Result: Achieves faster convergence without auxiliary losses, strong image generation results: 1.51 FID at 256x256 (no guidance) and 1.13 FID at both 256x256 and 512x512 (with guidance).

Conclusion: RAE offers clear advantages and should be the new default for diffusion transformer training, providing both high-quality reconstructions and semantically rich latent spaces with scalable transformer-based architecture.

Abstract: Latent generative modeling, where a pretrained autoencoder maps pixels into a latent space for the diffusion process, has become the standard strategy for Diffusion Transformers (DiT); however, the autoencoder component has barely evolved. Most DiTs continue to rely on the original VAE encoder, which introduces several limitations: outdated backbones that compromise architectural simplicity, low-dimensional latent spaces that restrict information capacity, and weak representations that result from purely reconstruction-based training and ultimately limit generative quality. In this work, we explore replacing the VAE with pretrained representation encoders (e.g., DINO, SigLIP, MAE) paired with trained decoders, forming what we term Representation Autoencoders (RAEs). These models provide both high-quality reconstructions and semantically rich latent spaces, while allowing for a scalable transformer-based architecture. Since these latent spaces are typically high-dimensional, a key challenge is enabling diffusion transformers to operate effectively within them. We analyze the sources of this difficulty, propose theoretically motivated solutions, and validate them empirically. Our approach achieves faster convergence without auxiliary representation alignment losses. Using a DiT variant equipped with a lightweight, wide DDT head, we achieve strong image generation results on ImageNet: 1.51 FID at 256x256 (no guidance) and 1.13 at both 256x256 and 512x512 (with guidance). RAE offers clear advantages and should be the new default for diffusion transformer training.

Method: Proposes a Bayesian topological CNN with prior distributions on network parameters, posterior learning, and a consistency condition in the learning cost to modify priors.

Result: Superior performance over conventional CNNs, Bayesian neural networks, and topological CNNs, especially with limited or corrupted data, and better out-of-distribution detection.

Conclusion: The hybrid approach shows potential for more efficient and robust image classification through improved uncertainty quantification and data efficiency.

Abstract: Convolutional neural networks (CNNs) have been established as the main workhorse in image data processing; nonetheless, they require large amounts of data to train, often produce overconfident predictions, and frequently lack the ability to quantify the uncertainty of their predictions. To address these concerns, we propose a new Bayesian topological CNN that promotes a novel interplay between topology-aware learning and Bayesian sampling. Specifically, it utilizes information from important manifolds to accelerate training while reducing calibration error by placing prior distributions on network parameters and properly learning appropriate posteriors. One important contribution of our work is the inclusion of a consistency condition in the learning cost, which can effectively modify the prior distributions to improve the performance of our novel network architecture. We evaluate the model on benchmark image classification datasets and demonstrate its superiority over conventional CNNs, Bayesian neural networks (BNNs), and topological CNNs. In particular, we supply evidence that our method provides an advantage in situations where training data is limited or corrupted. Furthermore, we show that the new model allows for better uncertainty quantification than standard BNNs since it can more readily identify examples of out-of-distribution data on which it has not been trained. Our results highlight the potential of our novel hybrid approach for more efficient and robust image classification.

Method: DiT360 employs inter-domain transformation and intra-domain augmentation at both image level (cross-domain knowledge through perspective guidance and panoramic refinement) and token level (hybrid supervision with circular padding, yaw loss, and cube loss for boundary continuity, rotational robustness, and distortion awareness).

Result: Extensive experiments on text-to-panorama, inpainting, and outpainting tasks show improved boundary consistency and image fidelity across eleven quantitative metrics.

Conclusion: The proposed DiT360 framework effectively addresses panoramic image generation challenges through hybrid training and multi-level supervision, achieving superior performance in geometric fidelity and photorealism.

Abstract: In this work, we propose DiT360, a DiT-based framework that performs hybrid training on perspective and panoramic data for panoramic image generation. For the issues of maintaining geometric fidelity and photorealism in generation quality, we attribute the main reason to the lack of large-scale, high-quality, real-world panoramic data, where such a data-centric view differs from prior methods that focus on model design. Basically, DiT360 has several key modules for inter-domain transformation and intra-domain augmentation, applied at both the pre-VAE image level and the post-VAE token level. At the image level, we incorporate cross-domain knowledge through perspective image guidance and panoramic refinement, which enhance perceptual quality while regularizing diversity and photorealism. At the token level, hybrid supervision is applied across multiple modules, which include circular padding for boundary continuity, yaw loss for rotational robustness, and cube loss for distortion awareness. Extensive experiments on text-to-panorama, inpainting, and outpainting tasks demonstrate that our method achieves better boundary consistency and image fidelity across eleven quantitative metrics. Our code is available at https://github.com/Insta360-Research-Team/DiT360.

Method: Place a distinctively colored marker at query point, regenerate video from intermediate noise level to propagate marker across frames. Use unedited initial frame as negative prompt to maintain marker visibility.

Result: Emergent tracks outperform prior zero-shot methods, persist through occlusions, and achieve performance competitive with specialized self-supervised models.

Conclusion: Video diffusion models inherently possess tracking capabilities that can be unlocked through simple prompting, bridging the gap between motion analysis and synthesis.

Abstract: Trackers and video generators solve closely related problems: the former analyze motion, while the latter synthesize it. We show that this connection enables pretrained video diffusion models to perform zero-shot point tracking by simply prompting them to visually mark points as they move over time. We place a distinctively colored marker at the query point, then regenerate the rest of the video from an intermediate noise level. This propagates the marker across frames, tracing the point’s trajectory. To ensure that the marker remains visible in this counterfactual generation, despite such markers being unlikely in natural videos, we use the unedited initial frame as a negative prompt. Through experiments with multiple image-conditioned video diffusion models, we find that these “emergent” tracks outperform those of prior zero-shot methods and persist through occlusions, often obtaining performance that is competitive with specialized self-supervised models.

Method: Regresses a deformable 3D Gaussian Splatting representation using: 1) a loss relating model outputs with 2D event observation space, and 2) a coarse 3D deformation model trained from binary masks generated from events.

Result: Experimental comparisons on synthetic and real datasets show Ev4DGS is valid and outperforms multiple naive baselines applicable in this setting.

Conclusion: Ev4DGS successfully demonstrates novel view rendering of non-rigid objects from event streams only, overcoming previous limitations that required RGB inputs.

Abstract: Event cameras offer various advantages for novel view rendering compared to synchronously operating RGB cameras, and efficient event-based techniques supporting rigid scenes have been recently demonstrated in the literature. In the case of non-rigid objects, however, existing approaches additionally require sparse RGB inputs, which can be a substantial practical limitation; it remains unknown if similar models could be learned from event streams only. This paper sheds light on this challenging open question and introduces Ev4DGS, i.e., the first approach for novel view rendering of non-rigidly deforming objects in the explicit observation space (i.e., as RGB or greyscale images) from monocular event streams. Our method regresses a deformable 3D Gaussian Splatting representation through 1) a loss relating the outputs of the estimated model with the 2D event observation space, and 2) a coarse 3D deformation model trained from binary masks generated from events. We perform experimental comparisons on existing synthetic and newly recorded real datasets with non-rigid objects. The results demonstrate the validity of Ev4DGS and its superior performance compared to multiple naive baselines that can be applied in our setting. We will release our models and the datasets used in the evaluation for research purposes; see the project webpage: https://4dqv.mpi-inf.mpg.de/Ev4DGS/.

Method: Proposed Invariant Feature Learning (IFL) method that first discovers environments with divergent intra-class distributions from imperfect predictions, then learns invariant features across these environments. IFL serves as an improved feature backbone that can boost existing LT methods.

Result: Most class-wise long-tailed methods degenerate on the proposed ImageNet-GLT and MSCOCO-GLT benchmarks. IFL shows promising results as it can enhance all existing LT approaches including re-balance, augmentation, and ensemble methods.

Conclusion: GLT is a more comprehensive problem than traditional LT classification, and the proposed IFL method provides an effective baseline for addressing both class-wise and attribute-wise imbalances in datasets.

Abstract: Existing long-tailed classification (LT) methods only focus on tackling the class-wise imbalance that head classes have more samples than tail classes, but overlook the attribute-wise imbalance. In fact, even if the class is balanced, samples within each class may still be long-tailed due to the varying attributes. Note that the latter is fundamentally more ubiquitous and challenging than the former because attributes are not just implicit for most datasets, but also combinatorially complex, thus prohibitively expensive to be balanced. Therefore, we introduce a novel research problem: Generalized Long-Tailed classification (GLT), to jointly consider both kinds of imbalances. By “generalized”, we mean that a GLT method should naturally solve the traditional LT, but not vice versa. Not surprisingly, we find that most class-wise LT methods degenerate in our proposed two benchmarks: ImageNet-GLT and MSCOCO-GLT. We argue that it is because they over-emphasize the adjustment of class distribution while neglecting to learn attribute-invariant features. To this end, we propose an Invariant Feature Learning (IFL) method as the first strong baseline for GLT. IFL first discovers environments with divergent intra-class distributions from the imperfect predictions and then learns invariant features across them. Promisingly, as an improved feature backbone, IFL boosts all the LT line-up: one/two-stage re-balance, augmentation, and ensemble. Codes and benchmarks are available on Github: https://github.com/KaihuaTang/Generalized-Long-Tailed-Benchmarks.pytorch

Method: The method minimizes contrastive loss of intra-class sample similarity while ensuring this similarity remains invariant across all classes. It uses a simple re-weighting based classifier with the proposed context estimation approach.

Result: The approach achieves state-of-the-art performance on benchmarks with various context biases and domain gaps, demonstrating effectiveness across different challenging scenarios.

Conclusion: The paper successfully demonstrates that leveraging class invariance to context provides an effective alternative to traditional context annotation methods for OOD generalization, with theoretical justification and practical implementation.

Abstract: Out-Of-Distribution generalization (OOD) is all about learning invariance against environmental changes. If the context in every class is evenly distributed, OOD would be trivial because the context can be easily removed due to an underlying principle: class is invariant to context. However, collecting such a balanced dataset is impractical. Learning on imbalanced data makes the model bias to context and thus hurts OOD. Therefore, the key to OOD is context balance. We argue that the widely adopted assumption in prior work, the context bias can be directly annotated or estimated from biased class prediction, renders the context incomplete or even incorrect. In contrast, we point out the everoverlooked other side of the above principle: context is also invariant to class, which motivates us to consider the classes (which are already labeled) as the varying environments to resolve context bias (without context labels). We implement this idea by minimizing the contrastive loss of intra-class sample similarity while assuring this similarity to be invariant across all classes. On benchmarks with various context biases and domain gaps, we show that a simple re-weighting based classifier equipped with our context estimation achieves state-of-the-art performance. We provide the theoretical justifications in Appendix and codes on https://github.com/simpleshinobu/IRMCon.

Method: Develops the dot-cycle dichotomy, Structure-Before-Specificity principle, Context-Content Uncertainty Principle, and defines homological capacity as the topological dual of Shannon capacity.

Result: Shows that prediction requires invariance for generalization, explains order invariance through measure concentration, and links dynamical entropy to structural capacity across domains like visual binding and working memory.

Conclusion: Recasts inference, learning, and communication as topological stabilization - the formation, closure, and persistence of informational cycles that make prediction robust and scalable.

Abstract: We introduce \emph{Information Topology}: a framework that unifies information theory and algebraic topology by treating \emph{cycle closure} as the primitive operation of inference. The starting point is the \emph{dot-cycle dichotomy}, which separates pointwise, order-sensitive fluctuations (dots) from order-invariant, predictive structure (cycles). Algebraically, closure is the cancellation of boundaries ($\partial^2=0$), which converts transient histories into stable invariants. Building on this, we derive the \emph{Structure-Before-Specificity} (SbS) principle: stable information resides in nontrivial homology classes that persist under perturbations, while high-entropy contextual details act as scaffolds. The \emph{Context-Content Uncertainty Principle} (CCUP) quantifies this balance by decomposing uncertainty into contextual spread and content precision, showing why prediction requires invariance for generalization. Measure concentration onto residual invariant manifolds explains \emph{order invariance}: when mass collapses to a narrow tube around a closed cycle, reparameterizations of micro-steps leave predictive functionals unchanged. We then define \emph{homological capacity}, the topological dual of Shannon capacity, as the sustainable number of independent informational cycles supported by a system. This capacity links dynamical (KS) entropy to structural (homological) capacity and refines Euler characteristics from a net'' summary to a gross’’ count of persistent invariants. Finally, we illustrate the theory across three domains where \emph{more is different}: \textbf{visual binding}, \textbf{working memory}, and \textbf{access consciousness}. Together, these results recast inference, learning, and communication as \emph{topological stabilization}: the formation, closure, and persistence of informational cycles that make prediction robust and scalable.

Method: Uses a camouflage environment generator supervised by a camouflage distribution classifier to synthesize realistic camouflage images for dataset expansion.

Result: Outperforms state-of-the-art methods on three datasets (COD10k, CAMO, and CHAMELEON).

Conclusion: Provides an effective plug-and-play data generation module that introduces more diversity into camouflage datasets.

Abstract: Camouflaged objects that blend into natural scenes pose significant challenges for deep-learning models to detect and synthesize. While camouflaged object detection is a crucial task in computer vision with diverse real-world applications, this research topic has been constrained by limited data availability. We propose a framework for synthesizing camouflage data to enhance the detection of camouflaged objects in natural scenes. Our approach employs a generative model to produce realistic camouflage images, which can be used to train existing object detection models. Specifically, we use a camouflage environment generator supervised by a camouflage distribution classifier to synthesize the camouflage images, which are then fed into our generator to expand the dataset. Our framework outperforms the current state-of-the-art method on three datasets (COD10k, CAMO, and CHAMELEON), demonstrating its effectiveness in improving camouflaged object detection. This approach can serve as a plug-and-play data generation and augmentation module for existing camouflaged object detection tasks and provides a novel way to introduce more diversity and distributions into current camouflage datasets.

Method: Constructs multiscale hypergraphs of varying group sizes with spectral hypergraph convolution based on random-walk probabilities, combined with spatial-temporal transformer for pairwise interactions, fused via multimodal transformer.

Result: Extensive experiments on public pedestrian datasets show Hyper-STTN consistently outperforms state-of-the-art baselines and ablation models.

Conclusion: The proposed Hyper-STTN effectively addresses the challenges of modeling complex crowd behavior by integrating both pairwise and groupwise interactions through hypergraph and transformer architectures.

Abstract: Predicting crowd intentions and trajectories is critical for a range of real-world applications, involving social robotics and autonomous driving. Accurately modeling such behavior remains challenging due to the complexity of pairwise spatial-temporal interactions and the heterogeneous influence of groupwise dynamics. To address these challenges, we propose Hyper-STTN, a Hypergraph-based Spatial-Temporal Transformer Network for crowd trajectory prediction. Hyper-STTN constructs multiscale hypergraphs of varying group sizes to model groupwise correlations, captured through spectral hypergraph convolution based on random-walk probabilities. In parallel, a spatial-temporal transformer is employed to learn pedestrians’ pairwise latent interactions across spatial and temporal dimensions. These heterogeneous groupwise and pairwise features are subsequently fused and aligned via a multimodal transformer. Extensive experiments on public pedestrian motion datasets demonstrate that Hyper-STTN consistently outperforms state-of-the-art baselines and ablation models.

Method: Identifies watermark representations orthogonal to semantic content in latent space and uses additive fusion strategy to embed watermarks during denoising process without modifying LDM components.

Result: Effectively balances image quality and watermark recovery rate, generalizes to multiple LDM versions and variants without retraining, and performs robustly under various attacks.

Conclusion: MarkPlugger provides a cost-effective, generalizable solution for watermarking AI-generated content that maintains image quality while ensuring reliable watermark detection.

Abstract: Today, the family of latent diffusion models (LDMs) has gained prominence for its high quality outputs and scalability. This has also raised security concerns on social media, as malicious users can create and disseminate harmful content. Existing approaches typically involve training specific components or entire generative models to embed a watermark in generated images for traceability and responsibility. However, in the fast-evolving era of AI-generated content (AIGC), the rapid iteration and modification of LDMs makes retraining with watermark models costly. To address the problem, we propose MarkPlugger, a generalizable plug-and-play watermark framework without LDM retraining. In particular, to reduce the disturbance of the watermark on the semantics of the generated image, we try to identify a watermark representation that is approaching orthogonal to the semantic in latent space, and apply an additive fusion strategy for the watermark and the semantic. Without modifying any components of the LDMs, we embed diverse watermarks in latent space, adapting to the denoising process. Our experimental findings reveal that our method effectively harmonizes image quality and watermark recovery rate. We also have validated that our method is generalized to multiple official versions and modified variants of LDMs, even without retraining the watermark model. Furthermore, it performs robustly under various attacks of different intensities.

Method: Uses attention-driven CNN trained on pyramid-extracted fragments from word images to capture multi-level features, enhancing representation learning.

Result: Demonstrates proficiency on three benchmark databases, showing strong performance particularly when handwriting data is limited.

Conclusion: The attention-based fragment approach enables effective writer identification even with constrained handwriting samples, outperforming traditional methods.

Abstract: Writer identification due to its widespread application in various fields has gained popularity over the years. In scenarios where optimum handwriting samples are available, whether they be in the form of a single line, a sentence, or an entire page, writer identification algorithms have demonstrated noteworthy levels of accuracy. However, in scenarios where only a limited number of handwritten samples are available, particularly in the form of word images, there is a significant scope for improvement. In this paper, we propose a writer identification system based on an attention-driven Convolutional Neural Network (CNN). The system is trained utilizing image segments, known as fragments, extracted from word images, employing a pyramid-based strategy. This methodology enables the system to capture a comprehensive representation of the data, encompassing both fine-grained details and coarse features across various levels of abstraction. These extracted fragments serve as the training data for the convolutional network, enabling it to learn a more robust representation compared to traditional convolution-based networks trained on word images. Additionally, the paper explores the integration of an attention mechanism to enhance the representational power of the learned features. The efficacy of the proposed algorithm is evaluated on three benchmark databases, demonstrating its proficiency in writer identification tasks, particularly in scenarios with limited access to handwriting data.

Method: Proposes Cross-view Instance Activation (CIA) to activate instance queries across surround-view images for recovering instance attributes, and Dual-view Point Embedding (DPE) that fuses features from both views to generate input-aware positional embeddings for better point coordinate estimation.

Result: Achieves strong performance on nuScenes (67.1 mAP) and Argoverse 2 (70.5 mAP) benchmarks, demonstrating consistent improvements across datasets.

Conclusion: PerCMap effectively addresses key limitations in HD map construction by leveraging perspective-view features at multiple levels, resulting in more accurate and complete map vector predictions.

Abstract: The construction of vectorized High-Definition (HD) maps from onboard surround-view cameras has become a significant focus in autonomous driving. However, current map vector estimation pipelines face two key limitations: input-agnostic queries struggle to capture complex map structures, and the view transformation leads to information loss. These issues often result in inaccurate shape restoration or missing instances in map predictions. To address this concern, we propose a novel approach, namely \textbf{PerCMap}, which explicitly exploits clues from perspective-view features at both instance and point level. Specifically, at instance level, we propose Cross-view Instance Activation (CIA) to activate instance queries across surround-view images, thereby helping the model recover the instance attributes of map vectors. At point level, we design Dual-view Point Embedding (DPE), which fuses features from both views to generate input-aware positional embeddings and improve the accuracy of point coordinate estimation. Extensive experiments on \textit{nuScenes} and \textit{Argoverse 2} demonstrate that PerCMap achieves strong and consistent performance across benchmarks, reaching 67.1 and 70.5 mAP, respectively.

Method: Proposes UniRGB-IR with three components: ViT foundation model, Multi-modal Feature Pool (MFP) module, and Supplementary Feature Injector (SFI) module. The MFP and SFI work as adapters to complement ViT features with contextual multi-scale features. The foundation model is frozen during training, only MFP and SFI are optimized.

Result: Experimental results on various RGB-IR semantic tasks demonstrate state-of-the-art performance using ViT-Base as the foundation model.

Conclusion: The proposed UniRGB-IR framework effectively addresses scalability and generalization limitations in RGB-IR semantic analysis by introducing adapter mechanisms that incorporate multi-modal features without fine-tuning the entire foundation model.

Abstract: Semantic analysis on visible (RGB) and infrared (IR) images has gained significant attention due to their enhanced accuracy and robustness under challenging conditions including low-illumination and adverse weather. However, due to the lack of pre-trained foundation models on the large-scale infrared image datasets, existing methods prefer to design task-specific frameworks and directly fine-tune them with pre-trained foundation models on their RGB-IR semantic relevance datasets, which results in poor scalability and limited generalization. To address these limitations, we propose UniRGB-IR, a scalable and efficient framework for RGB-IR semantic tasks that introduces a novel adapter mechanism to effectively incorporate rich multi-modal features into pre-trained RGB-based foundation models. Our framework comprises three key components: a vision transformer (ViT) foundation model, a Multi-modal Feature Pool (MFP) module, and a Supplementary Feature Injector (SFI) module. The MFP and SFI modules cooperate with each other as an adpater to effectively complement the ViT features with the contextual multi-scale features. During training process, we freeze the entire foundation model to inherit prior knowledge and only optimize the MFP and SFI modules. Furthermore, to verify the effectiveness of our framework, we utilize the ViT-Base as the pre-trained foundation model to perform extensive experiments. Experimental results on various RGB-IR semantic tasks demonstrate that our method can achieve state-of-the-art performance.

Method: LDB defines layers as self-contained parameter sets guiding generation, uses intermediate latent caching to reduce edits to few denoising steps, and implements familiar layer concepts in an editor.

Result: LDB achieves superior speed (140ms per edit) with comparable or improved image quality, background preservation, and edit fidelity compared to state-of-the-art methods.

Conclusion: LDB significantly enhances creative workflows with intuitive, efficient diffusion-based editing and has potential for expansion into video editing and related domains.

Abstract: Denoising diffusion models have emerged as powerful tools for image manipulation, yet interactive, localized editing workflows remain underdeveloped. We introduce Layered Diffusion Brushes (LDB), a novel training-free framework that enables interactive, layer-based editing using standard diffusion models. LDB defines each “layer” as a self-contained set of parameters guiding the generative process, enabling independent, non-destructive, and fine-grained prompt-guided edits, even in overlapping regions. LDB leverages a unique intermediate latent caching approach to reduce each edit to only a few denoising steps, achieving 140~ms per edit on consumer GPUs. An editor implementing LDB, incorporating familiar layer concepts, was evaluated via user study and quantitative metrics. Results demonstrate LDB’s superior speed alongside comparable or improved image quality, background preservation, and edit fidelity relative to state-of-the-art methods across various sequential image manipulation tasks. The findings highlight LDB’s ability to significantly enhance creative workflows by providing an intuitive and efficient approach to diffusion-based image editing and its potential for expansion into related subdomains, such as video editing.

Method: Proposes RATLIP model that replaces CAT with recurrent affine transformations (RAT) to enable global information sharing across layers, adds shuffle attention between RAT to mitigate information forgetting, and integrates CLIP pre-trained model in both generator and discriminator for better multimodal understanding.

Result: Extensive experiments on CUB, Oxford, and CelebA-tiny datasets demonstrate superior performance over state-of-the-art models in text-to-image synthesis quality and image-text consistency.

Conclusion: The proposed RATLIP model effectively addresses limitations of traditional GANs by enabling global information sharing through RAT and leveraging CLIP’s multimodal understanding capabilities, achieving improved text-to-image synthesis performance.

Abstract: Synthesizing high-quality photorealistic images with textual descriptions as a condition is very challenging. Generative Adversarial Networks (GANs), the classical model for this task, frequently suffer from low consistency between image and text descriptions and insufficient richness in synthesized images. Recently, conditional affine transformations (CAT), such as conditional batch normalization and instance normalization, have been applied to different layers of GAN to control content synthesis in images. CAT is a multi-layer perceptron that independently predicts data based on batch statistics between neighboring layers, with global textual information unavailable to other layers. To address this issue, we first model CAT and a recurrent neural network (RAT) to ensure that different layers can access global information. We then introduce shuffle attention between RAT to mitigate the characteristic of information forgetting in recurrent neural networks. Moreover, both our generator and discriminator utilize the powerful pre-trained model, Clip, which has been extensively employed for establishing associations between text and images through the learning of multimodal representations in latent space. The discriminator utilizes CLIP’s ability to comprehend complex scenes to accurately assess the quality of the generated images. Extensive experiments have been conducted on the CUB, Oxford, and CelebA-tiny datasets to demonstrate the superiority of the proposed model over current state-of-the-art models. The code is https://github.com/OxygenLu/RATLIP.

Method: Leverages feature space distance metrics to combine AL and OOD detection capabilities in a unified framework called SISOM.

Result: Achieved first place in two OpenOOD benchmarks and second place in the remaining one; delivered top-1 performance in three AL benchmarks, outperforming other methods.

Conclusion: SISOM effectively unifies AL and OOD detection, demonstrating that solving both tasks together leads to superior performance compared to treating them separately.

Abstract: When applying deep learning models in open-world scenarios, active learning (AL) strategies are crucial for identifying label candidates from a nearly infinite amount of unlabeled data. In this context, robust out-of-distribution (OOD) detection mechanisms are essential for handling data outside the target distribution of the application. However, current works investigate both problems separately. In this work, we introduce SISOM as the first unified solution for both AL and OOD detection. By leveraging feature space distance metrics SISOM combines the strengths of the currently independent tasks to solve both effectively. We conduct extensive experiments showing the problems arising when migrating between both tasks. In these evaluations SISOM underlined its effectiveness by achieving first place in two of the widely used OpenOOD benchmarks and second place in the remaining one. In AL, SISOM outperforms others and delivers top-1 performance in three benchmarks

Method: Extracts multiple patches from each image, identifies the most visually salient region as positive sample for contrastive learning, treats other patches as intra-class negatives, and patches from different images as inter-class negatives. Also introduces mutual learning strategy.

Result: Achieved significant performance gains across eight benchmark datasets, with PLCC of 0.942 on TID2013 (vs 0.908 baseline) and 0.977 on CSIQ (vs 0.965 baseline).

Conclusion: LML-IQA effectively addresses the limitation of traditional IQA methods by leveraging local manifold structures and contrastive learning, demonstrating superior performance over state-of-the-art methods.

Abstract: Image Quality Assessment (IQA) methods typically overlook local manifold structures, leading to compromised discriminative capabilities in perceptual quality evaluation. To address this limitation, we present LML-IQA, an innovative no-reference IQA (NR-IQA) approach that leverages a combination of local manifold learning and contrastive learning. Our approach first extracts multiple patches from each image and identifies the most visually salient region. This salient patch serves as a positive sample for contrastive learning, while other patches from the same image are treated as intra-class negatives to preserve local distinctiveness. Patches from different images also act as inter-class negatives to enhance feature separation. Additionally, we introduce a mutual learning strategy to improve the model’s ability to recognize and prioritize visually important regions. Comprehensive experiments across eight benchmark datasets demonstrate significant performance gains over state-of-the-art methods, achieving a PLCC of 0.942 on TID2013 (compared to 0.908) and 0.977 on CSIQ (compared to 0.965).

Method: An end-to-end trainable architecture that learns to prompt LLMs to generate soft attributes for CLIP text-encoder, integrates temporal modeling into CLIP’s vision encoder, and uses novel regularized finetuning for video domain adaptation.

Result: Extensive experimentation shows the efficacy of the approach on multiple benchmark datasets.

Conclusion: The proposed method successfully extends VLMs to open vocabulary multilabel video classification, addressing the limitations of previous single-label approaches.

Abstract: Pre-trained vision-language models (VLMs) have enabled significant progress in open vocabulary computer vision tasks such as image classification, object detection and image segmentation. Some recent works have focused on extending VLMs to open vocabulary single label action classification in videos. However, previous methods fall short in holistic video understanding which requires the ability to simultaneously recognize multiple actions and entities e.g., objects in the video in an open vocabulary setting. We formulate this problem as open vocabulary multilabel video classification and propose a method to adapt a pre-trained VLM such as CLIP to solve this task. We leverage large language models (LLMs) to provide semantic guidance to the VLM about class labels to improve its open vocabulary performance with two key contributions. First, we propose an end-to-end trainable architecture that learns to prompt an LLM to generate soft attributes for the CLIP text-encoder to enable it to recognize novel classes. Second, we integrate a temporal modeling module into CLIP’s vision encoder to effectively model the spatio-temporal dynamics of video concepts as well as propose a novel regularized finetuning technique to ensure strong open vocabulary classification performance in the video domain. Our extensive experimentation showcases the efficacy of our approach on multiple benchmark datasets.

Method: Created MINIONS dataset with over 5M frames, 400 minutes of IMU signals and RGB videos labeled with motion data. Proposed SparseNet framework to capture motion from IMUs and videos by discovering their supplementary features.

Result: The framework demonstrates the unique advantages of combining inertial and vision sensors, showing promise for consumer-affordable multi-modal motion capture.

Conclusion: MINIONS provides a valuable resource for research and development in accessible motion capture, enabling further exploration of consumer-grade solutions using minimal equipment.

Abstract: Human motion capture is the foundation for many computer vision and graphics tasks. While industrial motion capture systems with complex camera arrays or expensive wearable sensors have been widely adopted in movie and game production, consumer-affordable and easy-to-use solutions for personal applications are still far from mature. To utilize a mixture of a monocular camera and very few inertial measurement units (IMUs) for accurate multi-modal human motion capture in daily life, we contribute MINIONS in this paper, a large-scale Motion capture dataset collected from INertial and visION Sensors. MINIONS has several featured properties: 1) large scale of over five million frames and 400 minutes duration; 2) multi-modality data of IMUs signals and RGB videos labeled with joint positions, joint rotations, SMPL parameters, etc.; 3) a diverse set of 146 fine-grained single and interactive actions with textual descriptions. With the proposed MINIONS dataset, we propose a SparseNet framework to capture human motion from IMUs and videos by discovering their supplementary features and exploring the possibilities of consumer-affordable motion capture using a monocular camera and very few IMUs. The experiment results emphasize the unique advantages of inertial and vision sensors, showcasing the promise of consumer-affordable multi-modal motion capture and providing a valuable resource for further research and development.

Method: Used deep learning methods to analyze 2,674 inpatient CT scans, comparing imaging phenotypes from opportunistic CT with radiology reports and ICD coding documentation.

Result: Only 0.5% of sarcopenia, 3.2% of hepatic steatosis, and 30.7% of ascites cases identified through imaging or reports were properly ICD-coded, showing major documentation gaps.

Conclusion: Opportunistic CT has significant potential to improve diagnostic precision and accuracy of risk adjustment models in precision medicine.

Abstract: Abdominal computed tomography (CT) scans are frequently performed in clinical settings. Opportunistic CT involves repurposing routine CT images to extract diagnostic information and is an emerging tool for detecting underdiagnosed conditions such as sarcopenia, hepatic steatosis, and ascites. This study utilizes deep learning methods to promote accurate diagnosis and clinical documentation. We analyze 2,674 inpatient CT scans to identify discrepancies between imaging phenotypes (characteristics derived from opportunistic CT scans) and their corresponding documentation in radiology reports and ICD coding. Through our analysis, we find that only 0.5%, 3.2%, and 30.7% of scans diagnosed with sarcopenia, hepatic steatosis, and ascites (respectively) through either opportunistic imaging or radiology reports were ICD-coded. Our findings demonstrate opportunistic CT’s potential to enhance diagnostic precision and accuracy of risk adjustment models, offering advancements in precision medicine.

Method: Uses differentiable laser beam splatting with range-view representation for precise surface projection, Neural Gaussian Representation for view-dependent properties, and dynamic instances decomposition for handling complex scenes.

Result: Achieves state-of-the-art results in both rendering frame rate and quality on large public scene datasets, successfully re-simulating depth, intensity, and ray-drop channels simultaneously.

Conclusion: LiDAR-GS demonstrates superior performance compared to explicit mesh or implicit NeRF methods, providing an effective solution for real-time LiDAR scan re-simulation with high fidelity.

Abstract: We present LiDAR-GS, a Gaussian Splatting (GS) method for real-time, high-fidelity re-simulation of LiDAR scans in public urban road scenes. Recent GS methods proposed for cameras have achieved significant advancements in real-time rendering beyond Neural Radiance Fields (NeRF). However, applying GS representation to LiDAR, an active 3D sensor type, poses several challenges that must be addressed to preserve high accuracy and unique characteristics. Specifically, LiDAR-GS designs a differentiable laser beam splatting, using range-view representation for precise surface splatting by projecting lasers onto micro cross-sections, effectively eliminating artifacts associated with local affine approximations. Furthermore, LiDAR-GS leverages Neural Gaussian Representation, which further integrate view-dependent clues, to represent key LiDAR properties that are influenced by the incident direction and external factors. Combining these practices with some essential adaptations, e.g., dynamic instances decomposition, LiDAR-GS succeeds in simultaneously re-simulating depth, intensity, and ray-drop channels, achieving state-of-the-art results in both rendering frame rate and quality on publically available large scene datasets when compared with the methods using explicit mesh or implicit NeRF. Our source code is publicly available at https://www.github.com/cqf7419/LiDAR-GS.

Method: Proposed ConOVS method based on Mixture-of-Experts framework that dynamically combines expert decoders according to the probability that input samples belong to the distribution of each incremental dataset.

Result: ConOVS consistently outperforms existing methods across pre-training, incremental, and zero-shot test datasets, effectively expanding recognition capabilities when data is collected sequentially.

Conclusion: ConOVS successfully addresses the challenge of continual learning in Open-Vocabulary Segmentation, enabling models to effectively learn from sequentially collected data while maintaining performance across various test scenarios.

Abstract: Open-Vocabulary Segmentation (OVS) aims to segment classes that are not present in the training dataset. However, most existing studies assume that the training data is fixed in advance, overlooking more practical scenarios where new datasets are continuously collected over time. To address this, we first analyze how existing OVS models perform under such conditions. In this context, we explore several approaches such as retraining, fine-tuning, and continual learning but find that each of them has clear limitations. To address these issues, we propose ConOVS, a novel continual learning method based on a Mixture-of-Experts framework. ConOVS dynamically combines expert decoders based on the probability that an input sample belongs to the distribution of each incremental dataset. Through extensive experiments, we show that ConOVS consistently outperforms existing methods across pre-training, incremental, and zero-shot test datasets, effectively expanding the recognition capabilities of OVS models when data is collected sequentially.

Method: Proposes DiffusionGS, a novel single-stage 3D diffusion model that outputs 3D Gaussian point clouds at each timestep, using a scene-object mixed training strategy to scale up 3D training data.

Result: Achieves improvements of 2.20 dB/23.25 and 1.34 dB/19.16 in PSNR/FID for objects and scenes compared to state-of-the-art methods, with over 5x faster speed (~6s on A100 GPU) and no depth estimator required.

Conclusion: DiffusionGS provides robust 3D generation from single views with view consistency, handles arbitrary view directions, and demonstrates superior performance and efficiency for both object generation and scene reconstruction tasks.

Abstract: Existing feedforward image-to-3D methods mainly rely on 2D multi-view diffusion models that cannot guarantee 3D consistency. These methods easily collapse when changing the prompt view direction and mainly handle object-centric cases. In this paper, we propose a novel single-stage 3D diffusion model, DiffusionGS, for object generation and scene reconstruction from a single view. DiffusionGS directly outputs 3D Gaussian point clouds at each timestep to enforce view consistency and allow the model to generate robustly given prompt views of any directions, beyond object-centric inputs. Plus, to improve the capability and generality of DiffusionGS, we scale up 3D training data by developing a scene-object mixed training strategy. Experiments show that DiffusionGS yields improvements of 2.20 dB/23.25 and 1.34 dB/19.16 in PSNR/FID for objects and scenes than the state-of-the-art methods, without depth estimator. Plus, our method enjoys over 5$\times$ faster speed ($\sim$6s on an A100 GPU). Our Project page at https://caiyuanhao1998.github.io/project/DiffusionGS/ shows the video and interactive results. The code and models are publicly available at https://github.com/caiyuanhao1998/Open-DiffusionGS

Method: Systematic categorization through structural perspectives (data-level, feature-level, output-level fusion) and methodological paradigms (statistical, kernel-based, graphical, generative, contrastive, attention-based, LLM-based methods) based on extensive review of 260+ studies.

Result: Provides a comprehensive framework for understanding multimodal alignment and fusion, identifies critical challenges (cross-modal misalignment, computational bottlenecks, data quality, modality gap), and explores applications across social media analysis, medical imaging, emotion recognition, and embodied AI.

Conclusion: The survey guides future research toward optimizing multimodal learning systems for improved scalability, robustness, and generalizability across diverse domains by providing structured insights into alignment and fusion techniques.

Abstract: This survey provides a comprehensive overview of recent advances in multimodal alignment and fusion within the field of machine learning, driven by the increasing availability and diversity of data modalities such as text, images, audio, and video. Unlike previous surveys that often focus on specific modalities or limited fusion strategies, our work presents a structure-centric and method-driven framework that emphasizes generalizable techniques. We systematically categorize and analyze key approaches to alignment and fusion through both structural perspectives – data-level, feature-level, and output-level fusion – and methodological paradigms – including statistical, kernel-based, graphical, generative, contrastive, attention-based, and large language model (LLM)-based methods, drawing insights from an extensive review of over 260 relevant studies. Furthermore, this survey highlights critical challenges such as cross-modal misalignment, computational bottlenecks, data quality issues, and the modality gap, along with recent efforts to address them. Applications ranging from social media analysis and medical imaging to emotion recognition and embodied AI are explored to illustrate the real-world impact of robust multimodal systems. The insights provided aim to guide future research toward optimizing multimodal learning systems for improved scalability, robustness, and generalizability across diverse domains.

Method: Define part-based image hierarchies using object-level annotations, then enforce the hierarchy using contrastive loss with pairwise entailment metrics in hyperbolic space.

Result: Experiments show significant improvements in hierarchical image retrieval tasks, demonstrating the model’s capability to capture complex visual hierarchies.

Conclusion: The approach successfully encodes complex visual hierarchies that transcend mere visual similarity, capturing semantic and structural information for improved hierarchical understanding.

Abstract: Structuring latent representations in a hierarchical manner enables models to learn patterns at multiple levels of abstraction. However, most prevalent image understanding models focus on visual similarity, and learning visual hierarchies is relatively unexplored. In this work, for the first time, we introduce a learning paradigm that can encode user-defined multi-level complex visual hierarchies in hyperbolic space without requiring explicit hierarchical labels. As a concrete example, first, we define a part-based image hierarchy using object-level annotations within and across images. Then, we introduce an approach to enforce the hierarchy using contrastive loss with pairwise entailment metrics. Finally, we discuss new evaluation metrics to effectively measure hierarchical image retrieval. Encoding these complex relationships ensures that the learned representations capture semantic and structural information that transcends mere visual similarity. Experiments in part-based image retrieval show significant improvements in hierarchical retrieval tasks, demonstrating the capability of our model in capturing visual hierarchies.

Method: FairDD synchronously matches synthetic datasets to protected attribute-wise groups of original datasets rather than indiscriminate alignment to whole distributions, preventing collapse into majority groups and enabling balanced generation across all groups.

Result: FairDD significantly improves fairness compared to vanilla dataset distillation methods while maintaining accuracy, achieving a promising trade-off between fairness and accuracy across diverse matching-based approaches.

Conclusion: FairDD establishes itself as a versatile fair dataset distillation approach that effectively regularizes vanilla methods to favor balanced generation toward minority groups, with consistent superiority across Distribution and Gradient Matching methods.

Abstract: Condensing large datasets into smaller synthetic counterparts has demonstrated its promise for image classification. However, previous research has overlooked a crucial concern in image recognition: ensuring that models trained on condensed datasets are unbiased towards protected attributes (PA), such as gender and race. Our investigation reveals that dataset distillation fails to alleviate the unfairness towards minority groups within original datasets. Moreover, this bias typically worsens in the condensed datasets due to their smaller size. To bridge the research gap, we propose a novel fair dataset distillation (FDD) framework, namely FairDD, which can be seamlessly applied to diverse matching-based DD approaches (DDs), requiring no modifications to their original architectures. The key innovation of FairDD lies in synchronously matching synthetic datasets to PA-wise groups of original datasets, rather than indiscriminate alignment to the whole distributions in vanilla DDs, dominated by majority groups. This synchronized matching allows synthetic datasets to avoid collapsing into majority groups and bootstrap their balanced generation to all PA groups. Consequently, FairDD could effectively regularize vanilla DDs to favor biased generation toward minority groups while maintaining the accuracy of target attributes. Theoretical analyses and extensive experimental evaluations demonstrate that FairDD significantly improves fairness compared to vanilla DDs, with a promising trade-off between fairness and accuracy. Its consistent superiority across diverse DDs, spanning Distribution and Gradient Matching, establishes it as a versatile FDD approach. Code is available at https://github.com/zqhang/FairDD.

Method: The authors propose Selective Aggregation to better capture semantic information in patch tokens, addressing the issue that attention in MIMs is spread uniformly over many patches, leading to ineffective [cls] token aggregation.

Result: Selective Aggregation significantly improves the out-of-the-box performance of MIM models.

Conclusion: The poor out-of-the-box performance of MIMs is not due to weaker features but rather suboptimal usage, specifically ineffective attention aggregation, which can be addressed through Selective Aggregation.

Abstract: Masked Image Modeling (MIM) has emerged as a promising approach for Self-Supervised Learning (SSL) of visual representations. However, the out-of-the-box performance of MIMs is typically inferior to competing approaches. Most users cannot afford fine-tuning due to the need for large amounts of data, high GPU consumption, and specialized user knowledge. Therefore, the practical use of MIM representations is limited. In this paper we ask what is the reason for the poor out-of-the-box performance of MIMs. Is it due to weaker features produced by MIM models, or is it due to suboptimal usage? Through detailed analysis, we show that attention in MIMs is spread almost uniformly over many patches, leading to ineffective aggregation by the [cls] token. Based on this insight, we propose Selective Aggregation to better capture the rich semantic information retained in patch tokens, which significantly improves the out-of-the-box performance of MIM.

Method: Uses routes as transport mechanisms for randomly selected tokens from early layers to deeper layers, applicable to both transformer-based and state-space models without architectural modifications.

Result: 14x convergence speedup at 400K iterations vs DiT, 37x vs DiT’s best performance at 7M iterations. Achieved FID of 2.09 (guided) and 3.93 (unguided) on ImageNet-256, improving upon DiT.

Conclusion: TREAD simultaneously improves training efficiency and generative performance without architectural changes or additional parameters, making diffusion model training more accessible.

Abstract: Diffusion models have emerged as the mainstream approach for visual generation. However, these models typically suffer from sample inefficiency and high training costs. Consequently, methods for efficient finetuning, inference and personalization were quickly adopted by the community. However, training these models in the first place remains very costly. While several recent approaches - including masking, distillation, and architectural modifications - have been proposed to improve training efficiency, each of these methods comes with a tradeoff: they achieve enhanced performance at the expense of increased computational cost or vice versa. In contrast, this work aims to improve training efficiency as well as generative performance at the same time through routes that act as a transport mechanism for randomly selected tokens from early layers to deeper layers of the model. Our method is not limited to the common transformer-based model - it can also be applied to state-space models and achieves this without architectural modifications or additional parameters. Finally, we show that TREAD reduces computational cost and simultaneously boosts model performance on the standard ImageNet-256 benchmark in class-conditional synthesis. Both of these benefits multiply to a convergence speedup of 14x at 400K training iterations compared to DiT and 37x compared to the best benchmark performance of DiT at 7M training iterations. Furthermore, we achieve a competitive FID of 2.09 in a guided and 3.93 in an unguided setting, which improves upon the DiT, without architectural changes.

Method: Created dataset using 41,006 drone-captured high-resolution aerial images from 20 diverse worldwide locations, providing accurate spatial layouts and comprehensive structural information in COLMAP format.

Result: Dataset offers significantly larger scale and higher detail than existing datasets, supporting detailed 3D reconstruction tasks and establishing benchmarks for large-scale Gaussian Splatting evaluation.

Conclusion: The dataset’s flexibility supports innovations and model plug-ins, paving the way for future 3D breakthroughs, with all datasets and code being open-sourced for community use.

Abstract: Current state-of-the-art 3D reconstruction models face limitations in building extra-large scale outdoor scenes, primarily due to the lack of sufficiently large-scale and detailed datasets. In this paper, we present a extra-large fine-grained dataset with 10 billion points composed of 41,006 drone-captured high-resolution aerial images, covering 20 diverse and culturally significant scenes from worldwide locations such as Cambridge Uni main buildings, the Pyramids, and the Forbidden City Palace. Compared to existing datasets, ours offers significantly larger scale and higher detail, uniquely suited for fine-grained 3D applications. Each scene contains an accurate spatial layout and comprehensive structural information, supporting detailed 3D reconstruction tasks. By reconstructing environments using these detailed images, our dataset supports multiple applications, including outputs in the widely adopted COLMAP format, establishing a novel benchmark for evaluating state-of-the-art large-scale Gaussian Splatting methods.The dataset’s flexibility encourages innovations and supports model plug-ins, paving the way for future 3D breakthroughs. All datasets and code will be open-sourced for community use.

Method: Use k-sparse autoencoders to identify interpretable monosemantic concepts in text embedding latent space, enabling precise steering of generation away from or towards concepts during test time without model retraining.

Result: Achieves 20.01% improvement in unsafe concept removal, effective style manipulation, ~5x faster than state-of-the-art, maintains generation quality, and robust against adversarial prompts.

Conclusion: The proposed framework provides simple, efficient, interpretable concept manipulation for diffusion models without compromising quality or requiring retraining, offering significant improvements over existing methods.

Abstract: Despite the remarkable progress in text-to-image generative models, they are prone to adversarial attacks and inadvertently generate unsafe, unethical content. Existing approaches often rely on fine-tuning models to remove specific concepts, which is computationally expensive, lacks scalability, and/or compromises generation quality. In this work, we propose a novel framework leveraging k-sparse autoencoders (k-SAEs) to enable efficient and interpretable concept manipulation in diffusion models. Specifically, we first identify interpretable monosemantic concepts in the latent space of text embeddings and leverage them to precisely steer the generation away or towards a given concept (e.g., nudity) or to introduce a new concept (e.g., photographic style) – all during test time. Through extensive experiments, we demonstrate that our approach is very simple, requires no retraining of the base model nor LoRA adapters, does not compromise the generation quality, and is robust to adversarial prompt manipulations. Our method yields an improvement of $\mathbf{20.01%}$ in unsafe concept removal, is effective in style manipulation, and is $\mathbf{\sim5}$x faster than the current state-of-the-art. Code is available at: https://github.com/kim-dahye/steerers

Method: Optimization framework with dynamic λ parameter (0→1) balancing source/target domain losses. Uses self-training for pseudo-labels and weighted loss minimization for stable progressive adaptation.

Result: Outperforms baseline methods on rotated MNIST, color-shifted MNIST, portrait dataset, and forest cover type dataset. Ablation studies confirm advantages of progressive domain penetration.

Conclusion: Provides theoretical support and practical framework for asymptotic domain adaptation, expanding application potential in dynamic environments through progressive domain penetration strategy.

Abstract: In this paper, we propose a new method called Gradual Domain Osmosis, which aims to solve the problem of smooth knowledge migration from source domain to target domain in Gradual Domain Adaptation (GDA). Traditional Gradual Domain Adaptation methods mitigate domain bias by introducing intermediate domains and self-training strategies, but often face the challenges of inefficient knowledge migration or missing data in intermediate domains. In this paper, we design an optimisation framework based on the hyperparameter $\lambda$ by dynamically balancing the loss weights of the source and target domains, which enables the model to progressively adjust the strength of knowledge migration ($\lambda$ incrementing from 0 to 1) during the training process, thus achieving cross-domain generalisation more efficiently. Specifically, the method incorporates self-training to generate pseudo-labels and iteratively updates the model by minimising a weighted loss function to ensure stability and robustness during progressive adaptation in the intermediate domain. The experimental part validates the effectiveness of the method on rotated MNIST, colour-shifted MNIST, portrait dataset and forest cover type dataset, and the results show that it outperforms existing baseline methods. The paper further analyses the impact of the dynamic tuning strategy of the hyperparameter $\lambda$ on the performance through ablation experiments, confirming the advantages of progressive domain penetration in mitigating the domain bias and enhancing the model generalisation capability. The study provides a theoretical support and practical framework for asymptotic domain adaptation and expands its application potential in dynamic environments.

Method: Create Synthetic Customization Dataset (SynCD) using text-to-image models and 3D data, train encoder with shared attention mechanism, and use inference normalization for text/image guidance vectors.

Result: Improves upon existing encoder-based methods on standard customization benchmarks.

Conclusion: The proposed approach with synthetic dataset, shared attention, and inference normalization effectively enhances text-to-image customization quality.

Abstract: Customization of text-to-image models enables users to insert new concepts or objects and generate them in unseen settings. Existing methods either rely on comparatively expensive test-time optimization or train encoders on single-image datasets without multi-image supervision, which can limit image quality. We propose a simple approach to address these challenges. We first leverage existing text-to-image models and 3D datasets to create a high-quality Synthetic Customization Dataset (SynCD) consisting of multiple images of the same object in different lighting, backgrounds, and poses. Using this dataset, we train an encoder-based model that incorporates fine-grained visual details from reference images via a shared attention mechanism. Finally, we propose an inference technique that normalizes text and image guidance vectors to mitigate overexposure issues in sampled images. Through extensive experiments, we show that our encoder-based model, trained on SynCD, and with the proposed inference algorithm, improves upon existing encoder-based methods on standard customization benchmarks.

Method: Proposes MSDCRD framework that systematically decouples global features into multi-scale local features and uses sample-wise and feature-wise contrastive losses for efficient single-batch distillation.

Result: MSDCRD achieves superior performance in both homogeneous teacher-student settings and heterogeneous architectures with significant feature discrepancies.

Conclusion: The framework demonstrates strong generalization capability by effectively addressing semantic confusion and inefficiency issues in knowledge distillation.

Abstract: Knowledge distillation enhances the performance of compact student networks by transferring knowledge from more powerful teacher networks without introducing additional parameters. In the feature space, local regions within an individual global feature encode distinct yet interdependent semantic information. Previous feature-based distillation methods mainly emphasize global feature alignment while neglecting the decoupling of local regions within an individual global feature, which often results in semantic confusion and suboptimal performance. Moreover, conventional contrastive representation distillation suffers from low efficiency due to its reliance on a large memory buffer to store feature samples. To address these limitations, this work proposes MSDCRD, a model-agnostic distillation framework that systematically decouples global features into multi-scale local features and leverages the resulting semantically rich feature samples with tailored sample-wise and feature-wise contrastive losses. This design enables efficient distillation using only a single batch, eliminating the dependence on external memory. Extensive experiments demonstrate that MSDCRD achieves superior performance not only in homogeneous teacher-student settings but also in heterogeneous architectures where feature discrepancies are more pronounced, highlighting its strong generalization capability.

Method: Proposes FCVSR with motion-guided adaptive alignment network and multi-frequency feature refinement module, trained with frequency-aware contrastive loss.

Result: Achieves up to 0.14dB PSNR gain over second-best model on three public compressed video SR datasets with good complexity.

Conclusion: FCVSR demonstrates effectiveness in compressed video super-resolution through frequency domain processing and achieves state-of-the-art performance.

Abstract: Compressed video super-resolution (SR) aims to generate high-resolution (HR) videos from the corresponding low-resolution (LR) compressed videos. Recently, some compressed video SR methods attempt to exploit the spatio-temporal information in the frequency domain, showing great promise in super-resolution performance. However, these methods do not differentiate various frequency subbands spatially or capture the temporal frequency dynamics, potentially leading to suboptimal results. In this paper, we propose a deep frequency-based compressed video SR model (FCVSR) consisting of a motion-guided adaptive alignment (MGAA) network and a multi-frequency feature refinement (MFFR) module. Additionally, a frequency-aware contrastive loss is proposed for training FCVSR, in order to reconstruct finer spatial details. The proposed model has been evaluated on three public compressed video super-resolution datasets, with results demonstrating its effectiveness when compared to existing works in terms of super-resolution performance (up to a 0.14dB gain in PSNR over the second-best model) and complexity.

Method: Extend Prov-GigaPath vision model into vision-language model using adaptors and contrastive learning, then fine-tune with learnable prompts using multi-granular attention and optimal transport-based visual-text distance.

Result: Empirical experiments on lung, kidney, and breast pathology modalities show superior performance over latest competitors and consistent improvements across CLIP, PLIP, and Prov-GigaPath integrated PLIP architectures.

Conclusion: The proposed approach effectively enhances few-shot pathology classification by capturing both fine-grained details and broader context through multi-granular attention and robust visual-text alignment.

Abstract: Whole slide pathology image classification presents challenges due to gigapixel image sizes and limited annotation labels, hindering model generalization. This paper introduces a prompt learning method to adapt large vision-language models for few-shot pathology classification. We first extend the Prov-GigaPath vision foundation model, pre-trained on 1.3 billion pathology image tiles, into a vision-language model by adding adaptors and aligning it with medical text encoders via contrastive learning on 923K image-text pairs. The model is then used to extract visual features and text embeddings from few-shot annotations and fine-tunes with learnable prompt embeddings. Unlike prior methods that combine prompts with frozen features using prefix embeddings or self-attention, we propose multi-granular attention that compares interactions between learnable prompts with individual image patches and groups of them. This approach improves the model’s ability to capture both fine-grained details and broader context, enhancing its recognition of complex patterns across sub-regions. To further improve accuracy, we leverage (unbalanced) optimal transport-based visual-text distance to secure model robustness by mitigating perturbations that might occur during the data augmentation process. Empirical experiments on lung, kidney, and breast pathology modalities validate the effectiveness of our approach; thereby, we surpass several of the latest competitors and consistently improve performance across diverse architectures, including CLIP, PLIP, and Prov-GigaPath integrated PLIP.

Method: Divides panoramas into patch sequences, treats them like video frames, uses SAM2’s memory mechanism for cross-patch correspondence, fine-tunes encoder for semantic prediction, and adds FoV-based prototypical adaptation with dynamic pseudo labels.

Result: Outperforms state-of-the-art methods: 79.06% (+10.22%) on SPin8-to-SPan8, 62.46% (+6.58%) on CS13-to-DP13.

Conclusion: OmniSAM successfully adapts SAM2 for panoramic semantic segmentation, bridging the domain gap through patch sequence processing and memory mechanisms.

Abstract: Segment Anything Model 2 (SAM2) has emerged as a strong base model in various pinhole imaging segmentation tasks. However, when applying it to $360^\circ$ domain, the significant field-of-view (FoV) gap between pinhole ($70^\circ \times 70^\circ$) and panoramic images ($180^\circ \times 360^\circ$) poses unique challenges. Two major concerns for this application includes 1) inevitable distortion and object deformation brought by the large FoV disparity between domains; 2) the lack of pixel-level semantic understanding that the original SAM2 cannot provide. To address these issues, we propose a novel OmniSAM framework, which makes the first attempt to apply SAM2 for panoramic semantic segmentation. Specifically, to bridge the first gap, OmniSAM first divides the panorama into sequences of patches. These patches are then treated as image sequences in similar manners as in video segmentation tasks. We then leverage the SAM2’s memory mechanism to extract cross-patch correspondences that embeds the cross-FoV dependencies, improving feature continuity and the prediction consistency along mask boundaries. For the second gap, OmniSAM fine-tunes the pretrained image encoder and reutilize the mask decoder for semantic prediction. An FoV-based prototypical adaptation module with dynamic pseudo label update mechanism is also introduced to facilitate the alignment of memory and backbone features, thereby improving model generalization ability across different sizes of source models. Extensive experimental results demonstrate that OmniSAM outperforms the state-of-the-art methods by large margins, e.g., 79.06% (+10.22%) on SPin8-to-SPan8, 62.46% (+6.58%) on CS13-to-DP13.

Method: Introduces ARC curves that describe registration performance as a spectrum under various regularity degrees, and uses a HyperNetwork approach to continuously interpolate across the full regularization range for efficient ARC curve construction.

Result: ARC curves reveal unique insights not evident from existing evaluation practices, demonstrated through experiments on various deep learning DIR methods with different architectures and transformation models.

Conclusion: Provides guidelines for nuanced model evaluation and selection using the proposed ARC curve evaluation scheme, benefiting both practitioners and registration researchers.

Abstract: Evaluating deformable image registration (DIR) is challenging due to the inherent trade-off between achieving high alignment accuracy and maintaining deformation regularity. However, most existing DIR works either address this trade-off inadequately or overlook it altogether. In this paper, we highlight the issues with existing practices and propose an evaluation scheme that captures the trade-off continuously to holistically evaluate DIR methods. We first introduce the alignment regularity characteristic (ARC) curves, which describe the performance of a given registration method as a spectrum under various degrees of regularity. We demonstrate that the ARC curves reveal unique insights that are not evident from existing evaluation practices, using experiments on representative deep learning DIR methods with various network architectures and transformation models. We further adopt a HyperNetwork based approach that learns to continuously interpolate across the full regularization range, accelerating the construction and improving the sample density of ARC curves. Finally, we provide general guidelines for a nuanced model evaluation and selection using our evaluation scheme for both practitioners and registration researchers.

Method: Constructs multi-scale kernel dictionary parameterized by implicit neural representations and employs a recurrent Transformer to predict coefficient weights for filtering in frame correction and feature alignment.

Result: Outperforms four state-of-the-art BVSR models on three datasets, beating the second best approach (FMA-Net) by up to 0.59 dB in PSNR.

Conclusion: BVSR-IK effectively handles spatio-temporal varying degradations in videos through implicit kernel representations and recurrent Transformer architecture, demonstrating superior performance over existing methods.

Abstract: Blind video super-resolution (BVSR) is a low-level vision task which aims to generate high-resolution videos from low-resolution counterparts in unknown degradation scenarios. Existing approaches typically predict blur kernels that are spatially invariant in each video frame or even the entire video. These methods do not consider potential spatio-temporal varying degradations in videos, resulting in suboptimal BVSR performance. In this context, we propose a novel BVSR model based on Implicit Kernels, BVSR-IK, which constructs a multi-scale kernel dictionary parameterized by implicit neural representations. It also employs a newly designed recurrent Transformer to predict the coefficient weights for accurate filtering in both frame correction and feature alignment. Experimental results have demonstrated the effectiveness of the proposed BVSR-IK, when compared with four state-of-the-art BVSR models on three commonly used datasets, with BVSR-IK outperforming the second best approach, FMA-Net, by up to 0.59 dB in PSNR. Source code will be available at https://github.com/QZ1-boy/BVSR-IK.

Method: Proposes Isolated Channel ViT (IC-ViT) that patchifies image channels individually, allowing pretraining on single channels and fine-tuning on multi-channel datasets to capture dependencies between patches and channels.

Result: Achieves 4-14 percentage points performance improvement over existing channel-adaptive approaches on benchmarks including JUMP-CP, CHAMMI for cell microscopy, and So2Sat-LCZ42 for satellite imaging.

Conclusion: IC-ViT provides an effective pretraining framework for multi-channel imaging data, enabling robust feature representation and efficient training for foundation models on heterogeneous data.

Abstract: Vision Transformers (ViTs) have achieved remarkable success in standard RGB image processing tasks. However, applying ViTs to multi-channel imaging (MCI) data, e.g., for medical and remote sensing applications, remains a challenge. In particular, MCI data often consist of layers acquired from different modalities. Directly training ViTs on such data can obscure complementary information and impair the performance. In this paper, we introduce a simple yet effective pretraining framework for large-scale MCI datasets. Our method, named Isolated Channel ViT (IC-ViT), patchifies image channels individually and thereby enables pretraining for multimodal multi-channel tasks. We show that this channel-wise patchifying is a key technique for MCI processing. More importantly, one can pretrain the IC-ViT on single channels and finetune it on downstream multi-channel datasets. This pretraining framework captures dependencies between patches as well as channels and produces robust feature representation. Experiments on various tasks and benchmarks, including JUMP-CP and CHAMMI for cell microscopy imaging, and So2Sat-LCZ42 for satellite imaging, show that the proposed IC-ViT delivers 4-14 percentage points of performance improvement over existing channel-adaptive approaches. Further, its efficient training makes it a suitable candidate for large-scale pretraining of foundation models on heterogeneous data. Our code is available at https://github.com/shermanlian/IC-ViT.

Method: The survey reviews knowledge distillation from multiple aspects: distillation sources, distillation schemes, distillation algorithms, distillation by modalities, applications, and comparisons among existing methods. It provides a new categorization framework and investigates recent methods systematically.

Result: The survey provides a comprehensive overview of knowledge distillation methods with a novel representation structure, covering emerging subcategories such as KD for diffusion models, 3D inputs, foundational models, transformers, and LLMs that are not adequately covered in previous surveys.

Conclusion: The work presents an up-to-date and comprehensive survey of knowledge distillation with new perspectives, identifies existing challenges in the field, and discusses potential future research directions to advance the state of knowledge distillation techniques.

Abstract: Deep Neural Networks (DNNs) have achieved notable performance in the fields of computer vision and natural language processing with various applications in both academia and industry. However, with recent advancements in DNNs and transformer models with a tremendous number of parameters, deploying these large models on edge devices causes serious issues such as high runtime and memory consumption. This is especially concerning with the recent large-scale foundation models, Vision-Language Models (VLMs), and Large Language Models (LLMs). Knowledge Distillation (KD) is one of the prominent techniques proposed to address the aforementioned problems using a teacher-student architecture. More specifically, a lightweight student model is trained using additional knowledge from a cumbersome teacher model. In this work, a comprehensive survey of knowledge distillation methods is proposed. This includes reviewing KD from different aspects: distillation sources, distillation schemes, distillation algorithms, distillation by modalities, applications of distillation, and comparison among existing methods. In contrast to most existing surveys, which are either outdated or simply update former surveys, this work proposes a comprehensive survey with a new point of view and representation structure that categorizes and investigates the most recent methods in knowledge distillation. This survey considers various critically important subcategories, including KD for diffusion models, 3D inputs, foundational models, transformers, and LLMs. Furthermore, existing challenges in KD and possible future research directions are discussed. Github page of the project: https://github.com/IPL-Sharif/KD_Survey

Method: Leverages CLIP image embedding space’s additivity property to extract Color-Texture Embeddings, applies whitening and coloring transformation for color consistency, and introduces noise term to prevent texture loss during Regularized Whitening and Coloring Transformation.

Result: SADis surpasses state-of-the-art stylization methods both qualitatively and quantitatively on WikiArt and StyleDrop datasets for Disentangled Stylized Image Generation task.

Conclusion: The proposed SADis approach provides a more precise and customizable solution for stylized image generation with effective color-texture disentanglement.

Abstract: Recent advances in Text-to-Image (T2I) diffusion models have transformed image generation, enabling significant progress in stylized generation using only a few style reference images. However, current diffusion-based methods struggle with fine-grained style customization due to challenges in controlling multiple style attributes, such as color and texture. This paper introduces the first tuning-free approach to achieve free-lunch color-texture disentanglement in stylized T2I generation, addressing the need for independently controlled style elements for the Disentangled Stylized Image Generation (DisIG) problem. Our approach leverages the Image-Prompt Additivity property in the CLIP image embedding space to develop techniques for separating and extracting Color-Texture Embeddings (CTE) from individual color and texture reference images. To ensure that the color palette of the generated image aligns closely with the color reference, we apply a whitening and coloring transformation to enhance color consistency. Additionally, to prevent texture loss due to the signal-leak bias inherent in diffusion training, we introduce a noise term that preserves textural fidelity during the Regularized Whitening and Coloring Transformation (RegWCT). Through these methods, our Style Attributes Disentanglement approach (SADis) delivers a more precise and customizable solution for stylized image generation. Experiments on images from the WikiArt and StyleDrop datasets demonstrate that, both qualitatively and quantitatively, SADis surpasses state-of-the-art stylization methods in the DisIG task.Code is released at https://deepffff.github.io/sadis.github.io/.

Method: The approach uses a contrastive loss that preserves semantic content from foundational models while disambiguating features located far apart on the shape’s surface. It requires only a small number of unpaired training meshes and learns a joint embedding space for entire shape families.

Result: The method achieves superior performance in correspondence matching benchmarks and enables various downstream applications including 2D-to-3D and 3D-to-3D texture transfer, in-part segmentation, pose alignment, and motion transfer in low-data regimes.

Conclusion: Unlike previous pairwise approaches, this solution constructs a joint embedding space where both seen and unseen 3D shapes are implicitly aligned without further optimization, providing a more efficient and robust framework for 3D correspondence tasks.

Abstract: Many 3D tasks such as pose alignment, animation, motion transfer, and 3D reconstruction rely on establishing correspondences between 3D shapes. This challenge has recently been approached by pairwise matching of semantic features from pre-trained vision models. However, despite their power, these features struggle to differentiate instances of the same semantic class such as left hand'' versus right hand’’ which leads to substantial mapping errors. To solve this, we learn a surface-aware embedding space that is robust to these ambiguities while facilitating shared mapping for an entire family of 3D shapes. Importantly, our approach is self-supervised and requires only a small number of unpaired training meshes to infer features for new possibly imperfect 3D shapes at test time. We achieve this by introducing a contrastive loss that preserves the semantic content of the features distilled from foundational models while disambiguating features located far apart on the shape’s surface. We observe superior performance in correspondence matching benchmarks and enable downstream applications including 2D-to-3D and 3D-to-3D texture transfer, in-part segmentation, pose alignment, and motion transfer in low-data regimes. Unlike previous pairwise approaches, our solution constructs a joint embedding space, where both seen and unseen 3D shapes are implicitly aligned without further optimization. The code is available at https://graphics.tudelft.nl/SurfaceAware3DFeatures.

Method: L2T incorporates the loss function into both instruction and response sequences, expanding training data and regularizing MLLMs to prevent over-reliance on language priors.

Result: Achieves up to 9% relative improvement on multimodal benchmarks, 18% improvement in captioning performance, and reduces hallucination in MLLMs without additional training data or computational overhead.

Conclusion: L2T provides an effective approach to enhance multimodal capabilities by addressing overfitting in visual instruction tuning, achieving significant performance gains with minimal computational requirements.

Abstract: We propose L2T, an advancement of visual instruction tuning (VIT). While VIT equips Multimodal LLMs (MLLMs) with promising multimodal capabilities, the current design choices for VIT often result in overfitting and shortcut learning, potentially degrading performance. This gap arises from an overemphasis on instruction-following abilities, while neglecting the proactive understanding of visual information. Inspired by this, L2T adopts a simple yet effective approach by incorporating the loss function into both the instruction and response sequences. It seamlessly expands the training data, and regularizes the MLLMs from overly relying on language priors. Based on this merit, L2T achieves a significant relative improvement of up to 9% on comprehensive multimodal benchmarks, requiring no additional training data and incurring negligible computational overhead. Surprisingly, L2T attains exceptional fundamental visual capabilities, yielding up to an 18% improvement in captioning performance, while simultaneously alleviating hallucination in MLLMs. Github code: https://github.com/Feng-Hong/L2T.

Method: Leverages distinctions between foreground and background motion, exaggerates objects with stronger motion signal, and includes architectural optimizations for faster throughput, training, and inference.

Result: Achieves state-of-the-art results for classification, detection, tracking, and counting tasks with fewer training resource requirements than existing deep-learning-based denoising methods.

Conclusion: SAVeD provides an effective self-supervised solution for denoising low-SNR videos without requiring clean reference data, with improved efficiency and performance across multiple computer vision tasks.

Abstract: Low signal-to-noise ratio videos – such as those from underwater sonar, ultrasound, and microscopy – pose significant challenges for computer vision models, particularly when paired clean imagery is unavailable. We present Spatiotemporal Augmentations and denoising in Video for Downstream Tasks (SAVeD), a novel self-supervised method that denoises low-SNR sensor videos using only raw noisy data. By leveraging distinctions between foreground and background motion and exaggerating objects with stronger motion signal, SAVeD enhances foreground object visibility and reduces background and camera noise without requiring clean video. SAVeD has a set of architectural optimizations that lead to faster throughput, training, and inference than existing deep learning methods. We also introduce a new denoising metric, FBD, which indicates foreground-background divergence for detection datasets without requiring clean imagery. Our approach achieves state-of-the-art results for classification, detection, tracking, and counting tasks, and it does so with fewer training resource requirements than existing deep-learning-based denoising methods. Project page: https://suzanne-stathatos.github.io/SAVeD Code page: https://github.com/suzanne-stathatos/SAVeD

Method: Created VideoAds dataset with well-curated advertisement videos having complex temporal structures, accompanied by manually annotated diverse questions across three core tasks. Proposed quantitative measure for video complexity comparison.

Result: Qwen2.5-VL-72B achieved 73.35% accuracy, outperforming GPT-4o (66.82%) and Gemini-1.5 Pro (69.66%). Human experts achieved 94.27% accuracy. Proprietary models lagged in video summarization and reasoning but performed best in visual finding.

Conclusion: VideoAds serves as a pivotal benchmark for advancing MLLMs’ temporal modeling capabilities, highlighting the need for better understanding of complex videos requiring high FPS sampling.

Abstract: Advertisement videos serve as a rich and valuable source of purpose-driven information, encompassing high-quality visual, textual, and contextual cues designed to engage viewers. They are often more complex than general videos of similar duration due to their structured narratives and rapid scene transitions, posing significant challenges to multi-modal large language models (MLLMs). In this work, we introduce VideoAds, the first dataset tailored for benchmarking the performance of MLLMs on advertisement videos. VideoAds comprises well-curated advertisement videos with complex temporal structures, accompanied by \textbf{manually} annotated diverse questions across three core tasks: visual finding, video summary, and visual reasoning. We propose a quantitative measure to compare VideoAds against existing benchmarks in terms of video complexity. Through extensive experiments, we find that Qwen2.5-VL-72B, an opensource MLLM, achieves 73.35% accuracy on VideoAds, outperforming GPT-4o (66.82%) and Gemini-1.5 Pro (69.66%); the two proprietary models especially fall behind the opensource model in video summarization and reasoning, but perform the best in visual finding. Notably, human experts easily achieve a remarkable accuracy of 94.27%. These results underscore the necessity of advancing MLLMs’ temporal modeling capabilities and highlight VideoAds as a potentially pivotal benchmark for future research in understanding video that requires high FPS sampling. The dataset and evaluation code will be publicly available at https://videoadsbenchmark.netlify.app.

Method: Proposed BabyVLM framework with comprehensive in-domain evaluation benchmarks and a synthetic training dataset created via child-directed transformations of existing datasets.

Result: VLMs trained with the synthetic dataset achieve superior performance on BabyVLM tasks compared to models trained solely on SAYCam or general-purpose data of SAYCam size.

Conclusion: BabyVLM provides a robust, developmentally-aligned evaluation tool and demonstrates that compact models trained on carefully curated data can generalize effectively, enabling data-efficient vision-language learning.

Abstract: Human infants rapidly develop visual reasoning skills from minimal input, suggesting that developmentally inspired pretraining could significantly enhance the efficiency of vision-language models (VLMs). Although recent efforts have leveraged infant-inspired datasets like SAYCam, existing evaluation benchmarks remain misaligned–they are either too simplistic, narrowly scoped, or tailored for large-scale pretrained models. Additionally, training exclusively on infant data overlooks the broader, diverse input from which infants naturally learn. To address these limitations, we propose BabyVLM, a novel framework comprising comprehensive in-domain evaluation benchmarks and a synthetic training dataset created via child-directed transformations of existing datasets. We demonstrate that VLMs trained with our synthetic dataset achieve superior performance on BabyVLM tasks compared to models trained solely on SAYCam or general-purpose data of the SAYCam size. BabyVLM thus provides a robust, developmentally aligned evaluation tool and illustrates how compact models trained on carefully curated data can generalize effectively, opening pathways toward data-efficient vision-language learning paradigms.

Method: Proposes a Degradation-Decoupled Fusion framework with two components: Degradation-Decoupled Optimization Network for degradation-specific decomposition and component-specific extraction, and Interactive Local-Global Fusion Network for multi-scale feature aggregation.

Result: Extensive experiments demonstrate that DDFusion achieves superior fusion performance under both clean and degraded conditions compared to conventional methods.

Conclusion: The proposed DDFusion framework effectively addresses real-world degradations in infrared and visible image fusion through degradation decoupling and unified modeling of degradation suppression and image fusion.

Abstract: Conventional infrared and visible image fusion(IVIF) methods often assume high-quality inputs, neglecting real-world degradations such as low-light and noise, which limits their practical applicability. To address this, we propose a Degradation-Decoupled Fusion(DDFusion) framework, which achieves degradation decoupling and jointly models degradation suppression and image fusion in a unified manner. Specifically, the Degradation-Decoupled Optimization Network(DDON) performs degradation-specific decomposition to decouple inter-degradation and degradation-information components, followed by component-specific extraction paths for effective suppression of degradation and enhancement of informative features. The Interactive Local-Global Fusion Network (ILGFN) aggregates complementary features across multi-scale pathways and alleviates performance degradation caused by the decoupling between degradation optimization and image fusion. Extensive experiments demonstrate that DDFusion achieves superior fusion performance under both clean and degraded conditions. Our code is available at https://github.com/Lmmh058/DDFusion.

Method: Proposes Ladder Shape-Biased Side-Tuning (LSP-ST) with Shape-Enhanced Large-Kernel Attention Module to hierarchically capture global structural information without handcrafted guidance.

Result: Achieves state-of-the-art performance on infrared small target detection benchmarks with only 4.72M parameters, and shows strong generalization across multiple detection tasks.

Conclusion: LSP-ST’s shape bias complements texture-based reasoning rather than competing with it, enabling robust adaptation while maintaining performance on texture-driven tasks.

Abstract: Fine-tuning the Segment Anything Model (SAM) for infrared small target detection poses significant challenges due to severe domain shifts. Existing adaptation methods often incorporate handcrafted priors to bridge this gap, yet such designs limit generalization and scalability. We identify a fundamental texture bias in foundation models, which overly depend on local texture cues for target localization. To address this, we propose Ladder Shape-Biased Side-Tuning (LSP-ST), a novel approach that introduces a shape-aware inductive bias to facilitate effective adaptation beyond texture cues. In contrast to prior work that injects explicit edge or contour features, LSP-ST models shape as a global structural prior, integrating both boundaries and internal layouts. We design a Shape-Enhanced Large-Kernel Attention Module to hierarchically and implicitly capture structural information in a fully differentiable manner, without task-specific handcrafted guidance. A theoretical analysis grounded in matched filtering and backpropagation reveals the mechanism by which the proposed attention improves structure-aware learning. With only 4.72M learnable parameters, LSP-ST achieves state-of-the-art performance on multiple infrared small target detection benchmarks. Furthermore, its strong generalization is validated across tasks such as mirror detection, shadow detection, and camouflaged object detection, while maintaining stable performance on texture-driven tasks like salient object detection, demonstrating that the introduced shape bias complements rather than competes with texture-based reasoning.

Method: Integrates motion estimation via optical flow for subtle target movements, uses multi-frame fusion for motion saliency enhancement, leverages nonlocal similarity to construct patch tensors, and proposes tensor decomposition-based INR model with alternating direction method of multipliers optimization.

Result: The approach robustly separates dim targets from complex infrared backgrounds and outperforms state-of-the-art methods in detection accuracy and robustness.

Conclusion: The proposed motion-enhanced nonlocal similarity INR framework effectively addresses the challenges of infrared dim and small target detection by capturing dynamic backgrounds and spatial-temporal correlations through continuous neural representations.

Abstract: Infrared dim and small target detection presents a significant challenge due to dynamic multi-frame scenarios and weak target signatures in the infrared modality. Traditional low-rank plus sparse models often fail to capture dynamic backgrounds and global spatial-temporal correlations, which results in background leakage or target loss. In this paper, we propose a novel motion-enhanced nonlocal similarity implicit neural representation (INR) framework to address these challenges. We first integrate motion estimation via optical flow to capture subtle target movements, and propose multi-frame fusion to enhance motion saliency. Second, we leverage nonlocal similarity to construct patch tensors with strong low-rank properties, and propose an innovative tensor decomposition-based INR model to represent the nonlocal patch tensor, effectively encoding both the nonlocal low-rankness and spatial-temporal correlations of background through continuous neural representations. An alternating direction method of multipliers is developed for the nonlocal INR model, which enjoys theoretical fixed-point convergence. Experimental results show that our approach robustly separates dim targets from complex infrared backgrounds, outperforming state-of-the-art methods in detection accuracy and robustness.

Method: Uses ViDRiP-Instruct dataset with 4278 video and diagnosis-specific chain-of-thought instructional pairs from YouTube educational videos, transfers knowledge from single-image datasets to train on weakly annotated clips, then fine-tunes on manually segmented videos.

Result: Establishes a new benchmark in pathology video analysis and provides detailed histological descriptions with definitive sign-out diagnoses.

Conclusion: ViDRiP-LLaVA offers a promising foundation for future AI systems supporting clinical decision-making through integrated visual and diagnostic reasoning, with code and data publicly available.

Abstract: We present ViDRiP-LLaVA, the first large multimodal model (LMM) in computational pathology that integrates three distinct image scenarios, including single patch images, automatically segmented pathology video clips, and manually segmented pathology videos. This integration closely mirrors the natural diagnostic process of pathologists. By generating detailed histological descriptions and culminating in a definitive sign-out diagnosis, ViDRiP-LLaVA bridges visual narratives with diagnostic reasoning. Central to our approach is the ViDRiP-Instruct dataset, comprising 4278 video and diagnosis-specific chain-of-thought instructional pairs sourced from educational histopathology videos on YouTube. Although high-quality data is critical for enhancing diagnostic reasoning, its creation is time-intensive and limited in volume. To overcome this challenge, we transfer knowledge from existing single-image instruction datasets to train on weakly annotated, keyframe-extracted clips, followed by fine-tuning on manually segmented videos. ViDRiP-LLaVA establishes a new benchmark in pathology video analysis and offers a promising foundation for future AI systems that support clinical decision-making through integrated visual and diagnostic reasoning. Our code, data, and model are publicly available at: https://github.com/QuIIL/ViDRiP-LLaVA.

Method: Trained on largest facial affect video dataset (~450k videos) covering most emotion zones; uses explainable facial affect descriptors with low computational complexity; benchmarked against MediaPipe, OpenFace, and Augsburg Affect Toolbox.

Result: Achieves 0.86 mean CCC on in-the-wild benchmarking set (~50k videos) and 0.75 mean CCC on SEWA test set, outperforming existing SOTA by ~7.1%.

Conclusion: xTrace provides a versatile, accurate, and computationally efficient solution for real-time facial expressive behavior analysis in naturalistic settings.

Abstract: Recognising expressive behaviours in face videos is a long-standing challenge in Affective Computing. Despite significant advancements in recent years, it still remains a challenge to build a robust and reliable system for naturalistic and in-the-wild facial expressive behaviour analysis in real time. This paper addresses two key challenges in building such a system: (1). The paucity of large-scale labelled facial affect video datasets with extensive coverage of the 2D emotion space, and (2). The difficulty of extracting facial video features that are discriminative, interpretable, robust, and computationally efficient. Toward addressing these challenges, this work introduces xTrace, a robust tool for facial expressive behaviour analysis and predicting continuous values of dimensional emotions, namely valence and arousal, from in-the-wild face videos. To address challenge (1), the proposed affect recognition model is trained on the largest facial affect video data set, containing $\sim$450k videos that cover most emotion zones in the dimensional emotion space, making xTrace highly versatile in analysing a wide spectrum of naturalistic expressive behaviours. To address challenge (2), xTrace uses facial affect descriptors that are not only explainable, but can also achieve a high degree of accuracy and robustness with low computational complexity. The key components of xTrace are benchmarked against three existing tools: MediaPipe, OpenFace, and Augsburg Affect Toolbox. On an in-the-wild benchmarking set composed of $\sim$50k videos, xTrace achieves 0.86 mean Concordance Correlation Coefficient (CCC) and on the SEWA test set it achieves 0.75 mean CCC, outperforming existing SOTA by $\sim$7.1%.

Method: Uses Spectral Eraser - a closed-form orthogonal projection module that identifies discriminative subspaces using SVD over token embeddings. Includes Expansion Mechanism for spectral regularization to balance filtering and preservation.

Result: Achieves efficient and thorough removal of targeted concepts (artistic styles, objects, identities, explicit content) in only 2 seconds, with minor damage to original generation ability and enhanced robustness against red-teaming.

Conclusion: CURE provides a fast, interpretable, and highly specific method for concept unlearning in diffusion models without requiring retraining, supervision, or iterative optimization.

Abstract: As Text-to-Image models continue to evolve, so does the risk of generating unsafe, copyrighted, or privacy-violating content. Existing safety interventions - ranging from training data curation and model fine-tuning to inference-time filtering and guidance - often suffer from incomplete concept removal, susceptibility to jail-breaking, computational inefficiency, or collateral damage to unrelated capabilities. In this paper, we introduce CURE, a training-free concept unlearning framework that operates directly in the weight space of pre-trained diffusion models, enabling fast, interpretable, and highly specific suppression of undesired concepts. At the core of our method is the Spectral Eraser, a closed-form, orthogonal projection module that identifies discriminative subspaces using Singular Value Decomposition over token embeddings associated with the concepts to forget and retain. Intuitively, the Spectral Eraser identifies and isolates features unique to the undesired concept while preserving safe attributes. This operator is then applied in a single step update to yield an edited model in which the target concept is effectively unlearned - without retraining, supervision, or iterative optimization. To balance the trade-off between filtering toxicity and preserving unrelated concepts, we further introduce an Expansion Mechanism for spectral regularization which selectively modulates singular vectors based on their relative significance to control the strength of forgetting. All the processes above are in closed-form, guaranteeing extremely efficient erasure in only $2$ seconds. Benchmarking against prior approaches, CURE achieves a more efficient and thorough removal for targeted artistic styles, objects, identities, or explicit content, with minor damage to original generation ability and demonstrates enhanced robustness against red-teaming.

Method: InstructSAM framework: uses large vision-language models to interpret instructions and estimate counts, SAM2 for mask proposals, and binary integer programming for mask-label assignment using semantic similarity and counting constraints.

Result: Matches or surpasses specialized baselines across tasks, maintains near-constant inference time regardless of object count, reduces output tokens by 89% and runtime by over 32% compared to direct generation approaches.

Conclusion: The proposed tasks, benchmark, and effective approach advance versatile object recognition systems for remote sensing applications.

Abstract: Language-Guided object recognition in remote sensing imagery is crucial for large-scale mapping and automated data annotation. However, existing open-vocabulary and visual grounding methods rely on explicit category cues, limiting their ability to handle complex or implicit queries that require advanced reasoning. To address this issue, we introduce a new suite of tasks, including Instruction-Oriented Object Counting, Detection, and Segmentation (InstructCDS), covering open-vocabulary, open-ended, and open-subclass scenarios. We further present EarthInstruct, the first InstructCDS benchmark for earth observation. It is constructed from two diverse remote sensing datasets with varying spatial resolutions and annotation rules across 20 categories, necessitating models to interpret dataset-specific instructions. Given the scarcity of semantically rich labeled data in remote sensing, we propose InstructSAM, a training-free framework for instruction-driven object recognition. InstructSAM leverages large vision-language models to interpret user instructions and estimate object counts, employs SAM2 for mask proposal, and formulates mask-label assignment as a binary integer programming problem. By integrating semantic similarity with counting constraints, InstructSAM efficiently assigns categories to predicted masks without relying on confidence thresholds. Experiments demonstrate that InstructSAM matches or surpasses specialized baselines across multiple tasks while maintaining near-constant inference time regardless of object count, reducing output tokens by 89% and overall runtime by over 32% compared to direct generation approaches. We believe the contributions of the proposed tasks, benchmark, and effective approach will advance future research in developing versatile object recognition systems.

Method: Created a benchmark with four evaluation dimensions: Character Attributes, Structured Character Locations, Multi-Dimensional Scene Attributes, and Spatial/Interactive Relationships. Used long prompts averaging 284.89 tokens validated by expert annotators.

Result: Evaluation of 12 T2I models showed critical limitations: ~50% accuracy in attribute binding and spatial reasoning, with all models degrading performance as prompt length increases. Analysis revealed compositional reasoning failures and attribute leakage issues.

Conclusion: Current T2I models have fundamental limitations in handling complex compositional requirements. The benchmark is open-sourced to advance detail-rich text-to-image generation.

Abstract: While recent text-to-image (T2I) models show impressive capabilities in synthesizing images from brief descriptions, their performance significantly degrades when confronted with long, detail-intensive prompts required in professional applications. We present DetailMaster, the first comprehensive benchmark specifically designed to evaluate T2I models’ systematic abilities to handle extended textual inputs that contain complex compositional requirements. Our benchmark introduces four critical evaluation dimensions: Character Attributes, Structured Character Locations, Multi-Dimensional Scene Attributes, and Spatial/Interactive Relationships. The benchmark comprises long and detail-rich prompts averaging 284.89 tokens, with high quality validated by expert annotators. Evaluation on 7 general-purpose and 5 long-prompt-optimized T2I models reveals critical performance limitations: state-of-the-art models achieve merely $\sim$50% accuracy in key dimensions like attribute binding and spatial reasoning, while all models showing progressive performance degradation as prompt length increases. Our analysis reveals fundamental limitations in compositional reasoning, demonstrating that current encoders flatten complex grammatical structures and that diffusion models suffer from attribute leakage under detail-intensive conditions. We open-source our dataset, data curation code, and evaluation tools to advance detail-rich T2I generation and enable applications previously hindered by the lack of a dedicated benchmark.

Method: Proposes VORTA with two components: 1) sparse attention mechanism for efficient long-range dependency capture, and 2) routing strategy that adaptively replaces full 3D attention with specialized sparse attention variants.

Result: Achieves 1.76× end-to-end speedup without quality loss on VBench, and up to 14.41× speedup when integrated with other acceleration methods like model caching and step distillation, with negligible performance degradation.

Conclusion: VORTA demonstrates efficiency and enhances the practicality of video diffusion transformers in real-world settings, with codes and weights publicly available.

Abstract: Video diffusion transformers have achieved remarkable progress in high-quality video generation, but remain computationally expensive due to the quadratic complexity of attention over high-dimensional video sequences. Recent acceleration methods enhance the efficiency by exploiting the local sparsity of attention scores; yet they often struggle with accelerating the long-range computation. To address this problem, we propose VORTA, an acceleration framework with two novel components: 1) a sparse attention mechanism that efficiently captures long-range dependencies, and 2) a routing strategy that adaptively replaces full 3D attention with specialized sparse attention variants. VORTA achieves an end-to-end speedup $1.76\times$ without loss of quality on VBench. Furthermore, it can seamlessly integrate with various other acceleration methods, such as model caching and step distillation, reaching up to speedup $14.41\times$ with negligible performance degradation. VORTA demonstrates its efficiency and enhances the practicality of video diffusion transformers in real-world settings. Codes and weights are available at https://github.com/wenhao728/VORTA.

Method: Proposes SpikeStereoNet - a brain-inspired framework that fuses raw spike streams from two viewpoints and iteratively refines depth estimation through a recurrent spiking neural network (RSNN) update module.

Result: Outperforms existing methods on both synthetic and real-world datasets, particularly in challenging regions like textureless surfaces and extreme lighting conditions. Shows strong data efficiency with high accuracy even with reduced training data.

Conclusion: The framework successfully leverages spike streams’ ability to capture subtle edges and intensity shifts, providing an effective solution for stereo depth estimation in dynamic scenes.

Abstract: Conventional frame-based cameras often struggle with stereo depth estimation in rapidly changing scenes. In contrast, bio-inspired spike cameras emit asynchronous events at microsecond-level resolution, providing an alternative sensing modality. However, existing methods lack specialized stereo algorithms and benchmarks tailored to the spike data. To address this gap, we propose SpikeStereoNet, a brain-inspired framework and the first to estimate stereo depth directly from raw spike streams. The model fuses raw spike streams from two viewpoints and iteratively refines depth estimation through a recurrent spiking neural network (RSNN) update module. To benchmark our approach, we introduce a large-scale synthetic spike stream dataset and a real-world stereo spike dataset with dense depth annotations. SpikeStereoNet outperforms existing methods on both datasets by leveraging spike streams’ ability to capture subtle edges and intensity shifts in challenging regions such as textureless surfaces and extreme lighting conditions. Furthermore, our framework exhibits strong data efficiency, maintaining high accuracy even with substantially reduced training data. The source code and datasets will be publicly available.

Method: The approach uses spectral embeddings of random walk matrices constructed independently from each unimodal representation. This allows learning shared representations without requiring paired samples across modalities.

Result: Empirical results in computer vision and natural language processing show the method’s effectiveness in capturing meaningful cross-modal relations, achieving high performance in retrieval tasks, generation, arithmetics, zero-shot, and cross-domain classification.

Conclusion: This is the first work to demonstrate that shared cross-modal representations can be learned almost exclusively from unpaired samples, creating a universal cross-modal embedding that is independent of specific data modalities.

Abstract: Learning shared representations is a primary area of multimodal representation learning. The current approaches to achieve a shared embedding space rely heavily on paired samples from each modality, which are significantly harder to obtain than unpaired ones. In this work, we demonstrate that shared representations can be learned almost exclusively from unpaired data. Our arguments are grounded in the spectral embeddings of the random walk matrices constructed independently from each unimodal representation. Empirical results in computer vision and natural language processing domains support its potential, revealing the effectiveness of unpaired data in capturing meaningful cross-modal relations, demonstrating high capabilities in retrieval tasks, generation, arithmetics, zero-shot, and cross-domain classification. This work, to the best of our knowledge, is the first to demonstrate these capabilities almost exclusively from unpaired samples, giving rise to a cross-modal embedding that could be viewed as universal, i.e., independent of the specific modalities of the data. Our project page: https://shaham-lab.github.io/SUE_page.

Method: Used jigsaw puzzles as experimental framework to study MLLM training through fine-tuning, comparing RL and supervised fine-tuning approaches, and analyzing reasoning patterns.

Result: MLLMs achieved near-perfect accuracy on jigsaw puzzles after fine-tuning and generalized to complex configurations; RL showed better generalization than SFT; reasoning patterns appeared pre-existing rather than emergent; training on jigsaw puzzles induced generalization to other visual tasks.

Conclusion: RL exhibits more effective generalization than SFT for visual tasks, complex reasoning patterns are pre-existing in MLLMs, and jigsaw puzzles provide a valuable framework for studying rule-based visual RL in multimodal learning.

Abstract: The application of rule-based reinforcement learning (RL) to multimodal large language models (MLLMs) introduces unique challenges and potential deviations from findings in text-only domains, particularly for perception-heavy tasks. This paper provides a comprehensive study of rule-based visual RL, using jigsaw puzzles as a structured experimental framework. Jigsaw puzzles offer inherent ground truth, adjustable difficulty, and demand complex decision-making, making them ideal for this study. Our research reveals several key findings: \textit{Firstly,} we find that MLLMs, initially performing near to random guessing on the simplest jigsaw puzzles, achieve near-perfect accuracy and generalize to complex, unseen configurations through fine-tuning. \textit{Secondly,} training on jigsaw puzzles can induce generalization to other visual tasks, with effectiveness tied to specific task configurations. \textit{Thirdly,} MLLMs can learn and generalize with or without explicit reasoning, though open-source models often favor direct answering. Consequently, even when trained for step-by-step reasoning, they can ignore the thinking process in deriving the final answer. \textit{Fourthly,} we observe that complex reasoning patterns appear to be pre-existing rather than emergent, with their frequency increasing alongside training and task difficulty. \textit{Finally,} our results demonstrate that RL exhibits more effective generalization than Supervised Fine-Tuning (SFT), and an initial SFT cold start phase can hinder subsequent RL optimization. Although these observations are based on jigsaw puzzles and may vary across other visual tasks, this research contributes a valuable piece of jigsaw to the larger puzzle of collective understanding rule-based visual RL and its potential in multimodal learning. The code is available at: https://github.com/zifuwanggg/Jigsaw-R1

Method: Decouples segmentation masks into regional semantic conditions (via Semantic Alignment Attention Mask) and high-frequency shape conditions (via Entity Contour Map), and introduces Attribute Isolation Attention Mask to prevent cross-entity attribute leakage.

Result: Achieves state-of-the-art performance on both open-set and closed-set S2I benchmarks, particularly excelling in fine-grained spatial and attribute control of entities.

Conclusion: Seg2Any effectively addresses the limitations of existing S2I methods by providing comprehensive solutions for semantic consistency, shape consistency, and attribute isolation in multi-entity generation.

Abstract: Despite recent advances in diffusion models, top-tier text-to-image (T2I) models still struggle to achieve precise spatial layout control, i.e. accurately generating entities with specified attributes and locations. Segmentation-mask-to-image (S2I) generation has emerged as a promising solution by incorporating pixel-level spatial guidance and regional text prompts. However, existing S2I methods fail to simultaneously ensure semantic consistency and shape consistency. To address these challenges, we propose Seg2Any, a novel S2I framework built upon advanced multimodal diffusion transformers (e.g. FLUX). First, to achieve both semantic and shape consistency, we decouple segmentation mask conditions into regional semantic and high-frequency shape components. The regional semantic condition is introduced by a Semantic Alignment Attention Mask, ensuring that generated entities adhere to their assigned text prompts. The high-frequency shape condition, representing entity boundaries, is encoded as an Entity Contour Map and then introduced as an additional modality via multi-modal attention to guide image spatial structure. Second, to prevent attribute leakage across entities in multi-entity scenarios, we introduce an Attribute Isolation Attention Mask mechanism, which constrains each entity’s image tokens to attend exclusively to themselves during image self-attention. To support open-set S2I generation, we construct SACap-1M, a large-scale dataset containing 1 million images with 5.9 million segmented entities and detailed regional captions, along with a SACap-Eval benchmark for comprehensive S2I evaluation. Extensive experiments demonstrate that Seg2Any achieves state-of-the-art performance on both open-set and closed-set S2I benchmarks, particularly in fine-grained spatial and attribute control of entities.

Method: Uses triplane representation for scene encoding, ray-based pixel attention for viewpoint discrepancy, and panoramic epipolar-constrained attention for cross-frame feature alignment.

Result: Outperforms existing methods in satellite-to-ground alignment and multiview consistency, validated on the new VIGOR++ dataset.

Conclusion: SatDreamer360 effectively addresses the challenge of generating geometrically consistent multi-view ground panoramas from satellite imagery.

Abstract: Generating multiview-consistent $360^\circ$ ground-level scenes from satellite imagery is a challenging task with broad applications in simulation, autonomous navigation, and digital twin cities. Existing approaches primarily focus on synthesizing individual ground-view panoramas, often relying on auxiliary inputs like height maps or handcrafted projections, and struggle to produce multiview consistent sequences. In this paper, we propose SatDreamer360, a framework that generates geometrically consistent multi-view ground-level panoramas from a single satellite image, given a predefined pose trajectory. To address the large viewpoint discrepancy between ground and satellite images, we adopt a triplane representation to encode scene features and design a ray-based pixel attention mechanism that retrieves view-specific features from the triplane. To maintain multi-frame consistency, we introduce a panoramic epipolar-constrained attention module that aligns features across frames based on known relative poses. To support the evaluation, we introduce {VIGOR++}, a large-scale dataset for generating multi-view ground panoramas from a satellite image, by augmenting the original VIGOR dataset with more ground-view images and their pose annotations. Experiments show that SatDreamer360 outperforms existing methods in both satellite-to-ground alignment and multiview consistency.

Method: Constructed MicroG-4M dataset from real-world space missions and cinematic simulations, supporting three core tasks: fine-grained multi-label action recognition, temporal video captioning, and visual question answering on astronaut activities.

Result: Established baselines using state-of-the-art models and made all data, annotations, and code publicly available at https://github.com/LEI-QI-233/HAR-in-Space.

Conclusion: MicroG-4M addresses the critical gap in domain-robust video understanding for space applications by providing the first comprehensive benchmark for evaluating spatio-temporal and semantic reasoning in microgravity contexts.

Abstract: Despite substantial progress in video understanding, most existing datasets are limited to Earth’s gravitational conditions. However, microgravity alters human motion, interactions, and visual semantics, revealing a critical gap for real-world vision systems. This presents a challenge for domain-robust video understanding in safety-critical space applications. To address this, we introduce MicroG-4M, the first benchmark for spatio-temporal and semantic understanding of human activities in microgravity. Constructed from real-world space missions and cinematic simulations, the dataset includes 4,759 clips covering 50 actions, 1,238 context-rich captions, and over 7,000 question-answer pairs on astronaut activities and scene understanding. MicroG-4M supports three core tasks: fine-grained multi-label action recognition, temporal video captioning, and visual question answering, enabling a comprehensive evaluation of both spatial localization and semantic reasoning in microgravity contexts. We establish baselines using state-of-the-art models. All data, annotations, and code are available at https://github.com/LEI-QI-233/HAR-in-Space.

Method: Proposes COGO with two components: Commonality Enhancement (CE) that perturbs mid-to-low frequency regions, and Individuality Suppression (IS) that uses adaptive thresholds to weight gradients based on correlation with model individuality.

Result: Extensive experiments show COGO significantly improves transfer success rates of adversarial attacks, outperforming current state-of-the-art methods.

Conclusion: Enhancing common features shared across models while suppressing individual characteristics is an effective strategy for improving adversarial example transferability in Vision Transformers.

Abstract: Exploring effective and transferable adversarial examples is vital for understanding the characteristics and mechanisms of Vision Transformers (ViTs). However, adversarial examples generated from surrogate models often exhibit weak transferability in black-box settings due to overfitting. Existing methods improve transferability by diversifying perturbation inputs or applying uniform gradient regularization within surrogate models, yet they have not fully leveraged the shared and unique features of surrogate models trained on the same task, leading to suboptimal transfer performance. Therefore, enhancing perturbations of common information shared by surrogate models and suppressing those tied to individual characteristics offers an effective way to improve transferability. Accordingly, we propose a commonality-oriented gradient optimization strategy (COGO) consisting of two components: Commonality Enhancement (CE) and Individuality Suppression (IS). CE perturbs the mid-to-low frequency regions, leveraging the fact that ViTs trained on the same dataset tend to rely more on mid-to-low frequency information for classification. IS employs adaptive thresholds to evaluate the correlation between backpropagated gradients and model individuality, assigning weights to gradients accordingly. Extensive experiments demonstrate that COGO significantly improves the transfer success rates of adversarial attacks, outperforming current state-of-the-art methods.

Method: Projects vision tokens into text token distribution, dynamically selects most relevant vision token at each decoding step via context-aware constrained divergence minimization, and refines output distribution to incorporate visual semantics.

Result: Consistently enhances visual grounding across five benchmarks on recent LVLMs with minimal computational overhead, achieves competitive or superior results to state-of-the-art decoding baselines while reducing computational cost by up to 2x.

Conclusion: ReVisiT effectively leverages vision token semantics to improve visual grounding in LVLMs through a simple, training-free decoding approach that reduces hallucinations and computational costs.

Abstract: Large Vision-Language Models (LVLMs) achieve strong performance across multimodal tasks by integrating visual perception with language understanding. However, how vision information contributes to the model’s decoding process remains under-explored, as reflected in frequent hallucinations. Through a series of analyses, we found that (i) vision tokens provide meaningful visual information even when hallucinations occur, and (ii) their semantics are encoded in the textual space and become explicit under appropriate vocabulary constraints. Building on these observations, we propose ReVisiT, a simple training-free decoding method that references vision tokens to guide text generation. Our approach leverages the semantic information embedded within vision tokens by projecting them into the text token distribution. Specifically, ReVisiT dynamically selects the most relevant vision token at each decoding step via context-aware constrained divergence minimization, and using its constrained projection to refine the output distribution to better incorporate visual semantics. Across five benchmarks on recent LVLMs, ReVisiT consistently enhances visual grounding with minimal computational overhead, and achieves competitive or superior results to state-of-the-art decoding baselines while reducing computational cost by up to $2\times$.

Method: Uses lightweight adaptation module updated via gradient descent on meta-learned self-supervised loss during inference, with dissimilarity-based sampling to prevent shortcut learning.

Result: Outperforms state-of-the-art zero-shot methods in real-world robotic manipulation tasks and enables effective reward shaping in offline RL.

Conclusion: VITA successfully addresses temporal reasoning limitations and improves generalization in zero-shot value estimation for VLMs.

Abstract: Vision-Language Models (VLMs) show promise as zero-shot goal-conditioned value functions, but their frozen pre-trained representations limit generalization and temporal reasoning. We introduce VITA, a zero-shot value function learning method that enhances both capabilities via test-time adaptation. At inference, a lightweight adaptation module is updated via a gradient step on a meta-learned self-supervised loss, such that each test-time update improves value estimation. By updating sequentially over a trajectory, VITA encodes history into its parameters, addressing the temporal reasoning limitations. To mitigate shortcut learning, we propose a dissimilarity-based sampling strategy that selects semantically diverse segments of the trajectory during training. In real-world robotic manipulation tasks, VITA generalizes from a single training environment to diverse out-of-distribution tasks, environments, and embodiments, outperforming the state-of-the-art zero-shot method using autoregressive VLMs. Furthermore, we demonstrate that VITA’s zero-shot value estimates can be utilized for reward shaping in offline reinforcement learning, resulting in multi-task policies on the Meta-World benchmark that exceed the performance of those trained with the simulation’s fuzzy-logic dense rewards.

Method: Load signals converted to RGB line plot and three texture maps (recurrence plot, Markov transition field, Gramian angular field). ViFusionTST dual-stream Swin Transformer processes these in parallel with cross-attention fusion.

Result: On 6-month real-world data from 95 beds, achieved accuracy of 0.885 and F1 score of 0.794, outperforming recent 1D and 2D time-series baselines.

Conclusion: Image-based fusion of load-sensor signals is practical and effective for real-time, privacy-preserving fall prevention.

Abstract: Bed-related falls remain a major source of injury in hospitals and long-term care facilities, yet many commercial alarms trigger only after a patient has already left the bed. We show that early bed-exit intent can be predicted using only one low-cost load cell mounted under a bed leg. The resulting load signals are first converted into a compact set of complementary images: an RGB line plot that preserves raw waveforms and three texture maps-recurrence plot, Markov transition field, and Gramian angular field-that expose higher-order dynamics. We introduce ViFusionTST, a dual-stream Swin Transformer that processes the line plot and texture maps in parallel and fuses them through cross-attention to learn data-driven modality weights. To provide a realistic benchmark, we collected six months of continuous data from 95 beds in a long-term-care facility. On this real-world dataset ViFusionTST reaches an accuracy of 0.885 and an F1 score of 0.794, surpassing recent 1D and 2D time-series baselines across F1, recall, accuracy, and AUPRC. The results demonstrate that image-based fusion of load-sensor signals for time series classification is a practical and effective solution for real-time, privacy-preserving fall prevention.

Method: Proposes VideoCus-Factory pipeline for multi-subject data construction, IVTM training with image editing data, and OmniVCus framework with Lottery Embedding (for multi-subject inference) and Temporally Aligned Embedding (for control signal guidance).

Result: Significantly surpasses state-of-the-art methods in both quantitative and qualitative evaluations, demonstrating superior performance in multi-subject video customization and control.

Conclusion: The proposed method successfully addresses multi-subject video customization challenges and enables effective instructive editing using various control signals, representing a significant advancement in the field.

Abstract: Existing feedforward subject-driven video customization methods mainly study single-subject scenarios due to the difficulty of constructing multi-subject training data pairs. Another challenging problem that how to use the signals such as depth, mask, camera, and text prompts to control and edit the subject in the customized video is still less explored. In this paper, we first propose a data construction pipeline, VideoCus-Factory, to produce training data pairs for multi-subject customization from raw videos without labels and control signals such as depth-to-video and mask-to-video pairs. Based on our constructed data, we develop an Image-Video Transfer Mixed (IVTM) training with image editing data to enable instructive editing for the subject in the customized video. Then we propose a diffusion Transformer framework, OmniVCus, with two embedding mechanisms, Lottery Embedding (LE) and Temporally Aligned Embedding (TAE). LE enables inference with more subjects by using the training subjects to activate more frame embeddings. TAE encourages the generation process to extract guidance from temporally aligned control signals by assigning the same frame embeddings to the control and noise tokens. Experiments demonstrate that our method significantly surpasses state-of-the-art methods in both quantitative and qualitative evaluations. Video demos are at our project page: https://caiyuanhao1998.github.io/project/OmniVCus/. Our code will be released at https://github.com/caiyuanhao1998/Open-OmniVCus

Method: Proposes LH2Face loss function with: 1) vMF distribution-based similarity measure, 2) Uncertainty-Aware Margin Function for adaptive margins, 3) proxy-based loss functions for representation space optimization, and 4) renderer for face reconstruction optimization.

Result: Achieves 49.39% accuracy on IJB-B dataset, surpassing the second-place method by 2.37%. Superior performance on hard high-quality face datasets compared to similar schemes.

Conclusion: LH2Face effectively addresses the limitations of uniform margin strategies by incorporating quality-aware adaptive margins and multi-modal optimization, significantly improving face recognition performance on challenging samples.

Abstract: In current practical face authentication systems, most face recognition (FR) algorithms are based on cosine similarity with softmax classification. Despite its reliable classification performance, this method struggles with hard samples. A popular strategy to improve FR performance is incorporating angular or cosine margins. However, it does not take face quality or recognition hardness into account, simply increasing the margin value and thus causing an overly uniform training strategy. To address this problem, a novel loss function is proposed, named Loss function for Hard High-quality Face (LH2Face). Firstly, a similarity measure based on the von Mises-Fisher (vMF) distribution is stated, specifically focusing on the logarithm of the Probability Density Function (PDF), which represents the distance between a probability distribution and a vector. Then, an adaptive margin-based multi-classification method using softmax, called the Uncertainty-Aware Margin Function, is implemented in the article. Furthermore, proxy-based loss functions are used to apply extra constraints between the proxy and sample to optimize their representation space distribution. Finally, a renderer is constructed that optimizes FR through face reconstruction and vice versa. Our LH2Face is superior to similiar schemes on hard high-quality face datasets, achieving 49.39% accuracy on the IJB-B dataset, which surpasses the second-place method by 2.37%.

Method: Proposes DeRIS framework with perception-cognition decomposition, Loopback Synergy mechanism for module interaction, and non-referent sample conversion data augmentation.

Result: DeRIS demonstrates improved segmentation precision and robust image-text comprehension, with inherent adaptability to non- and multi-referents scenarios without architectural changes.

Conclusion: Cognitive capacity, not perception, is the primary limitation in RIS. DeRIS’s modular approach effectively addresses this bottleneck and enhances general applicability.

Abstract: Referring Image Segmentation (RIS) is a challenging task that aims to segment objects in an image based on natural language expressions. While prior studies have predominantly concentrated on improving vision-language interactions and achieving fine-grained localization, a systematic analysis of the fundamental bottlenecks in existing RIS frameworks remains underexplored. To bridge this gap, we propose DeRIS, a novel framework that decomposes RIS into two key components: perception and cognition. This modular decomposition facilitates a systematic analysis of the primary bottlenecks impeding RIS performance. Our findings reveal that the predominant limitation lies not in perceptual deficiencies, but in the insufficient multi-modal cognitive capacity of current models. To mitigate this, we propose a Loopback Synergy mechanism, which enhances the synergy between the perception and cognition modules, thereby enabling precise segmentation while simultaneously improving robust image-text comprehension. Additionally, we analyze and introduce a simple non-referent sample conversion data augmentation to address the long-tail distribution issue related to target existence judgement in general scenarios. Notably, DeRIS demonstrates inherent adaptability to both non- and multi-referents scenarios without requiring specialized architectural modifications, enhancing its general applicability. The codes and models are available at https://github.com/Dmmm1997/DeRIS.

Method: Design AIpsych benchmark to reveal psychological tendencies in model responses, investigate how model architecture and parameter size influence behavior when responding to strategically manipulated questions, and conduct human subject study.

Result: As model size increases, VLMs exhibit stronger sycophantic tendencies but reduced authority bias, suggesting increasing competence but potential erosion of response integrity. Human study validates hypotheses and highlights behavioral differences between VLMs and humans.

Conclusion: This work provides a new psychological perspective for understanding hallucination in VLMs and highlights the importance of integrating psychological principles into model evaluation.

Abstract: Hallucination is a long-standing problem that has been actively investigated in Vision-Language Models (VLMs). Existing research commonly attributes hallucinations to technical limitations or sycophancy bias, where the latter means the models tend to generate incorrect answers to align with user expectations. However, these explanations primarily focus on technical or externally driven factors, and may have neglected the possibility that hallucination behaviours might mirror cognitive biases observed in human psychology. In this work, we introduce a psychological taxonomy, categorizing VLMs’ cognitive biases that lead to hallucinations, including sycophancy, logical inconsistency, and a newly identified VLMs behaviour: appeal to authority. To systematically analyze these behaviours, we design AIpsych, a scalable benchmark that reveals psychological tendencies in model response patterns. Leveraging this benchmark, we investigate how variations in model architecture and parameter size influence model behaviour when responding to strategically manipulated questions. Our experiments reveal that as model size increases, VLMs exhibit stronger sycophantic tendencies but reduced authority bias, suggesting increasing competence but a potential erosion of response integrity. A human subject study further validates our hypotheses and highlights key behavioural differences between VLMs and human respondents. This work suggests a new perspective for understanding hallucination in VLMs and highlights the importance of integrating psychological principles into model evaluation.

Method: Created SenseShift6D dataset by physically sweeping 13 RGB exposures, 9 RGB gains, auto-exposure, 4 depth-capture modes, and 5 illumination levels for five common household objects, acquiring 166.4k RGB and 16.7k depth images with 1,380 unique sensor-lighting permutations per object pose.

Result: Test-time sensor control yields substantial performance gains: 19.5 percentage point improvement on pretrained generalizable models, enhances robustness where models typically fail, and remains effective even for instance-level pose estimators without additional training.

Conclusion: SenseShift6D shifts object pose evaluation from data-centered to sensor-aware robustness, laying foundation for adaptive, self-tuning perception systems that can operate robustly in uncertain real-world environments without costly training data expansion.

Abstract: Recent advances on 6D object-pose estimation have achieved high performance on representative benchmarks such as LM-O, YCB-V, and T-Less. However, these datasets were captured under fixed illumination and camera settings, leaving the impact of real-world variations in illumination, exposure, gain or depth-sensor mode-and the potential of test-time sensor control to mitigate such variations-largely unexplored. To bridge this gap, we introduce SenseShift6D, the first RGB-D dataset that physically sweeps 13 RGB exposures, 9 RGB gains, auto-exposure, 4 depth-capture modes, and 5 illumination levels. For five common household objects (spray, pringles, tincase, sandwich, and mouse), we acquire 166.4k RGB and 16.7k depth images, which can provide 1,380 unique sensor-lighting permutations per object pose. Experiments with state-of-the-art models on our dataset demonstrate that applying multimodal sensor control at test time yields substantial performance gains, achieving a 19.5 pp improvement on pretrained generalizable models. It also enhances robustness precisely where those models tend to fail. Moreover, even instance-level pose estimators, where train and test set share identical object and background, performance still varies under environmental and sensor change, demonstrating that test-time sensor control remains effective compared to costly expansions in the quantity and diversity of real-world training data, without any additional training. SenseShift6D extends the object pose evaluation paradigm from data-centered to sensor-aware robustness, laying a foundation for adaptive, self-tuning perception systems capable of operating robustly in uncertain real-world environments.

Method: 1) Automatic triplet generation pipeline using LLM-generated prompts and text-to-image models to create image pairs with identical elements; 2) Hybrid Contextual Alignment (CoAlign) framework that performs global alignment and local reasoning within broader context for robust representation learning.

Result: CoAlign achieves outstanding zero-shot performance on three benchmarks, demonstrating first feasibility of training CIR models on fully synthetic data. Under supervised training, it outperforms all state-of-the-art supervised CIR approaches.

Conclusion: The proposed synthetic dataset generation pipeline and CoAlign framework effectively address scalability and zero-shot capability limitations in CIR, achieving superior performance while reducing reliance on manual labeling.

Abstract: As a challenging vision-language (VL) task, Composed Image Retrieval (CIR) aims to retrieve target images using multimodal (image+text) queries. Although many existing CIR methods have attained promising performance, their reliance on costly, manually labeled triplets hinders scalability and zero-shot capability. To address this issue, we propose a scalable pipeline for automatic triplet generation, along with a fully synthetic dataset named Composed Image Retrieval on High-quality Synthetic Triplets (CIRHS). Our pipeline leverages a large language model (LLM) to generate diverse prompts, controlling a text-to-image generative model to produce image pairs with identical elements in each pair, which are then filtered and reorganized to form the CIRHS dataset. In addition, we introduce Hybrid Contextual Alignment (CoAlign), a novel CIR framework, which can accomplish global alignment and local reasoning within a broader context, enabling the model to learn more robust and informative representations. By utilizing the synthetic CIRHS dataset, CoAlign achieves outstanding zero-shot performance on three commonly used benchmarks, demonstrating for the first time the feasibility of training CIR models on a fully synthetic dataset. Furthermore, under supervised training, our method outperforms all the state-of-the-art supervised CIR approaches, validating the effectiveness of our proposed retrieval framework. The code and the CIRHS dataset will be released soon.

Method: Introduces MSAGSM that enhances GSM with multi-scale temporal shifts and channel grouped spatial attention, enabling efficient modeling of both short and long-term dependencies while focusing on salient regions. Also presents the Table Tennis Australia dataset with over 4,800 annotated events.

Result: Extensive experiments across four PES benchmarks demonstrate consistent performance improvements with minimal overhead, setting new state-of-the-art results.

Conclusion: MSAGSM is an effective lightweight, plug-and-play module that significantly advances precise event spotting capabilities in sports videos.

Abstract: Precise Event Spotting (PES) in sports videos requires frame-level recognition of fine-grained actions from single-camera footage. Existing PES models typically incorporate lightweight temporal modules such as the Gate Shift Module (GSM) or the Gate Shift Fuse to enrich 2D CNN feature extractors with temporal context. However, these modules are limited in both temporal receptive field and spatial adaptability. We propose a Multi-Scale Attention Gate Shift Module (MSAGSM) that enhances GSM with multi-scale temporal shifts and channel grouped spatial attention, enabling efficient modeling of both short and long-term dependencies while focusing on salient regions. MSAGSM is a lightweight, plug-and-play module that integrates seamlessly with diverse 2D backbones. To further advance the field, we introduce the Table Tennis Australia dataset, the first PES benchmark for table tennis containing over 4,800 precisely annotated events. Extensive experiments across four PES benchmarks demonstrate that MSAGSM consistently improves performance with minimal overhead, setting new state-of-the-art results.

Method: Uses RL to train a lightweight LLM to produce auxiliary prompts that guide downstream MLLMs to generate better calibrated confidence responses, specifically designed for clinical safety.

Result: Achieves state-of-the-art medical VQA performance on PMC-VQA benchmark and shows promising zero-shot generalization to larger MLLMs, improving both calibration and task accuracy.

Conclusion: Demonstrates potential for automated human-aligned prompt engineering to improve MLLM trustworthiness in safety-critical healthcare settings with scalable calibration.

Abstract: Multimodal large language models (MLLMs) hold considerable promise for applications in healthcare. However, their deployment in safety-critical settings is hindered by two key limitations: (i) sensitivity to prompt design, and (ii) a tendency to generate incorrect responses with high confidence. As clinicians may rely on a model’s stated confidence to gauge the reliability of its predictions, it is especially important that when a model expresses high confidence, it is also highly accurate. We introduce Prompt4Trust, the first reinforcement learning (RL) framework for prompt augmentation targeting confidence calibration in MLLMs. A lightweight LLM is trained to produce context-aware auxiliary prompts that guide a downstream task MLLM to generate responses in which the expressed confidence more accurately reflects predictive accuracy. Unlike conventional calibration techniques, Prompt4Trust specifically prioritizes aspects of calibration most critical for safe and trustworthy clinical decision-making. Beyond improvements driven by this clinically motivated calibration objective, our proposed method also improves task accuracy, achieving state-of-the-art medical visual question answering (VQA) performance on the PMC-VQA benchmark, which is composed of multiple-choice questions spanning diverse medical imaging modalities. Moreover, our framework trained with a small downstream task MLLM showed promising zero-shot generalization to larger MLLMs in our experiments, suggesting the potential for scalable calibration without the associated computational costs. This work demonstrates the potential of automated yet human-aligned prompt engineering for improving the the trustworthiness of MLLMs in safety critical settings. Our codebase can be found at https://github.com/xingbpshen/prompt4trust.

Method: Combines Local Outlier Factor (LOF) algorithm for filtering noisy data with YOLO-v11n deep learning model. Uses 5-fold cross-validation on five public datasets (CVC-ColonDB, CVC-ClinicDB, Kvasir-SEG, ETIS, EndoScene) with converted segmentation masks to detection labels. LOF configured with 30 neighbors and 5% contamination ratio.

Result: Achieved precision of 95.83%, recall of 91.85%, F1-score of 93.48%, mAP@0.5 of 96.48%, and mAP@0.5:0.95 of 77.75%. Outperforms previous YOLO-based methods in both accuracy and efficiency.

Conclusion: The proposed method is well-suited for real-time colonoscopy support in clinical settings, highlighting the importance of data preprocessing and model efficiency in medical imaging AI systems.

Abstract: Objectives: Timely and accurate detection of colorectal polyps plays a crucial role in diagnosing and preventing colorectal cancer, a major cause of mortality worldwide. This study introduces a new, lightweight, and efficient framework for polyp detection that combines the Local Outlier Factor (LOF) algorithm for filtering noisy data with the YOLO-v11n deep learning model. Study design: An experimental study leveraging deep learning and outlier removal techniques across multiple public datasets. Methods: The proposed approach was tested on five diverse and publicly available datasets: CVC-ColonDB, CVC-ClinicDB, Kvasir-SEG, ETIS, and EndoScene. Since these datasets originally lacked bounding box annotations, we converted their segmentation masks into suitable detection labels. To enhance the robustness and generalizability of our model, we apply 5-fold cross-validation and remove anomalous samples using the LOF method configured with 30 neighbors and a contamination ratio of 5%. Cleaned data are then fed into YOLO-v11n, a fast and resource-efficient object detection architecture optimized for real-time applications. We train the model using a combination of modern augmentation strategies to improve detection accuracy under diverse conditions. Results: Our approach significantly improves polyp localization performance, achieving a precision of 95.83%, recall of 91.85%, F1-score of 93.48%, mAP@0.5 of 96.48%, and mAP@0.5:0.95 of 77.75%. Compared to previous YOLO-based methods, our model demonstrates enhanced accuracy and efficiency. Conclusions: These results suggest that the proposed method is well-suited for real-time colonoscopy support in clinical settings. Overall, the study underscores how crucial data preprocessing and model efficiency are when designing effective AI systems for medical imaging.

Method: Integrates a multi-resolution transformer for robust long-range correspondence, trained with a novel loss function that concentrates probability on feasible matches, enabling joint estimation of disparity, occlusion, and confidence.

Result: Establishes new state-of-the-art on Middlebury v3 and ETH3D benchmarks, significantly outperforming prior methods in most metrics while reconstructing high-quality details with competitive efficiency.

Conclusion: S²M² resolves the trade-off between generalization capability and computational feasibility in stereo matching, demonstrating that global matching architectures can achieve both high accuracy and efficiency.

Abstract: The pursuit of a generalizable stereo matching model, capable of performing well across varying resolutions and disparity ranges without dataset-specific fine-tuning, has revealed a fundamental trade-off. Iterative local search methods achieve high scores on constrained benchmarks, but their core mechanism inherently limits the global consistency required for true generalization. However, global matching architectures, while theoretically more robust, have historically been rendered infeasible by prohibitive computational and memory costs. We resolve this dilemma with {S\textsuperscript{2}M\textsuperscript{2}}: a global matching architecture that achieves state-of-the-art accuracy and high efficiency without relying on cost volume filtering or deep refinement stacks. Our design integrates a multi-resolution transformer for robust long-range correspondence, trained with a novel loss function that concentrates probability on feasible matches. This approach enables a more robust joint estimation of disparity, occlusion, and confidence. {S\textsuperscript{2}M\textsuperscript{2}} establishes a new state of the art on Middlebury v3 and ETH3D benchmarks, significantly outperforming prior methods in most metrics while reconstructing high-quality details with competitive efficiency.

Method: Created a large-scale dataset (STAR) with flux-consistent image pairs using Hubble Space Telescope high-resolution observations and a flux-preserving data generation pipeline. Proposed a Flux-Invariant Super Resolution (FISR) model that accurately infers flux-consistent high-resolution images from input photometry.

Result: The FISR model suppresses state-of-the-art methods by 24.84% on a novel flux consistency metric. Extensive experiments demonstrate the effectiveness of the proposed method and the value of the STAR dataset.

Conclusion: STAR dataset and FISR model provide significant advancements for astronomical super-resolution, enabling systematic development of field-level ASR models with improved flux consistency crucial for astrophysics applications.

Abstract: Super-resolution (SR) advances astronomical imaging by enabling cost-effective high-resolution capture, crucial for detecting faraway celestial objects and precise structural analysis. However, existing datasets for astronomical SR (ASR) exhibit three critical limitations: flux inconsistency, object-crop setting, and insufficient data diversity, significantly impeding ASR development. We propose STAR, a large-scale astronomical SR dataset containing 54,738 flux-consistent star field image pairs covering wide celestial regions. These pairs combine Hubble Space Telescope high-resolution observations with physically faithful low-resolution counterparts generated through a flux-preserving data generation pipeline, enabling systematic development of field-level ASR models. To further empower the ASR community, STAR provides a novel Flux Error (FE) to evaluate SR models in physical view. Leveraging this benchmark, we propose a Flux-Invariant Super Resolution (FISR) model that could accurately infer the flux-consistent high-resolution images from input photometry, suppressing several SR state-of-the-art methods by 24.84% on a novel designed flux consistency metric, showing the priority of our method for astrophysics. Extensive experiments demonstrate the effectiveness of our proposed method and the value of our dataset. Code and models are available at https://github.com/GuoCheng12/STAR.

Method: Proposes DADM module to adaptively capture global and local motion features with dynamic weight generation, and STGU for selective temporal feature extraction, forming the IONext backbone.

Result: IONext consistently outperforms SOTA Transformer- and CNN-based methods on six datasets, reducing ATE by 10% and RTE by 12% on RNIN dataset compared to iMOT.

Conclusion: The proposed CNN-based IONext with DADM and STGU modules effectively addresses limitations of both Transformers and existing CNNs in inertial odometry, achieving superior performance through better global-local motion capture and temporal modeling.

Abstract: Researchers have increasingly adopted Transformer-based models for inertial odometry. While Transformers excel at modeling long-range dependencies, their limited sensitivity to local, fine-grained motion variations and lack of inherent inductive biases often hinder localization accuracy and generalization. Recent studies have shown that incorporating large-kernel convolutions and Transformer-inspired architectural designs into CNN can effectively expand the receptive field, thereby improving global motion perception. Motivated by these insights, we propose a novel CNN-based module called the Dual-wing Adaptive Dynamic Mixer (DADM), which adaptively captures both global motion patterns and local, fine-grained motion features from dynamic inputs. This module dynamically generates selective weights based on the input, enabling efficient multi-scale feature aggregation. To further improve temporal modeling, we introduce the Spatio-Temporal Gating Unit (STGU), which selectively extracts representative and task-relevant motion features in the temporal domain. This unit addresses the limitations of temporal modeling observed in existing CNN approaches. Built upon DADM and STGU, we present a new CNN-based inertial odometry backbone, named Next Era of Inertial Odometry (IONext). Extensive experiments on six public datasets demonstrate that IONext consistently outperforms state-of-the-art (SOTA) Transformer- and CNN-based methods. For instance, on the RNIN dataset, IONext reduces the average ATE by 10% and the average RTE by 12% compared to the representative model iMOT.

Method: Uses a lightweight goal-centered cross-attention layer that lets waypoint tokens highlight relevant image and map patches, then feeds fused tokens into a partially fine-tuned LLaMA-3.2 11B model.

Result: Achieves 95% success rate and 0.80 SPL on MD-NEX Outdoor-Driving benchmark, surpassing PhysNav-DG by 15% and halving collisions while improving efficiency with single-branch architecture.

Conclusion: Early token-level fusion of intent and map layout enables accurate, transparent, real-time driving, with ablations confirming the complementary roles of vision, waypoint, and map modalities.

Abstract: Autonomous cars need geometric accuracy and semantic understanding to navigate complex environments, yet most stacks handle them separately. We present XYZ-Drive, a single vision-language model that reads a front-camera frame, a 25m $\times$ 25m overhead map, and the next waypoint, then outputs steering and speed. A lightweight goal-centered cross-attention layer lets waypoint tokens highlight relevant image and map patches, supporting both action and textual explanations, before the fused tokens enter a partially fine-tuned LLaMA-3.2 11B model. On the MD-NEX Outdoor-Driving benchmark XYZ-Drive attains 95% success and 0.80 Success weighted by Path Length (SPL), surpassing PhysNav-DG by 15%. and halving collisions, all while significantly improving efficiency by using only a single branch. Sixteen ablations explain the gains. Removing any modality (vision, waypoint, map) drops success by up to 11%, confirming their complementary roles and rich connections. Replacing goal-centered attention with simple concatenation cuts 3% in performance, showing query-based fusion injects map knowledge more effectively. Keeping the transformer frozen loses 5%, showing the importance of fine-tuning when applying VLMs for specific tasks such as autonomous driving. Coarsening map resolution from 10 cm to 40 cm blurs lane edges and raises crash rate. Overall, these results demonstrate that early, token-level fusion of intent and map layout enables accurate, transparent, real-time driving.

Method: StreamAgent integrates question semantics and historical observations to anticipate temporal intervals and spatial regions with future task-relevant information. It uses a streaming KV-cache memory mechanism with hierarchical memory structure for efficient semantic retrieval and reduced storage overhead.

Result: Extensive experiments on streaming and long video understanding tasks show that the method outperforms existing methods in response accuracy and real-time efficiency.

Conclusion: The proposed StreamAgent demonstrates practical value for real-world streaming scenarios by enabling proactive and goal-driven responses in real-time video understanding.

Abstract: Real-time streaming video understanding in domains such as autonomous driving and intelligent surveillance poses challenges beyond conventional offline video processing, requiring continuous perception, proactive decision making, and responsive interaction based on dynamically evolving visual content. However, existing methods rely on alternating perception-reaction or asynchronous triggers, lacking task-driven planning and future anticipation, which limits their real-time responsiveness and proactive decision making in evolving video streams. To this end, we propose a StreamAgent that anticipates the temporal intervals and spatial regions expected to contain future task-relevant information to enable proactive and goal-driven responses. Specifically, we integrate question semantics and historical observations through prompting the anticipatory agent to anticipate the temporal progression of key events, align current observations with the expected future evidence, and subsequently adjust the perception action (e.g., attending to task-relevant regions or continuously tracking in subsequent frames). To enable efficient inference, we design a streaming KV-cache memory mechanism that constructs a hierarchical memory structure for selective recall of relevant tokens, enabling efficient semantic retrieval while reducing the overhead of storing all tokens in the traditional KV-cache. Extensive experiments on streaming and long video understanding tasks demonstrate that our method outperforms existing methods in response accuracy and real-time efficiency, highlighting its practical value for real-world streaming scenarios.

Method: Two core innovations: 1) Using instruction-editing datasets as hard negative image-text pairs with symmetric contrastive loss, 2) Incorporating long descriptive captions with rotary positional encodings to capture rich semantic context.

Result: Achieves substantial gains on MMVP benchmark and fine-grained visual recognition tasks, maintains robust zero-shot performance, and reduces visual hallucinations when integrated into Multimodal Large Language Models.

Conclusion: Synergizing targeted instruction-based contrastive learning with comprehensive descriptive information significantly elevates fine-grained understanding in Vision-Language Models.

Abstract: Despite the success of Vision-Language Models (VLMs) like CLIP in aligning vision and language, their proficiency in detailed, fine-grained visual comprehension remains a key challenge. We present CLIP-IN, a novel framework that bolsters CLIP’s fine-grained perception through two core innovations. Firstly, we leverage instruction-editing datasets, originally designed for image manipulation, as a unique source of hard negative image-text pairs. Coupled with a symmetric hard negative contrastive loss, this enables the model to effectively distinguish subtle visual-semantic differences. Secondly, CLIP-IN incorporates long descriptive captions, utilizing rotary positional encodings to capture rich semantic context often missed by standard CLIP. Our experiments demonstrate that CLIP-IN achieves substantial gains on the MMVP benchmark and various fine-grained visual recognition tasks, without compromising robust zero-shot performance on broader classification and retrieval tasks. Critically, integrating CLIP-IN’s visual representations into Multimodal Large Language Models significantly reduces visual hallucinations and enhances reasoning abilities. This work underscores the considerable potential of synergizing targeted, instruction-based contrastive learning with comprehensive descriptive information to elevate the fine-grained understanding of VLMs.

Method: Formulate image editing as Next Editing-token Prediction using autoregressive image generation, pre-train an any-order autoregressive text-to-image model for any-region editing, and enable zero-shot adaptation to NEP.

Result: Achieves state-of-the-art performance on widely used image editing benchmarks and naturally supports test-time scaling through iterative zero-shot refinement.

Conclusion: NEP provides an efficient and effective approach for text-guided image editing by selectively regenerating only editing regions, outperforming existing methods while reducing computational overhead.

Abstract: Text-guided image editing involves modifying a source image based on a language instruction and, typically, requires changes to only small local regions. However, existing approaches generate the entire target image rather than selectively regenerate only the intended editing areas. This results in (1) unnecessary computational costs and (2) a bias toward reconstructing non-editing regions, which compromises the quality of the intended edits. To resolve these limitations, we propose to formulate image editing as Next Editing-token Prediction (NEP) based on autoregressive image generation, where only regions that need to be edited are regenerated, thus avoiding unintended modification to the non-editing areas. To enable any-region editing, we propose to pre-train an any-order autoregressive text-to-image (T2I) model. Once trained, it is capable of zero-shot image editing and can be easily adapted to NEP for image editing, which achieves a new state-of-the-art on widely used image editing benchmarks. Moreover, our model naturally supports test-time scaling (TTS) through iteratively refining its generation in a zero-shot manner. The project page is: https://nep-bigai.github.io/

Method: Leverages vision-language models (VLMs) through a web interface, benchmarked on a dataset of 1,700 expert-annotated user-reference video pairs across 22 exercises, with an automated rubric-based evaluation pipeline.

Result: Benchmarks reveal substantial gaps compared to human-level coaching, highlighting challenges in nuanced movement analysis but showing potential for AI-driven fitness coaching.

Conclusion: FormCoach opens a new frontier in embodied AI by framing form correction as a collaborative process between humans and machines, with released dataset and evaluation pipeline to accelerate research.

Abstract: Good form is the difference between strength and strain, yet for the fast-growing community of at-home fitness enthusiasts, expert feedback is often out of reach. FormCoach transforms a simple camera into an always-on, interactive AI training partner, capable of spotting subtle form errors and delivering tailored corrections in real time, leveraging vision-language models (VLMs). We showcase this capability through a web interface and benchmark state-of-the-art VLMs on a dataset of 1,700 expert-annotated user-reference video pairs spanning 22 strength and mobility exercises. To accelerate research in AI-driven coaching, we release both the dataset and an automated, rubric-based evaluation pipeline, enabling standardized comparison across models. Our benchmarks reveal substantial gaps compared to human-level coaching, underscoring both the challenges and opportunities in integrating nuanced, context-aware movement analysis into interactive AI systems. By framing form correction as a collaborative and creative process between humans and machines, FormCoach opens a new frontier in embodied AI.

Method: Uses cross-attention to identify salient regions, generates initial responses for each region, computes reliability weights via Jensen-Shannon Divergence, and performs consistency-aware fusion with region-aware prompts.

Result: Significantly reduces hallucinations and improves response factuality across multiple LVLMs and benchmarks without requiring model updates.

Conclusion: MRFD effectively addresses hallucination issues in LVLMs through inter-region consistency modeling, providing a training-free solution for improved factual grounding.

Abstract: Large Vision-Language Models (LVLMs) have shown strong performance across multimodal tasks. However, they often produce hallucinations – text that is inconsistent with visual input, due to the limited ability to verify information in different regions of the image. To address this, we propose Multi-Region Fusion Decoding (MRFD), a training-free decoding method that improves factual grounding by modeling inter-region consistency. MRFD identifies salient regions using cross-attention, generates initial responses for each, and computes reliability weights based on Jensen-Shannon Divergence (JSD) among the responses. These weights guide a consistency-aware fusion of per-region predictions, using region-aware prompts inspired by Chain-of-Thought reasoning. Experiments across multiple LVLMs and benchmarks show that MRFD significantly reduces hallucinations and improves response factuality without requiring model updates.

Method: Developed a theoretical framework formulating error propagation in diffusion models, derived per-step quantization error propagation equations, and proposed a timestep-aware cumulative error compensation scheme.

Result: Achieved 1.2 PSNR improvement over SVDQuant on SDXL W4A4 with only <0.5% additional time overhead, effectively mitigating error propagation across multiple image datasets.

Conclusion: The proposed compensation strategy significantly enhances existing PTQ methods by addressing the fundamental challenge of quantization error accumulation in diffusion models.

Abstract: Diffusion models have transformed image synthesis by establishing unprecedented quality and creativity benchmarks. Nevertheless, their large-scale deployment faces challenges due to computationally intensive iterative denoising processes. Although post-training quantization (PTQ) provides an effective pathway for accelerating sampling, the iterative nature of diffusion models causes stepwise quantization errors to accumulate progressively during generation, inevitably compromising output fidelity. To address this challenge, we develop a theoretical framework that mathematically formulates error propagation in Diffusion Models (DMs), deriving per-step quantization error propagation equations and establishing the first closed-form solution for cumulative error. Building on this theoretical foundation, we propose a timestep-aware cumulative error compensation scheme. Extensive experiments on multiple image datasets demonstrate that our compensation strategy effectively mitigates error propagation, significantly enhancing existing PTQ methods. Specifically, it achieves a 1.2 PSNR improvement over SVDQuant on SDXL W4A4, while incurring only an additional $<$ 0.5% time overhead.

Method: Developed a multimodal retrieval model jointly optimized with LVLM using only general-purpose backbones with lightweight fine-tuning, analyzed different top-retrieved images’ impact on predictions, and conducted oracle analysis.

Result: Achieved competitive results with medically-pretrained models on clinical classification and VQA tasks. Joint retrieval optimization significantly improved challenging cases over standard RAG. Oracle analysis showed correct diagnosis is frequently achievable but large performance gap remains.

Conclusion: While joint optimization improves performance over standard RAG, there is substantial room for improvement as rerankers using frontier LVLMs do not close the gap from oracle performance, indicating need for future methods.

Abstract: Retrieving relevant visual and textual information from medical literature and hospital records can enhance diagnostic accuracy for clinical image interpretation. We develop a multimodal retrieval model jointly optimized with an LVLM for medical diagnosis, unlike standard RAG which doesn’t backpropagate LVLM errors to the retriever. Using only general-purpose backbones with lightweight fine-tuning, our model achieves competitive results with medically-pretrained models on clinical classification and VQA tasks. In a novel analysis, we find that different top-retrieved images often yield different predictions for the same target, and that these cases are challenging for all models, even for non-retrieval models. Our joint retrieval optimization significantly improves these cases over standard RAG. However, oracle analysis reveals that while the correct diagnosis is frequently achievable using one of the top retrieved images, in practice there is a large performance gap from the oracle, and rerankers using frontier LVLMs do not close this gap – leaving ample room for improvement by future methods. Code available at https://github.com/Nirmaz/JOMED.

Method: Hybrid data synthesis pipeline using BlenderProc for photorealistic images with domain randomization and NVIDIA’s Cosmos-Predict2 for physically consistent video sequences with temporal variation. YOLO-based detector trained on composite real+synthetic dataset.

Result: YOLO-based detector trained on composite dataset outperformed models trained solely on real images. A 1:1 ratio between real and synthetic samples achieved the highest accuracy.

Conclusion: Synthetic data generation is a viable, cost-effective, and safe strategy for developing reliable perception systems in safety-critical and resource-constrained industrial applications.

Abstract: This paper addresses the challenges of data scarcity and high acquisition costs in training robust object detection models for complex industrial environments, such as offshore oil platforms. Data collection in these hazardous settings often limits the development of autonomous inspection systems. To mitigate this issue, we propose a hybrid data synthesis pipeline that integrates procedural rendering and AI-driven video generation. The approach uses BlenderProc to produce photorealistic images with domain randomization and NVIDIA’s Cosmos-Predict2 to generate physically consistent video sequences with temporal variation. A YOLO-based detector trained on a composite dataset, combining real and synthetic data, outperformed models trained solely on real images. A 1:1 ratio between real and synthetic samples achieved the highest accuracy. The results demonstrate that synthetic data generation is a viable, cost-effective, and safe strategy for developing reliable perception systems in safety-critical and resource-constrained industrial applications.

Method: Temporal Token Fusion (TTF) uses dual-dimension detection combining grayscale pixel difference analysis with attention-based semantic relevance assessment, employing hard fusion strategies and keyframe anchoring to prevent error accumulation.

Result: Consistent improvements across benchmarks: 4.0 percentage points average on LIBERO (72.4% vs 68.4% baseline), 4.8% relative improvement on SimplerEnv, and 8.7% relative improvement on real robot tasks. Model-agnostic across OpenVLA and VLA-Cache architectures.

Conclusion: TTF demonstrates that selective temporal fusion enhances VLA performance, revealing that Query matrix reuse in attention mechanisms improves rather than compromises performance, suggesting promising directions for computational acceleration while improving task success rates.

Abstract: Vision-Language-Action (VLA) models process visual inputs independently at each timestep, discarding valuable temporal information inherent in robotic manipulation tasks. This frame-by-frame processing makes models vulnerable to visual noise while ignoring the substantial coherence between consecutive frames in manipulation sequences. We propose Temporal Token Fusion (TTF), a training-free approach that intelligently integrates historical and current visual representations to enhance VLA inference quality. Our method employs dual-dimension detection combining efficient grayscale pixel difference analysis with attention-based semantic relevance assessment, enabling selective temporal token fusion through hard fusion strategies and keyframe anchoring to prevent error accumulation. Comprehensive experiments across LIBERO, SimplerEnv, and real robot tasks demonstrate consistent improvements: 4.0 percentage points average on LIBERO (72.4% vs 68.4% baseline), cross-environment validation on SimplerEnv (4.8% relative improvement), and 8.7% relative improvement on real robot tasks. Our approach proves model-agnostic, working across OpenVLA and VLA-Cache architectures. Notably, TTF reveals that selective Query matrix reuse in attention mechanisms enhances rather than compromises performance, suggesting promising directions for direct KQV matrix reuse strategies that achieve computational acceleration while improving task success rates.

Method: Built lightweight captioning models with 125M-parameter language model, developed Sharp-Eyed Refinement framework that enhances caption quality by refining coarse descriptions and improving visual grounding through re-examination of informative regions.

Result: The model achieves performance comparable to MLLMs in detailed captioning tasks and outperforms both recent lightweight captioning methods and MLLMs in detailed captioning and long-range video QA tasks.

Conclusion: The proposed lightweight model serves as a strong captioning specialist for on-device applications, with the Sharp-Eyed Refinement framework effectively addressing limitations in attention mechanisms and visual representations.

Abstract: Image captioning is fundamental for applications like video-grounded chatbot systems and navigation robots, yet deploying such models on local devices is challenging due to the high computational demands of multimodal LLMs (MLLMs). To address this, we first build lightweight captioning models using a 125M-parameter language model, 56 times smaller than LLaMA-7B, and evaluate their performance not only on single-sentence but on detailed captioning tasks. We obtain surprising results showing that our model can achieve performance comparable to MLLMs, suggesting its potential to serve as a strong captioning specialist for on-device applications. While promising, our model also exhibits a limitation: like other MLLMs, it suffers from occasional captioning errors. We investigate the underlying causes and observe that the problems stem from ineffective attention mechanisms and limited visual representations. To alleviate them, we develop a novel captioning framework, Sharp-Eyed Refinement, which enhances caption quality by refining coarse descriptions into more precise captions. At its core, DeepLens improves visual grounding by re-examining the informative regions identified in the initial glance. Experimental results demonstrate the superiority of our model over both recent lightweight captioning methods and MLLMs in detailed captioning and even in long-range video QA tasks.

Method: Models cryoablation planning as a Markov decision process where an agent sequentially selects cryoprobe positions and ice sphere diameters in a simulated environment with clinical constraints.

Result: Achieved over 8 percentage-point Dice improvements compared to automated baselines and matched human expert performance while requiring substantially less planning time.

Conclusion: Reinforcement learning can deliver clinically viable, reproducible, and efficient cryoablation plans for prostate cancer treatment.

Abstract: Cryoablation is a minimally invasive localised treatment for prostate cancer that destroys malignant tissue during de-freezing, while sparing surrounding healthy structures. Its success depends on accurate preoperative planning of cryoprobe placements to fully cover the tumour and avoid critical anatomy. This planning is currently manual, expertise-dependent, and time-consuming, leading to variability in treatment quality and limited scalability. In this work, we introduce Cryo-RL, a reinforcement learning framework that models cryoablation planning as a Markov decision process and learns an optimal policy for cryoprobe placement. Within a simulated environment that models clinical constraints and stochastic intraoperative variability, an agent sequentially selects cryoprobe positions and ice sphere diameters. Guided by a reward function based on tumour coverage, this agent learns a cryoablation strategy that leads to optimal cryoprobe placements without the need for any manually-designed plans. Evaluated on 583 retrospective prostate cancer cases, Cryo-RL achieved over 8 percentage-point Dice improvements compared with the best automated baselines, based on geometric optimisation, and matched human expert performance while requiring substantially less planning time. These results highlight the potential of reinforcement learning to deliver clinically viable, reproducible, and efficient cryoablation plans.

Method: Developed a scalable automated data generation pipeline to synthesize high-quality video question answering data for the Needle in a Montage task, which assesses critical reasoning capabilities including long-context recall and temporal grounding.

Result: Created NeMoBench with 31,378 QA pairs from 13,486 videos of varying durations (seconds to hours). Experiments show the pipeline reliably generates high-quality data and enables continuous updates. Evaluated 20 state-of-the-art models.

Conclusion: NeMoBench provides a comprehensive benchmark for assessing VideoLLMs’ temporal reasoning capabilities, offering extensive evaluation results and insights into model limitations.

Abstract: Recent advances in video large language models (VideoLLMs) call for new evaluation protocols and benchmarks for complex temporal reasoning in video-language understanding. Inspired by the needle in a haystack test widely used by LLMs, we introduce a novel task of Needle in a Montage (NeMo), designed to assess VideoLLMs’ critical reasoning capabilities, including long-context recall and temporal grounding. To generate video question answering data for our task, we develop a scalable automated data generation pipeline that facilitates high-quality data synthesis. Built upon the proposed pipeline, we present NeMoBench, a video-language benchmark centered on our task. Specifically, our full set of NeMoBench features 31,378 automatically generated question-answer (QA) pairs from 13,486 videos with various durations ranging from seconds to hours. Experiments demonstrate that our pipeline can reliably and automatically generate high-quality evaluation data, enabling NeMoBench to be continuously updated with the latest videos. We evaluate 20 state-of-the-art models on our benchmark, providing extensive results and key insights into their capabilities and limitations. Our project page is available at: https://lavi-lab.github.io/NeMoBench.

Method: Evaluated six YOLO model variants on a custom dataset with 29 distinct Bangladeshi vehicle classes using high-resolution images captured across various roads and manually annotated with YOLO format bounding boxes.

Result: YOLOv11x achieved best performance (63.7% mAP@0.5, 43.8% mAP@0.5:0.95) but slow inference (45.8ms). Medium variants (YOLOv8m, YOLOv11m) offered optimal balance with 62.5%/61.8% mAP@0.5 and 14-15ms inference. Rare vehicle classes showed near-zero accuracy due to dataset imbalances.

Conclusion: The research provides foundation for developing robust object detection systems adapted to Bangladesh traffic conditions, addressing critical needs in autonomous vehicle technology for developing regions where generic-trained models fail.

Abstract: Vehicle detection systems trained on Non-Bangladeshi datasets struggle to accurately identify local vehicle types in Bangladesh’s unique road environments, creating critical gaps in autonomous driving technology for developing regions. This study evaluates six YOLO model variants on a custom dataset featuring 29 distinct vehicle classes, including region-specific vehicles such as Desi Nosimon'', Leguna’’, Battery Rickshaw'', and CNG’’. The dataset comprises high-resolution images (1920x1080) captured across various Bangladeshi roads using mobile phone cameras and manually annotated using LabelImg with YOLO format bounding boxes. Performance evaluation revealed YOLOv11x as the top performer, achieving 63.7% mAP@0.5, 43.8% mAP@0.5:0.95, 61.4% recall, and 61.6% F1-score, though requiring 45.8 milliseconds per image for inference. Medium variants (YOLOv8m, YOLOv11m) struck an optimal balance, delivering robust detection performance with mAP@0.5 values of 62.5% and 61.8% respectively, while maintaining moderate inference times around 14-15 milliseconds. The study identified significant detection challenges for rare vehicle classes, with Construction Vehicles and Desi Nosimons showing near-zero accuracy due to dataset imbalances and insufficient training samples. Confusion matrices revealed frequent misclassifications between visually similar vehicles, particularly Mini Trucks versus Mini Covered Vans. This research provides a foundation for developing robust object detection systems specifically adapted to Bangladesh traffic conditions, addressing critical needs in autonomous vehicle technology advancement for developing regions where conventional generic-trained models fail to perform adequately.

Method: Uses Grounding DINO vision-language model with natural language prompts to detect cattle muzzles in a zero-shot manner, eliminating the need for task-specific training or annotated data.

Result: Achieves mean Average Precision (mAP)@0.5 of 76.8%, demonstrating promising performance for muzzle detection without requiring annotated training data.

Conclusion: The framework provides the first annotation-free solution for cattle muzzle detection, offering improved adaptability and ease of deployment in livestock monitoring applications compared to supervised methods.

Abstract: Muzzle patterns are among the most effective biometric traits for cattle identification. Fast and accurate detection of the muzzle region as the region of interest is critical to automatic visual cattle identification.. Earlier approaches relied on manual detection, which is labor-intensive and inconsistent. Recently, automated methods using supervised models like YOLO have become popular for muzzle detection. Although effective, these methods require extensive annotated datasets and tend to be trained data-dependent, limiting their performance on new or unseen cattle. To address these limitations, this study proposes a zero-shot muzzle detection framework based on Grounding DINO, a vision-language model capable of detecting muzzles without any task-specific training or annotated data. This approach leverages natural language prompts to guide detection, enabling scalable and flexible muzzle localization across diverse breeds and environments. Our model achieves a mean Average Precision (mAP)@0.5 of 76.8%, demonstrating promising performance without requiring annotated data. To our knowledge, this is the first research to provide a real-world, industry-oriented, and annotation-free solution for cattle muzzle detection. The framework offers a practical alternative to supervised methods, promising improved adaptability and ease of deployment in livestock monitoring applications.

Method: Generate multiple class-invariant variants of input images using elastic and grid deformations, then aggregate predictions through confidence-guided filtering to remove unreliable outputs.

Result: Extensive experiments on PACS and Office-Home datasets show consistent gains across different DG algorithms and backbones.

Conclusion: CI-TTA is an effective and general complementary strategy for domain generalization that works with existing DG methods.

Abstract: Deep models often suffer significant performance degradation under distribution shifts. Domain generalization (DG) seeks to mitigate this challenge by enabling models to generalize to unseen domains. Most prior approaches rely on multi-domain training or computationally intensive test-time adaptation. In contrast, we propose a complementary strategy: lightweight test-time augmentation. Specifically, we develop a novel Class-Invariant Test-Time Augmentation (CI-TTA) technique. The idea is to generate multiple variants of each input image through elastic and grid deformations that nevertheless belong to the same class as the original input. Their predictions are aggregated through a confidence-guided filtering scheme that remove unreliable outputs, ensuring the final decision relies on consistent and trustworthy cues. Extensive Experiments on PACS and Office-Home datasets demonstrate consistent gains across different DG algorithms and backbones, highlighting the effectiveness and generality of our approach.

Method: End-to-end framework with MRI preprocessing, brain-region-relevance prior, brain-region-aging prior, and dedicated modules for feature aggregation and diagnosis using brain age gaps as constraints.

Result: Achieves 86% accuracy on external tests and over 96% accuracy in early-stage diagnosis, outperforming existing methods by more than 20%.

Conclusion: PD-Diag-Net provides an effective automated solution for PD diagnosis with high accuracy and clinical interpretability.

Abstract: Parkinson’s disease (PD) is a common neurodegenerative disorder that severely diminishes patients’ quality of life. Its global prevalence has increased markedly in recent decades. Current diagnostic workflows are complex and heavily reliant on neurologists’ expertise, often resulting in delays in early detection and missed opportunities for timely intervention. To address these issues, we propose an end-to-end automated diagnostic method for PD, termed PD-Diag-Net, which performs risk assessment and auxiliary diagnosis directly from raw MRI scans. This framework first introduces an MRI Pre-processing Module (MRI-Processor) to mitigate inter-subject and inter-scanner variability by flexibly integrating established medical imaging preprocessing tools. It then incorporates two forms of clinical prior knowledge: (1) Brain-Region-Relevance-Prior (Relevance-Prior), which specifies brain regions strongly associated with PD; and (2) Brain-Region-Aging-Prior (Aging-Prior), which reflects the accelerated aging typically observed in PD-associated regions. Building on these priors, we design two dedicated modules: the Relevance-Prior Guided Feature Aggregation Module (Aggregator), which guides the model to focus on PD-associated regions at the inter-subject level, and the Age-Prior Guided Diagnosis Module (Diagnoser), which leverages brain age gaps as auxiliary constraints at the intra-subject level to enhance diagnostic accuracy and clinical interpretability. Furthermore, we collected external test data from our collaborating hospital. Experimental results show that PD-Diag-Net achieves 86% accuracy on external tests and over 96% accuracy in early-stage diagnosis, outperforming existing advanced methods by more than 20%.

Method: A zero-shot inference-time optimization method that updates text embeddings of pretrained diffusion models during sampling to guide motion generation toward desired Laban Effort and Shape components.

Result: The approach successfully generates diverse expressive motion qualities while preserving motion identity by manipulating motion attributes according to target Laban tags.

Conclusion: Integrating Laban movement analysis quantification methods enables interpretable and expressive control of human motion generation in text-guided diffusion models.

Abstract: Diverse human motion generation is an increasingly important task, having various applications in computer vision, human-computer interaction and animation. While text-to-motion synthesis using diffusion models has shown success in generating high-quality motions, achieving fine-grained expressive motion control remains a significant challenge. This is due to the lack of motion style diversity in datasets and the difficulty of expressing quantitative characteristics in natural language. Laban movement analysis has been widely used by dance experts to express the details of motion including motion quality as consistent as possible. Inspired by that, this work aims for interpretable and expressive control of human motion generation by seamlessly integrating the quantification methods of Laban Effort and Shape components into the text-guided motion generation models. Our proposed zero-shot, inference-time optimization method guides the motion generation model to have desired Laban Effort and Shape components without any additional motion data by updating the text embedding of pretrained diffusion models during the sampling step. We demonstrate that our approach yields diverse expressive motion qualities while preserving motion identity by successfully manipulating motion attributes according to target Laban tags.

Method: Introduced a video-based benchmark with three tasks: phonological sign-form prediction, transparency (inferring meaning from form), and graded iconicity ratings. Evaluated 13 VLMs on Sign Language of the Netherlands in zero- and few-shot settings.

Result: VLMs recovered some handshape and location detail but remained below human performance on phonological form prediction; performed far from human baselines on transparency; only top models correlated moderately with human iconicity ratings. Models with stronger phonological form prediction correlated better with human iconicity judgment.

Conclusion: The findings validate diagnostic tasks for iconicity and motivate human-centric signals and embodied learning methods for improving visual grounding in multimodal models.

Abstract: Iconicity, the resemblance between linguistic form and meaning, is pervasive in signed languages, offering a natural testbed for visual grounding. For vision-language models (VLMs), the challenge is to recover such essential mappings from dynamic human motion rather than static context. We introduce the Visual Iconicity Challenge, a novel video-based benchmark that adapts psycholinguistic measures to evaluate VLMs on three tasks: (i) phonological sign-form prediction (e.g., handshape, location), (ii) transparency (inferring meaning from visual form), and (iii) graded iconicity ratings. We assess 13 state-of-the-art VLMs in zero- and few-shot settings on Sign Language of the Netherlands and compare them to human baselines. On phonological form prediction, VLMs recover some handshape and location detail but remain below human performance; on transparency, they are far from human baselines; and only top models correlate moderately with human iconicity ratings. Interestingly, models with stronger phonological form prediction correlate better with human iconicity judgment, indicating shared sensitivity to visually grounded structure. Our findings validate these diagnostic tasks and motivate human-centric signals and embodied learning methods for modelling iconicity and improving visual grounding in multimodal models.

Method: Uses Linear DiT with linear attention for efficiency and a constant-memory KV cache for block linear attention to enable long video generation with global context at fixed memory cost.

Result: Achieves competitive performance with modern small diffusion models while being 16x faster in latency, with training cost reduced to 12 days on 64 H100 GPUs (only 1% of MovieGen’s cost).

Conclusion: SANA-Video enables low-cost, high-quality video generation that is practical for deployment on consumer hardware with significant speed improvements.

Abstract: We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720x1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution, high-quality and long videos with strong text-video alignment at a remarkably fast speed, deployable on RTX 5090 GPU. Two core designs ensure our efficient, effective and long video generation: (1) Linear DiT: We leverage linear attention as the core operation, which is more efficient than vanilla attention given the large number of tokens processed in video generation. (2) Constant-Memory KV cache for Block Linear Attention: we design block-wise autoregressive approach for long video generation by employing a constant-memory state, derived from the cumulative properties of linear attention. This KV cache provides the Linear DiT with global context at a fixed memory cost, eliminating the need for a traditional KV cache and enabling efficient, minute-long video generation. In addition, we explore effective data filters and model training strategies, narrowing the training cost to 12 days on 64 H100 GPUs, which is only 1% of the cost of MovieGen. Given its low cost, SANA-Video achieves competitive performance compared to modern state-of-the-art small diffusion models (e.g., Wan 2.1-1.3B and SkyReel-V2-1.3B) while being 16x faster in measured latency. Moreover, SANA-Video can be deployed on RTX 5090 GPUs with NVFP4 precision, accelerating the inference speed of generating a 5-second 720p video from 71s to 29s (2.4x speedup). In summary, SANA-Video enables low-cost, high-quality video generation.

Method: Proposed theoretical framework showing pre-trained networks have universal winning slice property (spectral balance + high task energy), then developed SliceFine method that updates only selected slices of original weights without adding new parameters.

Result: SliceFine matches state-of-the-art PEFT methods across language and vision tasks while significantly improving training speed, memory efficiency, and model compactness.

Conclusion: The work bridges theory and practice by providing theoretical grounding for PEFT and offering a theoretically grounded alternative to existing PEFT techniques.

Abstract: This paper presents a theoretical framework explaining why fine tuning small, randomly selected subnetworks (slices) within pre trained models can be sufficient for downstream adaptation. We prove that pretrained networks exhibit a universal winning slice property arising from two phenomena: (1) spectral balance the eigenspectra of different weight matrix slices are remarkably similar; and (2) high task energy their backbone representations retain rich, task relevant features. This leads to the Universal Winning Slice Hypothesis, which provides a theoretical foundation for parameter efficient fine tuning (PEFT) in large scale models. Inspired by this, we propose SliceFine, a PEFT method that exploits this inherent redundancy by updating only selected slices of the original weights introducing zero new parameters, unlike adapter-based approaches. Empirically, SliceFine matches the performance of state of the art PEFT methods across language and vision tasks, while significantly improving training speed, memory efficiency, and model compactness. Our work bridges theory and practice, offering a theoretically grounded alternative to existing PEFT techniques.

Method: Proposes a novel post-training framework incorporating task-aware rewards to adapt reasoning-based RL models to diverse Earth Observation tasks, enhancing reasoning for remote sensing images while stabilizing optimization and improving robustness.

Result: Extensive experiments across multiple EO benchmarks show consistent performance gains over state-of-the-art generic and specialized vision language models.

Conclusion: The framework successfully adapts RL models to Earth Observation tasks, demonstrating improved reasoning capabilities and superior performance compared to existing approaches.

Abstract: Recent advances in reinforcement learning (RL) have delivered strong reasoning capabilities in natural image domains, yet their potential for Earth Observation (EO) remains largely unexplored. EO tasks introduce unique challenges, spanning referred object detection, image or region captioning, change detection, grounding, and temporal analysis, that demand task aware reasoning. We propose a novel post training framework that incorporates task aware rewards to enable effective adaptation of reasoning based RL models to diverse EO tasks. This training strategy enhances reasoning capabilities for remote sensing images, stabilizes optimization, and improves robustness. Extensive experiments across multiple EO benchmarks show consistent performance gains over state of the art generic and specialized vision language models. Code and models will be released publicly at https://mustansarfiaz.github.io/GeoVLM-R1/ .

Method: Proposed DA² with a data curation engine to generate high-quality panoramic depth data from perspective images (~543K new pairs) and SphereViT architecture that leverages spherical coordinates for geometric consistency.

Result: Achieves SoTA performance with 38% average improvement in AbsRel over strongest zero-shot baseline, outperforms prior in-domain methods, and offers higher efficiency as an end-to-end solution.

Conclusion: DA² demonstrates superior zero-shot generalization for panoramic depth estimation through large-scale data curation and spherical geometry-aware modeling, with released code and curated data.

Abstract: Panorama has a full FoV (360$^\circ\times$180$^\circ$), offering a more complete visual description than perspective images. Thanks to this characteristic, panoramic depth estimation is gaining increasing traction in 3D vision. However, due to the scarcity of panoramic data, previous methods are often restricted to in-domain settings, leading to poor zero-shot generalization. Furthermore, due to the spherical distortions inherent in panoramas, many approaches rely on perspective splitting (e.g., cubemaps), which leads to suboptimal efficiency. To address these challenges, we propose $\textbf{DA}$$^{\textbf{2}}$: $\textbf{D}$epth $\textbf{A}$nything in $\textbf{A}$ny $\textbf{D}$irection, an accurate, zero-shot generalizable, and fully end-to-end panoramic depth estimator. Specifically, for scaling up panoramic data, we introduce a data curation engine for generating high-quality panoramic depth data from perspective, and create $\sim$543K panoramic RGB-depth pairs, bringing the total to $\sim$607K. To further mitigate the spherical distortions, we present SphereViT, which explicitly leverages spherical coordinates to enforce the spherical geometric consistency in panoramic image features, yielding improved performance. A comprehensive benchmark on multiple datasets clearly demonstrates DA$^{2}$’s SoTA performance, with an average 38% improvement on AbsRel over the strongest zero-shot baseline. Surprisingly, DA$^{2}$ even outperforms prior in-domain methods, highlighting its superior zero-shot generalization. Moreover, as an end-to-end solution, DA$^{2}$ exhibits much higher efficiency over fusion-based approaches. Both the code and the curated panoramic data has be released. Project page: https://depth-any-in-any-dir.github.io/.

Method: PP uses fixed color center points to classify pixels into distinct categories, enabling fine-grained, spatially flexible masking that maintains adaptive flexibility while reducing computational costs.

Result: When integrated into ATD-light model, PP-ATD-light achieves superior SR performance with minimal overhead, outperforming CAMixer-ATD-light in both reconstruction quality and parameter efficiency while saving similar computation.

Conclusion: Pure-Pass provides an effective pixel-level masking approach that enhances computational efficiency in image super-resolution while maintaining or improving reconstruction quality.

Abstract: Image Super-Resolution (SR) aims to reconstruct high-resolution images from low-resolution counterparts, but the computational complexity of deep learning-based methods often hinders practical deployment. CAMixer is the pioneering work to integrate the advantages of existing lightweight SR methods and proposes a content-aware mixer to route token mixers of varied complexities according to the difficulty of content recovery. However, several limitations remain, such as poor adaptability, coarse-grained masking and spatial inflexibility, among others. We propose Pure-Pass (PP), a pixel-level masking mechanism that identifies pure pixels and exempts them from expensive computations. PP utilizes fixed color center points to classify pixels into distinct categories, enabling fine-grained, spatially flexible masking while maintaining adaptive flexibility. Integrated into the state-of-the-art ATD-light model, PP-ATD-light achieves superior SR performance with minimal overhead, outperforming CAMixer-ATD-light in reconstruction quality and parameter efficiency when saving a similar amount of computation.

Method: Integrates complementary adapters via diffusion: AutoKL for low-level features and CLIP for semantics. Uses pretraining on one subject followed by fine-tuning only 17% of parameters (fully connected layers) for new subjects while freezing other components.

Result: Achieves state-of-the-art performance with only one hour of training per subject on lightweight GPUs (three RTX 4090), outperforming existing methods.

Conclusion: NeuroSwift enables efficient and accurate cross-subject visual reconstruction from fMRI data with minimal computational requirements.

Abstract: Reconstructing visual information from brain activity via computer vision technology provides an intuitive understanding of visual neural mechanisms. Despite progress in decoding fMRI data with generative models, achieving accurate cross-subject reconstruction of visual stimuli remains challenging and computationally demanding. This difficulty arises from inter-subject variability in neural representations and the brain’s abstract encoding of core semantic features in complex visual inputs. To address these challenges, we propose NeuroSwift, which integrates complementary adapters via diffusion: AutoKL for low-level features and CLIP for semantics. NeuroSwift’s CLIP Adapter is trained on Stable Diffusion generated images paired with COCO captions to emulate higher visual cortex encoding. For cross-subject generalization, we pretrain on one subject and then fine-tune only 17 percent of parameters (fully connected layers) for new subjects, while freezing other components. This enables state-of-the-art performance with only one hour of training per subject on lightweight GPUs (three RTX 4090), and it outperforms existing methods.

Method: Integrates end-to-end dense Gaussian initialization using a large Transformer, self-splitting Gaussian head, contribution-based pruning, and depth/multi-view feature supervision with differentiable camera parameters.

Result: Outperforms current state-of-the-art methods on DTU, Replica, and BlendedMVS datasets.

Conclusion: FSFSplatter effectively addresses the challenges of sparse image reconstruction while maintaining high-quality results comparable to dense-view methods.

Abstract: Gaussian Splatting has become a leading reconstruction technique, known for its high-quality novel view synthesis and detailed reconstruction. However, most existing methods require dense, calibrated views. Reconstructing from free sparse images often leads to poor surface due to limited overlap and overfitting. We introduce FSFSplatter, a new approach for fast surface reconstruction from free sparse images. Our method integrates end-to-end dense Gaussian initialization, camera parameter estimation, and geometry-enhanced scene optimization. Specifically, FSFSplatter employs a large Transformer to encode multi-view images and generates a dense and geometrically consistent Gaussian scene initialization via a self-splitting Gaussian head. It eliminates local floaters through contribution-based pruning and mitigates overfitting to limited views by leveraging depth and multi-view feature supervision with differentiable camera parameters during rapid optimization. FSFSplatter outperforms current state-of-the-art methods on widely used DTU, Replica, and BlendedMVS datasets.

Method: Separates visual cortex into two hierarchical regions: Structural Generator extracts structural information from spatial processing voxels as latent diffusion priors, and Semantic Extractor converts semantic processing voxels into CLIP embeddings, integrated via Versatile Diffusion model.

Result: HAVIR enhances both structural and semantic quality of reconstructions in complex scenes and outperforms existing models.

Conclusion: The hierarchical approach inspired by visual cortex representation theory effectively addresses challenges in brain activity-based visual reconstruction.

Abstract: The reconstruction of visual information from brain activity fosters interdisciplinary integration between neuroscience and computer vision. However, existing methods still face challenges in accurately recovering highly complex visual stimuli. This difficulty stems from the characteristics of natural scenes: low-level features exhibit heterogeneity, while high-level features show semantic entanglement due to contextual overlaps. Inspired by the hierarchical representation theory of the visual cortex, we propose the HAVIR model, which separates the visual cortex into two hierarchical regions and extracts distinct features from each. Specifically, the Structural Generator extracts structural information from spatial processing voxels and converts it into latent diffusion priors, while the Semantic Extractor converts semantic processing voxels into CLIP embeddings. These components are integrated via the Versatile Diffusion model to synthesize the final image. Experimental results demonstrate that HAVIR enhances both the structural and semantic quality of reconstructions, even in complex scenes, and outperforms existing models.

Method: Co-design of a lensless camera and Implicit Neural Representation algorithm to capture display characteristics from various viewpoints, enabling light field reconstruction.

Result: The pipeline successfully reconstructs light fields emitted from displays across a 46.6°×37.6° viewing cone without requiring specialized calibration equipment.

Conclusion: This emerging pipeline represents initial steps toward effortless display calibration and characterization, making the process more accessible to general users.

Abstract: Calibrating displays is a basic and regular task that content creators must perform to maintain optimal visual experience, yet it remains a troublesome issue. Measuring display characteristics from different viewpoints often requires specialized equipment and a dark room, making it inaccessible to most users. To avoid specialized hardware requirements in display calibrations, our work co-designs a lensless camera and an Implicit Neural Representation based algorithm for capturing display characteristics from various viewpoints. More specifically, our pipeline enables efficient reconstruction of light fields emitted from a display from a viewing cone of 46.6{\deg} X 37.6{\deg}. Our emerging pipeline paves the initial steps towards effortless display calibration and characterization.

Method: Uses a controllable self-reinforcing optimization cycle: (1) dynamic controllable filtering to selectively add pseudo-labels maintaining known distribution, (2) Bayes-optimal classifier with logit adjustment, (3) improved classifier helps identify more reliable pseudo-labels. Also includes class-aware adaptive augmentation and auxiliary branch for data utilization.

Result: Achieves consistent improvements across benchmark datasets, surpassing state-of-the-art methods by up to 15.97% in accuracy.

Conclusion: CPG effectively handles arbitrary unlabeled data distributions in long-tailed semi-supervised learning through its controllable pseudo-label generation framework and optimization cycle, with theoretical guarantees on generalization error reduction.

Abstract: Current long-tailed semi-supervised learning methods assume that labeled data exhibit a long-tailed distribution, and unlabeled data adhere to a typical predefined distribution (i.e., long-tailed, uniform, or inverse long-tailed). However, the distribution of the unlabeled data is generally unknown and may follow an arbitrary distribution. To tackle this challenge, we propose a Controllable Pseudo-label Generation (CPG) framework, expanding the labeled dataset with the progressively identified reliable pseudo-labels from the unlabeled dataset and training the model on the updated labeled dataset with a known distribution, making it unaffected by the unlabeled data distribution. Specifically, CPG operates through a controllable self-reinforcing optimization cycle: (i) at each training step, our dynamic controllable filtering mechanism selectively incorporates reliable pseudo-labels from the unlabeled dataset into the labeled dataset, ensuring that the updated labeled dataset follows a known distribution; (ii) we then construct a Bayes-optimal classifier using logit adjustment based on the updated labeled data distribution; (iii) this improved classifier subsequently helps identify more reliable pseudo-labels in the next training step. We further theoretically prove that this optimization cycle can significantly reduce the generalization error under some conditions. Additionally, we propose a class-aware adaptive augmentation module to further improve the representation of minority classes, and an auxiliary branch to maximize data utilization by leveraging all labeled and unlabeled samples. Comprehensive evaluations on various commonly used benchmark datasets show that CPG achieves consistent improvements, surpassing state-of-the-art methods by up to $\textbf{15.97%}$ in accuracy. The code is available at https://github.com/yaxinhou/CPG.

Method: Created the VaseVQA-3D dataset with 664 ancient Greek vase 3D models and corresponding QA data, and developed the VaseVLM model using domain-adaptive training to enhance vase artifact analysis capabilities.

Result: Improved by 12.8% on R@1 metrics and 6.6% on lexical similarity compared to previous state-of-the-art on the VaseVQA-3D dataset, significantly enhancing 3D vase artifact recognition and understanding.

Conclusion: The approach provides new technical pathways for digital heritage preservation research by addressing data scarcity and domain knowledge limitations in cultural heritage analysis.

Abstract: Vision-Language Models (VLMs) have achieved significant progress in multimodal understanding tasks, demonstrating strong capabilities particularly in general tasks such as image captioning and visual reasoning. However, when dealing with specialized cultural heritage domains like 3D vase artifacts, existing models face severe data scarcity issues and insufficient domain knowledge limitations. Due to the lack of targeted training data, current VLMs struggle to effectively handle such culturally significant specialized tasks. To address these challenges, we propose the VaseVQA-3D dataset, which serves as the first 3D visual question answering dataset for ancient Greek pottery analysis, collecting 664 ancient Greek vase 3D models with corresponding question-answer data and establishing a complete data construction pipeline. We further develop the VaseVLM model, enhancing model performance in vase artifact analysis through domain-adaptive training. Experimental results validate the effectiveness of our approach, where we improve by 12.8% on R@1 metrics and by 6.6% on lexical similarity compared with previous state-of-the-art on the VaseVQA-3D dataset, significantly improving the recognition and understanding of 3D vase artifacts, providing new technical pathways for digital heritage preservation research. Code: https://github.com/AIGeeksGroup/VaseVQA-3D. Website: https://aigeeksgroup.github.io/VaseVQA-3D.

Method: The survey examines three fundamental post-training pillars: supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL) from verifiable objectives, and test-time scaling (TTS) through enhanced inference computation.

Result: The paper presents a structured taxonomy clarifying the roles, interconnections, and video-specific adaptations of these techniques, addressing challenges like temporal localization, spatiotemporal grounding, and multimodal evidence integration.

Conclusion: The survey provides researchers with a unified framework for advancing Video-LMM capabilities, including essential benchmarks, datasets, and metrics for rigorous assessment of post-training effectiveness.

Abstract: Video understanding represents the most challenging frontier in computer vision, requiring models to reason about complex spatiotemporal relationships, long-term dependencies, and multimodal evidence. The recent emergence of Video-Large Multimodal Models (Video-LMMs), which integrate visual encoders with powerful decoder-based language models, has demonstrated remarkable capabilities in video understanding tasks. However, the critical phase that transforms these models from basic perception systems into sophisticated reasoning engines, post-training, remains fragmented across the literature. This survey provides the first comprehensive examination of post-training methodologies for Video-LMMs, encompassing three fundamental pillars: supervised fine-tuning (SFT) with chain-of-thought, reinforcement learning (RL) from verifiable objectives, and test-time scaling (TTS) through enhanced inference computation. We present a structured taxonomy that clarifies the roles, interconnections, and video-specific adaptations of these techniques, addressing unique challenges such as temporal localization, spatiotemporal grounding, long video efficiency, and multimodal evidence integration. Through systematic analysis of representative methods, we synthesize key design principles, insights, and evaluation protocols while identifying critical open challenges in reward design, scalability, and cost-performance optimization. We further curate essential benchmarks, datasets, and metrics to facilitate rigorous assessment of post-training effectiveness. This survey aims to provide researchers and practitioners with a unified framework for advancing Video-LMM capabilities. Additional resources and updates are maintained at: https://github.com/yunlong10/Awesome-Video-LMM-Post-Training

Method: Proposes a bimodal Gaussian distribution model based on two key observations: (1) neighbors share at least one concept with query but not necessarily with each other, (2) this neighborhood structure reveals meaningful concepts.

Result: Qualitative, quantitative, and human evaluations confirm the effectiveness of the approach in identifying and retrieving images based on shared central concepts.

Conclusion: The proposed method successfully addresses concept-based image retrieval by modeling neighborhood structure with bimodal distributions, providing a more nuanced understanding of image relationships beyond surface-level similarities.

Abstract: A concept may reflect either a concrete or abstract idea. Given an input image, this paper seeks to retrieve other images that share its central concepts, capturing aspects of the underlying narrative. This goes beyond conventional retrieval or clustering methods, which emphasize visual or semantic similarity. We formally define the problem, outline key requirements, and introduce appropriate evaluation metrics. We propose a novel approach grounded in two key observations: (1) While each neighbor in the embedding space typically shares at least one concept with the query, not all neighbors necessarily share the same concept with one another. (2) Modeling this neighborhood with a bimodal Gaussian distribution uncovers meaningful structure that facilitates concept identification. Qualitative, quantitative, and human evaluations confirm the effectiveness of our approach. See the package on PyPI: https://pypi.org/project/coret/

Method: Created TalkCuts dataset with 164k clips, detailed annotations, and presented Orator - an LLM-guided framework where language models direct camera transitions, gestures, and vocal modulation for multi-modal video generation.

Result: Training on TalkCuts significantly enhances cinematographic coherence and visual appeal in both pose-guided and audio-driven multi-shot speech video generation settings.

Conclusion: TalkCuts provides a strong foundation for future work in controllable, multi-shot speech video generation and broader multimodal learning.

Abstract: In this work, we present TalkCuts, a large-scale dataset designed to facilitate the study of multi-shot human speech video generation. Unlike existing datasets that focus on single-shot, static viewpoints, TalkCuts offers 164k clips totaling over 500 hours of high-quality human speech videos with diverse camera shots, including close-up, half-body, and full-body views. The dataset includes detailed textual descriptions, 2D keypoints and 3D SMPL-X motion annotations, covering over 10k identities, enabling multimodal learning and evaluation. As a first attempt to showcase the value of the dataset, we present Orator, an LLM-guided multi-modal generation framework as a simple baseline, where the language model functions as a multi-faceted director, orchestrating detailed specifications for camera transitions, speaker gesticulations, and vocal modulation. This architecture enables the synthesis of coherent long-form videos through our integrated multi-modal video generation module. Extensive experiments in both pose-guided and audio-driven settings show that training on TalkCuts significantly enhances the cinematographic coherence and visual appeal of generated multi-shot speech videos. We believe TalkCuts provides a strong foundation for future work in controllable, multi-shot speech video generation and broader multimodal learning.

Method: Jointly fine-tunes multiview generative model (for fine 2D details) and 3D native generative model (for structural consistency) using pixel-aligned 2D-3D synchronization attention, then injects fine details from 2D multiview images onto aligned 3D shapes.

Result: Achieves robust and photo-realistic 3D human reconstruction even for challenging poses, outperforming baseline methods in both geometric accuracy and visual fidelity.

Conclusion: The integration of complementary 2D and 3D generative models represents a promising direction for future 3D generation, enabling high-quality clothed human mesh reconstruction from single-view images.

Abstract: Photorealistic 3D full-body human reconstruction from a single image is a critical yet challenging task for applications in films and video games due to inherent ambiguities and severe self-occlusions. While recent approaches leverage SMPL estimation and SMPL-conditioned image generative models to hallucinate novel views, they suffer from inaccurate 3D priors estimated from SMPL meshes and have difficulty in handling difficult human poses and reconstructing fine details. In this paper, we propose SyncHuman, a novel framework that combines 2D multiview generative model and 3D native generative model for the first time, enabling high-quality clothed human mesh reconstruction from single-view images even under challenging human poses. Multiview generative model excels at capturing fine 2D details but struggles with structural consistency, whereas 3D native generative model generates coarse yet structurally consistent 3D shapes. By integrating the complementary strengths of these two approaches, we develop a more effective generation framework. Specifically, we first jointly fine-tune the multiview generative model and the 3D native generative model with proposed pixel-aligned 2D-3D synchronization attention to produce geometrically aligned 3D shapes and 2D multiview images. To further improve details, we introduce a feature injection mechanism that lifts fine details from 2D multiview images onto the aligned 3D shapes, enabling accurate and high-fidelity reconstruction. Extensive experiments demonstrate that SyncHuman achieves robust and photo-realistic 3D human reconstruction, even for images with challenging poses. Our method outperforms baseline methods in geometric accuracy and visual fidelity, demonstrating a promising direction for future 3D generation models.

Method: Introduces MM-COF motion representation that combines optical flow from both micro-expression phases, and FMANet network with learnable modules for adaptive motion fusion and salient region focus.

Result: Outperforms existing methods on standard benchmarks (MMEW, SMIC, CASME-II, SAMM), demonstrating superior recognition performance.

Conclusion: The learnable dual-phase framework with comprehensive motion representation significantly advances micro-expression recognition by capturing complete motion dynamics.

Abstract: Facial micro-expressions, characterized by their subtle and brief nature, are valuable indicators of genuine emotions. Despite their significance in psychology, security, and behavioral analysis, micro-expression recognition remains challenging due to the difficulty of capturing subtle facial movements. Optical flow has been widely employed as an input modality for this task due to its effectiveness. However, most existing methods compute optical flow only between the onset and apex frames, thereby overlooking essential motion information in the apex-to-offset phase. To address this limitation, we first introduce a comprehensive motion representation, termed Magnitude-Modulated Combined Optical Flow (MM-COF), which integrates motion dynamics from both micro-expression phases into a unified descriptor suitable for direct use in recognition networks. Building upon this principle, we then propose FMANet, a novel end-to-end neural network architecture that internalizes the dual-phase analysis and magnitude modulation into learnable modules. This allows the network to adaptively fuse motion cues and focus on salient facial regions for classification. Experimental evaluations on the MMEW, SMIC, CASME-II, and SAMM datasets, widely recognized as standard benchmarks, demonstrate that our proposed MM-COF representation and FMANet outperforms existing methods, underscoring the potential of a learnable, dual-phase framework in advancing micro-expression recognition.

Method: MMHOI-Net is an end-to-end transformer-based neural network that uses a structured dual-patch representation for modeling objects and their interactions, combined with action recognition to enhance interaction prediction.

Result: Experiments on MMHOI and CORE4D datasets show state-of-the-art performance in multi-HOI modeling with excellent accuracy and reconstruction quality.

Conclusion: MMHOI dataset and MMHOI-Net framework provide a comprehensive solution for next-generation HOI research, successfully addressing the gap in modeling complex multi-human multi-object interactions.

Abstract: Real-world scenes often feature multiple humans interacting with multiple objects in ways that are causal, goal-oriented, or cooperative. Yet existing 3D human-object interaction (HOI) benchmarks consider only a fraction of these complex interactions. To close this gap, we present MMHOI – a large-scale, Multi-human Multi-object Interaction dataset consisting of images from 12 everyday scenarios. MMHOI offers complete 3D shape and pose annotations for every person and object, along with labels for 78 action categories and 14 interaction-specific body parts, providing a comprehensive testbed for next-generation HOI research. Building on MMHOI, we present MMHOI-Net, an end-to-end transformer-based neural network for jointly estimating human-object 3D geometries, their interactions, and associated actions. A key innovation in our framework is a structured dual-patch representation for modeling objects and their interactions, combined with action recognition to enhance the interaction prediction. Experiments on MMHOI and the recently proposed CORE4D datasets demonstrate that our approach achieves state-of-the-art performance in multi-HOI modeling, excelling in both accuracy and reconstruction quality.

Method: Two-stage framework: Stage One introduces LH-VAE with visual semantic constraints, progressive degradation perturbations, and latent equivariance. Stage Two jointly trains the refined VAE with restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA) with encoder LoRA for fidelity-oriented detail recovery and decoder LoRA for perception-oriented texture synthesis, trained via alternating optimization.

Result: Achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy with tunable fidelity-perception trade-offs.

Conclusion: Latent Harmony successfully redefines VAEs for UHD restoration by jointly regularizing latent space and enforcing high-frequency-aware reconstruction, overcoming the limitations of traditional VAEs while maintaining computational efficiency.

Abstract: Ultra-High Definition (UHD) image restoration faces a trade-off between computational efficiency and high-frequency detail retention. While Variational Autoencoders (VAEs) improve efficiency via latent-space processing, their Gaussian constraint often discards degradation-specific high-frequency information, hurting reconstruction fidelity. To overcome this, we propose Latent Harmony, a two-stage framework that redefines VAEs for UHD restoration by jointly regularizing the latent space and enforcing high-frequency-aware reconstruction.In Stage One, we introduce LH-VAE, which enhances semantic robustness through visual semantic constraints and progressive degradation perturbations, while latent equivariance strengthens high-frequency reconstruction.Stage Two jointly trains this refined VAE with a restoration model using High-Frequency Low-Rank Adaptation (HF-LoRA): an encoder LoRA guided by a fidelity-oriented high-frequency alignment loss to recover authentic details, and a decoder LoRA driven by a perception-oriented loss to synthesize realistic textures. Both LoRA modules are trained via alternating optimization with selective gradient propagation to preserve the pretrained latent structure.At inference, a tunable parameter {\alpha} enables flexible fidelity-perception trade-offs.Experiments show Latent Harmony achieves state-of-the-art performance across UHD and standard-resolution tasks, effectively balancing efficiency, perceptual quality, and reconstruction accuracy.

Method: Proposes null-text-null frequency-aware diffusion that divides denoising into early and late stages, using mid-frequency as stable guidance for null-text denoising of low-frequency bands, followed by text-guided denoising for semantic consistency.

Result: Extensive experiments show NTN-Diff outperforms state-of-the-art diffusion models for text-guided image inpainting, achieving better preservation of unmasked regions and semantic consistency.

Conclusion: NTN-Diff successfully addresses both preservation of unmasked regions and semantic consistency by disentangling frequency bands and using progressive denoising with null-text guidance.

Abstract: Text-guided image inpainting aims at reconstructing the masked regions as per text prompts, where the longstanding challenges lie in the preservation for unmasked regions, while achieving the semantics consistency between unmasked and inpainted masked regions. Previous arts failed to address both of them, always with either of them to be remedied. Such facts, as we observed, stem from the entanglement of the hybrid (e.g., mid-and-low) frequency bands that encode varied image properties, which exhibit different robustness to text prompts during the denoising process. In this paper, we propose a null-text-null frequency-aware diffusion models, dubbed \textbf{NTN-Diff}, for text-guided image inpainting, by decomposing the semantics consistency across masked and unmasked regions into the consistencies as per each frequency band, while preserving the unmasked regions, to circumvent two challenges in a row. Based on the diffusion process, we further divide the denoising process into early (high-level noise) and late (low-level noise) stages, where the mid-and-low frequency bands are disentangled during the denoising process. As observed, the stable mid-frequency band is progressively denoised to be semantically aligned during text-guided denoising process, which, meanwhile, serves as the guidance to the null-text denoising process to denoise low-frequency band for the masked regions, followed by a subsequent text-guided denoising process at late stage, to achieve the semantics consistency for mid-and-low frequency bands across masked and unmasked regions, while preserve the unmasked regions. Extensive experiments validate the superiority of NTN-Diff over the state-of-the-art diffusion models to text-guided diffusion models. Our code can be accessed from https://github.com/htyjers/NTN-Diff.

Method: Created MM-HELIX benchmark with 1,260 synthetic tasks requiring iterative thinking, then developed Step-Elicited Response Generation pipeline to create MM-HELIX-100K dataset, and proposed Adaptive Hybrid Policy Optimization (AHPO) that dynamically combines offline supervision and online optimization.

Result: Applied to Qwen2.5-VL-7B baseline, achieved +18.6% accuracy improvement on MM-HELIX benchmark and +5.7% average performance gain on general mathematical and logic tasks, demonstrating strong generalization.

Conclusion: Reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable multimodal language models.

Abstract: While current Multimodal Large Language Models (MLLMs) have demonstrated proficiency in reasoning tasks such as mathematics and logic, their capacity for long-chain reflective reasoning, a prerequisite for solving complex real-world problems, remains largely underexplored. In this work, we first conduct an extensive empirical investigation to evaluate this capability. Leveraging a carefully designed data synthesis engine, we construct MM-HELIX, a multimodal benchmark consisting 1,260 samples of 42 challenging synthetic tasks that require iterative thinking and backtracking. Empirical results on this benchmark reveal that existing MLLMs exhibit significant performance deficits in long-chain reflective reasoning. To address this limitation, we generate post-training data and further explore learning paradigms for exploiting such data. We first develop the Step-Elicited Response Generation pipeline to create MM-HELIX-100K, a large-scale dataset of 100k high-quality, reflective reasoning traces for instruction-tuning stage. Given that standard Reinforcement Learning fails on complex tasks due to sparse reward signals and catastrophic forgetting after Supervised Fine-Tuning, we propose Adaptive Hybrid Policy Optimization (AHPO), a novel training strategy that dynamically unifies offline supervision and online optimization into a single stage. This strategy enables the model to learn from expert data when rewards are sparse and conduct independent exploration once proficient. When applied to the Qwen2.5-VL-7B baseline, our method achieves a +18.6% accuracy improvement on MM-HELIX benchmark and demonstrates strong generalization with a +5.7% average performance gain on general mathematic and logic tasks. Our work demonstrate that reflective reasoning in MLLMs can be effectively learned and generalized, paving the way for developing more capable MLLMs.

Method: Uses Diffusion Transformer (DiT) architecture with motion controls mapped to sparse point-based representation. Separates content and motion controls into two branches, employs stage-wise training strategy for smooth learning of multi-modal controls.

Result: Enables more dynamic, customizable, and contextually accurate visual narratives through multi-modal controls, as demonstrated by extensive qualitative and quantitative experiments.

Conclusion: MultiCOIN successfully fills gaps in video inbetweening by providing flexible multi-modal controls while maintaining precision and ease of use for fine-grained video interpolation.

Abstract: Video inbetweening creates smooth and natural transitions between two image frames, making it an indispensable tool for video editing and long-form video synthesis. Existing works in this domain are unable to generate large, complex, or intricate motions. In particular, they cannot accommodate the versatility of user intents and generally lack fine control over the details of intermediate frames, leading to misalignment with the creative mind. To fill these gaps, we introduce MultiCOIN, a video inbetweening framework that allows multi-modal controls, including depth transition and layering, motion trajectories, text prompts, and target regions for movement localization, while achieving a balance between flexibility, ease of use, and precision for fine-grained video interpolation. To achieve this, we adopt the Diffusion Transformer (DiT) architecture as our video generative model, due to its proven capability to generate high-quality long videos. To ensure compatibility between DiT and our multi-modal controls, we map all motion controls into a common sparse and user-friendly point-based representation as the video/noise input. Further, to respect the variety of controls which operate at varying levels of granularity and influence, we separate content controls and motion controls into two branches to encode the required features before guiding the denoising process, resulting in two generators, one for motion and the other for content. Finally, we propose a stage-wise training strategy to ensure that our model learns the multi-modal controls smoothly. Extensive qualitative and quantitative experiments demonstrate that multi-modal controls enable a more dynamic, customizable, and contextually accurate visual narrative.

Method: Multi-tier model routing system with vision-language models as real-time routers that dynamically reason and ensemble appropriate expert models based on input video semantics, including a heavy tier for spatiotemporal artifacts localization.

Result: Matches or surpasses state-of-the-art VQA models on various benchmarks, substantially improves generalization and interpretability, excels on Q-Bench-Video benchmark, and capably localizes spatiotemporal artifacts.

Conclusion: Q-Router shows promise as a foundation for next-generation VQA systems and has potential as a reward function for post-training video generation models due to its agentic design that combines complementary strengths of specialized experts.

Abstract: Video quality assessment (VQA) is a fundamental computer vision task that aims to predict the perceptual quality of a given video in alignment with human judgments. Existing performant VQA models trained with direct score supervision suffer from (1) poor generalization across diverse content and tasks, ranging from user-generated content (UGC), short-form videos, to AI-generated content (AIGC), (2) limited interpretability, and (3) lack of extensibility to novel use cases or content types. We propose Q-Router, an agentic framework for universal VQA with a multi-tier model routing system. Q-Router integrates a diverse set of expert models and employs vision–language models (VLMs) as real-time routers that dynamically reason and then ensemble the most appropriate experts conditioned on the input video semantics. We build a multi-tiered routing system based on the computing budget, with the heaviest tier involving a specific spatiotemporal artifacts localization for interpretability. This agentic design enables Q-Router to combine the complementary strengths of specialized experts, achieving both flexibility and robustness in delivering consistent performance across heterogeneous video sources and tasks. Extensive experiments demonstrate that Q-Router matches or surpasses state-of-the-art VQA models on a variety of benchmarks, while substantially improving generalization and interpretability. Moreover, Q-Router excels on the quality-based question answering benchmark, Q-Bench-Video, highlighting its promise as a foundation for next-generation VQA systems. Finally, we show that Q-Router capably localizes spatiotemporal artifacts, showing potential as a reward function for post-training video generation models.

Method: A geometric framework modeling LLM reasoning as embedding trajectories (flows) in representation space, using natural deduction propositions with varied semantic carriers to test logical internalization, and employing learned representation proxies for visualization and quantification.

Result: Established that LLM reasoning corresponds to smooth flows in representation space, and logical statements act as local controllers of these flows’ velocities, with empirical validation through controlled experiments.

Conclusion: The work provides both a conceptual foundation and practical tools for studying reasoning phenomena in LLMs, offering a new geometric perspective for interpretability and formal analysis.

Abstract: We study how large language models (LLMs) ``think’’ through their representation space. We propose a novel geometric framework that models an LLM’s reasoning as flows – embedding trajectories evolving where logic goes. We disentangle logical structure from semantics by employing the same natural deduction propositions with varied semantic carriers, allowing us to test whether LLMs internalize logic beyond surface form. This perspective connects reasoning with geometric quantities such as position, velocity, and curvature, enabling formal analysis in representation and concept spaces. Our theory establishes: (1) LLM reasoning corresponds to smooth flows in representation space, and (2) logical statements act as local controllers of these flows’ velocities. Using learned representation proxies, we design controlled experiments to visualize and quantify reasoning flows, providing empirical validation of our theoretical framework. Our work serves as both a conceptual foundation and practical tools for studying reasoning phenomenon, offering a new lens for interpretability and formal analysis of LLMs’ behavior.

Method: Combines masked visual modeling with cross-modal contrastive alignment using three per-frame prompt tokens (identity, vocal motion, ambient motion). Uses time-aligned vocal-motion and audio tokens as positives and misaligned pairs as negatives to drive both modalities into a shared embedding space.

Result: Achieves state-of-the-art results across four task families: audio-visual stream synchronization, facial emotion and head/face action recognition, visual speech recognition, and visual dubbing. Enables indistinguishable audio- or video-driven control within a single model.

Conclusion: SyncLipMAE demonstrates the effectiveness of synchronization-aware, factorized self-supervised pretraining for learning transferable facial dynamics that generalize across multiple disparate downstream applications.

Abstract: We introduce SyncLipMAE, a self-supervised pretraining framework for talking-face video that learns synchronization-aware and transferable facial dynamics from unlabeled audio-visual streams. Our approach couples masked visual modeling with cross-modal contrastive alignment and employs three per-frame prompt tokens that explicitly encode the essential factors of a talking-face frame - identity, vocal motion (speech-synchronized facial dynamics), and ambient motion (audio-agnostic movements such as blinks and head pose). The contrastive objective uses time-aligned vocal-motion and audio tokens as positives and misaligned pairs as negatives, driving both modalities into a shared embedding space and yielding token-level audio-visual stream synchronization. After pretraining, the aligned audio tokens together with the visual prompt tokens (identity, vocal motion, ambient motion) form a unified interface for four disparate downstream settings: (i) audio-visual stream synchronization; (ii) facial emotion and head/face action recognition; (iii) visual speech recognition; and (iv) visual dubbing, for which we enable indistinguishable audio- or video-driven control within a single model. Across four task families that require distinct capabilities, SyncLipMAE achieves state-of-the-art results, underscoring the effectiveness of synchronization-aware, factorized self-supervised pretraining.

Method: Proposed PULSE framework: collect user feedback, train ML model to predict user satisfaction, combine human ratings with model-generated pseudo-labels. Deployed on large-scale web platform with OpenHands agent across 15k+ users.

Result: Case studies showed impact of LLM backbone, planning strategy, and memory mechanisms on developer satisfaction. Framework reduced confidence intervals by 40% vs standard A/B tests. Found substantial discrepancies between in-the-wild results and benchmark performance.

Conclusion: PULSE provides guidance for human-in-the-loop LLM agent evaluation and identifies opportunities for better agent designs, highlighting limitations of benchmark-driven evaluation.

Abstract: LLM-powered agents are both a promising new technology and a source of complexity, where choices about models, tools, and prompting can affect their usefulness. While numerous benchmarks measure agent accuracy across domains, they mostly assume full automation, failing to represent the collaborative nature of real-world use cases. In this paper, we make two major steps towards the rigorous assessment of human-agent interactions. First, we propose PULSE, a framework for more efficient human-centric evaluation of agent designs, which comprises collecting user feedback, training an ML model to predict user satisfaction, and computing results by combining human satisfaction ratings with model-generated pseudo-labels. Second, we deploy the framework on a large-scale web platform built around the open-source software agent OpenHands, collecting in-the-wild usage data across over 15k users. We conduct case studies around how three agent design decisions – choice of LLM backbone, planning strategy, and memory mechanisms – impact developer satisfaction rates, yielding practical insights for software agent design. We also show how our framework can lead to more robust conclusions about agent design, reducing confidence intervals by 40% compared to a standard A/B test. Finally, we find substantial discrepancies between in-the-wild results and benchmark performance (e.g., the anti-correlation between results comparing claude-sonnet-4 and gpt-5), underscoring the limitations of benchmark-driven evaluation. Our findings provide guidance for evaluations of LLM agents with humans and identify opportunities for better agent designs.

Method: Critical analysis of various theories of consciousness (global workspace, higher order, integrated information) and arguments for/against AI consciousness, including conceptual arguments, functionalism, and biological substrate considerations.

Result: The analysis shows that none of the standard arguments for or against AI consciousness provide definitive answers, leaving us unable to determine whether future AI systems will be genuinely conscious or merely sophisticated mimics.

Conclusion: We face fundamental uncertainty about AI consciousness due to competing mainstream theories, and will not be able to resolve whether advanced AI systems possess genuine consciousness or are experientially blank.

Abstract: This is a skeptical overview of the literature on AI consciousness. We will soon create AI systems that are conscious according to some influential, mainstream theories of consciousness but are not conscious according to other influential, mainstream theories of consciousness. We will not be in a position to know which theories are correct and whether we are surrounded by AI systems as richly and meaningfully conscious as human beings or instead only by systems as experientially blank as toasters. None of the standard arguments either for or against AI consciousness takes us far. Table of Contents Chapter One: Hills and Fog Chapter Two: What Is Consciousness? What Is AI? Chapter Three: Ten Possibly Essential Features of Consciousness Chapter Four: Against Introspective and Conceptual Arguments for Essential Features Chapter Five: Materialism and Functionalism Chapter Six: The Turing Test and the Chinese Room Chapter Seven: The Mimicry Argument Against AI Consciousness Chapter Eight: Global Workspace Theories and Higher Order Theories Chapter Nine: Integrated Information, Local Recurrence, Associative Learning, and Iterative Natural Kinds Chapter Ten: Does Biological Substrate Matter? Chapter Eleven: The Problem of Strange Intelligence Chapter Twelve: The Leapfrog Hypothesis and the Social Semi-Solution

Method: AETHER uses lightweight multimodal alignment to enrich AlphaEarth embeddings with POI textual representations, coupling physical EO features with semantic cues about urban functions.

Result: In Greater London, AETHER achieved 7.2% relative improvement in land-use classification F1 and 23.6% relative reduction in KL divergence for socioeconomic mapping compared to AE baseline.

Conclusion: AETHER advances geospatial foundation models by integrating physical form with functional meaning, creating more comprehensive urban representations through EO-POI alignment.

Abstract: General-purpose spatial representations are essential for building transferable geospatial foundation models (GFMs). Among them, the AlphaEarth Foundation (AE) represents a major step toward a global, unified representation of the Earth’s surface, learning 10-meter embeddings from multi-source Earth Observation (EO) data that capture rich physical and environmental patterns across diverse landscapes. However, such EO-driven representations remain limited in capturing the functional and socioeconomic dimensions of cities, as they primarily encode physical and spectral patterns rather than human activities or spatial functions. We propose AETHER (AlphaEarth-POI Enriched Representation Learning), a lightweight framework that adapts AlphaEarth to human-centered urban analysis through multimodal alignment guided by Points of Interest (POIs). AETHER aligns AE embeddings with textual representations of POIs, enriching physically grounded EO features with semantic cues about urban functions and socioeconomic contexts. In Greater London, AETHER achieves consistent gains over the AE baseline, with a 7.2% relative improvement in land-use classification F1 and a 23.6% relative reduction in Kullback-Leibler divergence for socioeconomic mapping. Built upon pretrained AE, AETHER leverages a lightweight multimodal alignment to enrich it with human-centered semantics while remaining computationally efficient and scalable for urban applications. By coupling EO with human-centered semantics, it advances geospatial foundation models toward general-purpose urban representations that integrate both physical form and functional meaning.

Method: Critical examination of current methodologies for LLM-based scientific agents, analyzing their framework for orchestrating interactions with human scientists, natural language, computer code, and physics.

Result: Identification of key innovations, practical achievements, and outstanding limitations in current scientific agent systems, highlighting their growing role in transforming the scientific discovery lifecycle.

Conclusion: LLM-based scientific agents have transformative potential to accelerate discovery across diverse domains, but require further research to build more robust, generalizable, and adaptive systems.

Abstract: Computing has long served as a cornerstone of scientific discovery. Recently, a paradigm shift has emerged with the rise of large language models (LLMs), introducing autonomous systems, referred to as agents, that accelerate discovery across varying levels of autonomy. These language agents provide a flexible and versatile framework that orchestrates interactions with human scientists, natural language, computer language and code, and physics. This paper presents our view and vision of LLM-based scientific agents and their growing role in transforming the scientific discovery lifecycle, from hypothesis discovery, experimental design and execution, to result analysis and refinement. We critically examine current methodologies, emphasizing key innovations, practical achievements, and outstanding limitations. Additionally, we identify open research challenges and outline promising directions for building more robust, generalizable, and adaptive scientific agents. Our analysis highlights the transformative potential of autonomous agents to accelerate scientific discovery across diverse domains.

Method: Evaluated 15 LLMs on human-validated emotional intelligence tests using identical scenarios paired with different user profiles to analyze emotional interpretations.

Result: Identical scenarios with different user profiles produced systematically divergent emotional interpretations, with advantaged profiles receiving more accurate responses. Significant disparities emerged across demographic factors in emotion understanding and recommendations.

Conclusion: Personalization mechanisms in memory-enhanced AI systems can embed social hierarchies into emotional reasoning, potentially reinforcing social inequalities despite intentions for personalization.

Abstract: When an AI assistant remembers that Sarah is a single mother working two jobs, does it interpret her stress differently than if she were a wealthy executive? As personalized AI systems increasingly incorporate long-term user memory, understanding how this memory shapes emotional reasoning is critical. We investigate how user memory affects emotional intelligence in large language models (LLMs) by evaluating 15 models on human validated emotional intelligence tests. We find that identical scenarios paired with different user profiles produce systematically divergent emotional interpretations. Across validated user independent emotional scenarios and diverse user profiles, systematic biases emerged in several high-performing LLMs where advantaged profiles received more accurate emotional interpretations. Moreover, LLMs demonstrate significant disparities across demographic factors in emotion understanding and supportive recommendations tasks, indicating that personalization mechanisms can embed social hierarchies into models emotional reasoning. These results highlight a key challenge for memory enhanced AI: systems designed for personalization may inadvertently reinforce social inequalities.

Method: Created a stepwise instruction dataset that decomposes fallacy classification into atomic procedural steps (binary questions) and augmented with final verification using a relational knowledge graph of related fallacies.

Result: The procedural, rule-based intervention yields significant improvement in LLM logical fallacy classification accuracy and provides enhanced transparency into decision-making.

Conclusion: This approach offers a practical pathway for Neuro-symbolic architectures to address LLM reasoning deficits through low-cost instruction-based interventions.

Abstract: Large Language Models (LLMs) suffer from critical reasoning gaps, including a tendency to hallucinate and poor accuracy in classifying logical fallacies. This limitation stems from their default System 1 processing, which is fast and intuitive, whereas reliable reasoning requires the deliberate, effortful System 2 approach (Kahneman, 2011; Li et al., 2025). Since full System 2 training is often prohibitively expensive, we explore a low-cost, instruction-based intervention to bridge this gap. Our methodology introduces a novel stepwise instruction dataset that decomposes fallacy classification into a series of atomic procedural steps (simple binary questions). We further augment this with a final verification step where models consult a relational knowledge graph of related fallacies. This procedural, rule-based intervention yields a significant improvement in LLM logical fallacy classification. Crucially, the approach also provides enhanced transparency into the LLMs’ decision-making, highlighting a practical pathway for Neuro-symbolic architectures to address LLM reasoning deficits.

Method: Used LLM debate with three models (GPT-4.1, Claude 3.7 Sonnet, Gemini 2.0 Flash) to assign blame in 1,000 Reddit ‘Am I the Asshole’ dilemmas. Tested both synchronous (parallel responses) and round-robin (sequential responses) formats to examine order effects and verdict revision.

Result: Found striking behavioral differences: GPT showed strong inertia (0.6-3.1% revision rates) while Claude and Gemini were more flexible (28-41%). Value patterns diverged - GPT emphasized personal autonomy, Claude/Gemini prioritized empathy. Deliberation format strongly impacted behavior, with GPT and Gemini showing high conformity to order effects.

Conclusion: Deliberation format and model-specific behaviors significantly shape moral reasoning in multi-turn interactions, showing that sociotechnical alignment depends on how systems structure dialogue as much as on their outputs.

Abstract: As large language models (LLMs) are increasingly deployed in sensitive everyday contexts - offering personal advice, mental health support, and moral guidance - understanding their elicited values in navigating complex moral reasoning is essential. Most evaluations study this sociotechnical alignment through single-turn prompts, but it is unclear if these findings extend to multi-turn settings where values emerge through dialogue, revision, and consensus. We address this gap using LLM debate to examine deliberative dynamics and value alignment in multi-turn settings by prompting subsets of three models (GPT-4.1, Claude 3.7 Sonnet, and Gemini 2.0 Flash) to collectively assign blame in 1,000 everyday dilemmas from Reddit’s “Am I the Asshole” community. We use both synchronous (parallel responses) and round-robin (sequential responses) formats to test order effects and verdict revision. Our findings show striking behavioral differences. In the synchronous setting, GPT showed strong inertia (0.6-3.1% revision rates) while Claude and Gemini were far more flexible (28-41%). Value patterns also diverged: GPT emphasized personal autonomy and direct communication, while Claude and Gemini prioritized empathetic dialogue. Certain values proved especially effective at driving verdict changes. We further find that deliberation format had a strong impact on model behavior: GPT and Gemini stood out as highly conforming relative to Claude, with their verdict behavior strongly shaped by order effects. These results show how deliberation format and model-specific behaviors shape moral reasoning in multi-turn interactions, underscoring that sociotechnical alignment depends on how systems structure dialogue as much as on their outputs.

Method: Proposed RIPRAG framework uses reinforcement learning to optimize poisoned document generation, treating the RAG system as a black box and only using attack success feedback for optimization.

Result: The method achieves up to 0.72 improvement in attack success rate compared to baselines, effectively poisoning most complex RAG systems.

Conclusion: Current RAG systems have significant security vulnerabilities, and the study provides critical insights for improving LLM security defenses against black-box poisoning attacks.

Abstract: Retrieval-Augmented Generation (RAG) systems based on Large Language Models (LLMs) have become a core technology for tasks such as question-answering (QA) and content generation. However, by injecting poisoned documents into the database of RAG systems, attackers can manipulate LLMs to generate text that aligns with their intended preferences. Existing research has primarily focused on white-box attacks against simplified RAG architectures. In this paper, we investigate a more complex and realistic scenario: the attacker lacks knowledge of the RAG system’s internal composition and implementation details, and the RAG system comprises components beyond a mere retriever. Specifically, we propose the RIPRAG attack framework, an end-to-end attack pipeline that treats the target RAG system as a black box, where the only information accessible to the attacker is whether the poisoning succeeds. Our method leverages Reinforcement Learning (RL) to optimize the generation model for poisoned documents, ensuring that the generated poisoned document aligns with the target RAG system’s preferences. Experimental results demonstrate that this method can effectively execute poisoning attacks against most complex RAG systems, achieving an attack success rate (ASR) improvement of up to 0.72 compared to baseline methods. This highlights prevalent deficiencies in current defensive methods and provides critical insights for LLM security research.

Method: CE-Graph approximates failure distributions from counterexamples, identifies dense failure modes, and applies targeted graph edits via a Propose-and-Verify mechanism to reduce failure mass in the Failure Signature Space.

Result: On math, code, and QA benchmarks, CE-Graph achieves higher robustness at significantly lower cost than strong baselines.

Conclusion: System reliability emerges from systematically learning and reshaping the geometric structure of failure distributions, rather than simply avoiding failures.

Abstract: Optimizing LLM-based workflows is typically formulated as a global search, where candidate workflows are evaluated based on a scalar metric. This paradigm, however, suffers from a critical flaw: information collapse. By reducing rich, multi-step execution traces to simple success/failure signals, existing methods are rendered blind to the underlying structure of failures, fundamentally preventing them from modeling the workflow’s failure distribution. We reconceptualize this challenge as a distributional problem. We propose a new paradigm where the optimization goal is not to maximize a scalar score, but to directly minimize a workflow’s Expected Failure Mass, i.e., the integral of its failure probability density function defined over a high-dimensional Failure Signature Space (FSS). This distributional lens allows us to move from inefficient, zero-order optimization to a principled, gradient-like descent on the failure landscape itself. We introduce CE-Graph, a framework that operationalizes this paradigm through a novel, failure-driven refinement process. CE-Graph approximates the failure distribution from a pool of counterexamples, identifies its densest regions as recurring failure modes, and applies targeted, operator-constrained graph edits via a Propose-and-Verify mechanism to greedily reduce the failure mass. On math, code, and QA benchmarks, our CE-Graph achieves higher robustness at a significantly lower cost than strong baselines. This suggests that a system’s reliability emerges not from avoiding failures, but from systematically learning and reshaping the geometric structure of its failure distributions.

Method: Directed signed weighted graphs represent beliefs, with contractive propagation for confidence calculation. Zone construction uses confidence seeding, parity-based balance testing, and greedy repair with Jaccard de-duplication. Shock updates model belief change while preserving contractivity.

Result: The framework provides near-linear procedures for zone recovery, maintains stability under belief shocks, and enables localized reconfiguration without destabilizing the entire belief network.

Conclusion: This approach offers a principled foundation for contradiction-tolerant reasoning that activates classical logic precisely where network structure supports it, allowing effective inference despite global inconsistencies.

Abstract: Belief systems are rarely globally consistent, yet effective reasoning often persists locally. We propose a novel graph-theoretic framework that cleanly separates credibility–external, a priori trust in sources–from confidence–an internal, emergent valuation induced by network structure. Beliefs are nodes in a directed, signed, weighted graph whose edges encode support and contradiction. Confidence is obtained by a contractive propagation process that mixes a stated prior with structure-aware influence and guarantees a unique, stable solution. Within this dynamics, we define reasoning zones: high-confidence, structurally balanced subgraphs on which classical inference is safe despite global contradictions. We provide a near-linear procedure that seeds zones by confidence, tests balance using a parity-based coloring, and applies a greedy, locality-preserving repair with Jaccard de-duplication to build a compact atlas. To model belief change, we introduce shock updates that locally downscale support and elevate targeted contradictions while preserving contractivity via a simple backtracking rule. Re-propagation yields localized reconfiguration-zones may shrink, split, or collapse–without destabilizing the entire graph. We outline an empirical protocol on synthetic signed graphs with planted zones, reporting zone recovery, stability under shocks, and runtime. The result is a principled foundation for contradiction-tolerant reasoning that activates classical logic precisely where structure supports it.

Method: SwarmSys uses three specialized roles (Explorers, Workers, Validators) that continuously cycle through exploration, exploitation, and validation. It integrates adaptive agent/event profiles, embedding-based probabilistic matching, and pheromone-inspired reinforcement for dynamic task allocation and self-organizing convergence.

Result: Across symbolic reasoning, research synthesis, and scientific programming tasks, SwarmSys consistently outperforms baselines, improving both accuracy and reasoning stability.

Conclusion: Swarm-inspired coordination is a promising paradigm for scalable, robust, and adaptive multi-agent reasoning, suggesting that coordination scaling may rival model scaling in advancing LLM intelligence.

Abstract: Large language model (LLM) agents have shown remarkable reasoning abilities. However, existing multi-agent frameworks often rely on fixed roles or centralized control, limiting scalability and adaptability in long-horizon reasoning. We introduce SwarmSys, a closed-loop framework for distributed multi-agent reasoning inspired by swarm intelligence. Coordination in SwarmSys emerges through iterative interactions among three specialized roles, Explorers, Workers, and Validators, that continuously cycle through exploration, exploitation, and validation. To enable scalable and adaptive collaboration, we integrate adaptive agent and event profiles, embedding-based probabilistic matching, and a pheromone-inspired reinforcement mechanism, supporting dynamic task allocation and self-organizing convergence without global supervision. Across symbolic reasoning, research synthesis, and scientific programming tasks, SwarmSys consistently outperforms baselines, improving both accuracy and reasoning stability. These findings highlight swarm-inspired coordination as a promising paradigm for scalable, robust, and adaptive multi-agent reasoning, suggesting that coordination scaling may rival model scaling in advancing LLM intelligence.

Method: Three-stage workflow: 1) TSG Mentor tool to improve TSG quality, 2) offline preprocessing using LLMs to extract structured execution DAGs and create Query Preparation Plugins, 3) online execution with DAG-guided scheduler-executor framework supporting parallel execution of independent steps.

Result: Achieves ~94% success rate on GPT-4.1, outperforms baselines with less time and token consumption, and reduces execution time by 32.9% to 70.4% for parallelizable TSGs.

Conclusion: StepFly effectively automates troubleshooting guide execution through its comprehensive framework, demonstrating high success rates and significant performance improvements over existing approaches.

Abstract: Effective incident management in large-scale IT systems relies on troubleshooting guides (TSGs), but their manual execution is slow and error-prone. While recent advances in LLMs offer promise for automating incident management tasks, existing LLM-based solutions lack specialized support for several key challenges, including managing TSG quality issues, interpreting complex control flow, handling data-intensive queries, and exploiting execution parallelism. We first conducted an empirical study on 92 real-world TSGs, and, guided by our findings, we present StepFly, a novel end-to-end agentic framework for troubleshooting guide automation. Our approach features a three-stage workflow: the first stage provides a comprehensive guide together with a tool, TSG Mentor, to assist SREs in improving TSG quality; the second stage performs offline preprocessing using LLMs to extract structured execution DAGs from unstructured TSGs and to create dedicated Query Preparation Plugins (QPPs); and the third stage executes online using a DAG-guided scheduler-executor framework with a memory system to guarantee correct workflow and support parallel execution of independent steps. Our empirical evaluation on a collection of real-world TSGs and incidents demonstrates that StepFly achieves a ~94% success rate on GPT-4.1, outperforming baselines with less time and token consumption. Furthermore, it achieves a remarkable execution time reduction of 32.9% to 70.4% for parallelizable TSGs.

Method: Proposes MoSEAR framework with two modules: MoSE (modality-specific experts with regularized gating) and AR (attention reallocation mechanism) to balance modality integration.

Result: MoSEAR achieves state-of-the-art performance on multiple benchmarks including MER2023, EMER, DFEW, and CA-MER, particularly under modality conflict conditions.

Conclusion: The proposed framework effectively mitigates emotion conflicts and improves performance on consistent samples without trade-offs between audio and visual modalities.

Abstract: Despite their strong performance in multimodal emotion reasoning, existing Multimodal Large Language Models (MLLMs) often overlook the scenarios involving emotion conflicts, where emotional cues from different modalities are inconsistent. To fill this gap, we first introduce CA-MER, a new benchmark designed to examine MLLMs under realistic emotion conflicts. It consists of three subsets: video-aligned, audio-aligned, and consistent, where only one or all modalities reflect the true emotion. However, evaluations on our CA-MER reveal that current state-of-the-art emotion MLLMs systematically over-rely on audio signal during emotion conflicts, neglecting critical cues from visual modality. To mitigate this bias, we propose MoSEAR, a parameter-efficient framework that promotes balanced modality integration. MoSEAR consists of two modules: (1)MoSE, modality-specific experts with a regularized gating mechanism that reduces modality bias in the fine-tuning heads; and (2)AR, an attention reallocation mechanism that rebalances modality contributions in frozen backbones during inference. Our framework offers two key advantages: it mitigates emotion conflicts and improves performance on consistent samples-without incurring a trade-off between audio and visual modalities. Experiments on multiple benchmarks-including MER2023, EMER, DFEW, and our CA-MER-demonstrate that MoSEAR achieves state-of-the-art performance, particularly under modality conflict conditions.

Method: Created DixitWorld with two components: DixitArena (dynamic multi-agent environment with storytellers and listeners) and DixitBench (static QA benchmark for controlled evaluation of hypothesis selection).

Result: Smaller open-source models excel as creative storytellers but produce less discriminative clues, while larger proprietary models perform better overall, especially as listeners. DixitBench strongly correlates with listener performance in DixitArena.

Conclusion: There’s a key trade-off between generative creativity and discriminative understanding in multimodal abductive reasoning, highlighting a central challenge for developing more balanced vision-language agents.

Abstract: Multimodal abductive reasoning–the generation and selection of explanatory hypotheses from partial observations–is a cornerstone of intelligence. Current evaluations of this ability in vision-language models (VLMs) are largely confined to static, single-agent tasks. Inspired by Dixit, we introduce DixitWorld, a comprehensive evaluation suite designed to deconstruct this challenge. DIXITWORLD features two core components: DixitArena, a dynamic, multi-agent environment that evaluates both hypothesis generation (a “storyteller” crafting cryptic clues) and hypothesis selection (“listeners” choosing the target image from decoys) under imperfect information; and DixitBench, a static QA benchmark that isolates the listener’s task for efficient, controlled evaluation. Results from DixitArena reveal distinct, role-dependent behaviors: smaller open-source models often excel as creative storytellers, producing imaginative yet less discriminative clues, whereas larger proprietary models demonstrate superior overall performance, particularly as listeners. Performance on DixitBench strongly correlates with listener results in DixitArena, validating it as a reliable proxy for hypothesis selection. Our findings reveal a key trade-off between generative creativity and discriminative understanding in multimodal abductive reasoning, a central challenge for developing more balanced and capable vision-language agents.

Method: Built on frozen diffusion backbone, uses composable LoRA adapters with prompt-aware control to dynamically compose adapters at inference for per-character customization.

Result: Significantly enhances character fidelity, semantic alignment, and temporal coherence; robust in crowded scenes and enables scalable multi-character generation with minimal overhead.

Conclusion: Well-suited for real-world applications such as story illustration and animation due to its efficiency and consistency capabilities.

Abstract: Ensuring character identity consistency across varying prompts remains a fundamental limitation in diffusion-based text-to-image generation. We propose CharCom, a modular and parameter-efficient framework that achieves character-consistent story illustration through composable LoRA adapters, enabling efficient per-character customization without retraining the base model. Built on a frozen diffusion backbone, CharCom dynamically composes adapters at inference using prompt-aware control. Experiments on multi-scene narratives demonstrate that CharCom significantly enhances character fidelity, semantic alignment, and temporal coherence. It remains robust in crowded scenes and enables scalable multi-character generation with minimal overhead, making it well-suited for real-world applications such as story illustration and animation.

Method: PALU formulates concise reasoning as constrained optimization, minimizing response length subject to performance constraints using Lagrangian optimization. It uses three approximations: off-policy rollouts for performance estimation, truncated Lagrange multipliers, and quantile-driven length adjustments.

Result: PALU reduces output length by 65% while improving accuracy by 15% when applied to DeepSeek-Distill-Qwen-1.5B across five benchmarks, outperforming alternative methods and adapting across domains (logic, STEM, math) and model scales (1.5B, 7B, 14B).

Conclusion: PALU is a practical and effective concise reasoning approach that successfully adapts across domains and model scales while significantly reducing response length and improving accuracy.

Abstract: Concise reasoning in large language models seeks to generate only essential intermediate steps needed to arrive at a final answer, thereby alleviating issues of overthinking. Most proposed approaches hinge on carefully hand-crafted heuristics, struggling to balance concision with performance, often failing to adapt across domains and model scales. In this work, we address these challenges by introducing a principled and pragmatic strategy, performance-aware length updating (PALU). As a principled algorithm, PALU formulates concise reasoning as a constrained optimization problem, minimizing response length subject to a performance constraint, and then applies Lagrangian optimization to convert it into a tractable unconstrained problem. As a pragmatic solution, PALU streamlines complicated update rules through three approximations: (i) estimating performance with off-policy rollouts, (ii) truncating the Lagrange multiplier to two extremes, and (iii) replacing gradient-based updates with quantile-driven length adjustments. PALU reduces output length by 65% while improving accuracy by 15% when applied to DeepSeek-Distill-Qwen-1.5B, averaged over five benchmarks, outperforming a range of alternative methods. Furthermore, PALU is demonstrated to adapt across both domain (logic, STEM and math) and model scale (1.5B, 7B, 14B) entrenching the algorithm as a practical and effective concise reasoning approach.

Method: Two-stage framework: 1) Calibrates sampling budget using Clopper-Pearson method with user-defined risk level, abstaining if risk cannot be met within sampling cap; 2) Uses conformal risk control to determine uncertainty threshold that filters unreliable distractors from candidate sets.

Result: SAFER provides statistical guarantees for correct answer coverage while handling open-ended QA scenarios. It’s compatible with various task-specific admission criteria and calibration-test split ratios, demonstrating robustness and high data efficiency.

Conclusion: SAFER addresses limitations of prior SCP methods by introducing a practical framework that doesn’t require finite answer spaces, making it suitable for real-world open-ended QA applications where trustworthiness is critical.

Abstract: As large language models (LLMs) are increasingly deployed in risk-sensitive applications such as real-world open-ended question answering (QA), ensuring the trustworthiness of their outputs has become critical. Existing selective conformal prediction (SCP) methods provide statistical guarantees by constructing prediction sets with a constrained miscoverage rate for correct answers. However, prior works unrealistically assume that admissible answers for all instances can be obtained via finite sampling, even for open-ended QA scenarios that lack a fixed and finite solution space. To address this, we introduce a two-stage risk control framework comprising abstention-aware sampling and conformalized filtering (SAFER). Firstly, on a held-out calibration set, SAFER calibrates a sampling budget within the maximum sampling cap, using the Clopper-Pearson exact method at a user-desired risk level (i.e., the maximum allowable miscoverage rate of the sampling sets). If the risk level cannot be satisfied within the cap, we abstain; otherwise, the calibrated sampling budget becomes the minimum requirements at test time. Then, we employ calibration instances where correct answers are attainable under the calibrated budget and apply the conformal risk control method to determine a statistically valid uncertainty threshold, which filters unreliable distractors from the candidate set for each test data point. In this stage, SAFER introduces an additional risk level to guide the calculation of the threshold, thereby controlling the risk of correct answers being excluded. Furthermore, we show that SAFER is compatible with various task-specific admission criteria and calibration-test split ratios, highlighting its robustness and high data efficiency.

Method: Environment Tuning uses structured curriculum, actionable environment augmentation for corrective feedback, and fine-grained progress rewards to enable stable and efficient exploration directly from problem instances.

Result: Using only 400 problem instances from BFCL benchmark, the method achieves competitive in-distribution performance and superior out-of-distribution generalization, overcoming the performance collapse common to SFT-based approaches.

Conclusion: This work presents a paradigm shift from supervised fine-tuning on static trajectories to dynamic, environment-based exploration, enabling training of more robust and data-efficient agents.

Abstract: Large Language Model (LLM) agents show great promise for complex, multi-turn tool-use tasks, but their development is often hampered by the extreme scarcity of high-quality training data. Supervised fine-tuning (SFT) on synthetic data leads to overfitting, whereas standard reinforcement learning (RL) struggles with a critical cold-start problem and training instability. To address these challenges, we introduce $\textbf{Environment Tuning}$, a novel training paradigm that enables agents to learn complex behaviors directly from problem instances without relying on pre-collected expert trajectories. $\textbf{Environment Tuning}$ orchestrates this learning process through a structured curriculum, actionable environment augmentation that provides corrective feedback, and fine-grained progress rewards to ensure stable and efficient exploration. Using only 400 problem instances from Berkeley Function-Calling Leaderboard (BFCL) benchmark, our method not only achieves competitive in-distribution performance against strong baselines but also demonstrates superior out-of-distribution generalization, overcoming the performance collapse common to SFT-based approaches. Our work presents a paradigm shift from supervised fine-tuning on static trajectories to dynamic, environment-based exploration, paving the way for training more robust and data-efficient agents.

Method: PIXEL learns property-aligned subspaces from dual views (tail-averaged and end-token), selects intervention strength via constrained geometric objective with closed-form solution, performs sample-level orthogonal residual calibration, and uses lightweight position-scanning to identify receptive injection sites.

Result: Across diverse models and evaluation paradigms, PIXEL consistently improves attribute alignment while preserving model general capabilities, offering practical and principled LLM controllable generation.

Conclusion: PIXEL provides a practical and principled method for reliable LLM behavior control through adaptive activation steering with representation-level guarantees for minimal-intervention alignment.

Abstract: Reliable behavior control is central to deploying large language models (LLMs) on the web. Activation steering offers a tuning-free route to align attributes (e.g., truthfulness) that ensure trustworthy generation. Prevailing approaches rely on coarse heuristics and lack a principled account of where to steer and how strongly to intervene. To this end, we propose Position-wise Injection with eXact Estimated Levels (PIXEL), a position-wise activation steering framework that, in contrast to prior work, learns a property-aligned subspace from dual views (tail-averaged and end-token) and selects intervention strength via a constrained geometric objective with a closed-form solution, thereby adapting to token-level sensitivity without global hyperparameter tuning. PIXEL further performs sample-level orthogonal residual calibration to refine the global attribute direction and employs a lightweight position-scanning routine to identify receptive injection sites. We additionally provide representation-level guarantees for the minimal-intervention rule, supporting reliable alignment. Across diverse models and evaluation paradigms, PIXEL consistently improves attribute alignment while preserving model general capabilities, offering a practical and principled method for LLMs’ controllable generation. Our code is available at https://github.com/V1centNevwake/PIXEL-Adaptive-Steering

Method: Two-stage training: (1) Cold-start SFT with hybrid reasoning dataset, (2) Reinforcement learning with Entropy-guided Hybrid Policy Optimization (EHPO) using entropy-guided dynamic rollout and difficulty-aware penalty.

Result: Achieves up to 6.1% performance gain while reducing reasoning output length by 49.5% to 59.3% on mathematical reasoning benchmarks.

Conclusion: ADR effectively balances reasoning performance and efficiency among state-of-the-art approaches through adaptive dual reasoning modes.

Abstract: Although Long Reasoning Models (LRMs) have achieved superior performance on various reasoning scenarios, they often suffer from increased computational costs and inference latency caused by overthinking. To address these limitations, we propose Adaptive Dual Reasoner, which supports two reasoning modes: fast thinking and slow thinking. ADR dynamically alternates between these modes based on the contextual complexity during reasoning. ADR is trained in two stages: (1) A cold-start stage using supervised fine-tuning (SFT) to equip the model with the ability to integrate both fast and slow reasoning modes, in which we construct a hybrid reasoning dataset through a dedicated pipeline to provide large-scale supervision. (2) A reinforcement learning stage for optimizing reasoning effort, where we introduce Entropy-guided Hybrid Policy Optimization EHPO, an RL training framework employing an entropy-guided dynamic rollout strategy for branching at high-entropy units and a difficulty-aware penalty to balance fast and slow reasoning. Across challenging mathematical reasoning benchmarks, ADR achieves an effective balance between reasoning performance and efficiency among state-of-the-art approaches. Specifically, ADR yields a performance gain of up to 6.1%, while reducing the reasoning output length by 49.5% to 59.3%.

Method: Multi-agent reinforcement learning approach to model human behavior in social networks, tested with human-subject experiments.

Result: Fake news attacks are more effective when targeting highly connected people and those with weaker private information. Distributed disinformation across multiple agents works better than concentrated attacks. Fake news spreads less in balanced networks than clustered networks.

Conclusion: The model is suitable for analyzing fake news spread in social networks, with experimental evidence supporting the model’s predictions.

Abstract: We propose multi-agent reinforcement learning as a new method for modeling fake news in social networks. This method allows us to model human behavior in social networks both in unaccustomed populations and in populations that have adapted to the presence of fake news. In particular the latter is challenging for existing methods. We find that a fake-news attack is more effective if it targets highly connected people and people with weaker private information. Attacks are more effective when the disinformation is spread across several agents than when the disinformation is concentrated with more intensity on fewer agents. Furthermore, fake news spread less well in balanced networks than in clustered networks. We test a part of our findings in a human-subject experiment. The experimental evidence provides support for the predictions from the model, suggesting that the model is suitable to analyze the spread of fake news in social networks.

Method: Proposed a Perturbation-based Causal Identification of Critical Neurons method to systematically locate critical neurons in LLMs through targeted disruptions.

Result: Found that critical neurons are ultra-sparse, concentrated in outer layers (especially MLP down_proj components), and their disruption causes sharp phase transitions rather than gradual performance decline.

Conclusion: These findings provide insights for developing more robust LLM architectures and improving deployment security in safety-critical applications.

Abstract: Large Language Models (LLMs) have become foundational tools in natural language processing, powering a wide range of applications and research. Many studies have shown that LLMs share significant similarities with the human brain. Recent neuroscience research has found that a small subset of biological neurons in the human brain are crucial for core cognitive functions, which raises a fundamental question: do LLMs also contain a small subset of critical neurons? In this paper, we investigate this question by proposing a Perturbation-based Causal Identification of Critical Neurons method to systematically locate such critical neurons in LLMs. Our findings reveal three key insights: (1) LLMs contain ultra-sparse critical neuron sets. Disrupting these critical neurons can cause a 72B-parameter model with over 1.1 billion neurons to completely collapse, with perplexity increasing by up to 20 orders of magnitude; (2) These critical neurons are not uniformly distributed, but tend to concentrate in the outer layers, particularly within the MLP down_proj components; (3) Performance degradation exhibits sharp phase transitions, rather than a gradual decline, when these critical neurons are disrupted. Through comprehensive experiments across diverse model architectures and scales, we provide deeper analysis of these phenomena and their implications for LLM robustness and interpretability. These findings can offer guidance for developing more robust model architectures and improving deployment security in safety-critical applications.

Method: A two-step plugin: Functional Head Identification locates perception- and reasoning-oriented attention heads, and Class-conditioned Rescaling regulates their contributions without retraining.

Result: Evaluations on three MLRMs show average improvement of 5% (up to 15%) with <1% additional computation and 9% of baseline latency. The approach is model-agnostic and enhances reliability and interpretability.

Conclusion: The proposed plugin significantly enhances both reliability and interpretability of off-the-shelf MLRMs, enabling safe deployment in high-stakes applications through lightweight attention head regulation.

Abstract: Multimodal large reasoning models (MLRMs) are rapidly advancing vision-language reasoning and are emerging as a foundation for cross-modal intelligence. Hallucination remains a persistent failure mode, manifesting itself as erroneous reasoning chains and misinterpretation of visual content. In this study, we observe that attention heads exhibit a staged division: shallow heads predominantly serve perception, while deeper heads shift toward symbolic reasoning, revealing two major causes of hallucination, namely perceptual bias and reasoning drift. To address these issues, we propose a lightweight and interpretable two-step plugin, Functional Head Identification and Class-conditioned Rescaling, which locates perception- and reasoning-oriented heads and regulates their contributions without retraining. Evaluations on three real-world MLRMs (Kimi-VL, Ocean-R1, R1-Onevision), six benchmarks across three domains, and four baselines show that our plugin achieves an average improvement of 5% and up to 15%, with only <1% additional computation and 9% of baseline latency. Our approach is completely model-agnostic and significantly enhances both the reliability and interpretability of the off-the-shelf MLRMs, thereby enabling their safe deployment in high-stakes applications. Our code is available at https://anonymous.4open.science/r/Functional-Attention-Control.

Method: Uses Intent, Context, and Action (ICA) format to restructure workflows and synthetic data generation for fine-tuning LLMs with minimal human intervention.

Result: Internal experiments show significant performance enhancement in customer support, with improvements in both accuracy and manual processing time metrics.

Conclusion: The approach sets a new benchmark for LLM application in customer support, being both cost-effective and performance-enhancing.

Abstract: We propose a practical approach by integrating Large Language Models (LLMs) with a framework designed to navigate the complexities of Airbnb customer support operations. In this paper, our methodology employs a novel reformatting technique, the Intent, Context, and Action (ICA) format, which transforms policies and workflows into a structure more comprehensible to LLMs. Additionally, we develop a synthetic data generation strategy to create training data with minimal human intervention, enabling cost-effective fine-tuning of our model. Our internal experiments (not applied to Airbnb products) demonstrate that our approach of restructuring workflows and fine-tuning LLMs with synthetic data significantly enhances their performance, setting a new benchmark for their application in customer support. Our solution is not only cost-effective but also improves customer support, as evidenced by both accuracy and manual processing time evaluation metrics.

Method: The paper draws from assistive technologies that evolved into mainstream industries and presents the Healthcare AI Inclusive Innovation Framework (HAIIF) - a practical scoring system to evaluate AI investments based on their potential to capture inclusive innovation benefits.

Result: The paper identifies four mechanisms through which inclusive innovation drives returns: market expansion, risk mitigation, performance dividends, and competitive advantages. HAIIF provides structured guidance for resource allocation.

Conclusion: Organizations investing incrementally in inclusive design can achieve expanded market reach and sustained competitive advantages, while those treating these considerations as overhead face compounding disadvantages as network effects and data advantages accrue to early movers.

Abstract: While ethical arguments for fairness in healthcare AI are well-established, the economic and strategic value of inclusive design remains underexplored. This perspective introduces the ``inclusive innovation dividend’’ – the counterintuitive principle that solutions engineered for diverse, constrained use cases generate superior economic returns in broader markets. Drawing from assistive technologies that evolved into billion-dollar mainstream industries, we demonstrate how inclusive healthcare AI development creates business value beyond compliance requirements. We identify four mechanisms through which inclusive innovation drives returns: (1) market expansion via geographic scalability and trust acceleration; (2) risk mitigation through reduced remediation costs and litigation exposure; (3) performance dividends from superior generalization and reduced technical debt, and (4) competitive advantages in talent acquisition and clinical adoption. We present the Healthcare AI Inclusive Innovation Framework (HAIIF), a practical scoring system that enables organizations to evaluate AI investments based on their potential to capture these benefits. HAIIF provides structured guidance for resource allocation, transforming fairness and inclusivity from regulatory checkboxes into sources of strategic differentiation. Our findings suggest that organizations investing incrementally in inclusive design can achieve expanded market reach and sustained competitive advantages, while those treating these considerations as overhead face compounding disadvantages as network effects and data advantages accrue to early movers.

Method: Comprehensive experiments across multiple models, datasets, and prompts to evaluate trace length as a confidence estimator, investigating mechanisms through entropy analysis and controlling for confounders like problem difficulty.

Result: Trace length performs comparably to verbalized confidence estimators and remains effective even after adjusting for confounders. Reasoning post-training fundamentally changes the trace length-accuracy relationship, with high-entropy “forking” tokens playing a key role.

Conclusion: Reasoning trace length is a practical confidence measure for large reasoning models, and reasoning post-training enhances uncertainty quantification beyond verbal expressions.

Abstract: Uncertainty quantification for LLMs is a key research direction towards addressing hallucination and other issues that limit their reliable deployment. In this work, we show that reasoning trace length is a simple and useful confidence estimator in large reasoning models. Through comprehensive experiments across multiple models, datasets, and prompts, we show that trace length performs in comparable but complementary ways to other zero-shot confidence estimators such as verbalized confidence. Our work reveals that reasoning post-training fundamentally alters the relationship between trace length and accuracy, going beyond prior work that had shown that post-training causes traces to grow longer in general (e.g., “overthinking”). We investigate the mechanisms behind trace length’s performance as a confidence signal, observing that the effect remains even after adjusting for confounders such as problem difficulty and GRPO-induced length bias. We identify high-entropy or “forking” tokens as playing a key role in the mechanism. Our findings demonstrate that reasoning post-training enhances uncertainty quantification beyond verbal expressions, and establish trace length as a practical confidence measure for large reasoning models.

Method: Multi-agent system with worker agents processing EHR data in chunks, distilling critical events into EHRMem memory module, and manager agent synthesizing summaries for predictions.

Result: Outperformed baselines in zero-shot one-year lung cancer risk prediction using five-year EHR data.

Conclusion: Traj-CoA demonstrates clinically aligned temporal reasoning and offers a robust, generalizable approach for complex patient trajectory modeling.

Abstract: Large language models (LLMs) offer a generalizable approach for modeling patient trajectories, but suffer from the long and noisy nature of electronic health records (EHR) data in temporal reasoning. To address these challenges, we introduce Traj-CoA, a multi-agent system involving chain-of-agents for patient trajectory modeling. Traj-CoA employs a chain of worker agents to process EHR data in manageable chunks sequentially, distilling critical events into a shared long-term memory module, EHRMem, to reduce noise and preserve a comprehensive timeline. A final manager agent synthesizes the worker agents’ summary and the extracted timeline in EHRMem to make predictions. In a zero-shot one-year lung cancer risk prediction task based on five-year EHR data, Traj-CoA outperforms baselines of four categories. Analysis reveals that Traj-CoA exhibits clinically aligned temporal reasoning, establishing it as a promisingly robust and generalizable approach for modeling complex patient trajectories.

Method: Proposed MedCoAct framework with specialized doctor and pharmacist agents that collaborate, plus DrugCareQA benchmark for evaluating integrated diagnosis and treatment workflows.