Method: Analyzed dense transformer architectures (Llama-3.1-70B and Qwen1.5-14B) exposed to large volumes of irrelevant context, examining KV cache growth effects. Extended analysis of Mixture-of-Experts (MoE) architecture at varying context scales.

Result: Identified non-linear performance degradation tied to KV cache growth. MoE architectures showed unique behavioral anomalies at different context scales, suggesting architectural benefits may be masked by infrastructure bottlenecks at high token volumes.

Conclusion: The trade-off between system performance and model quality is critical when scaling context windows. KV cache management is key to performance degradation, and MoE benefits may not scale linearly due to infrastructure limitations at high context volumes.

Abstract: The scaling trend in Large Language Models (LLMs) has prioritized increasing the maximum context window to facilitate complex, long-form reasoning and document analysis. However, managing this expanded context introduces severe computational overhead. This paper investigates the critical trade-off between system performance and model quality when dense transformer architectures–specifically Llama-3.1-70B and Qwen1.5-14B–are exposed to large volumes of irrelevant and distracting context. The research identifies a non-linear performance degradation tied to the growth of the Key-Value (KV) cache. Furthermore, an extended analysis of the Mixture-of-Experts (MoE) architecture reveals unique behavioral anomalies at varying context scales, suggesting that architectural benefits may be masked by infrastructure bottlenecks at high token volumes.

Method: 1. Class-Aware Masking (CAM): A lightweight modification to InfoNCE objective that suppresses invalid in-batch negatives during large-batch contrastive training with mixed task supervision. 2. Diversified training corpus construction: Uses context-grounded synthetic data generation, cross-lingual translation, and structured e-commerce data construction. 3. Production optimization: Combines robustness-driven large-batch training with spherical model merging to prevent catastrophic forgetting, and optimizes inference via vLLM and FP8 quantization.

Result: Compass-Embedding v4 achieves state-of-the-art performance on major Southeast Asian languages, significantly outperforming general-purpose embedding models in domain-specific retrieval and classification tasks. It maintains competitive performance on high-resource languages while being optimized for production deployment with high-throughput inference.

Conclusion: The framework successfully addresses the core challenges of multilingual embedding for Southeast Asian e-commerce through innovative techniques that improve semantic discrimination, enable robust multilingual learning, and meet production constraints, providing a practical solution for real-world e-commerce applications in emerging markets.

Abstract: As global e-commerce rapidly expands into emerging markets, the lack of high-quality semantic representations for low-resource languages has become a decisive bottleneck for retrieval, recommendation, and search systems. In this work, we present Compass-Embedding v4, a high-efficiency multilingual embedding framework specifically optimized for Southeast Asian (SEA) e-commerce scenarios, where data scarcity, noisy supervision, and strict production constraints jointly challenge representation learning. Compass-Embedding v4 addresses three core challenges. First, large-batch contrastive training under mixed task supervision introduces systematic false negatives that degrade semantic alignment. We propose Class-Aware Masking (CAM), a lightweight modification to the InfoNCE objective that suppresses invalid in-batch negatives and improves semantic discrimination without altering training efficiency. Second, low-resource SEA languages suffer from limited and uneven data coverage. We construct a diversified training corpus through context-grounded synthetic data generation, cross-lingual translation, and structured e-commerce data construction, enabling robust multilingual and domain-specific learning. Third, production deployment requires high-throughput inference while preserving embedding quality. We combine robustness-driven large-batch training with spherical model merging to mitigate catastrophic forgetting, and optimize inference via vLLM and FP8 quantization. Extensive evaluations across multilingual benchmarks and proprietary e-commerce tasks show that Compass-Embedding v4 achieves state-of-the-art performance on major SEA languages, significantly outperforming general-purpose embedding models in domain-specific retrieval and classification, while maintaining competitive performance on high-resource languages.

Method: Evaluated six medical LLMs using MCQ with human evaluation and clinical review in pediatric endocrinology. Examined prompt variation effects, self-assessment bias, output variability, and system-level perturbations across deterministic and stochastic settings.

Result: HuatuoGPT-o1-8B achieved highest performance and consistency, but high consistency doesn’t guarantee correctness. Models show self-assessment bias and dependency on explanation order. System-level perturbations cause statistically significant output shifts despite stable accuracy.

Conclusion: Small prompt perturbations cause divergent outputs, raising reproducibility concerns. Output variability under different stochastic regimes highlights need for broader diagnostic frameworks to understand pitfalls in clinical decision support scenarios.

Abstract: Small open-source medical large language models (LLMs) offer promising opportunities for low-resource deployment and broader accessibility. However, their evaluation is often limited to accuracy on medical multiple choice question (MCQ) benchmarks, and lacks evaluation of consistency, robustness, or reasoning behavior. We use MCQ coupled to human evaluation and clinical review to assess six small open-source medical LLMs (HuatuoGPT-o1 (Chen 2024), Diabetica-7B, Diabetica-o1 (Wei 2024), Meditron3-8B (Sallinen2025), MedFound-7B (Liu 2025), and ClinicaGPT-base-zh (Wang 2023)) in pediatric endocrinology. In deterministic settings, we examine the effect of prompt variation on models’ output and self-assessment bias. In stochastic settings, we evaluate output variability and investigate the relationship between consistency and correctness. HuatuoGPT-o1-8B achieved the highest performance. The results show that high consistency across the model response is not an indicator of correctness, although HuatuoGPT-o1-8B showed the highest consistency rate. When tasked with selecting correct reasoning, both HuatuoGPT-o1-8B and Diabetica-o1 exhibit self-assessment bias and dependency on the order of the candidate explanations. Expert review of incorrect reasoning rationales identified a mix of clinically acceptable responses and clinical oversight. We further show that system-level perturbations, such as differences in CUDA builds, can yield statistically significant shifts in model output despite stable accuracy. This work demonstrates that small, semantically negligible prompt perturbations lead to divergent outputs, raising concerns about reproducibility of LLM-based evaluations and highlights the output variability under different stochastic regimes, emphasizing the need of a broader diagnostic framework to understand potential pitfalls in real-world clinical decision support scenarios.

Method: Nine-step pipeline integrating diverse data sources, structured preprocessing, prompt-based QA generation via Google Gemini, NLI for quality control, semantic deduplication, clustering-based data splitting, and parameter-efficient fine-tuning using LoRA on three LLMs (LLaMA-3.2-3B, Qwen2.5-7B, Gemma).

Result: Qwen2.5-7B emerged as best performer with BLEU-4/ROUGE-1 improvements of 82%/70% for PRSGPT and 6%/18% for BioStarsGPT. Human evaluation showed PRSGPT achieved 61.9% accuracy comparable to Google Gemini (61.4%) but with richer methodological detail. BioStarsGPT demonstrated 59% conceptual accuracy. Open-source datasets include over 28,000 QA pairs for PRSGPT and 154,282 for BioStarsGPT.

Conclusion: The pipeline enables scalable, domain-specific fine-tuning of LLMs for privacy-preserving, locally deployable bioinformatics assistants, addressing challenges and limitations in their development and use.

Abstract: Large language models (LLMs) often lack specialized knowledge for complex bioinformatics applications. We present a reproducible pipeline for fine-tuning LLMs on specialized bioinformatics data, demonstrated through two use cases: PRSGPT, focused on polygenic risk score (PRS) tools, and BioStarsGPT, trained on community forum discussions. The nine-step pipeline integrates diverse data sources, structured preprocessing, prompt-based question-answer (QA) generation (via Google Gemini), natural language inference (NLI) for quality control, semantic deduplication, clustering-based data splitting, and parameter-efficient fine-tuning using LoRA. We fine-tuned three LLMs (LLaMA-3.2-3B, Qwen2.5-7B, Gemma) and benchmarked them on over 14 lexical and semantic metrics. Qwen2.5-7B emerged as the best performer, with BLEU-4 and ROUGE-1 improvements of 82% and 70% for PRSGPT and 6% and 18% for BioStarsGPT, respectively. The open-source datasets produced include over 28,000 QA pairs for PRSGPT and 154,282 for BioStarsGPT. Human evaluation of PRSGPT yielded 61.9% accuracy on the PRS tools comparison task, comparable to Google Gemini (61.4%), but with richer methodological detail and accurate citations. BioStarsGPT demonstrated 59% conceptual accuracy across 142 curated bioinformatics questions. Our pipeline enables scalable, domain-specific fine-tuning of LLMs. It enables privacy-preserving, locally deployable bioinformatics assistants, explores their practical applications, and addresses the challenges, limitations, and mitigation strategies associated with their development and use.

Method: Model LLM layers as contractive mappings in Iterated Function Systems (IFS) toward concept-specific attractors. Develop training-free methods that operate directly on these attractors to manipulate model behavior.

Result: Attractor-based interventions match or exceed specialized baselines across multiple tasks (language translation, hallucination reduction, guardrailing, synthetic data generation), offering efficient alternatives to heavy fine-tuning.

Conclusion: The IFS framework explains LLM internal representation patterns, and attractor-based methods provide simple, effective, training-free solutions for diverse practical tasks, outperforming specialized approaches in generalization scenarios.

Abstract: Large language models (LLMs) often map semantically related prompts to similar internal representations at specific layers, even when their surface forms differ widely. We show that this behavior can be explained through Iterated Function Systems (IFS), where layers act as contractive mappings toward concept-specific Attractors. We leverage this insight and develop simple, training-free methods that operate directly on these Attractors to solve a wide range of practical tasks, including language translation, hallucination reduction, guardrailing, and synthetic data generation. Despite their simplicity, these Attractor-based interventions match or exceed specialized baselines, offering an efficient alternative to heavy fine-tuning, generalizable in scenarios where baselines underperform.

Method: LimAgents uses a multi-agent framework with specialized roles: extraction agents for explicit limitations, methodological gap analyzers, peer reviewer simulators, and citation agents for contextual weaknesses. A Judge agent refines outputs and a Master agent consolidates them. The system also introduces a pointwise evaluation protocol using LLM-as-a-Judge for better coverage measurement.

Result: The RAG + multi-agent GPT-4o mini configuration achieved +15.51% coverage gain over zero-shot baselines, while the Llama 3 8B multi-agent setup yielded +4.41% improvement. The framework successfully identifies explicit, implicit, peer review-focused, and literature-informed limitations.

Conclusion: LimAgents provides a systematic approach to generating substantive research limitations by leveraging multi-agent collaboration and contextual information, significantly improving over superficial zero-shot approaches and addressing the shortcomings of traditional evaluation metrics.

Abstract: Identifying and articulating limitations is essential for transparent and rigorous scientific research. However, zero-shot large language models (LLMs) approach often produce superficial or general limitation statements (e.g., dataset bias or generalizability). They usually repeat limitations reported by authors without looking at deeper methodological issues and contextual gaps. This problem is made worse because many authors disclose only partial or trivial limitations. We propose LimAgents, a multi-agent LLM framework for generating substantive limitations. LimAgents integrates OpenReview comments and author-stated limitations to provide stronger ground truth. It also uses cited and citing papers to capture broader contextual weaknesses. In this setup, different agents have specific roles as sequential role: some extract explicit limitations, others analyze methodological gaps, some simulate the viewpoint of a peer reviewer, and a citation agent places the work within the larger body of literature. A Judge agent refines their outputs, and a Master agent consolidates them into a clear set. This structure allows for systematic identification of explicit, implicit, peer review-focused, and literature-informed limitations. Moreover, traditional NLP metrics like BLEU, ROUGE, and cosine similarity rely heavily on n-gram or embedding overlap. They often overlook semantically similar limitations. To address this, we introduce a pointwise evaluation protocol that uses an LLM-as-a-Judge to measure coverage more accurately. Experiments show that LimAgents substantially improve performance. The RAG + multi-agent GPT-4o mini configuration achieves a +15.51% coverage gain over zero-shot baselines, while the Llama 3 8B multi-agent setup yields a +4.41% improvement.

Method: Introduces ATOD benchmark with synthetic dialogue generation pipeline producing richly annotated conversations requiring long-term reasoning. Also proposes ATOD-Eval framework translating key dimensions into fine-grained metrics for reproducible offline/online evaluation, plus a strong agentic memory-based evaluator.

Result: ATOD-Eval enables comprehensive assessment across task completion, agentic capability, and response quality. The proposed evaluator offers better accuracy-efficiency tradeoff compared to existing memory- and LLM-based approaches under this evaluation setting.

Conclusion: ATOD addresses the gap in evaluating advanced agentic behaviors in modern TOD systems, providing a systematic benchmark and evaluation framework for comprehensive assessment of next-generation conversational agents.

Abstract: Recent advances in task-oriented dialogue (TOD) systems, driven by large language models (LLMs) with extensive API and tool integration, have enabled conversational agents to coordinate interleaved goals, maintain long-horizon context, and act proactively through asynchronous execution. These capabilities extend beyond traditional TOD systems, yet existing benchmarks lack systematic support for evaluating such agentic behaviors. To address this gap, we introduce ATOD, a benchmark and synthetic dialogue generation pipeline that produces richly annotated conversations requiring long-term reasoning. ATOD captures key characteristics of advanced TOD, including multi-goal coordination, dependency management, memory, adaptability, and proactivity. Building on ATOD, we propose ATOD-Eval, a holistic evaluation framework that translates these dimensions into fine-grained metrics and supports reproducible offline and online evaluation. We further present a strong agentic memory-based evaluator for benchmarking on ATOD. Experiments show that ATOD-Eval enables comprehensive assessment across task completion, agentic capability, and response quality, and that the proposed evaluator offers a better accuracy-efficiency tradeoff compared to existing memory- and LLM-based approaches under this evaluation setting.

Method: Extends Mistral 7B v0.2 architecture scaled to 11B parameters via depth up-scaling, with four-stage training: continuous pre-training, supervised fine-tuning (SFT), Direct Preference Optimization (DPO), and reinforcement learning.

Result: Significantly surpasses other specialized Polish language models and outperforms many larger models (2-6 times more parameters) on a wide range of tasks from basic linguistic understanding to complex reasoning.

Conclusion: Bielik 11B v3 advances Polish language AI capabilities and establishes a new benchmark for resource-efficient, high-performance models for less-represented languages, with effective deployment across diverse hardware configurations through parameter efficiency and extensive quantization options.

Abstract: We present Bielik 11B v3, a state-of-the-art language model highly optimized for the Polish language, while also maintaining strong capabilities in other European languages. This model extends the Mistral 7B v0.2 architecture, scaled to 11B parameters via depth up-scaling. Its development involved a comprehensive four-stage training pipeline: continuous pre-training, supervised fine-tuning (SFT), Direct Preference Optimization (DPO), and reinforcement learning. Comprehensive evaluations demonstrate that Bielik 11B v3 achieves exceptional performance. It significantly surpasses other specialized Polish language models and outperforms many larger models (with 2-6 times more parameters) on a wide range of tasks, from basic linguistic understanding to complex reasoning. The model’s parameter efficiency, combined with extensive quantization options, allows for effective deployment across diverse hardware configurations. Bielik 11B v3 not only advances AI capabilities for the Polish language but also establishes a new benchmark for developing resource-efficient, high-performance models for less-represented languages.

Method: Systematic study on vLLM inference engine covering multiple SD variants (n-gram, EAGLE/EAGLE-3, Draft-Model, Multi-Token Prediction) across diverse workloads, model scales, and batch sizes. Analyzes key performance factors and quantifies theoretical upper bound on SD speedup.

Result: Verification by target model dominates execution time. Acceptance length varies markedly across output token positions, requests, and datasets. Substantial gaps exist between observed performance and theoretical upper bounds.

Conclusion: The study reveals critical insights about SD performance in production settings and highlights new research opportunities for improving speculative decoding based on the observed gaps between theoretical and practical performance.

Abstract: Speculative decoding (SD) has become a popular technique to accelerate Large Language Model (LLM) inference, yet its real-world effectiveness remains unclear as prior evaluations rely on research prototypes and unrealistically small batch sizes. We present, to our knowledge, the first systematic study of SD on a production-grade and widely deployed inference engine (vLLM), covering multiple SD variants ($n$-gram, EAGLE/EAGLE-3, Draft-Model, Multi-Token Prediction) across diverse workloads, model scales, and batch sizes. We analyze key factors governing SD performance, and quantify a theoretical upper bound on SD speedup. Our results show that verification by the target model dominates the execution, while acceptance length varies markedly across output token positions, requests, and datasets. Comparing measured performance with theoretical bounds reveals substantial gaps between observed and theoretical upper bounds, and we leverage this observation to highlight new research opportunities that our study opens up in improving SD.

Method: Evaluated ELECTRA-small on SQuAD v1.1 and adversarial datasets (AddSent, AddOneSent), identified error patterns (lexical bias, numerical reasoning, entity recognition), and developed multi-domain debiasing framework with knowledge distillation, debiasing techniques, and domain expansion.

Result: Achieved up to 2.6 percentage point improvements in Exact Match (EM) and F1 scores across all test sets, with gains in adversarial contexts.

Conclusion: Targeted bias mitigation strategies can enhance robustness and reliability of natural language understanding systems.

Abstract: Question-answering (QA) models have advanced significantly in machine reading comprehension but often exhibit biases that hinder their performance, particularly with complex queries in adversarial conditions. This study evaluates the ELECTRA-small model on the Stanford Question Answering Dataset (SQuAD) v1.1 and adversarial datasets AddSent and AddOneSent. By identifying errors related to lexical bias, numerical reasoning, and entity recognition, we develop a multi-domain debiasing framework incorporating knowledge distillation, debiasing techniques, and domain expansion. Our results demonstrate up to 2.6 percentage point improvements in Exact Match (EM) and F1 scores across all test sets, with gains in adversarial contexts. These findings highlight the potential of targeted bias mitigation strategies to enhance the robustness and reliability of natural language understanding systems.

Method: WorldMind autonomously constructs symbolic World Knowledge Repository by synthesizing environmental feedback, unifying Process Experience (enforcing physical feasibility via prediction errors) and Goal Experience (guiding task optimality through successful trajectories).

Result: Experiments on EB-ALFRED and EB-Habitat show WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.

Conclusion: WorldMind effectively bridges the modal disconnect between LLMs’ semantic knowledge and physical world constraints through autonomous symbolic knowledge construction, enabling physically feasible planning without continuous retraining.

Abstract: Current Large Language Models (LLMs) exhibit a critical modal disconnect: they possess vast semantic knowledge but lack the procedural grounding to respect the immutable laws of the physical world. Consequently, while these agents implicitly function as world models, their simulations often suffer from physical hallucinations-generating plans that are logically sound but physically unexecutable. Existing alignment strategies predominantly rely on resource-intensive training or fine-tuning, which attempt to compress dynamic environmental rules into static model parameters. However, such parametric encapsulation is inherently rigid, struggling to adapt to the open-ended variability of physical dynamics without continuous, costly retraining. To bridge this gap, we introduce WorldMind, a framework that autonomously constructs a symbolic World Knowledge Repository by synthesizing environmental feedback. Specifically, it unifies Process Experience to enforce physical feasibility via prediction errors and Goal Experience to guide task optimality through successful trajectories. Experiments on EB-ALFRED and EB-Habitat demonstrate that WorldMind achieves superior performance compared to baselines with remarkable cross-model and cross-environment transferability.

Method: Introduces Entropic Context Shaping (ECS), which measures context utility via the shift in the model’s answer distribution toward the correct answer. Formalizes utility as signed change in answer probability and provides theoretical analysis showing task-irrelevant updates yield near-zero distribution shift.

Result: On fine-grained turn selection, ECS with Llama-3.1-8B achieves F1=0.265, a 71.83% relative improvement over TF-IDF (F1=0.154). Demonstrates pragmatic utility outperforms lexical similarity when precise context selection matters.

Conclusion: ECS provides an information-theoretic framework for measuring pragmatic utility of context, significantly outperforming lexical similarity methods for context selection in LLM agents.

Abstract: Context engineering for large language model (LLM) agents requires distinguishing pragmatically useful information from misleading distractors. We introduce Entropic Context Shaping (ECS), an information-theoretic framework that measures context utility via the shift in the model’s answer distribution toward the correct answer. Unlike lexical similarity methods that rely on word overlap, ECS captures pragmatic utility – whether a passage actually helps answer the question. We formalize utility as the signed change in answer probability and provide theoretical analysis showing that task-irrelevant updates yield near-zero distribution shift. We evaluate on multi-turn context selection tasks using LongMemEval (session-level) and LoCoMo (turn-level) benchmarks. On fine-grained turn selection, ECS with Llama-3.1-8B achieves F1=0.265, a 71.83% relative improvement over TF-IDF (F1=0.154), demonstrating that pragmatic utility outperforms lexical similarity when precise context selection matters. Code and data are available in the supplementary materials.

Method: TSDA (Temporal-Spatial Decouple before Act) decouples each modality into temporal dynamics and spatial structural context using separate temporal and spatial encoders. Factor-Consistent Cross-Modal Alignment aligns temporal features only with temporal counterparts across modalities, and spatial features only with spatial counterparts. Factor-specific supervision and decorrelation regularization reduce cross-factor leakage while preserving complementarity. A Gated Recouple module then recouples aligned streams for the final task.

Result: Extensive experiments show that TSDA outperforms baseline methods. Ablation analysis confirms the necessity and interpretability of the design choices.

Conclusion: Explicitly decoupling temporal and spatial factors before cross-modal interaction effectively addresses spatiotemporal heterogeneity in multimodal sentiment analysis, leading to improved performance through better alignment and reduced information leakage.

Abstract: Multimodal Sentiment Analysis integrates Linguistic, Visual, and Acoustic. Mainstream approaches based on modality-invariant and modality-specific factorization or on complex fusion still rely on spatiotemporal mixed modeling. This ignores spatiotemporal heterogeneity, leading to spatiotemporal information asymmetry and thus limited performance. Hence, we propose TSDA, Temporal-Spatial Decouple before Act, which explicitly decouples each modality into temporal dynamics and spatial structural context before any interaction. For every modality, a temporal encoder and a spatial encoder project signals into separate temporal and spatial body. Factor-Consistent Cross-Modal Alignment then aligns temporal features only with their temporal counterparts across modalities, and spatial features only with their spatial counterparts. Factor specific supervision and decorrelation regularization reduce cross factor leakage while preserving complementarity. A Gated Recouple module subsequently recouples the aligned streams for task. Extensive experiments show that TSDA outperforms baselines. Ablation analysis studies confirm the necessity and interpretability of the design.

Method: A hierarchical framework with Base LLM, operational SLM agent, Code-Generation LLM, and Teacher-LLM. When tasks fail, agents escalate to tool synthesis via Code-Gen LLM, then trigger evolution using Curriculum Learning, Reward-Based Learning, or Genetic Algorithms.

Result: Evolved agents consistently outperform original agents across all settings. CL provides fast recovery and strong generalization, RL excels on high-difficulty tasks, and GA offers high behavioral diversity.

Conclusion: The framework demonstrates robust, autonomous, self-improving agentic evolution, enabling LLM agents to continuously adapt and expand capabilities beyond their initial deployment state.

Abstract: Large Language Model (LLM) based agents are powerful yet fundamentally static after deployment, lacking the ability to autonomously expand capabilities, generate new tools, or evolve their reasoning. This work introduces a hierarchical self-evolving multi-agent framework that integrates a Base LLM, an operational SLM agent, a Code-Generation LLM, and a Teacher-LLM to enable continuous adaptation. The workflow begins with the agent attempting a task using reasoning and existing tools; if unsuccessful, it escalates to tool synthesis through the Code-Gen LLM, and when failures persist, it triggers an evolution phase using Curriculum Learning (CL), Reward-Based Learning (RL), or Genetic Algorithm (GA) evolution. Using the TaskCraft dataset rich in hierarchical tasks, tool-use traces, and difficulty scaling we evaluate these paradigms. CL delivers fast recovery and strong generalization, RL excels on high-difficulty tasks, and GA offers high behavioral diversity. Across all settings, evolved agents outperform their originals, demonstrating robust, autonomous, self-improving agentic evolution.

Method: 1) Created FutureOmni benchmark via LLM-assisted, human-in-the-loop pipeline with 919 videos and 1,034 QA pairs across 8 domains. 2) Evaluated 13 omni-modal and 7 video-only models. 3) Curated 7K-sample instruction-tuning dataset and proposed Omni-Modal Future Forecasting (OFF) training strategy to enhance forecasting capabilities.

Result: Current systems struggle with audio-visual future prediction, especially in speech-heavy scenarios. Best accuracy of 64.8% achieved by Gemini 3 Flash. The proposed OFF training strategy enhances future forecasting and generalization across FutureOmni and other benchmarks.

Conclusion: Future forecasting from audio-visual cues remains challenging for current MLLMs. The FutureOmni benchmark fills an important gap in evaluation, and the proposed OFF training strategy shows promise for improving multimodal future prediction capabilities. All resources are publicly released.

Abstract: Although Multimodal Large Language Models (MLLMs) demonstrate strong omni-modal perception, their ability to forecast future events from audio-visual cues remains largely unexplored, as existing benchmarks focus mainly on retrospective understanding. To bridge this gap, we introduce FutureOmni, the first benchmark designed to evaluate omni-modal future forecasting from audio-visual environments. The evaluated models are required to perform cross-modal causal and temporal reasoning, as well as effectively leverage internal knowledge to predict future events. FutureOmni is constructed via a scalable LLM-assisted, human-in-the-loop pipeline and contains 919 videos and 1,034 multiple-choice QA pairs across 8 primary domains. Evaluations on 13 omni-modal and 7 video-only models show that current systems struggle with audio-visual future prediction, particularly in speech-heavy scenarios, with the best accuracy of 64.8% achieved by Gemini 3 Flash. To mitigate this limitation, we curate a 7K-sample instruction-tuning dataset and propose an Omni-Modal Future Forecasting (OFF) training strategy. Evaluations on FutureOmni and popular audio-visual and video-only benchmarks demonstrate that OFF enhances future forecasting and generalization. We publicly release all code (https://github.com/OpenMOSS/FutureOmni) and datasets (https://huggingface.co/datasets/OpenMOSS-Team/FutureOmni).

Method: 1) Compare indexing strategies for retrieval; 2) Fine-tune an LLM to use retrieval context and generate evidence-based questions; 3) Apply contrastive preference optimization to favor questions supported by retrieved passages over ungrounded alternatives.

Result: RAC demonstrates significant improvements over baselines on four benchmarks. Novel metrics using NLI and data-to-text show enhanced faithfulness, with LLM-as-Judge assessments confirming superiority.

Conclusion: The RAC framework successfully generates corpus-faithful clarification questions by integrating retrieval augmentation and contrastive optimization, addressing the critical gap of grounding questions in available documents.

Abstract: Clarification questions help conversational search systems resolve ambiguous or underspecified user queries. While prior work has focused on fluency and alignment with user intent, especially through facet extraction, much less attention has been paid to grounding clarifications in the underlying corpus. Without such grounding, systems risk asking questions that cannot be answered from the available documents. We introduce RAC (Retrieval-Augmented Clarification), a framework for generating corpus-faithful clarification questions. After comparing several indexing strategies for retrieval, we fine-tune a large language model to make optimal use of research context and to encourage the generation of evidence-based question. We then apply contrastive preference optimization to favor questions supported by retrieved passages over ungrounded alternatives. Evaluated on four benchmarks, RAC demonstrate significant improvements over baselines. In addition to LLM-as-Judge assessments, we introduce novel metrics derived from NLI and data-to-text to assess how well questions are anchored in the context, and we demonstrate that our approach consistently enhances faithfulness.

Method: The authors introduce a 4x4 landscape framework crossing four levels of meaning-making (descriptive, categorical, interpretive, theoretical) with four levels of modeling (static structure, stages/timelines, causal pathways, feedback dynamics). They apply this framework to analyze prior LLM-based automation systems.

Result: Analysis reveals a strong skew in existing systems toward low-level meaning (descriptive/categorical) and low-commitment representations (static structures/stages), with few reliable attempts at interpretive/theoretical inference or dynamical modeling.

Conclusion: The paper outlines an agenda for developing LLM systems that make their interpretive and modeling commitments explicit, selectable, and governable, addressing the identified gap in current qualitative research automation.

Abstract: LLMs are increasingly used to support qualitative research, yet existing systems produce outputs that vary widely–from trace-faithful summaries to theory-mediated explanations and system models. To make these differences explicit, we introduce a 4$\times$4 landscape crossing four levels of meaning-making (descriptive, categorical, interpretive, theoretical) with four levels of modeling (static structure, stages/timelines, causal pathways, feedback dynamics). Applying the landscape to prior LLM-based automation highlights a strong skew toward low-level meaning and low-commitment representations, with few reliable attempts at interpretive/theoretical inference or dynamical modeling. Based on the revealed gap, we outline an agenda for applying and building LLM-systems that make their interpretive and modeling commitments explicit, selectable, and governable.

Method: Proposes LLM-as-RNN framework that represents LLM hidden state as natural-language memory (structured system-prompt summary), updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates under fixed token budget.

Result: Evaluated on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT families. Significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average while producing interpretable learning traces.

Conclusion: LLM-as-RRN enables frozen LLMs to perform online learning through language, correcting errors and retaining task-relevant patterns via recurrent natural-language memory states, offering interpretable learning traces absent in standard context accumulation.

Abstract: Large language models are strong sequence predictors, yet standard inference relies on immutable context histories. After making an error at generation step t, the model lacks an updatable memory mechanism that improves predictions for step t+1. We propose LLM-as-RNN, an inference-only framework that turns a frozen LLM into a recurrent predictor by representing its hidden state as natural-language memory. This state, implemented as a structured system-prompt summary, is updated at each timestep via feedback-driven text rewrites, enabling learning without parameter updates. Under a fixed token budget, LLM-as-RNN corrects errors and retains task-relevant patterns, effectively performing online learning through language. We evaluate the method on three sequential benchmarks in healthcare, meteorology, and finance across Llama, Gemma, and GPT model families. LLM-as-RNN significantly outperforms zero-shot, full-history, and MemPrompt baselines, improving predictive accuracy by 6.5% on average, while producing interpretable, human-readable learning traces absent in standard context accumulation.

Method: LIME-LLM replaces random noise with hypothesis-driven, controlled perturbations using LLMs. It enforces a strict “Single Mask-Single Sample” protocol and employs distinct neutral infill and boundary infill strategies to construct fluent, on-manifold neighborhoods that rigorously isolate feature effects.

Result: Evaluation across three benchmarks (CoLA, SST-2, HateXplain) using human-annotated rationales as ground truth shows LIME-LLM achieves significant improvements in local explanation fidelity compared to traditional perturbation-based methods (LIME, SHAP, Integrated Gradients) and recent generative alternatives (LLiMe).

Conclusion: LIME-LLM establishes a new benchmark for black-box NLP explainability by addressing limitations of both traditional perturbation-based methods and recent generative approaches through controlled, hypothesis-driven perturbations that better isolate feature effects.

Abstract: Local explanation methods such as LIME (Ribeiro et al., 2016) remain fundamental to trustworthy AI, yet their application to NLP is limited by a reliance on random token masking. These heuristic perturbations frequently generate semantically invalid, out-of-distribution inputs that weaken the fidelity of local surrogate models. While recent generative approaches such as LLiMe (Angiulli et al., 2025b) attempt to mitigate this by employing Large Language Models for neighborhood generation, they rely on unconstrained paraphrasing that introduces confounding variables, making it difficult to isolate specific feature contributions. We introduce LIME-LLM, a framework that replaces random noise with hypothesis-driven, controlled perturbations. By enforcing a strict “Single Mask-Single Sample” protocol and employing distinct neutral infill and boundary infill strategies, LIME-LLM constructs fluent, on-manifold neighborhoods that rigorously isolate feature effects. We evaluate our method against established baselines (LIME, SHAP, Integrated Gradients) and the generative LLiMe baseline across three diverse benchmarks: CoLA, SST-2, and HateXplain using human-annotated rationales as ground truth. Empirical results demonstrate that LIME-LLM establishes a new benchmark for black-box NLP explainability, achieving significant improvements in local explanation fidelity compared to both traditional perturbation-based methods and recent generative alternatives.

Method: Used logistic regression classifier trained on Reddit posts from curated subreddits. Conducted feature ablation, keyword masking experiments, varying-density difference analyses comparing anxious vs control groups, and external validation with clinically diagnosed participants.

Result: Model achieved strong performance with high accuracy even after sentiment removal or keyword masking. Early detection with minimal post history significantly outperformed random classification. Cross-domain analysis showed strong consistency with clinical interview data.

Conclusion: Transparent linguistic features can support reliable, generalizable, and keyword-robust anxiety detection. The framework provides reproducible baseline for interpretable mental health screening across diverse online contexts.

Abstract: Anxiety affects hundreds of millions of individuals globally, yet large-scale screening remains limited. Social media language provides an opportunity for scalable detection, but current models often lack interpretability, keyword-robustness validation, and rigorous user-level data integrity. This work presents a transparent approach to social media-based anxiety detection through linguistically interpretable feature-grounded modeling and cross-domain validation. Using a substantial dataset of Reddit posts, we trained a logistic regression classifier on carefully curated subreddits for training, validation, and test splits. Comprehensive evaluation included feature ablation, keyword masking experiments, and varying-density difference analyses comparing anxious and control groups, along with external validation using clinically interviewed participants with diagnosed anxiety disorders. The model achieved strong performance while maintaining high accuracy even after sentiment removal or keyword masking. Early detection using minimal post history significantly outperformed random classification, and cross-domain analysis demonstrated strong consistency with clinical interview data. Results indicate that transparent linguistic features can support reliable, generalizable, and keyword-robust anxiety detection. The proposed framework provides a reproducible baseline for interpretable mental health screening across diverse online contexts.

Method: TIDE framework with a novel granular topic modeling method based on large language models (LLMs), plus auxiliary features including document summarization, topic parenting, and distillation for business applications.

Result: TIDE’s topic modeling outperforms modern topic modeling methods on various public and real-world business datasets. The auxiliary components effectively support industrial business scenarios.

Conclusion: TIDE provides an effective granular topic modeling solution using LLMs with valuable business application features, demonstrating superior performance and practical utility for industrial scenarios.

Abstract: Topic modeling has extensive applications in text mining and data analysis across various industrial sectors. Although the concept of granularity holds significant value for business applications by providing deeper insights, the capability of topic modeling methods to produce granular topics has not been thoroughly explored. In this context, this paper introduces a framework called TIDE, which primarily provides a novel granular topic modeling method based on large language models (LLMs) as a core feature, along with other useful functionalities for business applications, such as summarizing long documents, topic parenting, and distillation. Through extensive experiments on a variety of public and real-world business datasets, we demonstrate that TIDE’s topic modeling approach outperforms modern topic modeling methods, and our auxiliary components provide valuable support for dealing with industrial business scenarios. The TIDE framework is currently undergoing the process of being open sourced.

Method: Proposes a Toxic Signal Detector (internal self-identification mechanism) coupled with systematic intervention to transform toxic text into non-toxic counterparts. Uses iterative procedure to create contrastive detoxification dataset for fine-tuning.

Result: Achieves better detoxification performance than state-of-the-art methods on benchmark datasets (DetoxLLM and ParaDetox) while preserving semantic fidelity.

Conclusion: Reveals the intrinsic self-detoxification ability of LLMs, offering a consistent and effective approach for mitigating harmful content generation without human intervention or external components, paving the way for truly self-regulated language models.

Abstract: Recent breakthroughs in Large Language Models (LLMs) have revealed remarkable generative capabilities and emerging self-regulatory mechanisms, including self-correction and self-rewarding. However, current detoxification techniques rarely exploit these built-in abilities; instead, they rely on external modules, labor-intensive data annotation, or human intervention –factors that hinder scalability and consistency. In this paper, we introduce a fully self-reflective detoxification framework that harnesses the inherent capacities of LLMs to detect, correct toxic content, and refine LLMs without external modules and data annotation. Specifically, we propose a Toxic Signal Detector –an internal self-identification mechanism, coupled with a systematic intervention process to transform toxic text into its non-toxic counterpart. This iterative procedure yields a contrastive detoxification dataset used to fine-tune the model, enhancing its ability for safe and coherent text generation. Experiments on benchmark datasets such as DetoxLLM and ParaDetox show that our method achieves better detoxification performance than state-of-the-art methods while preserving semantic fidelity. By obviating the need for human intervention or external components, this paper reveals the intrinsic self-detoxification ability of LLMs, offering a consistent and effective approach for mitigating harmful content generation. Ultimately, our findings underscore the potential for truly self-regulated language models, paving the way for more responsible and ethically guided text generation systems.

Method: Systematically evaluated 14 models (1B-72B parameters) across 9 diverse benchmarks and 7 translation metrics to test whether translation quality alone can indicate broader multilingual capabilities.

Result: Translation performance strongly correlates with downstream task success (e.g., Phi-4 shows median Pearson r: MetricX = 0.89, xCOMET = 0.91, SSA-COMET = 0.87). The representational abilities supporting faithful translation overlap with those required for multilingual understanding.

Conclusion: Translation quality emerges as a strong, inexpensive first-pass proxy for multilingual performance, enabling translation-first screening with targeted follow-up for specific tasks, addressing the multilingual evaluation gap.

Abstract: The rapid proliferation of LLMs has created a critical evaluation paradox: while LLMs claim multilingual proficiency, comprehensive non-machine-translated benchmarks exist for fewer than 30 languages, leaving >98% of the world’s 7,000 languages in an empirical void. Traditional benchmark construction faces scaling challenges such as cost, scarcity of domain experts, and data contamination. We evaluate the validity of a simpler alternative: can translation quality alone indicate a model’s broader multilingual capabilities? Through systematic evaluation of 14 models (1B-72B parameters) across 9 diverse benchmarks and 7 translation metrics, we find that translation performance is a good indicator of downstream task success (e.g., Phi-4, median Pearson r: MetricX = 0.89, xCOMET = 0.91, SSA-COMET = 0.87). These results suggest that the representational abilities supporting faithful translation overlap with those required for multilingual understanding. Translation quality, thus emerges as a strong, inexpensive first-pass proxy of multilingual performance, enabling a translation-first screening with targeted follow-up for specific tasks.

Method: Introduces concept-level prediction where concepts group multiple surface forms of the same idea (e.g., “mom”, “mommy”, “mother” → MOTHER). Proposes various methods for integrating conceptual supervision into LLM training.

Result: Concept-aware models achieve lower perplexity, improved robustness under domain shift, and stronger performance than NTP-based models on diverse NLP benchmarks.

Conclusion: Concept-level supervision serves as an improved training signal that better aligns LLMs with human semantic abstractions, addressing limitations of token-level next-token prediction.

Abstract: The next-token prediction (NTP) objective has been foundational in the development of modern large language models (LLMs), driving advances in fluency and generalization. However, NTP operates at the \textit{token} level, treating deviations from a single reference continuation as errors even when alternative continuations are equally plausible or semantically equivalent (e.g., mom'' vs. mother’’). As a result, token-level loss can penalize valid abstractions, paraphrases, or conceptually correct reasoning paths, biasing models toward surface form rather than underlying meaning. This mismatch between the training signal and semantic correctness motivates learning objectives that operate over higher-level representations. We propose a shift from token-level to concept-level prediction, where concepts group multiple surface forms of the same idea (e.g., mom,'' mommy,’’ ``mother’’ $\rightarrow$ \textit{MOTHER}). We introduce various methods for integrating conceptual supervision into LLM training and show that concept-aware models achieve lower perplexity, improved robustness under domain shift, and stronger performance than NTP-based models on diverse NLP benchmarks. This suggests \textit{concept-level supervision} as an improved training signal that better aligns LLMs with human semantic abstractions.

Method: Created TWeddit dataset by curating Reddit stories about triggering experiences faced by women, with manual annotation and linguistic analysis of topics and moral foundations.

Result: Annotated stories in TWeddit express distinct topics and moral foundations, making the dataset useful for future research on content moderation and support systems.

Conclusion: TWeddit provides a valuable resource for research on detecting triggering content and understanding how people share traumatic experiences online, with potential applications in content moderation and support systems.

Abstract: Warning: This paper may contain examples and topics that may be disturbing to some readers, especially survivors of miscarriage and sexual violence. People affected by abortion, miscarriage, or sexual violence often share their experiences on social media to express emotions and seek support. On public platforms like Reddit, where users can post long, detailed narratives (up to 40,000 characters), readers may be exposed to distressing content. Although Reddit allows manual trigger warnings, many users omit them due to limited awareness or uncertainty about which categories apply. There is scarcity of datasets on Reddit stories labeled for triggering experiences. We propose a curated Reddit dataset, TWeddit, covering triggering experiences related to issues majorly faced by women. Our linguistic analyses show that annotated stories in TWeddit express distinct topics and moral foundations, making the dataset useful for a wide range of future research.

Method: Organized a challenge framework with defined anonymization tasks, datasets for development/evaluation, attack models, objective evaluation metrics, six baseline systems, and participant submissions.

Result: Provided systematic overview of challenge framework, described baseline systems and participant approaches, and delivered key insights for future voice anonymization research.

Conclusion: The challenge successfully advanced voice anonymization technology and identified promising research directions for future development in speaker privacy protection.

Abstract: We present results and analyses from the third VoicePrivacy Challenge held in 2024, which focuses on advancing voice anonymization technologies. The task was to develop a voice anonymization system for speech data that conceals a speaker’s voice identity while preserving linguistic content and emotional state. We provide a systematic overview of the challenge framework, including detailed descriptions of the anonymization task and datasets used for both system development and evaluation. We outline the attack model and objective evaluation metrics for assessing privacy protection (concealing speaker voice identity) and utility (content and emotional state preservation). We describe six baseline anonymization systems and summarize the innovative approaches developed by challenge participants. Finally, we provide key insights and observations to guide the design of future VoicePrivacy challenges and identify promising directions for voice anonymization research.

Method: Proposes Cross-Tokenizer Preference Distillation (CTPD) with three innovations: (1) Aligned Span Projection mapping tokens to shared character-level spans, (2) cross-tokenizer adaptation of Token-level Importance Sampling (TIS-DPO), and (3) Teacher-Anchored Reference for DPO-style objectives.

Result: Experiments across multiple benchmarks show significant performance gains over existing methods, confirming CTPD’s effectiveness in transferring human-aligned behavior between models with heterogeneous tokenizers.

Conclusion: CTPD establishes a practical and general solution for preference distillation across diverse tokenization schemes, enabling more accessible and efficient alignment of language models.

Abstract: While knowledge distillation has seen widespread use in pre-training and instruction tuning, its application to aligning language models with human preferences remains underexplored, particularly in the more realistic cross-tokenizer setting. The incompatibility of tokenization schemes between teacher and student models has largely prevented fine-grained, white-box distillation of preference information. To address this gap, we propose Cross-Tokenizer Preference Distillation (CTPD), the first unified framework for transferring human-aligned behavior between models with heterogeneous tokenizers. CTPD introduces three key innovations: (1) Aligned Span Projection, which maps teacher and student tokens to shared character-level spans for precise supervision transfer; (2) a cross-tokenizer adaptation of Token-level Importance Sampling (TIS-DPO) for improved credit assignment; and (3) a Teacher-Anchored Reference, allowing the student to directly leverage the teacher’s preferences in a DPO-style objective. Our theoretical analysis grounds CTPD in importance sampling, and experiments across multiple benchmarks confirm its effectiveness, with significant performance gains over existing methods. These results establish CTPD as a practical and general solution for preference distillation across diverse tokenization schemes, opening the door to more accessible and efficient alignment of language models.

Method: Tested reasoning models from OpenAI, Grok, Gemini, Claude, and DeepSeek families on the medicine abstraction and reasoning corpus (mARC), an adversarial medical QA benchmark designed to induce the Einstellung effect (inflexible overreliance on learned patterns).

Result: Strong reasoning models avoided Einstellung-based traps more often than weaker models, achieving human-level performance on mARC. Top 5 models answered 55% to 70% correctly on questions most commonly missed by physicians, with high confidence, indicating less susceptibility to Einstellung effects than humans.

Conclusion: Advanced reasoning LLMs demonstrate improved flexibility in medical reasoning, performing on par with humans while potentially being less vulnerable to cognitive biases that affect human clinical decision-making.

Abstract: Large Language Models (LLMs) have achieved high accuracy on medical question-answer (QA) benchmarks, yet their capacity for flexible clinical reasoning has been debated. Here, we asked whether advances in reasoning LLMs improve their cognitive flexibility in clinical reasoning. We assessed reasoning models from the OpenAI, Grok, Gemini, Claude, and DeepSeek families on the medicine abstraction and reasoning corpus (mARC), an adversarial medical QA benchmark which utilizes the Einstellung effect to induce inflexible overreliance on learned heuristic patterns in contexts where they become suboptimal. We found that strong reasoning models avoided Einstellung-based traps more often than weaker reasoning models, achieving human-level performance on mARC. On questions most commonly missed by physicians, the top 5 performing models answered 55% to 70% correctly with high confidence, indicating that these models may be less susceptible than humans to Einstellung effects. Our results indicate that strong reasoning models demonstrate improved flexibility in medical reasoning, achieving performance on par with humans on mARC.

Method: GloCTM constructs enriched input representations by expanding bag-of-words with cross-lingual lexical neighborhoods, uses both local and global encoders with internal regularization for topic proportion inference, defines global topic-word distributions over combined vocabulary, and incorporates Centered Kernel Alignment (CKA) loss to align latent topic space with multilingual contextual embeddings.

Result: Experiments across multiple benchmarks demonstrate that GloCTM significantly improves topic coherence and cross-lingual alignment, outperforming strong baselines.

Conclusion: GloCTM provides a novel framework that enforces cross-lingual topic alignment through a unified semantic space spanning the entire model pipeline, effectively capturing deep cross-lingual semantics and improving multilingual topic modeling performance.

Abstract: Cross-lingual topic modeling seeks to uncover coherent and semantically aligned topics across languages - a task central to multilingual understanding. Yet most existing models learn topics in disjoint, language-specific spaces and rely on alignment mechanisms (e.g., bilingual dictionaries) that often fail to capture deep cross-lingual semantics, resulting in loosely connected topic spaces. Moreover, these approaches often overlook the rich semantic signals embedded in multilingual pretrained representations, further limiting their ability to capture fine-grained alignment. We introduce GloCTM (Global Context Space for Cross-Lingual Topic Model), a novel framework that enforces cross-lingual topic alignment through a unified semantic space spanning the entire model pipeline. GloCTM constructs enriched input representations by expanding bag-of-words with cross-lingual lexical neighborhoods, and infers topic proportions using both local and global encoders, with their latent representations aligned through internal regularization. At the output level, the global topic-word distribution, defined over the combined vocabulary, structurally synchronizes topic meanings across languages. To further ground topics in deep semantic space, GloCTM incorporates a Centered Kernel Alignment (CKA) loss that aligns the latent topic space with multilingual contextual embeddings. Experiments across multiple benchmarks demonstrate that GloCTM significantly improves topic coherence and cross-lingual alignment, outperforming strong baselines.

Method: Created MedCounterFact dataset with counterfactual medical QA requiring clinical comparison judgments. Real-world medical interventions systematically replaced with four types of counterfactual stimuli (unknown words to toxic substances). Evaluated multiple frontier LLMs on this dataset.

Result: Models overwhelmingly accept counterfactual evidence at face value even when dangerous/implausible, providing confident, uncaveated answers. No existing boundary between faithfulness and safety.

Conclusion: Current LLMs lack proper safety boundaries when faced with counterfactual medical evidence, accepting dangerous information without critical evaluation, highlighting a critical safety gap.

Abstract: In high-stakes domains like medicine, it may be generally desirable for models to faithfully adhere to the context provided. But what happens if the context does not align with model priors or safety protocols? In this paper, we investigate how LLMs behave and reason when presented with counterfactual or even adversarial medical evidence. We first construct MedCounterFact, a counterfactual medical QA dataset that requires the models to answer clinical comparison questions (i.e., judge the efficacy of certain treatments, with evidence consisting of randomized controlled trials provided as context). In MedCounterFact, real-world medical interventions within the questions and evidence are systematically replaced with four types of counterfactual stimuli, ranging from unknown words to toxic substances. Our evaluation across multiple frontier LLMs on MedCounterFact reveals that in the presence of counterfactual evidence, existing models overwhelmingly accept such “evidence” at face value even when it is dangerous or implausible, and provide confident and uncaveated answers. While it may be prudent to draw a boundary between faithfulness and safety, our findings reveal that there exists no such boundary yet.

Method: PPA-Plan proactively identifies potential logical pitfalls and false assumptions, formulates them as negative constraints, and conditions plan generation on explicitly avoiding these constraints before generating the plan.

Result: Experiments on long-context QA benchmarks show that executing plans generated by PPA-Plan consistently outperforms existing plan-and-execute methods and direct prompting.

Conclusion: Proactive planning that prevents failures before plan generation is more effective than reactive refinement for long-context reasoning with LLMs.

Abstract: Large language models (LLMs) struggle with reasoning over long contexts where relevant information is sparsely distributed. Although plan-and-execute frameworks mitigate this by decomposing tasks into planning and execution, their effectiveness is often limited by unreliable plan generation due to dependence on surface-level cues. Consequently, plans may be based on incorrect assumptions, and once a plan is formed, identifying what went wrong and revising it reliably becomes difficult, limiting the effectiveness of reactive refinement. To address this limitation, we propose PPA-Plan, a proactive planning strategy for long-context reasoning that focuses on preventing such failures before plan generation. PPA-Plan identifies potential logical pitfalls and false assumptions, formulates them as negative constraints, and conditions plan generation on explicitly avoiding these constraints. Experiments on long-context QA benchmarks show that executing plans generated by PPA-Plan consistently outperforms existing plan-and-execute methods and direct prompting.

Method: TRACE (Task-decomposed Reasoning for Affective Communication and Empathy) models empathy as a structured cognitive process by decomposing the task into a pipeline for analysis and synthesis, building comprehensive understanding before generation.

Result: Experimental results show TRACE significantly outperforms strong baselines in both automatic and LLM-based evaluations, demonstrating superior empathetic response generation capabilities.

Conclusion: The structured decomposition approach of TRACE is a promising paradigm for creating more capable and interpretable empathetic agents, successfully uniting deep analysis with expressive generation.

Abstract: Empathetic response generation is a crucial task for creating more human-like and supportive conversational agents. However, existing methods face a core trade-off between the analytical depth of specialized models and the generative fluency of Large Language Models (LLMs). To address this, we propose TRACE, Task-decomposed Reasoning for Affective Communication and Empathy, a novel framework that models empathy as a structured cognitive process by decomposing the task into a pipeline for analysis and synthesis. By building a comprehensive understanding before generation, TRACE unites deep analysis with expressive generation. Experimental results show that our framework significantly outperforms strong baselines in both automatic and LLM-based evaluations, confirming that our structured decomposition is a promising paradigm for creating more capable and interpretable empathetic agents. Our code is available at https://anonymous.4open.science/r/TRACE-18EF/README.md.

Method: LSTM-MAS organizes agents in a chained architecture with specialized roles: worker agents for segment comprehension, filter agents for redundancy reduction, judge agents for error detection, and manager agents for global information regulation, mimicking LSTM’s gated memory mechanisms.

Result: Achieved improvements of 40.93% on NarrativeQA, 43.70% on Qasper, 121.57% on HotpotQA, and 33.12% on MuSiQue compared to previous best multi-agent approach (CoA).

Conclusion: The LSTM-inspired multi-agent system effectively addresses long-context understanding challenges by enabling controlled information transfer and selective long-term dependency modeling, preventing error accumulation and hallucination propagation.

Abstract: Effectively processing long contexts remains a fundamental yet unsolved challenge for large language models (LLMs). Existing single-LLM-based methods primarily reduce the context window or optimize the attention mechanism, but they often encounter additional computational costs or constrained expanded context length. While multi-agent-based frameworks can mitigate these limitations, they remain susceptible to the accumulation of errors and the propagation of hallucinations. In this work, we draw inspiration from the Long Short-Term Memory (LSTM) architecture to design a Multi-Agent System called LSTM-MAS, emulating LSTM’s hierarchical information flow and gated memory mechanisms for long-context understanding. Specifically, LSTM-MAS organizes agents in a chained architecture, where each node comprises a worker agent for segment-level comprehension, a filter agent for redundancy reduction, a judge agent for continuous error detection, and a manager agent for globally regulates information propagation and retention, analogous to LSTM and its input gate, forget gate, constant error carousel unit, and output gate. These novel designs enable controlled information transfer and selective long-term dependency modeling across textual segments, which can effectively avoid error accumulation and hallucination propagation. We conducted an extensive evaluation of our method. Compared with the previous best multi-agent approach, CoA, our model achieves improvements of 40.93%, 43.70%,121.57% and 33.12%, on NarrativeQA, Qasper, HotpotQA, and MuSiQue, respectively.

Method: Uses existing open-source ASR corpora with linguistically-informed curriculum learning to support high- to low-resource Arabic dialects, featuring effective in-context learning without text diacritization.

Result: Outperforms leading commercial services in generation quality while maintaining extensibility, and creates the first systematic benchmark for multi-dialect Arabic speech synthesis.

Conclusion: Habibi bridges the Arabic dialect TTS research gap, provides evaluation standards, and establishes groundwork for future research through open-sourced models and benchmarks.

Abstract: A notable gap persists in speech synthesis research and development for Arabic dialects, particularly from a unified modeling perspective. Despite its high practical value, the inherent linguistic complexity of Arabic dialects, further compounded by a lack of standardized data, benchmarks, and evaluation guidelines, steers researchers toward safer ground. To bridge this divide, we present Habibi, a suite of specialized and unified text-to-speech models that harnesses existing open-source ASR corpora to support a wide range of high- to low-resource Arabic dialects through linguistically-informed curriculum learning. Our approach outperforms the leading commercial service in generation quality, while maintaining extensibility through effective in-context learning, without requiring text diacritization. We are committed to open-sourcing the model, along with creating the first systematic benchmark for multi-dialect Arabic speech synthesis. Furthermore, by identifying the key challenges in and establishing evaluation standards for the process, we aim to provide a solid groundwork for subsequent research. Resources at https://SWivid.github.io/Habibi/ .

Method: Develops a diagnostic paradigm with: 1) taxonomy categorizing LLM errors by source (model-specific vs. task-inherent) and type (boundary ambiguity vs. conceptual misidentification); 2) lightweight human annotation test to estimate task-inherent ambiguity; 3) computational method to decompose observed LLM errors following the taxonomy.

Result: Validated on four educational annotation tasks, demonstrating conceptual validity and practical utility. Shows why excessively high alignment is unrealistic for certain tasks and why single metrics inadequately reflect LLM annotation quality.

Conclusion: The paradigm serves as a low-cost diagnostic tool to assess task suitability for LLM annotation and provides actionable insights for technical optimization, moving beyond simplistic alignment metrics to understand error sources in subjective annotation.

Abstract: Large language models offer a scalable alternative to human coding for data annotation tasks, enabling the scale-up of research across data-intensive domains. While LLMs are already achieving near-human accuracy on objective annotation tasks, their performance on subjective annotation tasks, such as those involving psychological constructs, is less consistent and more prone to errors. Standard evaluation practices typically collapse all annotation errors into a single alignment metric, but this simplified approach may obscure different kinds of errors that affect final analytical conclusions in different ways. Here, we propose a diagnostic evaluation paradigm that incorporates a human-in-the-loop step to separate task-inherent ambiguity from model-driven inaccuracies and assess annotation quality in terms of their potential downstream impacts. We refine this paradigm on ordinal annotation tasks, which are common in subjective annotation. The refined paradigm includes: (1) a diagnostic taxonomy that categorizes LLM annotation errors along two dimensions: source (model-specific vs. task-inherent) and type (boundary ambiguity vs. conceptual misidentification); (2) a lightweight human annotation test to estimate task-inherent ambiguity from LLM annotations; and (3) a computational method to decompose observed LLM annotation errors following our taxonomy. We validate this paradigm on four educational annotation tasks, demonstrating both its conceptual validity and practical utility. Theoretically, our work provides empirical evidence for why excessively high alignment is unrealistic in specific annotation tasks and why single alignment metrics inadequately reflect the quality of LLM annotations. In practice, our paradigm can be a low-cost diagnostic tool that assesses the suitability of a given task for LLM annotation and provides actionable insights for further technical optimization.

Method: Conducted large-scale analysis of 69,745 publications from 2000-2025, identifying cyclical evolution patterns (define-ideate-test pattern) and using theme modeling workflow to determine stable thematic structure with five dominant axes.

Result: Identified five thematic axes: cancer types, supportive care, clinical endpoints, mechanisms, and methodology. Found cyclical evolution patterns in publication output, collaboration, and funding. Cross-theme integration is patient-centered and systems-oriented. Field shows progressive specialization and potential saturation. Evidence suggests system-wide positive reporting bias across publication types and thematic areas.

Conclusion: The field has matured its current research agenda and is likely at the cusp of something new. The analysis reveals both progressive specialization and potential defragmentation/saturation, along with concerning homogeneous positive reporting bias that appears agnostic to structural drivers.

Abstract: The integration of complementary medicine into oncology represents a paradigm shift that has seen to increasing adoption of Traditional Chinese Medicine (TCM) as an adjuvant to radiotherapy. About twenty-five years since the formal institutionalization of integrated oncology, it is opportune to synthesize the trajectory of evidence for TCM as an adjuvant to radiotherapy. Here we conduct a large-scale analysis of 69,745 publications (2000 - 2025), emerging a cyclical evolution defined by coordinated expansion and contraction in publication output, international collaboration, and funding commitments that mirrors a define-ideate-test pattern. Using a theme modeling workflow designed to determine a stable thematic structure of the field, we identify five dominant thematic axes - cancer types, supportive care, clinical endpoints, mechanisms, and methodology - that signal a focus on patient well-being, scientific rigor and mechanistic exploration. Cross-theme integration of TCM is patient-centered and systems-oriented. Together with the emergent cycles of evolution, the thematic structure demonstrates progressive specialization and potential defragmentation of the field or saturation of existing research agenda. The analysis points to a field that has matured its current research agenda and is likely at the cusp of something new. Additionally, the field exhibits positive reporting of findings that is homogeneous across publication types, thematic areas, and the cycles of evolution suggesting a system-wide positive reporting bias agnostic to structural drivers.

Method: Enhanced models with sentence context to provide bidirectional information, used Low-Rank Adaptation (LoRA) during finetuning to improve performance, and evaluated using Macro-F1 scores instead of Micro-F1 to better assess performance across long-tailed event classes.

Result: Models with sentence context outperformed canonical decoder-only baselines. LoRA finetuning provided substantial Macro-F1 improvements, especially for decoder-only models, demonstrating LoRA’s effectiveness for enhancing LLM performance on long-tailed event classes.

Conclusion: Bidirectional context and Macro-F1 evaluation are crucial for event detection. LoRA finetuning effectively enhances LLM performance on long-tailed event types, addressing architectural limitations and evaluation biases in current event detection research.

Abstract: The current state of event detection research has two notable re-occurring limitations that we investigate in this study. First, the unidirectional nature of decoder-only LLMs presents a fundamental architectural bottleneck for natural language understanding tasks that depend on rich, bidirectional context. Second, we confront the conventional reliance on Micro-F1 scores in event detection literature, which systematically inflates performance by favoring majority classes. Instead, we focus on Macro-F1 as a more representative measure of a model’s ability across the long-tail of event types. Our experiments demonstrate that models enhanced with sentence context achieve superior performance over canonical decoder-only baselines. Using Low-Rank Adaptation (LoRA) during finetuning provides a substantial boost in Macro-F1 scores in particular, especially for the decoder-only models, showing that LoRA can be an effective tool to enhance LLMs’ performance on long-tailed event classes.

Method: PRiSM standardizes transcription-based evaluation and assesses downstream utility through transcription and representation probes across clinical, educational, and multilingual settings. It evaluates various PR systems including encoder-CTC models and Large Audio Language Models.

Result: Diverse language exposure during training is key to PR performance, encoder-CTC models are the most stable, and specialized PR models still outperform Large Audio Language Models despite their scale.

Conclusion: PRiSM provides the first comprehensive benchmark for phone recognition evaluation, releasing code, recipes, and datasets to advance multilingual speech models with robust phonetic capabilities.

Abstract: Phone recognition (PR) serves as the atomic interface for language-agnostic modeling for cross-lingual speech processing and phonetic analysis. Despite prolonged efforts in developing PR systems, current evaluations only measure surface-level transcription accuracy. We introduce PRiSM, the first open-source benchmark designed to expose blind spots in phonetic perception through intrinsic and extrinsic evaluation of PR systems. PRiSM standardizes transcription-based evaluation and assesses downstream utility in clinical, educational, and multilingual settings with transcription and representation probes. We find that diverse language exposure during training is key to PR performance, encoder-CTC models are the most stable, and specialized PR models still outperform Large Audio Language Models. PRiSM releases code, recipes, and datasets to move the field toward multilingual speech models with robust phonetic ability: https://github.com/changelinglab/prism.

Method: DoublyCal employs a novel double-calibration principle: first uses a lightweight proxy model to generate KG evidence with calibrated confidence scores, then uses this calibrated evidence to guide black-box LLMs for final predictions.

Result: Experiments on knowledge-intensive benchmarks show DoublyCal significantly improves both accuracy and confidence calibration of black-box LLMs while maintaining low token cost.

Conclusion: The framework enables more trustworthy reasoning by providing traceable uncertainty from supporting evidence to final predictions, addressing the hallucination problem in LLMs.

Abstract: Trustworthy reasoning in Large Language Models (LLMs) is challenged by their propensity for hallucination. While augmenting LLMs with Knowledge Graphs (KGs) improves factual accuracy, existing KG-augmented methods fail to quantify epistemic uncertainty in both the retrieved evidence and LLMs’ reasoning. To bridge this gap, we introduce DoublyCal, a framework built on a novel double-calibration principle. DoublyCal employs a lightweight proxy model to first generate KG evidence alongside a calibrated evidence confidence. This calibrated supporting evidence then guides a black-box LLM, yielding final predictions that are not only more accurate but also well-calibrated, with confidence scores traceable to the uncertainty of the supporting evidence. Experiments on knowledge-intensive benchmarks show that DoublyCal significantly improves both the accuracy and confidence calibration of black-box LLMs with low token cost.

Method: Introduces CalConflictBench benchmark for sequential calendar conflict resolution with feedback. Proposes PEARL framework with reinforcement learning, external memory module, and optimized round-wise reward design to help agents infer and adapt to user preferences progressively.

Result: Current LLM agents perform poorly (e.g., Qwen-3-30B-Think has 35% average error rate). PEARL achieves 0.76 error reduction rate and 55% improvement in average error rate compared to the strongest baseline.

Conclusion: PEARL demonstrates that reinforcement learning with external memory and optimized rewards significantly improves LLM agents’ ability to resolve calendar conflicts by learning user preferences on-the-fly.

Abstract: Overlapping calendar invitations force busy professionals to repeatedly decide which meetings to attend, reschedule, or decline. We refer to this preference-driven decision process as calendar conflict resolution. Automating such process is crucial yet challenging. Scheduling logistics drain hours, and human delegation often fails at scale, which motivate we to ask: Can we trust large language model (LLM) or language agent to manager time? To enable systematic study of this question, we introduce CalConflictBench, a benchmark for long-horizon calendar conflict resolution. Conflicts are presented sequentially and agents receive feedback after each round, requiring them to infer and adapt to user preferences progressively. Our experiments show that current LLM agents perform poorly with high error rates, e.g., Qwen-3-30B-Think has 35% average error rate. To address this gap, we propose PEARL, a reinforcement-learning framework that augments language agent with an external memory module and optimized round-wise reward design, enabling agent to progressively infer and adapt to user preferences on-the-fly. Experiments on CalConflictBench shows that PEARL achieves 0.76 error reduction rate, and 55% improvement in average error rate compared to the strongest baseline.

Method: Created MemoryRewardBench benchmark covering both long-context comprehension and long-form generation tasks with 10 distinct settings featuring different memory management patterns. Evaluated 13 cutting-edge reward models on this benchmark.

Result: Evaluation shows diminishing performance gap between open-source and proprietary models, with newer-generation models consistently outperforming predecessors regardless of parameter count. The benchmark exposes both capabilities and fundamental limitations of current RMs in evaluating LLM memory management.

Conclusion: MemoryRewardBench provides a systematic framework for studying reward models’ ability to evaluate long-term memory management, revealing important trends in model performance and highlighting current limitations in memory assessment capabilities.

Abstract: Existing works increasingly adopt memory-centric mechanisms to process long contexts in a segment manner, and effective memory management is one of the key capabilities that enables large language models to effectively propagate information across the entire sequence. Therefore, leveraging reward models (RMs) to automatically and reliably evaluate memory quality is critical. In this work, we introduce $\texttt{MemoryRewardBench}$, the first benchmark to systematically study the ability of RMs to evaluate long-term memory management processes. $\texttt{MemoryRewardBench}$ covers both long-context comprehension and long-form generation tasks, featuring 10 distinct settings with different memory management patterns, with context length ranging from 8K to 128K tokens. Evaluations on 13 cutting-edge RMs indicate a diminishing performance gap between open-source and proprietary models, with newer-generation models consistently outperforming their predecessors regardless of parameter count. We further expose the capabilities and fundamental limitations of current RMs in evaluating LLM memory management across diverse settings.

Method: Two-stage approach: 1) SORG framework prompts LLMs to generate agree/disagree reasoning pairs for news titles without ground-truth labels. 2) ORCD model uses three BERT encoders for title and reasoning, with contrastive learning guided by LLM-generated credibility scores.

Result: Experimental evaluations on three benchmark datasets show the method consistently outperforms LLM prompting, fine-tuned smaller language models, and state-of-the-art clickbait detection baselines.

Conclusion: Sycophancy in LLMs can be productively leveraged rather than eliminated, enabling generation of contrasting reasoning that improves clickbait detection through a novel framework combining LLM reasoning generation with specialized detection models.

Abstract: The widespread proliferation of online content has intensified concerns about clickbait, deceptive or exaggerated headlines designed to attract attention. While Large Language Models (LLMs) offer a promising avenue for addressing this issue, their effectiveness is often hindered by Sycophancy, a tendency to produce reasoning that matches users’ beliefs over truthful ones, which deviates from instruction-following principles. Rather than treating sycophancy as a flaw to be eliminated, this work proposes a novel approach that initially harnesses this behavior to generate contrastive reasoning from opposing perspectives. Specifically, we design a Self-renewal Opposing-stance Reasoning Generation (SORG) framework that prompts LLMs to produce high-quality agree and disagree reasoning pairs for a given news title without requiring ground-truth labels. To utilize the generated reasoning, we develop a local Opposing Reasoning-based Clickbait Detection (ORCD) model that integrates three BERT encoders to represent the title and its associated reasoning. The model leverages contrastive learning, guided by soft labels derived from LLM-generated credibility scores, to enhance detection robustness. Experimental evaluations on three benchmark datasets demonstrate that our method consistently outperforms LLM prompting, fine-tuned smaller language models, and state-of-the-art clickbait detection baselines.

Method: Systematic evaluation of static and dynamic quantization methods on fairness (English, French, Dutch, Spanish, Turkish) and safety (English, Korean, Arabic) benchmarks measuring intrinsic/extrinsic bias and safety alignment. Introduces Critical Weight Protection technique that identifies and preserves fairness- and safety-critical weights during quantization.

Result: Quantization consistently degrades fairness and safety; dynamic methods show greater stability than static ones; fairness degradation varies across languages; safety deterioration is especially pronounced in non-English settings; Critical Weight Protection effectively mitigates bias and safety deterioration without costly retraining.

Conclusion: Quantization poses significant fairness and safety risks, particularly in multilingual contexts. Critical Weight Protection offers a practical solution to maintain trustworthiness while retaining efficiency benefits of quantization, addressing these risks without requiring expensive retraining or alignment procedures.

Abstract: Quantization is widely adopted to reduce the computational cost of large language models (LLMs); however, its implications for fairness and safety, particularly in dynamic quantization and multilingual contexts, remain underexplored. In this work, we conduct a systematic study of how static and dynamic quantization methods impact fairness and safety across benchmarks measuring intrinsic and extrinsic bias and safety alignment. For fairness, we evaluate English, French, Dutch, Spanish, and Turkish; for safety, we focus on English, Korean, and Arabic. Our findings reveal that quantization consistently degrades fairness and safety, with dynamic methods demonstrating greater stability than static ones. Moreover, fairness degradation varies across languages, while safety deterioration is especially pronounced in non-English settings. To address these risks, we introduce Critical Weight Protection, a novel technique that identifies and preserves fairness- and safety-critical weights during quantization. This approach effectively mitigates bias and safety deterioration without costly retraining or alignment, maintaining trustworthiness while retaining efficiency.

Method: PUMA uses a parameter-efficient adapter to bridge semantic gaps between models, combined with a group-based user selection strategy to reduce training costs. It decouples user assets from underlying models.

Result: Experiments on three large-scale datasets show PUMA matches or surpasses retraining performance while reducing computational cost by up to 98%. It generalizes across diverse architectures and handles advanced scenarios like chained/aggregated migrations.

Conclusion: PUMA offers a practical, sustainable solution for evolving personalized AI by enabling efficient prompt migration across model upgrades, decoupling user assets from foundation models.

Abstract: Personalization in Large Language Models (LLMs) often relies on user-specific soft prompts. However, these prompts become obsolete when the foundation model is upgraded, necessitating costly, full-scale retraining. To overcome this limitation, we propose the Prompt-level User Migration Adapter (PUMA), a lightweight framework to efficiently migrate personalized prompts across incompatible models. PUMA utilizes a parameter-efficient adapter to bridge the semantic gap, combined with a group-based user selection strategy to significantly reduce training costs. Experiments on three large-scale datasets show our method matches or even surpasses the performance of retraining from scratch, reducing computational cost by up to 98%. The framework demonstrates strong generalization across diverse model architectures and robustness in advanced scenarios like chained and aggregated migrations, offering a practical path for the sustainable evolution of personalized AI by decoupling user assets from the underlying models.

Method: Proposed codebook-injected segmentation that conditions boundary decisions on downstream annotation criteria. Evaluated LLM-based segmenters against standard and retrieval-augmented baselines. Introduced evaluation metrics for span consistency, distinctiveness, and human-AI distributional agreement without requiring gold labels.

Result: DA-awareness produces segments that are internally more consistent than text-only baselines. LLMs excel at creating construct-consistent spans, but coherence-based baselines remain superior at detecting global dialogue flow shifts. No single segmenter dominates across two datasets, with improvements in within-segment coherence trading off against boundary distinctiveness and human-AI distributional agreement.

Conclusion: Segmentation is a consequential design choice that should be optimized for downstream objectives rather than a single performance score. The trade-offs between different segmentation qualities suggest practitioners should select segmentation approaches based on their specific annotation goals and priorities.

Abstract: Dialogue Act (DA) annotation typically treats communicative or pedagogical intent as localized to individual utterances or turns. This leads annotators to agree on the underlying action while disagreeing on segment boundaries, reducing apparent reliability. We propose codebook-injected segmentation, which conditions boundary decisions on downstream annotation criteria, and evaluate LLM-based segmenters against standard and retrieval-augmented baselines. To assess these without gold labels, we introduce evaluation metrics for span consistency, distinctiveness, and human-AI distributional agreement. We found DA-awareness produces segments that are internally more consistent than text-only baselines. While LLMs excel at creating construct-consistent spans, coherence-based baselines remain superior at detecting global shifts in dialogue flow. Across two datasets, no single segmenter dominates. Improvements in within-segment coherence frequently trade off against boundary distinctiveness and human-AI distributional agreement. These results highlight segmentation as a consequential design choice that should be optimized for downstream objectives rather than a single performance score.

Method: Developed a comprehensive Bangla symptoms-disease dataset with 758 unique symptom-disease relationships spanning 85 diseases, then evaluated multiple machine learning models using both soft and hard voting ensemble approaches.

Result: Ensemble models achieved 98% accuracy in predicting diseases based on Bangla symptom inputs, demonstrating superior robustness and generalization compared to individual models.

Conclusion: The work establishes a foundational resource for disease prediction in Bangla, enhancing equitable access to health information for Bangla-speaking communities and paving the way for future localized health informatics advancements.

Abstract: Increased access to reliable health information is essential for non-English-speaking populations, yet resources in Bangla for disease prediction remain limited. This study addresses this gap by developing a comprehensive Bangla symptoms-disease dataset containing 758 unique symptom-disease relationships spanning 85 diseases. To ensure transparency and reproducibility, we also make our dataset publicly available. The dataset enables the prediction of diseases based on Bangla symptom inputs, supporting healthcare accessibility for Bengali-speaking populations. Using this dataset, we evaluated multiple machine learning models to predict diseases based on symptoms provided in Bangla and analyzed their performance on our dataset. Both soft and hard voting ensemble approaches combining top-performing models achieved 98% accuracy, demonstrating superior robustness and generalization. Our work establishes a foundational resource for disease prediction in Bangla, paving the way for future advancements in localized health informatics and diagnostic tools. This contribution aims to enhance equitable access to health information for Bangla-speaking communities, particularly for early disease detection and healthcare interventions.

Method: Extract an arbitration vector from model activations on curated datasets that disentangle irrelevant contexts (eliciting parametric recall) and relevant but false contexts (eliciting copying). The vector is computed as residual-stream centroid difference between these regimes across 27 relations, then injected as additive intervention at selected layers and token spans to steer behavior.

Result: Experiments on two architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling. Mechanistic analyses reveal asymmetry: inducing copying is an easy “reactivation” process, while restoring recall is a more fragile “suppression” process tied to object-token interventions.

Conclusion: The study provides mechanistic insights into how language models arbitrate between parametric and contextual knowledge, revealing fundamental asymmetries in the processes of copying versus recall that have implications for retrieval-augmented generation systems.

Abstract: Language models used in retrieval-augmented settings must arbitrate between parametric knowledge stored in their weights and contextual information in the prompt. This work presents a mechanistic study of that choice by extracting an \emph{arbitration vector} from model activations on a curated dataset designed to disentangle (i) irrelevant contexts that elicit parametric recall and (ii) relevant but false contexts that elicit copying. The vector is computed as the residual-stream centroid difference between these regimes across 27 relations, and is injected as an additive intervention at selected layers and token spans to steer behavior in two directions: Copy$\rightarrow$Recall (suppressing context use) and Recall$\rightarrow$Copy (inducing the model to copy any token from the context). Experiments on two architectures (decoder-only and encoder/decoder) and two open-domain QA benchmarks show consistent behavior shifts under moderate scaling while monitoring accuracy and fluency. Mechanistic analyses of attention routing, MLP contributions, and layer-wise probability trajectories reveal an asymmetry: inducing copying is an easy reactivation'' process that can be triggered at different locations in the input, while restoring recall is a suppression’’ process that is more fragile and strongly tied to object-token interventions.

Method: K-Function framework combines Kids-Weighted Finite State Transducer (K-WFST) for accurate sub-word transcription with Large Language Model (LLM)-driven scoring. K-WFST merges acoustic phoneme encoder with phoneme-similarity model to capture child-specific speech errors while maintaining interpretability.

Result: K-WFST achieves 1.39% phoneme error rate on MyST and 8.61% on Multitudes - absolute improvements of 10.47% and 7.06% over greedy-search decoder. LLM-based scoring for verbal skills, developmental milestones, reading, and comprehension aligns closely with human evaluators.

Conclusion: Precise phoneme recognition is essential for creating effective child language assessment frameworks, enabling scalable language screening for children through the combination of accurate speech recognition and LLM-driven evaluation.

Abstract: Evaluating young children’s language is challenging for automatic speech recognizers due to high-pitched voices, prolonged sounds, and limited data. We introduce K-Function, a framework that combines accurate sub-word transcription with objective, Large Language Model (LLM)-driven scoring. Its core, Kids-Weighted Finite State Transducer (K-WFST), merges an acoustic phoneme encoder with a phoneme-similarity model to capture child-specific speech errors while remaining fully interpretable. K-WFST achieves a 1.39 % phoneme error rate on MyST and 8.61 % on Multitudes-an absolute improvement of 10.47 % and 7.06 % over a greedy-search decoder. These high-quality transcripts are used by an LLM to grade verbal skills, developmental milestones, reading, and comprehension, with results that align closely with human evaluators. Our findings show that precise phoneme recognition is essential for creating an effective assessment framework, enabling scalable language screening for children.

Method: PURPLE treats profile construction as a set generation process using a Plackett-Luce ranking model to capture inter-record dependencies, trained with dense feedback from reference response likelihood to align retrieval with generation quality.

Result: Extensive experiments on nine personalization tasks show PURPLE consistently outperforms strong heuristic and retrieval-augmented baselines in both effectiveness and efficiency.

Conclusion: PURPLE establishes a principled and scalable solution for optimizing user profiles in LLM personalization by directly aligning retrieval with generation quality rather than relying on semantic relevance.

Abstract: Large Language Models (LLMs) excel at general-purpose tasks, yet adapting their responses to individual users remains challenging. Retrieval augmentation provides a lightweight alternative to fine-tuning by conditioning LLMs on user history records, and existing approaches typically select these records based on semantic relevance. We argue that relevance serves as an unreliable proxy for utility: a record may be semantically similar to a query yet fail to improve generation quality or even degrade it due to redundancy or conflicting information. To bridge this gap, we propose PURPLE, a contextual bandit framework that oPtimizes UseR Profiles for Llm pErsonalization. In contrast to a greedy selection of the most relevant records, PURPLE treats profile construction as a set generation process and utilizes a Plackett-Luce ranking model to capture complex inter-record dependencies. By training with dense feedback provided by the likelihood of the reference response, our method aligns retrieval directly with generation quality. Extensive experiments on nine personalization tasks demonstrate that PURPLE consistently outperforms strong heuristic and retrieval-augmented baselines in both effectiveness and efficiency, establishing a principled and scalable solution for optimizing user profiles.

Method: TurnGuide dynamically segments assistant speech into dialogue turns and interleaves turn-level text and speech generation. This approach allows FD-SLMs to leverage the semantic intelligence of LLMs while maintaining natural acoustic flow and precise time alignment for real-time spoken interactions.

Result: Extensive experiments show TurnGuide significantly improves e2e FD-SLMs to produce semantically meaningful, coherent speech and achieves state-of-the-art performance on various turn-taking events including interruptions, backchannels, and overlapping speech.

Conclusion: TurnGuide successfully addresses the conversational quality degradation in FD-SLMs by enabling effective text-speech interleaved generation without compromising natural acoustic flow, making real-time spoken interactions more human-like and semantically coherent.

Abstract: Full-Duplex Speech Language Models (FD-SLMs) are specialized foundation models designed to enable natural, real-time spoken interactions by modeling complex conversational turn-taking such as interruptions, backchannels, and overlapping speech. End-to-end (e2e) FD-SLMs leverage real-world double-channel conversational data to capture nuanced two-speaker dialogue patterns for human-like interactions, but their conversational abilities often degrade compared to pure-text conversation due to prolonged speech sequences and limited high-quality spoken dialogue data. Although interleaved text-speech generation could mitigate this degradation, integrating discrete text tokens into continuous double-channel audio streams could disrupt the precise time alignment required for fluid interaction. To address this, we propose TurnGuide, a novel text-speech interleaved generation approach for e2e FD-SLMs that dynamically segments assistant speech into dialogue turns and interleaves turn-level text and speech generation. This approach allows FD-SLMs to integrate the semantic intelligence of LLMs without compromising the natural acoustic flow. Extensive experiments show that TurnGuide not only significantly improves e2e FD-SLMs to produce semantically meaningful, coherent speech but also achieves state-of-the-art performance on various turn-taking events. Demos are available at https://dreamtheater123.github.io/TurnGuide-Demo/. Code will be available at https://github.com/dreamtheater123/TurnGuide.

Method: Two key innovations: 1) Dual-Resolution Speech Representations (DRSR) where the Shared LLM processes audio at efficient 5Hz via token grouping while the Speech Refined Head generates high-quality tokens at 25Hz; 2) Core-Cocktail Training, a two-stage fine-tuning with intermediate merging to mitigate catastrophic forgetting, followed by Multi-Task DPO Training to enhance robustness and capabilities.

Result: Fun-Audio-Chat 8B and MoE 30B-A3B achieve competitive performance on Speech-to-Text and Speech-to-Speech tasks, ranking top among similar-scale models on Spoken QA benchmarks. They also show competitive to superior performance on Audio Understanding, Speech Function Calling, Instruction-Following and Voice Empathy.

Conclusion: The proposed approach enables retention of text LLM knowledge while gaining powerful audio understanding, reasoning, and generation capabilities without requiring large-scale audio-text pre-training, instead leveraging pre-trained models and extensive post-training.

Abstract: Recent advancements in joint speech-text models show great potential for seamless voice interactions. However, existing models face critical challenges: temporal resolution mismatch between speech tokens (25Hz) and text tokens (~3Hz) dilutes semantic information, incurs high computational costs, and causes catastrophic forgetting of text LLM knowledge. We introduce Fun-Audio-Chat, a Large Audio Language Model addressing these limitations via two innovations from our previous work DrVoice. First, Dual-Resolution Speech Representations (DRSR): the Shared LLM processes audio at efficient 5Hz (via token grouping), while the Speech Refined Head generates high-quality tokens at 25Hz, balancing efficiency (~50% GPU reduction) and quality. Second, Core-Cocktail Training, a two-stage fine-tuning with intermediate merging that mitigates catastrophic forgetting. We then apply Multi-Task DPO Training to enhance robustness, audio understanding, instruction-following and voice empathy. This multi-stage post-training enables Fun-Audio-Chat to retain text LLM knowledge while gaining powerful audio understanding, reasoning, and generation. Unlike recent LALMs requiring large-scale audio-text pre-training, Fun-Audio-Chat leverages pre-trained models and extensive post-training. Fun-Audio-Chat 8B and MoE 30B-A3B achieve competitive performance on Speech-to-Text and Speech-to-Speech tasks, ranking top among similar-scale models on Spoken QA benchmarks. They also achieve competitive to superior performance on Audio Understanding, Speech Function Calling, Instruction-Following and Voice Empathy. We develop Fun-Audio-Chat-Duplex, a full-duplex variant with strong performance on Spoken QA and full-duplex interactions. We open-source Fun-Audio-Chat-8B with training and inference code, and provide an interactive demo, at https://github.com/FunAudioLLM/Fun-Audio-Chat .

Method: Evaluated 7 state-of-the-art LLMs on their ability to annotate 121 ritual features across 567 ethnographic excerpts, comparing performance against human inter-coder reliability benchmarks.

Result: LLM performance was limited and fell well below levels required for reliable automated annotation. Performance was particularly poor on longer texts, features requiring ordinal distinctions, and ambiguous constructs. Human inter-coder reliability set a ceiling on LLM accuracy, and even on features where humans reliably agreed, models fell short of human performance.

Conclusion: LLMs cannot yet substitute for human expertise in ethnographic annotation, though they may have potential as supplementary tools or for specific, well-defined annotation tasks.

Abstract: Large language models (LLMs) have shown promise for automated text annotation, raising hopes that they might accelerate cross-cultural research by extracting structured data from ethnographic texts. We evaluated 7 state-of-the-art LLMs on their ability to annotate 121 ritual features across 567 ethnographic excerpts. Performance was limited, falling well below levels required for reliable automated annotation. Longer texts, features requiring ordinal distinctions, and ambiguous constructs proved particularly difficult. Human inter-coder reliability set an approximate ceiling on LLM accuracy: features that human coders found difficult to agree upon were also difficult for LLMs. Yet even on features where humans reliably agreed, models fell short of human performance. Our findings suggest that LLMs cannot yet substitute for human expertise in ethnographic annotation.

Method: EZ-MIA introduces the Error Zone (EZ) score, which measures directional probability imbalances at error positions (tokens where the model predicts incorrectly but shows elevated probability for training examples) relative to a pretrained reference model, requiring only two forward passes and no model training.

Result: On WikiText with GPT-2: 3.8x higher detection than SOTA (66.3% vs 17.5% TPR at 1% FPR), near-perfect AUC of 0.98. At 0.1% FPR: 8x higher detection (14.0% vs 1.8%). On AG News with Llama-2-7B: 3x higher detection (46.7% vs 15.8% TPR at 1% FPR).

Conclusion: Privacy risks of fine-tuned language models are substantially greater than previously understood, with EZ-MIA establishing new state-of-the-art detection capabilities that have important implications for privacy auditing and deployment decisions.

Abstract: Fine-tuned language models pose significant privacy risks, as they may memorize and expose sensitive information from their training data. Membership inference attacks (MIAs) provide a principled framework for auditing these risks, yet existing methods achieve limited detection rates, particularly at the low false-positive thresholds required for practical privacy auditing. We present EZ-MIA, a membership inference attack that exploits a key observation: memorization manifests most strongly at error positions, specifically tokens where the model predicts incorrectly yet still shows elevated probability for training examples. We introduce the Error Zone (EZ) score, which measures the directional imbalance of probability shifts at error positions relative to a pretrained reference model. This principled statistic requires only two forward passes per query and no model training of any kind. On WikiText with GPT-2, EZ-MIA achieves 3.8x higher detection than the previous state-of-the-art under identical conditions (66.3% versus 17.5% true positive rate at 1% false positive rate), with near-perfect discrimination (AUC 0.98). At the stringent 0.1% FPR threshold critical for real-world auditing, we achieve 8x higher detection than prior work (14.0% versus 1.8%), requiring no reference model training. These gains extend to larger architectures: on AG News with Llama-2-7B, we achieve 3x higher detection (46.7% versus 15.8% TPR at 1% FPR). These results establish that privacy risks of fine-tuned language models are substantially greater than previously understood, with implications for both privacy auditing and deployment decisions. Code is available at https://github.com/JetBrains-Research/ez-mia.

Method: Evaluated three instruction-tuned LLMs (LLaMA 3.1 8B Instruct, LLaMA 3.2 3B Instruct, and Qwen 2.5 7B Instruct) on a Kaggle dataset of Prothom Alo newspaper articles using the same classification framework.

Result: Qwen 2.5 achieved highest accuracy of 72%, excelling in “Sports” category. LLaMA 3.1 and LLaMA 3.2 scored 53% and 56% respectively, demonstrating LLM effectiveness despite Bengali resource scarcity.

Conclusion: LLMs show promise for Bengali text classification despite limited resources. Future work should explore additional models, address class imbalance, and refine fine-tuning approaches to improve performance.

Abstract: Bengali text classification is a Significant task in natural language processing (NLP), where text is categorized into predefined labels. Unlike English, Bengali faces challenges due to the lack of extensive annotated datasets and pre-trained language models. This study explores the effectiveness of large language models (LLMs) in classifying Bengali newspaper articles. The dataset used, obtained from Kaggle, consists of articles from Prothom Alo, a major Bangladeshi newspaper. Three instruction-tuned LLMs LLaMA 3.1 8B Instruct, LLaMA 3.2 3B Instruct, and Qwen 2.5 7B Instruct were evaluated for this task under the same classification framework. Among the evaluated models, Qwen 2.5 achieved the highest classification accuracy of 72%, showing particular strength in the “Sports” category. In comparison, LLaMA 3.1 and LLaMA 3.2 attained accuracies of 53% and 56%, respectively. The findings highlight the effectiveness of LLMs in Bengali text classification, despite the scarcity of resources for Bengali NLP. Future research will focus on exploring additional models, addressing class imbalance issues, and refining fine-tuning approaches to improve classification performance.

Method: Analyzed 13 transcribed cancer patient interviews (132,722 words) using BERTopic and Top2Vec with similar preprocessing, chunking, and clustering configurations. Used GPT-4 for topic labeling, evaluated outputs through human assessment on coherence, clarity, and relevance. Selected BERTopic for further experimentation with three clinically oriented embedding models (BioClinicalBERT and others).

Result: BERTopic outperformed Top2Vec in preliminary evaluation. Domain-specific embeddings improved topic precision and interpretability, with BioClinicalBERT producing most consistent results. Global analysis revealed dominant topics: “Coordination and Communication in Cancer Care Management” and “Patient Decision-Making in Cancer Treatment Journey.”

Conclusion: Neural topic modeling, particularly BERTopic with domain-specific embeddings like BioClinicalBERT, can effectively extract useful themes from patient interviews to provide feedback to clinicians, supporting more efficient document navigation and strengthening patient voices in healthcare workflows.

Abstract: This study investigates the use of neural topic modeling and LLMs to uncover meaningful themes from patient storytelling data, to offer insights that could contribute to more patient-oriented healthcare practices. We analyze a collection of transcribed interviews with cancer patients (132,722 words in 13 interviews). We first evaluate BERTopic and Top2Vec for individual interview summarization by using similar preprocessing, chunking, and clustering configurations to ensure a fair comparison on Keyword Extraction. LLMs (GPT4) are then used for the next step topic labeling. Their outputs for a single interview (I0) are rated through a small-scale human evaluation, focusing on {coherence}, {clarity}, and {relevance}. Based on the preliminary results and evaluation, BERTopic shows stronger performance and is selected for further experimentation using three {clinically oriented embedding} models. We then analyzed the full interview collection with the best model setting. Results show that domain-specific embeddings improved topic \textit{precision} and \textit{interpretability}, with BioClinicalBERT producing the most consistent results across transcripts. The global analysis of the full dataset of 13 interviews, using the BioClinicalBERT embedding model, reveals the most dominant topics throughout all 13 interviews, namely Coordination and Communication in Cancer Care Management" and Patient Decision-Making in Cancer Treatment Journey’’. Although the interviews are machine translations from Dutch to English, and clinical professionals are not involved in this evaluation, the findings suggest that neural topic modeling, particularly BERTopic, can help provide useful feedback to clinicians from patient interviews. This pipeline could support more efficient document navigation and strengthen the role of patients’ voices in healthcare workflows.

Method: Trained BabyBERTa (a transformer model optimized for small datasets) on artificial grammars with varying training set sizes, unique sentence types, and proportions of rule-following vs. exception exemplars to test Tolerance Principle predictions.

Result: Unlike human infants, BabyBERTa’s learning dynamics do not align with the Tolerance Principle, showing different patterns of grammar rule acquisition from limited data.

Conclusion: Current transformer models have fundamentally different learning mechanisms from human infants when acquiring grammar rules from minimal data, suggesting limitations in modeling human language acquisition.

Abstract: Modern language models like GPT-3, BERT, and LLaMA require massive training data, yet with sufficient training they reliably learn to distinguish grammatical from ungrammatical sentences. Children aged as young as 14 months already have the capacity to learn abstract grammar rules from very few exemplars, even in the presence of non-rule-following exceptions. Yang’s (2016) Tolerance Principle defines a precise threshold for how many exceptions a rule can tolerate and still be learnable. The present study explored the minimal amount and quality of training data necessary for rules to be generalized by a transformer-based language model to test the predictions of the Tolerance Principle. We trained BabyBERTa (Huebner et al. 2021), a transformer model optimized for small datasets, on artificial grammars. The training sets varied in size, number of unique sentence types, and proportion of rule-following versus exception exemplars. We found that, unlike human infants, BabyBERTa’s learning dynamics do not align with the Tolerance Principle.

Method: Frames dialect identification as limited-vocabulary ASR using CTC loss. Uses two approaches for training: Language-Agnostic Heuristic (LAH) or pre-trained ASR model to estimate dialect tag repetitions in transcriptions.

Result: CTC-DID outperforms fine-tuned Whisper and ECAPA-TDNN models on Arabic Dialect Identification task, even with limited training data. Also excels in zero-shot evaluation on Casablanca dataset, showing robustness to shorter utterances and minimal degradation for streaming applications.

Conclusion: CTC-based approach provides an effective framework for dialect identification, offering superior performance, robustness, and real-time adaptability compared to existing methods.

Abstract: This paper proposes a Dialect Identification (DID) approach inspired by the Connectionist Temporal Classification (CTC) loss function as used in Automatic Speech Recognition (ASR). CTC-DID frames the dialect identification task as a limited-vocabulary ASR system, where dialect tags are treated as a sequence of labels for a given utterance. For training, the repetition of dialect tags in transcriptions is estimated either using a proposed Language-Agnostic Heuristic (LAH) approach or a pre-trained ASR model. The method is evaluated on the low-resource Arabic Dialect Identification (ADI) task, with experimental results demonstrating that an SSL-based CTC-DID model, trained on a limited dataset, outperforms both fine-tuned Whisper and ECAPA-TDNN models. Notably, CTC-DID also surpasses these models in zero-shot evaluation on the Casablanca dataset. The proposed approach is found to be more robust to shorter utterances and is shown to be easily adaptable for streaming, real-time applications, with minimal performance degradation.

Method: CoReflect uses a conversation planner to generate structured templates for diverse, goal-directed dialogues, a user simulator to execute these dialogues, and a reflective analyzer to identify behavioral patterns and automatically refine evaluation rubrics. The system operates through a co-evolutionary loop where analysis insights feed back to update conversation templates.

Result: The framework provides a scalable, self-refining methodology that minimizes human intervention while allowing evaluation protocols to adapt alongside advancing dialogue model capabilities. It ensures test case complexity and rubric diagnostic precision improve together.

Conclusion: CoReflect offers a unified approach to conversational evaluation that addresses the limitations of static methods by creating an adaptive, co-evolutionary system that can better capture and assess the diverse behaviors of modern dialogue models.

Abstract: Evaluating conversational systems in multi-turn settings remains a fundamental challenge. Conventional pipelines typically rely on manually defined rubrics and fixed conversational context$-$a static approach that limits coverage and fails to capture the diverse, emergent behaviors of dialogue models. To address this, we introduce CoReflect (Conversational Evaluation via Co-Evolutionary Simulation and Reflective Rubric Refinement), which unifies dialogue simulation and evaluation into an adaptive, iterative process. CoReflect employs a conversation planner that generates structured templates to guide a user simulator through diverse, goal-directed dialogues. Subsequently, a reflective analyzer processes these dialogues to identify systematic behavioral patterns and automatically refine the evaluation rubrics. Crucially, the insights from the conversation analysis are fed back into the planner to update conversation templates for subsequent iterations. This co-evolution loop ensures that the complexity of test cases and the diagnostic precision of rubrics improve in tandem. By minimizing human intervention, CoReflect provides a scalable and self-refining methodology that allows evaluation protocols to adapt alongside the rapidly advancing capabilities of dialogue models.

Method: Plan-Verify-Fill (PVF) paradigm with three steps: 1) Actively constructs hierarchical skeleton by prioritizing high-leverage semantic anchors, 2) Employs verification protocol for pragmatic structural stopping, 3) Grounds planning via quantitative validation without requiring training.

Result: PVF reduces Number of Function Evaluations (NFE) by up to 65% compared to confidence-based parallel decoding across benchmark datasets on LLaDA-8B-Instruct and Dream-7B-Instruct models, achieving superior efficiency without compromising accuracy.

Conclusion: PVF demonstrates that active planning and verification can significantly improve decoding efficiency for Diffusion Language Models, offering a promising training-free approach to leverage global bidirectional context for more efficient text generation.

Abstract: Diffusion Language Models (DLMs) present a promising non-sequential paradigm for text generation, distinct from standard autoregressive (AR) approaches. However, current decoding strategies often adopt a reactive stance, underutilizing the global bidirectional context to dictate global trajectories. To address this, we propose Plan-Verify-Fill (PVF), a training-free paradigm that grounds planning via quantitative validation. PVF actively constructs a hierarchical skeleton by prioritizing high-leverage semantic anchors and employs a verification protocol to operationalize pragmatic structural stopping where further deliberation yields diminishing returns. Extensive evaluations on LLaDA-8B-Instruct and Dream-7B-Instruct demonstrate that PVF reduces the Number of Function Evaluations (NFE) by up to 65% compared to confidence-based parallel decoding across benchmark datasets, unlocking superior efficiency without compromising accuracy.

Method: MGEO employs an alternating gradient-based optimization strategy to jointly optimize imperceptible image perturbations and fluent textual suffixes, exploiting the deep cross-modal coupling within VLMs rather than treating modalities in isolation.

Result: Extensive experiments on real-world datasets using state-of-the-art models show that the coordinated multimodal attack significantly outperforms text-only and image-only baselines.

Conclusion: Multimodal synergy, typically a strength of VLMs, can be weaponized to compromise search ranking integrity without triggering conventional content filters, revealing a critical vulnerability in VLM-based systems.

Abstract: Vision-Language Models (VLMs) are rapidly replacing unimodal encoders in modern retrieval and recommendation systems. While their capabilities are well-documented, their robustness against adversarial manipulation in competitive ranking scenarios remains largely unexplored. In this paper, we uncover a critical vulnerability in VLM-based product search: multimodal ranking attacks. We present Multimodal Generative Engine Optimization (MGEO), a novel adversarial framework that enables a malicious actor to unfairly promote a target product by jointly optimizing imperceptible image perturbations and fluent textual suffixes. Unlike existing attacks that treat modalities in isolation, MGEO employs an alternating gradient-based optimization strategy to exploit the deep cross-modal coupling within the VLM. Extensive experiments on real-world datasets using state-of-the-art models demonstrate that our coordinated attack significantly outperforms text-only and image-only baselines. These findings reveal that multimodal synergy, typically a strength of VLMs, can be weaponized to compromise the integrity of search rankings without triggering conventional content filters.

Method: Uses Markov chain Monte Carlo (MCMC) power sampling methods to sample from sharpened sequence-level probability distributions rather than token-level distributions. Incorporates annealing where the tempered distribution gradually shifts from high to low temperature.

Result: Strong Theory of Mind capability can be recovered directly from base models without additional weight updates or verifications. Annealing substantially improves ToM performance over fixed-temperature power sampling.

Conclusion: Sampling-based optimization provides a powerful way to extract latent capabilities from language models without retraining, challenging the assumption that autoregressive models inherently fail at Theory of Mind tasks.

Abstract: Autoregressive language models are next-token predictors and have been criticized for only optimizing surface plausibility (i.e., local coherence) rather than maintaining correct latent-state representations (i.e., global coherence). Because Theory of Mind (ToM) tasks crucially depend on reasoning about latent mental states of oneself and others, such models are therefore often thought to fail at ToM. While post-training methods can improve ToM performance, we show that strong ToM capability can be recovered directly from the base model without any additional weight updates or verifications. Our approach builds on recent power-sampling methods (Karan & Du, 2025) that use Markov chain Monte Carlo (MCMC) to sample from sharpened sequence-level (rather than token-level) probability distributions of autoregressive language models. We further find that incorporating annealing, where the tempered distribution is gradually shifted from high to low temperature, substantially improves ToM performance over fixed-temperature power sampling. Together, these results suggest that sampling-based optimization provides a powerful way to extract latent capabilities from language models without retraining.

Method: Combines Representation Engineering (RepE) with One-Class Support Vector Machine (OCSVM) to identify context-specific subspaces within LLM internal states. Trains OCSVM on in-context examples to establish boundaries in the hidden state latent space. Identifies optimal layers within LLM internal states that strongly associate with specific contexts. Evaluated on Llama and Qwen models.

Result: Promising results in identifying subspaces for specific contexts. The approach shows effectiveness in detecting in-context vs out-of-context conversation threads.

Conclusion: The method contributes to better LLM interpretation and provides a useful tool for detecting when LLMs generate out-of-context responses, addressing important safety concerns in LLM deployment.

Abstract: The increasing prevalence of Large Language Models (LLMs) demands effective safeguards for their operation, particularly concerning their tendency to generate out-of-context responses. A key challenge is accurately detecting when LLMs stray from expected conversational norms, manifesting as topic shifts, factual inaccuracies, or outright hallucinations. Traditional anomaly detection struggles to directly apply within contextual semantics. This paper outlines our experiment in exploring the use of Representation Engineering (RepE) and One-Class Support Vector Machine (OCSVM) to identify subspaces within the internal states of LLMs that represent a specific context. By training OCSVM on in-context examples, we establish a robust boundary within the LLM’s hidden state latent space. We evaluate out study with two open source LLMs - Llama and Qwen models in specific contextual domain. Our approach entailed identifying the optimal layers within the LLM’s internal state subspaces that strongly associates with the context of interest. Our evaluation results showed promising results in identifying the subspace for a specific context. Aside from being useful in detecting in or out of context conversation threads, this research work contributes to the study of better interpreting LLMs.

Method: Created TaxoBench from 72 highly-cited computer science surveys, manually extracting expert-authored taxonomy trees with 3,815 precisely categorized citations as ground truth. Supports two evaluation modes: Deep Research (end-to-end retrieval and organization) and Bottom-Up (isolating structuring capability with exact papers). Evaluated 7 leading deep research agents and 12 frontier LLMs.

Result: Reveals dual bottleneck: best agent recalls only 20.9% of expert-selected papers, and even with perfect input, best model achieves only 0.31 Adjusted Rand Index (ARI) in organization. Current deep research agents remain far from expert-level survey writing capabilities.

Conclusion: TaxoBench provides a diagnostic benchmark for evaluating core capabilities of deep research agents in survey generation, highlighting significant gaps between current systems and human expert performance in both retrieval and organization tasks.

Abstract: Deep Research Agents are increasingly used for automated survey generation. However, whether they can write surveys like human experts remains unclear. Existing benchmarks focus on fluency or citation accuracy, but none evaluates the core capabilities: retrieving essential papers and organizing them into coherent knowledge structures. We introduce TaxoBench, a diagnostic benchmark derived from 72 highly-cited computer science surveys. We manually extract expert-authored taxonomy trees containing 3,815 precisely categorized citations as ground truth. Our benchmark supports two evaluation modes: Deep Research mode tests end-to-end retrieval and organization given only a topic, while Bottom-Up mode isolates structuring capability by providing the exact papers human experts used. We evaluate 7 leading Deep Research agents and 12 frontier LLMs. Results reveal a dual bottleneck: the best agent recalls only 20.9% of expert-selected papers, and even with perfect input, the best model achieves only 0.31 ARI in organization. Current deep research agents remain far from expert-level survey writing. Our benchmark is publicly available at https://github.com/KongLongGeFDU/TaxoBench.

Method: Introduced a scalable bias-auditing framework using named entities as probes to measure structural disparities. Used synthetic data that reliably reproduces bias patterns observed in natural text, enabling large-scale analysis across 1.9 billion data points.

Result: Systematic biases found: models penalize right-wing politicians, favor left-wing politicians, prefer Western/wealthy nations over Global South, favor Western companies, and penalize defense/pharmaceutical firms. Instruction tuning reduces bias, but increasing model scale amplifies it. Chinese/Russian prompting doesn’t attenuate Western-aligned preferences.

Conclusion: LLMs exhibit systematic biases that require rigorous auditing before deployment in high-stakes applications. The framework provides a scalable approach to identify and measure these structural disparities across models, tasks, and languages.

Abstract: Existing approaches to bias evaluation in large language models (LLMs) trade ecological validity for statistical control, relying on artificial prompts that poorly reflect real-world use, or on naturalistic tasks that lack scale and rigor. We introduce a scalable bias-auditing framework using named entities as probes to measure structural disparities in model behavior. We show that synthetic data reliably reproduces bias patterns observed in natural text, enabling large-scale analysis. Using this approach, we conduct the largest bias audit to date, comprising 1.9 billion data points across multiple entity types, tasks, languages, models, and prompting strategies. Our results reveal systematic biases: models penalize right-wing politicians, favor left-wing politicians, prefer Western and wealthy nations over the Global South, favor Western companies, and penalize firms in the defense and pharmaceutical sectors. While instruction tuning reduces bias, increasing model scale amplifies it, and prompting in Chinese or Russian does not attenuate Western-aligned preferences. These results indicate that LLMs should undergo rigorous auditing before deployment in high-stakes applications.

Method: Left-Right Diffusion Watermarking (LR-DWM) biases token generation based on both left and right neighbors when available, enabling watermark embedding during the diffusion process with minimal overhead.

Result: LR-DWM achieves high detectability with negligible computational and memory overhead, performing close to non-watermarked baseline DLMs while enabling reliable statistical detection.

Conclusion: DLMs can be efficiently watermarked using the LR-DWM approach, which solves the computational overhead problem of previous methods while maintaining effective detection capabilities.

Abstract: Watermarking (WM) is a critical mechanism for detecting and attributing AI-generated content. Current WM methods for Large Language Models (LLMs) are predominantly tailored for autoregressive (AR) models: They rely on tokens being generated sequentially, and embed stable signals within the generated sequence based on the previously sampled text. Diffusion Language Models (DLMs) generate text via non-sequential iterative denoising, which requires significant modification to use WM methods designed for AR models. Recent work proposed to watermark DLMs by inverting the process when needed, but suffers significant computational or memory overhead. We introduce Left-Right Diffusion Watermarking (LR-DWM), a scheme that biases the generated token based on both left and right neighbors, when they are available. LR-DWM incurs minimal runtime and memory overhead, remaining close to the non-watermarked baseline DLM while enabling reliable statistical detection under standard evaluation settings. Our results demonstrate that DLMs can be watermarked efficiently, achieving high detectability with negligible computational and memory overhead.

Method: NADIR integrates a Differential Transformer and a Mixture-of-Experts mechanism to model complex character mappings without sequential dependencies, enabling robust non-autoregressive processing.

Result: NADIR achieves over 13x speed-up compared to state-of-the-art AR baseline, maintains competitive mean Character Error Rate of 15.78% (vs 14.44% for AR and 21.88% for standard NAR), and significantly reduces repetition (49.53%), substitution (24.45%), omission (32.92%), and insertion (16.87%) errors.

Conclusion: NADIR effectively bridges the gap between AR accuracy and real-time deployment demands, providing a practical blueprint for building fast and reliable non-autoregressive systems for tasks with local dependencies.

Abstract: In this work, we argue that not all sequence-to-sequence tasks require the strong inductive biases of autoregressive (AR) models. Tasks like multilingual transliteration, code refactoring, grammatical correction or text normalization often rely on local dependencies where the full modeling capacity of AR models can be overkill, creating a trade-off between their high accuracy and high inference latency. While non-autoregressive (NAR) models offer speed, they typically suffer from hallucinations and poor length control. To explore this trade-off, we focus on the multilingual transliteration task in Indic languages and introduce NADIR, a novel NAR architecture designed to strike a balance between speed and accuracy. NADIR integrates a Differential Transformer and a Mixture-of-Experts mechanism, enabling it to robustly model complex character mappings without sequential dependencies. NADIR achieves over a 13x speed-up compared to the state-of-the-art AR baseline. It maintains a competitive mean Character Error Rate of 15.78%, compared to 14.44% for the AR model and 21.88% for a standard NAR equivalent. Importantly, NADIR reduces Repetition errors by 49.53%, Substitution errors by 24.45%, Omission errors by 32.92%, and Insertion errors by 16.87%. This work provides a practical blueprint for building fast and reliable NAR systems, effectively bridging the gap between AR accuracy and the demands of real-time, large-scale deployment.

Method: Proposed comparative analysis framework for model-agnostic interpretability techniques, employing two rationale extraction methods that justify outcomes with human-interpretable text fragments. Evaluated via faithfulness (normalized sufficiency/comprehensiveness metrics) and plausibility (legal expert evaluation).

Result: Model’s “reasons” for predicting violations differ substantially from legal experts’ reasoning, despite promising quantitative results and reasonable classification performance. LLM-as-a-Judge feasibility assessed using expert evaluations.

Conclusion: There’s a significant gap between model explanations and human expert reasoning in legal domain, highlighting the need for better interpretability methods that align with expert judgment.

Abstract: Interpretability is critical for applications of large language models in the legal domain which requires trust and transparency. While some studies develop task-specific approaches, other use the classification model’s parameters to explain the decisions. However, which technique explains the legal outcome prediction best remains an open question. To address this challenge, we propose a comparative analysis framework for model-agnostic interpretability techniques. Among these, we employ two rationale extraction methods, which justify outcomes with human-interpretable and concise text fragments (i.e., rationales) from the given input text. We conduct comparison by evaluating faithfulness-via normalized sufficiency and comprehensiveness metrics along with plausibility-by asking legal experts to evaluate extracted rationales. We further assess the feasibility of LLM-as-a-Judge using legal expert evaluation results. We show that the model’s “reasons” for predicting a violation differ substantially from those of legal experts, despite highly promising quantitative analysis results and reasonable downstream classification performance. The source code of our experiments is publicly available at https://github.com/trusthlt/IntEval.

Method: The study evaluates a holistic, system-mediated framework that attributes attention imbalances to functionally redundant system weights. It causally redistributes attention from system modality to image and textual inputs to suppress the yes-bias hallucination.

Result: Redistributing attention from system modality to image and text inputs substantially suppresses the yes-bias hallucination, often outperforming existing approaches. System-mediated attention imbalances encourage reliance on coarse input representations unsuitable for some tasks.

Conclusion: System attention is a key factor in VLM hallucination and represents a promising lever for mitigation strategies, establishing the importance of considering all modalities (system, image, text) rather than just image-centric approaches.

Abstract: Vision-language model (VLM) hallucination is commonly linked to imbalanced allocation of attention across input modalities: system, image and text. However, existing mitigation strategies tend towards an image-centric interpretation of these imbalances, often prioritising increased image attention while giving less consideration to the roles of the other modalities. In this study, we evaluate a more holistic, system-mediated account, which attributes these imbalances to functionally redundant system weights that reduce attention to image and textual inputs. We show that this framework offers a useful empirical perspective on the yes-bias, a common form of hallucination in which VLMs indiscriminately respond ‘yes’. Causally redistributing attention from the system modality to image and textual inputs substantially suppresses this bias, often outperforming existing approaches. We further present evidence suggesting that system-mediated attention imbalances contribute to the yes-bias by encouraging a default reliance on coarse input representations, which are effective for some tasks but ill-suited to others. Taken together, these findings firmly establish system attention as a key factor in VLM hallucination and highlight its potential as a lever for mitigation.

Method: Two main components: 1) DeepReasonQA - a knowledge graph-driven synthesis framework that controllably constructs high-difficulty, multi-hop long-context QA pairs with inherent reasoning chains. 2) LongPAS (Long-context Process Advantage Shaping) - a simple yet effective method that performs fine-grained credit assignment by evaluating reasoning steps along Validity and Relevance dimensions to capture learning signals from “almost-there” trajectories.

Result: Experiments on three long-context reasoning benchmarks show the approach substantially outperforms RLVR baselines and matches frontier LLMs while using far fewer parameters. Further analysis confirms effectiveness in strengthening long-context reasoning while maintaining stable RL training.

Conclusion: The proposed DeepReasonQA and LongPAS effectively address the “almost-there” problem in long-context reasoning by providing high-difficulty training data and fine-grained credit assignment, enabling RL-based methods to achieve strong performance in long-context scenarios comparable to much larger models.

Abstract: Reinforcement Learning with Verifiable Rewards (RLVR) has proven effective in enhancing LLMs short-context reasoning, but its performance degrades in long-context scenarios that require both precise grounding and robust long-range reasoning. We identify the “almost-there” phenomenon in long-context reasoning, where trajectories are largely correct but fail at the final step, and attribute this failure to two factors: (1) the lack of high reasoning density in long-context QA data that push LLMs beyond mere grounding toward sophisticated multi-hop reasoning; and (2) the loss of valuable learning signals during long-context RL training due to the indiscriminate penalization of partially correct trajectories with incorrect outcomes. To overcome this bottleneck, we propose DeepReasonQA, a KG-driven synthesis framework that controllably constructs high-difficulty, multi-hop long-context QA pairs with inherent reasoning chains. Building on this, we introduce Long-context Process Advantage Shaping (LongPAS), a simple yet effective method that performs fine-grained credit assignment by evaluating reasoning steps along Validity and Relevance dimensions, which captures critical learning signals from “almost-there” trajectories. Experiments on three long-context reasoning benchmarks show that our approach substantially outperforms RLVR baselines and matches frontier LLMs while using far fewer parameters. Further analysis confirms the effectiveness of our methods in strengthening long-context reasoning while maintaining stable RL training.

Method: Introduces MedAbstain benchmark with conformal prediction, adversarial question perturbations, and explicit abstention options for medical MCQA. Systematically evaluates open- and closed-source LLMs on their abstention capabilities.

Result: Even state-of-the-art, high-accuracy LLMs often fail to abstain when uncertain. Explicit abstention options consistently increase model uncertainty and safer abstention, outperforming input perturbations. Scaling model size or advanced prompting provides little improvement.

Conclusion: Abstention mechanisms are central for trustworthy LLM deployment in high-stakes applications. The findings offer practical guidance for improving safety, emphasizing that explicit abstention options are more effective than other approaches.

Abstract: Current evaluation of large language models (LLMs) overwhelmingly prioritizes accuracy; however, in real-world and safety-critical applications, the ability to abstain when uncertain is equally vital for trustworthy deployment. We introduce MedAbstain, a unified benchmark and evaluation protocol for abstention in medical multiple-choice question answering (MCQA) – a discrete-choice setting that generalizes to agentic action selection – integrating conformal prediction, adversarial question perturbations, and explicit abstention options. Our systematic evaluation of both open- and closed-source LLMs reveals that even state-of-the-art, high-accuracy models often fail to abstain with uncertain. Notably, providing explicit abstention options consistently increases model uncertainty and safer abstention, far more than input perturbations, while scaling model size or advanced prompting brings little improvement. These findings highlight the central role of abstention mechanisms for trustworthy LLM deployment and offer practical guidance for improving safety in high-stakes applications.

Method: Three-way transformer architecture integrating author information, inferred capability presentation, and research ideas with flexible fusion mechanisms, plus a two-stage architecture for learning capability representation.

Result: Method significantly outperforms single-way models using finetuned BERT-base and BERT-large; capability prediction significantly increases final model’s predictive accuracy.

Conclusion: Proposed framework enables early-stage research outcome prediction and scientific resource allocation using only author info and ideas, without requiring full manuscripts.

Abstract: Predicting the outcomes of research ideas at their conceptual stage (i.e. before significant resources are committed) holds great potential for optimizing scientific resource allocation and research planning. While existing methods rely heavily on finished manuscripts or peer reviews, we propose a novel capability-aware framework that predicts paper acceptance and ratings using only author information and research ideas, without requiring full text or experimental results. Our approach integrates author information, (inferred) capability presentation, and research ideas through a three-way transformer architecture with flexible fusion mechanisms. We also introduce a two-stage architecture for learning the capability representation given the author information and idea. Experiments show that our method significantly outperform the single-way models by finetuning bert-base and bert-large, and the capability predicting significantly increase the predictive accuracy of the final model. The proposed method can be applied in both early-stage research outcome prediction and scientific resource allocation.

Method: DoPE uses FewSoRT-Q (LLM-guided pipeline for question-level semantic decoy generation) and FewSoRT-D (encapsulation into watermarked documents). It exploits render-parse discrepancies in MLLM pipelines by embedding semantic decoys into PDF/HTML at authoring time.

Result: 91.4% detection rate at 8.7% false-positive rate using LLM-as-Judge verifier; prevents successful completion or induces decoy-aligned failures in 96.3% of attempts against black-box MLLMs from OpenAI and Anthropic.

Conclusion: DoPE provides effective model-agnostic prevention and detection for academic integrity, with strong empirical performance. The authors release Integrity-Bench (1826 exams), toolkit, and evaluation code for reproducible study.

Abstract: Multimodal Large Language Models (MLLMs) can directly consume exam documents, threatening conventional assessments and academic integrity. We present DoPE (Decoy-Oriented Perturbation Encapsulation), a document-layer defense framework that embeds semantic decoys into PDF/HTML assessments to exploit render-parse discrepancies in MLLM pipelines. By instrumenting exams at authoring time, DoPE provides model-agnostic prevention (stop or confound automated solving) and detection (flag blind AI reliance) without relying on conventional one-shot classifiers. We formalize prevention and detection tasks, and introduce FewSoRT-Q, an LLM-guided pipeline that generates question-level semantic decoys and FewSoRT-D to encapsulate them into watermarked documents. We evaluate on Integrity-Bench, a novel benchmark of 1826 exams (PDF+HTML) derived from public QA datasets and OpenCourseWare. Against black-box MLLMs from OpenAI and Anthropic, DoPE yields strong empirical gains: a 91.4% detection rate at an 8.7% false-positive rate using an LLM-as-Judge verifier, and prevents successful completion or induces decoy-aligned failures in 96.3% of attempts. We release Integrity-Bench, our toolkit, and evaluation code to enable reproducible study of document-layer defenses for academic integrity.

Method: Uses round-trip bootstrapping with NLLB models: translates English → target low-resource language → back to English. Employs reinforcement learning with chrF++ and BLEU as reward function on reconstructed English sentences. Evaluates on 600M and 1.3B parameter NLLB models using NLLB-MD dataset.

Result: Consistent improvements observed for Central Aymara, Friulian, Wolof, and Russian. Qualitative analysis shows increased fluency and semantic fidelity in translation outputs. Method shows potential to benefit from scale, allowing models to leverage pretrained knowledge for self-improvement.

Conclusion: Self-supervised reinforcement learning with round-trip bootstrapping effectively improves low-resource machine translation. The approach enables models to leverage their pretrained knowledge and continue self-improving, with potential for further benefits from scaling up model size.

Abstract: Low-resource machine translation (MT) has gained increasing attention as parallel data from low-resource language communities is collected, but many potential methods for improving low-resource MT remain unexplored. We investigate a self-supervised reinforcement-learning-based fine-tuning for translation in low-resource settings using round-trip bootstrapping with the No Language Left Behind (NLLB) family of models. Our approach translates English into a target low-resource language and then back into English, using a combination of chrF++ and BLEU as the reward function on the reconstructed English sentences. Using the NLLB-MD dataset, we evaluate both the 600M and 1.3B parameter NLLB models and observe consistent improvements for the following languages: Central Aymara, Friulian, Wolof and Russian. Qualitative inspection of translation outputs indicates increased fluency and semantic fidelity. We argue that our method can further benefit from scale, enabling models to increasingly leverage their pretrained knowledge and continue self-improving.

Method: Developed a comparative framework using structured meaning generation tasks across nine languages with 100 high-polysemy English words. Measures when models resort to dominant-language meanings in generation sequences - stronger models do so later while weaker models do so earlier.

Result: Found significant variation in semantic robustness across both models and languages. Established a principled ranking system for model comparison without requiring definitive causal attribution of error sources.

Conclusion: Provides a scalable comparative benchmark and rigorous validation pipeline for multilingual semantic evaluation, offering critical tools for developing more linguistically balanced AI systems.

Abstract: Multilingual Large Language Models (LLMs) exhibit remarkable cross-lingual abilities, yet often exhibit a systematic bias toward the representations from other languages, resulting in semantic interference when generating content in non-English languages$-$a phenomenon we define as language spilling. This paper presents a novel comparative framework for evaluating multilingual semantic robustness by systematically measuring how models handle polysemous words across languages. Our methodology provides a relative measure of model performance: when required to generate exactly five meanings, both strong and weak models may resort to meanings from dominant languages, but semantically stronger models do so later in the generation sequence, producing more true meanings from the target language before failing, while weaker models resort to dominant-language meanings earlier in the sequence. We evaluate a diverse set of open and closed multilingual LLMs using a structured meaning generation task across nine languages, employing a carefully curated benchmark of 100 high-polysemy English words. Our findings reveal significant variation in semantic robustness across both models and languages, providing a principled ranking system for model comparison without requiring definitive causal attribution of error sources. We contribute both a scalable comparative benchmark for multilingual semantic evaluation and a rigorous validation pipeline$-$critical tools for developing more linguistically balanced AI systems.

Method: Constructed controlled prompts that preserve underlying facts while introducing referential mediation through contextual sentences. Used synthetic vs. real names across languages to disentangle contextual effects from name-specific associations. Evaluated multiple model families in five languages.

Result: Contextual mediation consistently degrades factual recall, with substantial variation across relations. Larger models are more robust to contextual mediation, showing reduced performance gap relative to direct queries. Effects of real names and name origin are mixed and unsystematic.

Conclusion: There’s a significant gap between isolated factual recall and context-dependent language understanding in multilingual LLMs, highlighting limitations in how models access factual knowledge in naturalistic settings.

Abstract: Large language models (LLMs) can recall a wide range of factual knowledge across languages. However, existing factual recall evaluations primarily assess fact retrieval in isolation, where the queried entity is explicitly named and the fact is requested directly. In natural language use, facts are often accessed through context, where the relevant entity is introduced only indirectly. In this work, we study contextually mediated factual recall, asking whether LLMs can reliably retrieve factual knowledge when the target entity is embedded in a naturalistic context rather than queried explicitly, across languages. We construct controlled prompts that preserve the underlying fact while introducing referential mediation through contextual sentences. To disentangle contextual effects from name-specific associations, we further compare performance using synthetic names and real names across languages. Evaluating multiple model families in five languages, we find that contextual mediation consistently degrades factual recall, with substantial variation across relations. Larger models are more robust to contextual mediation, exhibiting a reduced performance gap relative to direct queries, while the effect of real names and name origin is mixed and unsystematic. These findings highlight a gap between isolated factual recall and context-dependent language understanding in multilingual LLMs.

Method: A domain-agnostic, model-independent workflow with a supervisor agent coordinating multiple specialized agents with individual capabilities. The framework integrates tasks from literature review and data analysis to complex simulation runs, demonstrated through a proof-of-concept system for catalyst research.

Result: The system achieves 90% correct task routing to appropriate agents, with 97.5% task completion for synthetic tasks and 91% for real-world chemistry tasks. It maintains comparable or better accuracy than frontier models while providing detailed cost breakdowns for cloud services.

Conclusion: The framework provides a viable, cost-effective solution for scientific domains to implement agentic systems, demonstrating practical utility for complex scientific workflows while being replicable across different scientific fields.

Abstract: As Large Language Models (LLMs) become ubiquitous across various scientific domains, their lack of ability to perform complex tasks like running simulations or to make complex decisions limits their utility. LLM-based agents bridge this gap due to their ability to call external resources and tools and thus are now rapidly gaining popularity. However, coming up with a workflow that can balance the models, cloud providers, and external resources is very challenging, making implementing an agentic system more of a hindrance than a help. In this work, we present a domain-agnostic, model-independent workflow for an agentic framework that can act as a scientific assistant while being run entirely on cloud. Built with a supervisor agent marshaling an array of agents with individual capabilities, our framework brings together straightforward tasks like literature review and data analysis with more complex ones like simulation runs. We describe the framework here in full, including a proof-of-concept system we built to accelerate the study of Catalysts, which is highly important in the field of Chemistry and Material Science. We report the cost to operate and use this framework, including the breakdown of the cost by services use. We also evaluate our system on a custom-curated synthetic benchmark and a popular Chemistry benchmark, and also perform expert validation of the system. The results show that our system is able to route the task to the correct agent 90% of the time and successfully complete the assigned task 97.5% of the time for the synthetic tasks and 91% of the time for real-world tasks, while still achieving better or comparable accuracy to most frontier models, showing that this is a viable framework for other scientific domains to replicate.

Method: Use cosine similarity on LLM reasoning traces from multi-agent systems to detect, quantify, and interpret disagreements among agents coding tutoring dialog segments; combine quantitative similarity metrics with qualitative review.

Result: LLM agents’ semantic reasoning similarity robustly differentiates consensus from disagreement and correlates with human coding reliability; reveals nuanced instructional sub-functions and opportunities for codebook refinement.

Conclusion: Reasoning-trace disagreements represent a valuable new analytic signal that can improve methodological rigor and interpretive depth in educational research, especially for establishing inter-rater reliability in human-AI collaboration.

Abstract: Learning analytics researchers often analyze qualitative student data such as coded annotations or interview transcripts to understand learning processes. With the rise of generative AI, fully automated and human-AI workflows have emerged as promising methods for analysis. However, methodological standards to guide such workflows remain limited. In this study, we propose that reasoning traces generated by large language model (LLM) agents, especially within multi-agent systems, constitute a novel and rich form of process data to enhance interpretive practices in qualitative coding. We apply cosine similarity to LLM reasoning traces to systematically detect, quantify, and interpret disagreements among agents, reframing disagreement as a meaningful analytic signal. Analyzing nearly 10,000 instances of agent pairs coding human tutoring dialog segments, we show that LLM agents’ semantic reasoning similarity robustly differentiates consensus from disagreement and correlates with human coding reliability. Qualitative analysis guided by this metric reveals nuanced instructional sub-functions within codes and opportunities for conceptual codebook refinement. By integrating quantitative similarity metrics with qualitative review, our method has the potential to improve and accelerate establishing inter-rater reliability during coding by surfacing interpretive ambiguity, especially when LLMs collaborate with humans. We discuss how reasoning-trace disagreements represent a valuable new class of analytic signals advancing methodological rigor and interpretive depth in educational research.

Method: Created BioPulse-QA with 2,280 expert-verified QA pairs from newly published biomedical documents (drug labels, trial protocols, clinical guidelines). Includes both extractive and abstractive formats with perturbed variants. Evaluated four LLMs released before document publication dates.

Result: GPT-o1 achieved highest relaxed F1 score (0.92) on drug labels, followed by Gemini-2.0-Flash (0.90). Clinical trials were most challenging with extractive F1 scores as low as 0.36. Performance differences larger for paraphrasing than typographical errors, bias testing showed negligible differences.

Conclusion: BioPulse-QA provides a scalable, clinically relevant framework for evaluating biomedical LLMs that addresses limitations of existing benchmarks by using timely documents and testing robustness.

Abstract: Objective: Large language models (LLMs) are increasingly applied in biomedical settings, and existing benchmark datasets have played an important role in supporting model development and evaluation. However, these benchmarks often have limitations. Many rely on static or outdated datasets that fail to capture the dynamic, context-rich, and high-stakes nature of biomedical knowledge. They also carry increasing risk of data leakage due to overlap with model pretraining corpora and often overlook critical dimensions such as robustness to linguistic variation and potential demographic biases. Materials and Methods: To address these gaps, we introduce BioPulse-QA, a benchmark that evaluates LLMs on answering questions from newly published biomedical documents including drug labels, trial protocols, and clinical guidelines. BioPulse-QA includes 2,280 expert-verified question answering (QA) pairs and perturbed variants, covering both extractive and abstractive formats. We evaluate four LLMs - GPT-4o, GPT-o1, Gemini-2.0-Flash, and LLaMA-3.1 8B Instruct - released prior to the publication dates of the benchmark documents. Results: GPT-o1 achieves the highest relaxed F1 score (0.92), followed by Gemini-2.0-Flash (0.90) on drug labels. Clinical trials are the most challenging source, with extractive F1 scores as low as 0.36. Discussion and Conclusion: Performance differences are larger for paraphrasing than for typographical errors, while bias testing shows negligible differences. BioPulse-QA provides a scalable and clinically relevant framework for evaluating biomedical LLMs.

Method: Conducted controlled comparison of six fine-tuning objectives (Supervised Fine-Tuning, Direct Preference Optimization, Conditional Fine-Tuning, Inoculation Prompting, Odds Ratio Preference Optimization, and KL-regularized fine-tuning) while keeping data, domain, architecture, and optimization constant. Evaluated across closed-form reasoning and open-ended generation tasks at different training scales.

Result: Objective choice creates systematic, scale-dependent shifts in the safety-capability frontier. At small training budgets, robustness is similar across objectives but capabilities differ. At larger budgets, supervised and preference-based tuning tightly couple capability gains with increased adversarial vulnerability and persona drift, while constrained objectives (especially ORPO and KL-regularization) substantially mitigate both safety issues.

Conclusion: Fine-tuning objectives matter little for safety at small scales but become a primary driver of adversarial robustness and latent persona stability as training scale increases. The choice of objective significantly impacts the safety-capability trade-off in large-scale fine-tuning.

Abstract: Fine-tuning LLMs on benign data can still degrade alignment and adversarial robustness, yet direct analysis of the role of fine-tuning objectives in shaping these safety outcomes remain limited. We present a controlled comparison of six fine-tuning objectives – Supervised Fine-Tuning, Direct Preference Optimization, Conditional Fine-Tuning, Inoculation Prompting, Odds Ratio Preference Optimization, and KL-regularized fine-tuning – holding data, domain, architecture, and optimization fixed. Across closed-form reasoning and open-ended generation tasks, we find that objective choice induces systematic, scale-dependent shifts along the safety-capability frontier. At small training budgets, robustness is similar across objectives but capability differs. At larger budgets, objectives diverge sharply: supervised and preference-based tuning tightly couple capability gains to increased adversarial vulnerability and persona drift, while objectives that constrain learning signals – especially ORPO and KL-regularization – substantially mitigate both. Fine-tuning objectives therefore matter little for safety at small scales but become a primary driver of adversarial robustness and latent persona stability as training scale increases.

Method: Evaluated multiple local and commercial LLMs using instruction-based and chain-of-thought prompting to extract structured procedural information from 414 curated interventional radiology reports authored by residents (2018-2024). Assessed performance using sensitivity, specificity, F1-score, inference latency, and token efficiency.

Result: Both local and commercial models achieved strong extraction performance with best F1-scores approaching 0.87. Models exhibited different trade-offs between speed and cost, demonstrating feasibility of AI-assisted documentation.

Conclusion: LLM automation has potential to substantially reduce clerical burden for trainees and improve case logging consistency. Findings demonstrate feasibility of AI-assisted documentation in medical education, highlighting need for further validation across institutions and clinical workflows.

Abstract: Procedural case logs are a core requirement in radiology training, yet they are time-consuming to complete and prone to inconsistency when authored manually. This study investigates whether large language models (LLMs) can automate procedural case log documentation directly from free-text radiology reports. We evaluate multiple local and commercial LLMs under instruction-based and chain-of-thought prompting to extract structured procedural information from 414 curated interventional radiology reports authored by nine residents between 2018 and 2024. Model performance is assessed using sensitivity, specificity, and F1-score, alongside inference latency and token efficiency to estimate operational cost. Results show that both local and commercial models achieve strong extraction performance, with best F1-scores approaching 0.87, while exhibiting different trade-offs between speed and cost. Automation using LLMs has the potential to substantially reduce clerical burden for trainees and improve consistency in case logging. These findings demonstrate the feasibility of AI-assisted documentation in medical education and highlight the need for further validation across institutions and clinical workflows.

Method: SSRAG integrates query augmentation, agentic routing, and structured retrieval combining vector and graph-based techniques with context unification.

Result: Extensive evaluations on TruthfulQA, SQuAD, and WikiQA datasets across five LLMs show consistent improvement in response quality over standard RAG implementations.

Conclusion: SSRAG enhances QA quality by refining retrieval processes and improving contextual grounding, leading to better answer accuracy and informativeness.

Abstract: Retrieval-Augmented Generation (RAG) has emerged as a powerful technique for enhancing the quality of responses in Question-Answering (QA) tasks. However, existing approaches often struggle with retrieving contextually relevant information, leading to incomplete or suboptimal answers. In this paper, we introduce Structured-Semantic RAG (SSRAG), a hybrid architecture that enhances QA quality by integrating query augmentation, agentic routing, and a structured retrieval mechanism combining vector and graph based techniques with context unification. By refining retrieval processes and improving contextual grounding, our approach improves both answer accuracy and informativeness. We conduct extensive evaluations on three popular QA datasets, TruthfulQA, SQuAD and WikiQA, across five Large Language Models (LLMs), demonstrating that our proposed approach consistently improves response quality over standard RAG implementations.

Method: Created UbuntuGuard benchmark from 155 domain experts across sensitive fields (including healthcare) who authored adversarial queries. Derived context-specific safety policies and reference responses capturing culturally grounded risk signals. Evaluated 13 models (6 general-purpose LLMs, 7 guardian models) across static, dynamic, and multilingual variants.

Result: Existing English-centric benchmarks overestimate real-world multilingual safety. Cross-lingual transfer provides partial but insufficient coverage. Dynamic models, while better at leveraging policies at inference time, still struggle to fully localize African-language contexts.

Conclusion: There’s an urgent need for multilingual, culturally grounded safety benchmarks to develop reliable and equitable guardian models for low-resource languages. UbuntuGuard addresses this gap by providing a policy-based safety benchmark tailored to African contexts.

Abstract: Current guardian models are predominantly Western-centric and optimized for high-resource languages, leaving low-resource African languages vulnerable to evolving harms, cross-lingual safety failures, and cultural misalignment. Moreover, most guardian models rely on rigid, predefined safety categories that fail to generalize across diverse linguistic and sociocultural contexts. Robust safety, therefore, requires flexible, runtime-enforceable policies and benchmarks that reflect local norms, harm scenarios, and cultural expectations. We introduce UbuntuGuard, the first African policy-based safety benchmark built from adversarial queries authored by 155 domain experts across sensitive fields, including healthcare. From these expert-crafted queries, we derive context-specific safety policies and reference responses that capture culturally grounded risk signals, enabling policy-aligned evaluation of guardian models. We evaluate 13 models, comprising six general-purpose LLMs and seven guardian models across three distinct variants: static, dynamic, and multilingual. Our findings reveal that existing English-centric benchmarks overestimate real-world multilingual safety, cross-lingual transfer provides partial but insufficient coverage, and dynamic models, while better equipped to leverage policies at inference time, still struggle to fully localize African-language contexts. These findings highlight the urgent need for multilingual, culturally grounded safety benchmarks to enable the development of reliable and equitable guardian models for low-resource languages. Our code can be found online.\footnote{Code repository available at https://github.com/hemhemoh/UbuntuGuard.

Method: Introduces template-based rewriting on top of agent-driven iterative loop: 1) kernels are semantically refactored into explicitly parameterizable templates, 2) template parameters are optimized via search-based autotuning, 3) agentic tuner performs templating, testing, analysis, and planning with profiling feedback, 4) executes constrained parameter search under hardware resource limits.

Result: Achieves speedups exceeding 3x in best cases on real-world CUDA kernels from SGLang. Template-plus-search design significantly reduces randomness compared to agent-only direct rewriting, making optimization more interpretable and systematic.

Conclusion: Template-based approach with search autotuning provides more stable and higher-quality GPU kernel optimization, can be extended to other backends (OpenCL, HIP) for automated performance optimization in production workloads.

Abstract: GPU code optimization is a key performance bottleneck for HPC workloads as well as large-model training and inference. Although compiler optimizations and hand-written kernels can partially alleviate this issue, achieving near-hardware-limit performance still relies heavily on manual code refactoring and parameter tuning. Recent progress in LLM-agent-based kernel generation and optimization has been reported, yet many approaches primarily focus on direct code rewriting, where parameter choices are often implicit and hard to control, or require human intervention, leading to unstable performance gains. This paper introduces a template-based rewriting layer on top of an agent-driven iterative loop: kernels are semantically refactored into explicitly parameterizable templates, and template parameters are then optimized via search-based autotuning, yielding more stable and higher-quality speedups. Experiments on a set of real-world kernels demonstrate speedups exceeding 3x in the best case. We extract representative CUDA kernels from SGLang as evaluation targets; the proposed agentic tuner iteratively performs templating, testing, analysis, and planning, and leverages profiling feedback to execute constrained parameter search under hardware resource limits. Compared to agent-only direct rewriting, the template-plus-search design significantly reduces the randomness of iterative optimization, making the process more interpretable and enabling a more systematic approach toward high-performance configurations. The proposed method can be further extended to OpenCL, HIP, and other backends to deliver automated performance optimization for real production workloads.

Method: Trained linear probes on LLM internal representations using the AMC subset of Easy2Hard benchmark translated into 21 languages, analyzing both shallow (early-layer) and deep (later-layer) representations.

Result: Deep representation probes achieve high within-language accuracy but poor cross-lingual generalization, while shallow representation probes show lower within-language performance but substantially better cross-lingual generalization.

Conclusion: LLMs form problem-difficulty representations in two stages: first language-agnostic (shallow layers), then language-specific (deep layers), extending the abstract-to-specific processing pattern from semantic content to meta-cognitive properties like difficulty estimation.

Abstract: Predicting problem-difficulty in large language models (LLMs) refers to estimating how difficult a task is according to the model itself, typically by training linear probes on its internal representations. In this work, we study the multilingual geometry of problem-difficulty in LLMs by training linear probes using the AMC subset of the Easy2Hard benchmark, translated into 21 languages. We found that difficulty-related signals emerge at two distinct stages of the model internals, corresponding to shallow (early-layers) and deep (later-layers) internal representations, that exhibit functionally different behaviors. Probes trained on deep representations achieve high accuracy when evaluated on the same language but exhibit poor cross-lingual generalization. In contrast, probes trained on shallow representations generalize substantially better across languages, despite achieving lower within-language performance. Together, these results suggest that LLMs first form a language-agnostic representation of problem difficulty, which subsequently becomes language-specific. This closely aligns with existing findings in LLM interpretability showing that models tend to operate in an abstract conceptual space before producing language-specific outputs. We demonstrate that this two-stage representational process extends beyond semantic content to high-level meta-cognitive properties such as problem-difficulty estimation.

Method: Two-stage framework: 1) Labeling stage with reflection-aware label correction using LLM as judge to detect reflection/self-correction behaviors and suppress overestimated rewards. 2) Training stage with Noise-Aware Iterative Training (NAIT) enabling PRM to progressively refine noisy labels based on its own confidence.

Result: Method substantially improves step-level correctness discrimination, achieving up to 27% absolute gain in average F1 over PRMs trained with noisy supervision.

Conclusion: The proposed framework effectively mitigates noisy supervision in PRMs by combining LLM-based reflection detection with confidence-aware iterative training, significantly improving step correctness evaluation.

Abstract: Process Reward Models (PRMs) have achieved strong results in complex reasoning, but are bottlenecked by costly process-level supervision. A widely used alternative, Monte Carlo Estimation (MCE), defines process rewards as the probability that a policy model reaches the correct final answer from a given reasoning step. However, step correctness is an intrinsic property of the reasoning trajectory, and should be invariant to policy choice. Our empirical findings show that MCE producing policy-dependent rewards that induce label noise, including false positives that reward incorrect steps and false negatives that penalize correct ones. To address above challenges, we propose a two-stage framework to mitigate noisy supervision. In the labeling stage, we introduce a reflection-aware label correction mechanism that uses a large language model (LLM) as a judge to detect reflection and self-correction behaviors related to the current reasoning step, thereby suppressing overestimated rewards. In the training stage, we further propose a \underline{\textbf{N}}oise-\underline{\textbf{A}}ware \underline{\textbf{I}}terative \underline{\textbf{T}}raining framework that enables the PRM to progressively refine noisy labels based on its own confidence. Extensive Experiments show that our method substantially improves step-level correctness discrimination, achieving up to a 27% absolute gain in average F1 over PRMs trained with noisy supervision.

Method: VISPA uses dynamic selection and internal model activation steering to control value expression, operating without additional training and enabling direct manipulation of value representation.

Result: VISPA performs well across all pluralistic alignment modes in healthcare and other domains, works with different steering initiations, models, and values, showing adaptability and effectiveness.

Conclusion: Pluralistic alignment can be achieved through internal activation mechanisms, offering a scalable path toward language models that serve diverse human perspectives.

Abstract: As large language models are increasingly used in high-stakes domains, it is essential that their outputs reflect not average} human preference, rather range of varying perspectives. Achieving such pluralism, however, remains challenging. Existing approaches consider limited values or rely on prompt-level interventions, lacking value control and representation. To address this, we introduce VISPA, a training-free pluralistic alignment framework, that enables direct control over value expression by dynamic selection and internal model activation steering. Across extensive empirical studies spanning multiple models and evaluation settings, we show VISPA is performant across all pluralistic alignment modes in healthcare and beyond. Further analysis reveals VISPA is adaptable with different steering initiations, model, and/or values. These results suggest that pluralistic alignment can be achieved through internal activation mechanisms, offering a scalable path toward language models that serves all.

Method: Proposes LLM Associative Memory Agents (LAMA) framework with dual-agent architecture: Person Agent and Media Agent specialized in different knowledge domains recall famous individuals with same names in parallel, then aggregate nationalities through indirect reasoning with voting for Top-1 and conditional completion for Top-K predictions.

Result: Achieved 0.817 accuracy on 99-country nationality prediction task, substantially outperforming conventional LLM prompting methods and neural models. Found LLMs more reliable in recalling concrete examples than abstract reasoning, recall-based approaches robust to low-frequency nationalities, and dual-agent architecture produces complementary synergistic effects.

Conclusion: Demonstrates effectiveness of new multi-agent system that retrieves and aggregates LLM knowledge rather than prompting reasoning, showing recall-based approaches leverage LLMs’ associative memory capabilities better than direct reasoning methods.

Abstract: Large language models (LLMs) possess extensive world knowledge, yet methods for effectively eliciting this knowledge remain underexplored. Nationality and region prediction tasks require understanding of not only linguistic features but also cultural and historical background, making LLM world knowledge particularly valuable. However, conventional LLM prompting methods rely on direct reasoning approaches, which have limitations in applying abstract linguistic rules. We propose LLM Associative Memory Agents (LAMA), a novel framework that leverages LLM world knowledge as associative memory. Rather than directly inferring nationality from names, LAMA recalls famous individuals with the same name and aggregates their nationalities through indirect reasoning. A dual-agent architecture comprising a Person Agent and a Media Agent, specialized in different knowledge domains, recalls famous individuals in parallel, generating Top-1 predictions through voting and Top-K predictions through conditional completion. On a 99-country nationality prediction task, LAMA achieved 0.817 accuracy, substantially outperforming conventional LLM prompting methods and neural models. Our experiments reveal that LLMs exhibit higher reliability in recalling concrete examples than in abstract reasoning, that recall-based approaches are robust to low-frequency nationalities independent of data frequency distributions, and that the dual-agent architecture functions complementarily to produce synergistic effects. These results demonstrate the effectiveness of a new multi-agent system that retrieves and aggregates LLM knowledge rather than prompting reasoning.

Method: MEDASSESS-X applies alignment at inference time rather than through Supervised Fine-Tuning. It uses lightweight steering vectors to guide model activations toward medically consistent reasoning without updating model weights or requiring domain-specific retraining. This inference-time alignment layer stabilizes CQA performance across both general-purpose and specialized medical LLMs.

Result: MEDASSESS-X delivers consistent gains across all LLM families: improving Accuracy by up to +6%, Factual Consistency by +7%, and reducing Safety Error Rate by as much as 50%. The framework resolves the specialization fallacy by showing that inference-time alignment can achieve superior performance without domain-specific retraining.

Conclusion: The specialization fallacy in Clinical QA can be resolved through inference-time alignment rather than domain-specific fine-tuning. MEDASSESS-X provides a practical, cost-effective deployment framework that works across both general-purpose and specialized LLMs, challenging the industry assumption that medical specialization requires model retraining.

Abstract: Clinical Question-Answering (CQA) industry systems are increasingly rely on Large Language Models (LLMs), yet their deployment is often guided by the assumption that domain-specific fine-tuning is essential. Although specialised medical LLMs such as BioBERT, BioGPT, and PubMedBERT remain popular, they face practical limitations including narrow coverage, high retraining costs, and limited adaptability. Efforts based on Supervised Fine-Tuning (SFT) have attempted to address these assumptions but continue to reinforce what we term the SPECIALISATION FALLACY-the belief that specialised medical LLMs are inherently superior for CQA. To address this assumption, we introduce MEDASSESS-X, a deployment-industry-oriented CQA framework that applies alignment at inference time rather than through SFT. MEDASSESS-X uses lightweight steering vectors to guide model activations toward medically consistent reasoning without updating model weights or requiring domain-specific retraining. This inference-time alignment layer stabilises CQA performance across both general-purpose and specialised medical LLMs, thereby resolving the SPECIALISATION FALLACY. Empirically, MEDASSESS-X delivers consistent gains across all LLM families, improving Accuracy by up to +6%, Factual Consistency by +7%, and reducing Safety Error Rate by as much as 50%.

Method: JurisMMA framework decomposes trial tasks, standardizes processes, and organizes them into distinct stages. The authors also built JurisMM, a large dataset with over 100,000 recent Chinese judicial records containing both text and multimodal video-text data.

Result: Experiments on JurisMM and benchmark LawBench validate the framework’s effectiveness. The results show the framework works well for LJP and has broader applicability to other legal applications.

Conclusion: JurisMMA offers an effective approach for LJP that handles complex legal cases better than traditional methods, and the JurisMM dataset enables comprehensive evaluation. The framework provides new perspectives for future legal methods and datasets.

Abstract: Legal Judgment Prediction (LJP) aims to predict the outcomes of legal cases based on factual descriptions, serving as a fundamental task to advance the development of legal systems. Traditional methods often rely on statistical analyses or role-based simulations but face challenges with multiple allegations, diverse evidence, and lack adaptability. In this paper, we introduce JurisMMA, a novel framework for LJP that effectively decomposes trial tasks, standardizes processes, and organizes them into distinct stages. Furthermore, we build JurisMM, a large dataset with over 100,000 recent Chinese judicial records, including both text and multimodal video-text data, enabling comprehensive evaluation. Experiments on JurisMM and the benchmark LawBench validate our framework’s effectiveness. These results indicate that our framework is effective not only for LJP but also for a broader range of legal applications, offering new perspectives for the development of future legal methods and datasets.

Method: Project focuses on explanatory/collaborative sequences in French language classrooms, analyzing multimodal phenomena (speech, prosody, gestures, gaze, positioning). Uses discourse analysis and interactional linguistics to define explanatory discourse as tripartite sequences. Examines transcription conventions and methodological foundations, using corpora like INTER-EXPLIC (30 hours) and EXPLIC-LEXIC for manual annotation and as reference datasets.

Result: Demonstrates variability and interpretative dimension of transcription practices. Identifies transcription conventions, annotation categories, and analytical units compatible with machine learning. Establishes methodological framework rather than delivering fully operational automated system.

Conclusion: TRAIMA establishes rigorous methodological framework for automatic processing of multimodal pedagogical interactions, laying groundwork for future interdisciplinary research at intersection of didactics, discourse analysis, multimodality, and AI in education.

Abstract: The TRAIMA project (TRaitement Automatique des Interactions Multimodales en Apprentissage), conducted between March 2019 and June 2020, investigates the potential of automatic processing of multimodal interactions in educational settings. The project addresses a central methodological challenge in educational and interactional research: the analysis of verbal, paraverbal, and non-verbal data is currently carried out manually, making it extremely time-consuming and difficult to scale. TRAIMA explores how machine learning approaches could contribute to the categorisation and classification of such interactions. The project focuses specifically on explanatory and collaborative sequences occurring in classroom interactions, particularly in French as a Foreign Language (FLE) and French as a First Language (FLM) contexts. These sequences are analysed as inherently multimodal phenomena, combining spoken language with prosody, gestures, posture, gaze, and spatial positioning. A key theoretical contribution of the project is the precise linguistic and interactional definition of explanatory discourse as a tripartite sequence (opening, explanatory core, closure), drawing on discourse analysis and interactional linguistics. A substantial part of the research is devoted to the methodological foundations of transcription, which constitute a critical bottleneck for any form of automation. The report provides a detailed state of the art of existing transcription conventions (ICOR, Mondada, GARS, VALIBEL, Ferr{é}), highlighting their respective strengths and limitations when applied to multimodal classroom data. Through comparative analyses of manually transcribed sequences, the project demonstrates the inevitable variability and interpretative dimension of transcription practices, depending on theoretical positioning and analytical goals. Empirical work is based on several corpora, notably the INTER-EXPLIC corpus (approximately 30 hours of classroom interaction) and the EXPLIC-LEXIC corpus, which serve both as testing grounds for manual annotation and as reference datasets for future automation. Particular attention is paid to teacher gestures (kin{é}sic and proxemic resources), prosodic features, and their functional role in meaning construction and learner comprehension. The project also highlights the strategic role of the Techn{é}LAB platform, which provides advanced multimodal data capture (multi-camera video, synchronized audio, eye-tracking, digital interaction traces) and constitutes both a research infrastructure and a test environment for the development of automated tools. In conclusion, TRAIMA does not aim to deliver a fully operational automated system, but rather to establish a rigorous methodological framework for the automatic processing of multimodal pedagogical interactions. The project identifies transcription conventions, annotation categories, and analytical units that are compatible with machine learning approaches, while emphasizing the need for theoretical explicitness and researcher reflexivity. TRAIMA thus lays the groundwork for future interdisciplinary research at the intersection of didactics, discourse analysis, multimodality, and artificial intelligence in education.

Method: Analyzed three open-source models using interpretability pipeline combining probing, neuron-level attribution, and targeted intervention on two datasets (toxicity generation and clinical narrative understanding).

Result: Demographic information is distributed across internal units with substantial cross-model variation. Some units encode sensitive/stereotype associations, but identical demographic cues can induce different behaviors. Interventions reduce bias but leave substantial residual effects.

Conclusion: Findings suggest behavioral rather than representational change from interventions, motivating more systematic mitigation approaches for demographic bias in LLMs.

Abstract: Large language models (LLMs) increasingly operate in high-stakes settings including healthcare and medicine, where demographic attributes such as race and ethnicity may be explicitly stated or implicitly inferred from text. However, existing studies primarily document outcome-level disparities, offering limited insight into internal mechanisms underlying these effects. We present a mechanistic study of how race and ethnicity are represented and operationalized within LLMs. Using two publicly available datasets spanning toxicity-related generation and clinical narrative understanding tasks, we analyze three open-source models with a reproducible interpretability pipeline combining probing, neuron-level attribution, and targeted intervention. We find that demographic information is distributed across internal units with substantial cross-model variation. Although some units encode sensitive or stereotype-related associations from pretraining, identical demographic cues can induce qualitatively different behaviors. Interventions suppressing such neurons reduce bias but leave substantial residual effects, suggesting behavioral rather than representational change and motivating more systematic mitigation.

Method: Two-stage approach: 1) Offline preprocessing embeds information from related text chunks into each chunk, 2) Online reprocessing selectively recomputes KV cache for tokens that the model focuses on, enabling efficient KV cache reuse while preserving context.

Result: FusionRAG significantly improves generation quality at same recomputation ratio compared to SOTA solutions. With <15% token recomputation, achieves up to 70% higher normalized F1 scores than baselines and reduces TTFT by 2.66x-9.39x compared to Full Attention.

Conclusion: FusionRAG successfully addresses the KV cache reuse challenge in RAG by preserving cross-chunk context through preprocessing and selective recomputation, achieving superior quality-efficiency trade-offs for RAG inference acceleration.

Abstract: Retrieval-Augmented Generation enhances Large Language Models by integrating external knowledge, which reduces hallucinations but increases prompt length. This increase leads to higher computational costs and longer Time to First Token (TTFT). To mitigate this issue, existing solutions aim to reuse the preprocessed KV cache of each retrieved chunk to accelerate RAG. However, the lack of cross-chunk contextual information leads to a significant drop in generation quality, leaving the potential benefits of KV cache reuse largely unfulfilled. The challenge lies in how to reuse the precomputed KV cache of chunks while preserving generation quality. We propose FusionRAG, a novel inference framework that optimizes both the preprocessing and reprocessing stages of RAG. In the offline preprocessing stage, we embed information from other related text chunks into each chunk, while in the online reprocessing stage, we recompute the KV cache for tokens that the model focuses on. As a result, we achieve a better trade-off between generation quality and efficiency. According to our experiments, FusionRAG significantly improves generation quality at the same recomputation ratio compared to previous state-of-the-art solutions. By recomputing fewer than 15% of the tokens, FusionRAG achieves up to 70% higher normalized F1 scores than baselines and reduces TTFT by 2.66x-9.39x compared to Full Attention.

Method: Reframe test-time adaptation as a budget-constrained memory consolidation problem. Propose Gdwm (Gated Differentiable Working Memory) with a write controller that gates consolidation based on Contextual Utility - an information-theoretic measure of long-range contextual dependence. The controller allocates gradient steps to high-utility regions while maintaining global coverage.

Result: Experiments on ZeroSCROLLS and LongBench v2 show Gdwm achieves comparable or superior performance with 4× fewer gradient steps than uniform baselines, establishing a new efficiency-performance Pareto frontier for test-time adaptation.

Conclusion: Gdwm’s selective consolidation approach based on contextual utility effectively addresses the limitations of uniform write policies in test-time adaptation, significantly improving efficiency while maintaining or enhancing performance on long-context tasks.

Abstract: Long contexts challenge transformers: attention scores dilute across thousands of tokens, critical information is often lost in the middle, and models struggle to adapt to novel patterns at inference time. Recent work on test-time adaptation addresses this by maintaining a form of working memory – transient parameters updated on the current context – but existing approaches rely on uniform write policies that waste computation on low-utility regions and suffer from high gradient variance across semantically heterogeneous contexts. In this work, we reframe test-time adaptation as a budget-constrained memory consolidation problem, focusing on which parts of the context should be consolidated into working memory under limited computation. We propose Gdwm (Gated Differentiable Working Memory), a framework that introduces a write controller to gate the consolidation process. The controller estimates Contextual Utility, an information-theoretic measure of long-range contextual dependence, and allocates gradient steps accordingly while maintaining global coverage. Experiments on ZeroSCROLLS and LongBench v2 demonstrate that Gdwm achieves comparable or superior performance with 4$\times$ fewer gradient steps than uniform baselines, establishing a new efficiency-performance Pareto frontier for test-time adaptation.

Method: Constructed dataset from GitHub issues and reproducibility papers, proposed synthetic data generation method to scale dataset, analyzed discrepancy types/categories, and evaluated 21 LLMs on the task.

Result: Dataset contains 611 paper-code discrepancies (81 real, 530 synthetic) across diverse disciplines. LLMs struggle with SciCoQA, especially on omitted paper details, long-context inputs, and unfamiliar data. Best model (GPT-5) only detects 45.7% of real-world discrepancies.

Conclusion: Paper-code discrepancy detection is challenging for current LLMs, highlighting the need for better tools to ensure faithful implementations and reproducibility in computational science research.

Abstract: We present SciCoQA, a dataset for detecting discrepancies between scientific publications and their codebases to ensure faithful implementations. We construct SciCoQA from GitHub issues and reproducibility papers, and to scale our dataset, we propose a synthetic data generation method for constructing paper-code discrepancies. We analyze the paper-code discrepancies in detail and propose discrepancy types and categories to better understand the occurring mismatches. In total, our dataset consists of 611 paper-code discrepancies (81 real, 530 synthetic), spanning diverse computational science disciplines, including AI, Physics, Quantitative Biology, and others. Our evaluation of 21 LLMs highlights the difficulty of SciCoQA, particularly for instances involving omitted paper details, long-context inputs, and data outside the models’ pre-training corpus. The best performing model in our evaluation, GPT-5, can only detect 45.7% of real-world paper-code discrepancies.

Method: Created IndoSoSci dataset from 151 Indonesian social science journals, extracted cultural facts, and used retrieval-augmented generation with LLM-generated hypothetical documents as queries during retrieval.

Result: The proposed method yields strong performance gains over baselines on the IndoCulture benchmark, and combining IndoSoSci with Indonesian Wikipedia achieves state-of-the-art accuracy.

Conclusion: Local social science journals are valuable for cultural knowledge injection into LLMs, and the proposed RAG approach with LLM-generated queries effectively improves cultural understanding benchmarks.

Abstract: Recently there have been intensifying efforts to improve the understanding of Indonesian cultures by large language models (LLMs). An attractive source of cultural knowledge that has been largely overlooked is local journals of social science, which likely contain substantial cultural studies from a native perspective. We present a novel text dataset of journal article passages, created from 151 open-source Indonesian social science journals, called IndoSoSci. We demonstrate an effective recipe for injecting Indonesian cultural knowledge therein into LLMs: extracting the facts related to Indonesian culture, and apply retrieval-augmented generation (RAG) with LLM-generated hypothetical documents as queries during retrieval. The proposed recipe yields strong performance gains over several strong baselines on the IndoCulture benchmark. Additionally, by combining IndoSoSci with Indonesian Wikipedia, we set a new state-of-the-art accuracy on the IndoCulture benchmark.

Method: Systematic review of bidirectional interaction between LLMs and MABs at component level. Analyzes existing LLM-enhanced bandit systems and bandit-enhanced LLM systems, examining design, methodologies, and performance.

Result: Identifies key challenges and representative findings. Shows MAB algorithms address LLM challenges across pre-training, RAG, and personalization, while LLMs enhance MAB by redefining core components like arm definition and environment modeling.

Conclusion: The survey provides comprehensive insights into LLM-MAB interaction, establishes foundational understanding, and guides future research with accompanying GitHub repository for literature indexing.

Abstract: Large language models (LLMs) have become powerful and widely used systems for language understanding and generation, while multi-armed bandit (MAB) algorithms provide a principled framework for adaptive decision-making under uncertainty. This survey explores the potential at the intersection of these two fields. As we know, it is the first survey to systematically review the bidirectional interaction between large language models and multi-armed bandits at the component level. We highlight the bidirectional benefits: MAB algorithms address critical LLM challenges, spanning from pre-training to retrieval-augmented generation (RAG) and personalization. Conversely, LLMs enhance MAB systems by redefining core components such as arm definition and environment modeling, thereby improving decision-making in sequential tasks. We analyze existing LLM-enhanced bandit systems and bandit-enhanced LLM systems, providing insights into their design, methodologies, and performance. Key challenges and representative findings are identified to help guide future research. An accompanying GitHub repository that indexes relevant literature is available at https://github.com/bucky1119/Awesome-LLM-Bandit-Interaction.

Method: Combines opaque and transparent deep learning methods with a natural language generation module that produces clinician-friendly textual explanations, designed with dental experts using a rule-based approach.

Result: Dental experts rated the generated explanations 4.77±0.12 out of 5 across five dimensions, and the system scored 4.40±0.27 out of 5 on the ALTAI AI Trustworthiness Assessment checklist.

Conclusion: The proposed system successfully improves transparency and trust in dental age estimation AI by providing clinician-friendly explanations validated by experts, demonstrating strong performance in both explanation quality and trustworthiness assessment.

Abstract: Integrating deep learning into healthcare enables personalized care but raises trust issues due to model opacity. To improve transparency, we propose a system for dental age estimation from panoramic images that combines an opaque and a transparent method within a natural language generation (NLG) module. This module produces clinician-friendly textual explanations about the age estimations, designed with dental experts through a rule-based approach. Following the best practices in the field, the quality of the generated explanations was manually validated by dental experts using a questionnaire. The results showed a strong performance, since the experts rated 4.77+/-0.12 (out of 5) on average across the five dimensions considered. We also performed a trustworthy self-assessment procedure following the ALTAI checklist, in which it scored 4.40+/-0.27 (out of 5) across seven dimensions of the AI Trustworthiness Assessment List.

Method: The authors introduce a repair-aware evaluation setting that explicitly distinguishes answerable vs. unanswerable audio inputs using a semantic-acoustic masking protocol. They propose the Evaluability Awareness and Repair (EAR) score, a non-compensatory metric that jointly evaluates task competence under answerable conditions and repair behavior under unanswerable conditions.

Result: Experiments on two spoken QA benchmarks across diverse LALMs reveal a consistent gap: while many models perform well when inputs are answerable, most fail to recognize semantic unanswerability and initiate appropriate conversational repair. This exposes limitations of accuracy-centric evaluation practices.

Conclusion: The findings motivate reliability assessments that treat unanswerable inputs as cues for repair and continued interaction, rather than just measuring answer accuracy. The proposed EAR score and repair-aware evaluation framework provide a more comprehensive way to assess conversational reliability in LALMs.

Abstract: Large Audio-Language Models (LALMs) have demonstrated strong performance in spoken question answering (QA), with existing evaluations primarily focusing on answer accuracy and robustness to acoustic perturbations. However, such evaluations implicitly assume that spoken inputs remain semantically answerable, an assumption that often fails in real-world interaction when essential information is missing. In this work, we introduce a repair-aware evaluation setting that explicitly distinguishes between answerable and unanswerable audio inputs. We define answerability as a property of the input itself and construct paired evaluation conditions using a semantic-acoustic masking protocol. Based on this setting, we propose the Evaluability Awareness and Repair (EAR) score, a non-compensatory metric that jointly evaluates task competence under answerable conditions and repair behavior under unanswerable conditions. Experiments on two spoken QA benchmarks across diverse LALMs reveal a consistent gap between answer accuracy and conversational reliability: while many models perform well when inputs are answerable, most fail to recognize semantic unanswerability and initiate appropriate conversational repair. These findings expose a limitation of prevailing accuracy-centric evaluation practices and motivate reliability assessments that treat unanswerable inputs as cues for repair and continued interaction.

Method: Created SCMPE benchmark assessing both static objective tasks (knowledge-oriented) and multi-turn simulated patient interactions (workflow-based). Analyzed performance across these dimensions, examined RAG impact, and mapped guideline adherence vs decision quality.

Result: Models excel at static tasks but performance drops significantly in dynamic clinical dialogues. Main bottleneck is active information gathering and dynamic state tracking, not knowledge retention. General models show “High Efficacy, Low Safety” risk. RAG reduces hallucinations in static tasks but has limited/heterogeneous effectiveness in dynamic workflows, sometimes causing degradation.

Conclusion: External knowledge (RAG) alone cannot bridge reasoning gaps without domain-adaptive pre-training. Study provides roadmap for bridging standardized knowledge with safe autonomous clinical practice by identifying capability boundaries of dental LLMs.

Abstract: The transition of Large Language Models (LLMs) from passive knowledge retrievers to autonomous clinical agents demands a shift in evaluation-from static accuracy to dynamic behavioral reliability. To explore this boundary in dentistry, a domain where high-quality AI advice uniquely empowers patient-participatory decision-making, we present the Standardized Clinical Management & Performance Evaluation (SCMPE) benchmark, which comprehensively assesses performance from knowledge-oriented evaluations (static objective tasks) to workflow-based simulations (multi-turn simulated patient interactions). Our analysis reveals that while models demonstrate high proficiency in static objective tasks, their performance precipitates in dynamic clinical dialogues, identifying that the primary bottleneck lies not in knowledge retention, but in the critical challenges of active information gathering and dynamic state tracking. Mapping “Guideline Adherence” versus “Decision Quality” reveals a prevalent “High Efficacy, Low Safety” risk in general models. Furthermore, we quantify the impact of Retrieval-Augmented Generation (RAG). While RAG mitigates hallucinations in static tasks, its efficacy in dynamic workflows is limited and heterogeneous, sometimes causing degradation. This underscores that external knowledge alone cannot bridge the reasoning gap without domain-adaptive pre-training. This study empirically charts the capability boundaries of dental LLMs, providing a roadmap for bridging the gap between standardized knowledge and safe, autonomous clinical practice.

Method: Comprehensive evaluation of dLLMs (LLaDA, Dream) across two agentic paradigms: Embodied Agents (long-horizon planning) and Tool-Calling Agents (precise formatting). Introduced DiffuAgent, a multi-agent evaluation framework integrating dLLMs as plug-and-play cognitive cores.

Result: dLLMs fail as reliable agentic backbones: (1) In embodied settings, they suffer repeated attempts and fail to branch under temporal feedback; (2) In tool-calling settings, they fail to maintain symbolic precision (strict JSON schemas) under diffusion noise. dLLMs are only effective in non-causal roles like memory summarization and tool selection.

Conclusion: dLLMs require incorporation of causal, precise, and logically grounded reasoning mechanisms into the denoising process to be viable for agentic tasks. Current dLLMs cannot serve as reliable agentic backbones despite efficiency hype.

Abstract: The pursuit of real-time agentic interaction has driven interest in Diffusion-based Large Language Models (dLLMs) as alternatives to auto-regressive backbones, promising to break the sequential latency bottleneck. However, does such efficiency gains translate into effective agentic behavior? In this work, we present a comprehensive evaluation of dLLMs (e.g., LLaDA, Dream) across two distinct agentic paradigms: Embodied Agents (requiring long-horizon planning) and Tool-Calling Agents (requiring precise formatting). Contrary to the efficiency hype, our results on Agentboard and BFCL reveal a “bitter lesson”: current dLLMs fail to serve as reliable agentic backbones, frequently leading to systematically failure. (1) In Embodied settings, dLLMs suffer repeated attempts, failing to branch under temporal feedback. (2) In Tool-Calling settings, dLLMs fail to maintain symbolic precision (e.g. strict JSON schemas) under diffusion noise. To assess the potential of dLLMs in agentic workflows, we introduce DiffuAgent, a multi-agent evaluation framework that integrates dLLMs as plug-and-play cognitive cores. Our analysis shows that dLLMs are effective in non-causal roles (e.g., memory summarization and tool selection) but require the incorporation of causal, precise, and logically grounded reasoning mechanisms into the denoising process to be viable for agentic tasks.

Method: ChartAttack framework injects misleaders into chart designs to induce incorrect interpretations. The authors also create AttackViz, a chart QA dataset where each (chart specification, QA) pair is labeled with effective misleaders and their induced incorrect answers.

Result: ChartAttack significantly degrades MLLM QA performance, reducing accuracy by average 19.6 points (in-domain) and 14.9 points (cross-domain). Human study shows average 20.2 point accuracy drop for participants exposed to misleading charts.

Conclusion: Findings highlight urgent need for robustness and security considerations in design, evaluation, and deployment of MLLM-based chart generation systems. Code and data are made publicly available.

Abstract: Multimodal large language models (MLLMs) are increasingly used to automate chart generation from data tables, enabling efficient data analysis and reporting but also introducing new misuse risks. In this work, we introduce ChartAttack, a novel framework for evaluating how MLLMs can be misused to generate misleading charts at scale. ChartAttack injects misleaders into chart designs, aiming to induce incorrect interpretations of the underlying data. Furthermore, we create AttackViz, a chart question-answering (QA) dataset where each (chart specification, QA) pair is labeled with effective misleaders and their induced incorrect answers. Experiments in in-domain and cross-domain settings show that ChartAttack significantly degrades the QA performance of MLLM readers, reducing accuracy by an average of 19.6 points and 14.9 points, respectively. A human study further shows an average 20.2 point drop in accuracy for participants exposed to misleading charts generated by ChartAttack. Our findings highlight an urgent need for robustness and security considerations in the design, evaluation, and deployment of MLLM-based chart generation systems. We make our code and data publicly available.

Method: Proposes Graph Reasoning Paradigm (GRP) for structured symbolic reasoning using graph representations with step-level cognitive labels. Develops PASC-GRPO optimization that uses structured evaluation instead of semantic evaluation, process-aware verification via graph-structured outcome rewards, and stratified clipping to mitigate reward hacking.

Result: Experiments show significant improvements across mathematical reasoning and code generation tasks.

Conclusion: GRP with PASC-GRPO effectively addresses limitations of current RLVR-based LCoT methods by introducing structured reasoning and optimized training processes.

Abstract: Long Chain-of-Thought (LCoT), achieved by Reinforcement Learning with Verifiable Rewards (RLVR), has proven effective in enhancing the reasoning capabilities of Large Language Models (LLMs). However, reasoning in current LLMs is primarily generated as plain text, where performing semantic evaluation on such unstructured data creates a computational bottleneck during training. Despite RLVR-based optimization, existing methods still suffer from coarse-grained supervision, reward hacking, high training costs, and poor generalization. To address these issues, we propose the Graph Reasoning Paradigm (GRP), which realizes structured and symbolic reasoning, implemented via graph-structured representations with step-level cognitive labels. Building upon GRP, we further design Process-Aware Stratified Clipping Group Relative Policy Optimization (PASC-GRPO), which leverages structured evaluation to replace semantic evaluation, achieves process-aware verification through graph-structured outcome rewards, and mitigates reward hacking via stratified clipping advantage estimation. Experiments demonstrate significant improvements across mathematical reasoning and code generation tasks. Data, models, and code will be released later.

Method: Proposes BiAtt-BiRNN-HateXplain model with bidirectional attention mechanism and BiRNN layer to capture sequential aspects of input data. Uses multi-task learning to simultaneously perform classification and explainability tasks, ensuring consistent attention patterns and reducing unintentional bias.

Result: Experimental results on HateXplain dataset show clear improvements in detection performance, explainability quality, and reduction of unintentional bias compared to existing approaches.

Conclusion: The proposed bidirectional attention model with multi-task learning effectively addresses attention variability in hate speech detection, leading to more reliable, explainable, and less biased classification systems that are more transparent than complex LLMs.

Abstract: Technological advances in the Internet and online social networks have brought many benefits to humanity. At the same time, this growth has led to an increase in hate speech, the main global threat. To improve the reliability of black-box models used for hate speech detection, post-hoc approaches such as LIME, SHAP, and LRP provide the explanation after training the classification model. In contrast, multi-task approaches based on the HateXplain benchmark learn to explain and classify simultaneously. However, results from HateXplain-based algorithms show that predicted attention varies considerably when it should be constant. This attention variability can lead to inconsistent interpretations, instability of predictions, and learning difficulties. To solve this problem, we propose the BiAtt-BiRNN-HateXplain (Bidirectional Attention BiRNN HateXplain) model which is easier to explain compared to LLMs which are more complex in view of the need for transparency, and will take into account the sequential aspect of the input data during explainability thanks to a BiRNN layer. Thus, if the explanation is correctly estimated, thanks to multi-task learning (explainability and classification task), the model could classify better and commit fewer unintentional bias errors related to communities. The experimental results on HateXplain data show a clear improvement in detection performance, explainability and a reduction in unintentional bias.

Method: Developed CEDAR benchmark using a novel pipeline: 1) Leverage LLM-generated provisional labels to identify instances with cross-cultural emotional distinctions, 2) Apply rigorous human evaluation to derive reliable ground-truth annotations. The benchmark includes 10,962 instances across 7 languages and 14 fine-grained emotion categories, with multimodal (400 per language) and text-only (1,166 per language) samples.

Result: Evaluation of 17 representative multilingual models reveals a dissociation between language consistency and cultural alignment. Models show language consistency but struggle with culturally grounded affective understanding, indicating this remains a significant challenge.

Conclusion: Culturally grounded affective understanding is a distinct and challenging aspect of cultural alignment that current LLMs fail to adequately capture. The CEDAR benchmark provides a valuable tool for assessing this dimension of cultural competence in AI systems.

Abstract: Culture serves as a fundamental determinant of human affective processing and profoundly shapes how individuals perceive and interpret emotional stimuli. Despite this intrinsic link extant evaluations regarding cultural alignment within Large Language Models primarily prioritize declarative knowledge such as geographical facts or established societal customs. These benchmarks remain insufficient to capture the subjective interpretative variance inherent to diverse sociocultural lenses. To address this limitation, we introduce CEDAR, a multimodal benchmark constructed entirely from scenarios capturing Culturally \underline{\textsc{E}}licited \underline{\textsc{D}}istinct \underline{\textsc{A}}ffective \underline{\textsc{R}}esponses. To construct CEDAR, we implement a novel pipeline that leverages LLM-generated provisional labels to isolate instances yielding cross-cultural emotional distinctions, and subsequently derives reliable ground-truth annotations through rigorous human evaluation. The resulting benchmark comprises 10,962 instances across seven languages and 14 fine-grained emotion categories, with each language including 400 multimodal and 1,166 text-only samples. Comprehensive evaluations of 17 representative multilingual models reveal a dissociation between language consistency and cultural alignment, demonstrating that culturally grounded affective understanding remains a significant challenge for current models.

Method: Proposes SASA framework with two key components: 1) Separated attention mechanism to encode triples into decoupled contextual representations and fuse them through effective interactive way, and 2) Semantic-aware hierarchical contrastive learning as auxiliary training objective considering both local and global level contrastive learning to improve discriminative capabilities and semantic learning.

Result: Experimental results show SASA significantly outperforms state-of-the-art methods, advancing accuracy by +5.9% on FB15k-237 and +3.4% on YAGO3-10 benchmark datasets.

Conclusion: SASA effectively addresses the limitations of existing triple classification methods by enhancing semantic interaction through separated attention and improving semantic representation learning via hierarchical contrastive learning, achieving substantial performance improvements on standard benchmarks.

Abstract: Knowledge Graphs~(KGs) often suffer from unreliable knowledge, which restricts their utility. Triple Classification~(TC) aims to determine the validity of triples from KGs. Recently, text-based methods learn entity and relation representations from natural language descriptions, significantly improving the generalization capabilities of TC models and setting new benchmarks in performance. However, there are still two critical challenges. First, existing methods often ignore the effective semantic interaction among different KG components. Second, most approaches adopt single binary classification training objective, leading to insufficient semantic representation learning. To address these challenges, we propose \textbf{SASA}, a novel framework designed to enhance TC models via separated attention mechanism and semantic-aware contrastive learning~(CL). Specifically, we first propose separated attention mechanism to encode triples into decoupled contextual representations and then fuse them through a more effective interactive way. Then, we introduce semantic-aware hierarchical CL as auxiliary training objective to guide models in improving their discriminative capabilities and achieving sufficient semantic learning, considering both local level and global level CL. Experimental results across two benchmark datasets demonstrate that SASA significantly outperforms state-of-the-art methods. In terms of accuracy, we advance the state-of-the-art by +5.9% on FB15k-237 and +3.4% on YAGO3-10.

Method: Developed a FastConformer-Transducer model with rigorous text normalization pipeline, two-stage curriculum learning for Isan dialect adaptation, and created Typhoon ASR Benchmark for standardized evaluation.

Result: Compact 115M-parameter model achieves 45x computational cost reduction compared to Whisper Large-v3 while maintaining comparable accuracy; resolves systemic Thai transcription ambiguities.

Conclusion: Text normalization can match model scaling impact for efficiency; released Typhoon ASR Benchmark addresses reproducibility challenges and provides standardized evaluation for Thai ASR research.

Abstract: Large encoder-decoder models like Whisper achieve strong offline transcription but remain impractical for streaming applications due to high latency. However, due to the accessibility of pre-trained checkpoints, the open Thai ASR landscape remains dominated by these offline architectures, leaving a critical gap in efficient streaming solutions. We present Typhoon ASR Real-time, a 115M-parameter FastConformer-Transducer model for low-latency Thai speech recognition. We demonstrate that rigorous text normalization can match the impact of model scaling: our compact model achieves a 45x reduction in computational cost compared to Whisper Large-v3 while delivering comparable accuracy. Our normalization pipeline resolves systemic ambiguities in Thai transcription –including context-dependent number verbalization and repetition markers (mai yamok) –creating consistent training targets. We further introduce a two-stage curriculum learning approach for Isan (north-eastern) dialect adaptation that preserves Central Thai performance. To address reproducibility challenges in Thai ASR, we release the Typhoon ASR Benchmark, a gold-standard human-labeled datasets with transcriptions following established Thai linguistic conventions, providing standardized evaluation protocols for the research community.

Method: Introduces the Simplification Profiler that generates multidimensional, interpretable fingerprints from aggregated simplifications. Uses meta-evaluation with linear classifiers to test if fingerprints can reliably identify different model configurations, measuring descriptive power without human-rated datasets.

Result: The Profiler can distinguish high-level behavioral variations between prompting strategies and fine-grained changes from prompt engineering. Complete feature set achieves classification F1-scores up to 71.9%, improving upon simple baselines by over 48 percentage points.

Conclusion: The Simplification Profiler offers developers granular, actionable analysis to build more effective and truly adaptive text simplification systems by measuring models’ unique behavioral signatures as an alternative to correlating metrics with human preferences.

Abstract: While Large Language Models (LLMs) produce highly nuanced text simplifications, developers currently lack tools for a holistic, efficient, and reproducible diagnosis of their behavior. This paper introduces the Simplification Profiler, a diagnostic toolkit that generates a multidimensional, interpretable fingerprint of simplified texts. Multiple aggregated simplifications of a model result in a model’s fingerprint. This novel evaluation paradigm is particularly vital for languages, where the data scarcity problem is magnified when creating flexible models for diverse target groups rather than a single, fixed simplification style. We propose that measuring a model’s unique behavioral signature is more relevant in this context as an alternative to correlating metrics with human preferences. We operationalize this with a practical meta-evaluation of our fingerprints’ descriptive power, which bypasses the need for large, human-rated datasets. This test measures if a simple linear classifier can reliably identify various model configurations by their created simplifications, confirming that our metrics are sensitive to a model’s specific characteristics. The Profiler can distinguish high-level behavioral variations between prompting strategies and fine-grained changes from prompt engineering, including few-shot examples. Our complete feature set achieves classification F1-scores up to 71.9 %, improving upon simple baselines by over 48 percentage points. The Simplification Profiler thus offers developers a granular, actionable analysis to build more effective and truly adaptive text simplification systems.

Method: Created Alexandria dataset through community-driven, human-translated approach covering 13 Arab countries and 11 high-impact domains, with city-of-origin metadata, multi-turn conversational scenarios, and speaker-addressee gender configurations.

Result: Dataset comprises 107K total samples serving as both training resource and benchmark; evaluation of Arabic-aware LLMs exposes significant persistent challenges in translating across diverse Arabic dialects and sub-dialects.

Conclusion: Alexandria bridges the gap in dialectal Arabic MT by providing unprecedented granularity and authentic local varieties, enabling better evaluation and development of models for real-world Arabic language applications.

Abstract: Arabic is a highly diglossic language where most daily communication occurs in regional dialects rather than Modern Standard Arabic. Despite this, machine translation (MT) systems often generalize poorly to dialectal input, limiting their utility for millions of speakers. We introduce \textbf{Alexandria}, a large-scale, community-driven, human-translated dataset designed to bridge this gap. Alexandria covers 13 Arab countries and 11 high-impact domains, including health, education, and agriculture. Unlike previous resources, Alexandria provides unprecedented granularity by associating contributions with city-of-origin metadata, capturing authentic local varieties beyond coarse regional labels. The dataset consists of multi-turn conversational scenarios annotated with speaker-addressee gender configurations, enabling the study of gender-conditioned variation in dialectal use. Comprising 107K total samples, Alexandria serves as both a training resource and a rigorous benchmark for evaluating MT and Large Language Models (LLMs). Our automatic and human evaluation of Arabic-aware LLMs benchmarks current capabilities in translating across diverse Arabic dialects and sub-dialects, while exposing significant persistent challenges.

Method: Integration of LoRA-based fine-tuning of Qwen3-8B large language model with Retrieval-Augmented Generation (RAG) framework, tested on binary classification task using annotated data from British National Corpus.

Result: LoRA-fine-tuned Qwen3-8B model significantly outperformed both native Qwen3-MAX model and theory-only RAG system in identifying ditransitive constructions.

Conclusion: Fine-tuning shifts model judgment from surface-form pattern matching to more semantically grounded understanding, demonstrating the effectiveness of LoRA+RAG approach for construction identification tasks.

Abstract: This study investigates the automatic identification of the English ditransitive construction by integrating LoRA-based fine-tuning of a large language model with a Retrieval-Augmented Generation (RAG) framework.A binary classification task was conducted on annotated data from the British National Corpus. Results demonstrate that a LoRA-fine-tuned Qwen3-8B model significantly outperformed both a native Qwen3-MAX model and a theory-only RAG system. Detailed error analysis reveals that fine-tuning shifts the model’s judgment from a surface-form pattern matching towards a more semantically grounded understanding based.

Method: CORE-T enriches tables with LLM-generated purpose metadata and pre-computes a lightweight table-compatibility cache. At inference: DR returns top-K candidates, a single LLM call selects coherent joinable subsets, and an additive adjustment restores strongly compatible tables.

Result: Improves table-selection F1 by up to 22.7 points while retrieving up to 42% fewer tables. Improves multi-table execution accuracy by up to 5.0 points on Bird and 6.9 points on MMQA. Uses 4-5x fewer tokens than LLM-intensive baselines.

Conclusion: CORE-T provides an effective, scalable solution for multi-table text-to-SQL in open-book settings with heterogeneous table collections, balancing recall and precision while reducing computational overhead.

Abstract: Realistic text-to-SQL workflows often require joining multiple tables. As a result, accurately retrieving the relevant set of tables becomes a key bottleneck for end-to-end performance. We study an open-book setting where queries must be answered over large, heterogeneous table collections pooled from many sources, without clean scoping signals such as database identifiers. Here, dense retrieval (DR) achieves high recall but returns many distractors, while join-aware alternatives often rely on extra assumptions and/or incur high inference overhead. We propose CORE-T, a scalable, training-free framework that enriches tables with LLM-generated purpose metadata and pre-computes a lightweight table-compatibility cache. At inference time, DR returns top-K candidates; a single LLM call selects a coherent, joinable subset, and a simple additive adjustment step restores strongly compatible tables. Across Bird, Spider, and MMQA, CORE-T improves table-selection F1 by up to 22.7 points while retrieving up to 42% fewer tables, improving multi-table execution accuracy by up to 5.0 points on Bird and 6.9 points on MMQA, and using 4-5x fewer tokens than LLM-intensive baselines.

Method: Introduces a conversational agent that interleaves search and reasoning across dialogue turns, using reinforcement learning with tailored rewards to learn exploratory and adaptive behaviors for evolving user goals.

Result: The method surpasses several existing strong baselines across four widely used conversational benchmarks, demonstrating effectiveness in handling multi-turn interactions.

Conclusion: Interleaving search and reasoning across turns with RL training enables more effective conversational agents that can adapt to evolving user intent in multi-turn dialogues.

Abstract: Large Language Models (LLMs) have become a popular interface for human-AI interaction, supporting information seeking and task assistance through natural, multi-turn dialogue. To respond to users within multi-turn dialogues, the context-dependent user intent evolves across interactions, requiring contextual interpretation, query reformulation, and dynamic coordination between retrieval and generation. Existing studies usually follow static rewrite, retrieve, and generate pipelines, which optimize different procedures separately and overlook the mixed-initiative action optimization simultaneously. Although the recent developments in deep search agents demonstrate the effectiveness in jointly optimizing retrieval and generation via reasoning, these approaches focus on single-turn scenarios, which might lack the ability to handle multi-turn interactions. We introduce a conversational agent that interleaves search and reasoning across turns, enabling exploratory and adaptive behaviors learned through reinforcement learning (RL) training with tailored rewards towards evolving user goals. The experimental results across four widely used conversational benchmarks demonstrate the effectiveness of our methods by surpassing several existing strong baselines.

Method: Proposes adversarial alignment framework with three stages: continued pre-training, instruction fine-tuning, and adversarial training. The adversarial training uses three components: Attacker (generates controversial queries), Actor (generates value-consistent responses), and Critic (filters and ensures response quality).

Result: Trained VC-LLM model that outperforms existing mainstream models in both Chinese and English tests on a bilingual evaluation dataset, demonstrating effectiveness of the method.

Conclusion: The adversarial alignment framework successfully enhances value consistency in LLMs for sensitive domains, with VC-LLM showing superior performance in handling controversial topics while maintaining value alignment across languages.

Abstract: With the wide application of large language models (LLMs), the problems of bias and value inconsistency in sensitive domains have gradually emerged, especially in terms of race, society and politics. In this paper, we propose an adversarial alignment framework, which enhances the value consistency of the model in sensitive domains through continued pre-training, instruction fine-tuning and adversarial training. In adversarial training, we use the Attacker to generate controversial queries, the Actor to generate responses with value consistency, and the Critic to filter and ensure response quality. Furthermore, we train a Value-Consistent Large Language Model, VC-LLM, for sensitive domains, and construct a bilingual evaluation dataset in Chinese and English. The experimental results show that VC-LLM performs better than the existing mainstream models in both Chinese and English tests, verifying the effectiveness of the method. Warning: This paper contains examples of LLMs that are offensive or harmful in nature.

Method: SPTS uses two component-specific strategies: Partial Attention Probing (PAP) for multi-head attention selects informative tokens via partial forward attention computation, and Low-rank Transformation Probing (LTP) for feed forward network uses a low-rank proxy network to predict token transformations. Multi-Stage Delayed Pruning (MSDP) reallocates skipping budget and progressively prunes redundant tokens across layers.

Result: Achieves up to 2.46× speedup for prefilling and 2.29× speedup for end-to-end generation while maintaining state-of-the-art model performance.

Conclusion: SPTS effectively addresses limitations of existing token skipping methods, providing a training-free solution for efficient long-context LLM inference with significant speed improvements and maintained accuracy.

Abstract: Long-context inference enhances the reasoning capability of Large Language Models (LLMs) while incurring significant computational overhead. Token-oriented methods, such as pruning and skipping, have shown promise in reducing inference latency, but still suffer from inherently limited acceleration potential, outdated proxy signals, and redundancy interference, thus yielding suboptimal speed-accuracy trade-offs. To address these challenges, we propose SPTS (Self-Predictive Token Skipping), a training-free framework for efficient long-context LLM inference. Specifically, motivated by the thought of probing the influence of targeted skipping layers, we design two component-specific strategies for selective token skipping: Partial Attention Probing (PAP) for multi-head attention, which selects informative tokens by performing partial forward attention computation, and Low-rank Transformation Probing (LTP) for feed forward network, which constructs a low-rank proxy network to predict token transformations. Furthermore, a Multi-Stage Delayed Pruning (MSDP) strategy reallocates the skipping budget and progressively prunes redundant tokens across layers. Extensive experiments demonstrate the effectiveness of our method, achieving up to 2.46$\times$ and 2.29$\times$ speedups for prefilling and end-to-end generation, respectively, while maintaining state-of-the-art model performance. The source code will be publicly available upon paper acceptance.

Method: Formulates patient message triage as pairwise inference problem; creates PMR-Bench dataset with 1569 messages and 2000+ test pairs; develops automated annotation strategy; trains UrgentReward (Bradley-Terry) and UrgentSFT (next token prediction) models.

Result: UrgentSFT achieves top performance on PMR-Bench; both models show significant improvements over off-the-shelf models (15-16 point boost for 8B models); UrgentReward excels in low-resource settings.

Conclusion: Pairwise LLM approach effectively addresses medical triage for outpatient messages; PMR-Bench enables scalable training; specialized models significantly outperform general-purpose LLMs in medical urgency assessment.

Abstract: Medical triage is the task of allocating medical resources and prioritizing patients based on medical need. This paper introduces the first large-scale public dataset for studying medical triage in the context of asynchronous outpatient portal messages. Our novel task formulation views patient message triage as a pairwise inference problem, where we train LLMs to choose `“which message is more medically urgent” in a head-to-head tournament-style re-sort of a physician’s inbox. Our novel benchmark PMR-Bench contains 1569 unique messages and 2,000+ high-quality test pairs for pairwise medical urgency assessment alongside a scalable training data generation pipeline. PMR-Bench includes samples that contain both unstructured patient-written messages alongside real electronic health record (EHR) data, emulating a real-world medical triage scenario. We develop a novel automated data annotation strategy to provide LLMs with in-domain guidance on this task. The resulting data is used to train two model classes, UrgentReward and UrgentSFT, leveraging Bradley-Terry and next token prediction objective, respectively to perform pairwise urgency classification. We find that UrgentSFT achieves top performance on PMR-Bench, with UrgentReward showing distinct advantages in low-resource settings. For example, UrgentSFT-8B and UrgentReward-8B provide a 15- and 16-point boost, respectively, on inbox sorting metrics over off-the-shelf 8B models. Paper resources can be found at https://tinyurl.com/Patient-Message-Triage

Method: The OpenExempt Framework uses expert-crafted symbolic representations of U.S. Bankruptcy Code statutes to dynamically generate a large space of natural language reasoning tasks and their machine-computable solutions on demand, giving fine-grained control over task complexity and scope. The OpenExempt Benchmark contains 9,765 samples across nine evaluation suites designed to probe specific reasoning capabilities.

Result: Experiments on 13 diverse language models reveal sharp performance cliffs that emerge only under longer reasoning paths and in the presence of obfuscating statements, demonstrating the diagnostic value of the approach.

Conclusion: OpenExempt provides a powerful diagnostic tool for legal reasoning evaluation that can isolate specific failure modes and support research aimed at understanding and improving reasoning systems. The framework and benchmark are released publicly to advance research in this area.

Abstract: Reasoning benchmarks have played a crucial role in the progress of language models. Yet rigorous evaluation remains a significant challenge as static question-answer pairs provide only a snapshot of performance, compressing complex behavior into a single accuracy metric. This limitation is especially true in complex, rule-bound domains such as law, where existing benchmarks are costly to build and ill suited for isolating specific failure modes. To address this, we introduce OpenExempt, a framework and benchmark for diagnostic evaluation of legal reasoning. The OpenExempt Framework uses expert-crafted symbolic representations of U.S. Bankruptcy Code statutes to dynamically generate a large space of natural language reasoning tasks and their machine-computable solutions on demand. This gives users fine-grained control over task complexity and scope, allowing individual reasoning skills to be probed in isolation. Using this system, we construct the OpenExempt Benchmark, a diagnostic benchmark for legal reasoning with 9,765 samples across nine evaluation suites designed to carefully probe model capabilities. Experiments on 13 diverse language models reveal sharp performance cliffs that emerge only under longer reasoning paths and in the presence of obfuscating statements. We release the framework and benchmark publicly to support research aimed at understanding and improving the next generation of reasoning systems.

Method: Introduces Mr Dre evaluation suite with: (1) unified long-form report evaluation protocol covering comprehensiveness, factuality, and presentation, and (2) human-verified feedback simulation pipeline for multi-turn revision. Analyzes five diverse DRAs and tests inference-time fixes like prompt engineering and dedicated revision sub-agents.

Result: DRAs can address most user feedback but regress on 16-27% of previously covered content and citation quality. Over multiple revision turns, even best-performing agents disrupt content outside feedback scope and fail to preserve earlier edits. Issues persist despite prompt engineering and dedicated revision sub-agents.

Conclusion: Multi-turn report revision reveals critical limitations in current DRAs, with significant headroom for improvement. The problems are not easily fixable with current inference-time solutions, highlighting the need for more robust approaches to iterative report refinement.

Abstract: Existing benchmarks for Deep Research Agents (DRAs) treat report generation as a single-shot writing task, which fundamentally diverges from how human researchers iteratively draft and revise reports via self-reflection or peer feedback. Whether DRAs can reliably revise reports with user feedback remains unexplored. We introduce Mr Dre, an evaluation suite that establishes multi-turn report revision as a new evaluation axis for DRAs. Mr Dre consists of (1) a unified long-form report evaluation protocol spanning comprehensiveness, factuality, and presentation, and (2) a human-verified feedback simulation pipeline for multi-turn revision. Our analysis of five diverse DRAs reveals a critical limitation: while agents can address most user feedback, they also regress on 16-27% of previously covered content and citation quality. Over multiple revision turns, even the best-performing agents leave significant headroom, as they continue to disrupt content outside the feedback’s scope and fail to preserve earlier edits. We further show that these issues are not easily resolvable through inference-time fixes such as prompt engineering and a dedicated sub-agent for report revision.

Method: Reformulate diffusion-style training into structured multi-group prediction, extending AR factorization to arbitrary token groups and generation orders. Implement via two-stream attention architecture and progressive adaptation strategy that transitions pretrained AR models toward any-order prediction.

Result: A3 outperforms diffusion-based models on question answering, commonsense reasoning, and story infilling tasks while maintaining flexible decoding capabilities.

Conclusion: A3 provides a unified approach for flexible, efficient language modeling that preserves AR’s strengths while gaining diffusion models’ flexibility for parallel and bidirectional generation.

Abstract: Diffusion language models enable any-order generation and bidirectional conditioning, offering appealing flexibility for tasks such as infilling, rewriting, and self-correction. However, their formulation-predicting one part of a sequence from another within a single-step dependency-limits modeling depth and often yields lower sample quality and stability than autoregressive (AR) models. To address this, we revisit autoregressive modeling as a foundation and reformulate diffusion-style training into a structured multi-group prediction process. We propose Any-order Any-subset Autoregressive modeling (A3), a generalized framework that extends the standard AR factorization to arbitrary token groups and generation orders. A3 preserves the probabilistic rigor and multi-layer dependency modeling of AR while inheriting diffusion models’ flexibility for parallel and bidirectional generation. We implement A3 through a two-stream attention architecture and a progressive adaptation strategy that transitions pretrained AR models toward any-order prediction. Experiments on question answering, commonsense reasoning, and story infilling demonstrate that A3 outperforms diffusion-based models while maintaining flexible decoding. This work offers a unified approach for a flexible, efficient, and novel language modeling paradigm.

Method: Three main components: 1) Created a labeled dataset of 843,000 concept pairs (synonymy, antonymy, co-hyponymy) using Gemini 2.5-Flash LLM augmentation and human-curated dictionary verification. 2) Developed a specialized three-way semantic relation discriminator achieving 90% macro-F1. 3) Introduced a novel soft-to-hard clustering algorithm with topology-aware two-stage expansion-pruning and topological voting to prevent semantic drift and resolve polysemy.

Result: Processed 15 million lexical items, evaluated 520 million potential relationships, and generated 2.9 million high-precision semantic clusters. The system reliably assigns each term to exactly one semantically coherent cluster, preventing erroneous transitive chains like “hot -> spicy -> pain -> depression”.

Conclusion: The system successfully addresses the synonym-antonym discrimination problem in neural embeddings, enabling high-precision semantic search and retrieval-augmented generation, especially beneficial for morphologically rich and low-resource languages with limited existing resources.

Abstract: Neural embeddings have a notorious blind spot: they can’t reliably tell synonyms apart from antonyms. Consequently, increasing similarity thresholds often fails to prevent opposites from being grouped together. We’ve built a large-scale semantic clustering system specifically designed to tackle this problem head on. Our pipeline chews through 15 million lexical items, evaluates a massive 520 million potential relationships, and ultimately generates 2.9 million high-precision semantic clusters. The system makes three primary contributions. First, we introduce a labeled dataset of 843,000 concept pairs spanning synonymy, antonymy, and co-hyponymy, constructed via Gemini 2.5-Flash LLM augmentation and verified using human-curated dictionary resources. Second, we propose a specialized three-way semantic relation discriminator that achieves 90% macro-F1, enabling robust disambiguation beyond raw embedding similarity. Third, we introduce a novel soft-to-hard clustering algorithm that mitigates semantic drift preventing erroneous transitive chains (e.g., hot -> spicy -> pain -> depression) while simultaneously resolving polysemy. Our approach employs a topology-aware two-stage expansion-pruning procedure with topological voting, ensuring that each term is assigned to exactly one semantically coherent cluster. The resulting resource enables high-precision semantic search and retrieval-augmented generation, particularly for morphologically rich and low-resource languages where existing synonym databases remain sparse.

Method: Three-phase hybrid approach: 1) FastText embeddings with Agglomerative Clustering for semantic clusters, 2) Gemini 2.5-Flash for automated semantic relationship classification, 3) integration with curated dictionary sources.

Result: Created 843,000 unique Turkish semantic pairs across synonyms, antonyms, co-hyponyms (10x scale increase over existing resources at $65 cost). Validation shows 90% top-1 retrieval accuracy for embeddings and 90% F1-macro for classification.

Conclusion: Scalable protocol successfully addresses Turkish NLP data scarcity, demonstrates applicability to other low-resource languages, with publicly released dataset and models.

Abstract: We present a hybrid methodology for generating large-scale semantic relationship datasets in low-resource languages, demonstrated through a comprehensive Turkish semantic relations corpus. Our approach integrates three phases: (1) FastText embeddings with Agglomerative Clustering to identify semantic clusters, (2) Gemini 2.5-Flash for automated semantic relationship classification, and (3) integration with curated dictionary sources. The resulting dataset comprises 843,000 unique Turkish semantic pairs across three relationship types (synonyms, antonyms, co-hyponyms) representing a 10x scale increase over existing resources at minimal cost ($65). We validate the dataset through two downstream tasks: an embedding model achieving 90% top-1 retrieval accuracy and a classification model attaining 90% F1-macro. Our scalable protocol addresses critical data scarcity in Turkish NLP and demonstrates applicability to other low-resource languages. We publicly release the dataset and models.

Method: Proposes reframing tokenization as a core modeling decision with a context-aware framework that integrates tokenizer and model co-design. Emphasizes standardized evaluation and transparent reporting to make tokenization choices accountable and comparable.

Result: The paper argues for treating tokenization as a design problem guided by linguistic, domain, and deployment considerations rather than a technical afterthought.

Conclusion: By making tokenization a core design consideration through context-aware frameworks and standardized evaluation, language technologies can become fairer, more efficient, and more adaptable.

Abstract: Tokenization underlies every large language model, yet it remains an under-theorized and inconsistently designed component. Common subword approaches such as Byte Pair Encoding (BPE) offer scalability but often misalign with linguistic structure, amplify bias, and waste capacity across languages and domains. This paper reframes tokenization as a core modeling decision rather than a preprocessing step. We argue for a context-aware framework that integrates tokenizer and model co-design, guided by linguistic, domain, and deployment considerations. Standardized evaluation and transparent reporting are essential to make tokenization choices accountable and comparable. Treating tokenization as a core design problem, not a technical afterthought, can yield language technologies that are fairer, more efficient, and more adaptable.

Method: Structured overview categorizing unlearning methods into data-centric, parameter-centric, architecture-centric, hybrid, and other strategies.

Result: Comprehensive review of evaluation ecosystem including benchmarks, metrics, and datasets for measuring forgetting effectiveness, knowledge retention, and robustness.

Conclusion: Identifies key challenges like scalable efficiency, formal guarantees, cross-language/multimodal unlearning, and adversarial relearning robustness; serves as roadmap for responsible unlearning techniques.

Abstract: Large language models (LLMs) have achieved remarkable success across natural language processing tasks, yet their widespread deployment raises pressing concerns around privacy, copyright, security, and bias. Machine unlearning has emerged as a promising paradigm for selectively removing knowledge or data from trained models without full retraining. In this survey, we provide a structured overview of unlearning methods for LLMs, categorizing existing approaches into data-centric, parameter-centric, architecture-centric, hybrid, and other strategies. We also review the evaluation ecosystem, including benchmarks, metrics, and datasets designed to measure forgetting effectiveness, knowledge retention, and robustness. Finally, we outline key challenges and open problems, such as scalable efficiency, formal guarantees, cross-language and multimodal unlearning, and robustness against adversarial relearning. By synthesizing current progress and highlighting open directions, this paper aims to serve as a roadmap for developing reliable and responsible unlearning techniques in large language models.

Method: Train lightweight probes on LLM hidden states to predict labels during the same forward pass used for generation. Use a two-stage aggregator: (1) summarize tokens within each layer, (2) aggregate across layer summaries to form a single representation for classification. Three implementations: direct pooling, 100K-parameter scoring-attention gate, and downcast multi-head self-attention (MHA) probe with up to 35M parameters.

Result: Probes outperform logit-only reuse methods (like MULI) and are competitive with substantially larger task-specific baselines on safety and sentiment benchmarks. They preserve near-serving latency and avoid the VRAM and latency costs of separate guard-model pipelines.

Conclusion: Lightweight probes on LLM hidden states provide an efficient alternative to separate classification models, enabling safety and other classification tasks without the overhead of additional models while maintaining competitive performance.

Abstract: Production LLM systems often rely on separate models for safety and other classification-heavy steps, increasing latency, VRAM footprint, and operational complexity. We instead reuse computation already paid for by the serving LLM: we train lightweight probes on its hidden states and predict labels in the same forward pass used for generation. We frame classification as representation selection over the full token-layer hidden-state tensor, rather than committing to a fixed token or fixed layer (e.g., first-token logits or final-layer pooling). To implement this, we introduce a two-stage aggregator that (i) summarizes tokens within each layer and (ii) aggregates across layer summaries to form a single representation for classification. We instantiate this template with direct pooling, a 100K-parameter scoring-attention gate, and a downcast multi-head self-attention (MHA) probe with up to 35M trainable parameters. Across safety and sentiment benchmarks our probes improve over logit-only reuse (e.g., MULI) and are competitive with substantially larger task-specific baselines, while preserving near-serving latency and avoiding the VRAM and latency costs of a separate guard-model pipeline.

Method: Option injection augments MCQA with misleading directive options, creating OI-Bench with 3,000 questions across knowledge/reasoning/commonsense tasks and 16 directive types covering social compliance, bonus/threat framing, and instructional interference.

Result: Evaluation of 12 LLMs reveals substantial vulnerabilities and heterogeneous robustness; mitigation strategies from inference-time prompting to post-training alignment were investigated.

Conclusion: OI-Bench enables systematic assessment of LLM susceptibility to directive interference in choice-based interfaces, supporting more robust model evaluation.

Abstract: Benchmarking large language models (LLMs) is critical for understanding their capabilities, limitations, and robustness. In addition to interface artifacts, prior studies have shown that LLM decisions can be influenced by directive signals such as social cues, framing, and instructions. In this work, we introduce option injection, a benchmarking approach that augments the multiple-choice question answering (MCQA) interface with an additional option containing a misleading directive, leveraging standardized choice structure and scalable evaluation. We construct OI-Bench, a benchmark of 3,000 questions spanning knowledge, reasoning, and commonsense tasks, with 16 directive types covering social compliance, bonus framing, threat framing, and instructional interference. This setting combines manipulation of the choice interface with directive-based interference, enabling systematic assessment of model susceptibility. We evaluate 12 LLMs to analyze attack success rates, behavioral responses, and further investigate mitigation strategies ranging from inference-time prompting to post-training alignment. Experimental results reveal substantial vulnerabilities and heterogeneous robustness across models. OI-Bench is expected to support more systematic evaluation of LLM robustness to directive interference within choice-based interfaces.

Method: Interpretable end-to-end thematic discovery framework: clusters texts by semantic similarity, uses LLMs to generate human-interpretable theme labels. Evaluates themes against traditional topic modeling baselines using human judgments and LLM-based evaluator. Validates through downstream stance prediction and theme-guided retrieval tasks.

Result: Platform-level incentives reflected in thematic structure, stance alignment, and temporal responsiveness of climate narratives. Systematic differences found between paid climate messaging (Meta ads) and public climate discourse (Bluesky posts). Thematic prevalence shifts observed around major political events.

Conclusion: Framework supports comparative narrative analysis across heterogeneous communication environments. While focused on climate communication, approach generalizable to other domains. Platform incentives shape climate discourse structure and dynamics.

Abstract: Climate discourse online plays a crucial role in shaping public understanding of climate change and influencing political and policy outcomes. However, climate communication unfolds across structurally distinct platforms with fundamentally different incentive structures: paid advertising ecosystems incentivize targeted, strategic persuasion, while public social media platforms host largely organic, user-driven discourse. Existing computational studies typically analyze these environments in isolation, limiting our ability to distinguish institutional messaging from public expression. In this work, we present a comparative analysis of climate discourse across paid advertisements on Meta (previously known as Facebook) and public posts on Bluesky from July 2024 to September 2025. We introduce an interpretable, end-to-end thematic discovery and assignment framework that clusters texts by semantic similarity and leverages large language models (LLMs) to generate concise, human-interpretable theme labels. We evaluate the quality of the induced themes against traditional topic modeling baselines using both human judgments and an LLM-based evaluator, and further validate their semantic coherence through downstream stance prediction and theme-guided retrieval tasks. Applying the resulting themes, we characterize systematic differences between paid climate messaging and public climate discourse and examine how thematic prevalence shifts around major political events. Our findings show that platform-level incentives are reflected in the thematic structure, stance alignment, and temporal responsiveness of climate narratives. While our empirical analysis focuses on climate communication, the proposed framework is designed to support comparative narrative analysis across heterogeneous communication environments.

Method: Computational analysis of linguistic “dialectness” and audio quality proxies on training splits of widely used DA corpora; development of Arab Voices framework with unified access to 31 datasets across 14 dialects

Result: Found substantial heterogeneity in acoustic conditions and dialectal signal strength/consistency across datasets; established strong baselines for modern DA ASR through benchmarking of recent systems

Conclusion: Standardized characterization beyond coarse labels is needed; Arab Voices framework reduces fragmentation and supports reproducible evaluation for dialectal Arabic ASR

Abstract: Dialectal Arabic (DA) speech data vary widely in domain coverage, dialect labeling practices, and recording conditions, complicating cross-dataset comparison and model evaluation. To characterize this landscape, we conduct a computational analysis of linguistic ``dialectness’’ alongside objective proxies of audio quality on the training splits of widely used DA corpora. We find substantial heterogeneity both in acoustic conditions and in the strength and consistency of dialectal signals across datasets, underscoring the need for standardized characterization beyond coarse labels. To reduce fragmentation and support reproducible evaluation, we introduce Arab Voices, a standardized framework for DA ASR. Arab Voices provides unified access to 31 datasets spanning 14 dialects, with harmonized metadata and evaluation utilities. We further benchmark a range of recent ASR systems, establishing strong baselines for modern DA ASR.

Method: Analyzed tokenizers of ten popular LMs to understand designs and per-language tokenization premiums. Proposed post-hoc vocabulary expansion by adding tokens that coalesce multi-token characters into single tokens. Applied methodology to 12 low-resource languages.

Result: Demonstrated that original and compressed inputs often have similar last hidden states when run through the Llama 3.2 1B model, showing the method’s effectiveness in reducing tokenization premiums while maintaining model behavior.

Conclusion: Post-hoc vocabulary expansion can effectively reduce tokenization premiums for low-resource languages, potentially lowering computational costs and improving context window utilization without retraining models.

Abstract: Relative to English, low-resource languages suffer from substantial tokenization premiums in modern LMs, meaning that it generally requires several times as many tokens to encode a sentence in a low-resource language than to encode the analogous sentence in English. This tokenization premium results in increased API and energy costs and reduced effective context windows for these languages. In this paper we analyze the tokenizers of ten popular LMs to better understand their designs and per-language tokenization premiums. We also propose a mechanism to reduce tokenization premiums in pre-trained models, by post-hoc additions to the token vocabulary that coalesce multi-token characters into single tokens. We apply this methodology to 12 low-resource languages, demonstrating that the original and compressed inputs often have similar last hidden states when run through the Llama 3.2 1B model.

Method: RegCheck is a modular LLM-assisted tool that allows users to determine which features to compare between registrations and papers, presents relevant text for each feature to facilitate human discrepancy judgments, and generates shareable reports with unique IDs.

Result: The paper presents RegCheck as a working tool designed to be adaptable across scientific domains and formats, with an example use case demonstrating its potential as extensible infrastructure for reproducible science.

Conclusion: RegCheck represents a promising AI-assisted approach to making study registration checking more feasible and effective, supporting scientific transparency while maintaining essential human oversight in the verification process.

Abstract: Across the social and medical sciences, researchers recognize that specifying planned research activities (i.e., ‘registration’) prior to the commencement of research has benefits for both the transparency and rigour of science. Despite this, evidence suggests that study registrations frequently go unexamined, minimizing their effectiveness. In a way this is no surprise: manually checking registrations against papers is labour- and time-intensive, requiring careful reading across formats and expertise across domains. The advent of AI unlocks new possibilities in facilitating this activity. We present RegCheck, a modular LLM-assisted tool designed to help researchers, reviewers, and editors from across scientific disciplines compare study registrations with their corresponding papers. Importantly, RegCheck keeps human expertise and judgement in the loop by (i) ensuring that users are the ones who determine which features should be compared, and (ii) presenting the most relevant text associated with each feature to the user, facilitating (rather than replacing) human discrepancy judgements. RegCheck also generates shareable reports with unique RegCheck IDs, enabling them to be easily shared and verified by other users. RegCheck is designed to be adaptable across scientific domains, as well as registration and publication formats. In this paper we provide an overview of the motivation, workflow, and design principles of RegCheck, and we discuss its potential as an extensible infrastructure for reproducible science with an example use case.

Method: Developed AfroScope framework with AfroScope-Data (dataset covering 713 African languages) and AfroScope-Models (LID models). Introduced hierarchical classification approach using Mirror-Serengeti embedding model for 29 closely related languages to improve discrimination.

Result: Hierarchical classification improved macro F1 by 4.55 points on confusable language subset compared to best base model. Analyzed cross-linguistic transfer and domain effects for robust African LID systems.

Conclusion: AfroScope enables large-scale measurement of Africa’s linguistic landscape in digital text. Framework and models are publicly released as enabling technology for African language processing.

Abstract: Language Identification (LID) is the task of determining the language of a given text and is a fundamental preprocessing step that affects the reliability of downstream NLP applications. While recent work has expanded LID coverage for African languages, existing approaches remain limited in (i) the number of supported languages and (ii) their ability to make fine-grained distinctions among closely related varieties. We introduce AfroScope, a unified framework for African LID that includes AfroScope-Data, a dataset covering 713 African languages, and AfroScope-Models, a suite of strong LID models with broad language coverage. To better distinguish highly confusable languages, we propose a hierarchical classification approach that leverages Mirror-Serengeti, a specialized embedding model targeting 29 closely related or geographically proximate languages. This approach improves macro F1 by 4.55 on this confusable subset compared to our best base model. Finally, we analyze cross linguistic transfer and domain effects, offering guidance for building robust African LID systems. We position African LID as an enabling technology for large scale measurement of Africas linguistic landscape in digital text and release AfroScope-Data and AfroScope-Models publicly.

Method: The paper introduces “sockpuppetting” - a simple jailbreaking technique that inserts an acceptance sequence (e.g., “Sure, here is how to…”) at the start of a model’s output and allows it to complete the response. This requires only a single line of code and no optimization. The authors also explore a hybrid approach that optimizes adversarial suffixes within the assistant message block rather than the user prompt.

Result: Sockpuppetting achieves up to 80% higher attack success rate (ASR) than GCG on Qwen3-8B in per-prompt comparisons. The hybrid approach increases ASR by 64% over GCG on Llama-3.1-8B in a prompt-agnostic setting. Both methods demonstrate significantly better performance than existing approaches with minimal computational requirements.

Conclusion: Sockpuppetting establishes itself as an effective low-cost attack method accessible to unsophisticated adversaries, highlighting the urgent need for defenses against output-prefix injection in open-weight models. The simplicity and effectiveness of this approach reveal a significant vulnerability in current LLM security frameworks.

Abstract: As open-weight large language models (LLMs) increase in capabilities, safeguarding them against malicious prompts and understanding possible attack vectors becomes ever more important. While automated jailbreaking methods like GCG [Zou et al., 2023] remain effective, they often require substantial computational resources and specific expertise. We introduce “sockpuppetting’’, a simple method for jailbreaking open-weight LLMs by inserting an acceptance sequence (e.g., “Sure, here is how to…’’) at the start of a model’s output and allowing it to complete the response. Requiring only a single line of code and no optimization, sockpuppetting achieves up to 80% higher attack success rate (ASR) than GCG on Qwen3-8B in per-prompt comparisons. We also explore a hybrid approach that optimizes the adversarial suffix within the assistant message block rather than the user prompt, increasing ASR by 64% over GCG on Llama-3.1-8B in a prompt-agnostic setting. The results establish sockpuppetting as an effective low-cost attack accessible to unsophisticated adversaries, highlighting the need for defences against output-prefix injection in open-weight models.

Method: 1) Incorporates inter-step attention to analyze semantic correlations across reasoning steps; 2) Introduces hidden confidence mechanism to retain historical confidence information for long-horizon responses; 3) Combines stepwise confidence with historical confidence for more accurate overall uncertainty estimation.

Result: Outperforms state-of-the-art methods on GAOKAO math benchmark and CLadder causal reasoning dataset using mainstream open-source LLMs. Achieves superior balance between predictive quality and calibration with strong performance on Negative Log-Likelihood and Expected Calibration Error metrics.

Conclusion: The proposed method effectively addresses the temporal confidence spread problem in multi-step reasoning, providing more reliable uncertainty estimates that can help prevent misleading hallucinations in LLM applications.

Abstract: As reasoning modules, such as the chain-of-thought mechanism, are applied to large language models, they achieve strong performance on various tasks such as answering common-sense questions and solving math problems. The main challenge now is to assess the uncertainty of answers, which can help prevent misleading or serious hallucinations for users. Although current methods analyze long reasoning sequences by filtering unrelated tokens and examining potential connections between nearby tokens or sentences, the temporal spread of confidence is often overlooked. This oversight can lead to inflated overall confidence, even when earlier steps exhibit very low confidence. To address this issue, we propose a novel method that incorporates inter-step attention to analyze semantic correlations across steps. For handling long-horizon responses, we introduce a hidden confidence mechanism to retain historical confidence information, which is then combined with stepwise confidence to produce a more accurate overall estimate. We evaluate our method on the GAOKAO math benchmark and the CLadder causal reasoning dataset using mainstream open-source large language models. Our approach is shown to outperform state-of-the-art methods by achieving a superior balance between predictive quality and calibration, demonstrated by strong performance on both Negative Log-Likelihood and Expected Calibration Error.

Method: Propose using Signal Temporal Logic (STL) to characterize stepwise confidence signals. Use discriminative STL mining to discover temporal formulas that distinguish confidence signals of correct vs incorrect responses. Develop confidence estimation approach that informs STL blocks with parameter hypernetworks.

Result: STL patterns generalize across tasks, while numeric parameters show sensitivity to individual questions. Experiments on multiple reasoning tasks demonstrate that the proposed confidence scores are more calibrated than baseline methods.

Conclusion: Stepwise confidence analysis using Signal Temporal Logic provides more nuanced and calibrated confidence estimation for LLMs performing complex multi-step reasoning tasks, overcoming limitations of single-scalar approaches.

Abstract: Large Language Models (LLMs) increasingly rely on long-form, multi-step reasoning to solve complex tasks such as mathematical problem solving and scientific question answering. Despite strong performance, existing confidence estimation methods typically reduce an entire reasoning process to a single scalar score, ignoring how confidence evolves throughout the generation. As a result, these methods are often sensitive to superficial factors such as response length or verbosity, and struggle to distinguish correct reasoning from confidently stated errors. We propose to characterize the stepwise confidence signal using Signal Temporal Logic (STL). Using a discriminative STL mining procedure, we discover temporal formulas that distinguish confidence signals of correct and incorrect responses. Our analysis found that the STL patterns generalize across tasks, and numeric parameters exhibit sensitivity to individual questions. Based on these insights, we develop a confidence estimation approach that informs STL blocks with parameter hypernetworks. Experiments on multiple reasoning tasks show our confidence scores are more calibrated than the baselines.

Method: Collected unstructured interviews from 65 T2D patients aged 65+, used LLMs with retrieval-augmented generation to create qualitative summaries and structured SDOH ratings. Applied traditional ML models (Ridge, Lasso, Random Forest, XGBoost) using SDOH ratings alone and combined with biomarkers. Also evaluated LLMs directly predicting diabetes control from interview text.

Result: LLMs achieved 60% accuracy in predicting diabetes control levels (low, medium, high) directly from interview text with A1C values redacted. Structured SDOH ratings were successfully integrated into conventional risk prediction workflows.

Conclusion: LLMs can effectively translate unstructured SDOH-related data into structured insights, offering a scalable approach to augment clinical risk models and decision-making for diabetes management.

Abstract: Social determinants of health (SDOH) play a critical role in Type 2 Diabetes (T2D) management but are often absent from electronic health records and risk prediction models. Most individual-level SDOH data is collected through structured screening tools, which lack the flexibility to capture the complexity of patient experiences and unique needs of a clinic’s population. This study explores the use of large language models (LLMs) to extract structured SDOH information from unstructured patient life stories and evaluate the predictive value of both the extracted features and the narratives themselves for assessing diabetes control. We collected unstructured interviews from 65 T2D patients aged 65 and older, focused on their lived experiences, social context, and diabetes management. These narratives were analyzed using LLMs with retrieval-augmented generation to produce concise, actionable qualitative summaries for clinical interpretation and structured quantitative SDOH ratings for risk prediction modeling. The structured SDOH ratings were used independently and in combination with traditional laboratory biomarkers as inputs to linear and tree-based machine learning models (Ridge, Lasso, Random Forest, and XGBoost) to demonstrate how unstructured narrative data can be applied in conventional risk prediction workflows. Finally, we evaluated several LLMs on their ability to predict a patient’s level of diabetes control (low, medium, high) directly from interview text with A1C values redacted. LLMs achieved 60% accuracy in predicting diabetes control levels from interview text. This work demonstrates how LLMs can translate unstructured SDOH-related data into structured insights, offering a scalable approach to augment clinical risk models and decision-making.

Method: Created a benchmark with four types of problems: factual knowledge questions, seen construction problems from public sources, hand-crafted unseen problems with multiple constraints, and systematically generated unseen problems via Arden’s theorem. Evaluated multiple prompting strategies (direct, Chain-of-Thought, Tree-of-Thought) and a three-stage hint protocol for error correction.

Result: Models achieved perfect accuracy on factual questions and 84-90% on seen tasks, but accuracy dropped sharply by 30-64% on unseen problems. Failures stemmed from systematic misinterpretation of constraints, incorrect handling of Kleene-star semantics, and failure to preserve global consistency. The hint protocol corrected shallow errors but couldn’t fix globally inconsistent or structurally flawed automata.

Conclusion: There’s a fundamental gap between LLMs’ ability to generate syntactically plausible DFAs and their capacity for semantically correct formal reasoning. Errors persist regardless of prompting approach, suggesting current LLMs lack genuine symbolic reasoning capabilities for formal language tasks.

Abstract: Large language models (LLMs) have demonstrated strong performance on formal language tasks, yet whether this reflects genuine symbolic reasoning or pattern matching on familiar constructions remains unclear. We introduce a benchmark for deterministic finite automata (DFA) construction from regular languages, comprising factual knowledge questions, seen construction problems from public sources, and two types of unseen problems: hand-crafted instances with multiple interacting constraints and systematically generated problems via Arden’s theorem. Models achieve perfect accuracy on factual questions and 84-90% on seen tasks. However, accuracy drops sharply on unseen problems (by 30-64%), with failures stemming from systematic misinterpretation of language constraints, incorrect handling of Kleene-star semantics, and a failure to preserve global consistency. We evaluate a three-stage hint protocol that enables correction of shallow errors but does not reliably resolve globally inconsistent or structurally flawed automata. Our analysis across multiple prompting strategies (direct, Chain-of-Thought, Tree-of-Thought) reveals that errors persist regardless of prompting approach, exposing a fundamental gap between LLMs’ ability to generate syntactically plausible DFAs and their capacity for semantically correct formal reasoning.

Method: Evaluated 11 models across 4 datasets spanning mathematical, legal, and medical reasoning domains. Used personas representing four expertise levels per domain to test how source credibility affects model susceptibility to incorrect/misleading endorsements.

Result: Models show increasing susceptibility to incorrect endorsements as source expertise increases. Higher-authority sources cause not only accuracy degradation but also increased confidence in wrong answers. The authority bias is mechanistically encoded within models.

Conclusion: Language models exhibit systematic authority bias that can be exploited. However, models can be steered away from this bias, improving performance even when experts give misleading endorsements.

Abstract: Prior research demonstrates that performance of language models on reasoning tasks can be influenced by suggestions, hints and endorsements. However, the influence of endorsement source credibility remains underexplored. We investigate whether language models exhibit systematic bias based on the perceived expertise of the provider of the endorsement. Across 4 datasets spanning mathematical, legal, and medical reasoning, we evaluate 11 models using personas representing four expertise levels per domain. Our results reveal that models are increasingly susceptible to incorrect/misleading endorsements as source expertise increases, with higher-authority sources inducing not only accuracy degradation but also increased confidence in wrong answers. We also show that this authority bias is mechanistically encoded within the model and a model can be steered away from the bias, thereby improving its performance even when an expert gives a misleading endorsement.

Method: 1) Created the first multilingual OSLD benchmark by rearranging existing datasets and collecting new data from news domain (960K samples, 12 languages). 2) Proposed a novel multi-stage framework that integrates multiple stages to continuously discover and learn new classes.

Result: Established the MOSLD benchmark and evaluated several language models including their own framework, providing reference results for future work. The benchmark and framework are publicly released.

Conclusion: This work introduces the first multilingual benchmark for open-set learning and discovery in text categorization, addressing a gap in the field. The proposed framework and benchmark provide a foundation for future research in this challenging area of machine learning.

Abstract: Open-set learning and discovery (OSLD) is a challenging machine learning task in which samples from new (unknown) classes can appear at test time. It can be seen as a generalization of zero-shot learning, where the new classes are not known a priori, hence involving the active discovery of new classes. While zero-shot learning has been extensively studied in text classification, especially with the emergence of pre-trained language models, open-set learning and discovery is a comparatively new setup for the text domain. To this end, we introduce the first multilingual open-set learning and discovery (MOSLD) benchmark for text categorization by topic, comprising 960K data samples across 12 languages. To construct the benchmark, we (i) rearrange existing datasets and (ii) collect new data samples from the news domain. Moreover, we propose a novel framework for the OSLD task, which integrates multiple stages to continuously discover and learn new classes. We evaluate several language models, including our own, to obtain results that can be used as reference for future work. We release our benchmark at https://github.com/Adriana19Valentina/MOSLD-Bench.

Method: Developed PhysicsSolutionAgent (PSA) that generates physics explanation videos up to 6 minutes using Manim animations, with an assessment pipeline using 15 quantitative parameters and VLM feedback for iterative improvement.

Result: PSA achieved 100% video-completion rate with average automated score of 3.8/5 using GPT-5-mini, but qualitative analysis revealed systematic quality differences by problem difficulty/type and uncovered visual layout inconsistencies and interpretation errors.

Conclusion: The work exposes limitations in reliable Manim code generation and multimodal reasoning for visual explanations, highlighting the need for improved visual understanding, verification, and evaluation frameworks in educational systems.

Abstract: Explaining numerical physics problems often requires more than text-based solutions; clear visual reasoning can substantially improve conceptual understanding. While large language models (LLMs) demonstrate strong performance on many physics questions in textual form, their ability to generate long, high-quality visual explanations remains insufficiently explored. In this work, we introduce PhysicsSolutionAgent (PSA), an autonomous agent that generates physics-problem explanation videos of up to six minutes using Manim animations. To evaluate the generated videos, we design an assessment pipeline that performs automated checks across 15 quantitative parameters and incorporates feedback from a vision-language model (VLM) to iteratively improve video quality. We evaluate PSA on 32 videos spanning numerical and theoretical physics problems. Our results reveal systematic differences in video quality depending on problem difficulty and whether the task is numerical or theoretical. Using GPT-5-mini, PSA achieves a 100% video-completion rate with an average automated score of 3.8/5. However, qualitative analysis and human inspection uncover both minor and major issues, including visual layout inconsistencies and errors in how visual content is interpreted during feedback. These findings expose key limitations in reliable Manim code generation and highlight broader challenges in multimodal reasoning and evaluation for visual explanations of numerical physics problems. Our work underscores the need for improved visual understanding, verification, and evaluation frameworks in future multimodal educational systems

Method: Anonpsy reformulates de-identification as graph-guided semantic rewriting: (1) converts narratives into semantic graphs encoding clinical entities, temporal anchors, and typed relations; (2) applies graph-constrained perturbations modifying identifying context while preserving clinically essential structure; (3) regenerates text via graph-conditioned LLM generation.

Result: Evaluated on 90 clinician-authored psychiatric case narratives, Anonpsy preserves diagnostic fidelity while achieving consistently low re-identification risk under expert, semantic, and GPT-5-based evaluations. Compared with LLM-only rewriting baseline, Anonpsy yields substantially lower semantic similarity and identifiability.

Conclusion: Explicit structural representations combined with constrained generation provide an effective approach to de-identification for psychiatric narratives, demonstrating that graph-guided semantic rewriting outperforms text-level methods.

Abstract: Psychiatric narratives encode patient identity not only through explicit identifiers but also through idiosyncratic life events embedded in their clinical structure. Existing de-identification approaches, including PHI masking and LLM-based synthetic rewriting, operate at the text level and offer limited control over which semantic elements are preserved or altered. We introduce Anonpsy, a de-identification framework that reformulates the task as graph-guided semantic rewriting. Anonpsy (1) converts each narrative into a semantic graph encoding clinical entities, temporal anchors, and typed relations; (2) applies graph-constrained perturbations that modify identifying context while preserving clinically essential structure; and (3) regenerates text via graph-conditioned LLM generation. Evaluated on 90 clinician-authored psychiatric case narratives, Anonpsy preserves diagnostic fidelity while achieving consistently low re-identification risk under expert, semantic, and GPT-5-based evaluations. Compared with a strong LLM-only rewriting baseline, Anonpsy yields substantially lower semantic similarity and identifiability. These results demonstrate that explicit structural representations combined with constrained generation provide an effective approach to de-identification for psychiatric narratives.

Method: Systematic investigation using symmetric prompts with predicate-positive and predicate-negative constructions across four high-stakes evaluation tasks, testing 14 different LLM judges.

Result: Significant framing-induced discrepancies in model outputs, with clear susceptibility to framing across all tested LLMs and distinct tendencies toward agreement or rejection patterns by model family.

Conclusion: Framing bias is a structural property of current LLM-based evaluation systems, highlighting the need for framing-aware evaluation protocols to ensure more stable and impartial judgments.

Abstract: Large language models (LLMs) are known to produce varying responses depending on prompt phrasing, indicating that subtle guidance in phrasing can steer their answers. However, the impact of this framing bias on LLM-based evaluation, where models are expected to make stable and impartial judgments, remains largely underexplored. Drawing inspiration from the framing effect in psychology, we systematically investigate how deliberate prompt framing skews model judgments across four high-stakes evaluation tasks. We design symmetric prompts using predicate-positive and predicate-negative constructions and demonstrate that such framing induces significant discrepancies in model outputs. Across 14 LLM judges, we observe clear susceptibility to framing, with model families showing distinct tendencies toward agreement or rejection. These findings suggest that framing bias is a structural property of current LLM-based evaluation systems, underscoring the need for framing-aware protocols.

Method: HateXScore is a four-component metric suite that evaluates: (1) conclusion explicitness, (2) faithfulness and causal grounding of quoted spans, (3) protected group identification (policy-configurable), and (4) logical consistency among these elements.

Result: Evaluated on six diverse hate speech datasets, HateXScore reveals interpretability failures and annotation inconsistencies invisible to standard metrics like Accuracy or F1. Human evaluation shows strong agreement with HateXScore.

Conclusion: HateXScore serves as a diagnostic complement to standard metrics, providing a practical tool for trustworthy and transparent content moderation by evaluating reasoning quality in hate speech detection explanations.

Abstract: Hateful speech detection is a key component of content moderation, yet current evaluation frameworks rarely assess why a text is deemed hateful. We introduce \textsf{HateXScore}, a four-component metric suite designed to evaluate the reasoning quality of model explanations. It assesses (i) conclusion explicitness, (ii) faithfulness and causal grounding of quoted spans, (iii) protected group identification (policy-configurable), and (iv) logical consistency among these elements. Evaluated on six diverse hate speech datasets, \textsf{HateXScore} is intended as a diagnostic complement to reveal interpretability failures and annotation inconsistencies that are invisible to standard metrics like Accuracy or F1. Moreover, human evaluation shows strong agreement with \textsf{HateXScore}, validating it as a practical tool for trustworthy and transparent moderation. \textcolor{red}{Disclaimer: This paper contains sensitive content that may be disturbing to some readers.}

Method: Created SUDO dataset with 4,150 annotated review pairs (15,191 sentences) featuring bi-level structure capturing aspect-level mentions and review-level preferences. Benchmarked with traditional ML and language model baselines.

Result: Language model baselines outperform traditional ML approaches, but overall performance remains moderate, revealing the inherent difficulty of implicit comparative opinion mining.

Conclusion: SUDO establishes a challenging benchmark for implicit comparative opinion mining, highlighting the need for more advanced methods to tackle this underexplored but important task.

Abstract: Existing studies on comparative opinion mining have mainly focused on explicit comparative expressions, which are uncommon in real-world reviews. This leaves implicit comparisons - here users express preferences across separate reviews - largely underexplored. We introduce SUDO, a novel dataset for implicit comparative opinion mining from same-user reviews, allowing reliable inference of user preferences even without explicit comparative cues. SUDO comprises 4,150 annotated review pairs (15,191 sentences) with a bi-level structure capturing aspect-level mentions and review-level preferences. We benchmark this task using two baseline architectures: traditional machine learning- and language model-based baselines. Experimental results show that while the latter outperforms the former, overall performance remains moderate, revealing the inherent difficulty of the task and establishing SUDO as a challenging and valuable benchmark for future research.

Method: TREX trains small-scale proxy tokenizers on random data mixtures, collects compression statistics, and learns a regression model to predict compression performance from data mixtures, enabling efficient mixture search before large-scale training.

Result: Tokenizers trained with TREX’s predicted mixtures outperform LLaMA3 and uniform distribution mixtures by up to 12% in both in-distribution and out-of-distribution compression efficiency.

Conclusion: TREX provides a scalable, robust framework for multilingual tokenizer design that mitigates the accuracy-cost trade-off, demonstrating practical effectiveness for improving LLM training and inference efficiency.

Abstract: Building effective tokenizers for multilingual Large Language Models (LLMs) requires careful control over language-specific data mixtures. While a tokenizer’s compression performance critically affects the efficiency of LLM training and inference, existing approaches rely on heuristics or costly large-scale searches to determine optimal language ratios. We introduce Tokenizer Regression for Optimal Data MiXture (TREX), a regression-based framework that efficiently predicts the optimal data mixture for tokenizer training. TREX trains small-scale proxy tokenizers on random mixtures, gathers their compression statistics, and learns to predict compression performance from data mixtures. This learned model enables scalable mixture search before large-scale tokenizer training, mitigating the accuracy-cost trade-off in multilingual tokenizer design. Tokenizers trained with TReX’s predicted mixtures outperform mixtures based on LLaMA3 and uniform distributions by up to 12% in both inand out-of-distribution compression efficiency, demonstrating strong scalability, robustness, and practical effectiveness.

Method: Systematic evaluation using the SMCR communication framework across five mainstream LLMs and three domains (factual knowledge, medical QA, social bias). Analyzed persuasive strategies’ effects on belief stability over multiple turns, tested meta-cognition prompting, and evaluated adversarial fine-tuning as a defense.

Result: Smaller models show extreme compliance (80%+ belief changes at first persuasive turn). Meta-cognition prompting increases vulnerability rather than enhancing robustness. Adversarial fine-tuning effectiveness varies: GPT-4o-mini achieves 98.6% robustness, Mistral 7B improves from 35.7% to 79.3%, but Llama models remain highly susceptible (<14% even when fine-tuned on failure cases).

Conclusion: Current LLMs have substantial model-dependent limits in robustness to persuasion. Meta-cognition prompting backfires, and adversarial fine-tuning effectiveness varies dramatically. These findings highlight the need for better defense mechanisms and offer guidance for developing more trustworthy LLMs.

Abstract: Large Language Models (LLMs) are increasingly employed in various question-answering tasks. However, recent studies showcase that LLMs are susceptible to persuasion and could adopt counterfactual beliefs. We present a systematic evaluation of LLM susceptibility to persuasion under the Source–Message–Channel–Receiver (SMCR) communication framework. Across five mainstream Large Language Models (LLMs) and three domains (factual knowledge, medical QA, and social bias), we analyze how different persuasive strategies influence belief stability over multiple interaction turns. We further examine whether meta-cognition prompting (i.e., eliciting self-reported confidence) affects resistance to persuasion. Results show that smaller models exhibit extreme compliance, with over 80% of belief changes occurring at the first persuasive turn (average end turn of 1.1–1.4). Contrary to expectations, meta-cognition prompting increases vulnerability by accelerating belief erosion rather than enhancing robustness. Finally, we evaluate adversarial fine-tuning as a defense. While GPT-4o-mini achieves near-complete robustness (98.6%) and Mistral~7B improves substantially (35.7% $\rightarrow$ 79.3%), Llama models remain highly susceptible (<14%) even when fine-tuned on their own failure cases. Together, these findings highlight substantial model-dependent limits of current robustness interventions and offer guidance for developing more trustworthy LLMs.

Method: CauScientist uses hybrid initialization to select superior starting graphs, iteratively refines structures through LLM-proposed modifications validated by statistical criteria, and maintains error memory to guide efficient search space exploration.

Result: CauScientist substantially outperforms purely data-driven baselines with up to 53.8% F1 score improvement and enhances recall from 35.0% to 100.0%. It reduces structural hamming distance by 44.0% compared to Qwen3-32B on 37-node graphs.

Conclusion: The collaborative framework synergizing LLMs as hypothesis-generating “data scientists” with probabilistic statistics as rigorous “verifiers” provides an effective solution to overcome limitations of existing causal discovery approaches.

Abstract: Causal discovery is fundamental to scientific understanding and reliable decision-making. Existing approaches face critical limitations: purely data-driven methods suffer from statistical indistinguishability and modeling assumptions, while recent LLM-based methods either ignore statistical evidence or incorporate unverified priors that can mislead result. To this end, we propose CauScientist, a collaborative framework that synergizes LLMs as hypothesis-generating “data scientists” with probabilistic statistics as rigorous “verifiers”. CauScientist employs hybrid initialization to select superior starting graphs, iteratively refines structures through LLM-proposed modifications validated by statistical criteria, and maintains error memory to guide efficient search space. Experiments demonstrate that CauScientist substantially outperforms purely data-driven baselines, achieving up to 53.8% F1 score improvement and enhancing recall from 35.0% to 100.0%. Notably, while standalone LLM performance degrades with graph complexity, CauScientist reduces structural hamming distance (SHD) by 44.0% compared to Qwen3-32B on 37-node graphs. Our project page is at https://github.com/OpenCausaLab/CauScientist.

Method: AAAC (Activation-space Anchored Access Control) identifies geometric regularity in intermediate activations where representations for different permission scopes cluster distinctly. It constructs an anchor bank from a small offline sample set (one anchor per permission class) and uses multi-anchor steering to redirect query activations toward authorized regions at inference time.

Result: Extensive experiments across three LLM families show AAAC reduces permission violation rates by up to 86.5% and prompt-based attack success rates by 90.7%, while improving response usability with minor inference overhead compared to baselines.

Conclusion: AAAC provides an effective training-free solution for fine-grained access control in knowledge-base QA systems by leveraging geometric separability in LLM activations, preventing over-privileged generations while maintaining usability.

Abstract: Large language models (LLMs) are increasingly deployed over knowledge bases for efficient knowledge retrieval and question answering. However, LLMs can inadvertently answer beyond a user’s permission scope, leaking sensitive content, thus making it difficult to deploy knowledge-base QA under fine-grained access control requirements. In this work, we identify a geometric regularity in intermediate activations: for the same query, representations induced by different permission scopes cluster distinctly and are readily separable. Building on this separability, we propose Activation-space Anchored Access Control (AAAC), a training-free framework for multi-class permission control. AAAC constructs an anchor bank, with one permission anchor per class, from a small offline sample set and requires no fine-tuning. At inference time, a multi-anchor steering mechanism redirects each query’s activations toward the anchor-defined authorized region associated with the current user, thereby suppressing over-privileged generations by design. Finally, extensive experiments across three LLM families demonstrate that AAAC reduces permission violation rates by up to 86.5% and prompt-based attack success rates by 90.7%, while improving response usability with minor inference overhead compared to baselines.

Method: Proposes a unified detection framework that operates across multiple levels (document and token-levels), with formal definitions of text anomalies at both levels. Collects and annotates three benchmark datasets spanning spam, reviews, and grammar errors with token-level labels.

Result: Experimental results demonstrate that the proposed framework achieves better performance than six baseline methods, enabling precise anomaly localization in text.

Conclusion: Token-level anomaly detection opens new possibilities for precise anomaly localization in text, with all codes and data publicly available for further research.

Abstract: Despite significant progress in text anomaly detection for web applications such as spam filtering and fake news detection, existing methods are fundamentally limited to document-level analysis, unable to identify which specific parts of a text are anomalous. We introduce token-level anomaly detection, a novel paradigm that enables fine-grained localization of anomalies within text. We formally define text anomalies at both document and token-levels, and propose a unified detection framework that operates across multiple levels. To facilitate research in this direction, we collect and annotate three benchmark datasets spanning spam, reviews and grammar errors with token-level labels. Experimental results demonstrate that our framework get better performance than other 6 baselines, opening new possibilities for precise anomaly localization in text. All the codes and data are publicly available on https://github.com/charles-cao/TokenCore.

Method: Study examines two language bias types: (1) performance disparity across languages in same-language judging, and (2) bias toward major languages in cross-language judging. Investigates correlation between language bias and low-perplexity bias.

Result: Same-language judging shows significant performance disparities across language families (European > African), especially in culturally-related subjects. Cross-language judging reveals models favor English answers, with answer language more influential than question language. Language bias is only slightly correlated with perplexity and not fully explained by it.

Conclusion: LLM-as-a-judge exhibits substantial language bias that cannot be fully attributed to perplexity differences, highlighting the need for more equitable evaluation methods and better understanding of linguistic biases in LLM judging systems.

Abstract: Recent advances in Large Language Models (LLMs) have incentivized the development of LLM-as-a-judge, an application of LLMs where they are used as judges to decide the quality of a certain piece of text given a certain context. However, previous studies have demonstrated that LLM-as-a-judge can be biased towards different aspects of the judged texts, which often do not align with human preference. One of the identified biases is language bias, which indicates that the decision of LLM-as-a-judge can differ based on the language of the judged texts. In this paper, we study two types of language bias in pairwise LLM-as-a-judge: (1) performance disparity between languages when the judge is prompted to compare options from the same language, and (2) bias towards options written in major languages when the judge is prompted to compare options of two different languages. We find that for same-language judging, there exist significant performance disparities across language families, with European languages consistently outperforming African languages, and this bias is more pronounced in culturally-related subjects. For inter-language judging, we observe that most models favor English answers, and that this preference is influenced more by answer language than question language. Finally, we investigate whether language bias is in fact caused by low-perplexity bias, a previously identified bias of LLM-as-a-judge, and we find that while perplexity is slightly correlated with language bias, language bias cannot be fully explained by perplexity only.

Method: Two-step approach: (1) Temporal Knowledge Graph Forecasting generates plausible future quadruples, filtered to match original KB schema; (2) LLMs convert quadruples to text descriptions. Creates 4.2K future quadruples with corresponding text.

Result: LLM performance (tested on EDC framework) decreases when evaluated on the contamination-free future dataset compared to known facts datasets, revealing inflated performance in contaminated evaluations.

Conclusion: The synthetic future-facts dataset provides a robust, unbiased benchmark for TKGE evaluation, enabling continuous creation of contamination-free datasets for long-term assessment of LLM-based extraction systems.

Abstract: The automatic extraction of information is important for populating large web knowledge bases such as Wikidata. The temporal version of that task, temporal knowledge graph extraction (TKGE), involves extracting temporally grounded facts from text, represented as semantic quadruples (subject, relation, object, timestamp). Many recent systems take advantage of large language models (LLMs), which are becoming a new cornerstone of the web due to their performance on many tasks across the natural language processing (NLP) field. Despite the importance of TKGE, existing datasets for training and evaluation remain scarce, and contamination of evaluation data is an unaddressed issue, potentially inflating LLMs’ perceived performance due to overlaps between training and evaluation sets. To mitigate these challenges, we propose a novel synthetic evaluation dataset constructed from predicted future, previously unseen temporal facts, thereby eliminating contamination and enabling robust and unbiased benchmarking. Our dataset creation involves a two-step approach: (1) Temporal Knowledge Graph Forecasting (TKGF) generates plausible future quadruples, which are subsequently filtered to adhere to the original knowledge base schema; (2) LLMs perform quadruple-to-text generation, creating semantically aligned textual descriptions. We benchmark Extract, Define and Canonicalize (EDC), a state-of-the-art LLM-based extraction framework, demonstrating that LLM performance decreases when evaluated on our dataset compared to a dataset of known facts. We publicly release our dataset consisting of 4.2K future quadruples and corresponding textual descriptions, along with the generation methodology, enabling continuous creation of unlimited future temporal datasets to serve as long-term, contamination-free benchmarks for TKGE.

Method: Proposes community-level alignment approach and introduces CommunityBench, the first large-scale benchmark for community-level alignment evaluation with four tasks based on Common Identity and Common Bond theory. Conducts comprehensive evaluation of various foundation models on this benchmark.

Result: Current LLMs exhibit limited capacity to model community-specific preferences. Community-level alignment shows potential for facilitating individual modeling, offering a promising direction for scalable and pluralistic alignment.

Conclusion: Community-level alignment represents a viable “middle ground” approach that balances scalability with pluralism, addressing limitations of both universal and individual-level alignment strategies.

Abstract: Large language models (LLMs) alignment ensures model behaviors reflect human value. Existing alignment strategies primarily follow two paths: one assumes a universal value set for a unified goal (i.e., one-size-fits-all), while the other treats every individual as unique to customize models (i.e., individual-level). However, assuming a monolithic value space marginalizes minority norms, while tailoring individual models is prohibitively expensive. Recognizing that human society is organized into social clusters with high intra-group value alignment, we propose community-level alignment as a “middle ground”. Practically, we introduce CommunityBench, the first large-scale benchmark for community-level alignment evaluation, featuring four tasks grounded in Common Identity and Common Bond theory. With CommunityBench, we conduct a comprehensive evaluation of various foundation models on CommunityBench, revealing that current LLMs exhibit limited capacity to model community-specific preferences. Furthermore, we investigate the potential of community-level alignment in facilitating individual modeling, providing a promising direction for scalable and pluralistic alignment.

Method: HeteroCache categorizes attention heads based on stability and redundancy, applies fine-grained weighting to allocate larger cache budgets to heads with rapidly shifting attention, and uses hierarchical storage where representative heads monitor attention shifts and trigger asynchronous, on-demand retrieval from CPU to hide I/O latency.

Result: HeteroCache achieves state-of-the-art performance on multiple long-context benchmarks and accelerates decoding by up to 3× compared to the original model in 224K context settings.

Conclusion: HeteroCache effectively addresses KV cache memory bottleneck through training-free dynamic compression that leverages attention head heterogeneity, achieving significant performance improvements and speedups in long-context LLM inference.

Abstract: The linear memory growth of the KV cache poses a significant bottleneck for LLM inference in long-context tasks. Existing static compression methods often fail to preserve globally important information, principally because they overlook the attention drift phenomenon where token significance evolves dynamically. Although recent dynamic retrieval approaches attempt to address this issue, they typically suffer from coarse-grained caching strategies and incur high I/O overhead due to frequent data transfers. To overcome these limitations, we propose HeteroCache, a training-free dynamic compression framework. Our method is built on two key insights: attention heads exhibit diverse temporal heterogeneity, and there is significant spatial redundancy among heads within the same layer. Guided by these insights, HeteroCache categorizes heads based on stability and redundancy. Consequently, we apply a fine-grained weighting strategy that allocates larger cache budgets to heads with rapidly shifting attention to capture context changes, thereby addressing the inefficiency of coarse-grained strategies. Furthermore, we employ a hierarchical storage mechanism in which a subset of representative heads monitors attention shift, and trigger an asynchronous, on-demand retrieval of contexts from the CPU, effectively hiding I/O latency. Finally, experiments demonstrate that HeteroCache achieves state-of-the-art performance on multiple long-context benchmarks and accelerates decoding by up to $3\times$ compared to the original model in the 224K context. Our code will be open-source.

Method: 1) Created CDRD structure to capture clinical reasoning logic with construction pipeline; 2) Developed Dr. Assistant model with two-stage training: supervised fine-tuning followed by reinforcement learning with tailored reward function.

Result: Dr. Assistant outperforms open-source models and achieves competitive performance to closed-source models on diagnostic reasoning and inquiry benchmark.

Conclusion: Dr. Assistant provides effective solution for clinical diagnostic inquiry guidance by combining structured clinical reasoning with LLM capabilities.

Abstract: Clinical Decision Support Systems (CDSSs) provide reasoning and inquiry guidance for physicians, yet they face notable challenges, including high maintenance costs and low generalization capability. Recently, Large Language Models (LLMs) have been widely adopted in healthcare due to their extensive knowledge reserves, retrieval, and communication capabilities. While LLMs show promise and excel at medical benchmarks, their diagnostic reasoning and inquiry skills are constrained. To mitigate this issue, we propose (1) Clinical Diagnostic Reasoning Data (CDRD) structure to capture abstract clinical reasoning logic, and a pipeline for its construction, and (2) the Dr. Assistant, a clinical diagnostic model equipped with clinical reasoning and inquiry skills. Its training involves a two-stage process: SFT, followed by RL with a tailored reward function. We also introduce a benchmark to evaluate both diagnostic reasoning and inquiry. Our experiments demonstrate that the Dr. Assistant outperforms open-source models and achieves competitive performance to closed-source models, providing an effective solution for clinical diagnostic inquiry guidance.

Method: Uses OCR-oriented visual encoder to compress table structure/content into compact optical tokens, fine-tunes pretrained decoder for SQL generation while freezing encoder to isolate representation sufficiency.

Result: Retains strong execution accuracy on visualized Spider 2.0-Snow dataset while reducing table input tokens by an order of magnitude. Optical tokens preserve essential structural information under visual perturbations.

Conclusion: Compact optical representations can serve as an efficient interface for executable semantic parsing, enabling vision-driven SQL generation directly from table images with significantly reduced token overhead.

Abstract: Executable SQL generation is typically studied in text-to-SQL settings, where tables are provided as fully linearized textual schemas and contents. While effective, this formulation assumes access to structured text and incurs substantial token overhead, which is misaligned with many real-world scenarios where tables appear as visual artifacts in documents or webpages. We investigate whether compact optical representations can serve as an efficient interface for executable semantic parsing. We present OptiSQL, a vision-driven framework that generates executable SQL directly from table images and natural language questions using compact optical tokens. OptiSQL leverages an OCR-oriented visual encoder to compress table structure and content into a small set of optical tokens and fine-tunes a pretrained decoder for SQL generation while freezing the encoder to isolate representation sufficiency. Experiments on a visualized version of Spider 2.0-Snow show that OptiSQL retains strong execution accuracy while reducing table input tokens by an order of magnitude. Robustness analyses further demonstrate that optical tokens preserve essential structural information under visual perturbations.

Method: GRADFILTERING uses a small GPT-2 proxy with a LoRA ensemble to compute per-example gradients, then aggregates them into a Gradient Signal-to-Noise Ratio (G-SNR) utility score for data selection. The framework is objective-agnostic and uncertainty-aware.

Result: The method matches or surpasses random subsets and strong baselines in most LLM-as-a-judge evaluations and human assessments. GRADFILTERING-selected subsets converge faster than competitive filters under the same compute budget.

Conclusion: GRADFILTERING demonstrates the benefit of uncertainty-aware scoring for efficient data selection in instruction tuning, reducing computational costs while maintaining or improving model performance through better identification of important training examples.

Abstract: Instruction tuning is a standard paradigm for adapting large language models (LLMs), but modern instruction datasets are large, noisy, and redundant, making full-data fine-tuning costly and often unnecessary. Existing data selection methods either build expensive gradient datastores or assign static scores from a weak proxy, largely ignoring evolving uncertainty, and thus missing a key source of LLM interpretability. We propose GRADFILTERING, an objective-agnostic, uncertainty-aware data selection framework that utilizes a small GPT-2 proxy with a LoRA ensemble and aggregates per-example gradients into a Gradient Signal-to-Noise Ratio (G-SNR) utility. Our method matches or surpasses random subsets and strong baselines in most LLM-as-a-judge evaluations as well as in human assessment. Moreover, GRADFILTERING-selected subsets converge faster than competitive filters under the same compute budget, reflecting the benefit of uncertainty-aware scoring.

Method: Built GerAV benchmark from Twitter and Reddit data, with Reddit further divided into in-domain/cross-domain message-based subsets and profile-based subsets. Conducted systematic evaluation using training splits to test strong baselines and state-of-the-art models.

Result: Best approach (fine-tuned large language model) outperforms recent baselines by up to 0.09 absolute F1 score and surpasses GPT-5 in zero-shot setting by 0.08. Found trade-off between specialization and generalization: models trained on specific data perform best under matching conditions but generalize poorly across data regimes.

Conclusion: GerAV provides a challenging and versatile benchmark for advancing research on German and cross-domain authorship verification, with findings showing that combining training sources can mitigate generalization limitations.

Abstract: Authorship verification (AV) is the task of determining whether two texts were written by the same author and has been studied extensively, predominantly for English data. In contrast, large-scale benchmarks and systematic evaluations for other languages remain scarce. We address this gap by introducing GerAV, a comprehensive benchmark for German AV comprising over 600k labeled text pairs. GerAV is built from Twitter and Reddit data, with the Reddit part further divided into in-domain and cross-domain message-based subsets, as well as a profile-based subset. This design enables controlled analysis of the effects of data source, topical domain, and text length. Using the provided training splits, we conduct a systematic evaluation of strong baselines and state-of-the-art models and find that our best approach, a fine-tuned large language model, outperforms recent baselines by up to 0.09 absolute F1 score and surpasses GPT-5 in a zero-shot setting by 0.08. We further observe a trade-off between specialization and generalization: models trained on specific data types perform best under matching conditions but generalize less well across data regimes, a limitation that can be mitigated by combining training sources. Overall, GerAV provides a challenging and versatile benchmark for advancing research on German and cross-domain AV.

Method: The study conducted systematic testing across 477 competition-level questions and 9 models to evaluate whether Simulated Ignorance (SI) can approximate True Ignorance (TI). They examined cutoff instructions, chain-of-thought reasoning, and reasoning-optimized models to assess knowledge suppression capabilities.

Result: SI fails systematically: (1) cutoff instructions leave a 52% performance gap between SI and TI; (2) chain-of-thought reasoning fails to suppress prior knowledge, even when reasoning traces contain no explicit post-cutoff references; (3) reasoning-optimized models exhibit worse SI fidelity despite superior reasoning trace quality.

Conclusion: Prompts cannot reliably “rewind” model knowledge. Retrospective forecasting on pre-cutoff events is methodologically flawed, and the authors recommend against using SI-based retrospective setups to benchmark forecasting capabilities.

Abstract: Evaluating LLM forecasting capabilities is constrained by a fundamental tension: prospective evaluation offers methodological rigor but prohibitive latency, while retrospective forecasting (RF) – evaluating on already-resolved events – faces rapidly shrinking clean evaluation data as SOTA models possess increasingly recent knowledge cutoffs. Simulated Ignorance (SI), prompting models to suppress pre-cutoff knowledge, has emerged as a potential solution. We provide the first systematic test of whether SI can approximate True Ignorance (TI). Across 477 competition-level questions and 9 models, we find that SI fails systematically: (1) cutoff instructions leave a 52% performance gap between SI and TI; (2) chain-of-thought reasoning fails to suppress prior knowledge, even when reasoning traces contain no explicit post-cutoff references; (3) reasoning-optimized models exhibit worse SI fidelity despite superior reasoning trace quality. These findings demonstrate that prompts cannot reliably “rewind” model knowledge. We conclude that RF on pre-cutoff events is methodologically flawed; we recommend against using SI-based retrospective setups to benchmark forecasting capabilities.

Method: Formalized over-personalization into three types (Irrelevance, Repetition, Sycophancy), created OP-Bench with 1,700 verified instances from long-horizon dialogues, evaluated LLMs and memory-augmentation methods, and proposed Self-ReCheck - a lightweight, model-agnostic memory filtering mechanism.

Result: Over-personalization is widespread when memory is introduced. Agents tend to retrieve and over-attend to user memories even when unnecessary. Self-ReCheck effectively mitigates over-personalization while preserving personalization performance.

Conclusion: The work takes an initial step toward more controllable and appropriate personalization in memory-augmented dialogue systems by identifying the over-personalization problem and providing a practical solution.

Abstract: Memory-augmented conversational agents enable personalized interactions using long-term user memory and have gained substantial traction. However, existing benchmarks primarily focus on whether agents can recall and apply user information, while overlooking whether such personalization is used appropriately. In fact, agents may overuse personal information, producing responses that feel forced, intrusive, or socially inappropriate to users. We refer to this issue as \emph{over-personalization}. In this work, we formalize over-personalization into three types: Irrelevance, Repetition, and Sycophancy, and introduce \textbf{OP-Bench} a benchmark of 1,700 verified instances constructed from long-horizon dialogue histories. Using \textbf{OP-Bench}, we evaluate multiple large language models and memory-augmentation methods, and find that over-personalization is widespread when memory is introduced. Further analysis reveals that agents tend to retrieve and over-attend to user memories even when unnecessary. To address this issue, we propose \textbf{Self-ReCheck}, a lightweight, model-agnostic memory filtering mechanism that mitigates over-personalization while preserving personalization performance. Our work takes an initial step toward more controllable and appropriate personalization in memory-augmented dialogue systems.

Method: Systematically evaluate modern MT systems, identify ND-MT as distinct phenomenon, evaluate five state-of-the-art ND-MT systems across three open datasets using both lexical-based and semantic-based metrics at varying sampling sizes, and propose ExpectoSample strategy for metric reliability assessment.

Result: ND-MT addresses multi-modality issues and provides higher-quality candidates than D-MT under temperature constraints, but reveals a “Buckets effect” where the lowest-quality candidate determines overall system ranking across different sampling sizes for all reasonable metrics.

Conclusion: ND-MT shows significant potential for improving translation quality but introduces new evaluation challenges. Current D-MT evaluation frameworks fail with ND-MT, requiring new assessment strategies like ExpectoSample to reliably evaluate non-deterministic translation systems.

Abstract: In recent years, the non-deterministic properties of language models have garnered considerable attention and have shown a significant influence on real-world applications. However, such properties remain under-explored in machine translation (MT), a complex, non-deterministic NLP task. In this study, we systematically evaluate modern MT systems and identify temperature-constrained Non-Deterministic MT (ND-MT) as a distinct phenomenon. Additionally, we demonstrate that ND-MT exhibits significant potential in addressing the multi-modality issue that has long challenged MT research and provides higher-quality candidates than Deterministic MT (D-MT) under temperature constraints. However, ND-MT introduces new challenges in evaluating system performance. Specifically, the evaluation framework designed for D-MT fails to yield consistent evaluation results when applied to ND-MT. We further investigate this emerging challenge by evaluating five state-of-the-art ND-MT systems across three open datasets using both lexical-based and semantic-based metrics at varying sampling sizes. The results reveal a Buckets effect across these systems: the lowest-quality candidate generated by ND-MT consistently determines the overall system ranking across different sampling sizes for all reasonable metrics. Furthermore, we propose the ExpectoSample strategy to automatically assess the reliability of evaluation metrics for selecting robust ND-MT.

Method: 1) Identify key tokens in long contexts using loss gaps between long and short forward contexts, then find most relevant preceding paragraphs and summarize them using an LLM. 2) Equip models to autonomously generate and utilize retrospective summaries during inference, establishing a recursive memory mechanism across paragraphs.

Result: Substantial performance gains: 26.8% improvement on RULER benchmark and 9.44% improvement on LongBench benchmark, demonstrating effective enhancement of long-context understanding.

Conclusion: ARL offers a simple yet effective continued pretraining-based approach to strengthen long-context understanding in LLMs, advancing scalable memory augmentation and recursive memory mechanisms for processing extended sequences.

Abstract: In this paper, we propose active recap learning (ARL), a framework for enhancing large language model (LLM) in understanding long contexts. ARL enables models to revisit and summarize earlier content through targeted sequence construction during contined pretraining and retrospective summarization at inference. First, we identify key tokens in prepared long context based on loss gaps between long and short forward contexts and find most revant preceding paragraphs, then summarize them using an LLM. Second, ARL equips models with the ability to autonomously generate and utilize these retrospective summaries during inference, thereby establishing a recursive memory mechanism across paragraphs. Experimental results show substantial gains, with ARL achieving a 26.8% improvement on RULER and a 9.44% improvement on LongBench. Overall, ARL offers a simple yet effective continued pretraining-based approach to strengthen long-context understanding, advancing scalable memory augmentation in LLM

Method: Introduces Human Chain-of-Thought (HCoT) annotation protocol to separate evaluation into content, voice quality, and paralinguistics dimensions. TRACE converts inexpensive audio signals to textual blueprints, then prompts LLMs to render dimension-wise judgments and fuses them via deterministic policy.

Result: TRACE achieves higher agreement with human raters than both ALMs and transcript-only LLM judges while being significantly more cost-effective.

Conclusion: TRACE enables scalable, human-aligned S2S evaluation by leveraging LLMs’ reasoning over audio cues, offering a cost-effective alternative to opaque ALMs. The framework and HCoT annotations will be released.

Abstract: Large Language Model (LLM) judges exhibit strong reasoning capabilities but are limited to textual content. This leaves current automatic Speech-to-Speech (S2S) evaluation methods reliant on opaque and expensive Audio Language Models (ALMs). In this work, we propose TRACE (Textual Reasoning over Audio Cues for Evaluation), a novel framework that enables LLM judges to reason over audio cues to achieve cost-efficient and human-aligned S2S evaluation. To demonstrate the strength of the framework, we first introduce a Human Chain-of-Thought (HCoT) annotation protocol to improve the diagnostic capability of existing judge benchmarks by separating evaluation into explicit dimensions: content (C), voice quality (VQ), and paralinguistics (P). Using this data, TRACE constructs a textual blueprint of inexpensive audio signals and prompts an LLM to render dimension-wise judgments, fusing them into an overall rating via a deterministic policy. TRACE achieves higher agreement with human raters than ALMs and transcript-only LLM judges while being significantly more cost-effective. We will release the HCoT annotations and the TRACE framework to enable scalable and human-aligned S2S evaluation.

Method: Three complementary experiments: 1) Analyzing LLM recommendations for AI vs non-AI options in advice-seeking queries, 2) Comparing salary estimations for AI vs matched non-AI jobs, 3) Probing internal representations of open-weight models to measure similarity between “Artificial Intelligence” and generic prompts under different framings.

Result: Consistent pro-AI bias: LLMs disproportionately recommend AI-related options (proprietary models almost deterministically), overestimate AI salaries by 10 percentage points more than non-AI jobs, and show “Artificial Intelligence” has highest similarity to generic prompts across positive, negative, and neutral framings.

Conclusion: LLMs exhibit systematic pro-AI bias that could skew high-stakes decisions, suggesting need for awareness and mitigation of such biases in AI-assisted decision-making systems.

Abstract: Large language models (LLMs) are increasingly employed for decision-support across multiple domains. We investigate whether these models display a systematic preferential bias in favor of artificial intelligence (AI) itself. Across three complementary experiments, we find consistent evidence of pro-AI bias. First, we show that LLMs disproportionately recommend AI-related options in response to diverse advice-seeking queries, with proprietary models doing so almost deterministically. Second, we demonstrate that models systematically overestimate salaries for AI-related jobs relative to closely matched non-AI jobs, with proprietary models overestimating AI salaries more by 10 percentage points. Finally, probing internal representations of open-weight models reveals that ``Artificial Intelligence’’ exhibits the highest similarity to generic prompts for academic fields under positive, negative, and neutral framings alike, indicating valence-invariant representational centrality. These patterns suggest that LLM-generated advice and valuation can systematically skew choices and perceptions in high-stakes decisions.

Method: Constructed knowledge graph with 12K legal cases using IRAC framework, generated training data from KG, performed SFT and DPO on three SOTA LLMs (30B, 49B, 70B) with varied architectures.

Result: Post-trained models outperformed baselines on 4/5 diverse legal benchmarks (14 tasks). The 70B DPO model achieved best scores on 4/6 reasoning tasks, beating even a 141B SOTA legal LLM.

Conclusion: KG-assisted approach using domain-specific frameworks like IRAC effectively enhances LLMs’ reasoning capabilities in high-stakes professional domains like law, demonstrating generalizability to other domains.

Abstract: LLM post-training has primarily relied on large text corpora and human feedback, without capturing the structure of domain knowledge. This has caused models to struggle dealing with complex reasoning tasks, especially for high-stakes professional domains. In Law, reasoning requires deep understanding of the relations between various legal concepts, a key component missing in current LLM post-training. In this paper, we propose a knowledge graph (KG)-assisted approach for enhancing LLMs’ reasoning capability in Legal that is generalizable to other high-stakes domains. We model key legal concepts by following the \textbf{IRAC} (Issue, Rule, Analysis and Conclusion) framework, and construct a KG with 12K legal cases. We then produce training data using our IRAC KG, and conduct both Supervised Fine-Tuning (SFT) and Direct Preference Optimization (DPO) with three state-of-the-art (SOTA) LLMs (30B, 49B and 70B), varying architecture and base model family. Our post-trained models obtained better average performance on 4/5 diverse legal benchmarks (14 tasks) than baselines. In particular, our 70B DPO model achieved the best score on 4/6 reasoning tasks, among baselines and a 141B SOTA legal LLM, demonstrating the effectiveness of our KG for enhancing LLMs’ legal reasoning capability.

Method: Introduced a vocoder-based approach to control prosody and lexical cues in speech more cleanly than prior work, then used this to probe a voice-activity projection model (an S3R-based turn-taking model) with various cue manipulations.

Result: Prediction accuracy on prosody-matched unintelligible noise was similar to clean speech, showing both prosodic and lexical cues support turn-taking independently. When one cue is disrupted, models automatically exploit the other without retraining. Results consistent across CPC-based and wav2vec2.0 S3Rs.

Conclusion: Future turn-taking models may only require prosody, offering privacy benefits. Prosodic and lexical cues are encoded in S3Rs with limited interdependence, allowing flexible cue usage. Code released to support future research.

Abstract: Fluid turn-taking remains a key challenge in human-robot interaction. Self-supervised speech representations (S3Rs) have driven many advances, but it remains unclear whether S3R-based turn-taking models rely on prosodic cues, lexical cues or both. We introduce a vocoder-based approach to control prosody and lexical cues in speech more cleanly than prior work. This allows us to probe the voice-activity projection model, an S3R-based turn-taking model. We find that prediction on prosody-matched, unintelligible noise is similar to accuracy on clean speech. This reveals both prosodic and lexical cues support turn-taking, but either can be used in isolation. Hence, future models may only require prosody, providing privacy and potential performance benefits. When either prosodic or lexical information is disrupted, the model exploits the other without further training, indicating they are encoded in S3Rs with limited interdependence. Results are consistent in CPC-based and wav2vec2.0 S3Rs. We discuss our findings and highlight a number of directions for future work. All code is available to support future research.

Method: Four-component framework: text healing to restore language generation capabilities, LLM distillation to transfer pedagogical knowledge, safety training for educational environments, and pedagogical evaluation adjusted to science education contexts. Evaluated across five science demonstrations in physics, chemistry, biology, and earth science.

Result: Achieves comparable task performance to baseline models while producing contextually appropriate educational explanations. Evaluation assesses both task performance (success rate, protocol compliance, efficiency, safety) and pedagogical quality through teacher surveys and LLM-as-Judge assessment.

Conclusion: The Pedagogical VLA Framework enables lightweight VLA models to effectively support science demonstrations in educational settings by balancing computational efficiency with pedagogical quality and safety requirements.

Abstract: Science demonstrations are important for effective STEM education, yet teachers face challenges in conducting them safely and consistently across multiple occasions, where robotics can be helpful. However, current Vision-Language-Action (VLA) models require substantial computational resources and sacrifice language generation capabilities to maximize efficiency, making them unsuitable for resource-constrained educational settings that require interpretable, explanation-generating systems. We present \textit{Pedagogical VLA Framework}, a framework that applies pedagogical alignment to lightweight VLA models through four components: text healing to restore language generation capabilities, large language model (LLM) distillation to transfer pedagogical knowledge, safety training for educational environments, and pedagogical evaluation adjusted to science education contexts. We evaluate Pedagogical VLA Framework across five science demonstrations spanning physics, chemistry, biology, and earth science, using an evaluation framework developed in collaboration with science education experts. Our evaluation assesses both task performance (success rate, protocol compliance, efficiency, safety) and pedagogical quality through teacher surveys and LLM-as-Judge assessment. We additionally provide qualitative analysis of generated texts. Experimental results demonstrate that Pedagogical VLA Framework achieves comparable task performance to baseline models while producing contextually appropriate educational explanations.

Method: Developed a three-dimensional framework based on educational assessment theory: Knowledge (curriculum-aligned content), Skills (scenario-based competencies with four-level hierarchy), and Attitude (alignment consistency and deception resistance). Includes 124K+ items across subjects, roles, and Bloom’s taxonomy levels.

Result: Evaluation of seven frontier models shows distinct profiles: Claude-Opus-4.5 excels in practical skills despite lower content knowledge, Grok-4.1-fast leads in knowledge but shows alignment concerns. No single model dominates all dimensions.

Conclusion: OpenLearnLM provides an open, comprehensive framework for advancing LLM readiness in authentic educational contexts, validating the necessity of multi-axis evaluation for educational applications.

Abstract: Large Language Models are increasingly deployed as educational tools, yet existing benchmarks focus on narrow skills and lack grounding in learning sciences. We introduce OpenLearnLM Benchmark, a theory-grounded framework evaluating LLMs across three dimensions derived from educational assessment theory: Knowledge (curriculum-aligned content and pedagogical understanding), Skills (scenario-based competencies organized through a four-level center-role-scenario-subscenario hierarchy), and Attitude (alignment consistency and deception resistance). Our benchmark comprises 124K+ items spanning multiple subjects, educational roles, and difficulty levels based on Bloom’s taxonomy. The Knowledge domain prioritizes authentic assessment items from established benchmarks, while the Attitude domain adapts Anthropic’s Alignment Faking methodology to detect behavioral inconsistency under varying monitoring conditions. Evaluation of seven frontier models reveals distinct capability profiles: Claude-Opus-4.5 excels in practical skills despite lower content knowledge, while Grok-4.1-fast leads in knowledge but shows alignment concerns. Notably, no single model dominates all dimensions, validating the necessity of multi-axis evaluation. OpenLearnLM provides an open, comprehensive framework for advancing LLM readiness in authentic educational contexts.

Method: Extends IRT-based adaptive testing by replacing Bernoulli response distribution with heteroskedastic normal distribution for continuous bounded scores, plus uncertainty-aware ranker with adaptive stopping criteria.

Result: Method uses only 2% of test items while improving ranking correlation by 0.12 τ over random sampling, achieving 95% accuracy on confident predictions across five benchmarks with different scoring metrics.

Conclusion: The approach enables efficient and reliable LLM evaluation on generation tasks with continuous scoring, significantly reducing computational cost while maintaining or improving ranking accuracy.

Abstract: Computerized Adaptive Testing (CAT) has proven effective for efficient LLM evaluation on multiple-choice benchmarks, but modern LLM evaluation increasingly relies on generation tasks where outputs are scored continuously rather than marked correct/incorrect. We present a principled extension of IRT-based adaptive testing to continuous bounded scores (ROUGE, BLEU, LLM-as-a-Judge) by replacing the Bernoulli response distribution with a heteroskedastic normal distribution. Building on this, we introduce an uncertainty aware ranker with adaptive stopping criteria that achieves reliable model ranking while testing as few items and as cheaply as possible. We validate our method on five benchmarks spanning n-gram-based, embedding-based, and LLM-as-judge metrics. Our method uses 2% of the items while improving ranking correlation by 0.12 τ over random sampling, with 95% accuracy on confident predictions.

Method: Proposes RetroSum framework with two key components: 1) retrospective summarization mechanism that dynamically re-evaluates interaction history to prevent information loss and maintain logical coherence, and 2) evolving experience strategy that retrieves accumulated experience from a memory bank to bridge domain gaps.

Result: RetroSum achieves performance gains up to 29.16% over competitive baselines and reduces total interaction errors by up to 92.3% on the AgentEHR benchmark for complex EHR navigation tasks.

Conclusion: The RetroSum framework successfully addresses critical limitations in EHR navigation by preventing information loss and maintaining reasoning continuity, demonstrating significant improvements in performance and error reduction for realistic clinical decision-making tasks.

Abstract: Large Language Models have demonstrated profound utility in the medical domain. However, their application to autonomous Electronic Health Records~(EHRs) navigation remains constrained by a reliance on curated inputs and simplified retrieval tasks. To bridge the gap between idealized experimental settings and realistic clinical environments, we present AgentEHR. This benchmark challenges agents to execute complex decision-making tasks, such as diagnosis and treatment planning, requiring long-range interactive reasoning directly within raw and high-noise databases. In tackling these tasks, we identify that existing summarization methods inevitably suffer from critical information loss and fractured reasoning continuity. To address this, we propose RetroSum, a novel framework that unifies a retrospective summarization mechanism with an evolving experience strategy. By dynamically re-evaluating interaction history, the retrospective mechanism prevents long-context information loss and ensures unbroken logical coherence. Additionally, the evolving strategy bridges the domain gap by retrieving accumulated experience from a memory bank. Extensive empirical evaluations demonstrate that RetroSum achieves performance gains of up to 29.16% over competitive baselines, while significantly decreasing total interaction errors by up to 92.3%.

Method: HyperWalker constructs a dynamic hypergraph (iBrochure) to model EHR structural heterogeneity and high-order associations, uses a reinforcement learning agent (Walker) to navigate diagnostic paths, and incorporates a linger mechanism with multi-hop orthogonal retrieval to select clinically complementary neighborhood cases.

Result: Experiments on medical report generation with MIMIC and medical VQA on EHRXQA demonstrate that HyperWalker achieves state-of-the-art performance.

Conclusion: HyperWalker successfully overcomes the limitations of sample-isolated inference by integrating EHR data through dynamic hypergraphs and test-time training, enabling more comprehensive clinical reasoning and accurate diagnosis.

Abstract: Automated clinical diagnosis remains a core challenge in medical AI, which usually requires models to integrate multi-modal data and reason across complex, case-specific contexts. Although recent methods have advanced medical report generation (MRG) and visual question answering (VQA) with medical vision-language models (VLMs), these methods, however, predominantly operate under a sample-isolated inference paradigm, as such processing cases independently without access to longitudinal electronic health records (EHRs) or structurally related patient examples. This paradigm limits reasoning to image-derived information alone, which ignores external complementary medical evidence for potentially more accurate diagnosis. To overcome this limitation, we propose \textbf{HyperWalker}, a \textit{Deep Diagnosis} framework that reformulates clinical reasoning via dynamic hypergraphs and test-time training. First, we construct a dynamic hypergraph, termed \textbf{iBrochure}, to model the structural heterogeneity of EHR data and implicit high-order associations among multimodal clinical information. Within this hypergraph, a reinforcement learning agent, \textbf{Walker}, navigates to and identifies optimal diagnostic paths. To ensure comprehensive coverage of diverse clinical characteristics in test samples, we incorporate a \textit{linger mechanism}, a multi-hop orthogonal retrieval strategy that iteratively selects clinically complementary neighborhood cases reflecting distinct clinical attributes. Experiments on MRG with MIMIC and medical VQA on EHRXQA demonstrate that HyperWalker achieves state-of-the-art performance. Code is available at: https://github.com/Bean-Young/HyperWalker

Method: Multi-agent framework where language-model agents jointly propose feature definitions, extract feature values, and evaluate feature quality using dataset-level performance and interpretability feedback, with iterative prompt refinement.

Result: Proposes a principled mechanism for automatic feature discovery that optimizes over prompts inducing shared feature sets rather than per-example predictions, departing from prior per-sample supervision methods.

Conclusion: Formulates feature discovery as dataset-level prompt optimization, enabling automatic extraction of interpretable and discriminative features from unstructured text through multi-agent collaboration and structured feedback.

Abstract: Feature extraction from unstructured text is a critical step in many downstream classification pipelines, yet current approaches largely rely on hand-crafted prompts or fixed feature schemas. We formulate feature discovery as a dataset-level prompt optimization problem: given a labelled text corpus, the goal is to induce a global set of interpretable and discriminative feature definitions whose realizations optimize a downstream supervised learning objective. To this end, we propose a multi-agent prompt optimization framework in which language-model agents jointly propose feature definitions, extract feature values, and evaluate feature quality using dataset-level performance and interpretability feedback. Instruction prompts are iteratively refined based on this structured feedback, enabling optimization over prompts that induce shared feature sets rather than per-example predictions. This formulation departs from prior prompt optimization methods that rely on per-sample supervision and provides a principled mechanism for automatic feature discovery from unstructured text.

Method: COMPACT adaptively fuses supervisions from different teachers by dynamically weighting teacher gradients based on the student’s real-time compatibility evaluated by three metrics: (1) Graph-based Consensus to filter misleading rationales by identifying mainstream reasoning paths; (2) Mutual-Information-based Adaptability to detect “epiphany moments” for genuine understanding rather than imitation; and (3) Loss-based Difficulty to assess student receptivity and prevent negative transfer.

Result: Extensive experiments and latent space analysis demonstrate that COMPACT effectively integrates diverse reasoning capabilities without damaging the model’s original knowledge structure, achieving state-of-the-art performance on various benchmarks while mitigating catastrophic forgetting.

Conclusion: COMPACT provides an effective framework for multi-teacher CoT distillation that overcomes limitations of single-teacher approaches, enabling compact student models to benefit from diverse reasoning capabilities while maintaining knowledge integrity and preventing catastrophic forgetting.

Abstract: Chain-of-Thought (CoT) reasoning empowers Large Language Models (LLMs) with remarkable capabilities but typically requires prohibitive parameter scales. CoT distillation has emerged as a promising paradigm to transfer reasoning prowess into compact Student Models (SLMs), but existing approaches often rely on a solitary teacher, capping the student’s potential since individual LLMs often exhibit distinct capability biases and may suffer from catastrophic forgetting. While leveraging diverse teachers seems appealing, effectively fusing their supervisions remains challenging: teacher-student incompatibility risks amplifying hallucinations, and passive supervision fails to ensure genuine logic internalization. To address this, we introduce COMPACT, a framework that adaptively fuses supervisions from different teachers by dynamically weighting teacher gradients based on the student’s real-time compatibility evaluated by a multi-dimensional metric: (1) Graph-based Consensus to filter misleading rationales by identifying mainstream reasoning paths; (2) Mutual-Information-based Adaptability to detect “epiphany moments” for genuinely understanding the reasoning process rather than merely imitating; and (3) Loss-based Difficulty to assess student receptivity to the teacher’s guidance and prevent negative transfer. Extensive experiments and latent space analysis demonstrate that COMPACT effectively integrates diverse reasoning capabilities without damaging the model’s original knowledge structure, achieving state-of-the-art performance on various benchmarks while mitigating catastrophic forgetting.

Method: Proposes Tree-aware Aligned Global Sampling (TAGS): 1) Uses LLM-based tagger to extract atomic knowledge concepts, 2) Organizes concepts into global tree via hierarchical clustering, 3) Grounds data instances onto tree, 4) Uses tree-aware metric to quantify quality/diversity, 5) Implements controllable sampling maximizing tree-level information gain with leaf-level alignment via KL-divergence.

Result: Significantly outperforms state-of-the-art baselines. Surpasses full-dataset model by +5.84% using only 5% of data. Aligned sampling strategy further boosts average performance by +4.24%.

Conclusion: TAGS provides an effective framework for data selection in LLM instruction tuning by leveraging hierarchical knowledge structures, enabling precise control over quality, diversity, and alignment while achieving superior performance with minimal data.

Abstract: Effective and controllable data selection is critical for LLM instruction tuning, especially with massive open-source datasets. Existing approaches primarily rely on instance-level quality scores, or diversity metrics based on embedding clusters or semantic tags. However, constrained by the flatness of embedding spaces or the coarseness of tags, these approaches overlook fine-grained knowledge and its intrinsic hierarchical dependencies, consequently hindering precise data valuation and knowledge-aligned sampling. To address this challenge, we propose Tree-aware Aligned Global Sampling (TAGS), a unified framework that leverages a knowledge tree built from fine-grained tags, thereby enabling joint control of global quality, diversity, and target alignment. Using an LLM-based tagger, we extract atomic knowledge concepts, which are organized into a global tree through bottom-up hierarchical clustering. By grounding data instances onto this tree, a tree-aware metric then quantifies data quality and diversity, facilitating effective sampling. Our controllable sampling strategy maximizes tree-level information gain and enforces leaf-level alignment via KL-divergence for specific domains. Extensive experiments demonstrate that TAGS significantly outperforms state-of-the-art baselines. Notably, it surpasses the full-dataset model by \textbf{+5.84%} using only \textbf{5%} of the data, while our aligned sampling strategy further boosts average performance by \textbf{+4.24%}.

Method: Proposes a structured pipeline: “Locate, Steer, and Improve.” Categorizes Localizing (diagnosis) and Steering (intervention) methods based on specific Interpretable Objects to establish rigorous intervention protocols.

Result: Demonstrates how the framework enables tangible improvements in Alignment, Capability, and Efficiency, effectively operationalizing MI as an actionable methodology for model optimization.

Conclusion: The survey provides a practical framework that transforms mechanistic interpretability from observational analysis to actionable intervention, with curated resources available for implementation.

Abstract: Mechanistic Interpretability (MI) has emerged as a vital approach to demystify the opaque decision-making of Large Language Models (LLMs). However, existing reviews primarily treat MI as an observational science, summarizing analytical insights while lacking a systematic framework for actionable intervention. To bridge this gap, we present a practical survey structured around the pipeline: “Locate, Steer, and Improve.” We formally categorize Localizing (diagnosis) and Steering (intervention) methods based on specific Interpretable Objects to establish a rigorous intervention protocol. Furthermore, we demonstrate how this framework enables tangible improvements in Alignment, Capability, and Efficiency, effectively operationalizing MI as an actionable methodology for model optimization. The curated paper list of this work is available at https://github.com/rattlesnakey/Awesome-Actionable-MI-Survey.

Method: Introduces abstraction-grounding framework with three capacities: A-A (abstract interpretation), A-C (grounding in concrete events), and C-C (applying abstract principles to decisions). Uses probing (detecting value traces in activations) and steering (modifying representations to shift behavior) across six open-source LLMs and ten value dimensions.

Result: Probing shows cross-level transfer - probes trained on abstract descriptions detect same values in concrete narratives. Steering reveals asymmetry: intervening on value representations shifts concrete judgments but leaves abstract interpretations unchanged, suggesting abstract values function as stable anchors.

Conclusion: LLMs maintain structured value representations that bridge abstraction and action, providing mechanistic foundation for building value-driven autonomous AI systems with transparent, generalizable alignment and control.

Abstract: Do large language models (LLMs) genuinely understand abstract concepts, or merely manipulate them as statistical patterns? We introduce an abstraction-grounding framework that decomposes conceptual understanding into three capacities: interpretation of abstract concepts (Abstract-Abstract, A-A), grounding of abstractions in concrete events (Abstract-Concrete, A-C), and application of abstract principles to regulate concrete decisions (Concrete-Concrete, C-C). Using human values as a testbed - given their semantic richness and centrality to alignment - we employ probing (detecting value traces in internal activations) and steering (modifying representations to shift behavior). Across six open-source LLMs and ten value dimensions, probing shows that diagnostic probes trained solely on abstract value descriptions reliably detect the same values in concrete event narratives and decision reasoning, demonstrating cross-level transfer. Steering reveals an asymmetry: intervening on value representations causally shifts concrete judgments and decisions (A-C, C-C), yet leaves abstract interpretations unchanged (A-A), suggesting that encoded abstract values function as stable anchors rather than malleable activations. These findings indicate LLMs maintain structured value representations that bridge abstraction and action, providing a mechanistic and operational foundation for building value-driven autonomous AI systems with more transparent, generalizable alignment and control.

Method: Proposes RM-Distiller framework with three key components: (1) Refinement capability - synthesizes highly correlated response pairs for fine-grained contrastive signals; (2) Scoring capability - guides RM with margin-aware optimization to capture preference strength; (3) Generation capability - incorporates teacher’s generative distribution to preserve linguistic knowledge.

Result: Extensive experiments show RM-Distiller significantly outperforms traditional distillation methods on both RM benchmarks and reinforcement learning-based alignment.

Conclusion: Exploiting multifaceted teacher capabilities is critical for effective reward modeling, and this represents the first systematic research on RM distillation from generative LLMs.

Abstract: Reward models (RMs) play a pivotal role in aligning large language models (LLMs) with human preferences. Due to the difficulty of obtaining high-quality human preference annotations, distilling preferences from generative LLMs has emerged as a standard practice. However, existing approaches predominantly treat teacher models as simple binary annotators, failing to fully exploit the rich knowledge and capabilities for RM distillation. To address this, we propose RM-Distiller, a framework designed to systematically exploit the multifaceted capabilities of teacher LLMs: (1) Refinement capability, which synthesizes highly correlated response pairs to create fine-grained and contrastive signals. (2) Scoring capability, which guides the RM in capturing precise preference strength via a margin-aware optimization objective. (3) Generation capability, which incorporates the teacher’s generative distribution to regularize the RM to preserve its fundamental linguistic knowledge. Extensive experiments demonstrate that RM-Distiller significantly outperforms traditional distillation methods both on RM benchmarks and reinforcement learning-based alignment, proving that exploiting multifaceted teacher capabilities is critical for effective reward modeling. To the best of our knowledge, this is the first systematic research on RM distillation from generative LLMs.

Method: Identifies 10 fundamental challenges (architectural inertia, gradient sparsity, linear reasoning limits) and proposes strategic roadmap with 4 pillars: foundational infrastructure, algorithmic optimization, cognitive reasoning, and unified multimodal intelligence.

Result: Proposes transition to diffusion-native ecosystem with multi-scale tokenization, active remasking, and latent thinking to overcome causal horizon constraints.

Conclusion: Shifting to diffusion-native ecosystem is essential for next-gen AI with complex structural reasoning, dynamic self-correction, and seamless multimodal integration beyond AR limitations.

Abstract: The paradigm of Large Language Models (LLMs) is currently defined by auto-regressive (AR) architectures, which generate text through a sequential brick-by-brick'' process. Despite their success, AR models are inherently constrained by a causal bottleneck that limits global structural foresight and iterative refinement. Diffusion Language Models (DLMs) offer a transformative alternative, conceptualizing text generation as a holistic, bidirectional denoising process akin to a sculptor refining a masterpiece. However, the potential of DLMs remains largely untapped as they are frequently confined within AR-legacy infrastructures and optimization frameworks. In this Perspective, we identify ten fundamental challenges ranging from architectural inertia and gradient sparsity to the limitations of linear reasoning that prevent DLMs from reaching their GPT-4 moment’’. We propose a strategic roadmap organized into four pillars: foundational infrastructure, algorithmic optimization, cognitive reasoning, and unified multimodal intelligence. By shifting toward a diffusion-native ecosystem characterized by multi-scale tokenization, active remasking, and latent thinking, we can move beyond the constraints of the causal horizon. We argue that this transition is essential for developing next-generation AI capable of complex structural reasoning, dynamic self-correction, and seamless multimodal integration.

Method: Systematic analysis of MoE models examining routing behavior and expert specialization across languages and network depth, followed by layerwise interventions and development of a routing-guided steering method.

Result: Multilingual processing in MoE models is highly structured: routing aligns with linguistic families, expert utilization follows layerwise patterns, high-resource languages use shared experts while low-resource languages rely more on language-exclusive experts. Routing-guided steering improves multilingual performance, especially for linguistically related language pairs.

Conclusion: MoE models organize multilingual processing in a structured way, with early/late layers handling language-specific processing and middle layers serving as language-agnostic capacity hubs. The proposed routing-guided steering method effectively leverages these insights to improve multilingual performance.

Abstract: Mixture-of-Experts (MoE) architectures have shown strong multilingual capabilities, yet the internal mechanisms underlying performance gains and cross-language differences remain insufficiently understood. In this work, we conduct a systematic analysis of MoE models, examining routing behavior and expert specialization across languages and network depth. Our analysis reveals that multilingual processing in MoE models is highly structured: routing aligns with linguistic families, expert utilization follows a clear layerwise pattern, and high-resource languages rely on shared experts while low-resource languages depend more on language-exclusive experts despite weaker performance. Layerwise interventions further show that early and late MoE layers support language-specific processing, whereas middle layers serve as language-agnostic capacity hubs. Building on these insights, we propose a routing-guided steering method that adaptively guides routing behavior in middle layers toward shared experts associated with dominant languages at inference time, leading to consistent multilingual performance improvements, particularly for linguistically related language pairs. Our code is available at https://github.com/conctsai/Multilingualism-in-Mixture-of-Experts-LLMs.

Method: Uses a large teacher model to generate synthetic prompts and translate instruction datasets based only on language name input, then trains small language models on this generated data for 54 low-resource languages.

Result: Successfully produced training data and SLMs for 54 languages, with evaluations showing consistent performance improvements over base models across translation, classification, and question answering tasks, all for under $50 per language.

Conclusion: Kakugo provides a practical, affordable solution for democratizing AI development for low-resource languages, enabling communities to create their own language models with minimal cost and technical barriers.

Abstract: We present Kakugo, a novel and cost-effective pipeline designed to train general-purpose Small Language Models (SLMs) for low-resource languages using only the language name as input. By using a large teacher model to generate synthetic prompts and translate instruction datasets, we produced training data and SLMs for 54 low-resource languages. Evaluations across a diverse set of general natural language processing tasks, including translation, classification, and question answering, demonstrate that our pipeline consistently improves performance over base models. With a total generation and training cost of under $50 per language, Kakugo offers an accessible method for communities to develop language-specific AI.

Method: Created XCR-Bench benchmark integrating Newmark’s CSI framework with Hall’s Triad of Culture, covering 4.9k parallel sentences and 1,098 unique CSIs across three reasoning tasks with corresponding evaluation metrics.

Result: State-of-the-art LLMs show consistent weaknesses in identifying and adapting CSIs related to social etiquette and cultural references, and encode regional and ethno-religious biases even within single linguistic settings during cultural adaptation.

Conclusion: The benchmark reveals significant gaps in LLMs’ cross-cultural reasoning capabilities and biases, providing a valuable resource for future research in cross-cultural NLP through released corpus and code.

Abstract: Cross-cultural competence in large language models (LLMs) requires the ability to identify Culture-Specific Items (CSIs) and to adapt them appropriately across cultural contexts. Progress in evaluating this capability has been constrained by the scarcity of high-quality CSI-annotated corpora with parallel cross-cultural sentence pairs. To address this limitation, we introduce XCR-Bench, a Cross(X)-Cultural Reasoning Benchmark consisting of 4.9k parallel sentences and 1,098 unique CSIs, spanning three distinct reasoning tasks with corresponding evaluation metrics. Our corpus integrates Newmark’s CSI framework with Hall’s Triad of Culture, enabling systematic analysis of cultural reasoning beyond surface-level artifacts and into semi-visible and invisible cultural elements such as social norms, beliefs, and values. Our findings show that state-of-the-art LLMs exhibit consistent weaknesses in identifying and adapting CSIs related to social etiquette and cultural reference. Additionally, we find evidence that LLMs encode regional and ethno-religious biases even within a single linguistic setting during cultural adaptation. We release our corpus and code to facilitate future research on cross-cultural NLP.

Method: Analyzed extensive datasets from Community Notes (CNs), EUvsDisinfo, and Database of Known Fakes (DBKF) to quantify prevalence and types of persuasion techniques across different fact-checking platforms.

Result: No evidence that Community Notes contain higher average number of persuasion techniques than professional fact-checks. Identified systematic rhetorical differences reflecting institutional norms and topical coverage. Notes with more persuasive elements get slightly higher helpfulness ratings, but crowd raters effectively penalize problematic rhetorical techniques.

Conclusion: Community-written debunks are not more persuasive than professional ones, challenging prior assumptions. Crowd evaluation systems can effectively moderate persuasive language, and institutional differences shape rhetorical approaches in fact-checking.

Abstract: This study presents the first large-scale comparison of persuasion techniques present in crowd- versus professionally-written debunks. Using extensive datasets from Community Notes (CNs), EUvsDisinfo, and the Database of Known Fakes (DBKF), we quantify the prevalence and types of persuasion techniques across these fact-checking ecosystems. Contrary to prior hypothesis that community-produced debunks rely more heavily on subjective or persuasive wording, we find no evidence that CNs contain a higher average number of persuasion techniques than professional fact-checks. We additionally identify systematic rhetorical differences between CNs and professional debunking efforts, reflecting differences in institutional norms and topical coverage. Finally, we examine how the crowd evaluates persuasive language in CNs and show that, although notes with more persuasive elements receive slightly higher overall helpfulness ratings, crowd raters are effective at penalising the use of particular problematic rhetorical means

Method: Introduces Explanation Network (ExpNet), a lightweight neural network that learns an explicit mapping from transformer attention patterns to token-level importance scores, automatically discovering optimal attention feature combinations.

Result: Evaluated in a challenging cross-task setting and benchmarked against a broad spectrum of model-agnostic methods and attention-based techniques spanning four methodological families.

Conclusion: ExpNet provides a more flexible and automated approach to transformer interpretability by learning optimal attention feature combinations rather than relying on predetermined rules, addressing limitations of existing XAI methods.

Abstract: Explainable AI (XAI) has become critical as transformer-based models are deployed in high-stakes applications including healthcare, legal systems, and financial services, where opacity hinders trust and accountability. Transformers self-attention mechanisms have proven valuable for model interpretability, with attention weights successfully used to understand model focus and behavior (Xu et al., 2015); (Wiegreffe and Pinter, 2019). However, existing attention-based explanation methods rely on manually defined aggregation strategies and fixed attribution rules (Abnar and Zuidema, 2020a); (Chefer et al., 2021), while model-agnostic approaches (LIME, SHAP) treat the model as a black box and incur significant computational costs through input perturbation. We introduce Explanation Network (ExpNet), a lightweight neural network that learns an explicit mapping from transformer attention patterns to token-level importance scores. Unlike prior methods, ExpNet discovers optimal attention feature combinations automatically rather than relying on predetermined rules. We evaluate ExpNet in a challenging cross-task setting and benchmark it against a broad spectrum of model-agnostic methods and attention-based techniques spanning four methodological families.

Method: NewsRECON uses a corpus of 90,000+ articles with: (1) bi-encoder for retrieving event-relevant articles, (2) two cross-encoders for reranking articles by location and event consistency. Can be combined with multimodal LLM.

Result: Outperforms prior work on TARA and 5Pils-OOC datasets. Achieves new SOTA results in absence of RIS evidence when combined with multimodal LLM.

Conclusion: NewsRECON provides effective alternative to RIS for news image verification, making code available for practical use by journalists and forensic experts.

Abstract: Identifying when and where a news image was taken is crucial for journalists and forensic experts to produce credible stories and debunk misinformation. While many existing methods rely on reverse image search (RIS) engines, these tools often fail to return results, thereby limiting their practical applicability. In this work, we address the challenging scenario where RIS evidence is unavailable. We introduce NewsRECON, a method that links images to relevant news articles to infer their date and location from article metadata. NewsRECON leverages a corpus of over 90,000 articles and integrates: (1) a bi-encoder for retrieving event-relevant articles; (2) two cross-encoders for reranking articles by location and event consistency. Experiments on the TARA and 5Pils-OOC show that NewsRECON outperforms prior work and can be combined with a multimodal large language model to achieve new SOTA results in the absence of RIS evidence. We make our code available.

Method: End-to-end evaluation on Natural Questions dataset systematically varying chunking method (token, sentence, semantic, code), chunk size, overlap, and context length using standard industrial setup with SPLADE retrieval and Mistral-8B generator.

Result: Four key actionable lessons: (1) overlap provides no measurable benefit and increases indexing cost; (2) sentence chunking is most cost-effective, matching semantic chunking up to ~5k tokens; (3) quality drops beyond ~2.5k tokens (“context cliff”); (4) optimal context depends on goal - semantic quality peaks at small contexts while exact match benefits from larger contexts.

Conclusion: Systematic chunking evaluation reveals practical guidelines for cost-efficient RAG deployment in industry, challenging common heuristics and providing evidence-based recommendations for optimal chunking strategies.

Abstract: We study how document chunking choices impact the reliability of Retrieval-Augmented Generation (RAG) systems in industry. While practice often relies on heuristics, our end-to-end evaluation on Natural Questions systematically varies chunking method (token, sentence, semantic, code), chunk size, overlap, and context length. We use a standard industrial setup: SPLADE retrieval and a Mistral-8B generator. We derive actionable lessons for cost-efficient deployment: (i) overlap provides no measurable benefit and increases indexing cost; (ii) sentence chunking is the most cost-effective method, matching semantic chunking up to ~5k tokens; (iii) a “context cliff” reduces quality beyond ~2.5k tokens; and (iv) optimal context depends on the goal (semantic quality peaks at small contexts; exact match at larger ones).

Method: Pretraining-free diffusion-based approach framing bias mitigation as style transfer. Used CARMA Arabic mental health corpus with gender imbalance, focusing on male-to-female style transfer. Created five datasets capturing different linguistic/semantic aspects of gender expression and trained separate diffusion models for each.

Result: Quantitative evaluations show high semantic fidelity between source and generated text with meaningful stylistic divergence. Qualitative analysis confirms linguistically plausible gender transformations. Approach generates high-entropy, semantically faithful synthetic data without pretrained LLMs.

Conclusion: Diffusion-based style transfer provides effective and flexible framework for mitigating gender bias in sensitive, low-resource mental health domains, offering alternative to LLM-based approaches.

Abstract: Synthetic data offers a promising solution for mitigating data scarcity and demographic bias in mental health analysis, yet existing approaches largely rely on pretrained large language models (LLMs), which may suffer from limited output diversity and propagate biases inherited from their training data. In this work, we propose a pretraining-free diffusion-based approach for synthetic text generation that frames bias mitigation as a style transfer problem. Using the CARMA Arabic mental health corpus, which exhibits a substantial gender imbalance, we focus on male-to-female style transfer to augment underrepresented female-authored content. We construct five datasets capturing varying linguistic and semantic aspects of gender expression in Arabic and train separate diffusion models for each setting. Quantitative evaluations demonstrate consistently high semantic fidelity between source and generated text, alongside meaningful surface-level stylistic divergence, while qualitative analysis confirms linguistically plausible gender transformations. Our results show that diffusion-based style transfer can generate high-entropy, semantically faithful synthetic data without reliance on pretrained LLMs, providing an effective and flexible framework for mitigating gender bias in sensitive, low-resource mental health domains.

Method: Conducted systematic architectural analysis of LLMs on multiple-choice QA tasks, comparing CQO (context-question-options) vs QOC (question-options-context) ordering across various models and datasets to identify underlying mechanisms.

Result: Found that causal attention masking is the core mechanism: in QOC prompts, the causal mask prevents option tokens from attending to context, creating an information bottleneck where context becomes invisible to options, explaining the performance gap.

Conclusion: Prompt structure significantly impacts LLM performance due to architectural constraints like causal attention masking, revealing important insights about how information flow in transformers affects reasoning capabilities in multi-step tasks.

Abstract: Large language models exhibit surprising sensitivity to the structure of the prompt, but the mechanisms underlying this sensitivity remain poorly understood. In this work, we conduct an in-depth investigation on a striking case: in multiple-choice question answering, placing context before the questions and options (CQO) outperforms the reverse order (QOC) by over 14%p, consistently over a wide range of models and datasets. Through systematic architectural analysis, we identify causal attention as the core mechanism: in QOC prompts, the causal mask prevents option tokens from attending to context, creating an information bottleneck where context becomes invisible to options.

Method: A novel synthetic data generation approach that systematically produces high-quality, diverse, and legally accurate question-answer pairs directly from authoritative German statutes. Uses rigorous automated filtering methods and parameter-efficient fine-tuning techniques.

Result: LLMs adapted with the synthetic dataset significantly outperform their baseline counterparts on German legal question answering tasks.

Conclusion: Carefully designed synthetic data can serve as a robust alternative to manual annotation in high-stakes, knowledge-intensive domains like legal reasoning.

Abstract: Large language models (LLMs) often struggle in specialized domains such as legal reasoning due to limited expert knowledge, resulting in factually incorrect outputs or hallucinations. This paper presents an effective method for adapting advanced LLMs to German legal question answering through a novel synthetic data generation approach. In contrast to costly human-annotated resources or unreliable synthetic alternatives, our approach systematically produces high-quality, diverse, and legally accurate question-answer pairs directly from authoritative German statutes. Using rigorous automated filtering methods and parameter-efficient fine-tuning techniques, we demonstrate that LLMs adapted with our synthetic dataset significantly outperform their baseline counterparts on German legal question answering tasks. Our results highlight the feasibility of using carefully designed synthetic data as a robust alternative to manual annotation in high-stakes, knowledge-intensive domains.

Method: 1) Binary moral presence detection; 2) Comparison of presence-gated hierarchy vs direct multi-label classifier using DeBERTa-base with lightweight signals (prior-sentence context, lexica, topic features); 3) Benchmarking instruction-tuned LLMs (Gemma 2, Llama 3.1, Mistral, Qwen) in zero/few-shot and QLoRA setups; 4) Building simple ensembles.

Result: Binary moral presence detection achieves positive-class F1 ≈ 0.74. Hierarchy doesn’t outperform direct prediction. Soft-vote supervised ensemble reaches macro-F1 0.332, surpassing best single supervised model and prior baselines. Lightweight signals and small ensembles yield most reliable improvements.

Conclusion: Under 8GB GPU constraints, carefully tuned supervised encoders remain strong and compute-efficient for structured human value detection. Hierarchical gating offers limited benefit, while richer value structure and document context could further improve performance.

Abstract: We study sentence-level identification of the 19 values in the Schwartz motivational continuum as a concrete formulation of human value detection in text. The setting - out-of-context sentences from news and political manifestos - features sparse moral cues and severe class imbalance. This combination makes fine-grained sentence-level value detection intrinsically difficult, even for strong modern neural models. We first operationalize a binary moral presence task (“does any value appear?”) and show that it is learnable from single sentences (positive-class F1 $\approx$ 0.74 with calibrated thresholds). We then compare a presence-gated hierarchy to a direct multi-label classifier under matched compute, both based on DeBERTa-base and augmented with lightweight signals (prior-sentence context, LIWC-22/eMFD/MJD lexica, and topic features). The hierarchy does not outperform direct prediction, indicating that gate recall limits downstream gains. We also benchmark instruction-tuned LLMs - Gemma 2 9B, Llama 3.1 8B, Mistral 8B, and Qwen 2.5 7B - in zero-/few-shot and QLoRA setups and build simple ensembles; a soft-vote supervised ensemble reaches macro-F1 0.332, significantly surpassing the best single supervised model and exceeding prior English-only baselines. Overall, in this scenario, lightweight signals and small ensembles yield the most reliable improvements, while hierarchical gating offers limited benefit. We argue that, under an 8 GB single-GPU constraint and at the 7-9B scale, carefully tuned supervised encoders remain a strong and compute-efficient baseline for structured human value detection, and we outline how richer value structure and sentence-in-document context could further improve performance.

Method: Proposes small auxiliary networks (residual probes) that read hallucination risk from intermediate hidden states of question tokens. These probes are computationally cheap, can run in parallel with inference, and enable fast risk estimation.

Result: Achieves strong AUROC and AURAC across four QA benchmarks and multiple LLM families, generalizes under dataset shift, and reveals interpretable structure in intermediate representations.

Conclusion: Fast internal uncertainty readout via residual probes provides a principled foundation for reliable agentic AI, enabling selective generation and routing where confident queries get immediate answers while uncertain ones go to verification pipelines.

Abstract: Hallucination in large language models (LLMs) can be understood as a failure of faithful readout: although internal representations may encode uncertainty about a query, decoding pressures still yield a fluent answer. We propose lightweight residual probes that read hallucination risk directly from intermediate hidden states of question tokens, motivated by the hypothesis that these layers retain epistemic signals that are attenuated in the final decoding stage. The probe is a small auxiliary network whose computation is orders of magnitude cheaper than token generation and can be evaluated fully in parallel with inference, enabling near-instantaneous hallucination risk estimation with effectively zero added latency in low-risk cases. We deploy the probe as an agentic critic for fast selective generation and routing, allowing LLMs to immediately answer confident queries while delegating uncertain ones to stronger verification pipelines. Across four QA benchmarks and multiple LLM families, the method achieves strong AUROC and AURAC, generalizes under dataset shift, and reveals interpretable structure in intermediate representations, positioning fast internal uncertainty readout as a principled foundation for reliable agentic AI.

Method: MASCOT uses a bi-level optimization strategy with two components: 1) Persona-Aware Behavioral Alignment - an RLAIF-driven pipeline that finetunes individual agents for strict persona fidelity, and 2) Collaborative Dialogue Optimization - a meta-policy guided by group-level rewards to ensure diverse and productive discourse.

Result: Extensive evaluations across psychological support and workplace domains show MASCOT significantly outperforms state-of-the-art baselines, achieving improvements of up to +14.1 in Persona Consistency and +10.6 in Social Contribution.

Conclusion: MASCOT provides a practical roadmap for engineering the next generation of socially intelligent multi-agent systems that maintain individual identities while enabling productive group collaboration.

Abstract: Multi-agent systems (MAS) have recently emerged as promising socio-collaborative companions for emotional and cognitive support. However, these systems frequently suffer from persona collapse–where agents revert to generic, homogenized assistant behaviors–and social sycophancy, which produces redundant, non-constructive dialogue. We propose MASCOT, a generalizable framework for multi-perspective socio-collaborative companions. MASCOT introduces a novel bi-level optimization strategy to harmonize individual and collective behaviors: 1) Persona-Aware Behavioral Alignment, an RLAIF-driven pipeline that finetunes individual agents for strict persona fidelity to prevent identity loss; and 2) Collaborative Dialogue Optimization, a meta-policy guided by group-level rewards to ensure diverse and productive discourse. Extensive evaluations across psychological support and workplace domains demonstrate that MASCOT significantly outperforms state-of-the-art baselines, achieving improvements of up to +14.1 in Persona Consistency and +10.6 in Social Contribution. Our framework provides a practical roadmap for engineering the next generation of socially intelligent multi-agent systems.

Method: Created APEX-Agents benchmark with 480 tasks requiring agents to navigate realistic work environments with files and tools. Tested eight agents using Pass@1 metric. Also developed Archipelago infrastructure for agent execution and evaluation. All prompts, rubrics, gold outputs, files, and metadata are open-sourced.

Result: Gemini 3 Flash (Thinking=High) achieved the highest score of 24.0%, followed by GPT-5.2 (Thinking=High), Claude Opus 4.5 (Thinking=High), and Gemini 3 Pro (Thinking=High). Performance shows current AI agents still have significant room for improvement on complex professional tasks.

Conclusion: APEX-Agents provides a comprehensive benchmark for evaluating AI agents on realistic professional work, revealing current capabilities and limitations. The open-source release enables further research and development in agent systems for complex, cross-application tasks.

Abstract: We introduce the AI Productivity Index for Agents (APEX-Agents), a benchmark for assessing whether AI agents can execute long-horizon, cross-application tasks created by investment banking analysts, management consultants, and corporate lawyers. APEX-Agents requires agents to navigate realistic work environments with files and tools. We test eight agents for the leaderboard using Pass@1. Gemini 3 Flash (Thinking=High) achieves the highest score of 24.0%, followed by GPT-5.2 (Thinking=High), Claude Opus 4.5 (Thinking=High), and Gemini 3 Pro (Thinking=High). We open source the APEX-Agents benchmark (n=480) with all prompts, rubrics, gold outputs, files, and metadata. We also open-source Archipelago, our infrastructure for agent execution and evaluation.

Method: Propose Rank-Surprisal Ratio (RSR) - ratio of average token-wise rank to average negative log-likelihood - capturing both alignment (via rank) and informativeness (via surprisal).

Result: RSR strongly correlates with post-training performance (average Spearman 0.86 across 5 students and 11 teachers), outperforms existing metrics, and works well for trajectory and teacher selection.

Conclusion: RSR effectively assesses reasoning trajectory suitability for LLM distillation by balancing learning signal strength and behavioral alignment, providing practical utility for trajectory and teacher selection.

Abstract: Long chain-of-thought (CoT) trajectories provide rich supervision signals for distilling reasoning from teacher to student LLMs. However, both prior work and our experiments show that trajectories from stronger teachers do not necessarily yield better students, highlighting the importance of data-student suitability in distillation. Existing methods assess suitability primarily through student likelihood, favoring trajectories that closely align with the model’s current behavior but overlooking more informative ones. Addressing this, we propose Rank-Surprisal Ratio (RSR), a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory. RSR is motivated by the observation that effective trajectories typically combine low absolute probability with relatively high-ranked tokens under the student model, balancing learning signal strength and behavioral alignment. Concretely, RSR is defined as the ratio of a trajectory’s average token-wise rank to its average negative log-likelihood, and is straightforward to compute and interpret. Across five student models and reasoning trajectories from 11 diverse teachers, RSR strongly correlates with post-training performance (average Spearman 0.86), outperforming existing metrics. We further demonstrate its practical utility in both trajectory selection and teacher selection.

Method: Created MMT dataset with 1.7 million tweets covering 13 coarse-grained and 63 fine-grained topics in Indian context, annotated 5,346 tweets for Indian languages and code-mixed content, and evaluated existing tools on topic modeling and language identification tasks.

Result: Existing NLP tools fail to adequately capture linguistic diversity in the MMT dataset, demonstrating the need for better multilingual and code-mixed processing capabilities. The dataset is publicly available for research.

Conclusion: The MMT dataset addresses a critical gap in multilingual and code-mixed social media research, particularly for Indian languages, and highlights the need for improved NLP tools to handle linguistic diversity in real-world social media data.

Abstract: Social media plays a significant role in cross-cultural communication. A vast amount of this occurs in code-mixed and multilingual form, posing a significant challenge to Natural Language Processing (NLP) tools for processing such information, like language identification, topic modeling, and named-entity recognition. To address this, we introduce a large-scale multilingual, and multi-topic dataset (MMT) collected from Twitter (1.7 million Tweets), encompassing 13 coarse-grained and 63 fine-grained topics in the Indian context. We further annotate a subset of 5,346 tweets from the MMT dataset with various Indian languages and their code-mixed counterparts. Also, we demonstrate that the currently existing tools fail to capture the linguistic diversity in MMT on two downstream tasks, i.e., topic modeling and language identification. To facilitate future research, we have make the anonymized and annotated dataset available at https://huggingface.co/datasets/LingoIITGN/MMT.

Method: Developed a matrix with five categories (Very LRL, LRL, RL, HRL, Very HRL) based on typology of contextual features rather than counting tools. Provided motivation for each feature and characterization, with focus on African languages.

Result: Proposed a more nuanced 5-level scale for characterizing language resources that considers contextual features. This contextualization helps better understand where languages fall on the resource spectrum, particularly for African languages.

Conclusion: Characterizing language resources within a given scale is indispensable for project planning, especially for languages in the lower half of the scale. The proposed matrix provides better understanding of language resourcedness for research and implementation projects.

Abstract: Several widely used software applications involve some form of processing of natural language, with tasks ranging from digitising hardcopies and text processing to speech generation. Varied language resources are used to develop software systems to accomplish a wide range of natural language processing (NLP) tasks, such as the ubiquitous spellcheckers and chatbots. Languages are typically characterised as either low (LRL) or high resourced languages (HRL) with African languages having been characterised as resource-scarce languages and English by far the most well-resourced language. But what lies in-between? We argue that the dichotomous typology of LRL and HRL for all languages is problematic. Through a clear understanding of language resources situated in a society, a matrix is developed that characterises languages as Very LRL, LRL, RL, HRL and Very HRL. The characterisation is based on the typology of contextual features for each category, rather than counting tools. The motivation is provided for each feature and each characterisation. The contextualisation of resourcedness, with a focus on African languages in this paper, and an increased understanding of where on the scale the language used in a project is, may assist in, among others, better planning of research and implementation projects. We thus argue in this paper that the characterisation of language resources within a given scale in a project is an indispensable component, particularly for those in the lower half of the scale.

Method: Proposes GNN2R (Graph Neural Network-based Two-Step Reasoning) that uses only weak supervision from widely-available final answer annotations. It efficiently retrieves both final answers and corresponding reasoning subgraphs as verifiable rationales through a two-step reasoning approach.

Result: GNN2R substantially outperforms existing state-of-the-art KGQA methods in terms of effectiveness, efficiency, and the quality of generated explanations.

Conclusion: GNN2R addresses the explainability gap in KGQA by providing verifiable reasoning subgraphs alongside answers, using only weak supervision, while maintaining high efficiency and outperforming existing methods across multiple metrics.

Abstract: Despite the rapid progress of large language models (LLMs), knowledge graph-based question answering (KGQA) remains essential for producing verifiable and hallucination-resistant answers in many real-world settings where answer trustworthiness and computational efficiency are highly valued. However, most existing KGQA methods provide only final answers in the form of KG entities. Without explicit explanations – ideally in the form of intermediate reasoning process over relevant KG triples, the QA results are difficult to inspect and interpret. Moreover, this limitation prevents the rich and verifiable knowledge encoded in KGs, which is a key advantage of KGQA over LLMs, from being fully leveraged. However, addressing this issue remains highly challenging due to the lack of annotated intermediate reasoning process and the requirement of high efficiency in KGQA. In this paper, we propose a novel Graph Neural Network-based Two-Step Reasoning method (GNN2R) that can efficiently retrieve both final answers and corresponding reasoning subgraphs as verifiable rationales, using only weak supervision from widely-available final answer annotations. We extensively evaluated GNN2R and demonstrated that GNN2R substantially outperforms existing state-of-the-art KGQA methods in terms of effectiveness, efficiency, and the quality of generated explanations. The complete code and pre-trained models are available at https://github.com/ruijie-wang-uzh/GNN2R.

Method: The paper provides a taxonomy of LLM memorization literature across three dimensions (granularity, retrievability, desirability), discusses metrics and methods for quantifying memorization, analyzes causes and contributing factors, and explores mitigation strategies.

Result: The survey organizes existing research on LLM memorization, identifies it as a fundamental source of privacy/security attacks, and maintains a dedicated repository of references that will be regularly updated to reflect latest developments.

Conclusion: The paper concludes by identifying future research directions including balancing privacy and performance, and analyzing memorization in specific LLM contexts like conversational agents, retrieval-augmented generation, and diffusion language models.

Abstract: While recent research increasingly showcases the remarkable capabilities of Large Language Models (LLMs), it is equally crucial to examine their associated risks. Among these, privacy and security vulnerabilities are particularly concerning, posing significant ethical and legal challenges. At the heart of these vulnerabilities stands memorization, which refers to a model’s tendency to store and reproduce phrases from its training data. This phenomenon has been shown to be a fundamental source to various privacy and security attacks against LLMs. In this paper, we provide a taxonomy of the literature on LLM memorization, exploring it across three dimensions: granularity, retrievability, and desirability. Next, we discuss the metrics and methods used to quantify memorization, followed by an analysis of the causes and factors that contribute to memorization phenomenon. We then explore strategies that are used so far to mitigate the undesirable aspects of this phenomenon. We conclude our survey by identifying potential research topics for the near future, including methods to balance privacy and performance, and the analysis of memorization in specific LLM contexts such as conversational agents, retrieval-augmented generation, and diffusion language models. Given the rapid research pace in this field, we also maintain a dedicated repository of the references discussed in this survey which will be regularly updated to reflect the latest developments.

Method: Introduces a framework to automatically generate an intention knowledge graph that captures connections between user intentions, using the Amazon m2 dataset to construct a graph with 351 million edges.

Result: The constructed intention graph demonstrates high plausibility and acceptance. The model effectively predicts new session intentions and enhances product recommendations, outperforming previous state-of-the-art methods.

Conclusion: The approach shows practical utility for online platforms by improving intention understanding, behavior modeling, and recommendation systems through connected intention knowledge graphs.

Abstract: Understanding user intentions is challenging for online platforms. Recent work on intention knowledge graphs addresses this but often lacks focus on connecting intentions, which is crucial for modeling user behavior and predicting future actions. This paper introduces a framework to automatically generate an intention knowledge graph, capturing connections between user intentions. Using the Amazon m2 dataset, we construct an intention graph with 351 million edges, demonstrating high plausibility and acceptance. Our model effectively predicts new session intentions and enhances product recommendations, outperforming previous state-of-the-art methods and showcasing the approach’s practical utility.

Method: MIRAGE framework includes eight intricately crafted murder mystery scripts with diverse themes and styles. It employs four evaluation methods: Trust Inclination Index (TII) for trust dynamics, Clue Investigation Capability (CIC) for information gathering, Interactivity Capability Index (ICI) for role-playing, and Script Compliance Index (SCI) for instruction following.

Result: Experiments show that even popular models like GPT-4 face significant challenges in navigating the complexities presented by MIRAGE, indicating current limitations in advanced human behavior simulation.

Conclusion: MIRAGE provides a comprehensive evaluation framework for assessing LLMs’ role-playing abilities, revealing current limitations and offering open-source datasets and simulation codes for further research in this domain.

Abstract: Large Language Models (LLMs) have shown remarkable capabilities in environmental perception, reasoning-based decision-making, and simulating complex human behaviors, particularly in interactive role-playing contexts. This paper introduces the Multiverse Interactive Role-play Ability General Evaluation (MIRAGE), a comprehensive framework designed to assess LLMs’ proficiency in portraying advanced human behaviors through murder mystery games. MIRAGE features eight intricately crafted scripts encompassing diverse themes and styles, providing a rich simulation. To evaluate LLMs’ performance, MIRAGE employs four distinct methods: the Trust Inclination Index (TII) to measure dynamics of trust and suspicion, the Clue Investigation Capability (CIC) to measure LLMs’ capability of conducting information, the Interactivity Capability Index (ICI) to assess role-playing capabilities and the Script Compliance Index (SCI) to assess LLMs’ capability of understanding and following instructions. Our experiments indicate that even popular models like GPT-4 face significant challenges in navigating the complexities presented by the MIRAGE. The datasets and simulation codes are available in \href{https://github.com/lime728/MIRAGE}{github}.

Method: Mixed-methods approach combining social network analysis with qualitative content coding of 2,333 substance-related hashtags and 351 representative videos, identifying 16 distinct hashtag communities and characterizing structural/thematic relationships.

Result: Network analysis reveals highly interconnected small-world structure with recovery-focused hashtags (#addiction, #recovery, #sober) as central bridges. Recovery Advocacy (33.9%) and Satirical content (28.2%) dominate videos, while direct substance depiction appears in only 26% (active use in just 6.5%).

Conclusion: Algorithmically surfaced substance discourse on TikTok is oriented toward recovery/support rather than substance promotion, shaped by organic community formation within platform affordances rather than adversarial evasion of moderation, showing how algorithmic visibility shapes recovery community organization.

Abstract: TikTok has emerged as a major source of information and social interaction for youth, raising urgent questions about how substance use discourse manifests and circulates on the platform. This paper presents the first comprehensive analysis of publicly visible, algorithmically surfaced substance-related content on TikTok, drawing on hashtags spanning all major substance categories. Using a mixed-methods approach that combines social network analysis with qualitative content coding, we examined 2,333 substance-related hashtags, identifying 16 distinct hashtag communities and characterizing their structural and thematic relationships. Our network analysis reveals a highly interconnected small-world structure in which recovery-focused hashtags such as \textit{#addiction}, \textit{#recovery}, and \textit{#sober} serve as central bridges between communities. Qualitative analysis of 351 representative videos shows that Recovery Advocacy content (33.9%) and Satirical content (28.2%) dominate, while direct substance depiction appears in only 26% of videos, with active use shown in just 6.5% of them. These findings suggest that the algorithmically surfaced layer of substance-related discourse on TikTok is predominantly oriented toward recovery, support, and coping rather than explicit promotion of substance use. We further show that hashtag communities and video content are closely aligned, indicating that substance-related discourse on TikTok is shaped through organic community formation within platform affordances rather than widespread adversarial evasion of moderation. This work contributes to social computing research by showing how algorithmic visibility on TikTok shapes the organization of substance-related discourse and the formation of recovery and support communities.

Method: Proposes BLUE (Boundary Layer Update) strategy to enhance locate-then-edit methods by addressing the residual distribution errors through theoretical and empirical analysis, focusing on boundary layer updates rather than multi-layer residual distribution.

Result: BLUE achieves 35.59% average performance improvement in sequential batch editing experiments across three LLMs and two datasets, significantly advancing state-of-the-art in model editing while better preserving LLMs’ general capabilities.

Conclusion: BLUE effectively addresses the limitations of residual distribution in locate-then-edit methods, providing a more precise and robust approach to model editing that maintains both editing accuracy and model generalization capabilities.

Abstract: Model editing enables targeted updates to the knowledge of large language models (LLMs) with minimal retraining. Among existing approaches, locate-then-edit methods constitute a prominent paradigm: they first identify critical layers, then compute residuals at the final critical layer based on the target edit, and finally apply least-squares-based multi-layer updates via $\textbf{residual distribution}$. While empirically effective, we identify a counterintuitive failure mode: residual distribution, a core mechanism in these methods, introduces weight shift errors that undermine editing precision. Through theoretical and empirical analysis, we show that such errors increase with the distribution distance, batch size, and edit sequence length, ultimately leading to inaccurate or suboptimal edits. To address this, we propose the $\textbf{B}$oundary $\textbf{L}$ayer $\textbf{U}$pdat$\textbf{E (BLUE)}$ strategy to enhance locate-then-edit methods. Sequential batch editing experiments on three LLMs and two datasets demonstrate that BLUE not only delivers an average performance improvement of 35.59%, significantly advancing the state of the art in model editing, but also enhances the preservation of LLMs’ general capabilities. Our code is available at https://github.com/xpq-tech/BLUE.

Method: Developed JingFang (JF) with a Multi-Agent Collaborative Consultation Mechanism (MACCM) that integrates various TCM Specialist Agents to emulate real-world diagnostic workflows. Includes a dedicated Syndrome Differentiation Agent fine-tuned on preprocessed data, and a Dual-Stage Recovery Scheme (DSRS) within the Treatment Agent.

Result: JF demonstrates superior performance in medical consultation and shows improvements of at least 124% in syndrome differentiation precision compared to existing TCM models, and 21.1% improvement compared to State of the Art LLMs.

Conclusion: JF successfully addresses critical limitations in current TCM AI systems by providing personalized consultations, accurate syndrome differentiation, and collaborative diagnostic workflows, representing a significant advancement in AI-assisted TCM diagnosis and treatment.

Abstract: The practice of Traditional Chinese Medicine (TCM) requires profound expertise and extensive clinical experience. While Large Language Models (LLMs) offer significant potential in this domain, current TCM-oriented LLMs suffer two critical limitations: (1) a rigid consultation framework that fails to conduct comprehensive and patient-tailored interactions, often resulting in diagnostic inaccuracies; and (2) treatment recommendations generated without rigorous syndrome differentiation, which deviates from the core diagnostic and therapeutic principles of TCM. To address these issues, we develop \textbf{JingFang (JF)}, an advanced LLM-based multi-agent system for TCM that facilitates the implementation of AI-assisted TCM diagnosis and treatment. JF integrates various TCM Specialist Agents in accordance with authentic diagnostic and therapeutic scenarios of TCM, enabling personalized medical consultations, accurate syndrome differentiation and treatment recommendations. A \textbf{Multi-Agent Collaborative Consultation Mechanism (MACCM)} for TCM is constructed, where multiple Agents collaborate to emulate real-world TCM diagnostic workflows, enhancing the diagnostic ability of base LLMs to provide accurate and patient-tailored medical consultation. Moreover, we introduce a dedicated \textbf{Syndrome Differentiation Agent} fine-tuned on a preprocessed dataset, along with a designed \textbf{Dual-Stage Recovery Scheme (DSRS)} within the Treatment Agent, which together substantially improve the model’s accuracy of syndrome differentiation and treatment. Comprehensive evaluations and experiments demonstrate JF’s superior performance in medical consultation, and also show improvements of at least 124% and 21.1% in the precision of syndrome differentiation compared to existing TCM models and State of the Art (SOTA) LLMs, respectively.

Method: Proposed Personality Structured Interview (PSI) method that incorporates psychological insights into LLM simulation using psychometric scale-development procedures to capture personality-related linguistic information from a formal psychological perspective.

Result: Three experiments show PSI effectively improves human-like heterogeneity in LLM-simulated personality data and predicts personality-related behavioral outcomes. The method demonstrates reliability, structural validity, and external validity.

Conclusion: PSI provides a theoretical framework for designing theory-informed structured interviews to enhance LLM reliability and effectiveness in simulating human-like data for broader psychometric research, addressing the diversity gap in LLM simulations.

Abstract: Despite their potential as human proxies, LLMs often fail to generate heterogeneous data with human-like diversity, thereby diminishing their value in advancing social science research. To address this gap, we propose a novel method to incorporate psychological insights into LLM simulation through the Personality Structured Interview (PSI). PSI leverages psychometric scale-development procedures to capture personality-related linguistic information from a formal psychological perspective. To systematically evaluate simulation fidelity, we developed a measurement theory grounded evaluation procedure that considers the latent construct nature of personality and evaluates its reliability, structural validity, and external validity. Results from three experiments demonstrate that PSI effectively improves human-like heterogeneity in LLM-simulated personality data and predicts personality-related behavioral outcomes. We further offer a theoretical framework for designing theory-informed structured interviews to enhance the reliability and effectiveness of LLMs in simulating human-like data for broader psychometric research.

Method: Created WildFrame dataset with 1,000 real-world texts conveying clear sentiment, reframed each text positively/negatively, collected human sentiment annotations, and evaluated 11 LLMs on the same data.

Result: All LLMs responded to reframing in human-like manner (r≥0.52), both humans and models influenced more by positive than negative reframing, GPT models were least correlated with human behavior among tested models.

Conclusion: Findings raise important questions about LLM development goals: whether models should align with human behavior (preserving framing effects) or mitigate such biases for fairness and consistency.

Abstract: Humans are influenced by how information is presented, a phenomenon known as the framing effect. Prior work suggests that LLMs may also be susceptible to framing, but it has relied on synthetic data and did not compare to human behavior. To address this gap, we introduce WildFrame - a dataset for evaluating LLM responses to positive and negative framing in naturally-occurring sentences, alongside human responses on the same data. WildFrame consists of 1,000 real-world texts selected to convey a clear sentiment; we then reframe each text in either a positive or negative light and collect human sentiment annotations. Evaluating eleven LLMs on WildFrame, we find that all models respond to reframing in a human-like manner ($r\geq0.52$), and that both humans and models are influenced more by positive than negative reframing. Notably, GPT models are the least correlated with human behavior among all tested models. These findings raise a discussion around the goals of state-of-the-art LLM development and whether models should align closely with human behavior, to preserve cognitive phenomena such as the framing effect, or instead mitigate such biases in favor of fairness and consistency.

Method: Developed a novel annotation scheme based on Incongruity Theory, Conceptual Metaphor Theory, and VU Amsterdam Metaphor Corpus; created Hummus Dataset with 1k annotated image-caption pairs from New Yorker Caption Contest; tested state-of-the-art MLLMs.

Result: Current MLLMs struggle with processing humorous multimodal metaphors, particularly in integrating visual and textual information.

Conclusion: The study provides a valuable dataset for humor and metaphor research and highlights limitations in current MLLMs’ multimodal understanding capabilities.

Abstract: Metaphor and humor share a lot of common ground, and metaphor is one of the most common humorous mechanisms. This study focuses on the humorous capacity of multimodal metaphors, which has not received due attention in the community. We take inspiration from the Incongruity Theory of humor, the Conceptual Metaphor Theory, and the annotation scheme behind the VU Amsterdam Metaphor Corpus, and developed a novel annotation scheme for humorous multimodal metaphor use in image-caption pairs. We create the Hummus Dataset of Humorous Multimodal Metaphor Use, providing expert annotation on 1k image-caption pairs sampled from the New Yorker Caption Contest corpus. Using the dataset, we test state-of-the-art multimodal large language models (MLLMs) on their ability to detect and understand humorous multimodal metaphor use. Our experiments show that current MLLMs still struggle with processing humorous multimodal metaphors, particularly with regard to integrating visual and textual information. We release our dataset and code at github.com/xiaoyuisrain/humorous-multimodal-metaphor-use.

Method: RainbowPlus extends classical evolutionary algorithms like MAP-Elites with adaptive quality-diversity search, using a multi-element archive to store diverse high-quality prompts and a comprehensive fitness function that evaluates multiple prompts concurrently.

Result: Achieves superior attack success rate (81.1% average ASR on HarmBench) and diversity (Diverse-Score ≈0.84), generates up to 100x more unique prompts than baseline methods, and is 9x faster than AutoDAN-Turbo (1.45 vs. 13.50 hours).

Conclusion: RainbowPlus provides a scalable, effective tool for LLM vulnerability assessment, advancing LLM safety research through its open-source implementation and superior performance over state-of-the-art red-teaming methods.

Abstract: Large Language Models (LLMs) exhibit remarkable capabilities but are susceptible to adversarial prompts that exploit vulnerabilities to produce unsafe or biased outputs. Existing red-teaming methods often face scalability challenges, resource-intensive requirements, or limited diversity in attack strategies. We propose RainbowPlus, a novel red-teaming framework rooted in evolutionary computation, enhancing adversarial prompt generation through an adaptive quality-diversity (QD) search that extends classical evolutionary algorithms like MAP-Elites with innovations tailored for language models. By employing a multi-element archive to store diverse high-quality prompts and a comprehensive fitness function to evaluate multiple prompts concurrently, RainbowPlus overcomes the constraints of single-prompt archives and pairwise comparisons in prior QD methods like Rainbow Teaming. Experiments comparing RainbowPlus to QD methods across six benchmark datasets and four open-source LLMs demonstrate superior attack success rate (ASR) and diversity (Diverse-Score $\approx 0.84$), generating up to 100 times more unique prompts (e.g., 10,418 vs. 100 for Ministral-8B-Instruct-2410). Against nine state-of-the-art methods on the HarmBench dataset with twelve LLMs (ten open-source, two closed-source), RainbowPlus achieves an average ASR of 81.1%, surpassing AutoDAN-Turbo by 3.9%, and is 9 times faster (1.45 vs. 13.50 hours). Our open-source implementation fosters further advancements in LLM safety, offering a scalable tool for vulnerability assessment. Code and resources are publicly available at https://github.com/knoveleng/rainbowplus, supporting reproducibility and future research in LLM red-teaming.

Method: The paper conducts a systematic analysis with a novel three-dimensional taxonomy: 1) learning methods (supervised, unsupervised, few-shot, zero-shot), 2) data modalities (unimodal, multimodal, hybrid), and 3) target relationships (in-target, cross-target, multi-target). It also examines evaluation techniques, benchmark datasets, and performance trends.

Result: The survey highlights emerging trends, analyzes strengths and limitations of different LLM architectures, and discusses key applications including misinformation detection, political analysis, public health monitoring, and social media moderation.

Conclusion: The paper identifies critical challenges (implicit stance expression, cultural biases, computational constraints) and outlines promising future directions (explainable stance reasoning, low-resource adaptation, real-time deployment frameworks) to guide development of next-generation stance detection systems powered by LLMs.

Abstract: Stance detection is essential for understanding subjective content across various platforms such as social media, news articles, and online reviews. Recent advances in Large Language Models (LLMs) have revolutionized stance detection by introducing novel capabilities in contextual understanding, cross-domain generalization, and multimodal analysis. Despite these progressions, existing surveys often lack comprehensive coverage of approaches that specifically leverage LLMs for stance detection. To bridge this critical gap, our review article conducts a systematic analysis of stance detection, comprehensively examining recent advancements of LLMs transforming the field, including foundational concepts, methodologies, datasets, applications, and emerging challenges. We present a novel taxonomy for LLM-based stance detection approaches, structured along three key dimensions: 1) learning methods, including supervised, unsupervised, few-shot, and zero-shot; 2) data modalities, such as unimodal, multimodal, and hybrid; and 3) target relationships, encompassing in-target, cross-target, and multi-target scenarios. Furthermore, we discuss the evaluation techniques and analyze benchmark datasets and performance trends, highlighting the strengths and limitations of different architectures. Key applications in misinformation detection, political analysis, public health monitoring, and social media moderation are discussed. Finally, we identify critical challenges such as implicit stance expression, cultural biases, and computational constraints, while outlining promising future directions, including explainable stance reasoning, low-resource adaptation, and real-time deployment frameworks. Our survey highlights emerging trends, open challenges, and future directions to guide researchers and practitioners in developing next-generation stance detection systems powered by large language models.

Method: Introduced a framework to decompose errors into missing vs. unused knowledge, using patent classification with dense technical language. Models generate clarifying questions and are tested in three settings: raw performance, self-answered questions, and externally provided answers.

Result: Most errors stem from failures to deploy existing knowledge rather than true knowledge gaps. Smaller models generate simpler, more effective questions, while larger models produce complex but less effective questions, showing complementary strengths across scales.

Conclusion: Evaluation should shift from static fact recall to dynamic knowledge application for a more informative view of model capabilities, highlighting the importance of knowledge deployment over mere knowledge possession.

Abstract: While large language models (LLMs) excel at factual recall, the real challenge lies in knowledge application. A gap persists between their ability to answer complex questions and their effectiveness in performing tasks that require that knowledge. We investigate this gap using a patent classification problem that requires deep conceptual understanding to distinguish semantically similar but objectively different patents written in dense, strategic technical language. We find that LLMs often struggle with this distinction. To diagnose the source of these failures, we introduce a framework that decomposes model errors into two categories: missing knowledge and unused knowledge. Our method prompts models to generate clarifying questions and compares three settings – raw performance, self-answered questions that activate internal knowledge, and externally provided answers that supply missing knowledge (if any). We show that most errors stem from failures to deploy existing knowledge rather than from true knowledge gaps. We also examine how models differ in constructing task-specific question-answer databases. Smaller models tend to generate simpler questions that they, and other models, can retrieve and use effectively, whereas larger models produce more complex questions that are less effective, suggesting complementary strengths across model scales. Together, our findings highlight that shifting evaluation from static fact recall to dynamic knowledge application offers a more informative view of model capabilities.

Method: Three-stage approach: (1) LLM preference-tuned to generate query-conditioned rationales using DPO, (2) Evidence Chunk Selection Engine uses rationales to select evidence with adaptive cutoff via elbow detection (optionally expands context), (3) Verifier LLM uses rationales to detect/filter poisoned evidence before generation.

Result: Achieves 13.41% higher recall and 21.05% higher precision than strongest baseline; reduces evidence needed for comparable recall by 80%; improves downstream answer accuracy by 33.34%; strengthens adversarial defense (F1 from 0.10 to 0.44).

Conclusion: METEORA successfully addresses critical limitations of current RAG systems by introducing rationale-driven selection, providing interpretability, robustness against poisoning, and significant performance improvements across multiple metrics.

Abstract: In sensitive domains, Retrieval-Augmented Generation (RAG) must be interpretable and robust because errors do not just mislead, they invite lawsuits, undermine scholarly credibility, and breach compliance. Stakeholders require traceable evidence, clear rationales for why specific evidence is selected, and safeguards against poisoned or misleading content. Yet current RAG pipelines rely on similarity-based retrieval with arbitrary top-k cutoffs, provide no explanation for selections, and remain vulnerable to poisoning attacks. We propose METEORA, which replaces these drawbacks with rationale-driven selection, using explicit reasoning to guide evidence choice, explain decisions, and improve robustness to RAG poisoning. METEORA operates in three stages: (1) a general-purpose LLM is preference-tuned to generate query-conditioned rationales using direct preference optimization; (2) these rationales drive an Evidence Chunk Selection Engine that pairs rationales with retrieved evidence for query-specific relevance and applies elbow detection to choose an adaptive cutoff (optionally expanding context with neighboring chunks); and (3) a Verifier LLM uses the rationales to detect and filter poisoned or misleading evidence before generation. Across six datasets, METEORA achieves 13.41% higher recall and, without expansion, 21.05% higher precision than the strongest baseline. It reduces the evidence needed for comparable recall by 80%, improving downstream answer accuracy by 33.34%, and strengthens adversarial defense by increasing F1 from 0.10 to 0.44. Code is available at: https://anonymous.4open.science/r/METEORA-DC46/README.md

Method: Introduces KNN-SSD algorithm that leverages K-Nearest Neighbor (KNN) search to match different skipped layers with various domain inputs, improving the domain adaptation of Self-Speculative Decoding.

Result: Evaluation across various models and multiple tasks shows the algorithm achieves 1.3x-1.6x speedup in LLM inference compared to baseline Self-Speculative Decoding.

Conclusion: KNN-SSD effectively addresses the domain sensitivity problem in Self-Speculative Decoding, providing a practical solution for improving LLM inference acceleration across different domains.

Abstract: Speculative Decoding (SD) has emerged as a widely used paradigm to accelerate the inference of large language models (LLMs) without compromising generation quality. It works by efficiently drafting multiple tokens using a compact model and then verifying them in parallel using the target LLM. Notably, Self-Speculative Decoding proposes skipping certain layers to construct the draft model, which eliminates the need for additional parameters or training. Despite its strengths, we observe in this work that drafting with layer skipping exhibits significant sensitivity to domain shifts, leading to a substantial drop in acceleration performance. To enhance the domain generalizability of this paradigm, we introduce KNN-SSD, an algorithm that leverages K-Nearest Neighbor (KNN) search to match different skipped layers with various domain inputs. We evaluated our algorithm in various models and multiple tasks, observing that its application leads to 1.3x-1.6x speedup in LLM inference.

Method: Used model-specific testbeds, measured momentary belief via internal state probes trained on external datasets, conducted steering experiments to test causality, and applied supervised fine-tuning.

Result: LLMs retract rarely; retraction is predicted by momentary internal belief (not parametric knowledge); belief causally drives retraction and affects attention dynamics; fine-tuning improves retraction by aligning belief.

Conclusion: Retraction depends on LLMs’ momentary internal belief during generation, which often diverges from stored knowledge; improving belief accuracy via fine-tuning enhances retraction behavior.

Abstract: Can large language models (LLMs) admit their mistakes when they should know better? In this work, we study when and why LLMs choose to retract, i.e., spontaneously and immediately acknowledge their errors. Using model-specific testbeds, we find that while LLMs are capable of retraction, they do so only rarely, even when they can recognize their mistakes when asked in a separate interaction. We identify a reliable predictor of retraction: the model’s momentary belief, as measured by a probe on its internal states that is trained to predict correctness on external datasets unrelated to retraction. A model retracts only when it “believes” its answers to be incorrect during generation; these beliefs frequently diverge from models’ parametric knowledge as measured by factoid questions. Steering experiments further demonstrate that model belief causally drives retraction. In particular, when the model believes its answer to be incorrect, this not only encourages the model to attempt further verification, but also alters attention dynamics. Finally, we show that supervised fine-tuning improves retraction performance by helping the model learn more accurate internal belief. Code and datasets are available on https://github.com/ayyyq/llm-retraction .

Method: Created EVADE benchmark with expert-curated Chinese multimodal dataset (text and images) across six product categories. Two tasks: Single-Violation (fine-grained reasoning with short prompts) and All-in-One (long-context reasoning with unified instructions merging overlapping rules).

Result: Benchmarked 26 mainstream LLMs/VLMs showing substantial performance gaps. Even state-of-the-art models frequently misclassify evasive samples. All-in-One setting significantly narrows performance gap between partial and full-match accuracy, suggesting clearer rule definitions improve alignment.

Conclusion: EVADE provides the first rigorous standard for evaluating evasive-content detection, exposes fundamental limitations in current multimodal reasoning, and lays groundwork for safer, more transparent content moderation systems in e-commerce.

Abstract: E-commerce platforms increasingly rely on Large Language Models (LLMs) and Vision-Language Models (VLMs) to detect illicit or misleading product content. However, these models remain vulnerable to evasive content: inputs (text or images) that superficially comply with platform policies while covertly conveying prohibited claims. Unlike traditional adversarial attacks that induce overt failures, evasive content exploits ambiguity and context, making it far harder to detect. Existing robustness benchmarks provide little guidance for this demanding, real-world challenge. We introduce EVADE, the first expert-curated, Chinese, multimodal benchmark specifically designed to evaluate foundation models on evasive content detection in e-commerce. The dataset contains 2,833 annotated text samples and 13,961 images spanning six demanding product categories, including body shaping, height growth, and health supplements. Two complementary tasks assess distinct capabilities: Single-Violation, which probes fine-grained reasoning under short prompts, and All-in-One, which tests long-context reasoning by merging overlapping policy rules into unified instructions. Notably, the All-in-One setting significantly narrows the performance gap between partial and full-match accuracy, suggesting that clearer rule definitions improve alignment between human and model judgment. We benchmark 26 mainstream LLMs and VLMs and observe substantial performance gaps: even state-of-the-art models frequently misclassify evasive samples. By releasing EVADE and strong baselines, we provide the first rigorous standard for evaluating evasive-content detection, expose fundamental limitations in current multimodal reasoning, and lay the groundwork for safer and more transparent content moderation systems in e-commerce. The dataset is publicly available at https://huggingface.co/datasets/koenshen/EVADE-Bench.

Method: Created MathEDU dataset of student math problem-solving processes with teacher-written feedback, then systematically evaluated various models on three hierarchical tasks: answer correctness classification, error identification, and feedback generation.

Result: Fine-tuning improved performance on correctness classification and error location, but generated feedback showed significant gaps compared to teacher feedback - often verbose and lacking targeted explanations for underlying misconceptions.

Conclusion: There’s an urgent need for more trustworthy and pedagogy-aware AI feedback systems in education, as current LLMs fall short in providing effective, targeted feedback despite technical improvements.

Abstract: The increasing reliance on Large Language Models (LLMs) across various domains extends to education, where students progressively use generative AI as a tool for learning. While prior work has examined LLMs’ mathematical ability, their reliability in grading authentic student problem-solving processes and delivering effective feedback remains underexplored. This study introduces MathEDU, a dataset consisting of student problem-solving processes in mathematics and corresponding teacher-written feedback. We systematically evaluate the reliability of various models across three hierarchical tasks: answer correctness classification, error identification, and feedback generation. Experimental results show that fine-tuning strategies effectively improve performance in classifying correctness and locating erroneous steps. However, the generated feedback across models shows a considerable gap from teacher-written feedback. Critically, the generated feedback is often verbose and fails to provide targeted explanations for the student’s underlying misconceptions. This emphasizes the urgent need for trustworthy and pedagogy-aware AI feedback in education.

Method: Strategically select NLI fine-tuning data by: 1) prioritizing more complex training examples, and 2) replacing existing examples with LLM-generated synthetic data. Also prompt LLMs to create more complex synthetic data to address simplicity issues.

Result: Prioritizing complex examples improves performance on challenging OOD datasets. Synthetic data improves easier OOD datasets. More complex synthetic data improves both easy and challenging OOD datasets. Autoregressive LLMs are substantially more robust to distributional shifts than encoder models.

Conclusion: Data selection strategies can improve OOD robustness for fine-tuned LLMs under API constraints. Autoregressive LLMs should be preferred baselines for future NLI robustness research due to their superior robustness to distributional shifts.

Abstract: We investigate the robustness of fine-tuned Large Language Models (LLMs) for the task of Natural Language Inference (NLI), finding that the in-distribution gains from fine-tuning correspond to a large drop in out-of-distribution (OOD) performance. Despite the widespread use of closed-source LLMs, there are no robustness mitigation methods that work under their API fine-tuning constraints. Existing methods to improve robustness typically require changing the fine-tuning process or large-scale data augmentation, methods that are infeasible or cost prohibitive for closed-source models. To address this, we propose strategically selecting the NLI fine-tuning data, prioritising more complex examples or replacing existing training examples with LLM-generated data. Prioritising more complex training examples improves performance on challenging OOD NLI datasets, while training with synthetic data leads to substantial improvements on easier OOD datasets. We find that synthetic examples are often too simple, and by prompting LLMs to create more complex synthetic data we can improve performance on both easy and challenging OOD datasets. Finally, we show that recent autoregressive LLMs are substantially more robust to distributional shifts compared to encoder models, and should be a preferred baseline for future research.

Method: Integrates FlashAttention and Hybrid Tree Attention to accelerate prefill and verification steps. Proposes Cross-model Retrieval, a novel KV cache eviction strategy that uses the target model’s attention scores to dynamically select relevant context for the draft model.

Result: Accelerates speculative decoding by up to 2.84x on 16K-token long document summarization and up to 3.86x on long-form reasoning, while preserving short-input performance.

Conclusion: SpecExtend is an effective drop-in enhancement that significantly improves speculative decoding performance on long sequences without requiring additional training.

Abstract: Speculative decoding is a widely used technique for accelerating inference in large language models (LLMs), but its performance degrades as input length grows, with significant drops even at moderate lengths. Yet, this early degradation has remained largely underexplored. We introduce SpecExtend, a drop-in enhancement that improves speculative decoding on long sequences without additional training. SpecExtend integrates efficient attention mechanisms such as FlashAttention and Hybrid Tree Attention to accelerate prefill and verification steps. To improve both draft accuracy and speed on long inputs without retraining, we propose Cross-model Retrieval, a novel KV cache eviction strategy that leverages the target model’s attention scores to dynamically select relevant context for the smaller draft model. Extensive evaluations show that SpecExtend accelerates speculative decoding by up to 2.84x on 16K-token long document summarization and up to 3.86x on long-form reasoning, while preserving the short-input performance of state-of-the-art frameworks. Our code is available at https://github.com/jycha98/SpecExtend .

Method: Created BLUCK dataset with 2366 MCQs curated from college and job examination materials across 23 categories covering Bangladesh’s culture, history, and Bengali linguistics. Benchmarked 9 LLMs (6 proprietary, 3 open-source) including GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, Llama-3.3-70B-Instruct, and DeepSeekV3.

Result: LLMs perform reasonably well overall but struggle specifically with Bengali phonetics. Performance on Bengali cultural and linguistic contexts is not comparable to mainstream languages like English. Results indicate Bengali’s status as a mid-resource language.

Conclusion: BLUCK is the first MCQ-based benchmark centered on native Bengali culture, history, and linguistics. It reveals current LLM limitations in Bengali understanding, particularly in phonetics, and provides a foundation for improving LLM performance on Bengali language and cultural knowledge.

Abstract: In this work, we introduce BLUCK, a new dataset designed to measure the performance of Large Language Models (LLMs) in Bengali linguistic understanding and cultural knowledge. Our dataset comprises 2366 multiple-choice questions (MCQs) carefully curated from compiled collections of several college and job level examinations and spans 23 categories covering knowledge on Bangladesh’s culture and history and Bengali linguistics. We benchmarked BLUCK using 6 proprietary and 3 open-source LLMs - including GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro, Llama-3.3-70B-Instruct, and DeepSeekV3. Our results show that while these models perform reasonably well overall, they, however, struggles in some areas of Bengali phonetics. Although current LLMs’ performance on Bengali cultural and linguistic contexts is still not comparable to that of mainstream languages like English, our results indicate Bengali’s status as a mid-resource language. Importantly, BLUCK is also the first MCQ-based evaluation benchmark that is centered around native Bengali culture, history, and linguistics.

Method: Introduce a framework based on multi-dimensional goal-space modeling user goals and LLM outputs as vectors with dimensions corresponding to text attributes (e.g., reading difficulty). Applied to text-rewriting task, evaluating interventions like prompt engineering, best-of-N sampling, and reinforcement learning fine-tuning.

Result: Current LLMs induce unintended changes or side effects to text attributes, impeding steerability. Interventions have varying effectiveness but side effects remain problematic. Even strong LLMs struggle with steerability.

Conclusion: Existing alignment strategies may be insufficient for achieving reliable steerability. Authors open-source their evaluation framework to facilitate further research.

Abstract: Despite advances in large language models (LLMs) on reasoning and instruction-following tasks, it is unclear whether they can reliably produce outputs aligned with a variety of user goals, a concept called steerability. Two gaps in current LLM evaluation impede steerability evaluation: (1) many benchmarks are built with past LLM chats and Internet-scraped text, which may skew towards common requests, and (2) scalar measures of performance common in prior work could conceal behavioral shifts in LLM outputs in open-ended generation. Thus, we introduce a framework based on a multi-dimensional goal-space that models user goals and LLM outputs as vectors with dimensions corresponding to text attributes (e.g., reading difficulty). Applied to a text-rewriting task, we find that current LLMs induce unintended changes or side effects to text attributes, impeding steerability. Interventions to improve steerability, such as prompt engineering, best-of-N sampling, and reinforcement learning fine-tuning, have varying effectiveness but side effects remain problematic. Our findings suggest that even strong LLMs struggle with steerability, and existing alignment strategies may be insufficient. We open-source our steerability evaluation framework at https://github.com/MLD3/steerability.

Method: Proposes heterogeneous Mixture-of-Adapters (MoA) that dynamically integrates PEFT adapter experts with diverse structures, leveraging complementary representational capabilities. Offers two variants: Soft MoA (weighted fusion of all expert outputs) and Sparse MoA (sparsely activates experts based on contribution).

Result: Experimental results show heterogeneous MoA outperforms homogeneous MoE-LoRA methods in both performance and parameter efficiency.

Conclusion: Heterogeneous MoA approach effectively addresses limitations of homogeneous MoE-LoRA methods by fostering expert specialization and enhancing knowledge transfer to downstream tasks through diverse adapter structures.

Abstract: Recent studies integrate Low-Rank Adaptation (LoRA) and Mixture-of-Experts (MoE) to further enhance the performance of parameter-efficient fine-tuning (PEFT) methods in Large Language Model (LLM) applications. Existing methods employ \emph{homogeneous} MoE-LoRA architectures composed of LoRA experts with either similar or identical structures and capacities. However, these approaches often suffer from representation collapse and expert load imbalance, which negatively impact the potential of LLMs. To address these challenges, we propose a \emph{heterogeneous} \textbf{Mixture-of-Adapters (MoA)} approach. This method dynamically integrates PEFT adapter experts with diverse structures, leveraging their complementary representational capabilities to foster expert specialization, thereby enhancing the effective transfer of pre-trained knowledge to downstream tasks. MoA supports two variants: \textbf{(i)} \textit{Soft MoA} achieves fine-grained integration by performing a weighted fusion of all expert outputs; \textbf{(ii)} \textit{Sparse MoA} activates adapter experts sparsely based on their contribution, achieving this with negligible performance degradation. Experimental results demonstrate that heterogeneous MoA outperforms homogeneous MoE-LoRA methods in both performance and parameter efficiency. Our project is available at https://github.com/DCDmllm/MoA.

Method: 1) Show transformers with fixed precision (except attention) are expressively equivalent to C-RASP programming language, preserving depth. 2) Prove deeper C-RASP programs are more expressive than shallower ones. 3) Extend proof to transformers with positional encodings (RoPE, ALiBi). 4) Use temporal logic with counting operators equivalent to C-RASP. 5) Provide empirical evidence on sequential dependency tasks.

Result: Theoretical proof shows deeper transformers are more expressive than shallower transformers within the studied subclass. Empirical evidence confirms theory predicts required depth for transformers without positional encodings to length-generalize on sequential dependency tasks.

Conclusion: Deeper transformers have strictly greater expressive power than shallower ones, providing formal justification for the observed correlation between depth and capabilities. The theory also applies to transformers with positional encodings and has practical implications for task-specific depth requirements.

Abstract: It has been observed that transformers with greater depth (that is, more layers) have more capabilities, but can we establish formally which capabilities are gained? We answer this question with a theoretical proof followed by an empirical study. First, we consider transformers that round to fixed precision except inside attention. We show that this subclass of transformers is expressively equivalent to the programming language C-RASP and this equivalence preserves depth. Second, we prove that deeper C-RASP programs are more expressive than shallower C-RASP programs, implying that deeper transformers are more expressive than shallower transformers (within the subclass mentioned above). The same is also proven for transformers with positional encodings (like RoPE and ALiBi). These results are established by studying a temporal logic with counting operators equivalent to C-RASP. Finally, we provide empirical evidence that our theory predicts the depth required for transformers without positional encodings to length-generalize on a family of sequential dependency tasks.

Method: Propose a flexible external memory framework based on knowledge graphs with hybrid graph design (standard edges + two hyper-edge types) that LLMs automatically construct and update, supporting diverse retrieval mechanisms (A*, water-circle traversal, beam search, hybrid methods).

Result: Evaluated on TriviaQA, HotpotQA, and DiaASQ benchmarks, showing different memory/retrieval configurations yield optimal performance per task; extended DiaASQ with temporal annotations and contradictory statements, demonstrating robustness in managing temporal dependencies and context-aware reasoning.

Conclusion: The knowledge graph-based memory framework enables effective personalization and scaling in long-term LLM interactions through structured memory and flexible retrieval, addressing limitations of current RAG approaches.

Abstract: Personalizing language models that effectively incorporating user interaction history remains a central challenge in development of adaptive AI systems. While large language models (LLMs), combined with Retrieval-Augmented Generation (RAG), have improved factual accuracy, they often lack structured memory and fail to scale in complex, long-term interactions. To address this, we propose a flexible external memory framework based on knowledge graph, which construct and update memory model automatically by LLM itself. Building upon the AriGraph architecture, we introduce a novel hybrid graph design that supports both standard edges and two types of hyper-edges, enabling rich and dynamic semantic and temporal representations. Our framework also supports diverse retrieval mechanisms, including A*, water-circle traversal, beam search and hybrid methods, making it adaptable to different datasets and LLM capacities. We evaluate our system on three benchmarks: TriviaQA, HotpotQA, DiaASQ and demonstrate that different memory and retrieval configurations yield optimal performance depending on the task. Additionally, we extend the DiaASQ benchmark with temporal annotations and internally contradictory statements, showing that our system remains robust and effective in managing temporal dependencies and context-aware reasoning.

Method: Created dataset by manually translating 10,000 standard Bangla sentences into four dialects (Mymensingh, Noakhali, Sylhet, Chittagong). Used dual annotation: multiclass thematic labeling (Political, Religious, Neutral) and multilabel emotion annotation (7 emotions). Expert native translators performed translation/annotation with quality assurance via Cohen’s Kappa inter-annotator agreement and systematic data refinement.

Result: Achieved strong consistency across dialects through Cohen’s Kappa inter-annotator agreement. Created comprehensive dataset with balanced political/religious/neutral content covering contemporary socio-political landscape of Bangladesh.

Conclusion: ANUBHUTI fills a critical gap in resources for sentiment analysis in low-resource Bangla dialects, enabling more accurate and context-aware natural language processing for regional Bangla varieties.

Abstract: Sentiment analysis for regional dialects of Bangla remains an underexplored area due to linguistic diversity and limited annotated data. This paper introduces ANUBHUTI, a comprehensive dataset consisting of 10,000 sentences manually translated from standard Bangla into four major regional dialects Mymensingh, Noakhali, Sylhet, and Chittagong. The dataset predominantly features political and religious content, reflecting the contemporary socio political landscape of Bangladesh, alongside neutral texts to maintain balance. Each sentence is annotated using a dual annotation scheme: multiclass thematic labeling categorizes sentences as Political, Religious, or Neutral, and multilabel emotion annotation assigns one or more emotions from Anger, Contempt, Disgust, Enjoyment, Fear, Sadness, and Surprise. Expert native translators conducted the translation and annotation, with quality assurance performed via Cohens Kappa inter annotator agreement, achieving strong consistency across dialects. The dataset was further refined through systematic checks for missing data, anomalies, and inconsistencies. ANUBHUTI fills a critical gap in resources for sentiment analysis in low resource Bangla dialects, enabling more accurate and context aware natural language processing.

Method: POLAR (Policy Discriminative Learning) trains a reward model to discern identical policies and discriminate different ones, capturing relative differences between one policy and an arbitrary target policy with desired behaviors.

Result: POLAR substantially outperforms traditional methods, improving preference accuracy from 54.8% to 81.0% on STEM tasks and from 57.9% to 85.5% on creative writing. It also enhances RLHF performance, improving LLaMa3.1-8B from 47.36% to 56.33% and Qwen2.5-32B from 64.49% to 70.47% on 20 benchmarks.

Conclusion: POLAR demonstrates impressive performance, strong generalization, and clear scaling properties with power-law relationships, suggesting it’s a promising direction for developing general and strong reward models.

Abstract: We offer a novel perspective on reward modeling by formulating it as a policy discriminator, which quantifies the difference between two policies to generate a reward signal, guiding the training policy towards a target policy with desired behaviors. Based on this conceptual insight, we propose a scalable pre-training method named Policy Discriminative Learning (POLAR), which trains a reward model (RM) to discern identical policies and discriminate different ones. Unlike traditional reward modeling methods relying on absolute preferences, POLAR captures the relative difference between one policy and an arbitrary target policy, which is a scalable, high-level optimization objective suitable for modeling generic ranking relationships. Leveraging the POLAR pre-training paradigm, we present a series of RMs with parameter scales from 1.8B to 7B. Empirical results show that POLAR substantially outperforms traditional non-pre-trained methods, significantly enhancing RM performance. For instance, POLAR-7B could improve preference accuracy from 54.8% to 81.0% on STEM tasks and from 57.9% to 85.5% on creative writing tasks compared to SOTA baselines. POLAR also shows robust generalization capabilities in RLHF using Reinforcement Fine-tuning (RFT), providing reliable reward signals and markedly enhancing policy performance–improving LLaMa3.1-8B from an average of 47.36% to 56.33% and Qwen2.5-32B from 64.49% to 70.47% on 20 benchmarks. Moreover, scaling experiments reveal a clear power-law relationship between computation and performance, supported by linear correlation coefficients approaching 0.99. The impressive performance, strong generalization, and scaling properties suggest that POLAR is a promising direction for developing general and strong reward models.

Method: Introduce Causal CoT Graphs (CCGraphs) - directed acyclic graphs automatically extracted from reasoning traces that model fine-grained causal dependencies. Create KisMATH dataset with 1671 math problems from MATH500, GSM8K, and AIME with associated CCGraphs.

Result: Analysis with 15 open-weight LLMs shows: (1) reasoning nodes in CCGraphs are causal contributors to final answers (constitutive of reasoning), (2) LLMs emphasize reasoning paths captured by CCGraphs, indicating models internally realize similar structures.

Conclusion: KisMATH enables controlled, graph-aligned interventions and opens avenues for investigating CoT’s role in LLM reasoning by providing a structured framework to analyze causal dependencies in reasoning processes.

Abstract: Chain-of-thought (CoT) traces have been shown to improve performance of large language models on a plethora of reasoning tasks, yet there is no consensus on the mechanism by which this boost is achieved. To shed more light on this, we introduce Causal CoT Graphs (CCGraphs), which are directed acyclic graphs automatically extracted from reasoning traces that model fine-grained causal dependencies in language-model outputs. A collection of 1671 mathematical reasoning problems from MATH500, GSM8K, and AIME, together with their associated CCGraphs, has been compiled into our dataset – KisMATH. Our detailed empirical analysis with 15 open-weight LLMs shows that (i) reasoning nodes in the CCGraphs are causal contributors to the final answer, which we argue is constitutive of reasoning; and (ii) LLMs emphasize the reasoning paths captured by the CCGraphs, indicating that the models internally realize structures similar to our graphs. KisMATH enables controlled, graph-aligned interventions and opens avenues for further investigation into the role of CoT in LLM reasoning.

Method: A robust multi-stage framework with: 1) lightweight query router for efficiency, 2) query-aware retrieval and summarization pipeline, 3) dual-pathways generation, and 4) post-hoc verification. The conservative strategy prioritizes factual accuracy over completeness.

Result: Achieved 3rd place in Task 1 of KDD Cup 2025 Meta Comprehensive RAG Benchmark for Multi-modal, Multi-turn (CRAG-MM) challenge, demonstrating effectiveness of prioritizing answer reliability.

Conclusion: The proposed conservative multi-stage framework effectively minimizes hallucinations in complex multi-modal RAG systems, showing that prioritizing factual accuracy and truthfulness is crucial for reliable VLM applications in real-world scenarios.

Abstract: This paper presents the technical solution developed by team CRUISE for the KDD Cup 2025 Meta Comprehensive RAG Benchmark for Multi-modal, Multi-turn (CRAG-MM) challenge. The challenge aims to address a critical limitation of modern Vision Language Models (VLMs): their propensity to hallucinate, especially when faced with egocentric imagery, long-tail entities, and complex, multi-hop questions. This issue is particularly problematic in real-world applications where users pose fact-seeking queries that demand high factual accuracy across diverse modalities. To tackle this, we propose a robust, multi-stage framework that prioritizes factual accuracy and truthfulness over completeness. Our solution integrates a lightweight query router for efficiency, a query-aware retrieval and summarization pipeline, a dual-pathways generation and a post-hoc verification. This conservative strategy is designed to minimize hallucinations, which incur a severe penalty in the competition’s scoring metric. Our approach achieved 3rd place in Task 1, demonstrating the effectiveness of prioritizing answer reliability in complex multi-modal RAG systems. Our implementation is available at https://github.com/Breezelled/KDD-Cup-2025-Meta-CRAG-MM .

Method: Created MUDT using hybrid LLM pre-annotation with human correction (3,456 sentences), then developed AI tutoring system using MUDT-based syntactic analyses for pedagogical feedback.

Result: MUDT reduced crossing-arc rate from 7.35% to 0.06%, improved parsing accuracy, and AI tutoring with syntax-aware feedback significantly boosted learning gains for 35 language learners.

Conclusion: MUDT provides robust syntactic analysis foundation and demonstrates that linguistically informed NLP resources can bridge computational models with second-language learners’ cognitive needs.

Abstract: Developing effective educational technologies for low-resource agglutinative languages like Uyghur is often hindered by the mismatch between existing annotation frameworks and specific grammatical structures. To address this challenge, this study introduces the Modern Uyghur Dependency Treebank (MUDT), a linguistically grounded annotation framework specifically designed to capture the agglutinative complexity of Uyghur, including zero copula constructions and fine-grained case marking. Utilizing a hybrid pipeline that combines Large Language Model pre-annotation with rigorous human correction, a high-quality treebank consisting of 3,456 sentences was constructed. Intrinsic structural evaluation reveals that MUDT significantly improves dependency projectivity by reducing the crossing-arc rate from 7.35% in the Universal Dependencies standard to 0.06%. Extrinsic parsing experiments using UDPipe and Stanza further demonstrate that models trained on MUDT achieve superior in-domain accuracy and cross-domain generalization compared to UD-based baselines. To validate the practical utility of this computational resource, an AI-assisted grammar tutoring system was developed to translate MUDT-based syntactic analyses into interpretable pedagogical feedback. A controlled experiment involving 35 second-language learners indicated that students receiving syntax-aware feedback achieved significantly higher learning gains compared to those in a control group. These findings establish MUDT as a robust foundation for syntactic analysis and underscore the critical role of linguistically informed natural language processing resources in bridging the gap between computational models and the cognitive needs of second-language learners.

Method: Controlled evaluation of five transformer-based LLMs on question answering over U.S. 10-K filings using human evaluation, automated similarity metrics, and behavioral diagnostics under standardized prompting conditions.

Result: Models differ in performance across qualitative dimensions (relevance, completeness, clarity, conciseness, factual accuracy) with modest inter-rater agreement. Automated metrics show systematic differences in lexical overlap and semantic similarity. Behavioral diagnostics reveal variation in response stability and cross-prompt alignment. No single model consistently dominates across all evaluation perspectives.

Conclusion: Performance differences should be interpreted as relative tendencies rather than definitive indicators of general reliability. There’s a need for evaluation frameworks that account for human disagreement, behavioral variability, and interpretability when deploying LLMs in financially consequential applications.

Abstract: Large language models (LLMs) are increasingly used to support the analysis of complex financial disclosures, yet their reliability, behavioral consistency, and transparency remain insufficiently understood in high-stakes settings. This paper presents a controlled evaluation of five transformer-based LLMs applied to question answering over the Business sections of U.S. 10-K filings. To capture complementary aspects of model behavior, we combine human evaluation, automated similarity metrics, and behavioral diagnostics under standardized and context-controlled prompting conditions. Human assessments indicate that models differ in their average performance across qualitative dimensions such as relevance, completeness, clarity, conciseness, and factual accuracy, though inter-rater agreement is modest, reflecting the subjective nature of these criteria. Automated metrics reveal systematic differences in lexical overlap and semantic similarity across models, while behavioral diagnostics highlight variation in response stability and cross-prompt alignment. Importantly, no single model consistently dominates across all evaluation perspectives. Together, these findings suggest that apparent performance differences should be interpreted as relative tendencies under the tested conditions rather than definitive indicators of general reliability. The results underscore the need for evaluation frameworks that account for human disagreement, behavioral variability, and interpretability when deploying LLMs in financially consequential applications.

Method: Progressive reinforcement learning framework with three stages: 1) Develop logical reasoning capabilities on medical problems, 2) Adaptively optimize retrieval to align with knowledge corpus characteristics, 3) Jointly optimize retrieval-reasoning coordination.

Result: LLaMA3.1-8B-Instruct + Med-R³ surpasses GPT-4o-mini by 3.93% at comparable parameter scale, while Qwen2.5-14B + Med-R³ shows 13.53% improvement, achieving state-of-the-art performance.

Conclusion: Med-R³ effectively addresses the coordination gap between retrieval and reasoning in medical AI through progressive reinforcement learning, demonstrating significant performance gains and better generalization to novel medical problem contexts.

Abstract: In medical scenarios, effectively retrieving external knowledge and leveraging it for rigorous logical reasoning is of significant importance. Despite their potential, existing work has predominantly focused on enhancing either retrieval or reasoning capabilities of the models in isolation, with little attention given to their joint optimization, which leads to limited coordination between the two processes. Additionally, current methods rely heavily on supervised fine-tuning (SFT), which can cause models to memorize existing problem-solving pathways, thereby restricting their generalization ability when confronted with novel problem contexts. Furthermore, while some studies have explored to improve retrieval-augmented reasoning in general domains via reinforcement learning, their reward function designs do not adequately capture the specific demands of the medical domain. To address these challenges, we introduce Med-R$^3$, a Medical Retrieval-augmented Reasoning framework driven by progressive Reinforcement learning. In this framework, we first develop the model’s ability to perform logical reasoning over medical problems. Subsequently, on the basis of this foundation, we adaptively optimize the retrieval capability to better align with the characteristics of knowledge corpus and external information utilization throughout the reasoning process. Finally, we conduct joint optimization of the model’s retrieval and reasoning coordination. Extensive experiments indicate that Med-R$^3$ could achieve state-of-the-art performances, with LLaMA3.1-8B-Instruct + Med-R$^3$ surpassing closed-sourced GPT-4o-mini by 3.93% at a comparable parameter scale, while Qwen2.5-14B augmented with Med-R$^3$ shows a more substantial gain of 13.53%.

Method: Bias Association Discovery Framework (BADF) - a systematic approach for extracting associations between demographic identities and descriptive concepts from open-ended LLM outputs across multiple models and diverse real-world contexts.

Result: BADF enables robust mapping and analysis of varied concepts characterizing demographic identities, advancing understanding of biases in open-ended generation and providing a scalable tool for bias identification.

Conclusion: BADF provides a systematic framework for discovering both known and previously unrecognized bias associations in LLMs, offering a scalable tool for comprehensive bias analysis beyond traditional evaluation limitations.

Abstract: Social biases embedded in Large Language Models (LLMs) raise critical concerns, resulting in representational harms – unfair or distorted portrayals of demographic groups – that may be expressed in subtle ways through generated language. Existing evaluation methods often depend on predefined identity-concept associations, limiting their ability to surface new or unexpected forms of bias. In this work, we present the Bias Association Discovery Framework (BADF), a systematic approach for extracting both known and previously unrecognized associations between demographic identities and descriptive concepts from open-ended LLM outputs. Through comprehensive experiments spanning multiple models and diverse real-world contexts, BADF enables robust mapping and analysis of the varied concepts that characterize demographic identities. Our findings advance the understanding of biases in open-ended generation and provide a scalable tool for identifying and analyzing bias associations in LLMs.

Method: Apply PCA to knowledge base embeddings and score queries in compact subspaces selected by explained-variance retention or t-test ranking. Evaluate geometric semantic-search rules and lightweight classifiers across 16 domains.

Result: Low-dimensional detectors achieve competitive OOD performance while being faster, cheaper, and more interpretable than prompted LLM-based judges. OOD queries primarily degrade RAG output relevance.

Conclusion: Efficient external OOD detection is needed to maintain safe, in-scope behavior in RAG systems, especially in high-stakes domains where query relevance is critical.

Abstract: Retrieval-Augmented Generation (RAG) systems are increasingly deployed in high-stakes domains, where safety depends not only on how a system answers, but also on whether a query should be answered given a knowledge base (KB). Out-of-domain (OOD) queries can cause dense retrieval to surface weakly related context and lead the generator to produce fluent but unjustified responses. We study lightweight, KB-aligned OOD detection as an always-on gate for RAG systems. Our approach applies PCA to KB embeddings and scores queries in a compact subspace selected either by explained-variance retention (EVR) or by a separability-driven t-test ranking. We evaluate geometric semantic-search rules and lightweight classifiers across 16 domains, including high-stakes COVID-19 and Substance Use KBs, and stress-test robustness using both LLM-generated attacks and an in-the-wild 4chan attack. We find that low-dimensional detectors achieve competitive OOD performance while being faster, cheaper, and more interpretable than prompted LLM-based judges. Finally, human and LLM-based evaluations show that OOD queries primarily degrade the relevance of RAG outputs, showing the need for efficient external OOD detection to maintain safe, in-scope behavior.

Method: Treats reasoning model weights as foundational prior, uses a reasoning capability indicator to preserve core long CoT weights while selectively merging essential domain-specific weights.

Result: Improves domain task performance by 9.5% and 9.2% over state-of-the-art methods on Qwen2.5-7B, Llama3.1-8B, and Qwen2.5-1.5B models in BioMedicine and Finance domains, without significantly harming original long CoT reasoning capability.

Conclusion: RCP-Merging successfully addresses the challenge of merging reasoning models with domain-specific ones, providing a resource-efficient method to create dual-capability LLMs that maintain both long CoT reasoning and domain expertise.

Abstract: Large Language Models (LLMs) with long chain-of-thought (CoT) capability, termed Reasoning Models, demonstrate superior intricate problem-solving abilities through multi-step long CoT reasoning. To create a dual-capability model with long CoT capability and domain-specific knowledge without substantial computational and data costs, model merging emerges as a highly resource-efficient method. However, significant challenges lie in merging domain-specific LLMs with long CoT ones since nowadays merging methods suffer from reasoning capability degradation, even gibberish output and output collapse. To overcome this, we introduce RCP-Merging: Merging Long Chain-of-Thought Models with Domain-Specific Models by Considering Reasoning Capability as Prior, a novel merging framework designed to integrate domain-specific LLMs with long CoT capability, meanwhile maintaining model performance in the original domain. Treating reasoning model weights as foundational prior, our method utilizes a reasoning capability indicator to preserve core long CoT capability model weights while selectively merging essential domain-specific weights. We conducted extensive experiments on Qwen2.5-7B, Llama3.1-8B, and Qwen2.5-1.5B models in BioMedicine and Finance domains. Our results show that RCP-Merging successfully merges a reasoning model with domain-specific ones, improving domain task performance by 9.5% and 9.2% over state-of-the-art methods, without significantly harming the original long CoT reasoning capability.

Method: Created CultureCare dataset with 1729 distress messages, 1523 cultural signals, and 1041 support strategies across four cultures. Developed and tested four adaptation strategies for three state-of-the-art LLMs.

Result: Adapted LLMs outperform anonymous online peer responses; simple cultural role-play is insufficient for cultural sensitivity; LLMs show potential for clinical training in cultural competence.

Conclusion: CultureCare enables culturally sensitive emotional support from LLMs, with adapted models showing practical value and potential applications in clinical training for cultural competence.

Abstract: Large language models (LLMs) show promise in offering emotional support and generating empathetic responses for individuals in distress, but their ability to deliver culturally sensitive support remains underexplored due to a lack of resources. In this work, we introduce CultureCare, the first dataset designed for this task, spanning four cultures and including 1729 distress messages, 1523 cultural signals, and 1041 support strategies with fine-grained emotional and cultural annotations. Leveraging CultureCare, we (i) develop and test four adaptation strategies for guiding three state-of-the-art LLMs toward culturally sensitive responses; (ii) conduct comprehensive evaluations using LLM-as-a-Judge, in-culture human annotators, and clinical psychologists; (iii) show that adapted LLMs outperform anonymous online peer responses, and that simple cultural role-play is insufficient for cultural sensitivity; and (iv) explore the application of LLMs in clinical training, where experts highlight their potential in fostering cultural competence in novice therapists.

Method: Three-stage structured approach: 1) content extraction from submissions, 2) retrieval and synthesis of related work, 3) structured comparison for evidence-based assessment. The method is informed by analysis of human novelty reviews and captures patterns like independent claim verification and contextual reasoning.

Result: Evaluated on 182 ICLR 2025 submissions with human-annotated novelty assessments: 86.5% alignment with human reasoning and 75.3% agreement on novelty conclusions, substantially outperforming existing LLM-based baselines. Produces detailed, literature-aware analyses and improves consistency over ad hoc reviewer judgments.

Conclusion: Structured LLM-assisted approaches can support more rigorous and transparent peer review without displacing human expertise. The method demonstrates potential for automated novelty assessment in high-volume academic reviewing.

Abstract: Novelty assessment is a central yet understudied aspect of peer review, particularly in high volume fields like NLP where reviewer capacity is increasingly strained. We present a structured approach for automated novelty evaluation that models expert reviewer behavior through three stages: content extraction from submissions, retrieval and synthesis of related work, and structured comparison for evidence based assessment. Our method is informed by a large scale analysis of human written novelty reviews and captures key patterns such as independent claim verification and contextual reasoning. Evaluated on 182 ICLR 2025 submissions with human annotated reviewer novelty assessments, the approach achieves 86.5% alignment with human reasoning and 75.3% agreement on novelty conclusions - substantially outperforming existing LLM based baselines. The method produces detailed, literature aware analyses and improves consistency over ad hoc reviewer judgments. These results highlight the potential for structured LLM assisted approaches to support more rigorous and transparent peer review without displacing human expertise. Data and code are made available.

Method: 1) Created ToxiFrench dataset via semi-automated annotation pipeline (LLM pre-annotation + 10% human verification). 2) Benchmarked various models, discovering SLMs outperform larger models. 3) Proposed novel Chain-of-Thought fine-tuning with Dynamic Weighted Loss to improve faithfulness.

Result: The fine-tuned 4B model (Qwen3-4B) achieves state-of-the-art performance on the benchmark, improving balanced accuracy by 10% over baseline and outperforming GPT-4o and DeepSeek-R1 while retaining cross-lingual capabilities.

Conclusion: The work provides a valuable French toxicity detection resource and demonstrates that smaller models can be more robust for this task when properly fine-tuned with innovative techniques like CoT and Dynamic Weighted Loss.

Abstract: Detecting toxic content using language models is crucial yet challenging. While substantial progress has been made in English, toxicity detection in French remains underdeveloped, primarily due to the lack of culturally relevant, human-annotated, large-scale datasets. In this work, we release ToxiFrench, a dataset of 53,622 French online comments together with a balanced benchmark split for systematic evaluation. The dataset is constructed via a semi-automated annotation pipeline that reduces manual labeling to only 10% through high-confidence LLM-based pre-annotation and human verification, while ensuring statistical alignment with human-only annotation. We then benchmark a broad range of models and uncover a counterintuitive finding: Small Language Models (SLMs) often surpass larger models in robustness and generalization on this task. Motivated by this finding, we propose a novel Chain-of-Thought (CoT) fine-tuning strategy using a Dynamic Weighted Loss (DWL) that progressively emphasizes the model’s final decision and significantly improves faithfulness. Our fine-tuned 4B model (Qwen3-4B) achieves state-of-the-art performance on the benchmark. It improves its balanced accuracy by 10% over its baseline and achieves better performance than GPT-4o and DeepSeek-R1 on our benchmark, while successfully retaining cross-lingual capabilities.

Method: The authors analyze the limitations and identify Softmax as a contributing factor. They propose a novel scoring function called scaled signed averaging (SSA) to replace Softmax in the attention mechanism.

Result: SSA significantly improves performance on ICL tasks with quantifiers and linear functions. It also outperforms transformer models with Softmax on early learning NLP benchmarks and linguistic probing tasks in zero-shot and few-shot settings.

Conclusion: Softmax contributes to LLMs’ limitations on certain semantic tasks, and replacing it with SSA can mitigate these limitations while improving performance across various NLP tasks and settings.

Abstract: While Large Language models’ abilities for in-context learning (ICL) have had much success, they have limitations on simple semantic tasks involving quantifiers like {\em every} and {\em some}, as well as on tasks with linear functions. We analyze those limitations and identify Softmax, the scoring function in the attention mechanism, as a contributing factor to these limitations. Our \textbf{scaled signed averaging (SSA)}, a novel scoring function mitigates these limitations. SSA significantly improves performance on our ICL tasks. In addition, SSA outperforms transformer models with Softmax on several early learning NLP benchmarks and linguistic probing tasks on zero and few-shot settings.

Method: Proposes a unified co-training framework that integrates three safety behaviors within a single SFT stage: positive (lawful/prosocial), negative (unfiltered/risk-prone), and rejective (refusal-oriented/conservative). Each behavior is activated via system-level instructions or magic tokens, enabling dynamic switching at inference time.

Result: The method matches safety alignment quality of SFT+DPO, with their 8B model surpassing DeepSeek-R1 (671B) in safety performance. Creates a Safety Alignment Margin with well-separated response distributions for each safety mode, providing evidence of safety robustness and fine-grained control.

Conclusion: Presents a scalable, efficient, and highly controllable solution for LLM content safety that reduces both training complexity and deployment costs while enabling unprecedented fine-grained control.

Abstract: Current methods for content safety in Large Language Models (LLMs), such as Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF), often rely on multi-stage training pipelines and lack fine-grained, post-deployment controllability. To address these limitations, we propose a unified co-training framework that efficiently integrates multiple safety behaviors: positive (lawful/prosocial), negative (unfiltered/risk-prone) and rejective (refusal-oriented/conservative) within a single SFT stage. Notably, each behavior is dynamically activated via a simple system-level instruction, or magic token, enabling stealthy and efficient behavioral switching at inference time. This flexibility supports diverse deployment scenarios, such as positive for safe user interaction, negative for internal red-teaming, and rejective for context-aware refusals triggered by upstream moderation signals. This co-training strategy induces a distinct Safety Alignment Margin in the output space, characterized by well-separated response distributions corresponding to each safety mode. The existence of this margin provides empirical evidence for the model’s safety robustness and enables unprecedented fine-grained control. Experiments show that our method matches the safety alignment quality of SFT+DPO, with our 8B model notably surpassing DeepSeek-R1 (671B) in safety performance, while significantly reducing both training complexity and deployment costs. This work presents a scalable, efficient, and highly controllable solution for LLM content safety.

Method: Introduces SurGE benchmark with (1) test instances (topic descriptions, expert-written surveys, full citation sets) and (2) large-scale academic corpus of 1M+ papers. Proposes automated evaluation framework measuring four dimensions: comprehensiveness, citation accuracy, structural organization, and content quality.

Result: Evaluation of diverse LLM-based methods shows significant performance gap, revealing that even advanced agentic frameworks struggle with survey generation complexities.

Conclusion: SurGE bridges a critical gap in automated survey generation research, highlighting the need for future work in this area, with all code, data, and models open-sourced.

Abstract: The rapid growth of academic literature makes the manual creation of scientific surveys increasingly infeasible. While large language models show promise for automating this process, progress in this area is hindered by the absence of standardized benchmarks and evaluation protocols. To bridge this critical gap, we introduce SurGE (Survey Generation Evaluation), a new benchmark for scientific survey generation in computer science. SurGE consists of (1) a collection of test instances, each including a topic description, an expert-written survey, and its full set of cited references, and (2) a large-scale academic corpus of over one million papers. In addition, we propose an automated evaluation framework that measures the quality of generated surveys across four dimensions: comprehensiveness, citation accuracy, structural organization, and content quality. Our evaluation of diverse LLM-based methods demonstrates a significant performance gap, revealing that even advanced agentic frameworks struggle with the complexities of survey generation and highlighting the need for future research in this area. We have open-sourced all the code, data, and models at: https://github.com/oneal2000/SurGE

Method: MIRAGE performs dynamic multi-chain inference over structured medical knowledge graphs by: 1) decomposing queries into entity-grounded sub-questions, 2) executing parallel inference chains, 3) adaptively retrieving evidence via neighbor expansion and multi-hop traversal, and 4) integrating answers using cross-chain verification.

Result: Experiments on three medical QA benchmarks (GenMedGPT-5k, CMCQA, and ExplainCPE) show MIRAGE consistently outperforms GPT-4o, Tree-of-Thought variants, and other retrieval-augmented baselines in both automatic and human evaluations.

Conclusion: MIRAGE improves both accuracy and interpretability for complex medical reasoning by generating explicit reasoning chains traceable to knowledge graph evidence, making it well-suited for medical QA scenarios.

Abstract: Large reasoning models (LRMs) have shown significant progress in test-time scaling through chain-of-thought prompting. Current approaches like search-o1 integrate retrieval augmented generation (RAG) into multi-step reasoning processes but rely on a single, linear reasoning chain while incorporating unstructured textual information in a flat, context-agnostic manner. As a result, these approaches can lead to error accumulation throughout the reasoning chain, which significantly limits its effectiveness in medical question-answering (QA) tasks where both accuracy and traceability are critical requirements. To address these challenges, we propose MIRAGE (Multi-chain Inference with Retrieval-Augmented Graph Exploration), a novel test-time scalable reasoning framework that performs dynamic multi-chain inference over structured medical knowledge graphs. Specifically, MIRAGE 1) decomposes complex queries into entity-grounded sub-questions, 2) executes parallel inference chains, 3) retrieves evidence adaptively via neighbor expansion and multi-hop traversal, and 4) integrates answers using cross-chain verification to resolve contradictions. Experiments on three medical QA benchmarks (GenMedGPT-5k, CMCQA, and ExplainCPE) show that MIRAGE consistently outperforms GPT-4o, Tree-of-Thought variants, and other retrieval-augmented baselines in both automatic and human evaluations. Additionally, MIRAGE improves interpretability by generating explicit reasoning chains that trace each factual claim to concrete chains within the knowledge graph, making it well-suited for complex medical reasoning scenarios. The code will be available for further research.

Method: Introduces SciReas (diverse scientific reasoning benchmarks) and SciReas-Pro (complex reasoning subset), plus KRUX framework to probe knowledge and reasoning roles separately in LLMs.

Result: Key findings: (1) Knowledge retrieval from model parameters is a critical bottleneck; (2) External knowledge boosts reasoning models; (3) Better verbalized reasoning improves knowledge surfacing.

Conclusion: The proposed benchmarks and framework enable holistic evaluation of scientific reasoning, revealing knowledge retrieval as the primary limitation and showing that reasoning enhancement improves knowledge utilization.

Abstract: Scientific problem solving poses unique challenges for LLMs, requiring both deep domain knowledge and the ability to apply such knowledge through complex reasoning. While automated scientific reasoners hold great promise for assisting human scientists, there is currently no widely adopted holistic benchmark for evaluating scientific reasoning, and few approaches systematically disentangle the distinct roles of knowledge and reasoning in these tasks. To address these gaps, we introduce SciReas, a diverse suite of existing benchmarks for scientific reasoning tasks, and SciReas-Pro, a selective subset that requires more complex reasoning. Our holistic evaluation surfaces insights about scientific reasoning performance that remain hidden when relying on individual benchmarks alone. We then propose KRUX, a probing framework for studying the distinct roles of reasoning and knowledge in scientific tasks. Combining the two, we conduct an in-depth analysis that yields several key findings: (1) Retrieving task-relevant knowledge from model parameters is a critical bottleneck for LLMs in scientific reasoning; (2) Reasoning models consistently benefit from external knowledge added in-context on top of the reasoning enhancement; (3) Enhancing verbalized reasoning improves LLMs’ ability to surface task-relevant knowledge.

Method: Three-stage system: 1) UBM processes all samples, retains high-confidence ones, forwards low-confidence to MLLM (with regression-to-classification conversion for uncertainty calculation). 2) MLLM initial processing - accepts samples where predictions match UBM polarity. 3) MLLM secondary inference on remaining samples using prompts with prior predictions as references.

Result: Extensive experiments show U-ACS maintains superior performance while significantly reducing computational overhead and resource consumption compared to full MLLM approaches.

Conclusion: U-ACS effectively balances MSA performance and efficiency by intelligently routing samples between lightweight baseline model and MLLMs based on uncertainty, achieving accurate sentiment analysis with reduced computational costs.

Abstract: Multimodal Large Language Models (MLLMs) have notably enhanced the performance of Multimodal Sentiment Analysis (MSA), yet their massive parameter scale leads to excessive resource consumption in training and inference, severely limiting model efficiency. To balance performance and efficiency for MSA, this paper innovatively proposes a novel Uncertainty-Aware Collaborative System (U-ACS) that integrates Uncertainty-aware Baseline Model (UBM) with MLLMs. U-ACS operates in three stages: First, all samples are processed by the UBM, retain high-confidence samples and forward low-confidence samples to the MLLM. Notably, to address the challenge that continuous outputs of regression tasks hinder uncertainty calculation, we innovatively convert the continuous sentiment label prediction task to a classification task, enabling a more accurate calculation of entropy and uncertainty. Second, the MLLM performs initial process. In this stage, high-confidence samples or low-confidence samples whose predictive sentiment polarity matches that of the UBM are deemed acceptable, while unqualified samples are forwarded for further processing. Finally, the MLLM performs secondary inference on remaining low-confidence samples using prompts augmented with prior rounds predictions as references. By aggregating results from the three stages, U-ACS preserves high MSA prediction accuracy while drastically boosting efficiency via offloading most simple samples to the UBM and minimizing MLLM processing volume. Extensive experiments verify that U-ACS maintains superior performance while significantly reducing computational overhead and resource consumption.

Method: Derives uncertainty indicators from accessible top candidate log-probabilities during non-greedy decoding. Uses Entropy Production Rate (EPR) as baseline, then enhances with supervised learning using entropic contributions of top-ranked tokens within single generated sequences.

Result: Significantly improves token-level hallucination detection over state-of-the-art methods across diverse QA datasets and multiple LLMs. High performance achieved using only small sets of available log-probabilities (e.g., top-10 per token).

Conclusion: Provides lightweight technique to enhance LLM trustworthiness at token level after single generation pass, suitable for API-constrained deployments in QA and RAG systems, validated on public datasets and financial framework.

Abstract: Hallucinations in Large Language Model (LLM) outputs for Question Answering (QA) tasks can critically undermine their real-world reliability. This paper introduces a methodology for robust, one-shot hallucination detection, specifically designed for scenarios with limited data access, such as interacting with black-box LLM APIs that typically expose only a few top candidate log-probabilities per token. Our approach derives uncertainty indicators directly from these readily available log-probabilities generated during non-greedy decoding. We first derive an Entropy Production Rate (EPR) that offers baseline performance, later augmented with supervised learning. Our learned model leverages the entropic contributions of the accessible top-ranked tokens within a single generated sequence, without multiple re-runs per query. Evaluated across diverse QA datasets and multiple LLMs, this estimator significantly improves token-level hallucination detection over state-of-the-art methods. Crucially, high performance is demonstrated using only the typically small set of available log-probabilities (e.g., top-10 per token), confirming its practical efficiency and suitability for API-constrained deployments. This work provides a lightweight technique to enhance the trustworthiness of LLM responses, at the token level, after a single generation pass, for QA and Retrieval-Augmented Generation (RAG) systems. Our experiments confirmed the performance of our method against existing approaches on public dataset as well as for a financial framework analyzing annual company reports.

Method: Created synthetic parallel datasets (3M English fables + 15K Romanian references), then two-stage fine-tuning: instruction tuning for narrative style, then adapter compression for efficiency.

Result: Fine-tuned model achieves strong fluency and adequacy, narrowing gap to proprietary models in automated and human evaluation while being open and cost-effective.

Conclusion: TF2 provides reproducible pipeline for cost-efficient literary translation, enabling broader adoption of open models for culturally significant content in low-resource settings.

Abstract: Literary translation has recently gained attention as a distinct and complex task in machine translation research. However, the translation by small open models remains an open problem. We contribute to this ongoing research by introducing TINYFABULIST TRANSLATION FRAMEWORK (TF2), a unified framework for dataset creation, fine-tuning, and evaluation in English-Romanian literary translations, centred on the creation and open release of both a compact, fine-tuned language model (TF2-12B) and large-scale synthetic parallel datasets (DS-TF2-EN-RO-3M and DS-TF2-EN-RO-15K). Building on DS-TF1-EN-3M (TF1), the largest collection of synthetic English fables to date, we address the need for rich, high-quality literary datasets in low-resource languages such as Romanian. Our pipeline first generates 15k high-quality Romanian references from the TF1 pool using a high-performing LLM. We then apply a two-stage fine-tuning process to a 12B-parameter open-weight model: (i) instruction tuning to capture genre-specific narrative style, and (ii) adapter compression for efficient deployment. Evaluation combines corpus-level BLEU and a five-dimension LLM-based rubric (accuracy, fluency, coherence, style, cultural adaptation) to provide a nuanced assessment of translation quality. Results show that our fine-tuned model achieves strong fluency and adequacy, narrowing the gap to top-performing proprietary models under automated and human-anchored evaluation, while being open, accessible, and significantly more cost-effective. Alongside the finetuned model, and both datasets, we publicly release all scripts and evaluation prompts. TF2 thus provides an end-to-end, reproducible pipeline for research on cost-efficient translation, cross-lingual narrative generation, and the broad adoption of open models for culturally significant literary content in low-resource settings.

Method: Three text-based augmentation methods: 1) gloss-based replacement, 2) random replacement, and 3) LLM-based text generation. Generated text is converted to synthetic audio using TTS. The synthetic data is combined with original audio to fine-tune a pretrained Wav2Vec2-XLSR-53 model.

Result: Significant performance gains across all four low-resource languages (Vatlongos, Nashta, Shinekhen Buryat, Kakabe), with up to 14.3% absolute WER reduction for Nashta. Methods also show utility for high-resource languages like English.

Conclusion: The proposed text-to-audio data augmentation methods are effective for low-resource ASR, requiring only original annotated data, and demonstrate broad applicability across diverse languages.

Abstract: This paper introduces three self-contained data augmentation methods for low-resource Automatic Speech Recognition (ASR). Our techniques first generate novel text–using gloss-based replacement, random replacement, or an LLM-based approach–and then apply Text-to-Speech (TTS) to produce synthetic audio. We apply these methods, which leverage only the original annotated data, to four languages with extremely limited resources (Vatlongos, Nashta, Shinekhen Buryat, and Kakabe). Fine-tuning a pretrained Wav2Vec2-XLSR-53 model on a combination of the original audio and generated synthetic data yields significant performance gains, including a 14.3% absolute WER reduction for Nashta. The methods prove effective across all four low-resource languages and also show utility for high-resource languages like English, demonstrating their broad applicability.

Method: Two-stage 1-to-N framework: Stage I computes shared post-attention hidden states once; Stage II clones this representation into parallel branches and steers them via a policy-reference mechanism for objective-specific control while maintaining cross-objective consistency.

Result: AMBS consistently improves HHH alignment across multiple 7B LLM backbones. On DeepSeek-7B, it improves average alignment scores by +32.4% and reduces unsafe outputs by 11.0% compared to naive 1-to-N baseline, while remaining competitive with SOTA methods.

Conclusion: AMBS provides an effective framework for unified multi-objective alignment that addresses both catastrophic forgetting and inference fragmentation, enabling consistent improvement across HHH objectives while maintaining efficiency.

Abstract: Alignment of Large Language Models (LLMs) along multiple objectives-helpfulness, harmlessness, and honesty (HHH)-is critical for safe and reliable deployment. Prior work has used steering vector-small control signals injected into hidden states-to guide LLM outputs, typically via one-to-one (1-to-1) Transformer decoders. In this setting, optimizing a single alignment objective can inadvertently overwrite representations learned for other objectives, leading to catastrophic forgetting. More recent approaches extend steering vectors via one-to-many (1-to-N) Transformer decoders. While this alleviates catastrophic forgetting, naive multi-branch designs optimize each objective independently, which can cause inference fragmentation-outputs across HHH objectives may become inconsistent. We propose Adaptive Multi-Branch Steering (AMBS), a two-stage 1-to-N framework for unified and efficient multi-objective alignment. In Stage I, post-attention hidden states of the Transformer layer are computed once to form a shared representation. In Stage II, this representation is cloned into parallel branches and steered via a policy-reference mechanism, enabling objective-specific control while maintaining cross-objective consistency. Empirical evaluations on Alpaca, BeaverTails, and TruthfulQA show that AMBS consistently improves HHH alignment across multiple 7B LLM backbones. For example, on DeepSeek-7B, AMBS improves average alignment scores by +32.4% and reduces unsafe outputs by 11.0% compared to a naive 1-to-N baseline, while remaining competitive with state-of-the-art methods.

Method: LOGOS integrates LLM-driven coding, semantic clustering, graph reasoning, and an iterative refinement process to build reusable codebooks. It also introduces a 5-dimensional metric and train-test split protocol for standardized evaluation.

Result: Across five diverse corpora, LOGOS consistently outperforms strong baselines and achieves 80.4% average alignment with expert-developed schemas on complex datasets.

Conclusion: LOGOS demonstrates potential to democratize and scale qualitative research without sacrificing theoretical nuance, offering a fully automated solution to grounded theory analysis.

Abstract: Grounded theory offers deep insights from qualitative data, but its reliance on expert-intensive manual coding presents a major scalability bottleneck. Existing computational tools either fail on full automation or lack flexible schema construction. We introduce LOGOS, a novel, end-to-end framework that fully automates the grounded theory workflow, transforming raw text into a structured, hierarchical theory. LOGOS integrates LLM-driven coding, semantic clustering, graph reasoning, and a novel iterative refinement process to build highly reusable codebooks. To ensure fair comparison, we also introduce a principled 5-dimensional metric and a train-test split protocol for standardized, unbiased evaluation. Across five diverse corpora, LOGOS consistently outperforms strong baselines and achieves a remarkable average $80.4%$ alignment with an expert-developed schema on complex datasets. LOGOS demonstrates a potential to democratize and scale qualitative research without sacrificing theoretical nuance.

Method: Proposes CoT Referring strategy that: 1) parses textual structures into sequential referring steps with chain-of-thought training data, 2) restructures training data with new annotations, 3) creates evaluation benchmark for complex cases, 4) integrates detection/segmentation in unified MLLM framework with adaptive weighted loss.

Result: Achieves notable 2.5%+ improvement over baseline models on curated benchmark and RefCOCO/+/g datasets, demonstrating effectiveness in complex referring scenarios.

Conclusion: CoT Referring effectively enhances multimodal reasoning through structured chain-of-thought training, improving performance on referring expression comprehension and segmentation tasks, particularly for complex queries.

Abstract: Referring Expression Comprehension and Segmentation are critical tasks for assessing the integration of language understanding and image comprehension, serving as benchmarks for Multimodal Large Language Models (MLLMs) capabilities. To address these challenges, we propose a new strategy, CoT Referring, which enhances model reasoning across modalities through a structured, chain-of-thought training data structure. Our approach systematically parses textual structures to a sequential referring step, where in each step it identifies relationships and ensures consistent reference alignment, thereby improving accuracy in complex query scenarios. We restructure the training data to enforce a new output form, providing new annotations for existing datasets and compiling an evaluation benchmark from existing resources. This benchmark is designed explicitly for complex referring cases. We also integrate detection and segmentation capabilities into a unified MLLM framework, training it with a novel adaptive weighted loss to optimize performance. Experimental results on our curated benchmark and RefCOCO/+/g demonstrate the effectiveness of our approach, with a notable increase of 2.5%+ over baseline models.

Method: The authors systematically examine five popular factuality benchmarks and eight LLMs released across different years. They develop an up-to-date fact retrieval pipeline and three metrics to quantify benchmark aging and its impact on LLM factuality evaluation.

Result: Experimental results show that a considerable portion of samples in widely used factuality benchmarks are outdated, leading to unreliable assessments of LLM factuality. The aging benchmarks provide misleading evaluations of modern LLMs.

Conclusion: The work provides a testbed to assess benchmark reliability for LLM factuality evaluation and aims to inspire more research on the benchmark aging issue. The authors have made their code publicly available.

Abstract: The rapid evolution of large language models (LLMs) and the real world has outpaced the static nature of widely used evaluation benchmarks, raising concerns about their reliability for evaluating LLM factuality. While substantial works continue to rely on the popular but old benchmarks, their temporal misalignment with real-world facts and modern LLMs, and their effects on LLM factuality evaluation remain underexplored. Therefore, in this work, we present a systematic investigation of this issue by examining five popular factuality benchmarks and eight LLMs released across different years. An up-to-date fact retrieval pipeline and three metrics are tailored to quantify benchmark aging and its impact on LLM factuality evaluation. Experimental results and analysis illustrate that a considerable portion of samples in the widely used factuality benchmarks are outdated, leading to unreliable assessments of LLM factuality. We hope our work can provide a testbed to assess the reliability of a benchmark for LLM factuality evaluation and inspire more research on the benchmark aging issue. Codes are available in https://github.com/JiangXunyi/BenchAge.

Method: Finetune open-sourced LLMs on misaligned completions across diverse domains; test in downstream combined finetuning (1% misalignment data); simulate human-AI interactions with benign and biased users.

Result: LLMs show broadly misaligned dishonesty behavior; 1% misalignment data decreases honest behavior over 20%; 10% biased user population unintentionally exacerbates assistant dishonesty.

Conclusion: Emergent misalignment extends to dishonesty/deception in high-stakes scenarios, with risks arising through direct finetuning, downstream mixture tasks, and practical human-AI interactions.

Abstract: Previous research has shown that LLMs finetuned on malicious or incorrect completions within narrow domains (e.g., insecure code or incorrect medical advice) can become broadly misaligned to exhibit harmful behaviors, which is called emergent misalignment. In this work, we investigate whether this phenomenon can extend beyond safety behaviors to a broader spectrum of dishonesty and deception under high-stakes scenarios (e.g., lying under pressure and deceptive behavior). To explore this, we finetune open-sourced LLMs on misaligned completions across diverse domains. Experimental results demonstrate that LLMs show broadly misaligned behavior in dishonesty. Additionally, we further explore this phenomenon in a downstream combined finetuning setting, and find that introducing as little as 1% of misalignment data into a standard downstream task is sufficient to decrease honest behavior over 20%. Furthermore, we consider a more practical human-AI interaction environment where we simulate both benign and biased users to interact with the assistant LLM. Notably, we find that the assistant can be misaligned unintentionally to exacerbate its dishonesty with only 10% biased user population. In summary, we extend the study of emergent misalignment to the domain of dishonesty and deception under high-stakes scenarios, and demonstrate that this risk arises not only through direct finetuning, but also in downstream mixture tasks and practical human-AI interactions. Refer to https://github.com/hxhcreate/LLM_Deceive_Unintentionally for experimental resources.

Method: Proposed a formal three-stage reasoning framework: 1) models cases using factual predicates called factors, 2) organizes them into legal knowledge hierarchy, 3) defines verifiable rules for identifying distinctions, analyzing argumentative support, and evaluating significance. Evaluated modern reasoning LLMs on these tasks.

Result: Found a paradox: high accuracy on surface-level reasoning (Task 1), degraded performance on hierarchical reasoning (Task 2: 64.82%-92.09%), and collapsed performance on integrated analysis (Task 3: 11.46%-33.99%). Most strikingly, models consistently used more computational resources on incorrect responses than correct ones.

Conclusion: LLMs have fundamental limitations in complex legal reasoning domains. The work provides methodology for fine-grained analysis of LLM reasoning capabilities and reveals that “thinking longer” doesn’t mean “thinking smarter” in complex reasoning tasks.

Abstract: Case-based reasoning is a cornerstone of U.S. legal practice, requiring professionals to argue about a current case by drawing analogies to and distinguishing from past precedents. While Large Language Models (LLMs) have shown remarkable capabilities, their proficiency in this complex, nuanced form of reasoning needs further investigation. We propose a formal framework that decomposes the process of identifying significant distinctions between cases into three-stage reasoning tasks. Our framework models cases using factual predicates called factors, organizes them into a legal knowledge hierarchy, and defines verifiable rules for identifying distinctions, analyzing their argumentative support, and evaluating their significance. Through comprehensive evaluation of modern reasoning LLMs, we reveal a paradox: while models achieve high accuracy on surface-level reasoning (Task 1), performance degrades on hierarchical reasoning (Task 2: 64.82%-92.09%) and collapses on integrated analysis (Task 3: 11.46%-33.99%). Most strikingly, we find that models consistently expend more computational resources on incorrect responses than correct ones, suggesting that “thinking longer” does not always mean “thinking smarter.” Our work provides a methodology for fine-grained analysis of LLM reasoning capabilities in complex domains and reveals fundamental limitations that must be addressed for robust and trustworthy legal AI.

Method: Two-component framework: (1) Schema Retriever with vector database of pre-computed schema embeddings using hard-negative supervised contrastive learning to distinguish similar but irrelevant columns; (2) SQL Generator fine-tuned in two stages (supervised fine-tuning + execution-guided reinforcement) enabling self-correction without multi-candidate sampling.

Result: Achieves 72.10% execution accuracy on BIRD and 88.45% on Spider 1.0, comparable or superior to LLM-based methods despite using 2x to 30x fewer parameters.

Conclusion: High-quality Text-to-SQL generation is feasible with lightweight models, offering practical solutions for privacy-sensitive and resource-constrained environments while maintaining competitive performance.

Abstract: The Text-to-SQL task translates natural language questions into SQL queries, enabling intuitive database interaction for non-experts. While recent methods leveraging Large Language Models (LLMs) achieve strong performance, their reliance on proprietary models raise concerns about deployment feasibility and data privacy. In this work, we introduce LitE-SQL, a Lightweight and Efficient framework with two components: (i) a Schema Retriever that performs efficient schema linking using a vector database of pre-computed schema embeddings, optimized with a hard-negative supervised contrastive objective to distinguish semantically similar but functionally irrelevant columns, and (ii) a SQL Generator fine-tuned in two stages-supervised fine-tuning followed by execution-guided reinforcement-enabling execution-guided self-correction without multi-candidate sampling, which is commonly required by prior LLM-based approaches. On BIRD, LitE-SQL achieves 72.10% execution accuracy, and on Spider 1.0 it reaches 88.45%, demonstrating comparable or superior performance to LLM-based methods despite using 2x to 30x fewer parameters. Our findings demonstrate that high-quality Text-to-SQL generation is feasible with lightweight models, offering a practical solution for privacy-sensitive and resource-constrained settings.

Method: Tested four LLM families (Gemini, Kimi-K2, Grok, Qwen) with narrow in-context examples to see if they produce misaligned responses to benign, unrelated queries. Varied number of examples (2-16) and tested scaling effects and reasoning capabilities.

Result: Emergent misalignment does occur in ICL: misalignment rates ranged 1-24% with 16 examples, appearing with as few as 2 examples. Neither larger model scale nor explicit reasoning provided reliable protection. Safety prioritization reduced EM while context-following increased it.

Conclusion: ICL is a previously underappreciated vector for emergent misalignment that operates without parameter modification and resists scaling-based solutions, highlighting the need for safety measures in ICL applications.

Abstract: Recent work has shown that narrow finetuning can produce broadly misaligned LLMs, a phenomenon termed emergent misalignment (EM). While concerning, these findings were limited to finetuning and activation steering, leaving out in-context learning (ICL). We therefore ask: does EM emerge in ICL? We find that it does: across four model families (Gemini, Kimi-K2, Grok, and Qwen), narrow in-context examples cause models to produce misaligned responses to benign, unrelated queries. With 16 in-context examples, EM rates range from 1% to 24% depending on model and domain, appearing with as few as 2 examples. Neither larger model scale nor explicit reasoning provides reliable protection. We formulate and test a hypothesis, which explains in-context EM as conflict between safety objectives and context-following behavior. Consistent with this, instructing models to prioritize safety reduces EM while prioritizing context-following increases it. These findings establish ICL as a previously underappreciated vector for emergent misalignment that operates without parameter modification and resists simple scaling-based solutions.

Method: Used multilingual Wug Test adaptation to test 6 models across 4 languages (Catalan, English, Greek, Spanish) and compared with human speakers on morphological generalization with novel words.

Result: Models generalized morphological processes with human-like accuracy, but accuracy patterns aligned more with community size/data availability than structural complexity. Spanish/English (larger communities) showed higher accuracy than Catalan/Greek.

Conclusion: Model behavior is driven primarily by linguistic resource richness rather than sensitivity to grammatical complexity, reflecting only superficial resemblance to human linguistic competence.

Abstract: The linguistic abilities of Large Language Models are a matter of ongoing debate. This study contributes to this discussion by investigating model performance in a morphological generalization task that involves novel words. Using a multilingual adaptation of the Wug Test, six models were tested across four partially unrelated languages (Catalan, English, Greek, and Spanish) and compared with human speakers. The aim is to determine whether model accuracy approximates human competence and whether it is shaped primarily by linguistic complexity or by the size of the linguistic community, which affects the quantity of available training data. Consistent with previous research, the results show that the models are able to generalize morphological processes to unseen words with human-like accuracy. However, accuracy patterns align more closely with community size and data availability than with structural complexity, refining earlier claims in the literature. In particular, languages with larger speaker communities and stronger digital representation, such as Spanish and English, revealed higher accuracy than less-resourced ones like Catalan and Greek. Overall, our findings suggest that model behavior is mainly driven by the richness of linguistic resources rather than by sensitivity to grammatical complexity, reflecting a form of performance that resembles human linguistic competence only superficially.

Method: CoRE uses two consistency mechanisms: token-level consistency applies a low-pass filter to downweight uncertain tokens with high inconsistency (addressing token misalignment), while model-level consistency promotes outputs with high self-confidence and minimal divergence from others.

Result: Extensive experiments across diverse benchmarks, model combinations, and ensemble strategies show CoRE consistently improves ensemble performance and robustness.

Conclusion: CoRE provides an effective plug-and-play solution for robust LLM ensembles by addressing both fine-grained token-level and coarse model-level inconsistencies, enhancing reliability across various ensemble methods.

Abstract: Different large language models (LLMs) exhibit diverse strengths and weaknesses, and LLM ensemble serves as a promising approach to integrate their complementary capabilities. Despite substantial progress in improving ensemble quality, limited attention has been paid to the robustness of ensembles against potential erroneous signals, which often arise from heterogeneous tokenization schemes and varying model expertise. Our analysis shows that ensemble failures typically arise from both the token level and the model level: the former reflects severe disagreement in token predictions, while the latter involves low confidence and pronounced disparities among models. In light of this, we propose CoRE, a plug-and-play technique that harnesses model consistency for robust LLM ensemble, which can be seamlessly integrated with diverse ensemble methods. Token-level consistency captures fine-grained disagreements by applying a low-pass filter to downweight uncertain tokens with high inconsistency, often due to token misalignment, thereby improving robustness at a granular level. Model-level consistency models global agreement by promoting model outputs with high self-confidence and minimal divergence from others, enhancing robustness at a coarser level. Extensive experiments across diverse benchmarks, model combinations, and ensemble strategies demonstrate that CoRE consistently improves ensemble performance and robustness. Our code is available at https://github.com/zhichenz98/CoRE-EACL26.

Method: Created CenterBench dataset with center-embedded sentences (relative clauses nested recursively) ranging from simple to deeply nested structures. Each sentence has a syntactically identical but semantically implausible counterpart. Includes six comprehension questions testing surface understanding, syntactic dependencies, and causal reasoning. Tested six models on this dataset.

Result: Performance gaps between plausible and implausible sentences widen systematically with complexity (up to 26.8 percentage points), showing models abandon structural analysis for semantic associations as complexity increases. Semantic plausibility actually harms performance on action questions where causal reasoning matters more. Reasoning models improve accuracy but show semantic shortcuts, overthinking, and answer refusal. Unlike models, humans show variable semantic effects.

Conclusion: CenterBench provides the first framework to identify when language models shift from structural analysis to pattern matching, revealing systematic differences between model and human processing of complex syntax.

Abstract: When language models correctly parse “The cat that the dog chased meowed,” are they analyzing syntax or simply familiar with dogs chasing cats? Despite extensive benchmarking, we lack methods to distinguish structural understanding from semantic pattern matching. We introduce CenterBench, a dataset of 9,720 comprehension questions on center-embedded sentences (like “The cat [that the dog chased] meowed”) where relative clauses nest recursively, creating processing demands from simple to deeply nested structures. Each sentence has a syntactically identical but semantically implausible counterpart (e.g., mailmen prescribe medicine, doctors deliver mail) and six comprehension questions testing surface understanding, syntactic dependencies, and causal reasoning. Testing six models reveals that performance gaps between plausible and implausible sentences widen systematically with complexity, with models showing median gaps up to 26.8 percentage points, quantifying when they abandon structural analysis for semantic associations. Notably, semantic plausibility harms performance on questions about resulting actions, where following causal relationships matters more than semantic coherence. Reasoning models improve accuracy but their traces show semantic shortcuts, overthinking, and answer refusal. Unlike models whose plausibility advantage systematically widens with complexity, humans shows variable semantic effects. CenterBench provides the first framework to identify when models shift from structural analysis to pattern matching.

Method: Created FICSIM dataset with long-form, recently written fiction, scoring similarity along 12 axes informed by author-produced metadata and validated by digital humanities scholars, while prioritizing author agency and informed consent.

Result: Evaluation of embedding models shows they tend to focus on surface-level features rather than semantic categories useful for computational literary studies tasks.

Conclusion: FICSIM provides a needed evaluation benchmark for embedding models in literary studies, revealing limitations in current models’ ability to capture meaningful semantic similarities for literary analysis.

Abstract: As language models become capable of processing increasingly long and complex texts, there has been growing interest in their application within computational literary studies. However, evaluating the usefulness of these models for such tasks remains challenging due to the cost of fine-grained annotation for long-form texts and the data contamination concerns inherent in using public-domain literature. Current embedding similarity datasets are not suitable for evaluating literary-domain tasks because of a focus on coarse-grained similarity and primarily on very short text. We assemble and release FICSIM, a dataset of long-form, recently written fiction, including scores along 12 axes of similarity informed by author-produced metadata and validated by digital humanities scholars. We evaluate a suite of embedding models on this task, demonstrating a tendency across models to focus on surface-level features over semantic categories that would be useful for computational literary studies tasks. Throughout our data-collection process, we prioritize author agency and rely on continual, informed author consent.

Method: Comparative analysis of instruction-tuned Mistral-Small 3 24B and reasoning-augmented QWen2.5 32B LLMs, evaluated against human benchmarks using 21 metrics spanning readability, discourse fidelity, content safety, and distributional measures.

Result: Mistral showed tempered lexical simplification achieving human-level discourse fidelity (BERTScore 0.91) while enhancing readability. QWen also improved readability but had poorer balance between readability and accuracy (BERTScore 0.89). Correlation analysis revealed strong metric redundancies.

Conclusion: Instruction-tuned LLMs like Mistral have architectural advantages for biomedical text simplification, better balancing readability and accuracy. The metric analysis provides guidance for metric selection and domain adaptation in text simplification tasks.

Abstract: The increasing health-seeking behavior and digital consumption of biomedical information by the general public necessitate scalable solutions for automatically adapting complex scientific and technical documents into plain language. Automatic text simplification solutions, including advanced large language models (LLMs), however, continue to face challenges in reliably arbitrating the tension between optimizing readability performance and ensuring preservation of discourse fidelity. This report empirically assesses two major classes of general-purpose LLMs, demonstrating how they navigate the readability-accuracy tension compared to a human benchmark. Using a comparative analysis of the instruction-tuned Mistral-Small 3 24B and the reasoning-augmented QWen2.5 32B, we identify an architectural advantage in the instruction-tuned LLM. Mistral exhibits a tempered lexical simplification strategy that enhances readability across a suite of metrics while preserving human-level discourse with a BERTScore of 0.91. QWen also attains enhanced readability performance and a reasonable BERTScore of 0.89, but its operational strategy shows a disconnect in balancing between readability and accuracy. Additionally, a comprehensive correlation analysis of a suite of 21 metrics spanning readability, discourse fidelity, content safety, and underlying distributional measures for mechanistic insights, confirms strong functional redundancies, and informs metric selection and domain adaptation for text simplification.

Method: 1) RPTS metric: Organizes reasoning steps into tree structure, uses hierarchical information to assign weighted faithfulness scores to each step. 2) RPTS-Eval benchmark: 374 images, 390 reasoning instances with visual-textual clues as leaf nodes. 3) Defines three types of intermodal relationships to study their influence on reasoning.

Result: Evaluated representative LVLMs (GPT4o, Llava-Next), revealing their limitations in multimodal reasoning and highlighting differences between open-source and closed-source models. The benchmark provides detailed insights into where models fail in reasoning processes.

Conclusion: RPTS and RPTS-Eval address critical gaps in multimodal reasoning evaluation by assessing reasoning process quality, not just answer correctness. This contributes to advancing multimodal reasoning research by providing more comprehensive evaluation tools.

Abstract: Large Vision-Language Models (LVLMs) excel in multimodal reasoning and have shown impressive performance on various multimodal benchmarks. However, most of these benchmarks evaluate models primarily through multiple-choice or short-answer formats, which do not take the reasoning process into account. Although some benchmarks assess the reasoning process, their methods are often overly simplistic and only examine reasoning when answers are incorrect. This approach overlooks scenarios where flawed reasoning leads to correct answers. In addition, these benchmarks do not consider the impact of intermodal relationships on reasoning. To address this issue, we propose the Reasoning Process Tree Score (RPTS), a tree structure-based metric to assess reasoning processes. Specifically, we organize the reasoning steps into a reasoning tree and leverage its hierarchical information to assign weighted faithfulness scores to each reasoning step. By dynamically adjusting these weights, RPTS not only evaluates the overall correctness of the reasoning, but also pinpoints where the model fails in the reasoning. To validate RPTS in real-world multimodal scenarios, we construct a new benchmark, RPTS-Eval, comprising 374 images and 390 reasoning instances. Each instance includes reliable visual-textual clues that serve as leaf nodes of the reasoning tree. Furthermore, we define three types of intermodal relationships to investigate how intermodal interactions influence the reasoning process. We evaluated representative LVLMs (e.g., GPT4o, Llava-Next), uncovering their limitations in multimodal reasoning and highlighting the differences between open-source and closed-source commercial LVLMs. We believe that this benchmark will contribute to the advancement of research in the field of multimodal reasoning.

Method: EasyMED uses a multi-agent framework that separates case-grounded information disclosure from response generation to support stable, inquiry-conditioned patient behavior. Also introduces SPBench, a human-grounded benchmark with eight expert-defined criteria for interaction-level evaluation.

Result: EasyMED more closely matches human SP behavior than existing VSPs, particularly in case consistency and controlled disclosure. A four-week controlled study shows learning outcomes comparable to human SP training, with stronger early gains for novice learners and improved flexibility, psychological safety, and cost efficiency.

Conclusion: EasyMED provides a stable, scalable alternative to human standardized patients that maintains training effectiveness while offering additional benefits in flexibility, safety, and cost efficiency.

Abstract: Standardized patients (SPs) are indispensable for clinical skills training but remain expensive and difficult to scale. Although large language model (LLM)-based virtual standardized patients (VSPs) have been proposed as an alternative, their behavior remains unstable and lacks rigorous comparison with human standardized patients. We propose EasyMED, a multi-agent VSP framework that separates case-grounded information disclosure from response generation to support stable, inquiry-conditioned patient behavior. We also introduce SPBench, a human-grounded benchmark with eight expert-defined criteria for interaction-level evaluation. Experiments show that EasyMED more closely matches human SP behavior than existing VSPs, particularly in case consistency and controlled disclosure. A four-week controlled study further demonstrates learning outcomes comparable to human SP training, with stronger early gains for novice learners and improved flexibility, psychological safety, and cost efficiency.

Method: 1) Created gold-standard benchmark with expert clinicians labeling clinical impact (No/Minimal/Significant) of ASR errors in doctor-patient dialogues. 2) Showed WER and existing metrics correlate poorly with clinical impact. 3) Introduced LLM-as-a-Judge framework optimized using GEPA through DSPy to replicate expert clinical assessment.

Result: The optimized judge (Gemini-2.5-Pro) achieved 90% accuracy and strong Cohen’s kappa of 0.816, demonstrating human-comparable performance. This provides a validated automated framework for clinical safety assessment.

Conclusion: The work provides a necessary, scalable framework for moving ASR evaluation beyond simple textual fidelity to assess clinical safety. The LLM-as-a-Judge approach offers automated, human-comparable assessment of clinical impact in dialogue transcription.

Abstract: As Automatic Speech Recognition (ASR) is increasingly deployed in clinical dialogue, standard evaluations still rely heavily on Word Error Rate (WER). This paper challenges that standard, investigating whether WER or other common metrics correlate with the clinical impact of transcription errors. We establish a gold-standard benchmark by having expert clinicians compare ground-truth utterances to their ASR-generated counterparts, labeling the clinical impact of any discrepancies found in two distinct doctor-patient dialogue datasets. Our analysis reveals that WER and a comprehensive suite of existing metrics correlate poorly with the clinician-assigned risk labels (No, Minimal, or Significant Impact). To bridge this evaluation gap, we introduce an LLM-as-a-Judge, programmatically optimized using GEPA through DSPy to replicate expert clinical assessment. The optimized judge (Gemini-2.5-Pro) achieves human-comparable performance, obtaining 90% accuracy and a strong Cohen’s kappa of 0.816. This work provides a validated, automated framework for moving ASR evaluation beyond simple textual fidelity to a necessary, scalable assessment of safety in clinical dialogue.

Method: Use conformal prediction framework with embedding- and LLM-based scoring functions to filter irrelevant content while preserving recall of supporting evidence with statistical coverage guarantees.

Result: Conformal filtering consistently meets target coverage, reduces context by 2-3x, improves ARGUE F1 on NeuCLIR under strict filtering, and maintains accuracy at moderate coverage levels.

Conclusion: Conformal prediction provides reliable, coverage-controlled context reduction for RAG systems, offering a model-agnostic and principled approach to context engineering.

Abstract: Retrieval-Augmented Generation (RAG) enhances factual grounding in large language models (LLMs) by incorporating retrieved evidence, but LLM accuracy declines when long or noisy contexts exceed the model’s effective attention span. Existing pre-generation filters rely on heuristics or uncalibrated LLM confidence scores, offering no statistical control over retained evidence. We evaluate and demonstrate context engineering through conformal prediction, a coverage-controlled filtering framework that removes irrelevant content while preserving recall of supporting evidence. Using both embedding- and LLM-based scoring functions, we test this approach on the NeuCLIR and RAGTIME collections. Conformal filtering consistently meets its target coverage, ensuring that a specified fraction of relevant snippets are retained, and reduces retained context by 2-3x relative to unfiltered retrieval. On NeuCLIR, downstream factual accuracy measured by ARGUE F1 improves under strict filtering and remains stable at moderate coverage, indicating that most discarded material is redundant or irrelevant. These results demonstrate that conformal prediction enables reliable, coverage-controlled context reduction in RAG, offering a model-agnostic and principled approach to context engineering.

Method: Proposed P-StaT framework tests belief stability via controlled semantic perturbations in representational (probing) and behavioral (zero-shot prompting) settings across 16 open-source LLMs and 3 domains, comparing epistemically familiar fictional statements vs unfamiliar synthetic statements.

Result: Synthetic unfamiliar content aligns closely with factual representations and causes largest belief retractions (up to 32.7% in representational, 36.3% in behavioral evaluations), while familiar fictional content is more representationally distinct and stable.

Conclusion: Epistemic familiarity serves as a robust signal for belief stability under semantic reframing, complementing accuracy-based factuality evaluation with a notion of epistemic robustness.

Abstract: Large language models (LLMs) are increasingly used as information sources, yet small changes in semantic framing can destabilize their truth judgments. We propose P-StaT (Perturbation Stability of Truth), an evaluation framework for testing belief stability under controlled semantic perturbations in representational and behavioral settings via probing and zero-shot prompting. Across sixteen open-source LLMs and three domains, we compare perturbations involving epistemically familiar Neither statements drawn from well-known fictional contexts (Fictional) to those involving unfamiliar Neither statements not seen in training data (Synthetic). We find a consistent stability hierarchy: Synthetic content aligns closely with factual representations and induces the largest retractions of previously held beliefs, producing up to $32.7%$ retractions in representational evaluations and up to $36.3%$ in behavioral evaluations. By contrast, Fictional content is more representationally distinct and comparatively stable. Together, these results suggest that epistemic familiarity is a robust signal across instantiations of belief stability under semantic reframing, complementing accuracy-based factuality evaluation with a notion of epistemic robustness.

Method: Developed a multilingual speech dataset with design principles, ethical considerations, and data collection procedures covering agriculture, healthcare, and general domain topics across 7 South African languages.

Result: Created a 3000-hour dataset and presented baseline results from training/finetuning ASR models, comparing performance with other existing ASR datasets for the target languages.

Conclusion: Swivuriso fills important resource gaps for ASR development in South African languages and provides a valuable benchmark dataset for future research and technology development in this domain.

Abstract: This paper introduces Swivuriso, a 3000-hour multilingual speech dataset developed as part of the African Next Voices project, to support the development and benchmarking of automatic speech recognition (ASR) technologies in seven South African languages. Covering agriculture, healthcare, and general domain topics, Swivuriso addresses significant gaps in existing ASR datasets. We describe the design principles, ethical considerations, and data collection procedures that guided the dataset creation. We present baseline results of training/finetuning ASR models with this data and compare to other ASR datasets for the langauges concerned.

Method: Two complementary techniques: Progressive Reward Shaping (PRS) - curriculum-inspired reward design with dense, stage-wise feedback (first master tool calls, then optimize correctness). Value-based Sampling Policy Optimization (VSPO) - enhanced GRPO variant that replaces zero-advantage samples with prompts selected by task-value metric and applies value-smoothing clipping.

Result: Experiments on multiple short-form and long-form QA benchmarks show PRS consistently outperforms traditional binary rewards, and VSPO achieves superior stability, faster convergence, and higher final performance compared to SFT, PPO and GRPO baselines.

Conclusion: Together, PRS and VSPO yield LLM-based TIR agents that generalize better across domains by addressing sparse reward and gradient degradation challenges in Agentic RL.

Abstract: Large Language Models (LLMs) empowered with Tool-Integrated Reasoning (TIR) can iteratively plan, call external tools, and integrate returned information to solve complex, long-horizon reasoning tasks. Agentic Reinforcement Learning (Agentic RL) optimizes such models over full tool-interaction trajectories, but two key challenges hinder effectiveness: (1) Sparse, non-instructive rewards, such as binary 0-1 verifiable signals, provide limited guidance for intermediate steps and slow convergence; (2) Gradient degradation in Group Relative Policy Optimization (GRPO), where identical rewards within a rollout group yield zero advantage, which reducing sample efficiency. To address these challenges, we propose two complementary techniques: Progressive Reward Shaping (PRS) and Value-based Sampling Policy Optimization (VSPO). PRS is a curriculum-inspired reward design that introduces dense, stage-wise feedback - encouraging models to first master parseable and properly formatted tool calls, then optimize for factual correctness and answer quality. We instantiate PRS for short-form QA (with a length-aware BLEU to fairly score concise answers) and long-form QA (with LLM-as-a-Judge scoring to prevent reward hacking). VSPO is an enhanced GRPO variant that replaces zero advantages samples with prompts selected by a task-value metric balancing difficulty and uncertainty, and applies value-smoothing clipping to stabilize gradient updates. Experiments on multiple short-form and long-form QA benchmarks show that PRS consistently outperforms traditional binary rewards, and VSPO achieves superior stability, faster convergence, and higher final performance compared to SFT, PPO and GRPO baselines. Together, PRS and VSPO yield LLM-based TIR agents that generalize better across domains.

Method: Proposes two metrics to detect proxy-policy conflicts: localized Proxy-Policy Alignment Conflict Score (PACS) and global Kendall-Tau Distance. Develops SHF-CAS algorithm that selectively targets high-conflict QA pairs for additional human feedback to refine both reward model and policy efficiently.

Result: Experiments on two alignment tasks show the approach enhances general alignment performance even when trained with biased proxy rewards.

Conclusion: Provides a new framework for interpreting alignment failures and offers a principled pathway for targeted refinement in LLM training by focusing on areas of shared ignorance between policy and reward models.

Abstract: Reward-model-based fine-tuning is a central paradigm in aligning Large Language Models with human preferences. However, such approaches critically rely on the assumption that proxy reward models accurately reflect intended supervision, a condition often violated due to annotation noise, bias, or limited coverage. This misalignment can lead to undesirable behaviors, where models optimize for flawed signals rather than true human values. In this paper, we investigate a novel framework to identify and mitigate such misalignment by treating the fine-tuning process as a form of knowledge integration. We focus on detecting instances of proxy-policy conflicts, cases where the base model strongly disagrees with the proxy. We argue that such conflicts often signify areas of shared ignorance, where neither the policy nor the reward model possesses sufficient knowledge, making them especially susceptible to misalignment. To this end, we propose two complementary metrics for identifying these conflicts: a localized Proxy-Policy Alignment Conflict Score (PACS) and a global Kendall-Tau Distance measure. Building on this insight, we design an algorithm named Selective Human-in-the-loop Feedback via Conflict-Aware Sampling (SHF-CAS) that targets high-conflict QA pairs for additional feedback, refining both the reward model and policy efficiently. Experiments on two alignment tasks demonstrate that our approach enhances general alignment performance, even when trained with a biased proxy reward. Our work provides a new lens for interpreting alignment failures and offers a principled pathway for targeted refinement in LLM training.

Method: Introduced evaluation protocol combining long persona dialogues (100+ rounds) with evaluation datasets to create dialogue-conditioned benchmarks for measuring long-context effects on seven state-of-the-art open- and closed-weight LLMs.

Result: Persona fidelity degrades over dialogues, especially in goal-oriented conversations. There’s a trade-off between fidelity and instruction following - non-persona baselines initially outperform persona models, but as fidelity fades, responses become similar to baselines.

Conclusion: Persona applications are fragile in extended interactions. The proposed protocol provides systematic measurement of such failures, highlighting the need for better long-context persona maintenance in LLMs.

Abstract: Persona-assigned large language models (LLMs) are used in domains such as education, healthcare, and sociodemographic simulation. Yet, they are typically evaluated only in short, single-round settings that do not reflect real-world usage. We introduce an evaluation protocol that combines long persona dialogues (over 100 rounds) and evaluation datasets to create dialogue-conditioned benchmarks that can robustly measure long-context effects. We then investigate the effects of dialogue length on persona fidelity, instruction-following, and safety of seven state-of-the-art open- and closed-weight LLMs. We find that persona fidelity degrades over the course of dialogues, especially in goal-oriented conversations, where models must sustain both persona fidelity and instruction following. We identify a trade-off between fidelity and instruction following, with non-persona baselines initially outperforming persona-assigned models; as dialogues progress and fidelity fades, persona responses become increasingly similar to baseline responses. Our findings highlight the fragility of persona applications in extended interactions and our work provides a protocol to systematically measure such failures.

Method: Created MMRB2 benchmark with: (1) practical but challenging prompts, (2) responses from state-of-the-art models and agents, (3) preference pairs with strong human-expert consensus using ensemble filtering strategy. Evaluated existing judges including multimodal LLM-as-a-judge and human-preference-trained models.

Result: Gemini 3 Pro achieves 75-80% accuracy, GPT-5 and Gemini 2.5 Pro reach 66-75%, surpassing GPT-4o (59%). Best open-source model Qwen3-VL-32B matches Gemini 2.5 Flash (64%). Human accuracy exceeds 90%. MMRB2 performance strongly correlates with downstream task success via Best-of-N sampling.

Conclusion: MMRB2 provides the first comprehensive benchmark for multimodal reward models, revealing significant gaps between current models and human performance, and identifies key areas for improvement in reward models for multimodal tasks.

Abstract: Reward models (RMs) are essential for training large language models (LLMs), but remain underexplored for omni models that handle interleaved image and text sequences. We introduce Multimodal RewardBench 2 (MMRB2), the first comprehensive benchmark for reward models on multimodal understanding and (interleaved) generation. MMRB2 spans four tasks: text-to-image, image editing, interleaved generation, and multimodal reasoning (“thinking-with-images”), providing 1,000 expert-annotated preference pairs per task from 23 models and agents across 21 source tasks. MMRB2 is designed with: (1) practical but challenging prompts; (2) responses from state-of-the-art models and agents; and (3) preference pairs with strong human-expert consensus, curated via an ensemble filtering strategy. Using MMRB2, we study existing judges for each subtask, including multimodal LLM-as-a-judge and models trained with human preferences. The latest Gemini 3 Pro attains 75-80% accuracy. GPT-5 and Gemini 2.5 Pro reach 66-75% accuracy, compared to >90% for humans, yet surpass the widely used GPT-4o (59%). The best performing open-source model Qwen3-VL-32B achieves similar accuracies as Gemini 2.5 Flash (64%). We also show that MMRB2 performance strongly correlates with downstream task success using Best-of-N sampling and conduct an in-depth analysis that shows key areas to improve the reward models going forward.

Method: Proposes LiR³AG (Lightweight Rerank Reasoning Strategy Framework for RAG) that enables non-reasoning models to transfer reasoning strategies by restructuring retrieved evidence into coherent reasoning chains. The framework identifies two reasoning modes used by reasoning models: Context-Grounded Reasoning (relying on retrieved content) and Knowledge-Reconciled Reasoning (resolving conflicts/gaps using internal knowledge).

Result: LiR³AG significantly reduces computational overhead: 98% reduction in average output tokens and 58.6% reduction in inference time. It improves 8B non-reasoning model’s F1 performance by 6.2% to 22.5%, enabling it to surpass the performance of 32B reasoning models in RAG tasks.

Conclusion: The proposed LiR³AG framework offers a practical and efficient path forward for RAG systems by enabling non-reasoning models to achieve reasoning-level performance with dramatically reduced computational costs, effectively addressing the performance-efficiency trade-off in multi-hop QA tasks.

Abstract: Retrieval-Augmented Generation (RAG) effectively enhances Large Language Models (LLMs) by incorporating retrieved external knowledge into the generation process. Reasoning models improve LLM performance in multi-hop QA tasks, which require integrating and reasoning over multiple pieces of evidence across different documents to answer a complex question. However, they often introduce substantial computational costs, including increased token consumption and inference latency. To better understand and mitigate this trade-off, we conduct a comprehensive study of reasoning strategies for reasoning models in RAG multi-hop QA tasks. Our findings reveal that reasoning models adopt structured strategies to integrate retrieved and internal knowledge, primarily following two modes: Context-Grounded Reasoning, which relies directly on retrieved content, and Knowledge-Reconciled Reasoning, which resolves conflicts or gaps using internal knowledge. To this end, we propose a novel Lightweight Rerank Reasoning Strategy Framework for RAG (LiR$^3$AG) to enable non-reasoning models to transfer reasoning strategies by restructuring retrieved evidence into coherent reasoning chains. LiR$^3$AG significantly reduce the average 98% output tokens overhead and 58.6% inferencing time while improving 8B non-reasoning model’s F1 performance ranging from 6.2% to 22.5% to surpass the performance of 32B reasoning model in RAG, offering a practical and efficient path forward for RAG systems.

Method: Multi-stage training: 1) Continual Pre-training on financial corpora, 2) Fine-grained Post-training pipeline with increasing specificity: Financial SFT → Finance Reasoning RL → Finance Agentic RL → General RL aligned with real-world business scenarios.

Result: QianfanHuijin achieves superior performance across various authoritative financial benchmarks. Ablation studies confirm that Reasoning RL and Agentic RL stages yield significant gains in their respective capabilities.

Conclusion: The fine-grained, progressive post-training methodology is validated and poised to become a mainstream paradigm for various industrial-enhanced LLMs, addressing the growing demand for models with comprehensive financial capabilities.

Abstract: Domain-specific enhancement of Large Language Models (LLMs) within the financial context has long been a focal point of industrial application. While previous models such as BloombergGPT and Baichuan-Finance primarily focused on knowledge enhancement, the deepening complexity of financial services has driven a growing demand for models that possess not only domain knowledge but also robust financial reasoning and agentic capabilities. In this paper, we present QianfanHuijin, a financial domain LLM, and propose a generalizable multi-stage training paradigm for industrial model enhancement. Our approach begins with Continual Pre-training (CPT) on financial corpora to consolidate the knowledge base. This is followed by a fine-grained Post-training pipeline designed with increasing specificity: starting with Financial SFT, progressing to Finance Reasoning RL and Finance Agentic RL, and culminating in General RL aligned with real-world business scenarios. Empirical results demonstrate that QianfanHuijin achieves superior performance across various authoritative financial benchmarks. Furthermore, ablation studies confirm that the targeted Reasoning RL and Agentic RL stages yield significant gains in their respective capabilities. These findings validate our motivation and suggest that this fine-grained, progressive post-training methodology is poised to become a mainstream paradigm for various industrial-enhanced LLMs.

Method: CREST has two components: (1) offline calibration to identify cognitive heads (correlated with behaviors like verification/backtracking) and derive head-specific steering vectors, and (2) inference-time procedure that rotates hidden representations to suppress components along those vectors.

Result: Across diverse reasoning benchmarks and models, CREST improves accuracy by up to 17.5% while reducing token usage by 37.6%.

Conclusion: CREST offers a simple and effective pathway to faster, more reliable LLM reasoning by adaptively suppressing unproductive reasoning behaviors without requiring training.

Abstract: Large Language Models (LLMs) often rely on long chain-of-thought (CoT) reasoning to solve complex tasks. While effective, these trajectories are frequently inefficient, leading to high latency from excessive token generation, or unstable reasoning that alternates between underthinking (shallow, inconsistent steps) and overthinking (repetitive, verbose reasoning). In this work, we study the structure of reasoning trajectories and uncover specialized attention heads that correlate with distinct cognitive behaviors such as verification and backtracking. By lightly intervening on these heads at inference time, we can steer the model away from inefficient modes. Building on this insight, we propose CREST, a training-free method for Cognitive REasoning Steering at Test-time. CREST has two components: (1) an offline calibration step that identifies cognitive heads and derives head-specific steering vectors, and (2) an inference-time procedure that rotates hidden representations to suppress components along those vectors. CREST adaptively suppresses unproductive reasoning behaviors, yielding both higher accuracy and lower computational cost. Across diverse reasoning benchmarks and models, CREST improves accuracy by up to 17.5% while reducing token usage by 37.6%, offering a simple and effective pathway to faster, more reliable LLM reasoning.

Method: Analyzes dynamic topology of layer-wise attention evolution by modeling attention matrices as zigzag graph filtrations and using zigzag persistence to extract topological signatures that distinguish factual from hallucinated generations.

Result: HalluZig outperforms strong baselines on multiple benchmarks, and topological signatures are generalizable across different models, with detection possible using structural signatures from partial network depth.

Conclusion: Topological analysis of attention patterns provides an effective new paradigm for hallucination detection that captures internal reasoning failures and offers cross-model generalizability.

Abstract: The factual reliability of Large Language Models (LLMs) remains a critical barrier to their adoption in high-stakes domains due to their propensity to hallucinate. Current detection methods often rely on surface-level signals from the model’s output, overlooking the failures that occur within the model’s internal reasoning process. In this paper, we introduce a new paradigm for hallucination detection by analyzing the dynamic topology of the evolution of model’s layer-wise attention. We model the sequence of attention matrices as a zigzag graph filtration and use zigzag persistence, a tool from Topological Data Analysis, to extract a topological signature. Our core hypothesis is that factual and hallucinated generations exhibit distinct topological signatures. We validate our framework, HalluZig, on multiple benchmarks, demonstrating that it outperforms strong baselines. Furthermore, our analysis reveals that these topological signatures are generalizable across different models and hallucination detection is possible only using structural signatures from partial network depth.

Method: Deferred Commitment Decoding (DCD) maintains a certainty-aware sliding window over masked tokens, resolving low-uncertainty tokens early while deferring high-uncertainty tokens until sufficient contextual evidence becomes available, without requiring additional training.

Result: DCD improves generation accuracy by 1.73% on average compared to fixed block-based methods, with the most significant improvement reaching 16.5%, while maintaining comparable inference time across multiple diffusion language models and benchmarks.

Conclusion: Deferring token commitment based on uncertainty is a simple yet effective principle for improving both the quality and efficiency of diffusion language model decoding, particularly benefiting reasoning-intensive tasks.

Abstract: Diffusion language models (DLMs) have recently emerged as a strong alternative to autoregressive models by enabling parallel text generation. To improve inference efficiency and KV-cache compatibility, prior work commonly adopts block-based diffusion, decoding tokens block by block. However, this paradigm suffers from a structural limitation that we term Boundary-Induced Context Truncation (BICT): undecoded tokens near block boundaries are forced to commit without access to nearby future context, even when such context could substantially reduce uncertainty. This limitation degrades decoding certainty and generation quality, especially for tasks requiring precise reasoning, such as mathematical problem solving and code generation. We propose Deferred Commitment Decoding (DCD), a novel, training-free decoding strategy that mitigates this issue. DCD maintains a certainty-aware sliding window over masked tokens, resolving low-uncertainty tokens early while deferring high-uncertainty tokens until sufficient contextual evidence becomes available. Extensive experiments across multiple diffusion language models, benchmarks, and caching configurations show that DCD improves generation accuracy by 1.73% with comparable time on average compared to fixed block-based diffusion methods, with the most significant improvement reaching 16.5%. These results demonstrate that deferring token commitment based on uncertainty is a simple yet effective principle for improving both the quality and efficiency of diffusion language model decoding.

Method: DeCode is a training-free, model-agnostic framework that adapts existing LLMs to produce contextualized answers in clinical settings without requiring additional model training.

Result: DeCode improves the state-of-the-art performance on OpenAI HealthBench from 28.4% to 49.8%, representing a 75% relative improvement in clinical relevance and validity of LLM responses.

Conclusion: DeCode effectively improves clinical question answering by enabling LLMs to produce more contextualized, patient-aligned responses without requiring model retraining, demonstrating significant performance gains on comprehensive clinical benchmarks.

Abstract: Large language models (LLMs) exhibit strong medical knowledge and can generate factually accurate responses. However, existing models often fail to account for individual patient contexts, producing answers that are clinically correct yet poorly aligned with patients’ needs. In this work, we introduce DeCode, a training-free, model-agnostic framework that adapts existing LLMs to produce contextualized answers in clinical settings. We evaluate DeCode on OpenAI HealthBench, a comprehensive and challenging benchmark designed to assess clinical relevance and validity of LLM responses. DeCode improves the previous state of the art from $28.4%$ to $49.8%$, corresponding to a $75%$ relative improvement. Experimental results suggest the effectiveness of DeCode in improving clinical question answering of LLMs.

Method: Cross-Architecture Proxy Tuning (CAPT) uses model ensembling with contrastive decoding to selectively inject clinically relevant signals from existing clinical models into new general-domain models, supporting models with disjoint vocabularies without training.

Result: CAPT consistently outperforms both individual models and state-of-the-art ensembling approaches (+17.6% over UniTE, +41.4% over proxy tuning across six clinical tasks), amplifies clinically actionable language, reduces context errors, and increases clinical specificity.

Conclusion: CAPT provides an effective training-free approach for adapting new general-domain LLMs to clinical applications using existing clinical models, offering significant performance improvements while preserving general-domain reasoning and fluency.

Abstract: Adapting language models to the clinical domain through continued pretraining and fine-tuning requires costly retraining for each new model generation. We propose Cross-Architecture Proxy Tuning (CAPT), a model-ensembling approach that enables training-free adaptation of state-of-the-art general-domain models using existing clinical models. CAPT supports models with disjoint vocabularies, leveraging contrastive decoding to selectively inject clinically relevant signals while preserving the general-domain model’s reasoning and fluency. On six clinical classification and text-generation tasks, CAPT with a new-generation general-domain model and an older-generation clinical model consistently outperforms both models individually and state-of-the-art ensembling approaches (average +17.6% over UniTE, +41.4% over proxy tuning across tasks). Through token-level analysis and physician case studies, we demonstrate that CAPT amplifies clinically actionable language, reduces context errors, and increases clinical specificity.

Method: Proposes Self-Graph Reasoning (SGR) framework where LLMs explicitly represent reasoning as structured graphs before final answers. Constructs a graph-structured reasoning dataset by merging multiple candidate reasoning graphs into refined structures for model training.

Result: Experiments on five QA benchmarks show SGR consistently improves reasoning consistency with 17.74% gain over base model. LLaMA-3.3-70B fine-tuned with SGR performs comparably to GPT-4o and surpasses Claude-3.5-Haiku.

Conclusion: Graph-structured reasoning effectively addresses limitations of linear reasoning in LLMs, demonstrating significant improvements in consistency and performance across general and specialized domains.

Abstract: Large Language Models (LLMs) show strong reasoning ability in open-domain question answering, yet their reasoning processes are typically linear and often logically inconsistent. In contrast, real-world reasoning requires integrating multiple premises and solving subproblems in parallel. Existing methods, such as Chain-of-Thought (CoT), express reasoning in a linear textual form, which may appear coherent but frequently leads to inconsistent conclusions. Recent approaches rely on externally provided graphs and do not explore how LLMs can construct and use their own graph-structured reasoning, particularly in open-domain QA. To fill this gap, we novelly explore graph-structured reasoning of LLMs in general-domain question answering. We propose Self-Graph Reasoning (SGR), a framework that enables LLMs to explicitly represent their reasoning process as a structured graph before producing the final answer. We further construct a graph-structured reasoning dataset that merges multiple candidate reasoning graphs into refined graph structures for model training. Experiments on five QA benchmarks across both general and specialized domains show that SGR consistently improves reasoning consistency and yields a 17.74% gain over the base model. The LLaMA-3.3-70B model fine-tuned with SGR performs comparably to GPT-4o and surpasses Claude-3.5-Haiku, demonstrating the effectiveness of graph-structured reasoning.

Method: Two-stage approach: (1) ELO Pretraining - detach and train only the critically important first and last layers on target language data, reducing trainable parameters and forward pass computations; (2) Layer Alignment - reintegrate trained layers and perform brief full fine-tuning on small dataset to align parameters.

Result: Achieves up to 6.46x training speedup compared to existing methods, improves target language performance by up to 6.2% on qualitative benchmarks, and effectively preserves source language (English) capabilities.

Conclusion: ELO method provides an efficient solution for continual pretraining of multilingual LLMs, significantly reducing computational costs while enhancing target language performance and maintaining source language proficiency.

Abstract: We propose an efficient layer-specific optimization (ELO) method designed to enhance continual pretraining (CP) for specific languages in multilingual large language models (MLLMs). This approach addresses the common challenges of high computational cost and degradation of source language performance associated with traditional CP. The ELO method consists of two main stages: (1) ELO Pretraining, where a small subset of specific layers, identified in our experiments as the critically important first and last layers, are detached from the original MLLM and trained with the target language. This significantly reduces not only the number of trainable parameters but also the total parameters computed during the forward pass, minimizing GPU memory consumption and accelerating the training process. (2) Layer Alignment, where the newly trained layers are reintegrated into the original model, followed by a brief full fine-tuning step on a small dataset to align the parameters. Experimental results demonstrate that the ELO method achieves a training speedup of up to 6.46 times compared to existing methods, while improving target language performance by up to 6.2% on qualitative benchmarks and effectively preserving source language (English) capabilities.

Method: Membox introduces a hierarchical memory architecture with two key components: 1) Topic Loom - continuously monitors dialogue in sliding-window fashion, grouping consecutive same-topic turns into coherent “memory boxes” at storage time; 2) Trace Weaver - links sealed boxes into long-range event-timeline traces to recover macro-topic recurrences across discontinuities.

Result: Experiments on LoCoMo show Membox achieves up to 68% F1 improvement on temporal reasoning tasks, outperforming competitive baselines (Mem0, A-MEM). Notably, it attains these gains while using only a fraction of the context tokens required by existing methods, demonstrating superior balance between efficiency and effectiveness.

Conclusion: By explicitly modeling topic continuity, Membox offers a cognitively motivated mechanism that enhances both coherence and efficiency in LLM agents, addressing fundamental limitations of current fragmentation-compensation approaches to dialogue memory systems.

Abstract: Human-agent dialogues often exhibit topic continuity-a stable thematic frame that evolves through temporally adjacent exchanges-yet most large language model (LLM) agent memory systems fail to preserve it. Existing designs follow a fragmentation-compensation paradigm: they first break dialogue streams into isolated utterances for storage, then attempt to restore coherence via embedding-based retrieval. This process irreversibly damages narrative and causal flow, while biasing retrieval towards lexical similarity. We introduce membox, a hierarchical memory architecture centered on a Topic Loom that continuously monitors dialogue in a sliding-window fashion, grouping consecutive same-topic turns into coherent “memory boxes” at storage time. Sealed boxes are then linked by a Trace Weaver into long-range event-timeline traces, recovering macro-topic recurrences across discontinuities. Experiments on LoCoMo demonstrate that Membox achieves up to 68% F1 improvement on temporal reasoning tasks, outperforming competitive baselines (e.g., Mem0, A-MEM). Notably, Membox attains these gains while using only a fraction of the context tokens required by existing methods, highlighting a superior balance between efficiency and effectiveness. By explicitly modeling topic continuity, Membox offers a cognitively motivated mechanism for enhancing both coherence and efficiency in LLM agents.

Method: PRISM uses a Process Reward Model (PRM) to guide learning alongside the model’s internal confidence, effectively combining external process evaluation with self-certainty metrics to enable stable unsupervised training without ground-truth labels.

Result: The combination of PRM with self-certainty leads to both stable training and better test-time performance while keeping the model’s internal confidence in check.

Conclusion: PRISM provides a unified framework for effective unsupervised learning from unlabeled data by addressing the unreliability of pure internal consistency metrics through the integration of process reward models with self-certainty.

Abstract: Current techniques for post-training Large Language Models (LLMs) rely either on costly human supervision or on external verifiers to boost performance on tasks such as mathematical reasoning and code generation. However, as LLMs improve their problem-solving, any further improvement will potentially require high-quality solutions to difficult problems that are not available to humans. As a result, learning from unlabeled data is becoming increasingly attractive in the research community. Existing methods extract learning signal from a model’s consistency, either by majority voting or by converting the model’s internal confidence into reward. Although internal consistency metric such as entropy or self-certainty require no human intervention, as we show in this work, these are unreliable signals for large-scale and long-term training. To address the unreliability, we propose PRISM, a unified training framework that uses a Process Reward Model (PRM) to guide learning alongside model’s internal confidence in the absence of ground-truth labels. We show that effectively combining PRM with self-certainty can lead to both stable training and better test-time performance, and also keep the model’s internal confidence in check. Code available at https://github.com/ghimiremukesh/PRISM.

Method: Two complementary models: Qwen3-VL-Embedding uses multi-stage training (contrastive pre-training → reranking distillation) with Matryoshka Representation Learning for flexible dimensions; Qwen3-VL-Reranker uses cross-encoder architecture with cross-attention for fine-grained relevance estimation. Both support 30+ languages and 32k token inputs.

Result: Qwen3-VL-Embedding-8B achieves state-of-the-art results with 77.8 overall score on MMEB-V2 (ranked first as of Jan 8, 2025). Both 2B and 8B parameter models show effectiveness across multimodal retrieval tasks including image-text retrieval, VQA, and video-text matching.

Conclusion: The Qwen3-VL-Embedding and Qwen3-VL-Reranker series provide a comprehensive solution for multimodal search, demonstrating superior performance on benchmarks while offering flexible deployment options through different parameter sizes and supporting diverse real-world applications.

Abstract: In this report, we introduce the Qwen3-VL-Embedding and Qwen3-VL-Reranker model series, the latest extensions of the Qwen family built on the Qwen3-VL foundation model. Together, they provide an end-to-end pipeline for high-precision multimodal search by mapping diverse modalities, including text, images, document images, and video, into a unified representation space. The Qwen3-VL-Embedding model employs a multi-stage training paradigm, progressing from large-scale contrastive pre-training to reranking model distillation, to generate semantically rich high-dimensional vectors. It supports Matryoshka Representation Learning, enabling flexible embedding dimensions, and handles inputs up to 32k tokens. Complementing this, Qwen3-VL-Reranker performs fine-grained relevance estimation for query-document pairs using a cross-encoder architecture with cross-attention mechanisms. Both model series inherit the multilingual capabilities of Qwen3-VL, supporting more than 30 languages, and are released in $\textbf{2B}$ and $\textbf{8B}$ parameter sizes to accommodate diverse deployment requirements. Empirical evaluations demonstrate that the Qwen3-VL-Embedding series achieves state-of-the-art results across diverse multimodal embedding evaluation benchmarks. Specifically, Qwen3-VL-Embedding-8B attains an overall score of $\textbf{77.8}$ on MMEB-V2, ranking first among all models (as of January 8, 2025). This report presents the architecture, training methodology, and practical capabilities of the series, demonstrating their effectiveness on various multimodal retrieval tasks, including image-text retrieval, visual question answering, and video-text matching.

Method: Proposed a two-component framework: 1) query generator that creates optimal queries to retrieve persuasion-relevant records from user history, and 2) profiler that summarizes these records into profiles to inform persuasiveness prediction models.

Result: Evaluation on ChangeMyView Reddit dataset shows consistent improvements over existing methods across multiple predictor models, with gains up to +13.77%p in F1 score. Analysis reveals effective profiles are context-dependent and predictor-specific.

Conclusion: Task-oriented, context-dependent user profiling is crucial for personalized persuasiveness prediction, moving beyond static attributes or surface-level similarity to leverage user history effectively.

Abstract: Estimating the persuasiveness of messages is critical in various applications, from recommender systems to safety assessment of LLMs. While it is imperative to consider the target persuadee’s characteristics, such as their values, experiences, and reasoning styles, there is currently no established systematic framework to optimize leveraging a persuadee’s past activities (e.g., conversations) to the benefit of a persuasiveness prediction model. To address this problem, we propose a context-aware user profiling framework with two trainable components: a query generator that generates optimal queries to retrieve persuasion-relevant records from a user’s history, and a profiler that summarizes these records into a profile to effectively inform the persuasiveness prediction model. Our evaluation on the ChangeMyView Reddit dataset shows consistent improvements over existing methods across multiple predictor models, with gains of up to +13.77%p in F1 score. Further analysis shows that effective user profiles are context-dependent and predictor-specific, rather than relying on static attributes or surface-level similarity. Together, these results highlight the importance of task-oriented, context-dependent user profiling for personalized persuasiveness prediction.

Method: Uses label-balanced mean baseline and task-directional relevance scoring to identify sparse task-relevant attention heads in a proxy-language checkpoint, then transfers to target language by updating only those heads (plus LayerNorm) via head-level gradient masking.

Result: CT-SFT improves cross-lingual accuracy over continued full fine-tuning on NusaX-Senti and XNLI while updating only a small subset of parameters. Shows editing-preserving trade-off and substantially reduces catastrophic forgetting.

Conclusion: CT-SFT provides an effective approach for low-resource language adaptation that balances task transfer with preservation of source-language competence, addressing key challenges in cross-lingual LLM adaptation.

Abstract: Adapting LLMs to low-resource languages is difficult: labeled data is scarce, full-model fine-tuning is unstable, and continued cross-lingual tuning can cause catastrophic forgetting. We propose Circuit-Targeted Supervised Fine-Tuning (CT-SFT): a counterfactual-free adaptation of CD-T (Contextual Decomposition Transformer) that uses a label-balanced mean baseline and task-directional relevance scoring to identify a sparse set of task-relevant attention heads in a proxy-language checkpoint, then transfer learns to a target language by updating only those heads (plus LayerNorm) via head-level gradient masking. Across NusaX-Senti and XNLI, CT-SFT improves cross-lingual accuracy over continued full fine-tuning while updating only a small subset of model parameters. We find an editing-preserving trade-off: harder transfers favor editing circuit heads, while easier transfers often favor near-zero (i.e., low-relevance heads) updates, preserving the source mechanism. CT-SFT also substantially reduces catastrophic forgetting, preserving proxy/source-language competence during transfer.

Method: Med-CoReasoner uses language-informed co-reasoning: elicits parallel English and local-language reasoning, abstracts them into structured concepts, integrates local clinical knowledge into English logical scaffold via concept-level alignment and retrieval.

Result: Improves multilingual reasoning performance by average 5% across three benchmarks, with substantial gains in low-resource languages. Produces clinically sound and culturally grounded reasoning traces confirmed by expert evaluation.

Conclusion: Med-CoReasoner effectively bridges multilingual medical reasoning gap by combining structural robustness of English reasoning with practice-grounded expertise in local languages, enabling more equitable global medical AI deployment.

Abstract: While reasoning-enhanced large language models perform strongly on English medical tasks, a persistent multilingual gap remains, with substantially weaker reasoning in local languages, limiting equitable global medical deployment. To bridge this gap, we introduce Med-CoReasoner, a language-informed co-reasoning framework that elicits parallel English and local-language reasoning, abstracts them into structured concepts, and integrates local clinical knowledge into an English logical scaffold via concept-level alignment and retrieval. This design combines the structural robustness of English reasoning with the practice-grounded expertise encoded in local languages. To evaluate multilingual medical reasoning beyond multiple-choice settings, we construct MultiMed-X, a benchmark covering seven languages with expert-annotated long-form question answering and natural language inference tasks, comprising 350 instances per language. Experiments across three benchmarks show that Med-CoReasoner improves multilingual reasoning performance by an average of 5%, with particularly substantial gains in low-resource languages. Moreover, model distillation and expert evaluation analysis further confirm that Med-CoReasoner produces clinically sound and culturally grounded reasoning traces.

Method: STAGE defines four interconnected tasks: 1) knowledge graph construction, 2) scene-level event summarization, 3) long-context screenplay question answering, and 4) in-script character role-playing. All tasks are grounded in a shared narrative world representation. The benchmark provides cleaned scripts, curated knowledge graphs, and event/character annotations for 150 films in English and Chinese.

Result: The benchmark enables holistic evaluation of models’ abilities to: build world representations, abstract and verify narrative events, reason over long narratives, and generate character-consistent responses. It provides a comprehensive framework for assessing narrative understanding across multiple dimensions.

Conclusion: STAGE addresses the gap in evaluating coherent story world construction and consistent reasoning across multiple narrative tasks, offering a unified benchmark for comprehensive assessment of narrative understanding in long-form screenplays.

Abstract: Movie screenplays are rich long-form narratives that interleave complex character relationships, temporally ordered events, and dialogue-driven interactions. While prior benchmarks target individual subtasks such as question answering or dialogue generation, they rarely evaluate whether models can construct a coherent story world and use it consistently across multiple forms of reasoning and generation. We introduce STAGE (Screenplay Text, Agents, Graphs and Evaluation), a unified benchmark for narrative understanding over full-length movie screenplays. STAGE defines four tasks: knowledge graph construction, scene-level event summarization, long-context screenplay question answering, and in-script character role-playing, all grounded in a shared narrative world representation. The benchmark provides cleaned scripts, curated knowledge graphs, and event- and character-centric annotations for 150 films across English and Chinese, enabling holistic evaluation of models’ abilities to build world representations, abstract and verify narrative events, reason over long narratives, and generate character-consistent responses.

Method: Introduce OrderProbe benchmark using fixed four-character expressions in Chinese, Japanese, and Korean that have unique canonical order, enabling exact-match scoring. Develop diagnostic framework evaluating semantic fidelity, logical validity, consistency, robustness sensitivity, and information density.

Result: Experiments on twelve widely used LLMs show structural reconstruction remains difficult even for frontier systems: zero-shot recovery frequently falls below 35%. Consistent dissociation observed between semantic recall and structural planning.

Conclusion: Structural robustness is not an automatic byproduct of semantic competence. The OrderProbe benchmark provides a reliable way to evaluate LLMs’ structural reconstruction capabilities beyond semantic understanding.

Abstract: Large language models (LLMs) excel at semantic understanding, yet their ability to reconstruct internal structure from scrambled inputs remains underexplored. Sentence-level restoration is ill-posed for automated evaluation because multiple valid word orders often exist. We introduce OrderProbe, a deterministic benchmark for structural reconstruction using fixed four-character expressions in Chinese, Japanese, and Korean, which have a unique canonical order and thus support exact-match scoring. We further propose a diagnostic framework that evaluates models beyond recovery accuracy, including semantic fidelity, logical validity, consistency, robustness sensitivity, and information density. Experiments on twelve widely used LLMs show that structural reconstruction remains difficult even for frontier systems: zero-shot recovery frequently falls below 35%. We also observe a consistent dissociation between semantic recall and structural planning, suggesting that structural robustness is not an automatic byproduct of semantic competence.

Method: Comprehensive comparison of six neural models and six LLM prompting strategies across three granularity levels (nationality, region, continent). Includes frequency-based stratified analysis and error analysis to understand performance patterns.

Result: LLMs outperform neural models at all granularity levels, with gap narrowing as granularity becomes coarser. Simple ML methods show highest frequency robustness, while pre-trained models and LLs degrade for low-frequency nationalities. LLMs make “near-miss” errors (correct region, wrong nationality), while neural models show cross-regional errors and high-frequency bias.

Conclusion: LLM superiority comes from world knowledge, model selection should consider required granularity, and evaluation should account for error quality beyond accuracy. Practical implications for choosing appropriate models based on application needs.

Abstract: Predicting nationality from personal names has practical value in marketing, demographic research, and genealogical studies. Conventional neural models learn statistical correspondences between names and nationalities from task-specific training data, posing challenges in generalizing to low-frequency nationalities and distinguishing similar nationalities within the same region. Large language models (LLMs) have the potential to address these challenges by leveraging world knowledge acquired during pre-training. In this study, we comprehensively compare neural models and LLMs on nationality prediction, evaluating six neural models and six LLM prompting strategies across three granularity levels (nationality, region, and continent), with frequency-based stratified analysis and error analysis. Results show that LLMs outperform neural models at all granularity levels, with the gap narrowing as granularity becomes coarser. Simple machine learning methods exhibit the highest frequency robustness, while pre-trained models and LLMs show degradation for low-frequency nationalities. Error analysis reveals that LLMs tend to make ``near-miss’’ errors, predicting the correct region even when nationality is incorrect, whereas neural models exhibit more cross-regional errors and bias toward high-frequency classes. These findings indicate that LLM superiority stems from world knowledge, model selection should consider required granularity, and evaluation should account for error quality beyond accuracy.

Method: For each LLM response, compute output-entropy profile from final-layer next-token probabilities (using top-k logprobs), summarize with eleven statistics. Train lightweight classifier to predict instance correctness, average predicted probabilities to get domain-level accuracy estimates.

Result: Evaluated on ten STEM reasoning benchmarks with exhaustive train/test compositions across nine LLMs (3B-20B). Estimates often track held-out benchmark accuracy, several models show near-monotonic ordering of domains.

Conclusion: Output-entropy profiles provide accessible signal for scalable monitoring of LLM performance under domain shift and for targeting data acquisition to improve models.

Abstract: Deploying LLMs raises two coupled challenges: (1) monitoring - estimating where a model underperforms as traffic and domains drift - and (2) improvement - prioritizing data acquisition to close the largest performance gaps. We test whether an inference-time signal can estimate slice-level accuracy under domain shift. For each response, we compute an output-entropy profile from final-layer next-token probabilities (from top-k logprobs) and summarize it with eleven statistics. A lightweight classifier predicts instance correctness, and averaging predicted probabilities yields a domain-level accuracy estimate. We evaluate on ten STEM reasoning benchmarks with exhaustive train/test compositions (k in {1,2,3,4}; all “10 choose k” combinations), across nine LLMs from six families (3B-20B). Estimates often track held-out benchmark accuracy, and several models show near-monotonic ordering of domains. Output-entropy profiles are thus an accessible signal for scalable monitoring and for targeting data acquisition.

Method: Two-stage fine-tuning: 1) Supervised fine-tuning using mixture of high-quality synthetic parallel data (from state-of-the-art models) and human-translated data, 2) Reinforcement learning phase optimizing translation quality using ensemble reward models (MetricX-QE and AutoMQM).

Result: Demonstrated effectiveness with human evaluation on WMT25 test set (10 language pairs) and automatic evaluation on WMT24++ benchmark (55 language pairs). Showed consistent substantial gains over baseline Gemma 3 models across all sizes, with smaller TranslateGemma models achieving performance comparable to larger baselines. Models retain strong multimodal capabilities with enhanced performance on Vistra image translation benchmark.

Conclusion: TranslateGemma provides efficient, high-performance machine translation models that maintain multimodal capabilities, offering the research community powerful and adaptable tools for translation tasks.

Abstract: We present TranslateGemma, a suite of open machine translation models based on the Gemma 3 foundation models. To enhance the inherent multilingual capabilities of Gemma 3 for the translation task, we employ a two-stage fine-tuning process. First, supervised fine-tuning is performed using a rich mixture of high-quality large-scale synthetic parallel data generated via state-of-the-art models and human-translated parallel data. This is followed by a reinforcement learning phase, where we optimize translation quality using an ensemble of reward models, including MetricX-QE and AutoMQM, targeting translation quality. We demonstrate the effectiveness of TranslateGemma with human evaluation on the WMT25 test set across 10 language pairs and with automatic evaluation on the WMT24++ benchmark across 55 language pairs. Automatic metrics show consistent and substantial gains over the baseline Gemma 3 models across all sizes. Notably, smaller TranslateGemma models often achieve performance comparable to larger baseline models, offering improved efficiency. We also show that TranslateGemma models retain strong multimodal capabilities, with enhanced performance on the Vistra image translation benchmark. The release of the open TranslateGemma models aims to provide the research community with powerful and adaptable tools for machine translation.

Method: Provides taxonomy of disagreement sources (data, task, annotator factors), synthesizes modeling approaches using prediction targets and pooling structure framework, and reviews evaluation metrics.

Result: Identifies shift from consensus learning to explicit disagreement modeling and structured annotator relationships; notes most fairness evaluations remain descriptive rather than normative.

Conclusion: Identifies open challenges: integrating multiple variation sources, developing disagreement-aware interpretability frameworks, and addressing practical tradeoffs of perspectivist modeling.

Abstract: Annotator disagreement is widespread in NLP, particularly for subjective and ambiguous tasks such as toxicity detection and stance analysis. While early approaches treated disagreement as noise to be removed, recent work increasingly models it as a meaningful signal reflecting variation in interpretation and perspective. This survey provides a unified view of disagreement-aware NLP methods. We first present a domain-agnostic taxonomy of the sources of disagreement spanning data, task, and annotator factors. We then synthesize modeling approaches using a common framework defined by prediction targets and pooling structure, highlighting a shift from consensus learning toward explicitly modeling disagreement, and toward capturing structured relationships among annotators. We review evaluation metrics for both predictive performance and annotator behavior, and noting that most fairness evaluations remain descriptive rather than normative. We conclude by identifying open challenges and future directions, including integrating multiple sources of variation, developing disagreement-aware interpretability frameworks, and grappling with the practical tradeoffs of perspectivist modeling.

Method: Created MCGA corpus with 119 hours of audio (22,000 samples) covering six speech tasks: ASR, S2TT, SEC, SQA, SU, and SR. Evaluated ten MLLMs on this benchmark and introduced domain-specific metrics for SEC and speech-text consistency.

Result: Current MLLMs face substantial challenges on MCGA test set, demonstrating significant limitations in handling Chinese classical studies audio tasks despite their multimodal capabilities.

Conclusion: MCGA fills a critical gap in CCS audio resources and reveals MLLMs’ current weaknesses in this domain, providing a public benchmark to drive development of more robust multimodal models.

Abstract: With the rapid advancement of Multimodal Large Language Models (MLLMs), their potential has gained significant attention in Chinese Classical Studies (CCS). While existing research primarily focuses on text and visual modalities, the audio corpus within this domain remains largely underexplored. To bridge this gap, we introduce the Multi-task Classical Chinese Literary Genre Audio Corpus (MCGA), a 119-hour corpus comprising 22,000 audio samples. It encompasses a diverse range of literary genres across six tasks: Automatic Speech Recognition (ASR), Speech-to-Text Translation (S2TT), Speech Emotion Captioning (SEC), Spoken Question Answering (SQA), Speech Understanding (SU), and Speech Reasoning (SR). Through the evaluation of ten MLLMs, our experimental results demonstrate that current MLLMs still face substantial challenges on the MCGA test set. Furthermore, we introduce a domain-specific metric for SEC and a metric to measure the consistency between speech and text capabilities. We release MCGA to the public to facilitate the development of more robust MLLMs. MCGA Corpus: https://github.com/yxduir/MCGA

Method: The authors introduce entropy-based metrics to identify domain experts (experts strongly favored for particular domains) and causal-effect metrics to identify driver experts (experts whose activation causally contributes to model output). They analyze expert activation across three public domains and study token-level associations with expert activation.

Result: Three key findings: (1) Some activated experts show clear domain preferences while others exert strong causal influence on model performance; (2) Tokens occurring earlier in sentences are more likely to trigger driver experts; (3) Adjusting weights of domain and driver experts leads to significant performance gains across all three models and domains.

Conclusion: The findings provide insights into the internal mechanisms of MoE models, revealing functional specialization among experts similar to brain organization, and demonstrate that understanding domain vs. driver expert distinctions can lead to performance improvements, enhancing MoE model interpretability.

Abstract: Most interpretability work focuses on layer- or neuron-level mechanisms in Transformers, leaving expert-level behavior in MoE LLMs underexplored. Motivated by functional specialization in the human brain, we analyze expert activation by distinguishing domain and driver experts. In this work, we study expert activation in MoE models across three public domains and address two key questions: (1) which experts are activated, and whether certain expert types exhibit consistent activation patterns; and (2) how tokens are associated with and trigger the activation of specific experts. To answer these questions, we introduce entropy-based and causal-effect metrics to assess whether an expert is strongly favored for a particular domain, and how strongly expert activation contributes causally to the model’s output, thus identify domain and driver experts, respectively. Furthermore, we explore how individual tokens are associated with the activation of specific experts. Our analysis reveals that (1) Among the activated experts, some show clear domain preferences, while others exert strong causal influence on model performance, underscoring their decisive roles. (2) tokens occurring earlier in a sentence are more likely to trigger the driver experts, and (3) adjusting the weights of domain and driver experts leads to significant performance gains across all three models and domains. These findings shed light on the internal mechanisms of MoE models and enhance their interpretability.

Method: Three-step approach: (1) construct CoT dataset with step-level scores, (2) train VAE and quality estimation model to learn low-dimensional manifold of high-quality CoT trajectories, (3) steer hidden states of target LLMs toward higher-quality regions in latent space using gradient-based steering along the learned manifold.

Result: On GSM8k dataset using Qwen3 series: improved accuracy by 0.9 points and enhanced reasoning quality by 4.5 points on average compared to original LLMs.

Conclusion: GeoSteer provides an effective and controllable mechanism for improving intermediate reasoning quality in LLMs through geometric steering in learned manifold space.

Abstract: Recent advances in Large Language Models (LLMs) have demonstrated remarkable progress in their reasoning capabilities, such as Chain-of-Thought (CoT). Most approaches rely on CoT rationales. Previous studies have shown that LLMs often generate logically inconsistent reasoning steps even when their final answers are correct. These inconsistencies reduce the reliability of the reasoning process. We propose GeoSteer, a manifold-based framework that improves the quality of intermediate reasoning. The method consists of: (1) constructing a CoT dataset with step-level scores, (2) training a Variational Autoencoder (VAE) model and a quality estimation model to learn a low-dimensional manifold of high-quality CoT trajectories, and (3) steering hidden states of target LLMs toward higher-quality regions in the latent space. This last step enables steering of the hidden states by following gradients along the learned manifold. It facilitates geometrically coherent steering. Evaluation experiments were conducted on the GSM8k dataset using the Qwen3 series. We evaluated performance using two metrics: answer accuracy and overall reasoning quality. GeoSteer improved the accuracy by 0.9 points and enhanced the reasoning quality by 4.5 points on average, compared with those of original LLMs. These results indicate that GeoSteer improves an effective and controllable mechanism for improving the quality of intermediate reasoning in LLMs.

Method: LIBERTy framework constructs datasets with structural counterfactual pairs grounded in explicitly defined Structured Causal Models (SCMs). Interventions on concepts propagate through SCMs until LLMs generate counterfactuals. Includes three datasets (disease detection, CV screening, workplace violence) and introduces order-faithfulness metric.

Result: Evaluation of various methods across five models shows substantial room for improving concept-based explanations. Systematic analysis reveals proprietary LLMs show reduced sensitivity to demographic concepts, likely due to post-training mitigation. LIBERTy provides a comprehensive benchmark for developing faithful explainability methods.

Conclusion: LIBERTy addresses critical limitations in existing evaluation methods for concept-based explanations by providing a systematic, scalable benchmark grounded in causal models. It enables better development and assessment of faithful explainability methods while revealing important insights about model behavior and mitigation effects.

Abstract: Concept-based explanations quantify how high-level concepts (e.g., gender or experience) influence model behavior, which is crucial for decision-makers in high-stakes domains. Recent work evaluates the faithfulness of such explanations by comparing them to reference causal effects estimated from counterfactuals. In practice, existing benchmarks rely on costly human-written counterfactuals that serve as an imperfect proxy. To address this, we introduce a framework for constructing datasets containing structural counterfactual pairs: LIBERTy (LLM-based Interventional Benchmark for Explainability with Reference Targets). LIBERTy is grounded in explicitly defined Structured Causal Models (SCMs) of the text generation, interventions on a concept propagate through the SCM until an LLM generates the counterfactual. We introduce three datasets (disease detection, CV screening, and workplace violence prediction) together with a new evaluation metric, order-faithfulness. Using them, we evaluate a wide range of methods across five models and identify substantial headroom for improving concept-based explanations. LIBERTy also enables systematic analysis of model sensitivity to interventions: we find that proprietary LLMs show markedly reduced sensitivity to demographic concepts, likely due to post-training mitigation. Overall, LIBERTy provides a much-needed benchmark for developing faithful explainability methods.

Method: Proposes a novel method that automatically constructs unweighted FSTs by leveraging the hidden state geometry learned by recurrent neural networks. The approach extracts transducer structure from the learned representations of RNNs.

Result: The constructed FSTs achieve high accuracy and robustness on real-world datasets for morphological inflection, grapheme-to-phoneme prediction, and historical normalization. They substantially outperform classical transducer learning algorithms by up to 87% accuracy on held-out test sets.

Conclusion: The proposed method successfully automates FST construction using neural network representations, creating highly accurate transducers that significantly outperform traditional learning approaches for string-to-string rewriting tasks.

Abstract: Finite-State Transducers (FSTs) are effective models for string-to-string rewriting tasks, often providing the efficiency necessary for high-performance applications, but constructing transducers by hand is difficult. In this work, we propose a novel method for automatically constructing unweighted FSTs following the hidden state geometry learned by a recurrent neural network. We evaluate our methods on real-world datasets for morphological inflection, grapheme-to-phoneme prediction, and historical normalization, showing that the constructed FSTs are highly accurate and robust for many datasets, substantially outperforming classical transducer learning algorithms by up to 87% accuracy on held-out test sets.

Method: Use HierarchicalViT (HVT), a Swin-style hierarchical transformer, and evaluate on three agricultural disease datasets. Apply SimCLR self-supervised pre-training on 3,000 unlabeled agricultural images. Compare against Swin-Base and ViT-Base architectures with and without domain-specific pre-training.

Result: SimCLR pre-training provides +4.57% accuracy improvement, exceeding +3.70% gain from hierarchical architecture. SSL benefits are architecture-agnostic: +4.08% for Swin-Base, +4.20% for ViT-Base. HVT-Base (78M) achieves 88.91% vs. Swin-Base (88M) at 87.23% (+1.68%). Calibration analysis shows HVT achieves 3.56% ECE (1.52% after temperature scaling).

Conclusion: Domain-specific self-supervised pre-training provides greater performance gains than architectural improvements for agricultural disease classification. Practitioners should prioritize collecting domain-specific unlabeled data over complex architectural choices, as SSL benefits are consistent across different transformer architectures.

Abstract: We investigate the impact of domain-specific self-supervised pre-training on agricultural disease classification using hierarchical vision transformers. Our key finding is that SimCLR pre-training on just 3,000 unlabeled agricultural images provides a +4.57% accuracy improvement–exceeding the +3.70% gain from hierarchical architecture design. Critically, we show this SSL benefit is architecture-agnostic: applying the same pre-training to Swin-Base yields +4.08%, to ViT-Base +4.20%, confirming practitioners should prioritize domain data collection over architectural choices. Using HierarchicalViT (HVT), a Swin-style hierarchical transformer, we evaluate on three datasets: Cotton Leaf Disease (7 classes, 90.24%), PlantVillage (38 classes, 96.3%), and PlantDoc (27 classes, 87.1%). At matched parameter counts, HVT-Base (78M) achieves 88.91% vs. Swin-Base (88M) at 87.23%, a +1.68% improvement. For deployment reliability, we report calibration analysis showing HVT achieves 3.56% ECE (1.52% after temperature scaling). Code: https://github.com/w2sg-arnav/HierarchicalViT

Method: Proposes a 3D TransUNet image synthesis framework that predicts fractional anisotropy (FA) and mean diffusivity (MD) maps directly from T1-weighted MRI. The model generates synthetic diffusion MRI metrics that can be integrated into multi-modal diagnostic models for Alzheimer’s disease classification.

Result: The model achieves high-fidelity synthesis with SSIM >0.93 and Pearson correlation >0.94 with ground-truth dMRI. When used in diagnostic models, synthetic features boost AD classification accuracy by 5% (78.75% to 83.75%) and improve mild cognitive impairment detection by 12.5%.

Conclusion: High-quality diffusion microstructural information can be inferred from routinely acquired T1w MRI, effectively transferring multi-modality benefits to settings without diffusion data. This approach reduces scan time while preserving complementary information, potentially improving AD diagnosis accessibility, efficiency, and accuracy in clinical practice.

Abstract: Alzheimer’s disease (AD) is a progressive neurodegenerative disorder in which pathological changes begin many years before the onset of clinical symptoms, making early detection essential for timely intervention. T1-weighted (T1w) Magnetic Resonance Imaging (MRI) is routinely used in clinical practice to identify macroscopic brain alterations, but these changes typically emerge relatively late in the disease course. Diffusion MRI (dMRI), in contrast, is sensitive to earlier microstructural abnormalities by probing water diffusion in brain tissue. dMRI metrics, including fractional anisotropy (FA) and mean diffusivity (MD), provide complementary information about white matter integrity and neurodegeneration. However, dMRI acquisitions are time-consuming and susceptible to motion artifacts, limiting their routine use in clinical populations. To bridge this gap, I propose a 3D TransUNet image synthesis framework that predicts FA and MD maps directly from T1w MRI. My model generates high-fidelity maps, achieving a structural similarity index (SSIM) exceeding 0.93 and a strong Pearson correlation (>0.94) with ground-truth dMRI. When integrated into a multi-modal diagnostic model, these synthetic features boost AD classification accuracy by 5% (78.75%->83.75%) and, most importantly, improve mild cognitive impairment (MCI) detection by 12.5%. This study demonstrates that high-quality diffusion microstructural information can be inferred from routinely acquired T1w MRI, effectively transferring the benefits of multi-modality imaging to settings where diffusion data are unavailable. By reducing scan time while preserving complementary structural and microstructural information, the proposed approach has the potential to improve the accessibility, efficiency, and accuracy of AD diagnosis in clinical practice.

Method: Uses hierarchically constrained neural Gaussian representation to preserve structural relationships while generating Gaussian primitives; employs progressive pose optimization to mitigate depth sensor noise; utilizes dynamic neural representation graph that adjusts Gaussian node distribution based on local geometric complexity.

Result: Outperforms existing 3DGS-based SLAM methods in reconstruction accuracy and rendering quality, demonstrating advantages for large-scale AR and robotics applications.

Conclusion: PointSLAM++ successfully addresses structural consistency and depth noise challenges through its novel neural Gaussian representation approach, enabling high-precision 3D mapping and photorealistic scene rendering suitable for real-time robotics and AR applications.

Abstract: Real-time 3D reconstruction is crucial for robotics and augmented reality, yet current simultaneous localization and mapping(SLAM) approaches often struggle to maintain structural consistency and robust pose estimation in the presence of depth noise. This work introduces PointSLAM++, a novel RGB-D SLAM system that leverages a hierarchically constrained neural Gaussian representation to preserve structural relationships while generating Gaussian primitives for scene mapping. It also employs progressive pose optimization to mitigate depth sensor noise, significantly enhancing localization accuracy. Furthermore, it utilizes a dynamic neural representation graph that adjusts the distribution of Gaussian nodes based on local geometric complexity, enabling the map to adapt to intricate scene details in real time. This combination yields high-precision 3D mapping and photorealistic scene rendering. Experimental results show PointSLAM++ outperforms existing 3DGS-based SLAM methods in reconstruction accuracy and rendering quality, demonstrating its advantages for large-scale AR and robotics.

Method: One-class autoencoders trained on interpretable handcrafted features (Fourier-domain energy, Shannon entropy, global contrast, GLCM-based homogeneity, fractal complexity) using authenticated sketches from multiple museum collections, evaluated under biometric-style protocol with genuine and impostor trials.

Result: Pooled system achieves 83.3% True Acceptance Rate with 9.5% False Acceptance Rate; performance varies by artist with near-zero false acceptance for some and elevated confusability for others; false accepts show structured error pathways consistent with stylistic proximity.

Conclusion: The computational framework provides reproducible, quantitative evidence suitable for data-scarce settings, designed to complement rather than replace traditional connoisseurship in historical sketch attribution.

Abstract: Authentication and attribution of works on paper remain persistent challenges in cultural heritage, particularly when the available reference corpus is small and stylistic cues are primarily expressed through line and limited tonal variation. We present a verification-based computational framework for historical drawing authentication using one-class autoencoders trained on a compact set of interpretable handcrafted features. Ten artist-specific verifiers are trained using authenticated sketches from the Metropolitan Museum of Art open-access collection, the Ashmolean Collections Catalogue, the Morgan Library and Museum, the Royal Collection Trust (UK), the Victoria and Albert Museum Collections, and an online catalogue of the Casa Buonarroti collection and evaluated under a biometric-style protocol with genuine and impostor trials. Feature vectors comprise Fourier-domain energy, Shannon entropy, global contrast, GLCM-based homogeneity, and a box-counting estimate of fractal complexity. Across 900 verification decisions (90 genuine and 810 impostor trials), the pooled system achieves a True Acceptance Rate of 83.3% with a False Acceptance Rate of 9.5% at the chosen operating point. Performance varies substantially by artist, with near-zero false acceptance for some verifiers and elevated confusability for others. A pairwise attribution of false accepts indicates structured error pathways consistent with stylistic proximity and shared drawing conventions, whilst also motivating tighter control of digitisation artefacts and threshold calibration. The proposed methodology is designed to complement, rather than replace, connoisseurship by providing reproducible, quantitative evidence suitable for data-scarce settings common in historical sketch attribution.

Method: Propose SLT (Single-Layer Transformer) that distills FreeFlow’s 28-layer Transformer into a single shared DiT block. During training, it matches teacher’s intermediate features at depth patches, fuses patch-level representations, and aligns teacher’s final velocity prediction. The compressed model enables efficient noise space screening to select higher-quality initial points for the teacher model.

Result: Successfully compressed 28 independent Transformer Blocks into a single Transformer Block, reducing parameters from 675M to 4.3M. Within comparable time to two teacher samplings, SLT performs over 100 noise screenings and produces high-quality samples through teacher model using selected points, substantially improving stability and average generation quality.

Conclusion: SLT effectively addresses quality fluctuations in one-step generation by enabling efficient noise screening through model compression, improving both stability and generation quality while maintaining computational efficiency comparable to minimal teacher model sampling.

Abstract: Currently, Flow matching methods aim to compress the iterative generation process of diffusion models into a few or even a single step, with MeanFlow and FreeFlow being representative achievements of one-step generation based on Ordinary Differential Equations (ODEs). We observe that the 28-layer Transformer architecture of FreeFlow can be characterized as an Euler discretization scheme for an ODE along the depth axis, where the layer index serves as the discrete time step. Therefore, we distill the number of layers of the FreeFlow model, following the same derivation logic as FreeFlow, and propose SLT (Single-Layer Transformer), which uses a single shared DiT block to approximate the depth-wise feature evolution of the 28-layer teacher. During training, it matches the teacher’s intermediate features at several depth patches, fuses those patch-level representations, and simultaneously aligns the teacher’s final velocity prediction. Through distillation training, we compress the 28 independent Transformer Blocks of the teacher model DiT-XL/2 into a single Transformer Block, reducing the parameter count from 675M to 4.3M. Furthermore, leveraging its minimal parameters and rapid sampling speed, SLT can screen more candidate points in the noise space within the same timeframe, thereby selecting higher-quality initial points for the teacher model FreeFlow and ultimately enhancing the quality of generated images. Experimental results demonstrate that within a time budget comparable to two random samplings of the teacher model, our method performs over 100 noise screenings and produces a high-quality sample through the teacher model using the selected points. Quality fluctuations caused by low-quality initial noise under a limited number of FreeFlow sampling calls are effectively avoided, substantially improving the stability and average generation quality of one-step generation.

Method: CCPO introduces Coordinate-Aware Spatial Compression (CASC) that aggregates coordinates from multiple rollouts to identify target-relevant regions and progressively narrows historical attention around key visual areas. It also uses a Distance-Based Advantage that provides fine-grained learning signals based on distance rather than binary correctness.

Result: CCPO achieves state-of-the-art performance across four benchmarks with up to 55% token compression and 3.8× training speedup, demonstrating both improved grounding accuracy and compression quality.

Conclusion: CCPO effectively addresses context inflation in multi-turn GUI agents by coupling visual compression with policy optimization, enabling efficient long-term context management while preserving spatial structure and improving learning efficiency.

Abstract: Multi-turn GUI agents enable complex task completion through sequential decision-making, but suffer from severe context inflation as interaction history accumulates. Existing strategies either sacrifice long-term context via truncation or compromise spatial structure through token pruning. In this paper, we propose Coordinate Compression Policy Optimization (CCPO), an efficient policy optimization framework that couples visual compression with policy optimization for multi-turn GUI agents. CCPO introduces Coordinate-Aware Spatial Compression (CASC), which aggregates coordinates from multiple rollouts to capture target-relevant regions and progressively narrow historical attention around key visual areas. From interactions across rollouts, CASC adaptively constructs attention boundaries that concentrate computation on the most informative regions of the scene. We further design a Distance-Based Advantage that provides fine-grained learning signals based on distance rather than binary correctness, improving both grounding accuracy and compression quality. Extensive experiments demonstrate that CCPO achieves SOTA performance across four benchmarks with up to 55% token compression and 3.8$\times$ training speedup.

Method: KG-ViP proposes a unified framework with a novel retrieval-and-fusion pipeline that uses the query as a semantic bridge to progressively integrate both scene graphs and commonsense graphs, synthesizing a unified structured context for reliable multi-modal reasoning.

Result: Extensive experiments on FVQA 2.0+ and MVQA benchmarks demonstrate that KG-ViP significantly outperforms existing VQA methods.

Conclusion: The fusion of scene graphs and commonsense graphs through a unified framework effectively addresses the dual limitations of MLLMs in VQA, enabling more reliable multi-modal reasoning by leveraging complementary structured knowledge sources.

Abstract: Multi-modal Large Language Models (MLLMs) for Visual Question Answering (VQA) often suffer from dual limitations: knowledge hallucination and insufficient fine-grained visual perception. Crucially, we identify that commonsense graphs and scene graphs provide precisely complementary solutions to these respective deficiencies by providing rich external knowledge and capturing fine-grained visual details. However, prior works typically treat them in isolation, overlooking their synergistic potential. To bridge this gap, we propose KG-ViP, a unified framework that empowers MLLMs by fusing scene graphs and commonsense graphs. The core of the KG-ViP framework is a novel retrieval-and-fusion pipeline that utilizes the query as a semantic bridge to progressively integrate both graphs, synthesizing a unified structured context that facilitates reliable multi-modal reasoning. Extensive experiments on FVQA 2.0+ and MVQA benchmarks demonstrate that KG-ViP significantly outperforms existing VQA methods.

Method: Proposes cross-modal knowledge transfer from data-rich RGB modality to wireless sensors. Uses enhanced Temporal Correlation Adaptive Graph Convolution (TC-AGC) with frame interactive enhancement to handle noise and missing frames, plus dual temporal convolution for multiscale temporal modeling.

Result: SkeFi achieves state-of-the-art performance on both mmWave and LiDAR sensors, demonstrating accurate pose extraction and action recognition from noisy wireless sensor data.

Conclusion: SkeFi successfully enables privacy-preserving skeleton-based action recognition in dark environments by transferring knowledge from RGB to wireless sensors and addressing noise challenges through novel graph convolution and temporal modeling techniques.

Abstract: Skeleton-based action recognition leverages human pose keypoints to categorize human actions, which shows superior generalization and interoperability compared to regular end-to-end action recognition. Existing solutions use RGB cameras to annotate skeletal keypoints, but their performance declines in dark environments and raises privacy concerns, limiting their use in smart homes and hospitals. This paper explores non-invasive wireless sensors, i.e., LiDAR and mmWave, to mitigate these challenges as a feasible alternative. Two problems are addressed: (1) insufficient data on wireless sensor modality to train an accurate skeleton estimation model, and (2) skeletal keypoints derived from wireless sensors are noisier than RGB, causing great difficulties for subsequent action recognition models. Our work, SkeFi, overcomes these gaps through a novel cross-modal knowledge transfer method acquired from the data-rich RGB modality. We propose the enhanced Temporal Correlation Adaptive Graph Convolution (TC-AGC) with frame interactive enhancement to overcome the noise from missing or inconsecutive frames. Additionally, our research underscores the effectiveness of enhancing multiscale temporal modeling through dual temporal convolution. By integrating TC-AGC with temporal modeling for cross-modal transfer, our framework can extract accurate poses and actions from noisy wireless sensors. Experiments demonstrate that SkeFi realizes state-of-the-art performances on mmWave and LiDAR. The code is available at https://github.com/Huang0035/Skefi.

Method: Proposes ViEBench with 200 multi-scenario high-resolution images with expert-annotated visual evidence. Tasks are categorized by difficulty into perception and reasoning dimensions. Introduces a dual-axis matrix with four diagnostic quadrants for fine-grained metrics, enabling transparent diagnosis of model behavior across varying task complexities.

Result: Experiments reveal interesting observations: (1) VLMs can produce correct final answers despite grounding on irrelevant regions, and (2) they may successfully locate correct evidence but still fail to utilize it to reach accurate conclusions. ViEBench demonstrates capability to serve as a more explainable and practical benchmark.

Conclusion: ViEBench addresses critical limitations in current VLM evaluation by providing process-verifiable assessment of faithful visual reasoning, offering a more comprehensive and explainable benchmark for evaluating agentic VLMs’ reasoning effectiveness.

Abstract: Despite the remarkable progress of Vision-Language Models (VLMs) in adopting “Thinking-with-Images” capabilities, accurately evaluating the authenticity of their reasoning process remains a critical challenge. Existing benchmarks mainly rely on outcome-oriented accuracy, lacking the capability to assess whether models can accurately leverage fine-grained visual cues for multi-step reasoning. To address these limitations, we propose ViEBench, a process-verifiable benchmark designed to evaluate faithful visual reasoning. Comprising 200 multi-scenario high-resolution images with expert-annotated visual evidence, ViEBench uniquely categorizes tasks by difficulty into perception and reasoning dimensions, where reasoning tasks require utilizing localized visual details with prior knowledge. To establish comprehensive evaluation criteria, we introduce a dual-axis matrix that provides fine-grained metrics through four diagnostic quadrants, enabling transparent diagnosis of model behavior across varying task complexities. Our experiments yield several interesting observations: (1) VLMs can sometimes produce correct final answers despite grounding on irrelevant regions, and (2) they may successfully locate the correct evidence but still fail to utilize it to reach accurate conclusions. Our findings demonstrate that ViEBench can serve as a more explainable and practical benchmark for comprehensively evaluating the effectiveness agentic VLMs. The codes will be released at: https://github.com/Xuchen-Li/ViEBench.

Method: HVP-Net extracts and refines features from multiple intermediate layers of a vision encoder, progressively distilling salient visual concepts from raw patch-tokens at different semantic levels to mitigate redundancy while preserving crucial details.

Result: Achieves new state-of-the-art performance on challenging benchmarks including MSRVTT, DiDeMo, and ActivityNet.

Conclusion: The work validates the effectiveness of exploiting hierarchical features for advancing video-text retrieval, providing more robust video representations.

Abstract: Video-text retrieval (VTR) aims to locate relevant videos using natural language queries. Current methods, often based on pre-trained models like CLIP, are hindered by video’s inherent redundancy and their reliance on coarse, final-layer features, limiting matching accuracy. To address this, we introduce the HVP-Net (Hierarchical Visual Perception Network), a framework that mines richer video semantics by extracting and refining features from multiple intermediate layers of a vision encoder. Our approach progressively distills salient visual concepts from raw patch-tokens at different semantic levels, mitigating redundancy while preserving crucial details for alignment. This results in a more robust video representation, leading to new state-of-the-art performance on challenging benchmarks including MSRVTT, DiDeMo, and ActivityNet. Our work validates the effectiveness of exploiting hierarchical features for advancing video-text retrieval. Our codes are available at https://github.com/boyun-zhang/HVP-Net.

Method: Multimodal LLM agent system that automatically recalls short videos with potential new issues, applies two-stage clustering to group them (each cluster = new issue), and generates updated annotation policies from these clusters.

Result: Deployed in real system, the agent improves emerging-issue discovery effectiveness by over 20% F1 score, reduces problematic video view count by approximately 15%, and significantly accelerates policy iteration compared to manual discovery.

Conclusion: LLM-based automatic issue discovery effectively addresses the speed limitations of human-driven methods, enabling faster policy updates and better content governance on rapidly evolving short-video platforms.

Abstract: Trends on short-video platforms evolve at a rapid pace, with new content issues emerging every day that fall outside the coverage of existing annotation policies. However, traditional human-driven discovery of emerging issues is too slow, which leads to delayed updates of annotation policies and poses a major challenge for effective content governance. In this work, we propose an automatic issue discovery method based on multimodal LLM agents. Our approach automatically recalls short videos containing potential new issues and applies a two-stage clustering strategy to group them, with each cluster corresponding to a newly discovered issue. The agent then generates updated annotation policies from these clusters, thereby extending coverage to these emerging issues. Our agent has been deployed in the real system. Both offline and online experiments demonstrate that this agent-based method significantly improves the effectiveness of emerging-issue discovery (with an F1 score improvement of over 20%) and enhances the performance of subsequent issue governance (reducing the view count of problematic videos by approximately 15%). More importantly, compared to manual issue discovery, it greatly reduces time costs and substantially accelerates the iteration of annotation policies.

Method: The framework uses diffusion-based generative model for face inpainting guided by high-level attribute recognition and motion-aware expression transfer. It then animates de-identified faces through video-driven animation that takes both the de-identified face and original video as input.

Result: Extensive experiments on VoxCeleb2, CelebV-HQ, and HDTF datasets demonstrate effectiveness in obfuscating identity while maintaining visual realism and temporal consistency across diverse facial dynamics.

Conclusion: Anon-NET provides an effective solution for face video anonymization that preserves important facial attributes while removing identity information, with code to be publicly released.

Abstract: Face video anonymization is aimed at privacy preservation while allowing for the analysis of videos in a number of computer vision downstream tasks such as expression recognition, people tracking, and action recognition. We propose here a novel unified framework referred to as Anon-NET, streamlined to de-identify facial videos, while preserving age, gender, race, pose, and expression of the original video. Specifically, we inpaint faces by a diffusion-based generative model guided by high-level attribute recognition and motion-aware expression transfer. We then animate deidentified faces by video-driven animation, which accepts the de-identified face and the original video as input. Extensive experiments on the datasets VoxCeleb2, CelebV-HQ, and HDTF, which include diverse facial dynamics, demonstrate the effectiveness of AnonNET in obfuscating identity while retaining visual realism and temporal consistency. The code of AnonNet will be publicly released.

Method: The authors introduce a Diagnostic Micro-Benchmark that decouples Task Success Rate (TSR) from Correction Success Rate (CSR). They explicitly quantify diminishing returns of correction attempts and develop a Failure Taxonomy to identify reasoning bottlenecks, particularly focusing on Semantic Drift (loss of contextual state).

Result: TSR is 62% while CSR ranges only 25-33%, showing initial competence doesn’t predict repair ability. Correction effectiveness saturates after three retries. Semantic Drift accounts for about 28% of failures, identified as a major reasoning bottleneck.

Conclusion: The benchmark provides a reproducible framework for developing stateful, trustworthy multimodal agents by isolating specific reasoning bottlenecks like Semantic Drift, enabling targeted improvements in self-correction capabilities.

Abstract: Recent progress in multimodal foundation models has enabled Vision-Language Agents (VLAs) to decompose complex visual tasks into executable tool-based plans. While recent benchmarks have begun to evaluate iterative self-correction, its quantitative limits and dominant reasoning bottlenecks remain poorly characterized. This work introduces a Diagnostic Micro-Benchmark. Our analysis decouples Task Success Rate (TSR = 62 percent) from Correction Success Rate (CSR = 25 to 33 percent), revealing that initial competence does not predict repair ability. We explicitly quantify the diminishing returns of correction, which saturates after three retries. Our Failure Taxonomy reveals a frequent factor is Semantic Drift (about 28 percent of failures), a loss of contextual state. By isolating this reasoning bottleneck, this benchmark defines a reproducible framework toward stateful, trustworthy multimodal agents.

Method: Model-Driven Data Correction (MDDC) framework with automated error analysis categorizing failures into four types (false negatives, false positives, class confusion, localization errors), followed by structured train-fix-retrain pipeline with version-controlled data management.

Result: Consistent improvements of 5-25% in mAP at 0.5 across multiple weed detection datasets using fixed lightweight detector (YOLOv8n).

Conclusion: Systematic data quality optimization can effectively alleviate performance bottlenecks under fixed model capacity constraints, offering a data-centric alternative to model scaling for edge device applications.

Abstract: Agricultural weed detection on edge devices is subject to strict constraints on model capacity, computational resources, and real-time inference latency, which prevent performance improvements through model scaling or ensembling. This paper proposes Model-Driven Data Correction (MDDC), a data-centric framework that enhances detection performance by iteratively diagnosing and correcting data quality deficiencies. An automated error analysis procedure categorizes detection failures into four types: false negatives, false positives, class confusion, and localization errors. These error patterns are systematically addressed through a structured train-fix-retrain pipeline with version-controlled data management. Experimental results on multiple weed detection datasets demonstrate consistent improvements of 5-25 percent in mAP at 0.5 using a fixed lightweight detector (YOLOv8n), indicating that systematic data quality optimization can effectively alleviate performance bottlenecks under fixed model capacity constraints.

Method: Two-stage approach: 1) Uses prior information from early denoising steps with distributed mixing to create a linear approximation model for predicting mask patterns for specific denoising intervals. 2) Implements online block masking strategy that dynamically applies predicted masks while maintaining historical sparsity information, eliminating repetitive sampling operations.

Result: Extensive evaluations show consistent acceleration and quality improvements across multiple benchmarks and model architectures, validating effectiveness for efficient, high-quality video generation while overcoming computational limitations of traditional sparse attention approaches.

Conclusion: MOD-DiT provides an effective solution for efficient video generation by addressing the quadratic complexity problem of self-attention in Diffusion Transformers through a novel sampling-free dynamic attention framework that maintains generation quality while significantly improving computational efficiency.

Abstract: While Diffusion Transformers (DiTs) have achieved notable progress in video generation, this long-sequence generation task remains constrained by the quadratic complexity inherent to self-attention mechanisms, creating significant barriers to practical deployment. Although sparse attention methods attempt to address this challenge, existing approaches either rely on oversimplified static patterns or require computationally expensive sampling operations to achieve dynamic sparsity, resulting in inaccurate pattern predictions and degraded generation quality. To overcome these limitations, we propose a \underline{\textbf{M}}ixtrue-\underline{\textbf{O}}f-\underline{\textbf{D}}istribution \textbf{DiT} (\textbf{MOD-DiT}), a novel sampling-free dynamic attention framework that accurately models evolving attention patterns through a two-stage process. First, MOD-DiT leverages prior information from early denoising steps and adopts a {distributed mixing approach} to model an efficient linear approximation model, which is then used to predict mask patterns for a specific denoising interval. Second, an online block masking strategy dynamically applies these predicted masks while maintaining historical sparsity information, eliminating the need for repetitive sampling operations. Extensive evaluations demonstrate consistent acceleration and quality improvements across multiple benchmarks and model architectures, validating MOD-DiT’s effectiveness for efficient, high-quality video generation while overcoming the computational limitations of traditional sparse attention approaches.

Method: Created a 2D X-ray projection simulator (PSSF) from parametric anatomical models of distal femur and proximal tibia. Generated virtual cohort of 180 subjects (260 knees) with three imaging protocols. Used IBSI-standardized radiomic features and trained ML models (logistic regression, random forest, gradient boosting) for binary and three-class KL-grade prediction.

Result: Successfully generated synthetic X-ray scans without patient involvement. Evaluated ML models across IBSI protocol, cross-protocol, and multi-protocol scenarios. Assessed feature stability using intraclass correlation coefficients across acquisition changes.

Conclusion: The PSSF framework provides a privacy-preserving solution for generating synthetic knee X-ray data, enabling AI/radiomics research in osteoarthritis assessment without real patient data constraints.

Abstract: Knee osteoarthritis (OA) is a major cause of disability worldwide and is still largely assessed using subjective radiographic grading, most commonly the Kellgren-Lawrence (KL) scale. Artificial intelligence (AI) and radiomics offer quantitative tools for OA assessment but depend on large, well-annotated image datasets, mainly X-ray scans, that are often difficult to obtain because of privacy, governance and resourcing constraints. In this research, we introduce a physics-based synthetic simulation framework (PSSF) to fully generate controllable X-ray scans without patients’ involvement and violating their privacy and institutional constraints. This PSSF is a 2D X-ray projection simulator of anteroposterior knee radiographs from a parametric anatomical model of the distal femur and proximal tibia. Using PSSF, we create a virtual cohort of 180 subjects (260 knees), each is imaged under three protocols (reference, low-dose, and geometry-shift). Medial joint regions are automatically localized, preprocessed, and processed with the Image Biomarker Standardisation Initiative (IBSI). Practically, three machine learning (ML) models are utilized, logistic regression, random forest, and gradient boosting, to train binary (KL-like “0” vs. “2”) and three-class (0-2) prediction radiographic images. Robustness is assessed within IBSI protocol, cross-protocol, and multi-protocol scenarios. Finally, features stability is then evaluated using intraclass correlation coefficients across acquisition changes.

Method: Vision-based confidence estimation framework that validates VLM predictions through independent geometric verification using object detection. Fuses four signals via gradient boosting: geometric alignment between VLM claims and coordinates, spatial ambiguity from overlap, detection quality, and VLM internal uncertainty.

Result: Achieved 0.674 AUROC on BLIP-2 (34.0% improvement over text-based baselines) and 0.583 AUROC on CLIP (16.1% improvement). At 60% target accuracy, achieved 61.9% coverage vs 27.6% baseline (2.2x improvement) on BLIP-2. Vision-based signals contribute 87.4% of model importance vs 12.7% from VLM confidence.

Conclusion: External geometric verification outperforms self-assessment for VLM spatial predictions. Framework enables reliable scene graph construction where confidence-based pruning improves precision from 52.1% to 78.3% while retaining 68.2% of edges, supporting safe deployment in robotics.

Abstract: Vision-Language Models (VLMs) demonstrate impressive capabilities across multimodal tasks, yet exhibit systematic spatial reasoning failures, achieving only 49% (CLIP) to 54% (BLIP-2) accuracy on basic directional relationships. For safe deployment in robotics and autonomous systems, we need to predict when to trust VLM spatial predictions rather than accepting all outputs. We propose a vision-based confidence estimation framework that validates VLM predictions through independent geometric verification using object detection. Unlike text-based approaches relying on self-assessment, our method fuses four signals via gradient boosting: geometric alignment between VLM claims and coordinates, spatial ambiguity from overlap, detection quality, and VLM internal uncertainty. We achieve 0.674 AUROC on BLIP-2 (34.0% improvement over text-based baselines) and 0.583 AUROC on CLIP (16.1% improvement), generalizing across generative and classification architectures. Our framework enables selective prediction: at 60% target accuracy, we achieve 61.9% coverage versus 27.6% baseline (2.2x improvement) on BLIP-2. Feature analysis reveals vision-based signals contribute 87.4% of model importance versus 12.7% from VLM confidence, validating that external geometric verification outperforms self-assessment. We demonstrate reliable scene graph construction where confidence-based pruning improves precision from 52.1% to 78.3% while retaining 68.2% of edges.

Method: Proposes IMSAHLO framework with: 1) Multi-Scale Dense Blocks (MSDBs) to capture features at various receptive fields for handling cell density variations; 2) Hierarchical Attention mechanism to focus on salient morphological features and preserve ROI boundary details; 3) Hybrid loss function combining Tversky+Focal loss for class imbalance, topology-aware Centerline Dice loss, and Contour-Weighted Boundary loss for topological continuity and cell separation.

Result: Outperforms state-of-the-art on Fluorescent Neuronal Cells dataset: 81.4% precision, 82.7% macro F1, 83.3% micro F1, 99.5% balanced accuracy on difficult dense/sparse cases. Ablation studies validate synergistic benefits of multi-scale attention and hybrid loss terms.

Conclusion: Establishes foundation for generalizable segmentation models applicable to wide range of biomedical imaging modalities, pushing AI-assisted analysis toward high-throughput neurobiological pipelines.

Abstract: Accurate segmentation of neuronal cells in fluorescence microscopy is a fundamental task for quantitative analysis in computational neuroscience. However, it is significantly impeded by challenges such as the coexistence of densely packed and sparsely distributed cells, complex overlapping morphologies, and severe class imbalance. Conventional deep learning models often fail to preserve fine topological details or accurately delineate boundaries under these conditions. To address these limitations, we propose a novel deep learning framework, IMSAHLO (Integrating Multi-Scale Attention and Hybrid Loss Optimization), for robust and adaptive neuronal segmentation. The core of our model features Multi-Scale Dense Blocks (MSDBs) to capture features at various receptive fields, effectively handling variations in cell density, and a Hierarchical Attention (HA) mechanism that adaptively focuses on salient morphological features to preserve Region of Interest (ROI) boundary details. Furthermore, we introduce a novel hybrid loss function synergistically combining Tversky and Focal loss to combat class imbalance, alongside a topology-aware Centerline Dice (clDice) loss and a Contour-Weighted Boundary loss to ensure topological continuity and precise separation of adjacent cells. Large-scale experiments on the public Fluorescent Neuronal Cells (FNC) dataset demonstrate that our framework outperforms state-of-the-art architectures, achieving precision of 81.4%, macro F1 score of 82.7%, micro F1 score of 83.3%, and balanced accuracy of 99.5% on difficult dense and sparse cases. Ablation studies validate the synergistic benefits of multi-scale attention and hybrid loss terms. This work establishes a foundation for generalizable segmentation models applicable to a wide range of biomedical imaging modalities, pushing AI-assisted analysis toward high-throughput neurobiological pipelines.

Method: SI³ introduces pre-imputation assessment that evaluates imputation-relevant informativeness of each missing position in a training-free manner. It selectively imputes only when sufficient informative support is available. Under a multi-view generative assumption, SI³ integrates selective imputation into a variational inference framework for uncertainty-aware imputation at the latent distribution level and robust multi-view fusion.

Result: Extensive experiments on multiple benchmark datasets show SI³ consistently outperforms both imputation-based and imputation-free methods, particularly under challenging unbalanced missing scenarios. The method is lightweight, data-driven, model-agnostic, and can be incorporated as a plug-in strategy.

Conclusion: SI³ effectively addresses the trade-off between imputation utility and imputation risk through pre-imputation assessment, providing a superior selective imputation approach for incomplete multi-view clustering that is more efficient and robust than existing methods.

Abstract: Incomplete Multi-view Clustering (IMC) has emerged as a significant challenge in multi-view learning. A predominant line for IMC is data imputation; however, indiscriminate imputation can result in unreliable content. Recently, researchers have proposed selective imputation methods that use a post-imputation assessment strategy: (1) impute all or some missing values, and (2) evaluate their quality through clustering tasks. We observe that this strategy incurs substantial computational complexity and is heavily dependent on the performance of the clustering model. To address these challenges, we first introduce the concept of pre-imputation assessment. We propose an Implicit Informativeness-based Selective Imputation (SI$^3$) method for incomplete multi-view clustering, which explicitly addresses the trade-off between imputation utility and imputation risk. SI$^3$ evaluates the imputation-relevant informativeness of each missing position in a training-free manner, and selectively imputes data only when sufficient informative support is available. Under a multi-view generative assumption, SI$^3$ further integrates selective imputation into a variational inference framework, enabling uncertainty-aware imputation at the latent distribution level and robust multi-view fusion. Compared with existing selective imputation strategies, SI$^3$ is lightweight, data-driven, and model-agnostic, and can be seamlessly incorporated into existing incomplete multi-view clustering frameworks as a plug-in strategy. Extensive experiments on multiple benchmark datasets demonstrate that SI$^3$ consistently outperforms both imputation-based and imputation-free methods, particularly under challenging unbalanced missing scenarios.

Method: Analyzed 26,400 synthetic faces generated using Stable Diffusion 2.1 and 3.5 Medium, examining associations between facial attractiveness and attributes, and testing gender classification algorithms on these synthetic faces.

Result: Three key findings: 1) T2I models systematically associate attractiveness with positive attributes; 2) Gender classification shows bias with women’s faces (especially with negative attributes) having higher misclassification rates; 3) Newer models intensify aesthetic constraints through age homogenization, gendered exposure, and geographic reductionism.

Conclusion: Algorithmic lookism operates as systematic infrastructure across AI vision systems, compounding inequalities through both representation (generation) and recognition (classification), revealing how AI perpetuates and amplifies societal biases.

Abstract: This paper examines algorithmic lookism-the systematic preferential treatment based on physical appearance-in text-to-image (T2I) generative AI and a downstream gender classification task. Through the analysis of 26,400 synthetic faces created with Stable Diffusion 2.1 and 3.5 Medium, we demonstrate how generative AI models systematically associate facial attractiveness with positive attributes and vice-versa, mirroring socially constructed biases rather than evidence-based correlations. Furthermore, we find significant gender bias in three gender classification algorithms depending on the attributes of the input faces. Our findings reveal three critical harms: (1) the systematic encoding of attractiveness-positive attribute associations in T2I models; (2) gender disparities in classification systems, where women’s faces, particularly those generated with negative attributes, suffer substantially higher misclassification rates than men’s; and (3) intensifying aesthetic constraints in newer models through age homogenization, gendered exposure patterns, and geographic reductionism. These convergent patterns reveal algorithmic lookism as systematic infrastructure operating across AI vision systems, compounding existing inequalities through both representation and recognition. Disclaimer: This work includes visual and textual content that reflects stereotypical associations between physical appearance and socially constructed attributes, including gender, race, and traits associated with social desirability. Any such associations found in this study emerge from the biases embedded in generative AI systems-not from empirical truths or the authors’ views.

Method: 1) Low-level segmentation using MeanShift to capture color, texture, and segment shape. 2) Construct pixel-segment graph with edge weights determined by novel Pixel Segment Similarity Index (PSSI) - harmonic mean of inter-channel similarities incorporating intensity and spatial smoothness. 3) Partition using Maximum Spanning Tree (MaxST) to capture strongly connected local neighborhoods.

Result: Outperforms current graph-based methods (AMOE, OneCut, SSNCut) on GrabCut and Images250 datasets in terms of Jaccard Index (IoU), F1 score, execution time, and Mean Error (ME). PSSI has O(B) computational complexity where B is number of histogram bins.

Conclusion: The integration of PSSI, MeanShift, and MaxST effectively captures color similarity, smoothness, texture, shape, and strong local connectivity, providing robust and efficient interactive segmentation with superior performance over existing methods.

Abstract: Interactive graph-based segmentation methods partition an image into foreground and background regions with the aid of user inputs. However, existing approaches often suffer from high computational costs, sensitivity to user interactions, and degraded performance when the foreground and background share similar color distributions. A key factor influencing segmentation performance is the similarity measure used for assigning edge weights in the graph. To address these challenges, we propose a novel Pixel Segment Similarity Index (PSSI), which leverages the harmonic mean of inter-channel similarities by incorporating both pixel intensity and spatial smoothness features. The harmonic mean effectively penalizes dissimilarities in any individual channel, enhancing robustness. The computational complexity of PSSI is $\mathcal{O}(B)$, where $B$ denotes the number of histogram bins. Our segmentation framework begins with low-level segmentation using MeanShift, which effectively captures color, texture, and segment shape. Based on the resulting pixel segments, we construct a pixel-segment graph with edge weights determined by PSSI. For partitioning, we employ the Maximum Spanning Tree (MaxST), which captures strongly connected local neighborhoods beneficial for precise segmentation. The integration of the proposed PSSI, MeanShift, and MaxST allows our method to jointly capture color similarity, smoothness, texture, shape, and strong local connectivity. Experimental evaluations on the GrabCut and Images250 datasets demonstrate that our method consistently outperforms current graph-based interactive segmentation methods such as AMOE, OneCut, and SSNCut in terms of segmentation quality, as measured by Jaccard Index (IoU), $F_1$ score, execution time and Mean Error (ME). Code is publicly available at: https://github.com/KaustubhShejole/PSSI-MaxST.

Method: Masked Binary U-Net (MBU-Net) uses a cost-aware masking strategy that prioritizes masking where it yields highest accuracy-per-cost, plus a GPU execution framework that maps MBU-Net to Tensor Cores via subtractive bit-encoding scheme using native binary Tensor Core instructions.

Result: Across 3 segmentation benchmarks, MBU-Net achieves near full-precision accuracy (only 3% average drop) while delivering 2.04x speedup and 3.54x energy reductions over 16-bit floating point U-Net.

Conclusion: MBU-Net successfully reconciles accuracy with near-binary efficiency through masked binary quantization and GPU-optimized execution, enabling real-time segmentation on resource-constrained edge devices.

Abstract: Real-time image segmentation is a key enabler for AR/VR, robotics, drones, and autonomous systems, where tight accuracy, latency, and energy budgets must be met on resource-constrained edge devices. While U-Net offers a favorable balance of accuracy and efficiency compared to large transformer-based models, achieving real-time performance on high-resolution input remains challenging due to compute, memory, and power limits. Extreme quantization, particularly binary networks, is appealing for its hardware-friendly operations. However, two obstacles limit practicality: (1) severe accuracy degradation, and (2) a lack of end-to-end implementations that deliver efficiency on general-purpose GPUs. We make two empirical observations that guide our design. (1) An explicit zero state is essential: training with zero masking to binary U-Net weights yields noticeable sparsity. (2) Quantization sensitivity is uniform across layers. Motivated by these findings, we introduce Masked Binary U-Net (MBU-Net), obtained through a cost-aware masking strategy that prioritizes masking where it yields the highest accuracy-per-cost, reconciling accuracy with near-binary efficiency. To realize these gains in practice, we develop a GPU execution framework that maps MBU-Net to Tensor Cores via a subtractive bit-encoding scheme, efficiently implementing masked binary weights with binary activations. This design leverages native binary Tensor Core BMMA instructions, enabling high throughput and energy savings on widely available GPUs. Across 3 segmentation benchmarks, MBU-Net attains near full-precision accuracy (3% average drop) while delivering 2.04x speedup and 3.54x energy reductions over a 16-bit floating point U-Net.

Method: Based on YOLO11 architecture, uses thermal imaging from LWIR cameras, integrates depthwise separable convolutions and feature pyramid network (FPN) for computational efficiency and accuracy on edge devices.

Result: LTV-YOLO achieves strong performance in detecting small-scale, partially occluded, and thermally distinct VRUs while maintaining compact architecture suitable for real-time edge deployment.

Conclusion: Provides a practical, scalable thermal-only solution optimized specifically for young/small VRUs detection in adverse conditions, contributing to pedestrian safety in intelligent transportation systems.

Abstract: Detecting vulnerable road users (VRUs), particularly children and adolescents, in low light and adverse weather conditions remains a critical challenge in computer vision, surveillance, and autonomous vehicle systems. This paper presents a purpose-built lightweight object detection model designed to identify young pedestrians in various environmental scenarios. To address these challenges, our approach leverages thermal imaging from long-wave infrared (LWIR) cameras, which enhances detection reliability in conditions where traditional RGB cameras operating in the visible spectrum fail. Based on the YOLO11 architecture and customized for thermal detection, our model, termed LTV-YOLO (Lightweight Thermal Vision YOLO), is optimized for computational efficiency, accuracy and real-time performance on edge devices. By integrating separable convolutions in depth and a feature pyramid network (FPN), LTV-YOLO achieves strong performance in detecting small-scale, partially occluded, and thermally distinct VRUs while maintaining a compact architecture. This work contributes a practical and scalable solution to improve pedestrian safety in intelligent transportation systems, particularly in school zones, autonomous navigation, and smart city infrastructure. Unlike prior thermal detectors, our contribution is task-specific: a thermally only edge-capable design designed for young and small VRUs (children and distant adults). Although FPN and depthwise separable convolutions are standard components, their integration into a thermal-only pipeline optimized for short/occluded VRUs under adverse conditions is, to the best of our knowledge, novel.

Method: Synthesis of over 150 studies, proposing a workflow framework integrating RGB imagery, LiDAR, and thermal sensing with transformer-based architectures, dynamic path planning, and comprehensive step-by-step guidance for complex environments.

Result: UAVs demonstrate value in structural health monitoring, disaster response, urban infrastructure management, energy efficiency evaluations, and cultural heritage preservation, with innovations in path optimization, thermal integration, and ML models like YOLO and Faster R-CNN.

Conclusion: Future research should focus on lightweight AI models, adaptive flight planning, synthetic datasets, and richer modality fusion to streamline modern infrastructure inspections and address remaining challenges in real-time processing and generalizability.

Abstract: Unmanned Aerial Vehicles (UAVs) are transforming infrastructure inspections in the Architecture, Engineering, Construction, and Facility Management (AEC+FM) domain. By synthesizing insights from over 150 studies, this review paper highlights UAV-based methodologies for data acquisition, photogrammetric modeling, defect detection, and decision-making support. Key innovations include path optimization, thermal integration, and advanced machine learning (ML) models such as YOLO and Faster R-CNN for anomaly detection. UAVs have demonstrated value in structural health monitoring (SHM), disaster response, urban infrastructure management, energy efficiency evaluations, and cultural heritage preservation. Despite these advancements, challenges in real-time processing, multimodal data fusion, and generalizability remain. A proposed workflow framework, informed by literature and a case study, integrates RGB imagery, LiDAR, and thermal sensing with transformer-based architectures to improve accuracy and reliability in detecting structural defects, thermal anomalies, and geometric inconsistencies. The proposed framework ensures precise and actionable insights by fusing multimodal data and dynamically adapting path planning for complex environments, presented as a comprehensive step-by-step guide to address these challenges effectively. This paper concludes with future research directions emphasizing lightweight AI models, adaptive flight planning, synthetic datasets, and richer modality fusion to streamline modern infrastructure inspections.

Method: MATEX combines multi-layer attention rollout, text-guided spatial priors, and layer consistency analysis to generate precise, stable, and anatomically informed gradient attribution maps.

Result: On the MS-CXR dataset, MATEX outperforms state-of-the-art M2IB in both spatial precision and alignment with expert-annotated findings.

Conclusion: MATEX enhances trust and transparency in radiological AI applications by providing more faithful and interpretable model explanations through anatomically informed spatial reasoning.

Abstract: We introduce MATEX (Multi-scale Attention and Text-guided Explainability), a novel framework that advances interpretability in medical vision-language models by incorporating anatomically informed spatial reasoning. MATEX synergistically combines multi-layer attention rollout, text-guided spatial priors, and layer consistency analysis to produce precise, stable, and clinically meaningful gradient attribution maps. By addressing key limitations of prior methods, such as spatial imprecision, lack of anatomical grounding, and limited attention granularity, MATEX enables more faithful and interpretable model explanations. Evaluated on the MS-CXR dataset, MATEX outperforms the state-of-the-art M2IB approach in both spatial precision and alignment with expert-annotated findings. These results highlight MATEX’s potential to enhance trust and transparency in radiological AI applications.

Method: PISE combines adjoint operator initialization with semantic guidance in a physics-informed deep ghost imaging framework to enhance edge perception capabilities.

Result: PISE improves classification accuracy by 2.57% and reduces variance by 9x at 5% sampling rate compared to baseline methods.

Conclusion: The proposed PISE framework effectively addresses low-bandwidth edge perception challenges by leveraging physics-informed deep learning with adjoint operators and semantic guidance, achieving significant improvements in both accuracy and stability.

Abstract: We propose PISE, a physics-informed deep ghost imaging framework for low-bandwidth edge perception. By combining adjoint operator initialization with semantic guidance, PISE improves classification accuracy by 2.57% and reduces variance by 9x at 5% sampling.

Method: Developed MetamerGen, a latent diffusion model with dual-stream DINOv2 token representation that fuses detailed features from fixated areas with peripherally degraded context features. Evaluated using same-different behavioral experiments where participants compared generated images to originals.

Result: MetamerGen successfully generates image metamers that align with human scene representations. High-level semantic alignment most strongly predicts metamerism when conditioned on viewers’ own fixations, though it can generate metamers even with random fixations.

Conclusion: MetamerGen is a powerful tool for studying scene understanding, revealing specific visual processing features that contribute to human judgments. It bridges computational modeling with human perception by generating images that match latent scene representations.

Abstract: Human vision combines low-resolution “gist” information from the visual periphery with sparse but high-resolution information from fixated locations to construct a coherent understanding of a visual scene. In this paper, we introduce MetamerGen, a tool for generating scenes that are aligned with latent human scene representations. MetamerGen is a latent diffusion model that combines peripherally obtained scene gist information with information obtained from scene-viewing fixations to generate image metamers for what humans understand after viewing a scene. Generating images from both high and low resolution (i.e. “foveated”) inputs constitutes a novel image-to-image synthesis problem, which we tackle by introducing a dual-stream representation of the foveated scenes consisting of DINOv2 tokens that fuse detailed features from fixated areas with peripherally degraded features capturing scene context. To evaluate the perceptual alignment of MetamerGen generated images to latent human scene representations, we conducted a same-different behavioral experiment where participants were asked for a “same” or “different” response between the generated and the original image. With that, we identify scene generations that are indeed metamers for the latent scene representations formed by the viewers. MetamerGen is a powerful tool for understanding scene understanding. Our proof-of-concept analyses uncovered specific features at multiple levels of visual processing that contributed to human judgments. While it can generate metamers even conditioned on random fixations, we find that high-level semantic alignment most strongly predicts metamerism when the generated scenes are conditioned on viewers’ own fixated regions.

Method: 1) Cross-modal differentiated quantization framework for VLMs that implements differentiated strategies to reduce memory usage. 2) Scene-aware vectorized memory multi-agent system using perception-memory-reasoning workflows to provide environmental information beyond current view.

Result: Quantization reduced memory from 38GB to 11.3GB. The 19B-parameter quantized model only experienced 2.05% performance drop on MMBench and maintained 63.7 accuracy on OCR-VQA (original: 64.9), outperforming smaller models with equivalent memory. System achieved 2.83-3.52s latency to initial speech output.

Conclusion: The research advances computational efficiency and assistive technology by providing comprehensive assistance in scene perception, text recognition, and navigation through efficient VLM quantization and integrated multi-agent system design.

Abstract: Visually impaired individuals face significant challenges in environmental perception. Traditional assistive technologies often lack adaptive intelligence, focusing on individual components rather than integrated systems. While Vision-Language Models (VLMs) offer a promising path to richer, integrated understanding, their deployment is severely limited by substantial computational requirements, demanding dozens of gigabytes of memory. To address these gaps in computational efficiency and integrated design, this study proposes a dual technological innovation framework: a cross-modal differentiated quantization framework for VLMs and a scene-aware vectorized memory multi-agent system. The quantization framework implements differentiated strategies, reducing memory from 38GB to 11.3GB. The multi-agent system uses vectorized memory and perception-memory-reasoning workflows to provide environmental information beyond the current view, achieving 2.83-3.52s latency to initial speech output. Experiments show the quantized 19B-parameter model only experiences a 2.05% performance drop on MMBench and maintains 63.7 accuracy on OCR-VQA (original: 64.9), outperforming smaller models with equivalent memory. This research advances computational efficiency and assistive technology, offering comprehensive assistance in scene perception, text recognition, and navigation.

Method: Two-step approach: 1) Generate uniformly distributed virtual cameras for comprehensive coverage; 2) Trim Gaussian primitives based on 2D segmentation results from these virtual cameras.

Result: Extensive experiments show GaussianTrimmer effectively improves segmentation quality of existing 3D Gaussian segmentation methods as a plug-and-play solution.

Conclusion: GaussianTrimmer provides an efficient, plug-and-play boundary trimming solution that addresses the jagged boundary problem in 3D Gaussian segmentation without modifying the underlying segmentation methods.

Abstract: With the widespread application of 3D Gaussians in 3D scene representation, 3D scene segmentation methods based on 3D Gaussians have also gradually emerged. However, existing 3D Gaussian segmentation methods basically segment on the basis of Gaussian primitives. Due to the large variation range of the scale of 3D Gaussians, large-sized Gaussians that often span the foreground and background lead to jagged boundaries of segmented objects. To this end, we propose an online boundary trimming method, GaussianTrimmer, which is an efficient and plug-and-play post-processing method capable of trimming coarse boundaries for existing 3D Gaussian segmentation methods. Our method consists of two core steps: 1. Generating uniformly and well-covered virtual cameras; 2. Trimming Gaussian at the primitive level based on 2D segmentation results on virtual cameras. Extensive quantitative and qualitative experiments demonstrate that our method can improve the segmentation quality of existing 3D Gaussian segmentation methods as a plug-and-play method.

Method: Utilizes conformal points and calibrating conics as geometric tools to analyze image geometry, focusing on their relationships and applications for geometric computation.

Result: Developed concepts that enable intuitive visualization of image geometry and provide straightforward ways to compute geometric properties like angles and directions.

Conclusion: Conformal points and calibrating conics are valuable geometric concepts that facilitate intuitive understanding and computation of image geometry.

Abstract: This gives some information about the conformal point and the calibrating conic, and their relationship one to the other. These concepts are useful for visualizing image geometry, and lead to intuitive ways to compute geometry, such as angles and directions in an image.

Method: Train a shallow recurrent neural network on non-featurized trajectory data from 10 public handwriting datasets, testing against 7 different synthesizers (including Kinematic Theory, GANs, Transformers, Diffusion models). Evaluate in few-shot settings (10% training data) and out-of-domain scenarios.

Result: Excellent performance with 98.3% average AUC and 1.4% equal error rate across all synthesizers and datasets. Strong few-shot performance using only 10% training data, and competitive out-of-domain results.

Conclusion: The approach provides effective human presence verification with implications for computerized security systems, adding an additional layer of protection against attackers by distinguishing human from synthetic handwriting.

Abstract: Handwriting movements can be leveraged as a unique form of behavioral biometrics, to verify whether a real user is operating a device or application. This task can be framed as a reverse Turing test in which a computer has to detect if an input instance has been generated by a human or artificially. To tackle this task, we study ten public datasets of handwritten symbols (isolated characters, digits, gestures, pointing traces, and signatures) that are artificially reproduced using seven different synthesizers, including, among others, the Kinematic Theory (Sigma h model), generative adversarial networks, Transformers, and Diffusion models. We train a shallow recurrent neural network that achieves excellent performance (98.3 percent Area Under the ROC Curve (AUC) score and 1.4 percent equal error rate on average across all synthesizers and datasets) using nonfeaturized trajectory data as input. In few-shot settings, we show that our classifier achieves such an excellent performance when trained on just 10 percent of the data, as evaluated on the remaining 90% of the data as a test set. We further challenge our classifier in out-of-domain settings, and observe very competitive results as well. Our work has implications for computerized systems that need to verify human presence, and adds an additional layer of security to keep attackers at bay.

Method: FiCoP introduces: 1) object-centric disentanglement preprocessing to isolate semantic targets from noise, 2) Cross-Perspective Global Perception module for dual-view feature fusion with explicit context reasoning, and 3) Patch Correlation Predictor that generates block-wise association maps as spatial filters for fine-grained matching.

Result: On REAL275 and Toyota-Light datasets, FiCoP improves Average Recall by 8.0% and 6.1% respectively compared to state-of-the-art methods, demonstrating robust performance in complex open-world environments.

Conclusion: FiCoP successfully addresses the ambiguity problem in open-vocabulary 6D pose estimation by transitioning from global matching to spatially-constrained patch-level correspondence, enabling more robust and generalized perception for robotic manipulation in unconstrained environments.

Abstract: Open-vocabulary 6D object pose estimation empowers robots to manipulate arbitrary unseen objects guided solely by natural language. However, a critical limitation of existing approaches is their reliance on unconstrained global matching strategies. In open-world scenarios, trying to match anchor features against the entire query image space introduces excessive ambiguity, as target features are easily confused with background distractors. To resolve this, we propose Fine-grained Correspondence Pose Estimation (FiCoP), a framework that transitions from noise-prone global matching to spatially-constrained patch-level correspondence. Our core innovation lies in leveraging a patch-to-patch correlation matrix as a structural prior to narrowing the matching scope, effectively filtering out irrelevant clutter to prevent it from degrading pose estimation. Firstly, we introduce an object-centric disentanglement preprocessing to isolate the semantic target from environmental noise. Secondly, a Cross-Perspective Global Perception (CPGP) module is proposed to fuse dual-view features, establishing structural consensus through explicit context reasoning. Finally, we design a Patch Correlation Predictor (PCP) that generates a precise block-wise association map, acting as a spatial filter to enforce fine-grained, noise-resilient matching. Experiments on the REAL275 and Toyota-Light datasets demonstrate that FiCoP improves Average Recall by 8.0% and 6.1%, respectively, compared to the state-of-the-art method, highlighting its capability to deliver robust and generalized perception for robotic agents operating in complex, unconstrained open-world environments. The source code will be made publicly available at https://github.com/zjjqinyu/FiCoP.

Method: Aligns intermediate features with VLM’s semantically rich feature space for domain-invariance, enhanced with image-level augmentation and output-level regularization strategies.

Result: Achieves state-of-the-art results across four benchmarks, showing both qualitative and quantitative improvements over existing SSDG baselines.

Conclusion: The proposed VLM feature alignment approach with effective data utilization strategies successfully addresses SSDG challenges and outperforms existing methods.

Abstract: Semi-supervised Domain Generalization (SSDG) addresses the challenge of generalizing to unseen target domains with limited labeled data. Existing SSDG methods highlight the importance of achieving high pseudo-labeling (PL) accuracy and preventing model overfitting as the main challenges in SSDG. In this light, we show that the SSDG literature’s excessive focus on PL accuracy, without consideration for maximum data utilization during training, limits potential performance improvements. We propose a novel approach to the SSDG problem by aligning the intermediate features of our model with the semantically rich and generalized feature space of a Vision Language Model (VLM) in a way that promotes domain-invariance. The above approach is enhanced with effective image-level augmentation and output-level regularization strategies to improve data utilization and minimize overfitting. Extensive experimentation across four benchmarks against existing SSDG baselines suggests that our method achieves SOTA results both qualitatively and quantitatively. The code will be made publicly available.

Method: EID uses equivariant learning to exploit image symmetry, enforces haze consistency, and employs adversarial learning to model unknown haze physics without supervision.

Result: EID significantly outperforms state-of-the-art methods on scientific image benchmarks (cell microscopy, medical endoscopy) and natural image dehazing.

Conclusion: By combining equivariant learning with haze physics modeling, EID enables versatile and effective haze removal for scientific imaging applications.

Abstract: Image Dehazing (ID) aims to produce a clear image from an observation contaminated by haze. Current ID methods typically rely on carefully crafted priors or extensive haze-free ground truth, both of which are expensive or impractical to acquire, particularly in the context of scientific imaging. We propose a new unsupervised learning framework called Equivariant Image Dehazing (EID) that exploits the symmetry of image signals to restore clarity to hazy observations. By enforcing haze consistency and systematic equivariance, EID can recover clear patterns directly from raw, hazy images. Additionally, we propose an adversarial learning strategy to model unknown haze physics and facilitate EID learning. Experiments on two scientific image dehazing benchmarks (including cell microscopy and medical endoscopy) and on natural image dehazing have demonstrated that EID significantly outperforms state-of-the-art approaches. By unifying equivariant learning with modelling haze physics, we hope that EID will enable more versatile and effective haze removal in scientific imaging. Code and datasets will be published.

Method: Introduces the Spatial Relation Recognition Task (SpaRRTa) benchmark that generates arbitrary numbers of photorealistic images with diverse scenes and fully controllable object arrangements, along with freely accessible spatial annotations. Unlike traditional 3D objectives focusing on precise metric prediction, SpaRRTa probes fundamental capabilities underpinning human-like spatial understanding.

Result: Evaluation of state-of-the-art VFMs reveals significant disparities in their spatial reasoning abilities. The analysis provides insights into mechanisms that support or hinder spatial awareness in modern VFMs.

Conclusion: SpaRRTa serves as a useful tool for guiding development of future spatially aware visual models by systematically evaluating and understanding spatial reasoning capabilities in Visual Foundation Models.

Abstract: Visual Foundation Models (VFMs), such as DINO and CLIP, excel in semantic understanding of images but exhibit limited spatial reasoning capabilities, which limits their applicability to embodied systems. As a result, recent work incorporates some 3D tasks (such as depth estimation) into VFM training. However, VFM performance remains inconsistent across other spatial tasks, raising the question of whether these models truly have spatial awareness or overfit to specific 3D objectives. To address this question, we introduce the Spatial Relation Recognition Task (SpaRRTa) benchmark, which evaluates the ability of VFMs to identify relative positions of objects in the image. Unlike traditional 3D objectives that focus on precise metric prediction (e.g., surface normal estimation), SpaRRTa probes a fundamental capability underpinning more advanced forms of human-like spatial understanding. SpaRRTa generates an arbitrary number of photorealistic images with diverse scenes and fully controllable object arrangements, along with freely accessible spatial annotations. Evaluating a range of state-of-the-art VFMs, we reveal significant disparities between their spatial reasoning abilities. Through our analysis, we provide insights into the mechanisms that support or hinder spatial awareness in modern VFMs. We hope that SpaRRTa will serve as a useful tool for guiding the development of future spatially aware visual models.

Method: Taxonomy-decoupled architecture using classification-free region proposals and unified embeddings for similarity retrieval, plus LLM-as-a-Judge framework for zero-shot evaluation of visual similarity and category relevance.

Result: Deployed at scale on global home goods platform, improved retrieval quality and customer engagement; offline metrics strongly correlate with real-world outcomes.

Conclusion: The proposed approach enables more flexible and generalizable visual search while overcoming evaluation bottlenecks through LLM-based assessment, demonstrating practical success in production.

Abstract: Visual search is critical for e-commerce, especially in style-driven domains where user intent is subjective and open-ended. Existing industrial systems typically couple object detection with taxonomy-based classification and rely on catalog data for evaluation, which is prone to noise that limits robustness and scalability. We propose a taxonomy-decoupled architecture that uses classification-free region proposals and unified embeddings for similarity retrieval, enabling a more flexible and generalizable visual search. To overcome the evaluation bottleneck, we propose an LLM-as-a-Judge framework that assesses nuanced visual similarity and category relevance for query-result pairs in a zero-shot manner, removing dependence on human annotations or noise-prone catalog data. Deployed at scale on a global home goods platform, our system improves retrieval quality and yields a measurable uplift in customer engagement, while our offline evaluation metrics strongly correlate with real-world outcomes.

Method: Two key techniques: 1) Teacher-student architecture where a multi-view teacher model provides geometric supervision to the single-view student to address scale ambiguity and ensure geometric validity; 2) Extrapolation network that completes missing scene context for high-quality extrapolation.

Result: studentSplat achieves state-of-the-art single-view novel-view reconstruction quality, comparable performance to multi-view methods at scene level, and competitive performance as a self-supervised single-view depth estimation method.

Conclusion: The method demonstrates strong potential for general single-view 3D understanding tasks by effectively addressing scale ambiguity and extrapolation problems inherent in single-view scene reconstruction.

Abstract: Recent advance in feed-forward 3D Gaussian splatting has enable remarkable multi-view 3D scene reconstruction or single-view 3D object reconstruction but single-view 3D scene reconstruction remain under-explored due to inherited ambiguity in single-view. We present \textbf{studentSplat}, a single-view 3D Gaussian splatting method for scene reconstruction. To overcome the scale ambiguity and extrapolation problems inherent in novel-view supervision from a single input, we introduce two techniques: 1) a teacher-student architecture where a multi-view teacher model provides geometric supervision to the single-view student during training, addressing scale ambiguity and encourage geometric validity; and 2) an extrapolation network that completes missing scene context, enabling high-quality extrapolation. Extensive experiments show studentSplat achieves state-of-the-art single-view novel-view reconstruction quality and comparable performance to multi-view methods at the scene level. Furthermore, studentSplat demonstrates competitive performance as a self-supervised single-view depth estimation method, highlighting its potential for general single-view 3D understanding tasks.

Method: Uses two unsupervised image translators (CycleGAN and AdaIN-based model) to generate artificial datasets in the target domain, training object detectors on this generated data using only source domain annotations and unlabeled target domain data.

Result: Significant improvements in real-world autonomous driving scenarios, outperforming state-of-the-art methods in most cases and closing the performance gap toward the upper-bound (training with target data).

Conclusion: Proposes a simpler, more interpretable, and effective approach to unsupervised domain adaptation for object detection by generating artificial target domain datasets, demonstrating strong performance in autonomous driving applications.

Abstract: Unsupervised domain adaptation for object detection addresses the adaption of detectors trained in a source domain to work accurately in an unseen target domain. Recently, methods approaching the alignment of the intermediate features proven to be promising, achieving state-of-the-art results. However, these methods are laborious to implement and hard to interpret. Although promising, there is still room for improvements to close the performance gap toward the upper-bound (when training with the target data). In this work, we propose a method to generate an artificial dataset in the target domain to train an object detector. We employed two unsupervised image translators (CycleGAN and an AdaIN-based model) using only annotated data from the source domain and non-annotated data from the target domain. Our key contributions are the proposal of a less complex yet more effective method that also has an improved interpretability. Results on real-world scenarios for autonomous driving show significant improvements, outperforming state-of-the-art methods in most cases, further closing the gap toward the upper-bound.

Method: Multimodal deep learning framework integrating facial expressions (Transformer on landmark features), speech (Audio Spectrogram Transformer on mel spectrograms), and upper-body movements (MLP-Mixer on pose sequences), with attention-based fusion for cross-modal interactions.

Result: Achieved 95.83% accuracy and 96.00% F1-score on self-collected dataset of 222 videos from 37 subjects, outperforming unimodal baselines and detecting all stroke cases in test set with balanced sensitivity/specificity.

Conclusion: Multimodal learning shows strong potential for early stroke screening but requires larger, clinically representative datasets for reliable real-world deployment.

Abstract: Early identification of stroke symptoms is essential for enabling timely intervention and improving patient outcomes, particularly in prehospital settings. This study presents a fast, non-invasive multimodal deep learning framework for automatic binary stroke screening based on data collected during the F.A.S.T. assessment. The proposed approach integrates complementary information from facial expressions, speech signals, and upper-body movements to enhance diagnostic robustness. Facial dynamics are represented using landmark based features and modeled with a Transformer architecture to capture temporal dependencies. Speech signals are converted into mel spectrograms and processed using an Audio Spectrogram Transformer, while upper-body pose sequences are analyzed with an MLP-Mixer network to model spatiotemporal motion patterns. The extracted modality specific representations are combined through an attention-based fusion mechanism to effectively learn cross modal interactions. Experiments conducted on a self-collected dataset of 222 videos from 37 subjects demonstrate that the proposed multimodal model consistently outperforms unimodal baselines, achieving 95.83% accuracy and a 96.00% F1-score. The model attains a strong balance between sensitivity and specificity and successfully detects all stroke cases in the test set. These results highlight the potential of multimodal learning and transfer learning for early stroke screening, while emphasizing the need for larger, clinically representative datasets to support reliable real-world deployment.

Method: RemoteVAR conditions autoregressive prediction on multi-resolution fused bi-temporal features via cross-attention and employs an autoregressive training strategy specifically designed for change map prediction.

Result: Extensive experiments on standard change detection benchmarks show that RemoteVAR delivers consistent and significant improvements over strong diffusion-based and transformer-based baselines.

Conclusion: RemoteVAR establishes a competitive autoregressive alternative for remote sensing change detection, addressing the limitations of VARs for pixel-level discriminative tasks.

Abstract: Remote sensing change detection aims to localize and characterize scene changes between two time points and is central to applications such as environmental monitoring and disaster assessment. Meanwhile, visual autoregressive models (VARs) have recently shown impressive image generation capability, but their adoption for pixel-level discriminative tasks remains limited due to weak controllability, suboptimal dense prediction performance and exposure bias. We introduce RemoteVAR, a new VAR-based change detection framework that addresses these limitations by conditioning autoregressive prediction on multi-resolution fused bi-temporal features via cross-attention, and by employing an autoregressive training strategy designed specifically for change map prediction. Extensive experiments on standard change detection benchmarks show that RemoteVAR delivers consistent and significant improvements over strong diffusion-based and transformer-based baselines, establishing a competitive autoregressive alternative for remote sensing change detection. Code will be available \href{https://github.com/yilmazkorkmaz1/RemoteVAR}{\underline{here}}.

Method: Developed a dual-task model based on EfficientNetB4 architecture that simultaneously performs airborne object classification and threat-level prediction. Created the AODTA Dataset by aggregating and refining multiple public sources to address data scarcity. Benchmarked against AVD Dataset and compared with ResNet-50 baseline.

Result: EfficientNetB4 model achieved 96% accuracy in object classification and 90% accuracy in threat-level prediction, outperforming ResNet-50 baseline. The newly created AODTA Dataset successfully addressed data scarcity issues for training.

Conclusion: The dual-task EfficientNetB4 model shows strong promise for real-time threat assessment applications in surveillance, defense, and airspace management. Despite title referencing detection, the study focuses on classification and threat-level inference using pre-localized images from existing datasets.

Abstract: The rapid proliferation of airborne platforms, including commercial aircraft, drones, and UAVs, has intensified the need for real-time, automated threat assessment systems. Current approaches depend heavily on manual monitoring, resulting in limited scalability and operational inefficiencies. This work introduces a dual-task model based on EfficientNetB4 capable of performing airborne object classification and threat-level prediction simultaneously. To address the scarcity of clean, balanced training data, we constructed the AODTA Dataset by aggregating and refining multiple public sources. We benchmarked our approach on both the AVD Dataset and the newly developed AODTA Dataset and further compared performance against a ResNet-50 baseline, which consistently underperformed EfficientNetB4. Our EfficientNetB4 model achieved 96% accuracy in object classification and 90% accuracy in threat-level prediction, underscoring its promise for applications in surveillance, defense, and airspace management. Although the title references detection, this study focuses specifically on classification and threat-level inference using pre-localized airborne object images provided by existing datasets.

Method: Used center/surround retinex model with two light intensity encoding functions: logarithmic (original model) and Naka-Rushton (retinal photoreceptor model). Tested with color-variable LEDs illuminating targets under various lighting colors. Evaluated color discrimination using HSV color space and opponent color theory-based color planes.

Result: The Naka-Rushton function combined with double opponent color plane representation provided superior discrimination performance for identifying target colors under different illumination conditions compared to other combinations.

Conclusion: Using the Naka-Rushton function (modeling retinal photoreceptor response) with double opponent color representation in center/surround retinex models improves color constancy performance, offering better illumination-independent color discrimination for vision systems.

Abstract: Color is an important source of information for visual functions such as object recognition, but it is greatly affected by the color of illumination. The ability to perceive the color of a visual target independent of illumination color is called color constancy (CC), and is an important feature for vision systems that use color information. In this study, we investigated the effects of the light intensity encoding function on the performance of CC of the center/surround (C/S) retinex model, which is a well-known model inspired by CC of the visual nervous system. The functions used to encode light intensity are the logarithmic function used in the original C/S retinex model and the Naka-Rushton (N-R) function, which is a model of retinal photoreceptor response. Color-variable LEDs were used to illuminate visual targets with various lighting colors, and color information computed by each model was used to evaluate the degree to which the color of visual targets illuminated with different lighting colors could be discriminated. Color information was represented using the HSV color space and a color plane based on the classical opponent color theory. The results showed that the combination of the N-R function and the double opponent color plane representation provided superior discrimination performance.

Method: Proposes GW-VLM (Guess What Vision Language Model) with two key components: 1) Multi-Scale Visual Language Searching (MS-VLS) that uses multi-scale visual-language soft-alignment with VLMs to generate snippets from class-agnostic object detection results, and 2) Contextual Concept Prompt (CCP) that forms concept flow from MS-VLS to help LLMs understand snippets for OVOD. The approach engages pre-trained VLMs and LLMs in a “guess what” game without any training.

Result: Extensive experiments on natural datasets (COCO val, Pascal VOC) and remote sensing datasets (DIOR, NWPU-10) show that GW-VLM achieves superior OVOD performance compared to state-of-the-art methods, despite requiring no training steps.

Conclusion: GW-VLM successfully creates a universal understanding paradigm for open-vocabulary object detection by leveraging pre-trained foundation models through a novel training-free approach, demonstrating that effective OVOD can be achieved without additional training by better utilizing existing model capabilities.

Abstract: Open-Vocabulary Object Detection (OVOD) aims to develop the capability to detect anything. Although myriads of large-scale pre-training efforts have built versatile foundation models that exhibit impressive zero-shot capabilities to facilitate OVOD, the necessity of creating a universal understanding for any object cognition according to already pretrained foundation models is usually overlooked. Therefore, in this paper, a training-free Guess What Vision Language Model, called GW-VLM, is proposed to form a universal understanding paradigm based on our carefully designed Multi-Scale Visual Language Searching (MS-VLS) coupled with Contextual Concept Prompt (CCP) for OVOD. This approach can engage a pre-trained Vision Language Model (VLM) and a Large Language Model (LLM) in the game of “guess what”. Wherein, MS-VLS leverages multi-scale visual-language soft-alignment for VLM to generate snippets from the results of class-agnostic object detection, while CCP can form the concept of flow referring to MS-VLS and then make LLM understand snippets for OVOD. Finally, the extensive experiments are carried out on natural and remote sensing datasets, including COCO val, Pascal VOC, DIOR, and NWPU-10, and the results indicate that our proposed GW-VLM can achieve superior OVOD performance compared to the-state-of-the-art methods without any training step.

Method: Evaluation of a compact convolutional neural network across five publicly available real-world image datasets from Bangladesh covering urban encroachment, vehicle detection, road damage, and agricultural crops.

Result: The compact CNN demonstrates high classification accuracy, efficient convergence, low computational overhead, effectively captures discriminative features, and generalizes robustly across diverse scenarios.

Conclusion: Streamlined CNN architectures are suitable for small-class image classification tasks, offering good performance with reduced complexity and computational requirements.

Abstract: Convolutional neural networks (CNNs) have achieved state-of-the-art performance in image recognition tasks but often involve complex architectures that may overfit on small datasets. In this study, we evaluate a compact CNN across five publicly available, real-world image datasets from Bangladesh, including urban encroachment, vehicle detection, road damage, and agricultural crops. The network demonstrates high classification accuracy, efficient convergence, and low computational overhead. Quantitative metrics and saliency analyses indicate that the model effectively captures discriminative features and generalizes robustly across diverse scenarios, highlighting the suitability of streamlined CNN architectures for small-class image classification tasks.

Method: Proposes an intervention paradigm for representation space: uniformly models spurious correlations as a low-rank subspace, decomposes them via orthogonal low-rank projection, removes this subspace from original representations, and trains the orthogonal complement to capture authentic forgery-related features.

Result: Achieves state-of-the-art performance across several benchmarks with only 0.43M trainable parameters, demonstrating excellent robustness and generalization.

Conclusion: The low-rank subspace intervention effectively eliminates spurious correlation factors, ensuring classification decisions are based on authentic forgery cues, providing a practical solution to the generalization problem in face forgery detection.

Abstract: The generalization problem remains a critical challenge in face forgery detection. Some researches have discovered that ``a backdoor path” in the representations from forgery-irrelevant information to labels induces biased learning, thereby hindering the generalization. In this paper, these forgery-irrelevant information are collectively termed spurious correlations factors. Previous methods predominantly focused on identifying concrete, specific spurious correlation and designing corresponding solutions to address them. However, spurious correlations arise from unobservable confounding factors, making it impractical to identify and address each one individually. To address this, we propose an intervention paradigm for representation space. Instead of tracking and blocking various instance-level spurious correlation one by one, we uniformly model them as a low-rank subspace and intervene in them. Specifically, we decompose spurious correlation features into a low-rank subspace via orthogonal low-rank projection, subsequently removing this subspace from the original representation and training its orthogonal complement to capture forgery-related features. This low-rank projection removal effectively eliminates spurious correlation factors, ensuring that classification decision is based on authentic forgery cues. With only 0.43M trainable parameters, our method achieves state-of-the-art performance across several benchmarks, demonstrating excellent robustness and generalization.

Method: Used Gabor filters (modeling VNS feature extraction) as CNN preprocessing. Created dataset with images from different camera positions. Trained multiple CNN architectures with/without Gabor filters using limited training data from specific distances.

Result: Gabor filter preprocessing improved CNN generalization performance and contributed to reducing CNN size. CNNs with Gabor filters performed better when trained on limited data and tested on varied conditions.

Conclusion: Biological inspiration from visual nervous systems, implemented via Gabor filter preprocessing, offers effective solution for edge-based robot vision: improves generalization with limited training data while reducing model size.

Abstract: In this study, we propose a technique to improve the accuracy and reduce the size of convolutional neural networks (CNNs) running on edge devices for real-world robot vision applications. CNNs running on edge devices must have a small architecture, and CNNs for robot vision applications involving on-site object recognition must be able to be trained efficiently to identify specific visual targets from data obtained under a limited variation of conditions. The visual nervous system (VNS) is a good example that meets the above requirements because it learns from few visual experiences. Therefore, we used a Gabor filter, a model of the feature extractor of the VNS, as a preprocessor for CNNs to investigate the accuracy of the CNNs trained with small amounts of data. To evaluate how well CNNs trained on image data acquired under a limited variation of conditions generalize to data acquired under other conditions, we created an image dataset consisting of images acquired from different camera positions, and investigated the accuracy of the CNNs that trained using images acquired at a certain distance. The results were compared after training on multiple CNN architectures with and without Gabor filters as preprocessing. The results showed that preprocessing with Gabor filters improves the generalization performance of CNNs and contributes to reducing the size of CNNs.

Method: RADAR uses attention-based diffusion models to directly generate anomaly maps without reconstructing input images, eliminating the need for iterative reverse sampling and overcoming limitations of reconstruction-based approaches.

Result: RADAR outperforms state-of-the-art diffusion-based and statistical machine learning models on MVTec-AD and 3D-printed material datasets, improving F1 score by 7% on MVTec-AD and 13% on the 3D-printed material dataset.

Conclusion: RADAR provides a more efficient and accurate alternative to reconstruction-based anomaly detection by directly producing anomaly maps from diffusion models, enabling real-time applications while improving detection performance across multiple metrics.

Abstract: Generative models have demonstrated significant success in anomaly detection and segmentation over the past decade. Recently, diffusion models have emerged as a powerful alternative, outperforming previous approaches such as GANs and VAEs. In typical diffusion-based anomaly detection, a model is trained on normal data, and during inference, anomalous images are perturbed to a predefined intermediate step in the forward diffusion process. The corresponding normal image is then reconstructed through iterative reverse sampling. However, reconstruction-based approaches present three major challenges: (1) the reconstruction process is computationally expensive due to multiple sampling steps, making real-time applications impractical; (2) for complex or subtle patterns, the reconstructed image may correspond to a different normal pattern rather than the original input; and (3) Choosing an appropriate intermediate noise level is challenging because it is application-dependent and often assumes prior knowledge of anomalies, an assumption that does not hold in unsupervised settings. We introduce Reconstruction-free Anomaly Detection with Attention-based diffusion models in Real-time (RADAR), which overcomes the limitations of reconstruction-based anomaly detection. Unlike current SOTA methods that reconstruct the input image, RADAR directly produces anomaly maps from the diffusion model, improving both detection accuracy and computational efficiency. We evaluate RADAR on real-world 3D-printed material and the MVTec-AD dataset. Our approach surpasses state-of-the-art diffusion-based and statistical machine learning models across all key metrics, including accuracy, precision, recall, and F1 score. Specifically, RADAR improves F1 score by 7% on MVTec-AD and 13% on the 3D-printed material dataset compared to the next best model. Code available at: https://github.com/mehrdadmoradi124/RADAR

Method: 1) Subgraph-based training with weighted geometric overlapping relationships using soft supervised contrastive loss; 2) DiVLAD aggregator using DINO-inspired multi-head attention maps from ViT; 3) Learnable gating mechanism to combine semantic cues with visual features.

Result: Achieves state-of-the-art performance on GL3D dataset, significantly outperforming NetVLAD with minimal additional parameters. Training strategy provides consistent gains across different aggregation techniques.

Conclusion: SupScene effectively learns discriminative global descriptors for overlapping image pairs in SfM through geometric-aware training and attention-based feature aggregation.

Abstract: Image retrieval is a critical step for alleviating the quadratic complexity of image matching in unconstrained Structure-from-Motion (SfM). However, in this context, image retrieval typically focuses more on the image pairs of geometric matchability than on those of semantic similarity, a nuance that most existing deep learning-based methods guided by batched binaries (overlapping vs. non-overlapping pairs) fail to capture. In this paper, we introduce SupScene, a novel solution that learns global descriptors tailored for finding overlapping image pairs of similar geometric nature for SfM. First, to better underline co-visible regions, we employ a subgraph-based training strategy that moves beyond equally important isolated pairs, leveraging ground-truth geometric overlapping relationships with various weights to provide fine-grained supervision via a soft supervised contrastive loss. Second, we introduce DiVLAD, a DINO-inspired VLAD aggregator that leverages the inherent multi-head attention maps from the last block of ViT. And then, a learnable gating mechanism is designed to adaptively utilize these semantically salient cues with visual features, enabling a more discriminative global descriptor. Extensive experiments on the GL3D dataset demonstrate that our method achieves state-of-the-art performance, significantly outperforming NetVLAD while introducing a negligible number of additional trainable parameters. Furthermore, we show that the proposed training strategy brings consistent gains across different aggregation techniques. Code and models are available at https://anonymous.4open.science/r/SupScene-5B73.

Method: Fine-tuned Ultrasound Self-Supervised Foundation Model with Masked Autoencoding (USF-MAE) pretrained on 370,000+ unlabeled ultrasound images for binary classification of normal vs cystic hygroma cases. Used same dataset, preprocessing, and 4-fold cross-validation as DenseNet-169 baseline. Analyzed interpretability with Score-CAM visualizations.

Result: USF-MAE outperformed DenseNet-169 on all metrics: 0.96 accuracy vs 0.93, 0.94 sensitivity vs 0.92, 0.98 specificity vs 0.94, and 0.98 ROC-AUC vs 0.94. Score-CAM visualizations showed clinically relevant attention to fetal neck regions. Performance improvements were statistically significant (p=0.0057).

Conclusion: Ultrasound-specific self-supervised pretraining enables accurate, robust deep learning detection of cystic hygroma, overcoming limitations of small labeled datasets and supporting scalable early prenatal screening programs.

Abstract: Cystic hygroma is a high-risk prenatal ultrasound finding that portends high rates of chromosomal abnormalities, structural malformations, and adverse pregnancy outcomes. Automated detection can increase reproducibility and support scalable early screening programs, but supervised deep learning methods are limited by small labelled datasets. This study assesses whether ultrasound-specific self-supervised pretraining can facilitate accurate, robust deep learning detection of cystic hygroma in first-trimester ultrasound images. We fine-tuned the Ultrasound Self-Supervised Foundation Model with Masked Autoencoding (USF-MAE), pretrained on over 370,000 unlabelled ultrasound images, for binary classification of normal controls and cystic hygroma cases used in this study. Performance was evaluated on the same curated ultrasound dataset, preprocessing pipeline, and 4-fold cross-validation protocol as for the DenseNet-169 baseline, using accuracy, sensitivity, specificity, and the area under the receiver operating characteristic curve (ROC-AUC). Model interpretability was analyzed qualitatively using Score-CAM visualizations. USF-MAE outperformed the DenseNet-169 baseline on all evaluation metrics. The proposed model yielded a mean accuracy of 0.96, sensitivity of 0.94, specificity of 0.98, and ROC-AUC of 0.98 compared to 0.93, 0.92, 0.94, and 0.94 for the DenseNet-169 baseline, respectively. Qualitative Score-CAM visualizations of model predictions demonstrated clinical relevance by highlighting expected regions in the fetal neck for both positive and negative cases. Paired statistical analysis using a Wilcoxon signed-rank test confirmed that performance improvements achieved by USF-MAE were statistically significant (p = 0.0057).

Method: LMGait uses designed gait-related language cues to guide the capture of key motion features in gait sequences, creating a language-guided and motion-aware framework.

Result: The abstract doesn’t provide specific results, but the framework is presented as a solution to the mentioned challenges.

Conclusion: LMGait addresses the limitations of existing gait recognition methods by incorporating language guidance to better focus on dynamic motion features rather than static noise.

Abstract: Gait recognition is emerging as a promising technology and an innovative field within computer vision. However, existing methods typically rely on complex architectures to directly extract features from images and apply pooling operations to obtain sequence-level representations. Such designs often lead to overfitting on static noise (e.g., clothing), while failing to effectively capture dynamic motion regions.To address the above challenges, we present a Language guided and Motion-aware gait recognition framework, named LMGait.In particular, we utilize designed gait-related language cues to capture key motion features in gait sequences.

Method: Space-Reweighted Adversarial Warping (SRAW) generates adversarial examples through optimized spatial deformation with reweighted budgets across foreground and background regions, allowing for targeted perturbations.

Result: Extensive experiments show SRAW significantly degrades state-of-the-art SAR-ATR model performance and consistently outperforms existing methods in imperceptibility and adversarial transferability.

Conclusion: SRAW provides an effective and stealthy adversarial attack method for SAR-ATR systems, addressing the need for balanced performance between attack effectiveness and visual imperceptibility.

Abstract: Synthetic aperture radar (SAR) imagery exhibits intrinsic information sparsity due to its unique electromagnetic scattering mechanism. Despite the widespread adoption of deep neural network (DNN)-based SAR automatic target recognition (SAR-ATR) systems, they remain vulnerable to adversarial examples and tend to over-rely on background regions, leading to degraded adversarial robustness. Existing adversarial attacks for SAR-ATR often require visually perceptible distortions to achieve effective performance, thereby necessitating an attack method that balances effectiveness and stealthiness. In this paper, a novel attack method termed Space-Reweighted Adversarial Warping (SRAW) is proposed, which generates adversarial examples through optimized spatial deformation with reweighted budgets across foreground and background regions. Extensive experiments demonstrate that SRAW significantly degrades the performance of state-of-the-art SAR-ATR models and consistently outperforms existing methods in terms of imperceptibility and adversarial transferability. Code is made available at https://github.com/boremycin/SAR-ATR-TransAttack.

Method: Hierarchical Dense Attention Network with 3D spatial-channel attention mechanism and attention-guided dense upsampling strategy to enhance feature discrimination in low-contrast volumetric MRI data.

Result: Superior segmentation performance compared to state-of-the-art baselines, effectively tackling isointense tissue differentiation. Application confirms WM and GM volumes in preterm infants are significantly lower than in term infants.

Conclusion: The proposed network successfully addresses the isointense tissue segmentation challenge in preterm infants and provides imaging evidence of neurodevelopmental delays associated with preterm birth through volume measurements.

Abstract: Preterm infants (born between 28 and 37 weeks of gestation) face elevated risks of neurodevelopmental delays, making early identification crucial for timely intervention. While deep learning-based volumetric segmentation of brain MRI scans offers a promising avenue for assessing neonatal neurodevelopment, achieving accurate segmentation of white matter (WM) and gray matter (GM) in preterm infants remains challenging due to their comparable signal intensities (isointense appearance) on MRI during early brain development. To address this, we propose a novel segmentation neural network, named Hierarchical Dense Attention Network. Our architecture incorporates a 3D spatial-channel attention mechanism combined with an attention-guided dense upsampling strategy to enhance feature discrimination in low-contrast volumetric data. Quantitative experiments demonstrate that our method achieves superior segmentation performance compared to state-of-the-art baselines, effectively tackling the challenge of isointense tissue differentiation. Furthermore, application of our algorithm confirms that WM and GM volumes in preterm infants are significantly lower than those in term infants, providing additional imaging evidence of the neurodevelopmental delays associated with preterm birth. The code is available at: https://github.com/ICL-SUST/HDAN.

Method: Proposes DGS family: DGS-O (output), DGS-I (input), and DGS-L (layers) with closed-form solution for DGS-O. Uses joint reorienting and rescaling of gradients from watermark channel gradient leaking queries to prevent watermark remover convergence.

Result: DGS achieves 100% defense success rate across all settings in deraining and image generation tasks with state-of-the-art box-free watermarking, while preserving decoder output image quality.

Conclusion: DGS effectively protects box-free watermarking decoders against gradient-based attacks, addressing a critical vulnerability in current watermarking systems.

Abstract: Box-free model watermarking has gained significant attention in deep neural network (DNN) intellectual property protection due to its model-agnostic nature and its ability to flexibly manage high-entropy image outputs from generative models. Typically operating in a black-box manner, it employs an encoder-decoder framework for watermark embedding and extraction. While existing research has focused primarily on the encoders for the robustness to resist various attacks, the decoders have been largely overlooked, leading to attacks against the watermark. In this paper, we identify one such attack against the decoder, where query responses are utilized to obtain backpropagated gradients to train a watermark remover. To address this issue, we propose Decoder Gradient Shields (DGSs), a family of defense mechanisms, including DGS at the output (DGS-O), at the input (DGS-I), and in the layers (DGS-L) of the decoder, with a closed-form solution for DGS-O and provable performance for all DGS. Leveraging the joint design of reorienting and rescaling of the gradients from watermark channel gradient leaking queries, the proposed DGSs effectively prevent the watermark remover from achieving training convergence to the desired low-loss value, while preserving image quality of the decoder output. We demonstrate the effectiveness of our proposed DGSs in diverse application scenarios. Our experimental results on deraining and image generation tasks with the state-of-the-art box-free watermarking show that our DGSs achieve a defense success rate of 100% under all settings.

Method: Developed a unified pipeline integrating YOLOv8n for object detection, custom FaceNet-based embedding for facial recognition, and DeepFace’s CNN for emotion classification. The core innovation is an adaptive scheduling mechanism that selectively activates modules based on contextual triggers to reduce computational load.

Result: System achieved 65% reduction in computational load compared to continuous processing. Object detection AP: 0.861, facial recognition accuracy: 88%, emotion detection AUC up to 0.97 for specific emotions, operating at 5.6 FPS on Raspberry Pi 5.

Conclusion: Context-aware scheduling enables complex multi-modal AI on cost-effective edge hardware, making intelligent perception more accessible and privacy-preserving while maintaining good performance metrics.

Abstract: Intelligent surveillance systems often handle perceptual tasks such as object detection, facial recognition, and emotion analysis independently, but they lack a unified, adaptive runtime scheduler that dynamically allocates computational resources based on contextual triggers. This limits their holistic understanding and efficiency on low-power edge devices. To address this, we present a real-time multi-modal vision framework that integrates object detection, owner-specific face recognition, and emotion detection into a unified pipeline deployed on a Raspberry Pi 5 edge platform. The core of our system is an adaptive scheduling mechanism that reduces computational load by 65% compared to continuous processing by selectively activating modules such as, YOLOv8n for object detection, a custom FaceNet-based embedding system for facial recognition, and DeepFace’s CNN for emotion classification. Experimental results demonstrate the system’s efficacy, with the object detection module achieving an Average Precision (AP) of 0.861, facial recognition attaining 88% accuracy, and emotion detection showing strong discriminatory power (AUC up to 0.97 for specific emotions), while operating at 5.6 frames per second. Our work demonstrates that context-aware scheduling is the key to unlocking complex multi-modal AI on cost-effective edge hardware, making intelligent perception more accessible and privacy-preserving.

Method: AVIR framework: 1) Lightweight retrieval model scores page relevance, 2) Pages clustered by score distribution for adaptive selection, 3) Top-K screening keeps context compact, 4) For short documents, uses relevance probability threshold, 5) Selected pages fed to frozen LVLM without fine-tuning.

Result: Reduces average page count by 70%, achieves 84.58% ANLS on MP-DocVQA dataset (surpassing previous methods), verified on SlideVQA and DUDE benchmarks with significantly lower computational cost.

Conclusion: AVIR effectively addresses MP-DocVQA challenges by adaptively selecting relevant pages, reducing computational burden while maintaining high accuracy without model fine-tuning.

Abstract: Multi-page Document Visual Question Answering (MP-DocVQA) remains challenging because long documents not only strain computational resources but also reduce the effectiveness of the attention mechanism in large vision-language models (LVLMs). We tackle these issues with an Adaptive Visual In-document Retrieval (AVIR) framework. A lightweight retrieval model first scores each page for question relevance. Pages are then clustered according to the score distribution to adaptively select relevant content. The clustered pages are screened again by Top-K to keep the context compact. However, for short documents, clustering reliability decreases, so we use a relevance probability threshold to select pages. The selected pages alone are fed to a frozen LVLM for answer generation, eliminating the need for model fine-tuning. The proposed AVIR framework reduces the average page count required for question answering by 70%, while achieving an ANLS of 84.58% on the MP-DocVQA dataset-surpassing previous methods with significantly lower computational cost. The effectiveness of the proposed AVIR is also verified on the SlideVQA and DUDE benchmarks. The code is available at https://github.com/Li-yachuan/AVIR.

Method: RADAR uses retrieval-guided adaptation with: 1) Entropy Selection-Based Retrieval to find stable reference videos, 2) Stable Anchor-Guided Alignment for distribution-level matching, and 3) Target-Domain Aware Self-Training with pseudo-labels augmented by stable references.

Result: Extensive experiments show RADAR achieves superior performance for test-time fake news video detection, enabling strong on-the-fly adaptation to unseen fake news video topics.

Conclusion: RADAR is the first framework that enables effective test-time adaptation to unseen news videos, bridging the gap in detecting fake news videos tied to emerging events and unseen topics.

Abstract: Fake News Video Detection (FNVD) is critical for social stability. Existing methods typically assume consistent news topic distribution between training and test phases, failing to detect fake news videos tied to emerging events and unseen topics. To bridge this gap, we introduce RADAR, the first framework that enables test-time adaptation to unseen news videos. RADAR pioneers a new retrieval-guided adaptation paradigm that leverages stable (source-close) videos from the target domain to guide robust adaptation of semantically related but unstable instances. Specifically, we propose an Entropy Selection-Based Retrieval mechanism that provides videos with stable (low-entropy), relevant references for adaptation. We also introduce a Stable Anchor-Guided Alignment module that explicitly aligns unstable instances’ representations to the source domain via distribution-level matching with their stable references, mitigating severe domain discrepancies. Finally, our novel Target-Domain Aware Self-Training paradigm can generate informative pseudo-labels augmented by stable references, capturing varying and imbalanced category distributions in the target domain and enabling RADAR to adapt to the fast-changing label distributions. Extensive experiments demonstrate that RADAR achieves superior performance for test-time FNVD, enabling strong on-the-fly adaptation to unseen fake news video topics.

Method: Comprehensive AI-IoT system with glove-based gesture control for navigation, YOLOv8 for object detection with auditory feedback, ultrasonic sensors for collision avoidance, and continuous vital sign monitoring (heart rate, SpO2, ECG, temperature) uploaded to ThingSpeak cloud platform.

Result: Gesture control achieved 95.5% success rate, ultrasonic obstacle detection reached 94% accuracy, and YOLOv8 object detection delivered 91.5% Precision, 90.2% Recall, and 90.8% F1-score. System provides email alerts for critical health conditions.

Conclusion: Integrated multi-modal approach offers practical, scalable, affordable solution that enhances user autonomy, safety, and independence by bridging innovative research with real-world deployment.

Abstract: The growing number of differently-abled and elderly individuals demands affordable, intelligent wheelchairs that combine safe navigation with health monitoring. Traditional wheelchairs lack dynamic features, and many smart alternatives remain costly, single-modality, and limited in health integration. Motivated by the pressing demand for advanced, personalized, and affordable assistive technologies, we propose a comprehensive AI-IoT based smart wheelchair system that incorporates glove-based gesture control for hands-free navigation, real-time object detection using YOLOv8 with auditory feedback for obstacle avoidance, and ultrasonic for immediate collision avoidance. Vital signs (heart rate, SpO$_2$, ECG, temperature) are continuously monitored, uploaded to ThingSpeak, and trigger email alerts for critical conditions. Built on a modular and low-cost architecture, the gesture control achieved a 95.5% success rate, ultrasonic obstacle detection reached 94% accuracy, and YOLOv8-based object detection delivered 91.5% Precision, 90.2% Recall, and a 90.8% F1-score. This integrated, multi-modal approach offers a practical, scalable, and affordable solution, significantly enhancing user autonomy, safety, and independence by bridging the gap between innovative research and real-world deployment.

Method: Converts convolutional feature maps into patch-level graphs where nodes encode appearance and spatial coordinates, with edges reflecting local structural adjacency. Introduces custom structural propagation mechanism that explicitly models relative spatial relations, enabling structured inference as inductive bias.

Result: Demonstrated through chest X-ray case study showing how structural priors guide relational reasoning and improve interpretability. The framework supports both node-level lesion-aware predictions and graph-level diagnostic reasoning with intrinsic explainability through learned node importance scores.

Conclusion: The framework provides domain-agnostic graph-based reasoning for structure-aware and explainable learning, contributing to research on graphs as computational substrates for interpretable AI systems, particularly valuable in medical imaging contexts.

Abstract: We present a structural graph reasoning framework that incorporates explicit anatomical priors for explainable vision-based diagnosis. Convolutional feature maps are reinterpreted as patch-level graphs, where nodes encode both appearance and spatial coordinates, and edges reflect local structural adjacency. Unlike conventional graph neural networks that rely on generic message passing, we introduce a custom structural propagation mechanism that explicitly models relative spatial relations as part of the reasoning process. This design enables the graph to act as an inductive bias for structured inference rather than a passive relational representation. The proposed model jointly supports node-level lesion-aware predictions and graph-level diagnostic reasoning, yielding intrinsic explainability through learned node importance scores without relying on post-hoc visualization techniques. We demonstrate the approach through a chest X-ray case study, illustrating how structural priors guide relational reasoning and improve interpretability. While evaluated in a medical imaging context, the framework is domain-agnostic and aligns with the broader vision of graph-based reasoning across artificial intelligence systems. This work contributes to the growing body of research exploring graphs as computational substrates for structure-aware and explainable learning.

Method: 1) Created DAOS dataset: 9,787 video clips with 36 fine-grained driver actions and 15 object classes (2.5M object instances), multi-modal (RGB, IR, depth), multi-view (front, face, left, right). 2) Proposed AOR-Net: Action-Object-Relation Network with multi-level reasoning and chain-of-action prompting to model logical relationships among actions, objects, and relations. Includes Mixture of Thoughts module for dynamic knowledge selection.

Result: Extensive experiments show AOR-Net outperforms state-of-the-art methods on various datasets, demonstrating effectiveness in handling object-rich and object-scarce conditions.

Conclusion: The DAOS dataset addresses critical annotation gaps in driver monitoring, and AOR-Net effectively leverages action-object relations for robust driver action recognition through multi-level reasoning and dynamic knowledge selection.

Abstract: In driver activity monitoring, movements are mostly limited to the upper body, which makes many actions look similar. To tell these actions apart, human often rely on the objects the driver is using, such as holding a phone compared with gripping the steering wheel. However, most existing driver-monitoring datasets lack accurate object-location annotations or do not link objects to their associated actions, leaving a critical gap for reliable action recognition. To address this, we introduce the Driver Action with Object Synergy (DAOS) dataset, comprising 9,787 video clips annotated with 36 fine-grained driver actions and 15 object classes, totaling more than 2.5 million corresponding object instances. DAOS offers multi-modal, multi-view data (RGB, IR, and depth) from front, face, left, and right perspectives. Although DAOS captures a wide range of cabin objects, only a few are directly relevant to each action for prediction, so focusing on task-specific human-object relations is essential. To tackle this challenge, we propose the Action-Object-Relation Network (AOR-Net). AOR-Net comprehends complex driver actions through multi-level reasoning and a chain-of-action prompting mechanism that models the logical relationships among actions, objects, and their relations. Additionally, the Mixture of Thoughts module is introduced to dynamically select essential knowledge at each stage, enhancing robustness in object-rich and object-scarce conditions. Extensive experiments demonstrate that our model outperforms other state-of-the-art methods on various datasets.

Method: Coarse-to-fine pipeline: 1) VLMs for coarse image-text filtering, 2) database of successful mining cases to few-shot condition LLMs, 3) text-trajectory contrastive learning to refine candidate trajectories in shared embedding space.

Result: Experiments on public datasets show substantial improvements in both retrieval quality and efficiency compared to existing methods.

Conclusion: SMc2f addresses limitations of trajectory-based retrieval by leveraging vision-language models and contrastive learning, providing more robust scenario mining for autonomous vehicle safety validation.

Abstract: The safety validation of autonomous robotic vehicles hinges on systematically testing their planning and control stacks against rare, safety-critical scenarios. Mining these long-tail events from massive real-world driving logs is therefore a critical step in the robotic development lifecycle. The goal of the Scenario Mining task is to retrieve useful information to enable targeted re-simulation, regression testing, and failure analysis of the robot’s decision-making algorithms. RefAV, introduced by the Argoverse team, is an end-to-end framework that uses large language models (LLMs) to spatially and temporally localize scenarios described in natural language. However, this process performs retrieval on trajectory labels, ignoring the direct connection between natural language and raw RGB images, which runs counter to the intuition of video retrieval; it also depends on the quality of upstream 3D object detection and tracking. Further, inaccuracies in trajectory data lead to inaccuracies in downstream spatial and temporal localization. To address these issues, we propose Robust Scenario Mining for Robotic Autonomy from Coarse to Fine (SMc2f), a coarse-to-fine pipeline that employs vision-language models (VLMs) for coarse image-text filtering, builds a database of successful mining cases on top of RefAV and automatically retrieves exemplars to few-shot condition the LLM for more robust retrieval, and introduces text-trajectory contrastive learning to pull matched pairs together and push mismatched pairs apart in a shared embedding space, yielding a fine-grained matcher that refines the LLM’s candidate trajectories. Experiments on public datasets demonstrate substantial gains in both retrieval quality and efficiency.

Method: A hybrid deep learning model called DeepSegFusion that integrates SegNet and DeepLabV3+ architectures with an attention-based feature fusion mechanism to improve boundary precision and contextual understanding for oil spill segmentation in SAR images.

Result: The model achieves 94.85% accuracy, 0.5685 IoU, and 0.9330 ROC-AUC on SAR datasets including ALOS PALSAR imagery, with more than three times fewer false detections (64.4% reduction) compared to baseline models and traditional methods.

Conclusion: DeepSegFusion is a stable model under various marine conditions that can be used for near real-time oil spill monitoring, offering significant improvements over existing approaches for environmental surveillance and maritime safety.

Abstract: Detection of oil spills from satellite images is essential for both environmental surveillance and maritime safety. Traditional threshold-based methods frequently encounter performance degradation due to very high false alarm rates caused by look-alike phenomena such as wind slicks and ship wakes. Here, a hybrid deep learning model, DeepSegFusion, is presented for oil spill segmentation in Synthetic Aperture Radar (SAR) images. The model uses SegNet and DeepLabV3+ integrated with an attention-based feature fusion mechanism to achieve better boundary precision as well as improved contextual understanding. Results obtained on SAR oil spill datasets, including ALOS PALSAR imagery, confirm that the proposed DeepSegFusion model achieves an accuracy of 94.85%, an Intersection over Union (IoU) of 0.5685, and a ROC-AUC score of 0.9330. The proposed method delivers more than three times fewer false detections compared to individual baseline models and traditional non-segmentation methods, achieving a reduction of 64.4%. These results indicate that DeepSegFusion is a stable model under various marine conditions and can therefore be used in near real-time oil spill monitoring scenarios.

Method: Fine-tune diffusion models for novel-view synthesis and relighting conditioned on estimated geometry, trained on less than 7% of the dataset. Introduce illumination-independent geometric priors: multi-view silhouette consistency loss and multi-view depth consistency loss to stabilize reconstruction under sparse supervision.

Result: Achieves state-of-the-art quality in relightable Gaussian rendering across all sparsity regimes, reducing real capture requirements by up to 90% compared to SSS-3DGS, and can replace up to 95% of missing captures with photorealistic augmentations.

Conclusion: DIAMOND-SSS provides a data-efficient framework for high-fidelity translucent reconstruction from extremely sparse supervision, significantly reducing the data acquisition burden for subsurface scattering modeling in neural rendering.

Abstract: Subsurface scattering (SSS) gives translucent materials – such as wax, jade, marble, and skin – their characteristic soft shadows, color bleeding, and diffuse glow. Modeling these effects in neural rendering remains challenging due to complex light transport and the need for densely captured multi-view, multi-light datasets (often more than 100 views and 112 OLATs). We present DIAMOND-SSS, a data-efficient framework for high-fidelity translucent reconstruction from extremely sparse supervision – even as few as ten images. We fine-tune diffusion models for novel-view synthesis and relighting, conditioned on estimated geometry and trained on less than 7 percent of the dataset, producing photorealistic augmentations that can replace up to 95 percent of missing captures. To stabilize reconstruction under sparse or synthetic supervision, we introduce illumination-independent geometric priors: a multi-view silhouette consistency loss and a multi-view depth consistency loss. Across all sparsity regimes, DIAMOND-SSS achieves state-of-the-art quality in relightable Gaussian rendering, reducing real capture requirements by up to 90 percent compared to SSS-3DGS.

Method: FocaLogic identifies minimal interpretable subsets of visual regions called “visual focuses” that decisively influence model predictions, then translates these into precise logical expressions. It also introduces quantitative metrics (focus precision, recall, divergence) for objective evaluation.

Result: Empirical analyses show FocaLogic can uncover critical insights including training-induced concentration, improved focus accuracy through generalization, and anomalous focuses under biases and adversarial attacks.

Conclusion: FocaLogic provides a systematic, scalable, and quantitative solution for interpreting visual models, addressing limitations of existing methods and enabling transparent analysis of model decision-making.

Abstract: Interpretability of modern visual models is crucial, particularly in high-stakes applications. However, existing interpretability methods typically suffer from either reliance on white-box model access or insufficient quantitative rigor. To address these limitations, we introduce FocaLogic, a novel model-agnostic framework designed to interpret and quantify visual model decision-making through logic-based representations. FocaLogic identifies minimal interpretable subsets of visual regions-termed visual focuses-that decisively influence model predictions. It translates these visual focuses into precise and compact logical expressions, enabling transparent and structured interpretations. Additionally, we propose a suite of quantitative metrics, including focus precision, recall, and divergence, to objectively evaluate model behavior across diverse scenarios. Empirical analyses demonstrate FocaLogic’s capability to uncover critical insights such as training-induced concentration, increasing focus accuracy through generalization, and anomalous focuses under biases and adversarial attacks. Overall, FocaLogic provides a systematic, scalable, and quantitative solution for interpreting visual models.

Method: Masked Jigsaw Puzzle (MJP) framework: 1) Random token shuffling to break token order, 2) Learnable unknown position embedding to mask PEs of shuffled tokens, disrupting local spatial information and forcing models to learn less position-dependent representations.

Result: MJP improves robustness against gradient attacks while boosting performance in both vision (ImageNet-1K classification) and text (Yelp/Amazon sentiment analysis) tasks. Works as unified framework for different Transformer-based models.

Conclusion: MJP provides effective defense against gradient attacks in federated learning while enhancing Transformer performance across CV and NLP domains through position embedding disruption.

Abstract: In federated learning, Transformer, as a popular architecture, faces critical challenges in defending against gradient attacks and improving model performance in both Computer Vision (CV) and Natural Language Processing (NLP) tasks. It has been revealed that the gradient of Position Embeddings (PEs) in Transformer contains sufficient information, which can be used to reconstruct the input data. To mitigate this issue, we introduce a Masked Jigsaw Puzzle (MJP) framework. MJP starts with random token shuffling to break the token order, and then a learnable \textit{unknown (unk)} position embedding is used to mask out the PEs of the shuffled tokens. In this manner, the local spatial information which is encoded in the position embeddings is disrupted, and the models are forced to learn feature representations that are less reliant on the local spatial information. Notably, with the careful use of MJP, we can not only improve models’ robustness against gradient attacks, but also boost their performance in both vision and text application scenarios, such as classification for images (\textit{e.g.,} ImageNet-1K) and sentiment analysis for text (\textit{e.g.,} Yelp and Amazon). Experimental results suggest that MJP is a unified framework for different Transformer-based models in both vision and language tasks. Code is publicly available via https://github.com/ywxsuperstar/transformerattack

Method: Proposes TDP-CR with Prompt-Guided Fusion (PGF) mechanism that uses learnable degradation prompts to encode cloud thickness and spatial uncertainty. Combines global channel context with local prompt-conditioned spatial bias to adaptively integrate SAR information only where optical data is corrupted. Uses parameter-efficient two-phase training that decouples reconstruction and semantic representation learning.

Result: On LuojiaSET-OSFCR dataset: surpasses state-of-the-art baselines by 0.18 dB in PSNR while using only 15% of parameters, and achieves 1.4% improvement in mIoU consistently against multi-task competitors.

Conclusion: TDP-CR effectively bridges the gap between visually plausible restoration and semantic utility, delivering analysis-ready data through joint cloud removal and segmentation with efficient multimodal fusion.

Abstract: Optical remote sensing imagery is indispensable for Earth observation, yet persistent cloud occlusion limits its downstream utility. Most cloud removal (CR) methods are optimized for low-level fidelity and can over-smooth textures and boundaries that are critical for analysis-ready data (ARD), leading to a mismatch between visually plausible restoration and semantic utility. To bridge this gap, we propose TDP-CR, a task-driven multimodal framework that jointly performs cloud removal and land-cover segmentation. Central to our approach is a Prompt-Guided Fusion (PGF) mechanism, which utilizes a learnable degradation prompt to encode cloud thickness and spatial uncertainty. By combining global channel context with local prompt-conditioned spatial bias, PGF adaptively integrates Synthetic Aperture Radar (SAR) information only where optical data is corrupted. We further introduce a parameter-efficient two-phase training strategy that decouples reconstruction and semantic representation learning. Experiments on the LuojiaSET-OSFCR dataset demonstrate the superiority of our framework: TDP-CR surpasses heavy state-of-the-art baselines by 0.18 dB in PSNR while using only 15% of the parameters, and achieves a 1.4% improvement in mIoU consistently against multi-task competitors, effectively delivering analysis-ready data.

Method: Generated calibration (n=110) and validation (n=55) sets from open-source data for U-net architecture search and stopping point optimization. Developed AUTO-DIP pipeline that automatically transfers parameters from calibration dataset to test images based on image metadata similarity (microscope type, imaged specimen).

Result: Parameter transfer based on metadata similarity outperforms quantitative image similarity measures. AUTO-DIP beats baseline DIP (original parameters) and state-of-the-art variational denoising approaches on multiple test datasets, especially for very noisy inputs. Validated on locally acquired fluorescence microscopy images.

Conclusion: AUTO-DIP enables efficient, optimization-free DIP denoising for fluorescence microscopy by leveraging metadata-based parameter transfer, making DIP practical for processing large image collections without per-image optimization.

Abstract: Unsupervised deep image prior (DIP) addresses shortcomings of training data requirements and limited generalization associated with supervised deep learning. The performance of DIP depends on the network architecture and the stopping point of its iterative process. Optimizing these parameters for a new image requires time, restricting DIP application in domains where many images need to be processed. Focusing on fluorescence microscopy data, we hypothesize that similar images share comparable optimal parameter configurations for DIP-based denoising, potentially enabling optimization-free DIP for fluorescence microscopy. We generated a calibration (n=110) and validation set (n=55) of semantically different images from an open-source dataset for a network architecture search targeted towards ideal U-net architectures and stopping points. The calibration set represented our transfer basis. The validation set enabled the assessment of which image similarity criterion yields the best results. We then implemented AUTO-DIP, a pipeline for automatic parameter transfer, and compared it to the originally published DIP configuration (baseline) and a state-of-the-art image-specific variational denoising approach. We show that a parameter transfer from the calibration dataset to a test image based on only image metadata similarity (e.g., microscope type, imaged specimen) leads to similar and better performance than a transfer based on quantitative image similarity measures. AUTO-DIP outperforms the baseline DIP (DIP with original DIP parameters) as well as the variational denoising approaches for several open-source test datasets of varying complexity, particularly for very noisy inputs. Applications to locally acquired fluorescence microscopy images further proved superiority of AUTO-DIP.

Method: Proposes LSMRL with three modules: 1) STFL module for parameter-efficient spatial-temporal modeling using CLIP with minimal modifications, 2) SD module to diffuse modality-shared language prompts into visible/infrared features for preliminary modal consistency, and 3) CMI module using bidirectional cross-modal self-attention to eliminate residual modality gaps. Also introduces two modality-level losses for better discriminative ability and generalization.

Result: Extensive experiments on large-scale VVI-ReID datasets demonstrate superiority over all other tested methods (AOTA methods).

Conclusion: LSMRL effectively addresses limitations of existing methods through efficient spatial-temporal modeling, enhanced cross-modal interaction, and explicit modality-level loss guidance, achieving state-of-the-art performance in video-based visible-infrared person re-identification.

Abstract: The core of video-based visible-infrared person re-identification (VVI-ReID) lies in learning sequence-level modal-invariant representations across different modalities. Recent research tends to use modality-shared language prompts generated by CLIP to guide the learning of modal-invariant representations. Despite achieving optimal performance, such methods still face limitations in efficient spatial-temporal modeling, sufficient cross-modal interaction, and explicit modality-level loss guidance. To address these issues, we propose the language-driven sequence-level modal-invariant representation learning (LSMRL) method, which includes spatial-temporal feature learning (STFL) module, semantic diffusion (SD) module and cross-modal interaction (CMI) module. To enable parameter- and computation-efficient spatial-temporal modeling, the STFL module is built upon CLIP with minimal modifications. To achieve sufficient cross-modal interaction and enhance the learning of modal-invariant features, the SD module is proposed to diffuse modality-shared language prompts into visible and infrared features to establish preliminary modal consistency. The CMI module is further developed to leverage bidirectional cross-modal self-attention to eliminate residual modality gaps and refine modal-invariant representations. To explicitly enhance the learning of modal-invariant representations, two modality-level losses are introduced to improve the features’ discriminative ability and their generalization to unseen categories. Extensive experiments on large-scale VVI-ReID datasets demonstrate the superiority of LSMRL over AOTA methods.

Method: Proposes stochastic bridge model for video-to-video translation, establishing direct path from source video (with objects) to target video (objects removed). Uses adaptive mask modulation to dynamically modulate input embeddings based on mask characteristics, balancing background fidelity with generative flexibility.

Result: Extensive experiments show significant outperformance over existing methods in both visual quality and temporal consistency.

Conclusion: Reformulating video object removal as video-to-video translation with stochastic bridge model effectively leverages input video as structural prior, enabling precise removal while maintaining logical consistency with surrounding environment.

Abstract: Existing video object removal methods predominantly rely on diffusion models following a noise-to-data paradigm, where generation starts from uninformative Gaussian noise. This approach discards the rich structural and contextual priors present in the original input video. Consequently, such methods often lack sufficient guidance, leading to incomplete object erasure or the synthesis of implausible content that conflicts with the scene’s physical logic. In this paper, we reformulate video object removal as a video-to-video translation task via a stochastic bridge model. Unlike noise-initialized methods, our framework establishes a direct stochastic path from the source video (with objects) to the target video (objects removed). This bridge formulation effectively leverages the input video as a strong structural prior, guiding the model to perform precise removal while ensuring that the filled regions are logically consistent with the surrounding environment. To address the trade-off where strong bridge priors hinder the removal of large objects, we propose a novel adaptive mask modulation strategy. This mechanism dynamically modulates input embeddings based on mask characteristics, balancing background fidelity with generative flexibility. Extensive experiments demonstrate that our approach significantly outperforms existing methods in both visual quality and temporal consistency.

Method: ARMARecon integrates Autoregressive Moving Average (ARMA) graph filtering with a reconstruction-driven objective to enhance feature representation. It models both local and global connectivity using 20-bin Fractional Anisotropy (FA) histogram features from white-matter regions while mitigating over-smoothing.

Result: ARMARecon achieves superior performance compared to state-of-the-art methods on multi-site dMRI datasets ADNI and NIFD.

Conclusion: The proposed ARMARecon framework effectively captures disease propagation patterns in white-matter regions and improves classification accuracy for neurodegenerative disease detection.

Abstract: Early detection of neurodegenerative diseases such as Alzheimer’s Disease (AD) and Frontotemporal Dementia (FTD) is essential for reducing the risk of progression to severe disease stages. As AD and FTD propagate along white-matter regions in a global, graph-dependent manner, graph-based neural networks are well suited to capture these patterns. Hence, we introduce ARMARecon, a unified graph learning framework that integrates Autoregressive Moving Average (ARMA) graph filtering with a reconstruction-driven objective to enhance feature representation and improve classification accuracy. ARMARecon effectively models both local and global connectivity by leveraging 20-bin Fractional Anisotropy (FA) histogram features extracted from white-matter regions, while mitigating over-smoothing. Overall, ARMARecon achieves superior performance compared to state-of-the-art methods on the multi-site dMRI datasets ADNI and NIFD.

Method: Two main contributions: 1) RS-RVOS Bench - first large-scale benchmark with 111 videos, 25K frames, 213K annotations using causality-aware annotation strategy; 2) MQC-SAM framework with temporal motion consistency module for initial memory calibration using motion trajectory priors, and decoupled attention-based memory integration with dynamic quality assessment to selectively update high-confidence features while filtering unreliable information.

Result: Extensive experiments on RS-RVOS Bench demonstrate that MQC-SAM achieves state-of-the-art performance in remote sensing video referring object segmentation.

Conclusion: The paper advances RS-RVOS research through both data (RS-RVOS Bench benchmark) and methodology (MQC-SAM framework) contributions, effectively addressing key challenges in the field and establishing new state-of-the-art performance.

Abstract: Remote sensing video referring object segmentation (RS-RVOS) is challenged by weak target saliency and severe visual information truncation in dynamic scenes, making it extremely difficult to maintain discriminative target representations during segmentation. Moreover, progress in this field is hindered by the absence of large-scale dedicated benchmarks, while existing models are often affected by biased initial memory construction that impairs accurate instance localization in complex scenarios, as well as indiscriminate memory accumulation that encodes noise from occlusions or misclassifications, leading to persistent error propagation. This paper advances RS-RVOS research through dual contributions in data and methodology. First, we construct RS-RVOS Bench, the first large-scale benchmark comprising 111 video sequences, about 25,000 frames, and 213,000 temporal referring annotations. Unlike common RVOS benchmarks where many expressions are written with access to the full video context, our dataset adopts a strict causality-aware annotation strategy in which linguistic references are generated solely from the target state in the initial frame. Second, we propose a memory-quality-aware online referring segmentation framework, termed Memory Quality Control with Segment Anything Model (MQC-SAM). MQC-SAM introduces a temporal motion consistency module for initial memory calibration, leveraging short-term motion trajectory priors to correct structural deviations and establish accurate memory anchoring. Furthermore, it incorporates a decoupled attention-based memory integration mechanism with dynamic quality assessment, selectively updating high-confidence semantic features while filtering unreliable information, thereby effectively preventing error accumulation and propagation. Extensive experiments on RS-RVOS Bench demonstrate that MQC-SAM achieves state-of-the-art performance.

Method: 1) Construct EmoLat emotion latent space with emotion semantic graph capturing relations among emotions, objects, and visual attributes. 2) Use adversarial regularization to align latent emotion distributions across modalities. 3) Build cross-modal sentiment transfer framework with joint embedding of text and EmoLat features. 4) Optimize with multi-objective loss (semantic consistency, emotion alignment, adversarial regularization). 5) Create EmoSpace Set benchmark dataset with dense emotion, object, and attribute annotations.

Result: Significantly outperforms existing state-of-the-art methods on EmoSpace Set in both quantitative metrics and qualitative transfer fidelity. Establishes new paradigm for controllable image sentiment editing guided by textual input.

Conclusion: EmoLat enables fine-grained, text-driven image sentiment transfer through effective modeling of cross-modal emotion correlations. The approach demonstrates superior performance and introduces a valuable benchmark dataset for future research in emotion-aware image editing.

Abstract: We propose EmoLat, a novel emotion latent space that enables fine-grained, text-driven image sentiment transfer by modeling cross-modal correlations between textual semantics and visual emotion features. Within EmoLat, an emotion semantic graph is constructed to capture the relational structure among emotions, objects, and visual attributes. To enhance the discriminability and transferability of emotion representations, we employ adversarial regularization, aligning the latent emotion distributions across modalities. Building upon EmoLat, a cross-modal sentiment transfer framework is proposed to manipulate image sentiment via joint embedding of text and EmoLat features. The network is optimized using a multi-objective loss incorporating semantic consistency, emotion alignment, and adversarial regularization. To support effective modeling, we construct EmoSpace Set, a large-scale benchmark dataset comprising images with dense annotations on emotions, object semantics, and visual attributes. Extensive experiments on EmoSpace Set demonstrate that our approach significantly outperforms existing state-of-the-art methods in both quantitative metrics and qualitative transfer fidelity, establishing a new paradigm for controllable image sentiment editing guided by textual input. The EmoSpace Set and all the code are available at http://github.com/JingVIPLab/EmoLat.

Method: 1) Depth-Aware Distillation: Transfer depth-related knowledge for better foreground representation. 2) Domain-invariant learning: Treat synthetic data processing as domain adaptation problem to focus on foreground learning. 3) Object-Oriented Decoder: Receives both visual and language prompts to predict referring targets for interactive prediction.

Result: Experimental results show the method quantitatively and qualitatively outperforms state-of-the-art methods.

Conclusion: FCLM effectively addresses foreground consistency, data scarcity, and interactive prediction challenges in high-precision scene parsing through its integrated approach of depth-aware knowledge transfer, domain adaptation, and multi-modal prompting.

Abstract: High-precision scene parsing tasks, including image matting and dichotomous segmentation, aim to accurately predict masks with extremely fine details (such as hair). Most existing methods focus on salient, single foreground objects. While interactive methods allow for target adjustment, their class-agnostic design restricts generalization across different categories. Furthermore, the scarcity of high-quality annotation has led to a reliance on inharmonious synthetic data, resulting in poor generalization to real-world scenarios. To this end, we propose a Foreground Consistent Learning model, dubbed as FCLM, to address the aforementioned issues. Specifically, we first introduce a Depth-Aware Distillation strategy where we transfer the depth-related knowledge for better foreground representation. Considering the data dilemma, we term the processing of synthetic data as domain adaptation problem where we propose a domain-invariant learning strategy to focus on foreground learning. To support interactive prediction, we contribute an Object-Oriented Decoder that can receive both visual and language prompts to predict the referring target. Experimental results show that our method quantitatively and qualitatively outperforms SOTA methods.

Method: Proposes HistoCRF, a CRF-based framework with a novel pairwise potential definition that promotes label diversity and leverages expert annotations. The method adapts Conditional Random Fields to histopathological applications without additional training, working in three modes: without annotations, with expert annotations, and with iterative human-in-the-loop annotations.

Result: Experiments on five patch-level classification datasets show average accuracy gains of 16.0% without annotations, 27.5% with only 100 annotations, and 32.6% with human-in-the-loop annotations compared to zero-shot VLM predictions.

Conclusion: HistoCRF effectively refines VLM predictions for histopathology analysis, achieving significant accuracy improvements with minimal expert input, demonstrating practical value for clinical applications where labeled data is scarce.

Abstract: Assisting pathologists in the analysis of histopathological images has high clinical value, as it supports cancer detection and staging. In this context, histology foundation models have recently emerged. Among them, Vision-Language Models (VLMs) provide strong yet imperfect zero-shot predictions. We propose to refine these predictions by adapting Conditional Random Fields (CRFs) to histopathological applications, requiring no additional model training. We present HistoCRF, a CRF-based framework, with a novel definition of the pairwise potential that promotes label diversity and leverages expert annotations. We consider three experiments: without annotations, with expert annotations, and with iterative human-in-the-loop annotations that progressively correct misclassified patches. Experiments on five patch-level classification datasets covering different organs and diseases demonstrate average accuracy gains of 16.0% without annotations and 27.5% with only 100 annotations, compared to zero-shot predictions. Moreover, integrating a human in the loop reaches a further gain of 32.6% with the same number of annotations. The code will be made available on https://github.com/tgodelaine/HistoCRF.

Method: The method exploits the cuboid geometry of industrial bins by first detecting intermediate 3D line segments corresponding to their top edges. It extends the 2D line segment detection network LeTR to operate on structured point cloud data. Detected 3D line segments are then processed using a simple geometric procedure to robustly determine the bin’s 6DoF pose.

Result: The method achieves state-of-the-art performance with 3 cm translation error and 8.2° rotation error. Incorporating synthetic training data significantly improves pose estimation accuracy on real scans. The approach outperforms current state-of-the-art methods while not requiring instance-specific CAD models during inference.

Conclusion: The proposed method provides an effective solution for 6DoF pose estimation of industrial bins that works well in data-scarce environments, leverages geometric priors, and eliminates the need for CAD models during inference, making it practical for real-world industrial applications.

Abstract: The task of 6DoF object pose estimation is one of the fundamental problems of 3D vision with many practical applications such as industrial automation. Traditional deep learning approaches for this task often require extensive training data or CAD models, limiting their application in real-world industrial settings where data is scarce and object instances vary. We propose a novel method for 6DoF pose estimation focused specifically on bins used in industrial settings. We exploit the cuboid geometry of bins by first detecting intermediate 3D line segments corresponding to their top edges. Our approach extends the 2D line segment detection network LeTR to operate on structured point cloud data. The detected 3D line segments are then processed using a simple geometric procedure to robustly determine the bin’s 6DoF pose. To evaluate our method, we extend an existing dataset with a newly collected and annotated dataset, which we make publicly available. We show that incorporating synthetic training data significantly improves pose estimation accuracy on real scans. Moreover, we show that our method significantly outperforms current state-of-the-art 6DoF pose estimation methods in terms of the pose accuracy (3 cm translation error, 8.2$^\circ$ rotation error) while not requiring instance-specific CAD models during inference.

Method: Knowledge distillation from high-capacity CNNs trained in data centers to compact CNNs via ensemble learning. Integration of dense tiny pairs through simple and optimized ensembling to enhance accuracy while maintaining computational and energy constraints.

Result: Distilled tiny ensembles achieved competitive accuracy with prior work while significantly reducing energy consumption and carbon footprint on a curated coffee leaf dataset.

Conclusion: Lightweight models, when properly distilled and ensembled, can provide practical diagnostic solutions for IoT applications in agriculture, enabling sustainable on-device disease diagnosis.

Abstract: Coffee yields are contingent on the timely and accurate diagnosis of diseases; however, assessing leaf diseases in the field presents significant challenges. Although Artificial Intelligence (AI) vision models achieve high accuracy, their adoption is hindered by the limitations of constrained devices and intermittent connectivity. This study aims to facilitate sustainable on-device diagnosis through knowledge distillation: high-capacity Convolutional Neural Networks (CNNs) trained in data centers transfer knowledge to compact CNNs through Ensemble Learning (EL). Furthermore, dense tiny pairs were integrated through simple and optimized ensembling to enhance accuracy while adhering to strict computational and energy constraints. On a curated coffee leaf dataset, distilled tiny ensembles achieved competitive with prior work with significantly reduced energy consumption and carbon footprint. This indicates that lightweight models, when properly distilled and ensembled, can provide practical diagnostic solutions for Internet of Things (IoT) applications.

Method: Proposes Real-Centered Detection Network (RCDN) with Xception backbone, using frequency-spatial CNN framework that anchors representation space around authentic facial images. Employs dual-branch architecture and real-centered loss design to focus on consistency of real images rather than modeling diverse forgery patterns.

Result: Extensive experiments on DiFF dataset (FE, I2I, T2I forgery types) show RCDN achieves state-of-the-art in-domain accuracy and significantly stronger cross-domain generalization. Reduces generalization gap compared to baselines and achieves highest cross/in-domain stability ratio.

Conclusion: RCDN demonstrates potential as a practical solution for defending against evolving and unseen image forgery techniques by focusing on real image consistency rather than modeling diverse forgery patterns, enabling better cross-domain generalization.

Abstract: Image forgery has become a critical threat with the rapid proliferation of AI-based generation tools, which make it increasingly easy to synthesize realistic but fraudulent facial content. Existing detection methods achieve near-perfect performance when training and testing are conducted within the same domain, yet their effectiveness deteriorates substantially in crossdomain scenarios. This limitation is problematic, as new forgery techniques continuously emerge and detectors must remain reliable against unseen manipulations. To address this challenge, we propose the Real-Centered Detection Network (RCDN), a frequency spatial convolutional neural networks(CNN) framework with an Xception backbone that anchors its representation space around authentic facial images. Instead of modeling the diverse and evolving patterns of forgeries, RCDN emphasizes the consistency of real images, leveraging a dual-branch architecture and a real centered loss design to enhance robustness under distribution shifts. Extensive experiments on the DiFF dataset, focusing on three representative forgery types (FE, I2I, T2I), demonstrate that RCDN achieves both state-of-the-art in-domain accuracy and significantly stronger cross-domain generalization. Notably, RCDN reduces the generalization gap compared to leading baselines and achieves the highest cross/in-domain stability ratio, highlighting its potential as a practical solution for defending against evolving and unseen image forgery techniques.

Method: Created CARLA-Round dataset using systematic simulation design with 25 controlled scenarios (5 weather conditions × 5 traffic density levels). The dataset includes realistic driving behavior mixtures and explicit annotations, enabling precise analysis of how different conditions influence prediction performance.

Result: Validation experiments show traffic density dominates prediction difficulty with strong monotonic effects, while weather shows non-linear impacts. The best model achieves 0.312m ADE on real-world rounD dataset, demonstrating effective sim-to-real transfer.

Conclusion: CARLA-Round provides a systematically designed simulation dataset that enables quantification of factor impacts impossible to isolate in confounded real-world datasets, advancing roundabout trajectory prediction research with controlled experimental conditions.

Abstract: Accurate trajectory prediction of vehicles at roundabouts is critical for reducing traffic accidents, yet it remains highly challenging due to their circular road geometry, continuous merging and yielding interactions, and absence of traffic signals. Developing accurate prediction algorithms relies on reliable, multimodal, and realistic datasets; however, such datasets for roundabout scenarios are scarce, as real-world data collection is often limited by incomplete observations and entangled factors that are difficult to isolate. We present CARLA-Round, a systematically designed simulation dataset for roundabout trajectory prediction. The dataset varies weather conditions (five types) and traffic density levels (spanning Level-of-Service A-E) in a structured manner, resulting in 25 controlled scenarios. Each scenario incorporates realistic mixtures of driving behaviors and provides explicit annotations that are largely absent from existing datasets. Unlike randomly sampled simulation data, this structured design enables precise analysis of how different conditions influence trajectory prediction performance. Validation experiments using standard baselines (LSTM, GCN, GRU+GCN) reveal traffic density dominates prediction difficulty with strong monotonic effects, while weather shows non-linear impacts. The best model achieves 0.312m ADE on real-world rounD dataset, demonstrating effective sim-to-real transfer. This systematic approach quantifies factor impacts impossible to isolate in confounded real-world datasets. Our CARLA-Round dataset is available at https://github.com/Rebecca689/CARLA-Round.

Method: Introduces SAMA with Multi-View Localization Encoder (MVLE) for detailed local features, Localization Adapter for boundary refinement, and dual prediction heads for segmentation and matting tasks.

Result: Achieves state-of-the-art performance across multiple segmentation and matting benchmarks, demonstrating adaptability and effectiveness in diverse downstream tasks.

Conclusion: SAMA successfully unifies segmentation and matting in a lightweight extension of SAM, delivering high-quality interactive results with minimal parameter overhead.

Abstract: Segment Anything (SAM) has recently pushed the boundaries of segmentation by demonstrating zero-shot generalization and flexible prompting after training on over one billion masks. Despite this, its mask prediction accuracy often falls short of the precision required in real-world applications. While several refinement modules have been proposed to boost SAM’s segmentation quality, achieving highly accurate object delineation within a single, unified framework remains an open challenge. Furthermore, interactive image matting, which aims to generate fine-grained alpha mattes guided by diverse user hints, has not yet been explored in the context of SAM. Insights from recent studies highlight strong correlations between segmentation and matting, suggesting the feasibility of a unified model capable of both tasks. In this paper, we introduce Segment And Matte Anything (SAMA), a lightweight extension of SAM that delivers high-quality interactive image segmentation and matting with minimal extra parameters. Our Multi-View Localization Encoder (MVLE) captures detailed features from local views, while the Localization Adapter (Local-Adapter) refines mask outputs by recovering subtle boundary details. We also incorporate two prediction heads for each task into the architecture to generate segmentation and matting masks, simultaneously. Trained on a diverse dataset aggregated from publicly available sources, SAMA achieves state-of-the-art performance across multiple segmentation and matting benchmarks, showcasing its adaptability and effectiveness in a wide range of downstream tasks.

Method: Proposes THz-SSDD network using: 1) Recorrupted-to-Recorrupted self-supervised learning to capture noise features through invariance under repeated corruption, 2) PCA decomposition and reconstruction to restore images across both low and high frequencies.

Result: Evaluated on four sample types, showing effective denoising and deblurring across different material properties and measurement modes. Quantitative analysis validates improved image quality while preserving original signal characteristics. Training requires only small set of unlabeled noisy images.

Conclusion: THz-SSDD provides an effective self-supervised solution for simultaneous denoising and deblurring of THz images, overcoming limitations of conventional methods and reducing need for manual intervention.

Abstract: Terahertz (THz) systems inherently introduce frequency-dependent degradation effects, resulting in low-frequency blurring and high-frequency noise in amplitude images. Conventional image processing techniques cannot simultaneously address both issues, and manual intervention is often required due to the unknown boundary between denoising and deblurring. To tackle this challenge, we propose a principal component analysis (PCA)-based THz self-supervised denoising and deblurring network (THz-SSDD). The network employs a Recorrupted-to-Recorrupted self-supervised learning strategy to capture the intrinsic features of noise by exploiting invariance under repeated corruption. PCA decomposition and reconstruction are then applied to restore images across both low and high frequencies. The performance of the THz-SSDD network was evaluated on four types of samples. Training requires only a small set of unlabeled noisy images, and testing across samples with different material properties and measurement modes demonstrates effective denoising and deblurring. Quantitative analysis further validates the network feasibility, showing improvements in image quality while preserving the physical characteristics of the original signals.

Method: Proposes an efficient inference strategy that sparsifies attention using spatially aware neighboring blocks and filters out non-informative tokens through global attention scores, reducing GPU memory and runtime while preserving performance.

Result: Achieves up to 7.67% improvement in ROI classification and compatible results in segmentation, enabling inference at higher resolutions under the same GPU budget with substantially reduced memory and runtime.

Conclusion: The method overcomes limitations of pathology foundation models by enabling efficient high-resolution whole-slide image inference while preserving and even improving downstream performance, making practical deployment more feasible.

Abstract: Despite their prominent performance on tasks such as ROI classification and segmentation, many pathology foundation models remain constrained by a specific input size e.g. 224 x 224, creating substantial inefficiencies when applied to whole-slide images (WSIs), which span thousands of resolutions. A naive strategy is to either enlarge inputs or downsample the WSIs. However, enlarging inputs results in prohibitive GPU memory consumption, while downsampling alters the microns-per-pixel resolution and obscures critical morphological details. To overcome these limitations, we propose an space- and time- efficient inference strategy that sparsifies attention using spatially aware neighboring blocks and filters out non-informative tokens through global attention scores. This design substantially reduces GPU memory and runtime during high-resolution WSI inference while preserving and even improving the downstream performance, enabling inference at higher resolutions under the same GPU budget. The experimental results show that our method can achieves up to an 7.67% improvement in the ROI classification and compatible results in segmentation.

Method: Collaborative inverse rendering combines photometric consistency from multi-view images with persistent homology priors that capture topological features like tunnel loops and handle loops. Uses gradient-based optimization within a mesh-based framework rather than neural networks.

Result: Method achieves lower Chamfer Distance and higher Volume IoU compared to state-of-the-art mesh-based methods, demonstrating improved geometric accuracy and robustness against topological failures.

Conclusion: Incorporating persistent homology priors effectively resolves reconstruction ambiguities for complex high-genus surfaces, providing a robust solution that prevents catastrophic topological failures.

Abstract: Reconstructing 3D objects from images is inherently an ill-posed problem due to ambiguities in geometry, appearance, and topology. This paper introduces collaborative inverse rendering with persistent homology priors, a novel strategy that leverages topological constraints to resolve these ambiguities. By incorporating priors that capture critical features such as tunnel loops and handle loops, our approach directly addresses the difficulty of reconstructing high-genus surfaces. The collaboration between photometric consistency from multi-view images and homology-based guidance enables recovery of complex high-genus geometry while circumventing catastrophic failures such as collapsing tunnels or losing high-genus structure. Instead of neural networks, our method relies on gradient-based optimization within a mesh-based inverse rendering framework to highlight the role of topological priors. Experimental results show that incorporating persistent homology priors leads to lower Chamfer Distance (CD) and higher Volume IoU compared to state-of-the-art mesh-based methods, demonstrating improved geometric accuracy and robustness against topological failure.

Method: Uses a shared MLLM backbone with contrastive alignment of visual and textual embeddings, trained efficiently with LoRA on 700K paired samples. Supports embedding-based candidate search and can be adapted for reranking without additional training for moment retrieval.

Result: Surpasses other MLLM-based methods on zero-shot video retrieval, achieves competitive results on zero-shot moment retrieval, state-of-the-art on zero-shot composed video retrieval, and with reranking matches specialized models trained on much larger datasets.

Conclusion: VIRTUE demonstrates that a single MLLM-based architecture can effectively unify diverse video retrieval tasks while achieving performance comparable to specialized systems, bridging the gap between multimodal query support and retrieval effectiveness.

Abstract: Modern video retrieval systems are expected to handle diverse tasks ranging from corpus-level retrieval and fine-grained moment localization to flexible multimodal querying. Specialized architectures achieve strong retrieval performance by training modality-specific encoders on massive datasets, but they lack the ability to process composed multimodal queries. In contrast, multimodal LLM (MLLM)-based methods support rich multimodal search but their retrieval performance remains well below that of specialized systems. We present VIRTUE, an MLLM-based versatile video retrieval framework that integrates corpus and moment-level retrieval capabilities while accommodating composed multimodal queries within a single architecture. We use contrastive alignment of visual and textual embeddings generated using a shared MLLM backbone to facilitate efficient embedding-based candidate search. Our embedding model, trained efficiently using low-rank adaptation (LoRA) on 700K paired visual-text data samples, surpasses other MLLM-based methods on zero-shot video retrieval tasks. Additionally, we demonstrate that the same model can be adapted without further training to achieve competitive results on zero-shot moment retrieval, and state of the art results for zero-shot composed video retrieval. With additional training for reranking candidates identified in the embedding-based search, our model substantially outperforms existing MLLM-based retrieval systems and achieves retrieval performance comparable to state of the art specialized models which are trained on orders of magnitude larger data.

Method: SurgRef uses a motion-guided framework that grounds free-form language expressions in instrument motion, capturing how tools move and interact across time. The method is trained on Ref-IMotion, a diverse multi-institutional video dataset with dense spatiotemporal masks and motion-centric expressions.

Result: SurgRef achieves state-of-the-art accuracy and generalization across surgical procedures, setting a new benchmark for robust, language-driven surgical video segmentation, particularly effective under occlusion, ambiguity, or unfamiliar terminology.

Conclusion: The motion-guided approach enables more intuitive language-driven interaction with surgical scenes, representing a critical step toward intelligent operating rooms and autonomous surgical robotic assistance.

Abstract: Enabling intuitive, language-driven interaction with surgical scenes is a critical step toward intelligent operating rooms and autonomous surgical robotic assistance. However, the task of referring segmentation, localizing surgical instruments based on natural language descriptions, remains underexplored in surgical videos, with existing approaches struggling to generalize due to reliance on static visual cues and predefined instrument names. In this work, we introduce SurgRef, a novel motion-guided framework that grounds free-form language expressions in instrument motion, capturing how tools move and interact across time, rather than what they look like. This allows models to understand and segment instruments even under occlusion, ambiguity, or unfamiliar terminology. To train and evaluate SurgRef, we present Ref-IMotion, a diverse, multi-institutional video dataset with dense spatiotemporal masks and rich motion-centric expressions. SurgRef achieves state-of-the-art accuracy and generalization across surgical procedures, setting a new benchmark for robust, language-driven surgical video segmentation.

Method: Uses diffusion model to detect artifacts as outliers among clean images. Requires only clean images for training, no pixel-level annotations. Enhanced with contrastive learning module to explicitly enlarge distribution separation between artifact and clean images.

Result: Superior performance to state-of-the-art methods, offers cross-stain generalization capacity, requires significantly less data and annotations.

Conclusion: DiffusionQC provides an effective, annotation-efficient solution for artifact detection in digital pathology with strong generalization capabilities.

Abstract: Digital pathology plays a vital role across modern medicine, offering critical insights for disease diagnosis, prognosis, and treatment. However, histopathology images often contain artifacts introduced during slide preparation and digitization. Detecting and excluding them is essential to ensure reliable downstream analysis. Traditional supervised models typically require large annotated datasets, which is resource-intensive and not generalizable to novel artifact types. To address this, we propose DiffusionQC, which detects artifacts as outliers among clean images using a diffusion model. It requires only a set of clean images for training rather than pixel-level artifact annotations and predefined artifact types. Furthermore, we introduce a contrastive learning module to explicitly enlarge the distribution separation between artifact and clean images, yielding an enhanced version of our method. Empirical results demonstrate superior performance to state-of-the-art and offer cross-stain generalization capacity, with significantly less data and annotations.

Method: PRISM is a three-stage framework: 1) Adaptive visual sampling to reduce frame count, 2) Label-driven keyframe anchoring to identify meaningful procedural transitions, and 3) Contextual validation using a large language model (LLM) to filter out generic or hallucinated content and ensure contextual coherence.

Result: The method achieves strong performance on instructional and activity datasets, retaining 84% of semantic content while sampling fewer than 5% of original frames. It improves over baselines by up to 33% and generalizes well across procedural and domain-specific video tasks with strong semantic alignment and precision.

Conclusion: PRISM provides an effective framework for semantically grounded video summarization that captures meaningful procedural transitions while filtering irrelevant content. The approach demonstrates strong generalization across domains and achieves significant improvements over existing baselines with minimal frame sampling.

Abstract: Video summarization helps turn long videos into clear, concise representations that are easier to review, document, and analyze, especially in high-stakes domains like surgical training. Prior work has progressed from using basic visual features like color, motion, and structural changes to using pre-trained vision-language models that can better understand what’s happening in the video (semantics) and capture temporal flow, resulting in more context-aware video summarization. We propose a three-stage framework, PRISM: Procedural Representation via Integrated Semantic and Multimodal analysis, that produces semantically grounded video summaries. PRISM combines adaptive visual sampling, label-driven keyframe anchoring, and contextual validation using a large language model (LLM). Our method ensures that selected frames reflect meaningful and procedural transitions while filtering out generic or hallucinated content, resulting in contextually coherent summaries across both domain-specific and instructional videos. We evaluate our method on instructional and activity datasets, using reference summaries for instructional videos. Despite sampling fewer than 5% of the original frames, our summaries retain 84% semantic content while improving over baselines by as much as 33%. Our approach generalizes across procedural and domain-specific video tasks, achieving strong performance with both semantic alignment and precision.

Method: Proposes an innovative framework integrating Pyramid Adaptive Atrous Convolution (PAAC) and Transformer architectures. Uses Multi-Scale Feature Fusion to enhance feature extraction from benign and malignant tissues. Combines Dice Loss and Focal Loss functions to improve model learning. Trained on comprehensive dataset from INbreast, MIAS, and DDSM with preprocessing including data augmentation, contrast enhancement, and resizing to 227x227 pixels.

Result: Achieved state-of-the-art performance with accuracy of 98.5%, sensitivity of 97.8%, specificity of 96.3%, F1-score of 98.2%, and precision of 97.9%. Outperformed foundational models including BreastNet, DeepMammo, Multi-Scale CNN, Swin-Unet, and SegFormer.

Conclusion: The proposed model demonstrates significant improvement over traditional methods and confirms effectiveness in identifying cancerous masses in complex scenarios and large datasets. Shows potential as a reliable and efficient tool for breast cancer diagnosis that can be integrated into medical diagnostic systems.

Abstract: Breast cancer is one of the most common cancers among women worldwide, and its accurate and timely diagnosis plays a critical role in improving treatment outcomes. This thesis presents an innovative framework for detecting malignant masses in mammographic images by integrating the Pyramid Adaptive Atrous Convolution (PAAC) and Transformer architectures. The proposed approach utilizes Multi-Scale Feature Fusion to enhance the extraction of features from benign and malignant tissues and combines Dice Loss and Focal Loss functions to improve the model’s learning process, effectively reducing errors in binary breast cancer classification and achieving high accuracy and efficiency. In this study, a comprehensive dataset of breast cancer images from INbreast, MIAS, and DDSM was preprocessed through data augmentation and contrast enhancement and resized to 227x227 pixels for model training. Leveraging the Transformer’s ability to manage long-range dependencies with Self-Attention mechanisms, the proposed model achieved high accuracy in detecting cancerous masses, outperforming foundational models such as BreastNet, DeepMammo, Multi-Scale CNN, Swin-Unet, and SegFormer. The final evaluation results for the proposed model include an accuracy of 98.5%, sensitivity of 97.8%, specificity of 96.3%, F1-score of 98.2%, and overall precision of 97.9%. These metrics demonstrate a significant improvement over traditional methods and confirm the model’s effectiveness in identifying cancerous masses in complex scenarios and large datasets. This model shows potential as a reliable and efficient tool for breast cancer diagnosis and can be effectively integrated into medical diagnostic systems.

Method: FedDCG introduces a domain grouping strategy where class-generalized networks are trained within each group to prevent decision boundary confusion. It uses a learnable network to enhance class generalization and a decoupling mechanism to separate general and domain-specific knowledge. During inference, it aggregates class-generalized results based on domain similarity.

Result: Extensive experiments across various datasets show that FedDCG outperforms state-of-the-art baselines in terms of accuracy and robustness.

Conclusion: FedDCG provides an effective framework for jointly addressing both class and domain generalization in federated learning, demonstrating superior performance compared to existing methods.

Abstract: Efficient fine-tuning of visual-language models like CLIP has become crucial due to their large-scale parameter size and extensive pretraining requirements. Existing methods typically address either the issue of unseen classes or unseen domains in isolation, without considering a joint framework for both. In this paper, we propose \textbf{Fed}erated Joint Learning for \textbf{D}omain and \textbf{C}lass \textbf{G}eneralization, termed \textbf{FedDCG}, a novel approach that addresses both class and domain generalization in federated learning settings. Our method introduces a domain grouping strategy where class-generalized networks are trained within each group to prevent decision boundary confusion. During inference, we aggregate class-generalized results based on domain similarity, effectively integrating knowledge from both class and domain generalization. Specifically, a learnable network is employed to enhance class generalization capabilities, and a decoupling mechanism separates general and domain-specific knowledge, improving generalization to unseen domains. Extensive experiments across various datasets show that \textbf{FedDCG} outperforms state-of-the-art baselines in terms of accuracy and robustness.

Method: Proposed a novel reformulation of light transport model that decomposes hidden scenes into light-occluding and non-light-occluding components, creating a separable non-linear least squares (SNLLS) inverse problem. Developed two solutions: gradient-based optimization and a physics-inspired neural network called Soft Shadow diffusion (SSD).

Result: Successfully demonstrated 3D reconstruction of hidden scenes from ordinary NLOS photographs in real experimental scenarios. SSD shows strong generalization from simulation to unseen classes in both simulation and real-world NLOS scenes, with surprising robustness to noise and ambient illumination.

Conclusion: The approach generalizes passive NLOS methods to full 3D reconstruction, overcoming previous limitations through novel light transport modeling and effective optimization/neural network solutions that handle the challenging ill-conditioned inverse problem.

Abstract: Conventional imaging requires a line of sight to create accurate visual representations of a scene. In certain circumstances, however, obtaining a suitable line of sight may be impractical, dangerous, or even impossible. Non-line-of-sight (NLOS) imaging addresses this challenge by reconstructing the scene from indirect measurements. Recently, passive NLOS methods that use an ordinary photograph of the subtle shadow cast onto a visible wall by the hidden scene have gained interest. These methods are currently limited to 1D or low-resolution 2D color imaging or to localizing a hidden object whose shape is approximately known. Here, we generalize this class of methods and demonstrate a 3D reconstruction of a hidden scene from an ordinary NLOS photograph. To achieve this, we propose a novel reformulation of the light transport model that conveniently decomposes the hidden scene into \textit{light-occluding} and \textit{non-light-occluding} components to yield a separable non-linear least squares (SNLLS) inverse problem. We develop two solutions: A gradient-based optimization method and a physics-inspired neural network approach, which we call Soft Shadow diffusion (SSD). Despite the challenging ill-conditioned inverse problem encountered here, our approaches are effective on numerous 3D scenes in real experimental scenarios. Moreover, SSD is trained in simulation but generalizes well to unseen classes in simulation and real-world NLOS scenes. SSD also shows surprising robustness to noise and ambient illumination.

Method: A framework with three specialized LLM agents: Profiling Agent analyzes model architecture and MAC distributions; Master Agent orchestrates workflow with divergence monitoring; Analysis Agent (Claude 3.5 Sonnet) learns optimal strategies from historical attempts via in-context learning. Builds on isomorphic pruning with context-aware adaptation across pruning iterations.

Result: On ImageNet-1K: ResNet-50 achieves 1.77G MACs with 77.04% accuracy (+0.91% vs baseline); ResNet-101 achieves 4.22G MACs with 78.94% accuracy (+1.56% vs baseline). ConvNeXt-Small gets 8.17G MACs with 1.41x GPU and 1.07x CPU speedup, 45% parameter reduction. Vision Transformers achieve MAC-budget compliance within tolerance bands (+1% to +5% overshoot, -5% to -15% undershoot).

Conclusion: AgenticPruner enables automatic convergence to target MAC budgets within user-defined tolerance bands, establishing feasibility for deployment scenarios requiring strict computational guarantees while maintaining or improving model accuracy.

Abstract: Neural network pruning remains essential for deploying deep learning models on resource-constrained devices, yet existing approaches primarily target parameter reduction without directly controlling computational cost. This yields unpredictable inference latency in deployment scenarios where strict Multiply-Accumulate (MAC) operation budgets must be met. We propose AgenticPruner, a framework utilizing large language models to achieve MAC-constrained optimization through iterative strategy learning. Our approach coordinates three specialized agents: a Profiling Agent that analyzes model architecture and MAC distributions, a Master Agent that orchestrates the workflow with divergence monitoring, and an Analysis Agent powered by Claude 3.5 Sonnet that learns optimal strategies from historical attempts. Through in-context learning, the Analysis Agent improves convergence success rate from 48% to 71% compared to grid search. Building upon isomorphic pruning’s graph-based structural grouping, our method adds context-aware adaptation by analyzing patterns across pruning iterations, enabling automatic convergence to target MAC budgets within user-defined tolerance bands. We validate our framework on ImageNet-1K across ResNet, ConvNeXt, and DeiT architectures. On CNNs, our approach achieves MAC targeting while maintaining or improving accuracy: ResNet-50 reaches 1.77G MACs with 77.04% accuracy (+0.91% vs baseline); ResNet-101 achieves 4.22G MACs with 78.94% accuracy (+1.56% vs baseline). For ConvNeXt-Small, pruning to 8.17G MACs yields 1.41x GPU and 1.07x CPU speedup with 45% parameter reduction. On Vision Transformers, we demonstrate MAC-budget compliance within user-defined tolerance bands (typically +1% to +5% overshoot, -5% to -15% undershoot), establishing feasibility for deployment scenarios requiring strict computational guarantees.

Method: Developed CytoCLIP, a suite of vision-language models based on pre-trained CLIP frameworks. Two variants: one trained on low-resolution whole-region images for overall patterns, another on high-resolution image tiles for cellular details. Trained on NISSL-stained fetal brain sections with 86 regions (low-res) and 384 regions (high-res).

Result: CytoCLIP outperforms existing methods, achieving F1 scores of 0.87 for whole-region classification and 0.91 for high-resolution tile classification. Shows strong generalization across different ages and sectioning planes.

Conclusion: CytoCLIP provides an effective automated solution for brain cytoarchitecture analysis, reducing reliance on human experts while maintaining high accuracy in region identification.

Abstract: The functions of different regions of the human brain are closely linked to their distinct cytoarchitecture, which is defined by the spatial arrangement and morphology of the cells. Identifying brain regions by their cytoarchitecture enables various scientific analyses of the brain. However, delineating these areas manually in brain histological sections is time-consuming and requires specialized knowledge. An automated approach is necessary to minimize the effort needed from human experts. To address this, we propose CytoCLIP, a suite of vision-language models derived from pre-trained Contrastive Language-Image Pre-Training (CLIP) frameworks to learn joint visual-text representations of brain cytoarchitecture. CytoCLIP comprises two model variants: one is trained using low-resolution whole-region images to understand the overall cytoarchitectural pattern of an area, and the other is trained on high-resolution image tiles for detailed cellular-level representation. The training dataset is created from NISSL-stained histological sections of developing fetal brains of different gestational weeks. It includes 86 distinct regions for low-resolution images and 384 brain regions for high-resolution tiles. We evaluate the model’s understanding of the cytoarchitecture and generalization ability using region classification and cross-modal retrieval tasks. Multiple experiments are performed under various data setups, including data from samples of different ages and sectioning planes. Experimental results demonstrate that CytoCLIP outperforms existing methods. It achieves an F1 score of 0.87 for whole-region classification and 0.91 for high-resolution image tile classification.

Method: SDiT introduces a training-free framework with three components: (1) semantic-aware clustering using fast Quickshift-based segmentation, (2) complexity-driven regional scheduling to selectively update informative areas, and (3) boundary-aware refinement to maintain spatial coherence.

Result: Without any model retraining or architectural modification, SDiT achieves up to 3.0x acceleration while preserving nearly identical perceptual and semantic quality to full-attention inference.

Conclusion: SDiT demonstrates that adaptive computation based on regional complexity can significantly accelerate Diffusion Transformers while maintaining output quality, offering a practical solution for efficient text-to-image synthesis.

Abstract: Diffusion Transformers (DiTs) achieve state-of-the-art performance in text-to-image synthesis but remain computationally expensive due to the iterative nature of denoising and the quadratic cost of global attention. In this work, we observe that denoising dynamics are spatially non-uniform-background regions converge rapidly while edges and textured areas evolve much more actively. Building on this insight, we propose SDiT, a Semantic Region-Adaptive Diffusion Transformer that allocates computation according to regional complexity. SDiT introduces a training-free framework combining (1) semantic-aware clustering via fast Quickshift-based segmentation, (2) complexity-driven regional scheduling to selectively update informative areas, and (3) boundary-aware refinement to maintain spatial coherence. Without any model retraining or architectural modification, SDiT achieves up to 3.0x acceleration while preserving nearly identical perceptual and semantic quality to full-attention inference.

Method: Leverage radiance fields anchored to parametric face models to learn radiance manifolds in 3D space, extracting explicit layered mesh with appearance and warp textures. Use linear blending and alpha compositing for animation control.

Result: Achieves controllable volumetric rendering of complex facial features (hair, skin, eyes) with efficient streaming and rendering on legacy graphics platforms.

Conclusion: The explicit representation enables photorealistic 3D face avatars to be efficiently streamed and rendered using classical mesh-based rendering without requiring custom engineering.

Abstract: We introduce a novel representation for efficient classical rendering of photorealistic 3D face avatars. Leveraging recent advances in radiance fields anchored to parametric face models, our approach achieves controllable volumetric rendering of complex facial features, including hair, skin, and eyes. At enrollment time, we learn a set of radiance manifolds in 3D space to extract an explicit layered mesh, along with appearance and warp textures. During deployment, this allows us to control and animate the face through simple linear blending and alpha compositing of textures over a static mesh. This explicit representation also enables the generated avatar to be efficiently streamed online and then rendered using classical mesh and shader-based rendering on legacy graphics platforms, eliminating the need for any custom engineering or integration.

Method: 1) Automatically extracts visual concepts from pre-trained encoder; 2) Uses multimodal LLMs to label/filter concepts based on visual identifiability and task relevance; 3) Selects independent concept subset via reconstruction-guided optimization; 4) Leverages CLIP’s visual-text alignment to decompose image representations into linear combination of concept embeddings for CBM framework.

Result: Extensive experiments across 11 image classification tasks show PCBM-ReD achieves state-of-the-art accuracy, narrows performance gap with end-to-end models, and exhibits better interpretability.

Conclusion: PCBM-ReD provides an effective pipeline for retrofitting interpretability onto pretrained opaque models through automatic concept extraction, multimodal LLM-based concept refinement, and representation decomposition using CLIP alignment.

Abstract: Deep learning has achieved remarkable success in image recognition, yet their inherent opacity poses challenges for deployment in critical domains. Concept-based interpretations aim to address this by explaining model reasoning through human-understandable concepts. However, existing post-hoc methods and ante-hoc concept bottleneck models (CBMs), suffer from limitations such as unreliable concept relevance, non-visual or labor-intensive concept definitions, and model or data-agnostic assumptions. This paper introduces Post-hoc Concept Bottleneck Model via Representation Decomposition (PCBM-ReD), a novel pipeline that retrofits interpretability onto pretrained opaque models. PCBM-ReD automatically extracts visual concepts from a pre-trained encoder, employs multimodal large language models (MLLMs) to label and filter concepts based on visual identifiability and task relevance, and selects an independent subset via reconstruction-guided optimization. Leveraging CLIP’s visual-text alignment, it decomposes image representations into linear combination of concept embeddings to fit into the CBMs abstraction. Extensive experiments across 11 image classification tasks show PCBM-ReD achieves state-of-the-art accuracy, narrows the performance gap with end-to-end models, and exhibits better interpretability.

Method: Two-stage globally-diverse attack framework: 1) Textual perturbations via globally-diverse strategy combining candidate text expansion with globally-aware replacement, 2) Visual perturbations using multi-scale resizing and block-shuffle rotation to enhance diversity.

Result: Extensive experiments show 2S-GDA consistently improves attack success rates over state-of-the-art methods, achieving gains up to 11.17% in black-box settings. The framework is modular and can be combined with existing methods.

Conclusion: 2S-GDA effectively addresses diversity limitations in multimodal adversarial attacks, demonstrating superior performance in black-box scenarios while maintaining modularity for integration with existing attack methods.

Abstract: Vision-language pre-training (VLP) models are vulnerable to adversarial examples, particularly in black-box scenarios. Existing multimodal attacks often suffer from limited perturbation diversity and unstable multi-stage pipelines. To address these challenges, we propose 2S-GDA, a two-stage globally-diverse attack framework. The proposed method first introduces textual perturbations through a globally-diverse strategy by combining candidate text expansion with globally-aware replacement. To enhance visual diversity, image-level perturbations are generated using multi-scale resizing and block-shuffle rotation. Extensive experiments on VLP models demonstrate that 2S-GDA consistently improves attack success rates over state-of-the-art methods, with gains of up to 11.17% in black-box settings. Our framework is modular and can be easily combined with existing methods to further enhance adversarial transferability.

Method: Three key innovations: 1) Correlation-guided feature pyramids for capturing scale-invariant patterns, 2) Adaptive Channel Correlation Module (ACCM) for learning dynamic cross-scale relationships, 3) Correlation-guided meta-learning that uses correlation patterns instead of conventional prototype averaging. Trained from scratch with only ~600K parameters.

Result: Achieves up to 86.65% accuracy in 5-way 5-shot classification on multiple remote sensing datasets (EuroSAT, NWPU-RESISC45, UC Merced Land Use, AID). Offers 20× fewer parameters than ResNet-18 while maintaining high efficiency (<50ms per image inference).

Conclusion: AMC-MetaNet establishes a computationally efficient, scale-aware framework for real-world few-shot remote sensing applications, providing a lightweight alternative to heavy pre-trained models while effectively handling multi-scale challenges.

Abstract: Few-shot learning in remote sensing remains challenging due to three factors: the scarcity of labeled data, substantial domain shifts, and the multi-scale nature of geospatial objects. To address these issues, we introduce Adaptive Multi-Scale Correlation Meta-Network (AMC-MetaNet), a lightweight yet powerful framework with three key innovations: (i) correlation-guided feature pyramids for capturing scale-invariant patterns, (ii) an adaptive channel correlation module (ACCM) for learning dynamic cross-scale relationships, and (iii) correlation-guided meta-learning that leverages correlation patterns instead of conventional prototype averaging. Unlike prior approaches that rely on heavy pre-trained models or transformers, AMC-MetaNet is trained from scratch with only $\sim600K$ parameters, offering $20\times$ fewer parameters than ResNet-18 while maintaining high efficiency ($<50$ms per image inference). AMC-MetaNet achieves up to 86.65% accuracy in 5-way 5-shot classification on various remote sensing datasets, including EuroSAT, NWPU-RESISC45, UC Merced Land Use, and AID. Our results establish AMC-MetaNet as a computationally efficient, scale-aware framework for real-world few-shot remote sensing.

Method: Builds upon CurConMix spatial framework with curriculum-guided contrastive learning, structured hard-pair sampling, and feature-level mixup. CurConMix+ extends this with Multi-Resolution Temporal Transformer (MRTT) that adaptively fuses multi-scale temporal features and dynamically balances spatio-temporal cues. Also introduces LLS48 benchmark dataset.

Result: Outperforms state-of-the-art approaches in triplet recognition on CholecT45 and LLS48 datasets. Exhibits strong cross-level generalization with fine-grained features effectively transferring to higher-level phase and step recognition tasks.

Conclusion: The framework and LLS48 dataset provide a unified foundation for hierarchy-aware, reproducible, and interpretable surgical workflow understanding. Code and dataset will be publicly released to facilitate reproducibility and further research.

Abstract: Surgical action triplet recognition aims to understand fine-grained surgical behaviors by modeling the interactions among instruments, actions, and anatomical targets. Despite its clinical importance for workflow analysis and skill assessment, progress has been hindered by severe class imbalance, subtle visual variations, and the semantic interdependence among triplet components. Existing approaches often address only a subset of these challenges rather than tackling them jointly, which limits their ability to form a holistic understanding. This study builds upon CurConMix, a spatial representation framework. At its core, a curriculum-guided contrastive learning strategy learns discriminative and progressively correlated features, further enhanced by structured hard-pair sampling and feature-level mixup. Its temporal extension, CurConMix+, integrates a Multi-Resolution Temporal Transformer (MRTT) that achieves robust, context-aware understanding by adaptively fusing multi-scale temporal features and dynamically balancing spatio-temporal cues. Furthermore, we introduce LLS48, a new, hierarchically annotated benchmark for complex laparoscopic left lateral sectionectomy, providing step-, task-, and action-level annotations. Extensive experiments on CholecT45 and LLS48 demonstrate that CurConMix+ not only outperforms state-of-the-art approaches in triplet recognition, but also exhibits strong cross-level generalization, as its fine-grained features effectively transfer to higher-level phase and step recognition tasks. Together, the framework and dataset provide a unified foundation for hierarchy-aware, reproducible, and interpretable surgical workflow understanding. The code and dataset will be publicly released on GitHub to facilitate reproducibility and further research.

Method: Proposes S²F-Net framework that exploits inherent spectral discrepancies between real and synthetic textures. Uses a learnable frequency attention module that adaptively weights and enhances discriminative frequency bands by combining spatial texture analysis with spectral dependencies. Focuses on detecting frequency-domain artifacts left by upsampling operations in both texture-poor and texture-rich regions.

Result: On AIGCDetectBenchmark (17 categories of generative models), S²F-Net achieves 90.49% detection accuracy, significantly outperforming various existing baseline methods in cross-domain detection scenarios.

Conclusion: S²F-Net effectively addresses generalization challenges in AI-generated content detection by leveraging spectral discrepancies, demonstrating strong performance across diverse generative models and improving cross-domain detection capabilities.

Abstract: The rapid development of generative models has imposed an urgent demand for detection schemes with strong generalization capabilities. However, existing detection methods generally suffer from overfitting to specific source models, leading to significant performance degradation when confronted with unseen generative architectures. To address these challenges, this paper proposes a cross-model detection framework called S 2 F-Net, whose core lies in exploring and leveraging the inherent spectral discrepancies between real and synthetic textures. Considering that upsampling operations leave unique and distinguishable frequency fingerprints in both texture-poor and texture-rich regions, we focus our research on the detection of frequency-domain artifacts, aiming to fundamentally improve the generalization performance of the model. Specifically, we introduce a learnable frequency attention module that adaptively weights and enhances discriminative frequency bands by synergizing spatial texture analysis and spectral dependencies.On the AIGCDetectBenchmark, which includes 17 categories of generative models, S 2 F-Net achieves a detection accuracy of 90.49%, significantly outperforming various existing baseline methods in cross-domain detection scenarios.

Method: Uses CLIP global features conditioned with learnable prototype banks (illumination, head pose, background, direction), fuses these with CLIP patch tokens and high-resolution CNN tokens in unified attention space, and replaces FFN blocks with routed/shared Mixture of Experts for increased conditional capacity.

Result: Achieves new state-of-the-art angular errors: 2.49° on MPIIFaceGaze, 3.22° on EYEDIAP, 10.16° on Gaze360, and 1.44° on ETH-XGaze, demonstrating up to 64% relative improvement over previous methods.

Conclusion: The proposed Gazeformer model effectively addresses 3D gaze estimation challenges through semantic conditioning, multi-scale fusion, and conditional capacity enhancement, setting new benchmarks across multiple datasets.

Abstract: We present a semantics modulated, multi scale Transformer for 3D gaze estimation. Our model conditions CLIP global features with learnable prototype banks (illumination, head pose, background, direction), fuses these prototype-enriched global vectors with CLIP patch tokens and high-resolution CNN tokens in a unified attention space, and replaces several FFN blocks with routed/shared Mixture of Experts to increase conditional capacity. Evaluated on MPIIFaceGaze, EYEDIAP, Gaze360 and ETH-XGaze, our model achieves new state of the art angular errors of 2.49°, 3.22°, 10.16°, and 1.44°, demonstrating up to a 64% relative improvement over previously reported results. ablations attribute gains to prototype conditioning, cross scale fusion, MoE and hyperparameter. Our code is publicly available at https://github. com/AIPMLab/Gazeformer.

Method: Proposes a lightweight descriptor-learning architecture that augments a Siamese CNN with: (1) hypernetwork modules that compute adaptive, per-channel scaling and shifting, and (2) conditional instance normalization for modality-specific adaptation in shallow layers. Trained with triplet loss and hard-negative mining.

Result: Achieves state-of-the-art results on VIS-NIR and other VIS-IR benchmarks, matches or surpasses prior methods on additional datasets despite their higher inference cost. Also releases GAP-VIR dataset with 500K cross-platform (ground/aerial) VIS-IR patch pairs for domain shift evaluation.

Conclusion: Hypernetworks provide an effective mechanism for multimodal patch matching by enabling adaptive, modality-specific feature adaptation while preserving inference efficiency. The approach successfully addresses appearance shifts between modalities and the released dataset enables better evaluation of cross-domain generalization.

Abstract: Hypernetworks are models that generate or modulate the weights of another network. They provide a flexible mechanism for injecting context and task conditioning and have proven broadly useful across diverse applications without significant increases in model size. We leverage hypernetworks to improve multimodal patch matching by introducing a lightweight descriptor-learning architecture that augments a Siamese CNN with (i) hypernetwork modules that compute adaptive, per-channel scaling and shifting and (ii) conditional instance normalization that provides modality-specific adaptation (e.g., visible vs. infrared, VIS-IR) in shallow layers. This combination preserves the efficiency of descriptor-based methods during inference while increasing robustness to appearance shifts. Trained with a triplet loss and hard-negative mining, our approach achieves state-of-the-art results on VIS-NIR and other VIS-IR benchmarks and matches or surpasses prior methods on additional datasets, despite their higher inference cost. To spur progress on domain shift, we also release GAP-VIR, a cross-platform (ground/aerial) VIS-IR patch dataset with 500K pairs, enabling rigorous evaluation of cross-domain generalization and adaptation.

Method: Proposes EmoKGEdit with two key components: 1) Multimodal Sentiment Association Knowledge Graph (MSA-KG) that encodes causal chains among objects, attributes, scenes, visual clues and emotions to guide reasoning, and 2) A disentangled structure-emotion editing module that separates emotional attributes from layout features in latent space to maintain spatial coherence while injecting target emotions.

Result: Extensive experiments show EmoKGEdit achieves excellent performance in both emotion fidelity and content preservation, outperforming state-of-the-art methods.

Conclusion: EmoKGEdit successfully bridges the gap in image emotion editing by effectively disentangling emotional cues from content representations through knowledge graph-guided reasoning and explicit latent space separation, resulting in precise emotion editing with preserved visual structure.

Abstract: Existing image emotion editing methods struggle to disentangle emotional cues from latent content representations, often yielding weak emotional expression and distorted visual structures. To bridge this gap, we propose EmoKGEdit, a novel training-free framework for precise and structure-preserving image emotion editing. Specifically, we construct a Multimodal Sentiment Association Knowledge Graph (MSA-KG) to disentangle the intricate relationships among objects, scenes, attributes, visual clues and emotion. MSA-KG explicitly encode the causal chain among object-attribute-emotion, and as external knowledge to support chain of thought reasoning, guiding the multimodal large model to infer plausible emotion-related visual cues and generate coherent instructions. In addition, based on MSA-KG, we design a disentangled structure-emotion editing module that explicitly separates emotional attributes from layout features within the latent space, which ensures that the target emotion is effectively injected while strictly maintaining visual spatial coherence. Extensive experiments demonstrate that EmoKGEdit achieves excellent performance in both emotion fidelity and content preservation, and outperforms the state-of-the-art methods.

Method: Proposes FlowIID, a novel architecture based on latent flow matching. Combines a VAE-guided latent space with a flow matching module to enable stable decomposition of albedo and shading. The model is designed to be parameter-efficient and produces results in a single inference step.

Result: FlowIID delivers competitive and superior results compared to existing models across various benchmarks. Despite its compact design, it maintains high performance while being parameter-efficient and suitable for single-step inference.

Conclusion: FlowIID is well-suited for deployment in resource-constrained and real-time vision applications due to its parameter efficiency, competitive performance, and single-step inference capability.

Abstract: Intrinsic Image Decomposition (IID) separates an image into albedo and shading components. It is a core step in many real-world applications, such as relighting and material editing. Existing IID models achieve good results, but often use a large number of parameters. This makes them costly to combine with other models in real-world settings. To address this problem, we propose a flow matching-based solution. For this, we design a novel architecture, FlowIID, based on latent flow matching. FlowIID combines a VAE-guided latent space with a flow matching module, enabling a stable decomposition of albedo and shading. FlowIID is not only parameter-efficient, but also produces results in a single inference step. Despite its compact design, FlowIID delivers competitive and superior results compared to existing models across various benchmarks. This makes it well-suited for deployment in resource-constrained and real-time vision applications.

Method: Combined cluster sparsity prior with GoDec algorithm by incorporating it into the S-step. Modeled cluster sparsity using Markov random field and computed anomaly marginal probabilities via message passing on factor graph. High-probability locations form the sparse component in Turbo-GoDec.

Result: Experiments on three real HSI datasets show Turbo-GoDec outperforms vanilla GoDec (LSMAD) and state-of-the-art methods, especially for detecting small-size anomalies.

Conclusion: Incorporating cluster sparsity prior significantly improves hyperspectral anomaly detection performance, particularly for small anomalies, demonstrating the value of exploiting spatial distribution characteristics of anomalies.

Abstract: As a key task in hyperspectral image processing, hyperspectral anomaly detection has garnered significant attention and undergone extensive research. Existing methods primarily relt on two prior assumption: low-rank background and sparse anomaly, along with additional spatial assumptions of the background. However, most methods only utilize the sparsity prior assumption for anomalies and rarely expand on this hypothesis. From observations of hyperspectral images, we find that anomalous pixels exhibit certain spatial distribution characteristics: they often manifest as small, clustered groups in space, which we refer to as cluster sparsity of anomalies. Then, we combined the cluster sparsity prior with the classical GoDec algorithm, incorporating the cluster sparsity prior into the S-step of GoDec. This resulted in a new hyperspectral anomaly detection method, which we called Turbo-GoDec. In this approach, we modeled the cluster sparsity prior of anomalies using a Markov random field and computed the marginal probabilities of anomalies through message passing on a factor graph. Locations with high anomalous probabilities were treated as the sparse component in the Turbo-GoDec. Experiments are conducted on three real hyperspectral image (HSI) datasets which demonstrate the superior performance of the proposed Turbo-GoDec method in detecting small-size anomalies comparing with the vanilla GoDec (LSMAD) and state-of-the-art anomaly detection methods. The code is available at https://github.com/jiahuisheng/Turbo-GoDec.

Method: Introduces MMDR-Bench with 140 expert-crafted tasks across 21 domains, each providing image-text bundles. Proposes three evaluation components: FLAE for report quality, TRACE for citation-grounded evidence alignment, and MOSAIC for text-visual integrity.

Result: Experiments across 25 state-of-the-art models reveal systematic trade-offs between generation quality, citation discipline, and multimodal grounding, showing strong prose alone doesn’t guarantee faithful evidence use and multimodal integrity remains a key bottleneck.

Conclusion: MMDR-Bench addresses the gap in evaluating multimodal deep research agents, highlighting the importance of faithful evidence use and multimodal integrity beyond just generation quality, with the proposed evaluation pipeline providing fine-grained diagnostic signals.

Abstract: Deep Research Agents (DRAs) generate citation-rich reports via multi-step search and synthesis, yet existing benchmarks mainly target text-only settings or short-form multimodal QA, missing end-to-end multimodal evidence use. We introduce MMDeepResearch-Bench (MMDR-Bench), a benchmark of 140 expert-crafted tasks across 21 domains, where each task provides an image-text bundle to evaluate multimodal understanding and citation-grounded report generation. Compared to prior setups, MMDR-Bench emphasizes report-style synthesis with explicit evidence use, where models must connect visual artifacts to sourced claims and maintain consistency across narrative, citations, and visual references. We further propose a unified, interpretable evaluation pipeline: Formula-LLM Adaptive Evaluation (FLAE) for report quality, Trustworthy Retrieval-Aligned Citation Evaluation (TRACE) for citation-grounded evidence alignment, and Multimodal Support-Aligned Integrity Check (MOSAIC) for text-visual integrity, each producing fine-grained signals that support error diagnosis beyond a single overall score. Experiments across 25 state-of-the-art models reveal systematic trade-offs between generation quality, citation discipline, and multimodal grounding, highlighting that strong prose alone does not guarantee faithful evidence use and that multimodal integrity remains a key bottleneck for deep research agents.

Method: Proposes SimpleMatch with: 1) Lightweight upsample decoder that progressively recovers spatial detail by upsampling deep features to 1/4 resolution, 2) Multi-scale supervised loss to retain discriminative features across spatial scales, 3) Sparse matching and window-based localization to optimize training memory usage.

Result: Achieves 84.1% PCK@0.1 on SPair-71k benchmark at 252x252 resolution (3.3x smaller than current SOTA), reduces training memory by 51%, and provides efficient performance with lower computational requirements.

Conclusion: SimpleMatch offers a practical and efficient baseline for semantic correspondence research by addressing feature fusion issues in downsampling, enabling strong performance at low resolutions with reduced computational overhead, making it suitable for real-world applications.

Abstract: Recent advances in semantic correspondence have been largely driven by the use of pre-trained large-scale models. However, a limitation of these approaches is their dependence on high-resolution input images to achieve optimal performance, which results in considerable computational overhead. In this work, we address a fundamental limitation in current methods: the irreversible fusion of adjacent keypoint features caused by deep downsampling operations. This issue is triggered when semantically distinct keypoints fall within the same downsampled receptive field (e.g., 16x16 patches). To address this issue, we present SimpleMatch, a simple yet effective framework for semantic correspondence that delivers strong performance even at low resolutions. We propose a lightweight upsample decoder that progressively recovers spatial detail by upsampling deep features to 1/4 resolution, and a multi-scale supervised loss that ensures the upsampled features retain discriminative features across different spatial scales. In addition, we introduce sparse matching and window-based localization to optimize training memory usage and reduce it by 51%. At a resolution of 252x252 (3.3x smaller than current SOTA methods), SimpleMatch achieves superior performance with 84.1% PCK@0.1 on the SPair-71k benchmark. We believe this framework provides a practical and efficient baseline for future research in semantic correspondence. Code is available at: https://github.com/hailong23-jin/SimpleMatch.

Method: Agentic framework with three specialized agents: Descriptor agent uses chain-of-symbols prompting to assess scene criticality; Planner agent constructs high-level sub-goals via in-context learning; Generator agent synthesizes executable BT sub-trees in XML format.

Result: Successfully demonstrated in CARLA+Nav2 simulation, the system triggers only when baseline BT fails and can navigate around unexpected obstacles like street blockages without human intervention.

Conclusion: The approach is a proof-of-concept that extends to diverse driving scenarios, showing potential for adaptive behavior planning in autonomous vehicles using LLM/LVM-based systems.

Abstract: Autonomous vehicles (AVs) require adaptive behavior planners to navigate unpredictable, real-world environments safely. Traditional behavior trees (BTs) offer structured decision logic but are inherently static and demand labor-intensive manual tuning, limiting their applicability at SAE Level 5 autonomy. This paper presents an agentic framework that leverages large language models (LLMs) and multi-modal vision models (LVMs) to generate and adapt BTs on the fly. A specialized Descriptor agent applies chain-of-symbols prompting to assess scene criticality, a Planner agent constructs high-level sub-goals via in-context learning, and a Generator agent synthesizes executable BT sub-trees in XML format. Integrated into a CARLA+Nav2 simulation, our system triggers only upon baseline BT failure, demonstrating successful navigation around unexpected obstacles (e.g., street blockage) with no human intervention. Compared to a static BT baseline, this approach is a proof-of-concept that extends to diverse driving scenarios.

Method: Builds on previous DepthCropSeg work to create a large-scale dataset (28,406 images across 30+ species and 15 environmental conditions). Uses ViT-Adapter architecture enhanced with dynamic upsampling for detail awareness, trained with a two-stage self-training pipeline.

Result: Achieves 93.11% mIoU on comprehensive testing, outperforming supervised baselines (+0.36%) and general-purpose vision foundation models like SAM (+48.57%). Excels in challenging scenarios: night-time (86.90% mIoU), high-density canopies (90.09% mIoU), and unseen crop varieties (90.09% mIoU).

Conclusion: DepthCropSeg++ represents a new state-of-the-art for crop segmentation, demonstrating strong generalization capabilities across diverse crop species and environmental conditions through large-scale data and enhanced architecture with self-training.

Abstract: DepthCropSeg++: a foundation model for crop segmentation, capable of segmenting different crop species under open in-field environment. Crop segmentation is a fundamental task for modern agriculture, which closely relates to many downstream tasks such as plant phenotyping, density estimation, and weed control. In the era of foundation models, a number of generic large language and vision models have been developed. These models have demonstrated remarkable real world generalization due to significant model capacity and largescale datasets. However, current crop segmentation models mostly learn from limited data due to expensive pixel-level labelling cost, often performing well only under specific crop types or controlled environment. In this work, we follow the vein of our previous work DepthCropSeg, an almost unsupervised approach to crop segmentation, to scale up a cross-species and crossscene crop segmentation dataset, with 28,406 images across 30+ species and 15 environmental conditions. We also build upon a state-of-the-art semantic segmentation architecture ViT-Adapter architecture, enhance it with dynamic upsampling for improved detail awareness, and train the model with a two-stage selftraining pipeline. To systematically validate model performance, we conduct comprehensive experiments to justify the effectiveness and generalization capabilities across multiple crop datasets. Results demonstrate that DepthCropSeg++ achieves 93.11% mIoU on a comprehensive testing set, outperforming both supervised baselines and general-purpose vision foundation models like Segmentation Anything Model (SAM) by significant margins (+0.36% and +48.57% respectively). The model particularly excels in challenging scenarios including night-time environment (86.90% mIoU), high-density canopies (90.09% mIoU), and unseen crop varieties (90.09% mIoU), indicating a new state of the art for crop segmentation.

Method: Two-stack architecture: 1) On-board driving stack with stereo camera for localization and perception (object detection, feature extraction including velocity, yaw, TTC, time-headway), 2) Digital twin stack using Unreal Engine 5 replica that receives real-time data via ROS2 messages over 4G V2I communication.

Result: The system successfully processes 20-fps stereo camera images, extracts safety metrics, and enables real-time digital twin monitoring with safety alerts. Tests confirm validity and real-time response across various driving scenarios.

Conclusion: CD-TWINSAFE provides a functional V2I digital twin architecture for autonomous vehicles that enables real-time safety monitoring and alerting through synchronized physical and virtual environments.

Abstract: In this paper, the CD-TWINSAFE is introduced, a V2I-based digital twin for Autonomous Vehicles. The proposed architecture is composed of two stacks running simultaneously, an on-board driving stack that includes a stereo camera for scene understanding, and a digital twin stack that runs an Unreal Engine 5 replica of the scene viewed by the camera as well as returning safety alerts to the cockpit. The on-board stack is implemented on the vehicle side including 2 main autonomous modules; localization and perception. The position and orientation of the ego vehicle are obtained using on-board sensors. Furthermore, the perception module is responsible for processing 20-fps images from stereo camera and understands the scene through two complementary pipelines. The pipeline are working on object detection and feature extraction including object velocity, yaw and the safety metrics time-to-collision and time-headway. The collected data form the driving stack are sent to the infrastructure side through the ROS-enabled architecture in the form of custom ROS2 messages and sent over UDP links that ride a 4G modem for V2I communication. The environment is monitored via the digital twin through the shared messages which update the information of the spawned ego vehicle and detected objects based on the real-time localization and perception data. Several tests with different driving scenarios to confirm the validity and real-time response of the proposed architecture.

Method: Train a score-based generative model (SGM) on all spectra from the HSI. At test time, perturb each spectrum through a perturbation kernel, feed it to the trained SGM to obtain estimated scores. The score field captures the data distribution gradient, enabling discrimination between on-manifold (background) and off-manifold (anomalous) spectra.

Result: Experiments on four hyperspectral datasets demonstrate the effectiveness of the proposed ScoreAD method for hyperspectral anomaly detection.

Conclusion: ScoreAD successfully leverages score-based generative models and the hyperspectral manifold hypothesis to detect anomalies by distinguishing between background spectra on low-dimensional manifolds and anomalous spectra as outliers, with code publicly available.

Abstract: Hyperspectral images (HSIs) are a type of image that contains abundant spectral information. As a type of real-world data, the high-dimensional spectra in hyperspectral images are actually determined by only a few factors, such as chemical composition and illumination. Thus, spectra in hyperspectral images are highly likely to satisfy the manifold hypothesis. Based on the hyperspectral manifold hypothesis, we propose a novel hyperspectral anomaly detection method (named ScoreAD) that leverages the time-dependent gradient field of the data distribution (i.e., the score), as learned by a score-based generative model (SGM). Our method first trains the SGM on the entire set of spectra from the hyperspectral image. At test time, each spectrum is passed through a perturbation kernel, and the resulting perturbed spectrum is fed into the trained SGM to obtain the estimated score. The manifold hypothesis of HSIs posits that background spectra reside on one or more low-dimensional manifolds. Conversely, anomalous spectra, owing to their unique spectral signatures, are considered outliers that do not conform to the background manifold. Based on this fundamental discrepancy in their manifold distributions, we leverage a generative SGM to achieve hyperspectral anomaly detection. Experiments on the four hyperspectral datasets demonstrate the effectiveness of the proposed method. The code is available at https://github.com/jiahuisheng/ScoreAD.

Method: Evaluated embeddings from 10 foundation models across three domains (general CV, general medical, dermatology-specific) using DERM12345 dataset with 40 lesion subclasses. Used frozen embeddings with lightweight adapter models and 5-fold cross-validation. Introduced hierarchical evaluation framework across four granularity levels.

Result: MedImageInsights achieved 97.52% weighted F1-Score on binary malignancy detection but declined to 65.50% on 40-class subtype classification. MedSigLip (69.79%) and dermatology-specific models excelled at fine-grained classification but had lower overall performance on broader tasks, revealing a “granularity gap.”

Conclusion: General medical foundation models are effective for high-level screening, but specialized modeling strategies are necessary for the granular distinctions required in diagnostic support systems.

Abstract: Foundation models have transformed medical image analysis by providing robust feature representations that reduce the need for large-scale task-specific training. However, current benchmarks in dermatology often reduce the complex diagnostic taxonomy to flat, binary classification tasks, such as distinguishing melanoma from benign nevi. This oversimplification obscures a model’s ability to perform fine-grained differential diagnoses, which is critical for clinical workflow integration. This study evaluates the utility of embeddings derived from ten foundation models, spanning general computer vision, general medical imaging, and dermatology-specific domains, for hierarchical skin lesion classification. Using the DERM12345 dataset, which comprises 40 lesion subclasses, we calculated frozen embeddings and trained lightweight adapter models using a five-fold cross-validation. We introduce a hierarchical evaluation framework that assesses performance across four levels of clinical granularity: 40 Subclasses, 15 Main Classes, 2 and 4 Superclasses, and Binary Malignancy. Our results reveal a “granularity gap” in model capabilities: MedImageInsights achieved the strongest overall performance (97.52% weighted F1-Score on Binary Malignancy detection) but declined to 65.50% on fine-grained 40-class subtype classification. Conversely, MedSigLip (69.79%) and dermatology-specific models (Derm Foundation and MONET) excelled at fine-grained 40-class subtype discrimination while achieving lower overall performance than MedImageInsights on broader classification tasks. Our findings suggest that while general medical foundation models are highly effective for high-level screening, specialized modeling strategies are necessary for the granular distinctions required in diagnostic support systems.

Method: Introduces Class-Partitioned Vector Quantized Variational Autoencoder (CPVQ-VAE) with class-partitioned codebook where codevectors are labeled by class, and class-aware running average update to prevent codebook collapse. Uses Latent-space Flow Matching Model (LFMM) to generate object features and class labels, then CPVQ-VAE performs class-aware inverse look-up to map latents to codebook entries for decoding to class-specific point cloud shapes.

Result: Achieves pure point cloud generation without external object database retrieval, with up to 70.4% reduction in Chamfer error and 72.3% reduction in Point2Mesh error on complex living room scenes.

Conclusion: The proposed CPVQ-VAE with class-partitioned codebook and class-aware training effectively decodes object latent features into class-specific point cloud shapes, enabling reliable point cloud scene generation without database retrieval dependency.

Abstract: Most 3D scene generation methods are limited to only generating object bounding box parameters while newer diffusion methods also generate class labels and latent features. Using object size or latent feature, they then retrieve objects from a predefined database. For complex scenes of varied, multi-categorical objects, diffusion-based latents cannot be effectively decoded by current autoencoders into the correct point cloud objects which agree with target classes. We introduce a Class-Partitioned Vector Quantized Variational Autoencoder (CPVQ-VAE) that is trained to effectively decode object latent features, by employing a pioneering $\textit{class-partitioned codebook}$ where codevectors are labeled by class. To address the problem of $\textit{codebook collapse}$, we propose a $\textit{class-aware}$ running average update which reinitializes dead codevectors within each partition. During inference, object features and class labels, both generated by a Latent-space Flow Matching Model (LFMM) designed specifically for scene generation, are consumed by the CPVQ-VAE. The CPVQ-VAE’s class-aware inverse look-up then maps generated latents to codebook entries that are decoded to class-specific point cloud shapes. Thereby, we achieve pure point cloud generation without relying on an external objects database for retrieval. Extensive experiments reveal that our method reliably recovers plausible point cloud scenes, with up to 70.4% and 72.3% reduction in Chamfer and Point2Mesh errors on complex living room scenes.

Method: The method involves: 1) conducting an in-depth review of existing literature, 2) selecting three best-performing neural network solutions, 3) choosing three diverse datasets with varied image numbers, 4) training the selected networks, and 5) performing comparative experiments including cross-dataset testing.

Result: The experiments revealed several weaknesses in existing emotion detection solutions: 1) differences between datasets affecting model performance, 2) unequal difficulty levels in recognizing certain emotions, and 3) challenges in differentiating between closely related emotions.

Conclusion: The study concludes that current emotion detection systems have significant limitations related to dataset inconsistencies and difficulty distinguishing similar emotions, highlighting the need for more robust solutions and standardized evaluation approaches.

Abstract: Emotion detection from faces is one of the machine learning problems needed for human-computer interaction. The variety of methods used is enormous, which motivated an in-depth review of articles and scientific studies. Three of the most interesting and best solutions are selected, followed by the selection of three datasets that stood out for the diversity and number of images in them. The selected neural networks are trained, and then a series of experiments are performed to compare their performance, including testing on different datasets than a model was trained on. This reveals weaknesses in existing solutions, including differences between datasets, unequal levels of difficulty in recognizing certain emotions and the challenges in differentiating between closely related emotions.

Method: Formulate camera-projected points as lines/rays, use epipolar distance and 3D ray distance as cost functions in optimal transport problems. Create efficiently solvable assignment problems for feature matching, and extend to hierarchical OT for object matching.

Result: Developed algorithms for efficient feature and object matching, demonstrated in numerical experiments focused on facial analysis applications.

Conclusion: Optimal transport with line constraints provides effective unsupervised sparse matching solution for stereo vision challenges, particularly useful for matching different facial landmarking conventions.

Abstract: Stereo vision between images faces a range of challenges, including occlusions, motion, and camera distortions, across applications in autonomous driving, robotics, and face analysis. Due to parameter sensitivity, further complications arise for stereo matching with sparse features, such as facial landmarks. To overcome this ill-posedness and enable unsupervised sparse matching, we consider line constraints of the camera geometry from an optimal transport (OT) viewpoint. Formulating camera-projected points as (half)lines, we propose the use of the classical epipolar distance as well as a 3D ray distance to quantify matching quality. Employing these distances as a cost function of a (partial) OT problem, we arrive at efficiently solvable assignment problems. Moreover, we extend our approach to unsupervised object matching by formulating it as a hierarchical OT problem. The resulting algorithms allow for efficient feature and object matching, as demonstrated in our numerical experiments. Here, we focus on applications in facial analysis, where we aim to match distinct landmarking conventions.

Method: The paper surveys three defense approaches: 1) Training-time Defense (adversarial fine-tuning), 2) Test-time Adaptation Defense (updating parameters at inference), and 3) Training-free Defense (altering inputs/features without model modification).

Result: The survey reviews latest advancements in adversarial defense strategies for VLMs, analyzing strengths and limitations of each approach and identifying ongoing challenges in VLM robustness.

Conclusion: While various defense paradigms exist for protecting VLMs against adversarial attacks, each has trade-offs in effectiveness, computational cost, and generalization, with ongoing research needed to enhance VLM robustness across different attack scenarios.

Abstract: The widespread use of Vision Language Models (VLMs, e.g. CLIP) has raised concerns about their vulnerability to sophisticated and imperceptible adversarial attacks. These attacks could compromise model performance and system security in cross-modal tasks. To address this challenge, three main defense paradigms have been proposed: Training-time Defense, Test-time Adaptation Defense, and Training-free Defense. Training-time Defense involves modifying the training process, typically through adversarial fine-tuning to improve the robustness to adversarial examples. While effective, this approach requires substantial computational resources and may not generalize across all adversarial attacks. Test-time Adaptation Defense focuses on adapting the model at inference time by updating its parameters to handle unlabeled adversarial examples, offering flexibility but often at the cost of increased complexity and computational overhead. Training-free Defense avoids modifying the model itself, instead focusing on altering the adversarial inputs or their feature embeddings, which enforces input perturbations to mitigate the impact of attacks without additional training. This survey reviews the latest advancements in adversarial defense strategies for VLMs, highlighting the strengths and limitations of such approaches and discussing ongoing challenges in enhancing the robustness of VLMs.

Method: Created large-scale multi-source benchmark with 100k+ 2D EM images across cell types and 5 organelle classes, using connectivity-aware 3D Label Propagation Algorithm with expert refinement. Benchmarked U-Net, SAM variants, and Mask2Former.

Result: Current models struggle to generalize across heterogeneous EM data and perform poorly on organelles with global, distributed morphologies (e.g., Endoplasmic Reticulum).

Conclusion: There’s a fundamental mismatch between local-context models and the need to model long-range structural continuity in real-world EM data. The benchmark and labeling tool will be publicly released.

Abstract: Accurate instance-level segmentation of organelles in electron microscopy (EM) is critical for quantitative analysis of subcellular morphology and inter-organelle interactions. However, current benchmarks, based on small, curated datasets, fail to capture the inherent heterogeneity and large spatial context of in-the-wild EM data, imposing fundamental limitations on current patch-based methods. To address these limitations, we developed a large-scale, multi-source benchmark for multi-organelle instance segmentation, comprising over 100,000 2D EM images across variety cell types and five organelle classes that capture real-world variability. Dataset annotations were generated by our designed connectivity-aware Label Propagation Algorithm (3D LPA) with expert refinement. We further benchmarked several state-of-the-art models, including U-Net, SAM variants, and Mask2Former. Our results show several limitations: current models struggle to generalize across heterogeneous EM data and perform poorly on organelles with global, distributed morphologies (e.g., Endoplasmic Reticulum). These findings underscore the fundamental mismatch between local-context models and the challenge of modeling long-range structural continuity in the presence of real-world variability. The benchmark dataset and labeling tool will be publicly released soon.

Method: DCAC maintains separate caches for each ID class to collect high-entropy samples during testing. It uses a lightweight two-layer module that leverages cached visual features and predicted probabilities to calibrate raw predictions, mitigating overconfidence on OOD samples.

Result: Extensive experiments show DCAC significantly enhances existing OOD detection methods across multiple benchmarks, reducing FPR95 by 6.55% when integrated with ASH-S on ImageNet OOD benchmark.

Conclusion: DCAC provides an effective, training-free calibration module that can be seamlessly integrated with various OOD detection methods across unimodal and vision-language models with minimal computational overhead.

Abstract: Out-of-distribution (OOD) detection remains a fundamental challenge for deep neural networks, particularly due to overconfident predictions on unseen OOD samples during testing. We reveal a key insight: OOD samples predicted as the same class, or given high probabilities for it, are visually more similar to each other than to the true in-distribution (ID) samples. Motivated by this class-specific observation, we propose DCAC (Dynamic Class-Aware Cache), a training-free, test-time calibration module that maintains separate caches for each ID class to collect high-entropy samples and calibrate the raw predictions of input samples. DCAC leverages cached visual features and predicted probabilities through a lightweight two-layer module to mitigate overconfident predictions on OOD samples. This module can be seamlessly integrated with various existing OOD detection methods across both unimodal and vision-language models while introducing minimal computational overhead. Extensive experiments on multiple OOD benchmarks demonstrate that DCAC significantly enhances existing methods, achieving substantial improvements, i.e., reducing FPR95 by 6.55% when integrated with ASH-S on ImageNet OOD benchmark.

Method: 1) Reconstruct coarse surface geometry using multi-view stereo. 2) Use VLM to retrieve realistic fur length/structure information for each body part. 3) Construct furless geometry and grow strands atop it. 4) Supervise with geometric and photometric losses from multi-view images. 5) Use VLM to guide strand growth direction and incorporate gravity vector as loss to mitigate orientation ambiguities.

Result: Shows generalization across various animals with different fur types using the novel VLM-guided 3D reconstruction schema.

Conclusion: Presents the first multi-view method for high-fidelity 3D animal fur modeling using strand-based representation, leveraging VLMs to overcome the lack of fur datasets and achieve realistic reconstruction across diverse animal types.

Abstract: Reconstructing realistic animal fur geometry from images is a challenging task due to the fine-scale details, self-occlusion, and view-dependent appearance of fur. In contrast to human hairstyle reconstruction, there are also no datasets that can be leveraged to learn a fur prior for different animals. In this work, we present a first multi-view-based method for high-fidelity 3D fur modeling of animals using a strand-based representation, leveraging the general knowledge of a vision language model. Given multi-view RGB images, we first reconstruct a coarse surface geometry using traditional multi-view stereo techniques. We then use a vision language model (VLM) system to retrieve information about the realistic length structure of the fur for each part of the body. We use this knowledge to construct the animal’s furless geometry and grow strands atop it. The fur reconstruction is supervised with both geometric and photometric losses computed from multi-view images. To mitigate orientation ambiguities stemming from the Gabor filters that are applied to the input images, we additionally utilize the VLM to guide the strands’ growth direction and their relation to the gravity vector that we incorporate as a loss. With this new schema of using a VLM to guide 3D reconstruction from multi-view inputs, we show generalization across a variety of animals with different fur types. For additional results and code, please refer to https://neuralfur.is.tue.mpg.de.

Method: 1) Histopath-C benchmark with realistic synthetic corruptions mimicking real histopathology domain shifts. 2) LATTE: transductive low-rank adaptation using multiple text templates to reduce sensitivity to text input variations.

Result: LATTE outperforms state-of-the-art TTA methods designed for natural images across multiple histopathology datasets, demonstrating effective robust adaptation.

Conclusion: The proposed Histopath-C benchmark and LATTE adaptation strategy effectively address domain shift challenges in histopathology VLMs, enabling more robust performance under realistic corruptions.

Abstract: Medical Vision-language models (VLMs) have shown remarkable performances in various medical imaging domains such as histo-pathology by leveraging pre-trained, contrastive models that exploit visual and textual information. However, histopathology images may exhibit severe domain shifts, such as staining, contamination, blurring, and noise, which may severely degrade the VLM’s downstream performance. In this work, we introduce Histopath-C, a new benchmark with realistic synthetic corruptions designed to mimic real-world distribution shifts observed in digital histopathology. Our framework dynamically applies corruptions to any available dataset and evaluates Test-Time Adaptation (TTA) mechanisms on the fly. We then propose LATTE, a transductive, low-rank adaptation strategy that exploits multiple text templates, mitigating the sensitivity of histopathology VLMs to diverse text inputs. Our approach outperforms state-of-the-art TTA methods originally designed for natural images across a breadth of histopathology datasets, demonstrating the effectiveness of our proposed design for robust adaptation in histopathology images. Code and data are available at https://github.com/Mehrdad-Noori/Histopath-C.

Method: GD3A: Global Density Map Decomposition via Descriptor Association - density map-based video counting using pixel-level descriptor correspondences via optimal transport with adaptive dustbin score. DVTrack: Descriptor voting mechanism converts descriptor-level matching to instance-level associations for tracking.

Result: Significant performance improvements: 47.4% reduction in counting error and 39.2% improvement in tracking performance compared to existing methods on MovingDroneCrowd++ dataset under dense crowds and complex motion.

Conclusion: Proposed drone-based approach with GD3A and DVTrack effectively addresses limitations of fixed-camera methods, enabling accurate large-scale dense crowd analysis through flexible aerial video capture and novel density decomposition techniques.

Abstract: Counting and tracking dense crowds in large-scale scenes is highly challenging, yet existing methods mainly rely on datasets captured by fixed cameras, which provide limited spatial coverage and are inadequate for large-scale dense crowd analysis. To address this limitation, we propose a flexible solution using moving drones to capture videos and perform video-level crowd counting and tracking of unique pedestrians across entire scenes. We introduce MovingDroneCrowd++, the largest video-level dataset for dense crowd counting and tracking captured by moving drones, covering diverse and complex conditions with varying flight altitudes, camera angles, and illumination. Existing methods fail to achieve satisfactory performance on this dataset. To this end, we propose GD3A (Global Density Map Decomposition via Descriptor Association), a density map-based video individual counting method that avoids explicit localization. GD3A establishes pixel-level correspondences between pedestrian descriptors across consecutive frames via optimal transport with an adaptive dustbin score, enabling the decomposition of global density maps into shared, inflow, and outflow components. Building on this framework, we further introduce DVTrack, which converts descriptor-level matching into instance-level associations through a descriptor voting mechanism for pedestrian tracking. Experimental results show that our methods significantly outperform existing approaches under dense crowds and complex motion, reducing counting error by 47.4 percent and improving tracking performance by 39.2 percent.

Method: Proposes SDCoNet with: 1) Swin transformer-based shared encoder for cross-task feature collaboration, 2) Multi-scale saliency prediction module to select key tokens and focus on weak object regions while suppressing background clutter, 3) Gradient routing strategy to stabilize detection semantics and guide SR gradients toward detection-oriented directions.

Result: Experiments on NWPU VHR-10-Split, DOTAv1.5-Split, and HRSSD-Split datasets show SDCoNet significantly outperforms existing mainstream algorithms in small object detection on low-quality remote sensing images while maintaining competitive computational efficiency.

Conclusion: SDCoNet effectively addresses the limitations of serial SR-detection pipelines by enabling collaborative optimization through shared features, saliency-guided attention, and gradient routing, achieving superior performance for small object detection in challenging remote sensing conditions.

Abstract: In remote sensing images, complex backgrounds, weak object signals, and small object scales make accurate detection particularly challenging, especially under low-quality imaging conditions. A common strategy is to integrate single-image super-resolution (SR) before detection; however, such serial pipelines often suffer from misaligned optimization objectives, feature redundancy, and a lack of effective interaction between SR and detection. To address these issues, we propose a Saliency-Driven multi-task Collaborative Network (SDCoNet) that couples SR and detection through implicit feature sharing while preserving task specificity. SDCoNet employs the swin transformer-based shared encoder, where hierarchical window-shifted self-attention supports cross-task feature collaboration and adaptively balances the trade-off between texture refinement and semantic representation. In addition, a multi-scale saliency prediction module produces importance scores to select key tokens, enabling focused attention on weak object regions, suppression of background clutter, and suppression of adverse features introduced by multi-task coupling. Furthermore, a gradient routing strategy is introduced to mitigate optimization conflicts. It first stabilizes detection semantics and subsequently routes SR gradients along a detection-oriented direction, enabling the framework to guide the SR branch to generate high-frequency details that are explicitly beneficial for detection. Experiments on public datasets, including NWPU VHR-10-Split, DOTAv1.5-Split, and HRSSD-Split, demonstrate that the proposed method, while maintaining competitive computational efficiency, significantly outperforms existing mainstream algorithms in small object detection on low-quality remote sensing images. Our code is available at https://github.com/qiruo-ya/SDCoNet.

Method: CycleGAN-based model for unsupervised medical-image domain adaptation that learns bidirectional mappings between source and target domains without paired training data, using content and disparity loss to preserve anatomical integrity.

Result: Experiments on MRI datasets demonstrate efficacy in bidirectional domain adaptation without labeled data, improving model performance and reducing domain-related variability for more precise medical image analysis.

Conclusion: The approach offers promising avenues for improving diagnostic accuracy in healthcare by enabling effective domain adaptation while maintaining image integrity, contributing to more consistent medical image analysis.

Abstract: Magnetic Resonance Imaging (MRI) scans acquired from different scanners or institutions often suffer from domain shifts owing to variations in hardware, protocols, and acquisition parameters. This discrepancy degrades the performance of deep learning models trained on source domain data when applied to target domain images. In this study, we propose a Cycle-GAN-based model for unsupervised medical-image domain adaptation. Leveraging CycleGANs, our model learns bidirectional mappings between the source and target domains without paired training data, preserving the anatomical content of the images. By leveraging Cycle-GAN capabilities with content and disparity loss for adaptation tasks, we ensured image-domain adaptation while maintaining image integrity. Several experiments on MRI datasets demonstrated the efficacy of our model in bidirectional domain adaptation without labelled data. Furthermore, research offers promising avenues for improving the diagnostic accuracy of healthcare. The statistical results confirm that our approach improves model performance and reduces domain-related variability, thus contributing to more precise and consistent medical image analysis.

Method: Uses deep feature proximity to compute smooth semantic deformation weights without additional regularization. Introduces barycentric feature distillation pipeline that efficiently uses visual signals from shape renders to minimize distillation cost. Preserves classical method properties through feature space constraints and locality weighting.

Result: Achieves real-time computation of deformation weights for any surface point, enables co-deformation of semantic parts, handles meshes up to 1 million faces in real-time on consumer-grade machines, and computes weights for high-resolution meshes in under a minute (vs. hours for previous methods).

Conclusion: Successfully fuses semantic prior from data with precise control and speed of traditional frameworks, offering a simple yet effective solution that enables semantic, precise, and real-time mesh deformation with automatic symmetry detection and preservation.

Abstract: Handle-based mesh deformation has been a long-standing paradigm in computer graphics, enabling intuitive shape edits from sparse controls. Classic techniques offer precise and rapid deformation control. However, they solve an optimization problem with constraints defined by control handle placement, requiring a user to know apriori the ideal distribution of handles on the shape to accomplish the desired edit. The mapping from handle set to deformation behavior is often unintuitive and, importantly, non-semantic. Modern data-driven methods, on the other hand, leverage a data prior to obtain semantic edits, but are slow and imprecise. We propose a technique that fuses the semantic prior of data with the precise control and speed of traditional frameworks. Our approach is surprisingly simple yet effective: deep feature proximity makes for smooth and semantic deformation weights, with no need for additional regularization. The weights can be computed in real-time for any surface point, whereas prior methods require optimization for new handles. Moreover, the semantic prior from deep features enables co-deformation of semantic parts. We introduce an improved feature distillation pipeline, barycentric feature distillation, which efficiently uses the visual signal from shape renders to minimize distillation cost. This allows our weights to be computed for high resolution meshes in under a minute, in contrast to potentially hours for both classical and neural methods. We preserve and extend properties of classical methods through feature space constraints and locality weighting. Our field representation allows for automatic detection of semantic symmetries, which we use to produce symmetry-preserving deformations. We show a proof-of-concept application which can produce deformations for meshes up to 1 million faces in real-time on a consumer-grade machine.

Method: Cross-attention-based architecture that operates on image patches centered at matched keypoints, predicting refined coordinates without relying on detector representations.

Result: Consistently improves geometric estimation accuracy on MegaDepth, KITTI, and ScanNet datasets, outperforming existing refinement methods while maintaining runtime efficiency.

Conclusion: XRefine provides a generalizable, detector-agnostic solution for sub-pixel keypoint refinement that can be extended to multi-view feature tracks.

Abstract: Sparse keypoint matching is crucial for 3D vision tasks, yet current keypoint detectors often produce spatially inaccurate matches. Existing refinement methods mitigate this issue through alignment of matched keypoint locations, but they are typically detector-specific, requiring retraining for each keypoint detector. We introduce XRefine, a novel, detector-agnostic approach for sub-pixel keypoint refinement that operates solely on image patches centered at matched keypoints. Our cross-attention-based architecture learns to predict refined keypoint coordinates without relying on internal detector representations, enabling generalization across detectors. Furthermore, XRefine can be extended to handle multi-view feature tracks. Experiments on MegaDepth, KITTI, and ScanNet demonstrate that the approach consistently improves geometric estimation accuracy, achieving superior performance compared to existing refinement methods while maintaining runtime efficiency. Our code and trained models can be found at https://github.com/boschresearch/xrefine.

Method: The authors created the BirdsEye-RU dataset by collecting 2,978 images containing over 8,000 annotated faces. The dataset includes both drone images and smartphone-captured images from high altitudes, specifically designed to capture small and distant faces across diverse environments.

Result: The paper presents a comprehensive dataset that is now publicly available on Kaggle. The dataset addresses the specific challenges of face detection in overhead imagery by providing annotated examples of small and distant faces in various environmental conditions.

Conclusion: The BirdsEye-RU dataset fills an important gap in computer vision research by providing a specialized resource for developing and evaluating face detection algorithms in overhead imagery, particularly for small and distant faces in challenging aerial conditions.

Abstract: Detecting faces in overhead images remains a significant challenge due to extreme scale variations and environmental clutter. To address this, we created the BirdsEye-RU dataset, a comprehensive collection of 2,978 images containing over eight thousand annotated faces. This dataset is specifically designed to capture small and distant faces across diverse environments, containing both drone images and smartphone-captured images from high altitude. We present a detailed description of the BirdsEye-RU dataset in this paper. We made our dataset freely available to the public, and it can be accessed at https://www.kaggle.com/datasets/mdahanafarifkhan/birdseye-ru.

Method: Develop novel gaze detection model using self-supervised eye movement reconstruction (inspired by language model pretraining) that leverages unlabeled video. Use encoder embeddings to fine-tune on two downstream tasks: 1) aligning eye movement with directional emotion estimates from speech, 2) predicting three momentary emotional behaviors (laughing, crying/sobbing, sighing). Data from USC Shoah Foundation’s Holocaust survivor interviews.

Result: New model is predictive of emotion outcomes. Positive correlation observed between pretraining performance and emotion processing performance for both experiments.

Conclusion: Self-supervised eye movement reconstruction is an effective method for encoding the affective signal carried by eye movements, enabling emotion prediction from low-resolution video data.

Abstract: The relationship between emotional expression and eye movement is well-documented, with literature establishing gaze patterns are reliable indicators of emotion. However, most studies utilize specialized, high-resolution eye-tracking equipment, limiting the potential reach of findings. We investigate how eye movement can be used to predict multimodal markers of emotional expression from naturalistic, low-resolution videos. We utilize a collection of video interviews from the USC Shoah Foundation’s Visual History Archive with Holocaust survivors as they recount their experiences in the Auschwitz concentration camp. Inspired by pretraining methods on language models, we develop a novel gaze detection model that uses self-supervised eye movement reconstruction that can effectively leverage unlabeled video. We use this model’s encoder embeddings to fine-tune models on two downstream tasks related to emotional expression. The first is aligning eye movement with directional emotion estimates from speech. The second task is using eye gaze as a predictor of three momentary manifestations of emotional behaviors: laughing, crying/sobbing, and sighing. We find our new model is predictive of emotion outcomes and observe a positive correlation between pretraining performance and emotion processing performance for both experiments. We conclude self-supervised eye movement reconstruction is an effective method for encoding the affective signal they carry.

Method: Combines classical quadratic reconstruction error formulation with Cayley parameterization of rotations and least-squares optimization. Uses deterministic selection of starting points based on analysis of reconstruction error for two canonical vectors to avoid costly search procedures. Creates hybrid cost formulation by analytically eliminating reconstruction-error surrogate for translation.

Result: Achieves practically the same projection accuracy as optimal SQPnP and slightly higher than IPPE (both OpenCV PnP procedures). Maintains significantly simpler algorithmic structure. Provides intuitive insight into convergence process through analysis of optimization trajectories in Cayley space.

Conclusion: PnP-ProCay78 offers an effective solution to planar PnP problems with competitive accuracy, simpler implementation, and geometric transparency. The method is attractive for both practical applications and educational purposes due to its intuitive convergence behavior.

Abstract: Estimating the position and orientation of a camera with respect to an observed scene is one of the central problems in computer vision, particularly in the context of camera calibration and multi-sensor systems. This paper addresses the planar Perspective–$n$–Point problem, with special emphasis on the initial estimation of the pose of a calibration object. As a solution, we propose the \texttt{PnP-ProCay78} algorithm, which combines the classical quadratic formulation of the reconstruction error with a Cayley parameterization of rotations and least-squares optimization. The key component of the method is a deterministic selection of starting points based on an analysis of the reconstruction error for two canonical vectors, allowing costly solution-space search procedures to be avoided. Experimental validation is performed using data acquired also from high-resolution RGB cameras and very low-resolution thermal cameras in an integrated RGB–IR setup. The results demonstrate that the proposed algorithm achieves practically the same projection accuracy as optimal \texttt{SQPnP} and slightly higher than \texttt{IPPE}, both prominent \texttt{PnP-OpenCV} procedures. However, \texttt{PnP-ProCay78} maintains a significantly simpler algorithmic structure. Moreover, the analysis of optimization trajectories in Cayley space provides an intuitive insight into the convergence process, making the method attractive also from a didactic perspective. Unlike existing PnP solvers, the proposed \texttt{PnP-ProCay78} algorithm combines projection error minimization with an analytically eliminated reconstruction-error surrogate for translation, yielding a hybrid cost formulation that is both geometrically transparent and computationally efficient.

Method: Search for confluence of spatial representations, use linear models to test causal explanations, conduct rigorous causal interventions to demonstrate ID mediation, and extend analysis to video VLMs for temporal IDs.

Result: VLMs encode object locations by linearly binding spatial IDs to textual activations, with these IDs mediating model beliefs across layers; spatial IDs also serve as diagnostic tools and learning signals; analogous temporal ID mechanism found in video VLMs.

Conclusion: The identified spatiotemporal ID mechanism elucidates previously underexplored internal reasoning processes in VLMs, contributing to improved interpretability and principled design of more aligned and capable models.

Abstract: Spatio-temporal reasoning is a remarkable capability of Vision Language Models (VLMs), but the underlying mechanisms of such abilities remain largely opaque. We postulate that visual/geometrical and textual representations of spatial structure must be combined at some point in VLM computations. We search for such confluence, and ask whether the identified representation can causally explain aspects of input-output model behavior through a linear model. We show empirically that VLMs encode object locations by linearly binding \textit{spatial IDs} to textual activations, then perform reasoning via language tokens. Through rigorous causal interventions we demonstrate that these IDs, which are ubiquitous across the model, can systematically mediate model beliefs at intermediate VLM layers. Additionally, we find that spatial IDs serve as a diagnostic tool for identifying limitations in existing VLMs, and as a valuable learning signal. We extend our analysis to video VLMs and identify an analogous linear temporal ID mechanism. By characterizing our proposed spatiotemporal ID mechanism, we elucidate a previously underexplored internal reasoning process in VLMs, toward improved interpretability and the principled design of more aligned and capable models. We release our code for reproducibility: https://github.com/Raphoo/linear-mech-vlms.

Method: Uses Swin-Transformer based U-Net (Swin-BathyUNet) with leave-one-band out study for spectral importance ranking, adapts ablation-based CAM to regression (A-CAM-R) for explainability, performs attention ablations to analyze decoder cross-attention, and conducts cross-region inference tests.

Result: Green/blue channels most important for depth inference; A-CAM-R reliably localizes evidence; decoder cross-attention improves robustness to glint/foam; cross-region inference shows depth-dependent degradation with MAE rising linearly with depth; bimodal depth distributions exacerbate mid/deep errors.

Conclusion: Practical guidance: maintain wide receptive fields, preserve radiometric fidelity in green/blue channels, pre-filter bright high variance near shore, and pair light target site fine tuning with depth aware calibration for cross-region transfer.

Abstract: Deploying Sentinel-2 satellite derived bathymetry (SDB) robustly across sites remains challenging. We analyze a Swin-Transformer based U-Net model (Swin-BathyUNet) to understand how it infers depth and when its predictions are trustworthy. A leave-one-band out study ranks spectral importance to the different bands consistent with shallow water optics. We adapt ablation-based CAM to regression (A-CAM-R) and validate the reliability via a performance retention test: keeping only the top-p% salient pixels while neutralizing the rest causes large, monotonic RMSE increase, indicating explanations localize on evidence the model relies on. Attention ablations show decoder conditioned cross attention on skips is an effective upgrade, improving robustness to glint/foam. Cross-region inference (train on one site, test on another) reveals depth-dependent degradation: MAE rises nearly linearly with depth, and bimodal depth distributions exacerbate mid/deep errors. Practical guidance follows: maintain wide receptive fields, preserve radiometric fidelity in green/blue channels, pre-filter bright high variance near shore, and pair light target site fine tuning with depth aware calibration to transfer across regions.

Method: Mixed precision framework for PointPillars: 1) PTQ-based layer sensitivity search to identify top-k sensitive layers kept as FP, 2) Greedy search for mixed precision layer combinations, 3) Finalization with PTQ or QAT, 4) Small calibration data strategy to handle outliers by reducing outlier likelihood.

Result: PTQ pipeline produces mixed precision models without training, QAT pipeline achieves performance competitive to FP models. TensorRT deployment reduces latency by up to 2.35x and model size by up to 2.26x.

Conclusion: The proposed mixed precision quantization framework effectively addresses LIDAR’s numerical challenges, enabling efficient real-time 3D object detection with significant speed and size improvements while maintaining accuracy.

Abstract: LIDAR 3D object detection is one of the important tasks for autonomous vehicles. Ensuring that this task operates in real-time is crucial. Toward this, model quantization can be used to accelerate the runtime. However, directly applying model quantization often leads to performance degradation due to LIDAR’s wide numerical distributions and extreme outliers. To address the wide numerical distribution, we proposed a mixed precision framework designed for PointPillars. Our framework first searches for sensitive layers with post-training quantization (PTQ) by quantizing one layer at a time to 8-bit integer (INT8) and evaluating each model for average precision (AP). The top-k most sensitive layers are assigned as floating point (FP). Combinations of these layers are greedily searched to produce candidate mixed precision models, which are finalized with either PTQ or quantization-aware training (QAT). Furthermore, to handle outliers, we observe that using a very small number of calibration data reduces the likelihood of encountering outliers, thereby improving PTQ performance. Our methods provides mixed precision models without training in the PTQ pipeline, while our QAT pipeline achieves the performance competitive to FP models. With TensorRT deployment, our models offer less latency and sizes by up to 2.35 and 2.26 times, respectively.

Method: The study performs centralized Bayesian hyperparameter optimization on individual cancer imaging datasets (ovarian and colorectal cancers), then transfers dataset-specific optima to non-IID FL setups. The key innovation is a simple cross-dataset aggregation heuristic that combines configurations by averaging learning rates and considering modal optimizers and batch sizes.

Result: The combined configuration achieved competitive classification performance, demonstrating that hyperparameters optimized on one cancer imaging dataset can generalize effectively across non-IID federated scenarios.

Conclusion: The proposed aggregation heuristic enables effective hyperparameter transfer in FL for cancer histopathology, addressing privacy constraints while maintaining competitive performance across different cancer types and non-IID data distributions.

Abstract: Deep learning for cancer histopathology training conflicts with privacy constraints in clinical settings. Federated Learning (FL) mitigates this by keeping data local; however, its performance depends on hyperparameter choices under non-independent and identically distributed (non-IID) client datasets. This paper examined whether hyperparameters optimized on one cancer imaging dataset generalized across non-IID federated scenarios. We considered binary histopathology tasks for ovarian and colorectal cancers. We perform centralized Bayesian hyperparameter optimization and transfer dataset-specific optima to the non-IID FL setup. The main contribution of this study is the introduction of a simple cross-dataset aggregation heuristic by combining configurations by averaging the learning rates and considering the modal optimizers and batch sizes. This combined configuration achieves a competitive classification performance.

Method: Proposes a framework using neural implicit representations for depth and BRDF modeling under mono-chromaticity assumption (uniform chromaticity and homogeneous material). This alleviates ill-posedness of color photometric stereo and enables single-shot reconstruction. Also designs a compact optical tactile sensor for validation.

Result: Experiments on both synthetic and real-world datasets demonstrate that the method achieves accurate and robust surface reconstruction from just one image.

Conclusion: The proposed framework successfully extends color photometric stereo to more practical conditions (near-light, non-Lambertian) using neural implicit representations and mono-chromaticity assumption, enabling single-shot surface reconstruction validated by optical tactile sensing.

Abstract: Color photometric stereo enables single-shot surface reconstruction, extending conventional photometric stereo that requires multiple images of a static scene under varying illumination to dynamic scenarios. However, most existing approaches assume ideal distant lighting and Lambertian reflectance, leaving more practical near-light conditions and non-Lambertian surfaces underexplored. To overcome this limitation, we propose a framework that leverages neural implicit representations for depth and BRDF modeling under the assumption of mono-chromaticity (uniform chromaticity and homogeneous material), which alleviates the inherent ill-posedness of color photometric stereo and allows for detailed surface recovery from just one image. Furthermore, we design a compact optical tactile sensor to validate our approach. Experiments on both synthetic and real-world datasets demonstrate that our method achieves accurate and robust surface reconstruction.

Method: Evaluated convolutional neural networks in federated learning setting, comparing models trained on original vs preprocessed MRI images (resizing, grayscale conversion, normalization, filtering, histogram equalization). Tested preprocessing alone and combined with test-time augmentation.

Result: Preprocessing alone yielded negligible gains. Combined with test-time augmentation, it delivered consistent, statistically significant improvements in federated MRI classification (p<0.001).

Conclusion: Test-time augmentation should be the default inference strategy in FL-based medical imaging. When computational budget permits, pairing TTA with light preprocessing provides additional reliable gains.

Abstract: Efficient brain tumor diagnosis is crucial for early treatment; however, it is challenging because of lesion variability and image complexity. We evaluated convolutional neural networks (CNNs) in a federated learning (FL) setting, comparing models trained on original versus preprocessed MRI images (resizing, grayscale conversion, normalization, filtering, and histogram equalization). Preprocessing alone yielded negligible gains; combined with test-time augmentation (TTA), it delivered consistent, statistically significant improvements in federated MRI classification (p<0.001). In practice, TTA should be the default inference strategy in FL-based medical imaging; when the computational budget permits, pairing TTA with light preprocessing provides additional reliable gains.

Method: VILTA integrates a VLM directly into closed-loop AD training. The VLM actively participates by comprehending the dynamic driving environment and strategically generating challenging scenarios through direct, fine-grained editing of surrounding agents’ future trajectories.

Result: The approach substantially enhances the safety and robustness of AD policies, particularly in navigating critical long-tail events, by creating a diverse curriculum of plausible yet challenging scenarios beyond traditional methods.

Conclusion: VILTA’s direct-editing approach fully leverages VLM’s powerful generalization capabilities to address the long-tail problem in autonomous driving, overcoming limitations of existing two-stage frameworks and producing more diverse and effective training scenarios.

Abstract: The safe deployment of autonomous driving (AD) systems is fundamentally hindered by the long-tail problem, where rare yet critical driving scenarios are severely underrepresented in real-world data. Existing solutions including safety-critical scenario generation and closed-loop learning often rely on rule-based heuristics, resampling methods and generative models learned from offline datasets, limiting their ability to produce diverse and novel challenges. While recent works leverage Vision Language Models (VLMs) to produce scene descriptions that guide a separate, downstream model in generating hazardous trajectories for agents, such two-stage framework constrains the generative potential of VLMs, as the diversity of the final trajectories is ultimately limited by the generalization ceiling of the downstream algorithm. To overcome these limitations, we introduce VILTA (VLM-In-the-Loop Trajectory Adversary), a novel framework that integrates a VLM into the closed-loop training of AD agents. Unlike prior works, VILTA actively participates in the training loop by comprehending the dynamic driving environment and strategically generating challenging scenarios through direct, fine-grained editing of surrounding agents’ future trajectories. This direct-editing approach fully leverages the VLM’s powerful generalization capabilities to create a diverse curriculum of plausible yet challenging scenarios that extend beyond the scope of traditional methods. We demonstrate that our approach substantially enhances the safety and robustness of the resulting AD policy, particularly in its ability to navigate critical long-tail events.

Method: 1) Multi-exposure image fusion for thermal radiation: decompose images into positive/negative channels, parallel processing, optimize via multi-exposure fusion. 2) Heat haze reduction: use FSIM as objective function for iterative optimization, apply grayscale average algorithm to equalize anomalous gray values.

Result: Effective computation area increased from 26% to 50% for under-exposed images and 32% to 40% for over-exposed images. Static thermal deformation errors reduced: ε_xx by 85.3%, ε_yy by 36.0%, γ_xy by 36.4%.

Conclusion: Proposed fusion-restoration methods effectively suppress thermal radiation and heat haze interference, improve DIC measurement accuracy for high-temperature deformation with practical application value.

Abstract: In the deformation measurement of high-temperature structures, image degradation caused by thermal radiation and random errors introduced by heat haze restrict the accuracy and effectiveness of deformation measurement. To suppress thermal radiation and heat haze using fusion-restoration image processing methods, thereby improving the accuracy and effectiveness of DIC in the measurement of high-temperature deformation. For image degradation caused by thermal radiation, based on the image layered representation, the image is decomposed into positive and negative channels for parallel processing, and then optimized for quality by multi-exposure image fusion. To counteract the high-frequency, random errors introduced by heat haze, we adopt the FSIM as the objective function to guide the iterative optimization of model parameters, and the grayscale average algorithm is applied to equalize anomalous gray values, thereby reducing measurement error. The proposed multi-exposure image fusion algorithm effectively suppresses image degradation caused by complex illumination conditions, boosting the effective computation area from 26% to 50% for under-exposed images and from 32% to 40% for over-exposed images without degrading measurement accuracy in the experiment. Meanwhile, the image restoration combined with the grayscale average algorithm reduces static thermal deformation measurement errors. The error in ε_xx is reduced by 85.3%, while the errors in ε_yy and γ_xy are reduced by 36.0% and 36.4%, respectively. We present image processing methods to suppress the interference of thermal radiation and heat haze in high-temperature deformation measurement using DIC. The experimental results verify that the proposed method can effectively improve image quality, reduce deformation measurement errors, and has potential application value in thermal deformation measurement.

Method: IVGF framework reconstructs scene geometry from multimodal 2D inputs using 3D Gaussian splatting. Includes cross-modal adjustment (CMA) module to modulate Gaussian opacity for resolving cross-modal conflicts, and fusion loss to guide CMA optimization for preserving modality-specific features.

Result: Comprehensive qualitative and quantitative experiments demonstrate the effectiveness of the proposed method in producing high-quality fused images.

Conclusion: The IVGF framework successfully addresses limitations of traditional 2D fusion methods by enabling 3D scene reconstruction and direct rendering of fused images while preserving critical characteristics from both infrared and visible modalities.

Abstract: Infrared-visible image fusion aims to integrate infrared and visible information into a single fused image. Existing 2D fusion methods focus on fusing images from fixed camera viewpoints, neglecting a comprehensive understanding of complex scenarios, which results in the loss of critical information about the scene. To address this limitation, we propose a novel Infrared-Visible Gaussian Fusion (IVGF) framework, which reconstructs scene geometry from multimodal 2D inputs and enables direct rendering of fused images. Specifically, we propose a cross-modal adjustment (CMA) module that modulates the opacity of Gaussians to solve the problem of cross-modal conflicts. Moreover, to preserve the distinctive features from both modalities, we introduce a fusion loss that guides the optimization of CMA, thus ensuring that the fused image retains the critical characteristics of each modality. Comprehensive qualitative and quantitative experiments demonstrate the effectiveness of the proposed method.

Method: P2L-CA integrates a Prompt-to-Label module (using class-specific prompts to disentangle multi-label representations with linguistic priors) and a Continuous Adapter module (lightweight adapters to mitigate domain gaps between pre-trained models and downstream tasks).

Result: Extensive experiments on MS-COCO and PASCAL VOC show P2L-CA achieves substantial improvements over state-of-the-art methods, demonstrates strong generalization in CIL scenarios, requires minimal trainable parameters, and eliminates memory buffers.

Conclusion: P2L-CA provides an effective parameter-efficient solution for multi-label class-incremental learning that addresses computational cost, storage overhead, and feature confusion issues while maintaining strong performance.

Abstract: Multi-label Class-Incremental Learning aims to continuously recognize novel categories in complex scenes where multiple objects co-occur. However, existing approaches often incur high computational costs due to full-parameter fine-tuning and substantial storage overhead from memory buffers, or they struggle to address feature confusion and domain discrepancies adequately. To overcome these limitations, we introduce P2L-CA, a parameter-efficient framework that integrates a Prompt-to-Label module with a Continuous Adapter module. The P2L module leverages class-specific prompts to disentangle multi-label representations while incorporating linguistic priors to enforce stable semantic-visual alignment. Meanwhile, the CA module employs lightweight adapters to mitigate domain gaps between pre-trained models and downstream tasks, thereby enhancing model plasticity. Extensive experiments across standard and challenging MLCIL settings on MS-COCO and PASCAL VOC show that P2L-CA not only achieves substantial improvements over state-of-the-art methods but also demonstrates strong generalization in CIL scenarios, all while requiring minimal trainable parameters and eliminating the need for memory buffers.

Method: Proposes RSOD teacher-student framework: 1) Calculates reliability scores by assessing teacher prediction consistency across different views, 2) Introduces object mixed pseudo-label method to address labeled data shortage, 3) Implements reliability-guided adaptive constraint to optimize student performance by leveraging unlabeled data.

Result: On UATD dataset, using only 5% labeled data, RSOD achieves results competitive with baseline algorithm trained on 100% labeled data. Authors also collected a new dataset to provide more valuable data for sonar research.

Conclusion: RSOD effectively mitigates the impact of limited labels in sonar image object detection by fully learning sonar image characteristics and developing appropriate pseudo-label strategies, enabling good performance even with extremely limited labeled data.

Abstract: Object detection in sonar images is a key technology in underwater detection systems. Compared to natural images, sonar images contain fewer texture details and are more susceptible to noise, making it difficult for non-experts to distinguish subtle differences between classes. This leads to their inability to provide precise annotation data for sonar images. Therefore, designing effective object detection methods for sonar images with extremely limited labels is particularly important. To address this, we propose a teacher-student framework called RSOD, which aims to fully learn the characteristics of sonar images and develop a pseudo-label strategy suitable for these images to mitigate the impact of limited labels. First, RSOD calculates a reliability score by assessing the consistency of the teacher’s predictions across different views. To leverage this score, we introduce an object mixed pseudo-label method to tackle the shortage of labeled data in sonar images. Finally, we optimize the performance of the student by implementing a reliability-guided adaptive constraint. By taking full advantage of unlabeled data, the student can perform well even in situations with extremely limited labels. Notably, on the UATD dataset, our method, using only 5% of labeled data, achieves results that can compete against those of our baseline algorithm trained on 100% labeled data. We also collected a new dataset to provide more valuable data for research in the field of sonar.

Method: Introduces S2DiT with: 1) LinConv Hybrid Attention (LCHA) and Stride Self-Attention (SSA) for efficient token processing, 2) sandwich design discovered via budget-aware dynamic programming search, and 3) 2-in-1 distillation framework to transfer knowledge from large teacher models to compact few-step models.

Result: Achieves video generation quality comparable to state-of-the-art server models while streaming at over 10 FPS on an iPhone, making real-time mobile video generation feasible.

Conclusion: S2DiT successfully bridges the gap between high-quality video generation and mobile deployment, enabling efficient streaming video generation on resource-constrained devices without sacrificing quality.

Abstract: Diffusion Transformers (DiTs) have recently improved video generation quality. However, their heavy computational cost makes real-time or on-device generation infeasible. In this work, we introduce S2DiT, a Streaming Sandwich Diffusion Transformer designed for efficient, high-fidelity, and streaming video generation on mobile hardware. S2DiT generates more tokens but maintains efficiency with novel efficient attentions: a mixture of LinConv Hybrid Attention (LCHA) and Stride Self-Attention (SSA). Based on this, we uncover the sandwich design via a budget-aware dynamic programming search, achieving superior quality and efficiency. We further propose a 2-in-1 distillation framework that transfers the capacity of large teacher models (e.g., Wan 2.2-14B) to the compact few-step sandwich model. Together, S2DiT achieves quality on par with state-of-the-art server video models, while streaming at over 10 FPS on an iPhone.

Method: 1) Residual-guided complementary fusion: Anchors representations in DINOv2 feature space while injecting CLIP semantics through learned residual correction. 2) Vector of Local Aggregated Queries (VLAQ): A query-residual global aggregation scheme that encodes local tokens by their residual responses to learnable queries, improving stability and preserving fine-grained discriminative cues.

Result: Extensive experiments on standard VPR benchmarks (Pitts30k, Tokyo24/7, MSLS, Nordland, SPED, AmsterTime) show DC-VLAQ consistently outperforms strong baselines and achieves state-of-the-art performance, particularly under challenging domain shifts and long-term appearance changes.

Conclusion: DC-VLAQ successfully addresses the challenge of fusing complementary visual foundation models while maintaining aggregation stability, resulting in robust global representations for visual place recognition that excel under difficult conditions like domain shifts and appearance changes.

Abstract: One of the central challenges in visual place recognition (VPR) is learning a robust global representation that remains discriminative under large viewpoint changes, illumination variations, and severe domain shifts. While visual foundation models (VFMs) provide strong local features, most existing methods rely on a single model, overlooking the complementary cues offered by different VFMs. However, exploiting such complementary information inevitably alters token distributions, which challenges the stability of existing query-based global aggregation schemes. To address these challenges, we propose DC-VLAQ, a representation-centric framework that integrates the fusion of complementary VFMs and robust global aggregation. Specifically, we first introduce a lightweight residual-guided complementary fusion that anchors representations in the DINOv2 feature space while injecting complementary semantics from CLIP through a learned residual correction. In addition, we propose the Vector of Local Aggregated Queries (VLAQ), a query–residual global aggregation scheme that encodes local tokens by their residual responses to learnable queries, resulting in improved stability and the preservation of fine-grained discriminative cues. Extensive experiments on standard VPR benchmarks, including Pitts30k, Tokyo24/7, MSLS, Nordland, SPED, and AmsterTime, demonstrate that DC-VLAQ consistently outperforms strong baselines and achieves state-of-the-art performance, particularly under challenging domain shifts and long-term appearance changes.

Method: Harnesses pre-trained 3D prior of LRM and incorporates FLAME-based 2D Gaussian Splatting into LRM’s rendering pipeline. Projects LRM’s triplane features into FLAME parameter space for geometry recovery, and models appearance via 2D Gaussian primitives coupled to the FLAME mesh.

Result: Achieves superior reconstruction accuracy and cross-view consistency on both controlled and in-the-wild benchmarks, while existing methods remain sensitive to viewpoint variations.

Conclusion: KaoLRM successfully leverages LRM’s rich 3D prior to create a FLAME regressor aware of 3D structure, enabling accurate and robust facial reconstructions under challenging conditions like self-occlusions and diverse viewpoints.

Abstract: We propose KaoLRM to re-target the learned prior of the Large Reconstruction Model (LRM) for parametric 3D face reconstruction from single-view images. Parametric 3D Morphable Models (3DMMs) have been widely used for facial reconstruction due to their compact and interpretable parameterization, yet existing 3DMM regressors often exhibit poor consistency across varying viewpoints. To address this, we harness the pre-trained 3D prior of LRM and incorporate FLAME-based 2D Gaussian Splatting into LRM’s rendering pipeline. Specifically, KaoLRM projects LRM’s pre-trained triplane features into the FLAME parameter space to recover geometry, and models appearance via 2D Gaussian primitives that are tightly coupled to the FLAME mesh. The rich prior enables the FLAME regressor to be aware of the 3D structure, leading to accurate and robust reconstructions under self-occlusions and diverse viewpoints. Experiments on both controlled and in-the-wild benchmarks demonstrate that KaoLRM achieves superior reconstruction accuracy and cross-view consistency, while existing methods remain sensitive to viewpoint variations. The code is released at https://github.com/CyberAgentAILab/KaoLRM.

Method: Proposes SSPFormer with two key strategies: 1) Inverse frequency projection masking that prioritizes reconstruction of high-frequency anatomical regions for structure-aware learning, and 2) Frequency-weighted FFT noise enhancement that injects physiologically realistic noise into Fourier domain for artifact robustness.

Result: Achieves state-of-the-art performance on segmentation, super-resolution, and denoising tasks, verifying ability to capture fine-grained MRI fidelity and adapt to clinical requirements.

Conclusion: SSPFormer effectively learns domain-specific feature representations from unlabeled MRI data, addressing domain gap and data scarcity through frequency-based self-supervised strategies, enabling domain-invariant and artifact-robust feature learning.

Abstract: The pre-trained transformer demonstrates remarkable generalization ability in natural image processing. However, directly transferring it to magnetic resonance images faces two key challenges: the inability to adapt to the specificity of medical anatomical structures and the limitations brought about by the privacy and scarcity of medical data. To address these issues, this paper proposes a Self-Supervised Pretrained Transformer (SSPFormer) for MRI images, which effectively learns domain-specific feature representations of medical images by leveraging unlabeled raw imaging data. To tackle the domain gap and data scarcity, we introduce inverse frequency projection masking, which prioritizes the reconstruction of high-frequency anatomical regions to enforce structure-aware representation learning. Simultaneously, to enhance robustness against real-world MRI artifacts, we employ frequency-weighted FFT noise enhancement that injects physiologically realistic noise into the Fourier domain. Together, these strategies enable the model to learn domain-invariant and artifact-robust features directly from raw scans. Through extensive experiments on segmentation, super-resolution, and denoising tasks, the proposed SSPFormer achieves state-of-the-art performance, fully verifying its ability to capture fine-grained MRI image fidelity and adapt to clinical application requirements.

Method: 1) Train a diffusion model for motion perception, shifting from image-to-video generation to video-to-dense-tracking. 2) Construct a motion-labeled dataset to identify features encoding the strongest motion information. 3) Inject these motion features into a structurally identical video generation model, enabling zero-shot motion transfer without additional adapters.

Result: The proposed Moaw framework enables motion transfer by leveraging the homogeneity between motion perception and video generation networks, allowing natural adaptation of motion features in a zero-shot manner.

Conclusion: This work provides a new paradigm for bridging generative modeling and motion understanding, paving the way for more unified and controllable video learning frameworks that can leverage the inherent motion awareness of video diffusion models.

Abstract: Video diffusion models, trained on large-scale datasets, naturally capture correspondences of shared features across frames. Recent works have exploited this property for tasks such as optical flow prediction and tracking in a zero-shot setting. Motivated by these findings, we investigate whether supervised training can more fully harness the tracking capability of video diffusion models. To this end, we propose Moaw, a framework that unleashes motion awareness for video diffusion models and leverages it to facilitate motion transfer. Specifically, we train a diffusion model for motion perception, shifting its modality from image-to-video generation to video-to-dense-tracking. We then construct a motion-labeled dataset to identify features that encode the strongest motion information, and inject them into a structurally identical video generation model. Owing to the homogeneity between the two networks, these features can be naturally adapted in a zero-shot manner, enabling motion transfer without additional adapters. Our work provides a new paradigm for bridging generative modeling and motion understanding, paving the way for more unified and controllable video learning frameworks.

Method: Proposes DSOD with: 1) Unified Feature Injection (UFI) module integrating VFM features into CNN backbone via Simple-Scale Extension (SSE) and Domain-aware Adaptive Weighting (DAAW); 2) Semantic-aware Feature Regularization (SAFR) to prevent overfitting to source domain; 3) VFM-free variant DSOD-distill using Dual-Teacher distillation for computation-restricted scenarios.

Result: Achieves 48.1% AP on Normal-to-Foggy weather adaptation, 39.3% AP on Cross-scene adaptation, and 61.4% AP on Synthetic-to-Real adaptation, outperforming state-of-the-art SFOD methods across multiple benchmarks.

Conclusion: DSOD effectively mitigates source bias in source-free object detection by leveraging VFM features and regularization techniques, demonstrating superior performance across diverse domain adaptation scenarios while offering a computationally efficient variant for resource-constrained environments.

Abstract: Source-Free Object Detection (SFOD) has garnered much attention in recent years by eliminating the need of source-domain data in cross-domain tasks, but existing SFOD methods suffer from the Source Bias problem, i.e. the adapted model remains skewed towards the source domain, leading to poor generalization and error accumulation during self-training. To overcome this challenge, we propose Debiased Source-free Object Detection (DSOD), a novel VFM-assisted SFOD framework that can effectively mitigate source bias with the help of powerful VFMs. Specifically, we propose Unified Feature Injection (UFI) module that integrates VFM features into the CNN backbone through Simple-Scale Extension (SSE) and Domain-aware Adaptive Weighting (DAAW). Then, we propose Semantic-aware Feature Regularization (SAFR) that constrains feature learning to prevent overfitting to source domain characteristics. Furthermore, we propose a VFM-free variant, termed DSOD-distill for computation-restricted scenarios through a novel Dual-Teacher distillation scheme. Extensive experiments on multiple benchmarks demonstrate that DSOD outperforms state-of-the-art SFOD methods, achieving 48.1% AP on Normal-to-Foggy weather adaptation, 39.3% AP on Cross-scene adaptation, and 61.4% AP on Synthetic-to-Real adaptation.

Method: Two main modules: 1) Spatial Perception Enhancement (SPE) integrates panoramic filtering with specialized door and region experts for spatially coherent representations; 2) Explored Multi-expert Reasoning (EMR) uses parallel LLM experts for waypoint-level semantics and region-level transitions, with query-and-explore mechanism to resolve perceptual ambiguities.

Result: Achieves state-of-the-art performance on VLN-CE using only low-cost LLMs. Introduces value-based waypoint sampling strategy to bridge Sim2Real gap, with extensive real-world evaluations confirming superior generalization and robustness.

Conclusion: Spatial-VLN effectively addresses spatial perception bottlenecks in zero-shot VLN, demonstrating both strong performance in simulation and practical real-world applicability through innovative perception-guided exploration.

Abstract: Zero-shot Vision-and-Language Navigation (VLN) agents leveraging Large Language Models (LLMs) excel in generalization but suffer from insufficient spatial perception. Focusing on complex continuous environments, we categorize key perceptual bottlenecks into three spatial challenges: door interaction,multi-room navigation, and ambiguous instruction execution, where existing methods consistently suffer high failure rates. We present Spatial-VLN, a perception-guided exploration framework designed to overcome these challenges. The framework consists of two main modules. The Spatial Perception Enhancement (SPE) module integrates panoramic filtering with specialized door and region experts to produce spatially coherent, cross-view consistent perceptual representations. Building on this foundation, our Explored Multi-expert Reasoning (EMR) module uses parallel LLM experts to address waypoint-level semantics and region-level spatial transitions. When discrepancies arise between expert predictions, a query-and-explore mechanism is activated, prompting the agent to actively probe critical areas and resolve perceptual ambiguities. Experiments on VLN-CE demonstrate that Spatial VLN achieves state-of-the-art performance using only low-cost LLMs. Furthermore, to validate real-world applicability, we introduce a value-based waypoint sampling strategy that effectively bridges the Sim2Real gap. Extensive real-world evaluations confirm that our framework delivers superior generalization and robustness in complex environments. Our codes and videos are available at https://yueluhhxx.github.io/Spatial-VLN-web/.

Method: 1) Model 3D head avatars with Gaussian primitives embedded on parametric face model in UV space. 2) Leverage pretrained 3D GAN for global full-head feature extraction and multi-view supervision. 3) Fuse local fine-grained input image features with global textures using UV space symmetry.

Result: Achieves high-quality 3D full-head modeling and real-time animation, improving realism of 3D talking avatars. Handles large camera pose variations effectively.

Conclusion: Proposed framework enables one-shot 3D full-head animatable avatar reconstruction in single feed-forward pass, supporting real-time animation and 360° rendering with improved realism.

Abstract: Building 3D animatable head avatars from a single image is an important yet challenging problem. Existing methods generally collapse under large camera pose variations, compromising the realism of 3D avatars. In this work, we propose a new framework to tackle the novel setting of one-shot 3D full-head animatable avatar reconstruction in a single feed-forward pass, enabling real-time animation and simultaneous 360$^\circ$ rendering views. To facilitate efficient animation control, we model 3D head avatars with Gaussian primitives embedded on the surface of a parametric face model within the UV space. To obtain knowledge of full-head geometry and textures, we leverage rich 3D full-head priors within a pretrained 3D generative adversarial network (GAN) for global full-head feature extraction and multi-view supervision. To increase the fidelity of the 3D reconstruction of the input image, we take advantage of the symmetric nature of the UV space and human faces to fuse local fine-grained input image features with the global full-head textures. Extensive experiments demonstrate the effectiveness of our method, achieving high-quality 3D full-head modeling as well as real-time animation, thereby improving the realism of 3D talking avatars.

Method: RetCLIP constructs a masked segment feature database from paired image-text data. At inference, masked segment features from input images query this database to retrieve similar features and associated class labels. Classification scores are based on query-retrieval similarity and combined with CLIP-based scores for final output.

Result: When trained on COCO, RetCLIP achieves 30.9 PQ, 19.3 mAP, 44.0 mIoU on ADE20k dataset, representing +4.5 PQ, +2.5 mAP, +10.0 mIoU absolute improvement over baseline (FC-CLIP).

Conclusion: Retrieval-augmented panoptic segmentation significantly improves performance on unseen classes, demonstrating the effectiveness of combining retrieval-based classification with CLIP scores for open-vocabulary segmentation tasks.

Abstract: Given an input image and set of class names, panoptic segmentation aims to label each pixel in an image with class labels and instance labels. In comparison, Open Vocabulary Panoptic Segmentation aims to facilitate the segmentation of arbitrary classes according to user input. The challenge is that a panoptic segmentation system trained on a particular dataset typically does not generalize well to unseen classes beyond the training data. In this work, we propose RetCLIP, a retrieval-augmented panoptic segmentation method that improves the performance of unseen classes. In particular, we construct a masked segment feature database using paired image-text data. At inference time, we use masked segment features from the input image as query keys to retrieve similar features and associated class labels from the database. Classification scores for the masked segment are assigned based on the similarity between query features and retrieved features. The retrieval-based classification scores are combined with CLIP-based scores to produce the final output. We incorporate our solution with a previous SOTA method (FC-CLIP). When trained on COCO, the proposed method demonstrates 30.9 PQ, 19.3 mAP, 44.0 mIoU on the ADE20k dataset, achieving +4.5 PQ, +2.5 mAP, +10.0 mIoU absolute improvement over the baseline.

Method: Proposes SKANet - a cognitive deep learning framework with dual-stream architecture integrating Time-Frequency Images (TFIs) and Power Spectral Density (PSD). Uses Multi-Branch Selective Kernel (SK) module combined with Asymmetric Convolution Blocks (ACBs) to dynamically adjust receptive fields, acting as an adaptive filter. Also integrates Squeeze-and-Excitation (SE) mechanism at fusion stage to adaptively recalibrate heterogeneous feature contributions.

Result: Achieves 96.99% overall accuracy on dataset of 405,000 samples. Demonstrates superior robustness for compound jamming classification, particularly under low Jamming-to-Noise Ratio (JNR) regimes.

Conclusion: SKANet effectively addresses the challenge of compound interference classification by dynamically adapting receptive fields to capture both micro-scale transient features and macro-scale spectral trends, outperforming conventional methods through its cognitive dual-stream architecture.

Abstract: As the electromagnetic environment becomes increasingly complex, Global Navigation Satellite Systems (GNSS) face growing threats from sophisticated jamming interference. Although Deep Learning (DL) effectively identifies basic interference, classifying compound interference remains difficult due to the superposition of diverse jamming sources. Existing single-domain approaches often suffer from performance degradation because transient burst signals and continuous global signals require conflicting feature extraction scales. We propose the Selective Kernel and Asymmetric convolution Network(SKANet), a cognitive deep learning framework built upon a dual-stream architecture that integrates Time-Frequency Images (TFIs) and Power Spectral Density (PSD). Distinct from conventional fusion methods that rely on static receptive fields, the proposed architecture incorporates a Multi-Branch Selective Kernel (SK) module combined with Asymmetric Convolution Blocks (ACBs). This mechanism enables the network to dynamically adjust its receptive fields, acting as an adaptive filter that simultaneously captures micro-scale transient features and macro-scale spectral trends within entangled compound signals. To complement this spatial-temporal adaptation, a Squeeze-and-Excitation (SE) mechanism is integrated at the fusion stage to adaptively recalibrate the contribution of heterogeneous features from each modality. Evaluations on a dataset of 405,000 samples demonstrate that SKANet achieves an overall accuracy of 96.99%, exhibiting superior robustness for compound jamming classification, particularly under low Jamming-to-Noise Ratio (JNR) regimes.

Method: Uses Jensen-Shannon divergence to measure sample cleanliness likelihood considering nearest neighbors. Implements self-adaptive, data-driven thresholding for per-class selection. Clean samples use conventional training, in-distribution noisy samples use partial label learning, and out-of-distribution noisy samples use negative learning. Adds triplet consistency regularization for self-prediction, neighbor-prediction, and feature consistency.

Result: Extensive experiments on various benchmark datasets and comprehensive ablation studies demonstrate the effectiveness and superiority of the approach over existing state-of-the-art methods.

Conclusion: Jo-SNC effectively addresses label noise challenges by jointly handling sample selection and model regularization, particularly addressing imbalances in label noise and out-of-distribution noisy data through a comprehensive consistency-based framework.

Abstract: Label noise is pervasive in various real-world scenarios, posing challenges in supervised deep learning. Deep networks are vulnerable to such label-corrupted samples due to the memorization effect. One major stream of previous methods concentrates on identifying clean data for training. However, these methods often neglect imbalances in label noise across different mini-batches and devote insufficient attention to out-of-distribution noisy data. To this end, we propose a noise-robust method named Jo-SNC (\textbf{Jo}int sample selection and model regularization based on \textbf{S}elf- and \textbf{N}eighbor-\textbf{C}onsistency). Specifically, we propose to employ the Jensen-Shannon divergence to measure the ``likelihood’’ of a sample being clean or out-of-distribution. This process factors in the nearest neighbors of each sample to reinforce the reliability of clean sample identification. We design a self-adaptive, data-driven thresholding scheme to adjust per-class selection thresholds. While clean samples undergo conventional training, detected in-distribution and out-of-distribution noisy samples are trained following partial label learning and negative learning, respectively. Finally, we advance the model performance further by proposing a triplet consistency regularization that promotes self-prediction consistency, neighbor-prediction consistency, and feature consistency. Extensive experiments on various benchmark datasets and comprehensive ablation studies demonstrate the effectiveness and superiority of our approach over existing state-of-the-art methods.

Method: Introduces PhyG-MoE framework with spectrum-based gating mechanism that routes signals based on spectral feature entanglement. Uses high-capacity TransNeXt expert for complex saturated scenarios and lightweight experts for fundamental signals to minimize latency.

Result: Achieves 97.58% overall accuracy on 21 jamming categories. Significantly reduces computational overhead without performance degradation by dynamically matching model capacity to signal complexity.

Conclusion: PhyG-MoE resolves the conflict between static computing and dynamic electromagnetic environments, offering a viable solution for resource-constrained cognitive receivers in GNSS interference recognition.

Abstract: Complex electromagnetic interference increasingly compromises Global Navigation Satellite Systems (GNSS), threatening the reliability of Space-Air-Ground Integrated Networks (SAGIN). Although deep learning has advanced interference recognition, current static models suffer from a \textbf{fundamental limitation}: they impose a fixed computational topology regardless of the input’s physical entropy. This rigidity leads to severe resource mismatch, where simple primitives consume the same processing cost as chaotic, saturated mixtures. To resolve this, this paper introduces PhyG-MoE (Physics-Guided Mixture-of-Experts), a framework designed to \textbf{dynamically align model capacity with signal complexity}. Unlike static architectures, the proposed system employs a spectrum-based gating mechanism that routes signals based on their spectral feature entanglement. A high-capacity TransNeXt expert is activated on-demand to disentangle complex features in saturated scenarios, while lightweight experts handle fundamental signals to minimize latency. Evaluations on 21 jamming categories demonstrate that PhyG-MoE achieves an overall accuracy of 97.58%. By resolving the intrinsic conflict between static computing and dynamic electromagnetic environments, the proposed framework significantly reduces computational overhead without performance degradation, offering a viable solution for resource-constrained cognitive receivers.

Method: Created a controllable 1D image-text testbed with lightweight Transformer encoders trained end-to-end on paired descriptions of one- and two-object scenes. Used CLIP-style contrastive objective and systematically varied label and layout diversity. Performed attention decomposition to analyze mechanisms.

Result: Contrastive training learns left-right relations, with label diversity being the primary driver of generalization rather than layout diversity. Attention analysis shows interactions between positional and token embeddings induce horizontal attention gradients that break left-right symmetry.

Conclusion: Provides mechanistic insight into when and how CLIP-style models acquire relational competence, showing that positional embeddings play a crucial role in enabling spatial understanding through attention gradient formation.

Abstract: Spatial understanding remains a key challenge in vision-language models. Yet it is still unclear whether such understanding is truly acquired, and if so, through what mechanisms. We present a controllable 1D image-text testbed to probe how left-right relational understanding emerges in Transformer-based vision and text encoders trained with a CLIP-style contrastive objective. We train lightweight Transformer-based vision and text encoders end-to-end on paired descriptions of one- and two-object scenes and evaluate generalization to unseen object pairs while systematically varying label and layout diversity. We find that contrastive training learns left-right relations and that label diversity, more than layout diversity, is the primary driver of generalization in this setting. To gain the mechanistic understanding, we perform an attention decomposition and show that interactions between positional and token embeddings induce a horizontal attention gradient that breaks left-right symmetry in the encoders; ablating this contribution substantially reduces left-right discrimination. Our results provide a mechanistic insight of when and how CLIP-style models acquire relational competence.

Method: Integrates semantic learning into compression pipeline with lightweight implicit neural representation-based hyperprior for efficient entropy coding of color and semantic attributes. Uses compression-guided segmentation learning with quantization-aware training and quality-aware weighting to suppress unreliable Gaussian primitives.

Result: Extensive experiments on LERF and 3D-OVS datasets show significant reduction in transmission cost while preserving high rendering quality and strong segmentation performance.

Conclusion: Proposes first unified framework for joint rate-distortion-optimized compression and segmentation of 3DGS, enabling efficient transmission for decoder-side applications with improved performance over independent approaches.

Abstract: We present the first unified framework for rate-distortion-optimized compression and segmentation of 3D Gaussian Splatting (3DGS). While 3DGS has proven effective for both real-time rendering and semantic scene understanding, prior works have largely treated these tasks independently, leaving their joint consideration unexplored. Inspired by recent advances in rate-distortion-optimized 3DGS compression, this work integrates semantic learning into the compression pipeline to support decoder-side applications–such as scene editing and manipulation–that extend beyond traditional scene reconstruction and view synthesis. Our scheme features a lightweight implicit neural representation-based hyperprior, enabling efficient entropy coding of both color and semantic attributes while avoiding costly grid-based hyperprior as seen in many prior works. To facilitate compression and segmentation, we further develop compression-guided segmentation learning, consisting of quantization-aware training to enhance feature separability and a quality-aware weighting mechanism to suppress unreliable Gaussian primitives. Extensive experiments on the LERF and 3D-OVS datasets demonstrate that our approach significantly reduces transmission cost while preserving high rendering quality and strong segmentation performance.

Method: 1) Dual-stream encoders decouple CT and PET representation learning with cross-modal interaction module; 2) Hierarchical context modeling combines local windows with global attention for long-range dependencies; 3) Uses anatomical segmentation masks as semantic anchors for voxel-mask-text alignment during pre-training on 10,000+ patients.

Result: Outperforms strong task-specific and clinical-reference baselines across tumor segmentation, low-dose lesion detection, and multilingual diagnostic report generation while reducing localization errors and hallucinated findings. Enables system-wide metabolic profiling and reveals tumor-associated inter-organ metabolic network interactions.

Conclusion: SDF-HOLO provides a scalable computational foundation for total-body PET/CT diagnostics and system-level precision oncology, moving beyond focal interpretation to enable holistic analysis of system-wide metabolic interactions.

Abstract: Total-body PET/CT enables system-wide molecular imaging, but heterogeneous anatomical and metabolic signals, approximately 2 m axial coverage, and structured radiology semantics challenge existing medical AI models that assume single-modality inputs, localized fields of view, and coarse image-text alignment. We introduce SDF-HOLO (Systemic Dual-stream Fusion Holo Model), a multimodal foundation model for holistic total-body PET/CT, pre-trained on more than 10,000 patients. SDF-HOLO decouples CT and PET representation learning with dual-stream encoders and couples them through a cross-modal interaction module, allowing anatomical context to refine PET aggregation while metabolic saliency guides subtle morphological reasoning. To model long-range dependencies across the body, hierarchical context modeling combines efficient local windows with global attention. To bridge voxels and clinical language, we use anatomical segmentation masks as explicit semantic anchors and perform voxel-mask-text alignment during pre-training. Across tumor segmentation, low-dose lesion detection, and multilingual diagnostic report generation, SDF-HOLO outperforms strong task-specific and clinical-reference baselines while reducing localization errors and hallucinated findings. Beyond focal interpretation, the model enables system-wide metabolic profiling and reveals tumor-associated fingerprints of inter-organ metabolic network interactions, providing a scalable computational foundation for total-body PET/CT diagnostics and system-level precision oncology.

Method: TreeDGS leverages 3D Gaussian Splatting as a continuous, densifiable scene representation. After SfM-MVS initialization and Gaussian optimization, it extracts dense point sets using RaDe-GS’s depth-aware cumulative-opacity integration, associates each sample with multi-view opacity reliability scores, isolates trunk points, and estimates DBH using opacity-weighted solid-circle fitting.

Result: Evaluated on 10 plots with field-measured DBH, TreeDGS achieves 4.79cm RMSE (about 2.6 pixels at this GSD), outperforming a state-of-the-art LiDAR baseline (7.91cm RMSE).

Conclusion: Densified splat-based geometry enables accurate, low-cost aerial DBH measurement, demonstrating that 3D Gaussian Splatting can overcome the limitations of conventional reconstruction methods for sparse aerial observations of natural objects.

Abstract: Aerial remote sensing enables efficient large-area surveying, but accurate direct object-level measurement remains difficult in complex natural scenes. Recent advancements in 3D vision, particularly learned radiance-field representations such as NeRF and 3D Gaussian Splatting, have begun to raise the ceiling on reconstruction fidelity and densifiable geometry from posed imagery. Nevertheless, direct aerial measurement of important natural attributes such as tree diameter at breast height (DBH) remains challenging. Trunks in aerial forest scans are distant and sparsely observed in image views: at typical operating altitudes, stems may span only a few pixels. With these constraints, conventional reconstruction methods leave breast-height trunk geometry weakly constrained. We present TreeDGS, an aerial image reconstruction method that leverages 3D Gaussian Splatting as a continuous, densifiable scene representation for trunk measurement. After SfM-MVS initialization and Gaussian optimization, we extract a dense point set from the Gaussian field using RaDe-GS’s depth-aware cumulative-opacity integration and associate each sample with a multi-view opacity reliability score. We then estimate DBH from trunk-isolated points using opacity-weighted solid-circle fitting. Evaluated on 10 plots with field-measured DBH, TreeDGS reaches 4.79,cm RMSE (about 2.6 pixels at this GSD) and outperforms a state-of-the-art LiDAR baseline (7.91,cm RMSE), demonstrating that densified splat-based geometry can enable accurate, low-cost aerial DBH measurement.

Method: Evaluated five architectures (ResNet-50, ResNet-101, DenseNet-161, EfficientNet-B0, ViT-Base-Patch16-224) on IQ-OTH/NCCD lung cancer dataset. Introduced quantitative evaluation framework combining localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability across architectures.

Result: Grad-CAM effectively highlights salient tumor regions in most convolutional networks, but interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior. Cross-model comparisons show substantial variability in saliency localization, indicating Grad-CAM explanations may not always correspond to true diagnostic evidence used by networks.

Conclusion: Exposes critical limitations of current saliency-based XAI approaches in medical imaging, emphasizing need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Urges cautious adoption of visual explanation tools and rethinking what it means to “trust” a model’s explanation.

Abstract: Explainable Artificial Intelligence (XAI) techniques, such as Gradient-weighted Class Activation Mapping (Grad-CAM), have become indispensable for visualizing the reasoning process of deep neural networks in medical image analysis. Despite their popularity, the faithfulness and reliability of these heatmap-based explanations remain under scrutiny. This study critically investigates whether Grad-CAM truly represents the internal decision-making of deep models trained for lung cancer image classification. Using the publicly available IQ-OTH/NCCD dataset, we evaluate five representative architectures: ResNet-50, ResNet-101, DenseNet-161, EfficientNet-B0, and ViT-Base-Patch16-224, to explore model-dependent variations in Grad-CAM interpretability. We introduce a quantitative evaluation framework that combines localization accuracy, perturbation-based faithfulness, and explanation consistency to assess Grad-CAM reliability across architectures. Experimental findings reveal that while Grad-CAM effectively highlights salient tumor regions in most convolutional networks, its interpretive fidelity significantly degrades for Vision Transformer models due to non-local attention behavior. Furthermore, cross-model comparisons indicate substantial variability in saliency localization, implying that Grad-CAM explanations may not always correspond to the true diagnostic evidence used by the networks. This work exposes critical limitations of current saliency-based XAI approaches in medical imaging and emphasizes the need for model-aware interpretability methods that are both computationally sound and clinically meaningful. Our findings aim to inspire a more cautious and rigorous adoption of visual explanation tools in medical AI, urging the community to rethink what it truly means to “trust” a model’s explanation.

Method: Proposes FGTBT framework with two key components: 1) Fine-Grained Multi-Task Balancing loss (FMB-loss) that assigns weights to individual landmarks based on dataset occurrence for effective unified training, and 2) Frequency-Guided Structure-Aware (FGSA) model using frequency-guided structure injection and regularization to learn facial structure constraints.

Result: Extensive experiments on popular benchmark datasets show the proposed FGTBT framework achieves performance comparable to state-of-the-art methods.

Conclusion: The integration of FMB-loss and FGSA model in the FGTBT framework effectively addresses challenges in facial landmark detection, particularly in difficult scenarios, through frequency-domain modeling and improved multi-dataset training strategies.

Abstract: Recently, deep learning based facial landmark detection (FLD) methods have achieved considerable success. However, in challenging scenarios such as large pose variations, illumination changes, and facial expression variations, they still struggle to accurately capture the geometric structure of the face, resulting in performance degradation. Moreover, the limited size and diversity of existing FLD datasets hinder robust model training, leading to reduced detection accuracy. To address these challenges, we propose a Frequency-Guided Task-Balancing Transformer (FGTBT), which enhances facial structure perception through frequency-domain modeling and multi-dataset unified training. Specifically, we propose a novel Fine-Grained Multi-Task Balancing loss (FMB-loss), which moves beyond coarse task-level balancing by assigning weights to individual landmarks based on their occurrence across datasets. This enables more effective unified training and mitigates the issue of inconsistent gradient magnitudes. Additionally, a Frequency-Guided Structure-Aware (FGSA) model is designed to utilize frequency-guided structure injection and regularization to help learn facial structure constraints. Extensive experimental results on popular benchmark datasets demonstrate that the integration of the proposed FMB-loss and FGSA model into our FGTBT framework achieves performance comparable to state-of-the-art methods. The code is available at https://github.com/Xi0ngxinyu/FGTBT.

Method: 1) Discover that vanilla CLIP has intrinsic defensive capabilities against adversarial examples from different CLIP architectures (proxy adversarial robustness). 2) Propose Heterogeneous Proxy Transfer (HPT) framework to establish cross-architectural robustness distillation channels. 3) Design Generalization-Pivot Decoupling (GPD) using learning rate scheduling differences to separate generalization maintenance from robustness transfer, preventing overfitting.

Result: Extensive experiments on 15 zero-shot datasets demonstrate the effectiveness of HPT-GPD in achieving equilibrium between natural generalization and adversarial robustness without expensive adversarial training.

Conclusion: The proposed HPT-GPD framework successfully enables adversarial robustness transfer for vision-language models by leveraging proxy robustness between different CLIP architectures while maintaining zero-shot generalization capabilities, offering a computationally efficient alternative to traditional adversarial training.

Abstract: As a pivotal technique for improving the defense of deep models, adversarial robustness transfer via distillation has demonstrated remarkable success in conventional image classification tasks. However, this paradigm encounters critical challenges when applied to vision-language models (VLM) (e.g., CLIP): constructing adversarially robust teacher for large-scale multi-modal models demands prohibitively high computational resources. We bridge this gap by revealing an interesting phenomenon: vanilla CLIP (without adversarial training) exhibits intrinsic defensive capabilities against adversarial examples generated by another CLIP with different architectures. We formally define this as proxy adversarial robustness, and naturally propose a Heterogeneous Proxy Transfer (HPT) framework that establishes cross-architectural robustness distillation channels between CLIP variants, effortlessly enabling the VLM robustness transfer from proxy to target models. Yet, such proxy transfer paradigm easily induces severe overfitting, leading to a sharp degradation in zero-shot natural generalization. To resolve that, we design Generalization-Pivot Decoupling (GPD) by leveraging the difference in learning rate scheduling. This decouples the proxy transfer process into a generalization-anchored warm-up that maintains generalization and a generalization-pulled HPT that promotes adversarial robustness, to achieve an equilibrium between natural generalization and adversarial robustness. Extensive experiments on 15 zero-shot datasets demonstrate the effectiveness of our HPT-GPD method. The code is available at the website of github.com/fxw13/HPT-GPD.

Method: THFEM framework combines AD-THG models with SPFEM, using adjacent frame learning strategy to finetune AD-THG models for predicting consecutive frames to improve realism.

Result: The framework effectively preserves mouth shapes during expression manipulations, demonstrating benefits of integrating AD-THG with SPFEM.

Conclusion: Integration of audio-driven talking head generation with speech-preserving facial expression manipulation successfully addresses lip synchronization challenges in facial expression manipulation.

Abstract: Speech-Preserving Facial Expression Manipulation (SPFEM) is an innovative technique aimed at altering facial expressions in images and videos while retaining the original mouth movements. Despite advancements, SPFEM still struggles with accurate lip synchronization due to the complex interplay between facial expressions and mouth shapes. Capitalizing on the advanced capabilities of audio-driven talking head generation (AD-THG) models in synthesizing precise lip movements, our research introduces a novel integration of these models with SPFEM. We present a new framework, Talking Head Facial Expression Manipulation (THFEM), which utilizes AD-THG models to generate frames with accurately synchronized lip movements from audio inputs and SPFEM-altered images. However, increasing the number of frames generated by AD-THG models tends to compromise the realism and expression fidelity of the images. To counter this, we develop an adjacent frame learning strategy that finetunes AD-THG models to predict sequences of consecutive frames. This strategy enables the models to incorporate information from neighboring frames, significantly improving image quality during testing. Our extensive experimental evaluations demonstrate that this framework effectively preserves mouth shapes during expression manipulations, highlighting the substantial benefits of integrating AD-THG with SPFEM.

Method: YOLO26 introduces three key innovations: 1) MuSGD optimizer for stabilizing lightweight backbones, 2) STAL (small-target-aware assignment) for better small object detection, and 3) ProgLoss for dynamic supervision, enabling native end-to-end learning without NMS.

Result: YOLO26 establishes a new Pareto front, outperforming predecessors and state-of-the-art competitors (RTMDet, DAMO-YOLO) in both inference speed and detection accuracy, resolving the historical trade-off between latency and precision.

Conclusion: By decoupling representation learning from heuristic post-processing, YOLO26 represents the next evolutionary step in edge-based computer vision, successfully eliminating the NMS bottleneck that constrained previous YOLO iterations.

Abstract: The “You Only Look Once” (YOLO) framework has long served as the benchmark for real-time object detection, yet traditional iterations (YOLOv1 through YOLO11) remain constrained by the latency and hyperparameter sensitivity of Non-Maximum Suppression (NMS) post-processing. This paper analyzes a comprehensive analysis of YOLO26, an architecture that fundamentally redefines this paradigm by eliminating NMS in favor of a native end-to-end learning strategy. This study examines the critical innovations that enable this transition, specifically the introduction of the MuSGD optimizer for stabilizing lightweight backbones, STAL for small-target-aware assignment, and ProgLoss for dynamic supervision. Through a systematic review of official performance benchmarks, the results demonstrate that YOLO26 establishes a new Pareto front, outperforming a comprehensive suite of predecessors and state-of-the-art competitors (including RTMDet and DAMO-YOLO) in both inference speed and detection accuracy. The analysis confirms that by decoupling representation learning from heuristic post-processing, YOLOv26 successfully resolves the historical trade-off between latency and precision, signaling the next evolutionary step in edge-based computer vision.

Method: Developed an Ensemble Deep Learning framework integrating VGG16, ResNet50, and InceptionV3 with optimized weighted averaging. Trained on a comprehensive dataset of 10,516 expert-annotated images from 18 farms across India, Brazil, and the USA for simultaneous LSD and FMD detection.

Result: Achieved state-of-the-art accuracy of 98.2%, with macro-averaged precision of 98.2%, recall of 98.1%, F1-score of 98.1%, and AUC-ROC of 99.5%. Successfully addressed symptom overlap challenge in multi-disease detection.

Conclusion: The framework enables early, precise, and automated diagnosis of LSD and FMD, with potential to enhance disease management, support global agricultural sustainability, and designed for future deployment in resource-limited settings.

Abstract: Lumpy Skin Disease (LSD) and Foot-and-Mouth Disease (FMD) are highly contagious viral diseases affecting cattle, causing significant economic losses and welfare challenges. Their visual diagnosis is complicated by significant symptom overlap with each other and with benign conditions like insect bites or chemical burns, hindering timely control measures. Leveraging a comprehensive dataset of 10,516 expert-annotated images from 18 farms across India, Brazil, and the USA, this study presents a novel Ensemble Deep Learning framework integrating VGG16, ResNet50, and InceptionV3 with optimized weighted averaging for simultaneous LSD and FMD detection. The model achieves a state-of-the-art accuracy of 98.2%, with macro-averaged precision of 98.2%, recall of 98.1%, F1-score of 98.1%, and an AUC-ROC of 99.5%. This approach uniquely addresses the critical challenge of symptom overlap in multi-disease detection, enabling early, precise, and automated diagnosis. This tool has the potential to enhance disease management, support global agricultural sustainability, and is designed for future deployment in resource-limited settings.

Method: The model integrates a Swin Transformer backbone with Feature Pyramid Network (FPN) and UNet-style decoder, enhanced with Convolutional Block Attention Module (CBAM). It employs a dual-head architecture for joint optimization of detection (binary classification) and segmentation (localization) tasks using uncertainty-weighted multi-task learning.

Result: Extensive experiments on FantasyIDiap dataset show 84.31% accuracy, 90.78% AUC for classification, 57.24% mean Dice score for localization, and 88.61% F1-score for binary classification. The model maintains computational efficiency suitable for real-world deployment through FastAPI implementation.

Conclusion: The proposed TwoHead-SwinFPN architecture effectively addresses the growing threat of synthetic manipulations in ID documents by providing simultaneous detection and precise localization capabilities, with comprehensive evaluation across 10 languages and 3 acquisition devices demonstrating robust performance.

Abstract: The proliferation of sophisticated generative AI models has significantly escalated the threat of synthetic manipulations in identity documents, particularly through face swapping and text inpainting attacks. This paper presents TwoHead-SwinFPN, a unified deep learning architecture that simultaneously performs binary classification and precise localization of manipulated regions in ID documents. Our approach integrates a Swin Transformer backbone with Feature Pyramid Network (FPN) and UNet-style decoder, enhanced with Convolutional Block Attention Module (CBAM) for improved feature representation. The model employs a dual-head architecture for joint optimization of detection and segmentation tasks, utilizing uncertainty-weighted multi-task learning. Extensive experiments on the FantasyIDiap dataset demonstrate superior performance with 84.31% accuracy, 90.78% AUC for classification, and 57.24% mean Dice score for localization. The proposed method achieves an F1-score of 88.61% for binary classification while maintaining computational efficiency suitable for real-world deployment through FastAPI implementation. Our comprehensive evaluation includes ablation studies, cross-device generalization analysis, and detailed performance assessment across 10 languages and 3 acquisition devices.

Method: Proposes Supervision-by-Hallucination-and-Transfer (SHT) with two modules: Dual Hallucination Learning Network (DHLN) for learning high-resolution representations from low-resolution inputs, and Facial Pose Transfer Network (FPTN) for improving landmark heatmaps through pose transformation.

Result: Experimental results show the method surpasses state-of-the-art techniques in both face hallucination and facial landmark detection tasks.

Conclusion: This is the first weakly-supervised FLD framework integrating face hallucination and facial pose transfer, demonstrating improved robustness and precision for facial landmark detection.

Abstract: High-precision facial landmark detection (FLD) relies on high-resolution deep feature representations. However, low-resolution face images or the compression (via pooling or strided convolution) of originally high-resolution images hinder the learning of such features, thereby reducing FLD accuracy. Moreover, insufficient training data and imprecise annotations further degrade performance. To address these challenges, we propose a weakly-supervised framework called Supervision-by-Hallucination-and-Transfer (SHT) for more robust and precise FLD. SHT contains two novel mutually enhanced modules: Dual Hallucination Learning Network (DHLN) and Facial Pose Transfer Network (FPTN). By incorporating FLD and face hallucination tasks, DHLN is able to learn high-resolution representations with low-resolution inputs for recovering both facial structures and local details and generating more effective landmark heatmaps. Then, by transforming faces from one pose to another, FPTN can further improve landmark heatmaps and faces hallucinated by DHLN for detecting more accurate landmarks. To the best of our knowledge, this is the first study to explore weakly-supervised FLD by integrating face hallucination and facial pose transfer tasks. Experimental results of both face hallucination and FLD demonstrate that our method surpasses state-of-the-art techniques.

Method: Dual-Stream Collaborative Transformer (DSCT) with Pattern-Specific Mutual Attention Encoders (PSMAEs) and Dynamic Nomination Decoders (DNDs). PSMAE highlights private information of region and segmentation features by mutual querying. DND dynamically selects relevant learning blocks and exploits homogeneous features between consolidated features.

Result: Outperforms state-of-the-art image captioning models on popular benchmark datasets. First study to fuse different pattern-specific features dynamically for captioning.

Conclusion: DSCT effectively addresses semantic inconsistencies and spatial misalignment by dynamically fusing region and segmentation features, leading to more accurate and descriptive image captions.

Abstract: Current region feature-based image captioning methods have progressed rapidly and achieved remarkable performance. However, they are still prone to generating irrelevant descriptions due to the lack of contextual information and the over-reliance on generated partial descriptions for predicting the remaining words. In this paper, we propose a Dual-Stream Collaborative Transformer (DSCT) to address this issue by introducing the segmentation feature. The proposed DSCT consolidates and then fuses the region and segmentation features to guide the generation of caption sentences. It contains multiple Pattern-Specific Mutual Attention Encoders (PSMAEs) and Dynamic Nomination Decoders (DNDs). The PSMAE effectively highlights and consolidates the private information of two representations by querying each other. The DND dynamically searches for the most relevant learning blocks to the input textual representations and exploits the homogeneous features between the consolidated region and segmentation features to generate more accurate and descriptive caption sentences. To the best of our knowledge, this is the first study to explore how to fuse different pattern-specific features in a dynamic way to bypass their semantic inconsistencies and spatial misalignment issues for image captioning. The experimental results from popular benchmark datasets demonstrate that our DSCT outperforms the state-of-the-art image captioning models in the literature.

Method: Developed tailored MINT architectures for object recognition using convolutional layers to capture activation patterns. Experiments involved object detection models, embedding extractors, and MINT modules tested on three public databases with over 174K images.

Result: Achieved precision rates of 70-80% for identifying training data membership, with performance varying based on the depth of detection module layer used as input to the MINT module. Also analyzed factors influencing MINT module performance for more transparent training processes.

Conclusion: The proposed specialized MINT architectures are effective for membership inference in object recognition, providing a practical solution with reasonable precision while offering insights into factors that affect membership inference performance and training transparency.

Abstract: In this research, we analyze the performance of Membership Inference Tests (MINT), focusing on determining whether given data were utilized during the training phase, specifically in the domain of object recognition. Within the area of object recognition, we propose and develop architectures tailored for MINT models. These architectures aim to optimize performance and efficiency in data utilization, offering a tailored solution to tackle the complexities inherent in the object recognition domain. We conducted experiments involving an object detection model, an embedding extractor, and a MINT module. These experiments were performed in three public databases, totaling over 174K images. The proposed architecture leverages convolutional layers to capture and model the activation patterns present in the data during the training process. Through our analysis, we are able to identify given data used for testing and training, achieving precision rates ranging between 70% and 80%, contingent upon the depth of the detection module layer chosen for input to the MINT module. Additionally, our studies entail an analysis of the factors influencing the MINT Module, delving into the contributing elements behind more transparent training processes.

Method: QASA decouples slot selection from reconstruction to eliminate mutual constraints, proposes an unsupervised Slot-Quality metric to assess per-slot quality, designs a Quality-Guided Slot Selection scheme that dynamically selects high-quality slots, and uses a gated decoder for reconstruction during training. At inference, token-wise competition yields K-adaptive outcomes.

Result: QASA substantially outperforms existing K-adaptive methods on both real and synthetic datasets, and on real-world datasets it even surpasses K-fixed methods.

Conclusion: The proposed quality-guided approach effectively addresses the limitations of existing K-adaptive slot attention methods, achieving superior performance by providing principled quality assessment and eliminating conflicting optimization objectives.

Abstract: Slot Attention, an approach that binds different objects in a scene to a set of “slots”, has become a leading method in unsupervised object-centric learning. Most methods assume a fixed slot count K, and to better accommodate the dynamic nature of object cardinality, a few works have explored K-adaptive variants. However, existing K-adaptive methods still suffer from two limitations. First, they do not explicitly constrain slot-binding quality, so low-quality slots lead to ambiguous feature attribution. Second, adding a slot-count penalty to the reconstruction objective creates conflicting optimization goals between reducing the number of active slots and maintaining reconstruction fidelity. As a result, they still lag significantly behind strong K-fixed baselines. To address these challenges, we propose Quality-Guided K-Adaptive Slot Attention (QASA). First, we decouple slot selection from reconstruction, eliminating the mutual constraints between the two objectives. Then, we propose an unsupervised Slot-Quality metric to assess per-slot quality, providing a principled signal for fine-grained slot–object binding. Based on this metric, we design a Quality-Guided Slot Selection scheme that dynamically selects a subset of high-quality slots and feeds them into our newly designed gated decoder for reconstruction during training. At inference, token-wise competition on slot attention yields a K-adaptive outcome. Experiments show that QASA substantially outperforms existing K-adaptive methods on both real and synthetic datasets. Moreover, on real-world datasets QASA surpasses K-fixed methods.

Method: Uses diffusion models conditioned on 2D pose, subject surroundings, and scene context. Represents 3D gaze as an additional body joint at fixed distance from eyes, allowing joint denoising with pose during diffusion process to generate multiple plausible hypotheses.

Result: Achieves state-of-the-art performance on three benchmark datasets for 3D gaze estimation, even surpassing methods that use temporal information, demonstrating effectiveness of joint gaze-pose estimation.

Conclusion: GazeD successfully demonstrates that treating gaze as a body joint and jointly estimating it with pose using diffusion models leads to superior 3D gaze estimation from single RGB images, handling uncertainty through multiple plausible hypotheses.

Abstract: We introduce GazeD, a new 3D gaze estimation method that jointly provides 3D gaze and human pose from a single RGB image. Leveraging the ability of diffusion models to deal with uncertainty, it generates multiple plausible 3D gaze and pose hypotheses based on the 2D context information extracted from the input image. Specifically, we condition the denoising process on the 2D pose, the surroundings of the subject, and the context of the scene. With GazeD we also introduce a novel way of representing the 3D gaze by positioning it as an additional body joint at a fixed distance from the eyes. The rationale is that the gaze is usually closely related to the pose, and thus it can benefit from being jointly denoised during the diffusion process. Evaluations across three benchmark datasets demonstrate that GazeD achieves state-of-the-art performance in 3D gaze estimation, even surpassing methods that rely on temporal information. Project details will be available at https://aimagelab.ing.unimore.it/go/gazed.

Method: Introduces StyMam, a Mamba-based generator with two key components: 1) a residual dual-path strip scanning mechanism to efficiently capture local texture features, and 2) a channel-reweighted spatial attention module to model global dependencies.

Result: Extensive experiments show the proposed method outperforms state-of-the-art algorithms in both quality and speed, producing high-quality stylized images without artifacts and disharmonious patterns.

Conclusion: The Mamba-based approach successfully addresses the limitations of both GAN and diffusion-based methods for style transfer, achieving better quality results with faster inference by effectively capturing both local and global dependencies.

Abstract: Image style transfer aims to integrate the visual patterns of a specific artistic style into a content image while preserving its content structure. Existing methods mainly rely on the generative adversarial network (GAN) or stable diffusion (SD). GAN-based approaches using CNNs or Transformers struggle to jointly capture local and global dependencies, leading to artifacts and disharmonious patterns. SD-based methods reduce such issues but often fail to preserve content structures and suffer from slow inference. To address these issues, we revisit GAN and propose a mamba-based generator, termed as StyMam, to produce high-quality stylized images without introducing artifacts and disharmonious patterns. Specifically, we introduce a mamba-based generator with a residual dual-path strip scanning mechanism and a channel-reweighted spatial attention module. The former efficiently captures local texture features, while the latter models global dependencies. Finally, extensive qualitative and quantitative experiments demonstrate that the proposed method outperforms state-of-the-art algorithms in both quality and speed.

Method: Developed a spatial affinity component that can be integrated into existing self-supervised learning frameworks. This component uses HR imagery to learn better representations of MR imagery by leveraging spatial relationships between different resolution levels.

Result: The spatial affinity component was tested on two self-supervised learning frameworks and demonstrated superior performance compared to models pretrained solely on HR or MR images alone.

Conclusion: Incorporating high-resolution imagery through a spatial affinity component enhances self-supervised pretraining for mid-resolution remote sensing tasks, leading to improved representation learning and downstream segmentation performance.

Abstract: Self-supervised pretraining in remote sensing is mostly done using mid-spatial resolution (MR) image datasets due to their high availability. Given the release of high-resolution (HR) datasets, we ask how HR datasets can be included in self-supervised pretraining to enhance MR image representation learning and downstream segmentation performance on MR tasks. We design a spatial affinity component that can be added to existing self-supervised learning frameworks and that uses HR imagery to learn better representations of MR imagery. We test the spatial affinity component on two self-supervised learning frameworks and show that it outperforms models pretrained on HR or MR images alone.

Method: Tabular transformer (TabTrans) architecture processing longitudinal health records and bone-related DXA data from 1,382 Qatari subjects, with SMOTE/SMOTE-ENN resampling for class imbalance, compared against conventional ML and generative AI models (Claude 3.5, GPT-4, Gemini Pro).

Result: TabTrans achieved ROC AUC ≥ 79.7% for T2DM prediction, outperforming both generative AI and conventional ML models. Key predictors identified: visceral adipose tissue (VAT) mass/volume, ward BMD/BMC, T/Z-scores, and L1-L4 scores.

Conclusion: TabTrans shows significant potential for analyzing complex tabular healthcare data, providing a powerful tool for proactive T2DM management and personalized clinical interventions in the Qatari population.

Abstract: This study introduces a novel approach for early Type 2 Diabetes Mellitus (T2DM) risk prediction using a tabular transformer (TabTrans) architecture to analyze longitudinal patient data. By processing patients longitudinal health records and bone-related tabular data, our model captures complex, long-range dependencies in disease progression that conventional methods often overlook. We validated our TabTrans model on a retrospective Qatar BioBank (QBB) cohort of 1,382 subjects, comprising 725 men (146 diabetic, 579 healthy) and 657 women (133 diabetic, 524 healthy). The study integrated electronic health records (EHR) with dual-energy X-ray absorptiometry (DXA) data. To address class imbalance, we employed SMOTE and SMOTE-ENN resampling techniques. The proposed models performance is evaluated against conventional machine learning (ML) and generative AI models, including Claude 3.5 Sonnet (Anthropics constitutional AI), GPT-4 (OpenAIs generative pre-trained transformer), and Gemini Pro (Google`s multimodal language model). Our TabTrans model demonstrated superior predictive performance, achieving ROC AUC $\geq$ 79.7 % for T2DM prediction compared to both generative AI models and conventional ML approaches. Feature interpretation analysis identified key risk indicators, with visceral adipose tissue (VAT) mass and volume, ward bone mineral density (BMD) and bone mineral content (BMC), T and Z-scores, and L1-L4 scores emerging as the most important predictors associated with diabetes development in Qatari adults. These findings demonstrate the significant potential of TabTrans for analyzing complex tabular healthcare data, providing a powerful tool for proactive T2DM management and personalized clinical interventions in the Qatari population. Index Terms: tabular transformers, multimodal data, DXA data, diabetes, T2DM, feature interpretation, tabular data

Method: AsyncBEV estimates 2D flow from BEV features of two sensor modalities considering known time offsets, then warps and spatially aligns feature maps. It can be integrated into various BEV detector architectures (grid-based and token-based).

Result: AsyncBEV significantly improves robustness against both small and large asynchrony between LiDAR or camera sensors. It outperforms ego motion compensated baselines by 16.6% and 11.9% NDS on dynamic objects in worst-case 0.5s time offset scenarios.

Conclusion: AsyncBEV provides an effective trainable solution to sensor asynchrony in multi-modal 3D perception, enhancing robustness for dynamic object detection in autonomous driving systems.

Abstract: In autonomous driving, multi-modal perception tasks like 3D object detection typically rely on well-synchronized sensors, both at training and inference. However, despite the use of hardware- or software-based synchronization algorithms, perfect synchrony is rarely guaranteed: Sensors may operate at different frequencies, and real-world factors such as network latency, hardware failures, or processing bottlenecks often introduce time offsets between sensors. Such asynchrony degrades perception performance, especially for dynamic objects. To address this challenge, we propose AsyncBEV, a trainable lightweight and generic module to improve the robustness of 3D Birds’ Eye View (BEV) object detection models against sensor asynchrony. Inspired by scene flow estimation, AsyncBEV first estimates the 2D flow from the BEV features of two different sensor modalities, taking into account the known time offset between these sensor measurements. The predicted feature flow is then used to warp and spatially align the feature maps, which we show can easily be integrated into different current BEV detector architectures (e.g., BEV grid-based and token-based). Extensive experiments demonstrate AsyncBEV improves robustness against both small and large asynchrony between LiDAR or camera sensors in both the token-based CMT and grid-based UniBEV, especially for dynamic objects. We significantly outperform the ego motion compensated CMT and UniBEV baselines, notably by $16.6$ % and $11.9$ % NDS on dynamic objects in the worst-case scenario of a $0.5 s$ time offset. Code will be released upon acceptance.

Method: Think3D leverages 3D reconstruction models to recover point clouds and camera poses from images/videos, enabling VLM agents to actively manipulate space through camera-based operations and ego/global-view switching, transforming spatial reasoning into an interactive 3D chain-of-thought process.

Result: Think3D significantly improves spatial reasoning performance: +7.8% average gains on BLINK Multi-view and MindCube, +4.7% on VSI-Bench for advanced models like GPT-4.1 and Gemini 2.5 Pro. Smaller models benefit from RL policy for viewpoint selection, increasing tool usage benefit from +0.7% to +6.8%.

Conclusion: Training-free, tool-augmented spatial exploration is a viable path toward more flexible and human-like 3D reasoning in multimodal agents, establishing a new dimension of multimodal intelligence.

Abstract: Understanding and reasoning about the physical world requires spatial intelligence: the ability to interpret geometry, perspective, and spatial relations beyond 2D perception. While recent vision large models (VLMs) excel at visual understanding, they remain fundamentally 2D perceivers and struggle with genuine 3D reasoning. We introduce Think3D, a framework that enables VLM agents to think with 3D space. By leveraging 3D reconstruction models that recover point clouds and camera poses from images or videos, Think3D allows the agent to actively manipulate space through camera-based operations and ego/global-view switching, transforming spatial reasoning into an interactive 3D chain-of-thought process. Without additional training, Think3D significantly improves the spatial reasoning performance of advanced models such as GPT-4.1 and Gemini 2.5 Pro, yielding average gains of +7.8% on BLINK Multi-view and MindCube, and +4.7% on VSI-Bench. We further show that smaller models, which struggle with spatial exploration, benefit significantly from a reinforcement learning policy that enables the model to select informative viewpoints and operations. With RL, the benefit from tool usage increases from +0.7% to +6.8%. Our findings demonstrate that training-free, tool-augmented spatial exploration is a viable path toward more flexible and human-like 3D reasoning in multimodal agents, establishing a new dimension of multimodal intelligence. Code and weights are released at https://github.com/zhangzaibin/spagent.

Method: The dataset pairs high-density LiDAR point clouds with high-resolution oblique imagery, annotated into 11 semantic classes. The authors provide predefined splits, mIoU metrics, and establish unimodal (LiDAR-only, image-only) and multi-modal fusion baselines for comparison.

Result: Fusion models outperform the best unimodal baseline by +5.55 mIoU, demonstrating the complementarity of geometric (LiDAR) and appearance (imagery) information for 3D semantic segmentation of electrical infrastructure.

Conclusion: GridNet-HD fills an important gap in available datasets for power-line asset analysis, showing that multi-modal fusion significantly improves segmentation performance over unimodal approaches, with the dataset, baselines, and codes made publicly available.

Abstract: This paper presents GridNet-HD, a multi-modal dataset for 3D semantic segmentation of overhead electrical infrastructures, pairing high-density LiDAR with high-resolution oblique imagery. The dataset comprises 7,694 images and 2.5 billion points annotated into 11 classes, with predefined splits and mIoU metrics. Unimodal (LiDAR-only, image-only) and multi-modal fusion baselines are provided. On GridNet-HD, fusion models outperform the best unimodal baseline by +5.55 mIoU, highlighting the complementarity of geometry and appearance. As reviewed in Sec. 2, no public dataset jointly provides high-density LiDAR and high-resolution oblique imagery with 3D semantic labels for power-line assets. Dataset, baselines, and codes are available: https://huggingface.co/collections/heig-vd-geo/gridnet-hd.

Method: Proposes a dual-branch prototype learning network integrating Retinex theory with few-shot learning. Uses Retinex-based reflectance components for illumination-invariant global representation learning, and metric learning to reduce dependence on large annotated datasets. Includes cross-similarity prior mask generation module to compute high-dimensional similarities between query and support features, and multi-scale feature enhancement module that fuses multi-scale features with prior mask to alleviate spatial inconsistency.

Result: Extensive experiments on multiple benchmarks demonstrate consistent state-of-the-art performance under low-light conditions.

Conclusion: The proposed approach effectively addresses low-light crack segmentation challenges by combining illumination-invariant representation learning with few-shot learning, reducing annotation burden while maintaining high accuracy in challenging lighting conditions.

Abstract: Crack detection is critical for concrete infrastructure safety, but real-world cracks often appear in low-light environments like tunnels and bridge undersides, degrading computer vision segmentation accuracy. Pixel-level annotation of low-light crack images is extremely time-consuming, yet most deep learning methods require large, well-illuminated datasets. We propose a dual-branch prototype learning network integrating Retinex theory with few-shot learning for low-light crack segmentation. Retinex-based reflectance components guide illumination-invariant global representation learning, while metric learning reduces dependence on large annotated datasets. We introduce a cross-similarity prior mask generation module that computes high-dimensional similarities between query and support features to capture crack location and structure, and a multi-scale feature enhancement module that fuses multi-scale features with the prior mask to alleviate spatial inconsistency. Extensive experiments on multiple benchmarks demonstrate consistent state-of-the-art performance under low-light conditions. Code: https://github.com/YulunGuo/CrackFSS.

Method: HyperAdapt encodes clinically relevant attributes (age, sex) into compact embeddings that condition a hypernetwork-style module. This module generates small residual modulation parameters for selected layers of a shared backbone model, preserving general medical knowledge while enabling patient-specific adjustments. Adaptations are constrained through low-rank and bottlenecked parameterizations for efficiency and robustness.

Result: Experiments across multiple public medical imaging benchmarks show consistent improvement in subgroup-level performance without sacrificing overall accuracy. On the PAD-UFES-20 dataset, HyperAdapt outperforms the strongest competing baseline by 4.1% in recall and 4.4% in F1 score, with larger gains for underrepresented patient populations.

Conclusion: HyperAdapt provides an effective approach for subgroup-aware medical AI models that incorporates patient context rather than suppressing it, improving reliability across diverse patient populations while maintaining diagnostic accuracy and efficiency.

Abstract: AI models for medical diagnosis often exhibit uneven performance across patient populations due to heterogeneity in disease prevalence, imaging appearance, and clinical risk profiles. Existing algorithmic fairness approaches typically seek to reduce such disparities by suppressing sensitive attributes. However, in medical settings these attributes often carry essential diagnostic information, and removing them can degrade accuracy and reliability, particularly in high-stakes applications. In contrast, clinical decision making explicitly incorporates patient context when interpreting diagnostic evidence, suggesting a different design direction for subgroup-aware models. In this paper, we introduce HyperAdapt, a patient-conditioned adaptation framework that improves subgroup reliability while maintaining a shared diagnostic model. Clinically relevant attributes such as age and sex are encoded into a compact embedding and used to condition a hypernetwork-style module, which generates small residual modulation parameters for selected layers of a shared backbone. This design preserves the general medical knowledge learned by the backbone while enabling targeted adjustments that reflect patient-specific variability. To ensure efficiency and robustness, adaptations are constrained through low-rank and bottlenecked parameterizations, limiting both model complexity and computational overhead. Experiments across multiple public medical imaging benchmarks demonstrate that the proposed approach consistently improves subgroup-level performance without sacrificing overall accuracy. On the PAD-UFES-20 dataset, our method outperforms the strongest competing baseline by 4.1% in recall and 4.4% in F1 score, with larger gains observed for underrepresented patient populations.

Method: Uses a revised U-Net-like architecture with Convolutional Block Attention Modules and residual paths (Residual U-Net Blocks with Attention), trained with modified triplet loss and curriculum learning-inspired hard negative mining for stability.

Result: Outperforms original keypoint descriptors on HPatches, MegaDepth-1500, and Image Matching Challenge 2021 benchmarks. Also introduces new urban 4K dataset where SANDesc shows substantial gains over existing descriptors with limited computational resources.

Conclusion: SANDesc provides an effective, lightweight solution for descriptor extraction that enhances matching performance when combined with existing keypoint detectors, demonstrating strong results across multiple benchmarks with only 2.4M parameters.

Abstract: We introduce SANDesc, a Streamlined Attention-Based Network for Descriptor extraction that aims to improve on existing architectures for keypoint description. Our descriptor network learns to compute descriptors that improve matching without modifying the underlying keypoint detector. We employ a revised U-Net-like architecture enhanced with Convolutional Block Attention Modules and residual paths, enabling effective local representation while maintaining computational efficiency. We refer to the building blocks of our model as Residual U-Net Blocks with Attention. The model is trained using a modified triplet loss in combination with a curriculum learning-inspired hard negative mining strategy, which improves training stability. Extensive experiments on HPatches, MegaDepth-1500, and the Image Matching Challenge 2021 show that training SANDesc on top of existing keypoint detectors leads to improved results on multiple matching tasks compared to the original keypoint descriptors. At the same time, SANDesc has a model complexity of just 2.4 million parameters. As a further contribution, we introduce a new urban dataset featuring 4K images and pre-calibrated intrinsics, designed to evaluate feature extractors. On this benchmark, SANDesc achieves substantial performance gains over the existing descriptors while operating with limited computational resources.

Method: Single-shot, optimization-free framework that directly modulates the phase in the VAE latent frequency domain, analyzing four different modulation variants.

Result: Thousands of times faster than optimization-based techniques while achieving state-of-the-art resilience against severe attacks including regeneration, without degrading image quality.

Conclusion: PhaseMark demonstrates a new paradigm where efficient, resilient watermarking is achieved by exploiting intrinsic latent properties of diffusion models.

Abstract: The proliferation of hyper-realistic images from Latent Diffusion Models (LDMs) demands robust watermarking, yet existing post-hoc methods are prohibitively slow due to iterative optimization or inversion processes. We introduce PhaseMark, a single-shot, optimization-free framework that directly modulates the phase in the VAE latent frequency domain. This approach makes PhaseMark thousands of times faster than optimization-based techniques while achieving state-of-the-art resilience against severe attacks, including regeneration, without degrading image quality. We analyze four modulation variants, revealing a clear performance-quality trade-off. PhaseMark demonstrates a new paradigm where efficient, resilient watermarking is achieved by exploiting intrinsic latent properties.

Method: Integrates Vision-Language Models on top of 3D Gaussian Splatting. First identifies pre-captured images correlated with query questions, then adjusts them into novel viewpoints to capture better visual information for VLM reasoning.

Result: Outperforms existing methods on several benchmarks, demonstrating effectiveness of integrating VLM-based reasoning with 3DGS for embodied tasks.

Conclusion: GaussExplorer successfully enables question-driven exploration and reasoning in 3D scenes by combining 3DGS with VLMs, addressing limitations of previous approaches for complex compositional queries.

Abstract: We present GaussExplorer, a framework for embodied exploration and reasoning built on 3D Gaussian Splatting (3DGS). While prior approaches to language-embedded 3DGS have made meaningful progress in aligning simple text queries with Gaussian embeddings, they are generally optimized for relatively simple queries and struggle to interpret more complex, compositional language queries. Alternative studies based on object-centric RGB-D structured memories provide spatial grounding but are constrained by pre-fixed viewpoints. To address these issues, GaussExplorer introduces Vision-Language Models (VLMs) on top of 3DGS to enable question-driven exploration and reasoning within 3D scenes. We first identify pre-captured images that are most correlated with the query question, and subsequently adjust them into novel viewpoints to more accurately capture visual information for better reasoning by VLMs. Experiments show that ours outperforms existing methods on several benchmarks, demonstrating the effectiveness of integrating VLM-based reasoning with 3DGS for embodied tasks.

Method: CLASP leverages CLIP to generate low-level (body parts) and high-level (attributes) semantic pseudo-labels, integrates these into visual representations, and uses a Prompt-Controlled Mixture-of-Experts module to dynamically adapt feature extraction based on task-specific prompts. It employs multi-task pre-training guided by part- and attribute-level pseudo-labels.

Result: Extensive experiments across multiple benchmarks demonstrate that CLASP consistently outperforms existing unsupervised pre-training methods in human-centric visual analysis tasks.

Conclusion: CLASP advances the field of human-centric visual analysis by providing an effective unsupervised pre-training framework that can adapt to varying semantic granularity requirements of different downstream tasks through its innovative use of CLIP-generated pseudo-labels and prompt-controlled MoE architecture.

Abstract: Human-centric visual analysis plays a pivotal role in diverse applications, including surveillance, healthcare, and human-computer interaction. With the emergence of large-scale unlabeled human image datasets, there is an increasing need for a general unsupervised pre-training model capable of supporting diverse human-centric downstream tasks. To achieve this goal, we propose CLASP (CLIP-guided Adaptable Self-suPervised learning), a novel framework designed for unsupervised pre-training in human-centric visual tasks. CLASP leverages the powerful vision-language model CLIP to generate both low-level (e.g., body parts) and high-level (e.g., attributes) semantic pseudo-labels. These multi-level semantic cues are then integrated into the learned visual representations, enriching their expressiveness and generalizability. Recognizing that different downstream tasks demand varying levels of semantic granularity, CLASP incorporates a Prompt-Controlled Mixture-of-Experts (MoE) module. MoE dynamically adapts feature extraction based on task-specific prompts, mitigating potential feature conflicts and enhancing transferability. Furthermore, CLASP employs a multi-task pre-training strategy, where part- and attribute-level pseudo-labels derived from CLIP guide the representation learning process. Extensive experiments across multiple benchmarks demonstrate that CLASP consistently outperforms existing unsupervised pre-training methods, advancing the field of human-centric visual analysis.

Method: 1) Created TVWorld - offline graph-based abstraction of real-world TV navigation; 2) Derived two benchmarks: TVWorld-N (topology-aware navigation) and TVWorld-G (focus-aware grounding); 3) Proposed Topology-Aware Training framework to inject topology awareness into LVLMs; 4) Developed TVTheseus foundation model specialized for TV navigation.

Result: TVTheseus achieves 68.3% success rate on TVWorld-N, surpassing strong closed-source baselines like Gemini 3 Flash and establishing state-of-the-art performance. Benchmarks reveal key limitation: insufficient topology awareness for focus-based, long-horizon TV navigation.

Conclusion: The work fills the gap in RC interaction research, provides comprehensive evaluation benchmarks, identifies topology awareness as critical limitation, and demonstrates that specialized training can significantly improve TV navigation capabilities in LVLMs.

Abstract: Recent large vision-language models (LVLMs) have demonstrated strong potential for device control. However, existing research has primarily focused on point-and-click (PnC) interaction, while remote-control (RC) interaction commonly encountered in everyday TV usage remains largely underexplored. To fill this gap, we introduce \textbf{TVWorld}, an offline graph-based abstraction of real-world TV navigation that enables reproducible and deployment-free evaluation. On this basis, we derive two complementary benchmarks that comprehensively assess TV-use capabilities: \textbf{TVWorld-N} for topology-aware navigation and \textbf{TVWorld-G} for focus-aware grounding. These benchmarks expose a key limitation of existing agents: insufficient topology awareness for focus-based, long-horizon TV navigation. Motivated by this finding, we propose a \emph{Topology-Aware Training} framework that injects topology awareness into LVLMs. Using this framework, we develop \textbf{TVTheseus}, a foundation model specialized for TV navigation. TVTheseus achieves a success rate of $68.3%$ on TVWorld-N, surpassing strong closed-source baselines such as Gemini 3 Flash and establishing state-of-the-art (SOTA) performance. Additional analyses further provide valuable insights into the development of effective TV-use agents.

Method: Uses multi-view captures to create animatable 3D face and dynamic 3D body models rendered with Gaussian splatting. Combines SWinGS++ for body reconstruction and HeadGaS++ for face reconstruction with artifact-free merging. Equips avatar with LLM for conversation, using audio speech to drive facial animation for synchronization.

Result: Developed a complete system demonstrating real-time conversation with photorealistic 3D avatars, achieving precise audio-visual synchronization and artifact-free face-body integration suitable for immersive applications.

Conclusion: ICo3D provides a fully integrated virtual avatar experience supporting oral/written interactions, applicable to gaming, virtual assistance, and personalized education, representing significant advancement in interactive 3D human simulation.

Abstract: This work presents Interactive Conversational 3D Virtual Human (ICo3D), a method for generating an interactive, conversational, and photorealistic 3D human avatar. Based on multi-view captures of a subject, we create an animatable 3D face model and a dynamic 3D body model, both rendered by splatting Gaussian primitives. Once merged together, they represent a lifelike virtual human avatar suitable for real-time user interactions. We equip our avatar with an LLM for conversational ability. During conversation, the audio speech of the avatar is used as a driving signal to animate the face model, enabling precise synchronization. We describe improvements to our dynamic Gaussian models that enhance photorealism: SWinGS++ for body reconstruction and HeadGaS++ for face reconstruction, and provide as well a solution to merge the separate face and body models without artifacts. We also present a demo of the complete system, showcasing several use cases of real-time conversation with the 3D avatar. Our approach offers a fully integrated virtual avatar experience, supporting both oral and written form interactions in immersive environments. ICo3D is applicable to a wide range of fields, including gaming, virtual assistance, and personalized education, among others. Project page: https://ico3d.github.io/

Method: U-Net CNN architecture combined with strategies leveraging anatomical priors and neuroimaging domain knowledge, specifically designed for the SSL3D and FOMO25 challenges.

Result: Ranked first in tracks of both contests; models trained 1-2 orders of magnitude faster and were 10 times smaller than competing transformer-based approaches.

Conclusion: CNN-based approaches with domain-specific priors can outperform transformer-based methods in medical imaging tasks while being significantly more efficient in training time and model size.

Abstract: Developing Foundation Models for medical image analysis is essential to overcome the unique challenges of radiological tasks. The first challenges of this kind for 3D brain MRI, SSL3D and FOMO25, were held at MICCAI 2025. Our solution ranked first in tracks of both contests. It relies on a U-Net CNN architecture combined with strategies leveraging anatomical priors and neuroimaging domain knowledge. Notably, our models trained 1-2 orders of magnitude faster and were 10 times smaller than competing transformer-based approaches. Models are available here: https://github.com/jbanusco/BrainFM4Challenges.

Method: Created GTPred benchmark with 370 globally distributed images spanning over 120 years. Evaluated 15 MLLMs (8 proprietary, 7 open-source) using joint year and hierarchical location sequence matching, plus assessment of intermediate reasoning chains against annotated ground-truth reasoning processes.

Result: Current MLLMs show strong visual perception but limited world knowledge and geo-temporal reasoning capabilities. Incorporating temporal information significantly enhances location inference performance compared to location-only prediction.

Conclusion: Temporal information is crucial for accurate geo-localization and current models need improvement in world knowledge and temporal reasoning. The GTPred benchmark provides a valuable tool for evaluating and advancing geo-temporal prediction capabilities in MLLMs.

Abstract: Geo-localization aims to infer the geographic location where an image was captured using observable visual evidence. Traditional methods achieve impressive results through large-scale training on massive image corpora. With the emergence of multi-modal large language models (MLLMs), recent studies have explored their applications in geo-localization, benefiting from improved accuracy and interpretability. However, existing benchmarks largely ignore the temporal information inherent in images, which can further constrain the location. To bridge this gap, we introduce GTPred, a novel benchmark for geo-temporal prediction. GTPred comprises 370 globally distributed images spanning over 120 years. We evaluate MLLM predictions by jointly considering year and hierarchical location sequence matching, and further assess intermediate reasoning chains using meticulously annotated ground-truth reasoning processes. Experiments on 8 proprietary and 7 open-source MLLMs show that, despite strong visual perception, current models remain limited in world knowledge and geo-temporal reasoning. Results also demonstrate that incorporating temporal information significantly enhances location inference performance.

Method: Proposes Additive U-Net with gated additive connections instead of concatenation. Each skip pathway is scaled by a learnable non-negative scalar, providing explicit control over encoder contributions while avoiding channel inflation.

Result: Achieves competitive PSNR/SSIM on Kodak-17 denoising benchmark at noise levels σ = 15, 25, 50. Shows robustness across kernel schedules and depths. Model learns natural progression from high-frequency to band-pass to low-frequency features without explicit down/up-sampling.

Conclusion: Additive skips offer a lightweight, interpretable alternative to concatenation, enabling efficient design and clearer understanding of multi-scale information transfer in reconstruction networks.

Abstract: Skip connections are central to U-Net architectures for image denoising, but standard concatenation doubles channel dimensionality and obscures information flow, allowing uncontrolled noise transfer. We propose the Additive U-Net, which replaces concatenative skips with gated additive connections. Each skip pathway is scaled by a learnable non-negative scalar, offering explicit and interpretable control over encoder contributions while avoiding channel inflation. Evaluations on the Kodak-17 denoising benchmark show that Additive U-Net achieves competitive PSNR/SSIM at noise levels σ = 15, 25, 50, with robustness across kernel schedules and depths. Notably, effective denoising is achieved even without explicit down/up-sampling or forced hierarchies, as the model naturally learns a progression from high-frequency to band-pass to low-frequency features. These results position additive skips as a lightweight and interpretable alternative to concatenation, enabling both efficient design and a clearer understanding of multi-scale information transfer in reconstruction networks.

Method: Created a 120-participant VR dataset of street-crossing navigation with accurate gaze data, complete object state-space, variable scenario complexities, and rich annotations (panoptic segmentation, depth, vehicle keypoints). Proposed oSIM metric for object-based attention evaluation and developed SUMGraph model using Mamba U-Net architecture with graph representation of critical scene objects (vehicles).

Result: Explicitly optimizing for object-based attention improves oSIM performance and enhances model performance on common metrics. SUMGraph outperforms several state-of-the-art visual attention prediction methods by encoding vehicles in a graph representation.

Conclusion: The work addresses critical gaps in object-based attention research by providing a comprehensive dataset, novel evaluation metric, and improved model architecture. The resources will be publicly released to advance computational visual attention modeling.

Abstract: The object-based nature of human visual attention is well-known in cognitive science, but has only played a minor role in computational visual attention models so far. This is mainly due to a lack of suitable datasets and evaluation metrics for object-based attention. To address these limitations, we present \dataset~ – a novel 120-participant dataset of spatial street-crossing navigation in virtual reality specifically geared to object-based attention evaluations. The uniqueness of the presented dataset lies in the ethical and safety affiliated challenges that make collecting comparable data in real-world environments highly difficult. \dataset~ not only features accurate gaze data and a complete state-space representation of objects in the virtual environment, but it also offers variable scenario complexities and rich annotations, including panoptic segmentation, depth information, and vehicle keypoints. We further propose object-based similarity (oSIM) as a novel metric to evaluate the performance of object-based visual attention models, a previously unexplored performance characteristic. Our evaluations show that explicitly optimising for object-based attention not only improves oSIM performance but also leads to an improved model performance on common metrics. In addition, we present SUMGraph, a Mamba U-Net-based model, which explicitly encodes critical scene objects (vehicles) in a graph representation, leading to further performance improvements over several state-of-the-art visual attention prediction methods. The dataset, code and models will be publicly released.

Method: Created BLEMORE dataset with over 3,000 multimodal clips from 58 actors, covering 6 basic emotions and 10 distinct blends, each with 3 salience configurations (50/50, 70/30, 30/70). Evaluated state-of-the-art video classification approaches on two tasks: emotion presence prediction and relative salience prediction.

Result: Unimodal classifiers achieved up to 29% presence accuracy and 13% salience accuracy. Multimodal methods showed clear improvements: ImageBind + WavLM reached 35% presence accuracy and HiCMAE achieved 18% salience accuracy on validation set. On test set, best models achieved 33% presence accuracy (VideoMAEv2 + HuBERT) and 18% salience accuracy (HiCMAE).

Conclusion: BLEMORE provides a valuable resource for advancing emotion recognition systems that account for the complexity of blended emotion expressions, demonstrating that multimodal approaches significantly outperform unimodal methods for this challenging task.

Abstract: Humans often experience not just a single basic emotion at a time, but rather a blend of several emotions with varying salience. Despite the importance of such blended emotions, most video-based emotion recognition approaches are designed to recognize single emotions only. The few approaches that have attempted to recognize blended emotions typically cannot assess the relative salience of the emotions within a blend. This limitation largely stems from the lack of datasets containing a substantial number of blended emotion samples annotated with relative salience. To address this shortcoming, we introduce BLEMORE, a novel dataset for multimodal (video, audio) blended emotion recognition that includes information on the relative salience of each emotion within a blend. BLEMORE comprises over 3,000 clips from 58 actors, performing 6 basic emotions and 10 distinct blends, where each blend has 3 different salience configurations (50/50, 70/30, and 30/70). Using this dataset, we conduct extensive evaluations of state-of-the-art video classification approaches on two blended emotion prediction tasks: (1) predicting the presence of emotions in a given sample, and (2) predicting the relative salience of emotions in a blend. Our results show that unimodal classifiers achieve up to 29% presence accuracy and 13% salience accuracy on the validation set, while multimodal methods yield clear improvements, with ImageBind + WavLM reaching 35% presence accuracy and HiCMAE 18% salience accuracy. On the held-out test set, the best models achieve 33% presence accuracy (VideoMAEv2 + HuBERT) and 18% salience accuracy (HiCMAE). In sum, the BLEMORE dataset provides a valuable resource to advancing research on emotion recognition systems that account for the complexity and significance of blended emotion expressions.

Method: ConvMambaNet integrates Convolutional Neural Networks (CNNs) with Mamba Structured State Space Model (SSM). The hybrid architecture embeds Mamba-SSM blocks within CNN framework to capture both spatial features and long-range temporal dependencies in EEG signals.

Result: Achieved 99% accuracy on CHB-MIT Scalp EEG dataset. Demonstrated robust performance under severe class imbalance conditions, showing strong generalization capability.

Conclusion: ConvMambaNet offers precise and efficient seizure detection, providing a viable path toward real-time, automated epilepsy monitoring in clinical environments by effectively handling EEG’s temporal complexity.

Abstract: Epilepsy is a chronic neurological disorder marked by recurrent seizures that can severely impact quality of life. Electroencephalography (EEG) remains the primary tool for monitoring neural activity and detecting seizures, yet automated analysis remains challenging due to the temporal complexity of EEG signals. This study introduces ConvMambaNet, a hybrid deep learning model that integrates Convolutional Neural Networks (CNNs) with the Mamba Structured State Space Model (SSM) to enhance temporal feature extraction. By embedding the Mamba-SSM block within a CNN framework, the model effectively captures both spatial and long-range temporal dynamics. Evaluated on the CHB-MIT Scalp EEG dataset, ConvMambaNet achieved a 99% accuracy and demonstrated robust performance under severe class imbalance. These results underscore the model’s potential for precise and efficient seizure detection, offering a viable path toward real-time, automated epilepsy monitoring in clinical environments.

Method: A two-stage, parameterized perturbation model based on semantic decoupling: Stage 1 models global rainfall effects through low-dimensional global modulation to weaken semantic decision boundaries; Stage 2 introduces structured rain variations by explicitly modeling multi-scale raindrop appearance and rainfall-induced illumination changes, optimizing the non-differentiable weather space to induce semantic shifts.

Result: Experiments show that physically plausible, highly constrained weather perturbations can induce substantial semantic misalignment in mainstream VLMs. Ablations confirm that illumination modeling and multi-scale raindrop structures are key drivers of these semantic shifts.

Conclusion: The framework reveals that even realistic weather perturbations pose significant safety and reliability risks for VLMs in real-world deployment, highlighting the need for more robust models that can maintain semantic alignment under adverse weather conditions.

Abstract: Vision-Language Models (VLMs) are trained on image-text pairs collected under canonical visual conditions and achieve strong performance on multimodal tasks. However, their robustness to real-world weather conditions, and the stability of cross-modal semantic alignment under such structured perturbations, remain insufficiently studied. In this paper, we focus on rainy scenarios and introduce the first adversarial framework that exploits realistic weather to attack VLMs, using a two-stage, parameterized perturbation model based on semantic decoupling to analyze rain-induced shifts in decision-making. In Stage 1, we model the global effects of rainfall by applying a low-dimensional global modulation to condition the embedding space and gradually weaken the original semantic decision boundaries. In Stage 2, we introduce structured rain variations by explicitly modeling multi-scale raindrop appearance and rainfall-induced illumination changes, and optimize the resulting non-differentiable weather space to induce stable semantic shifts. Operating in a non-pixel parameter space, our framework generates perturbations that are both physically grounded and interpretable. Experiments across multiple tasks show that even physically plausible, highly constrained weather perturbations can induce substantial semantic misalignment in mainstream VLMs, posing potential safety and reliability risks in real-world deployment. Ablations further confirm that illumination modeling and multi-scale raindrop structures are key drivers of these semantic shifts.

Method: Proposes learning-based 3D semantic segmentation framework using Calyo Pulse solid-state ultrasound sensors. Uses 3D U-Net architecture trained on spatial ultrasound data for volumetric segmentation.

Result: Demonstrates robust segmentation performance from Calyo Pulse sensors. Shows potential for improvement with larger datasets, refined ground truth, and weighted loss functions.

Conclusion: 3D ultrasound sensing is a promising complementary modality for reliable autonomy in harsh environments, offering alternative to traditional LiDAR/camera systems.

Abstract: Developing cost-efficient and reliable perception systems remains a central challenge for automated vehicles. LiDAR and camera-based systems dominate, yet they present trade-offs in cost, robustness and performance under adverse conditions. This work introduces a novel framework for learning-based 3D semantic segmentation using Calyo Pulse, a modular, solid-state 3D ultrasound sensor system for use in harsh and cluttered environments. A 3D U-Net architecture is introduced and trained on the spatial ultrasound data for volumetric segmentation. Results demonstrate robust segmentation performance from Calyo Pulse sensors, with potential for further improvement through larger datasets, refined ground truth, and weighted loss functions. Importantly, this study highlights 3D ultrasound sensing as a promising complementary modality for reliable autonomy.

Method: Proposes Enginuity - an open, large-scale, multi-domain engineering diagram dataset with comprehensive structural annotations that capture hierarchical component relationships, connections, and semantic elements across diverse engineering domains.

Result: The dataset enables multimodal large language models to address critical downstream tasks including structured diagram parsing, cross-modal information retrieval, and AI-assisted engineering simulation.

Conclusion: Enginuity would be transformative for AI for Scientific Discovery by breaking down the fundamental barrier that prevents AI from fully participating in scientific workflows requiring diagram interpretation and visual reasoning.

Abstract: We propose Enginuity - the first open, large-scale, multi-domain engineering diagram dataset with comprehensive structural annotations designed for automated diagram parsing. By capturing hierarchical component relationships, connections, and semantic elements across diverse engineering domains, our proposed dataset would enable multimodal large language models to address critical downstream tasks including structured diagram parsing, cross-modal information retrieval, and AI-assisted engineering simulation. Enginuity would be transformative for AI for Scientific Discovery by enabling artificial intelligence systems to comprehend and manipulate the visual-structural knowledge embedded in engineering diagrams, breaking down a fundamental barrier that currently prevents AI from fully participating in scientific workflows where diagram interpretation, technical drawing analysis, and visual reasoning are essential for hypothesis generation, experimental design, and discovery.

Method: 1) Created CausalSpatial benchmark with four tasks (Collision, Compatibility, Occlusion, Trajectory) to evaluate causal spatial reasoning. 2) Proposed Causal Object World model (COW) framework that externalizes simulation by generating videos of hypothetical dynamics to provide explicit visual cues of causality.

Result: Humans score 84% on CausalSpatial benchmark while GPT-5 achieves only 54%. Analysis reveals MLLMs over-rely on textual chain-of-thought reasoning that drifts from visual evidence, causing spatially ungrounded hallucinations.

Conclusion: MLLMs fundamentally lack causal spatial reasoning capabilities. The COW framework addresses this by grounding reasoning in physical reality through visual simulation rather than linguistic priors, enabling better anticipation of object motion consequences.

Abstract: Humans can look at a static scene and instantly predict what happens next – will moving this object cause a collision? We call this ability Causal Spatial Reasoning. However, current multimodal large language models (MLLMs) cannot do this, as they remain largely restricted to static spatial perception, struggling to answer “what-if” questions in a 3D scene. We introduce CausalSpatial, a diagnostic benchmark evaluating whether models can anticipate consequences of object motions across four tasks: Collision, Compatibility, Occlusion, and Trajectory. Results expose a severe gap: humans score 84% while GPT-5 achieves only 54%. Why do MLLMs fail? Our analysis uncovers a fundamental deficiency: models over-rely on textual chain-of-thought reasoning that drifts from visual evidence, producing fluent but spatially ungrounded hallucinations. To address this, we propose the Causal Object World model (COW), a framework that externalizes the simulation process by generating videos of hypothetical dynamics. With explicit visual cues of causality, COW enables models to ground their reasoning in physical reality rather than linguistic priors. We make the dataset and code publicly available here: https://github.com/CausalSpatial/CausalSpatial

Method: MultiST uses a unified multimodal framework with cross-attention-based fusion to jointly model spatial topology, gene expression, and tissue morphology. It employs graph-based gene encoders with adversarial alignment for robust spatial representations and integrates color-normalized histological features to capture molecular-morphological dependencies.

Result: Tested on 13 diverse ST datasets from human brain cortex and breast cancer tissue, MultiST produces spatial domains with clearer and more coherent boundaries than existing methods, leading to more stable pseudotime trajectories and more biologically interpretable cell-cell interaction patterns.

Conclusion: MultiST provides an effective multimodal framework for spatial transcriptomics analysis that better integrates tissue morphology with molecular profiles, improving spatial domain boundary resolution and biological interpretability.

Abstract: Spatial transcriptomics (ST) enables transcriptome-wide profiling while preserving the spatial context of tissues, offering unprecedented opportunities to study tissue organization and cell-cell interactions in situ. Despite recent advances, existing methods often lack effective integration of histological morphology with molecular profiles, relying on shallow fusion strategies or omitting tissue images altogether, which limits their ability to resolve ambiguous spatial domain boundaries. To address this challenge, we propose MultiST, a unified multimodal framework that jointly models spatial topology, gene expression, and tissue morphology through cross-attention-based fusion. MultiST employs graph-based gene encoders with adversarial alignment to learn robust spatial representations, while integrating color-normalized histological features to capture molecular-morphological dependencies and refine domain boundaries. We evaluated the proposed method on 13 diverse ST datasets spanning two organs, including human brain cortex and breast cancer tissue. MultiST yields spatial domains with clearer and more coherent boundaries than existing methods, leading to more stable pseudotime trajectories and more biologically interpretable cell-cell interaction patterns. The MultiST framework and source code are available at https://github.com/LabJunBMI/MultiST.git.

Method: 1) Methodology for generating controlled multi-level dust concentrations in cluttered environments; 2) New 4D mmWave radar dataset augmented by camera and LiDAR; 3) Threshold-based noise filtering framework using radar parameters (RCS, velocity, azimuth, elevation) to suppress ghost targets and multipath reflections; 4) Cluster-level, rule-based classification pipeline using radar semantics (velocity, RCS, volumetric spread) for pedestrian detection.

Result: Experimental results confirm the integrated approach significantly enhances clutter mitigation, detection robustness, and overall system resilience in dust-laden mining environments, enabling reliable real-time pedestrian detection without extensive domain-specific training.

Conclusion: The proposed methodology and framework successfully address the challenges of mm-wave sensing in dust-laden environments, providing a comprehensive solution for reliable pedestrian detection in harsh industrial settings through controlled dust generation, advanced filtering, and semantic-based classification.

Abstract: This paper introduces a novel methodology for generating controlled, multi-level dust concentrations in a highly cluttered environment representative of harsh, enclosed environments, such as underground mines, road tunnels, or collapsed buildings, enabling repeatable mm-wave propagation studies under severe electromagnetic constraints. We also present a new 4D mmWave radar dataset, augmented by camera and LiDAR, illustrating how dust particles and reflective surfaces jointly impact the sensing functionality. To address these challenges, we develop a threshold-based noise filtering framework leveraging key radar parameters (RCS, velocity, azimuth, elevation) to suppress ghost targets and mitigate strong multipath reflections at the raw data level. Building on the filtered point clouds, a cluster-level, rule-based classification pipeline exploits radar semantics-velocity, RCS, and volumetric spread-to achieve reliable, real-time pedestrian detection without extensive domainspecific training. Experimental results confirm that this integrated approach significantly enhances clutter mitigation, detection robustness, and overall system resilience in dust-laden mining environments.

Method: 1) Spherical Geometry Representation: anchors geometric signals to uniform spherical coordinates, guaranteeing regular point distribution and robust mesh reconstruction. 2) Spherical Geometry Diffusion: a conditional diffusion framework built on the 2D unwrapped sphere map that jointly models geometry and texture, with geometry explicitly conditioning texture synthesis.

Result: The method successfully handles text-to-3D generation, face reconstruction, and text-based 3D editing, substantially outperforming existing methods in geometric quality, textual fidelity, and inference efficiency.

Conclusion: Constraining 3D face geometry to a topological sphere provides a simple yet effective solution for high-quality text-to-3D face generation, enabling robust mesh reconstruction and seamless integration with powerful 2D generative models.

Abstract: A fundamental challenge in text-to-3D face generation is achieving high-quality geometry. The core difficulty lies in the arbitrary and intricate distribution of vertices in 3D space, making it challenging for existing models to establish clean connectivity and resulting in suboptimal geometry. To address this, our core insight is to simplify the underlying geometric structure by constraining the distribution onto a simple and regular manifold, a topological sphere. Building on this, we first propose the Spherical Geometry Representation, a novel face representation that anchors geometric signals to uniform spherical coordinates. This guarantees a regular point distribution, from which the mesh connectivity can be robustly reconstructed. Critically, this canonical sphere can be seamlessly unwrapped into a 2D map, creating a perfect synergy with powerful 2D generative models. We then introduce Spherical Geometry Diffusion, a conditional diffusion framework built upon this 2D map. It enables diverse and controllable generation by jointly modeling geometry and texture, where the geometry explicitly conditions the texture synthesis process. Our method’s effectiveness is demonstrated through its success in a wide range of tasks: text-to-3D generation, face reconstruction, and text-based 3D editing. Extensive experiments show that our approach substantially outperforms existing methods in geometric quality, textual fidelity, and inference efficiency.

Method: Developed ORACLE-CT with two key components: 1) Organ-Masked Attention (mask-restricted, per-organ pooling for spatial evidence) and 2) Organ-Scalar Fusion (lightweight fusion of normalized volume and mean-HU cues). Built on a supervised baseline with Global Average Pooling.

Result: Achieved AUROC 0.86 on chest CT-RATE dataset, and AUROC 0.85 on abdomen MERLIN dataset (30 findings). Outperformed all reported linear-probe VLMs and established new supervised state-of-the-art across both chest and abdomen CT.

Conclusion: ORACLE-CT delivers state-of-the-art supervised classification for CT triage across chest and abdomen domains, providing calibrated predictions with localized evidence while being encoder-agnostic and protocol-robust.

Abstract: There is an urgent need for triage and classification of high-volume medical imaging modalities such as computed tomography (CT), which can improve patient care and mitigate radiologist burnout. Study-level CT triage requires calibrated predictions with localized evidence; however, off-the-shelf Vision Language Models (VLM) struggle with 3D anatomy, protocol shifts, and noisy report supervision. This study used the two largest publicly available chest CT datasets: CT-RATE and RADCHEST-CT (held-out external test set). Our carefully tuned supervised baseline (instantiated as a simple Global Average Pooling head) establishes a new supervised state of the art, surpassing all reported linear-probe VLMs. Building on this baseline, we present ORACLE-CT, an encoder-agnostic, organ-aware head that pairs Organ-Masked Attention (mask-restricted, per-organ pooling that yields spatial evidence) with Organ-Scalar Fusion (lightweight fusion of normalized volume and mean-HU cues). In the chest setting, ORACLE-CT masked attention model achieves AUROC 0.86 on CT-RATE; in the abdomen setting, on MERLIN (30 findings), our supervised baseline exceeds a reproduced zero-shot VLM baseline obtained by running publicly released weights through our pipeline, and adding masked attention plus scalar fusion further improves performance to AUROC 0.85. Together, these results deliver state-of-the-art supervised classification performance across both chest and abdomen CT under a unified evaluation protocol. The source code is available at https://github.com/lavsendahal/oracle-ct.

Method: Fully model-driven 4D radar perception framework with: domain-aware multi-threshold filtering, ego-motion compensated temporal accumulation, KD tree Euclidean clustering with Doppler-aware refinement, and rule-based 3D classifier. Designed for real-time execution on embedded edge hardware.

Result: Framework evaluated in dust-filled enclosed trailer and real underground mining tunnels. Radar-based detector maintained stable pedestrian identification while camera and LiDAR modalities failed under severe visibility degradation.

Conclusion: Model-driven approach provides robust, interpretable, and computationally efficient perception for safety-critical applications in harsh industrial and subterranean environments.

Abstract: Pervasive sensing in industrial and underground environments is severely constrained by airborne dust, smoke, confined geometry, and metallic structures, which rapidly degrade optical and LiDAR based perception. Elevation resolved 4D mmWave radar offers strong resilience to such conditions, yet there remains a limited understanding of how to process its sparse and anisotropic point clouds for reliable human detection in enclosed, visibility degraded spaces. This paper presents a fully model-driven 4D radar perception framework designed for real-time execution on embedded edge hardware. The system uses radar as its sole perception modality and integrates domain aware multi threshold filtering, ego motion compensated temporal accumulation, KD tree Euclidean clustering with Doppler aware refinement, and a rule based 3D classifier. The framework is evaluated in a dust filled enclosed trailer and in real underground mining tunnels, and in the tested scenarios the radar based detector maintains stable pedestrian identification as camera and LiDAR modalities fail under severe visibility degradation. These results suggest that the proposed model-driven approach provides robust, interpretable, and computationally efficient perception for safety-critical applications in harsh industrial and subterranean environments.

Method: Proposes DIP-ℓ₀, a deep image prior framework incorporating ℓ₀ gradient regularizer. Uses an alternating direction method of multipliers algorithm with an off-the-shelf ℓ₀ gradient minimization solver to handle the nonconvex, nonsmooth ℓ₀ “norm” in the loss function.

Result: Numerical experiments show DIP-ℓ₀ outperforms many image smoothing algorithms in edge-preserving image smoothing and JPEG artifact removal, demonstrating high-quality smoothing without training data.

Conclusion: DIP-ℓ₀ successfully combines deep image prior with ℓ₀ regularization to achieve state-of-the-art image smoothing performance without requiring training data, addressing the challenge of dataset construction for this task.

Abstract: Image smoothing is a fundamental image processing operation that preserves the underlying structure, such as strong edges and contours, and removes minor details and textures in an image. Many image smoothing algorithms rely on computing local window statistics or solving an optimization problem. Recent state-of-the-art methods leverage deep learning, but they require a carefully curated training dataset. Because constructing a proper training dataset for image smoothing is challenging, we propose DIP-$\ell_0$, a deep image prior framework that incorporates the $\ell_0$ gradient regularizer. This framework can perform high-quality image smoothing without any training data. To properly minimize the associated loss function that has the nonconvex, nonsmooth $\ell_0$ ``norm”, we develop an alternating direction method of multipliers algorithm that utilizes an off-the-shelf $\ell_0$ gradient minimization solver. Numerical experiments demonstrate that the proposed DIP-$\ell_0$ outperforms many image smoothing algorithms in edge-preserving image smoothing and JPEG artifact removal.

Method: Comparative analysis of three SSOD approaches (MixPL, Semi-DETR, Consistent-Teacher) on MS-COCO and Pascal VOC benchmarks, plus a custom Beetle dataset, examining performance variation with different amounts of labeled images.

Result: Findings reveal trade-offs between accuracy, model size, and latency across methods, providing insights into which approaches work best in low-data regimes and specialized datasets with fewer object categories.

Conclusion: The study provides practical guidance for selecting appropriate semi-supervised object detection methods based on specific requirements (accuracy vs. efficiency) in data-scarce environments.

Abstract: Learning in data-scarce settings has recently gained significant attention in the research community. Semi-supervised object detection(SSOD) aims to improve detection performance by leveraging a large number of unlabeled images alongside a limited number of labeled images(a.k.a.,few-shot learning). In this paper, we present a comprehensive comparison of three state-of-the-art SSOD approaches, including MixPL, Semi-DETR and Consistent-Teacher, with the goal of understanding how performance varies with the number of labeled images. We conduct experiments using the MS-COCO and Pascal VOC datasets, two popular object detection benchmarks which allow for standardized evaluation. In addition, we evaluate the SSOD approaches on a custom Beetle dataset which enables us to gain insights into their performance on specialized datasets with a smaller number of object categories. Our findings highlight the trade-offs between accuracy, model size, and latency, providing insights into which methods are best suited for low-data regimes.

Method: Proposes QVLM (Quantitative Vision-Language Model) - a code-generation architecture that decouples language understanding from visual analysis. Instead of encoding images into embeddings, QVLM generates executable code that calls a segmentation model to obtain pixel-level masks, then operates directly on these masks to preserve spatial indexing throughout reasoning.

Result: QVLM using GPT-5 as coder achieves 42.0% accuracy on the new SQuID benchmark (2,000 satellite image QA pairs), compared to 28.1% for a standard VLM prompted with image-question pairs.

Conclusion: For quantitative spatial reasoning tasks, architectural decoupling that maintains pixel precision enables better accuracy than standard VLM approaches that compress spatial information through patch embeddings.

Abstract: Current Vision-Language Models (VLMs) fail at quantitative spatial reasoning because their architectures destroy pixel-level information required for counting and measurements. Vision encoders compress images through patch embeddings, reducing spatial indexing and losing the precise pixel-level tracking required for accurate counting. We present two contributions to address this fundamental limitation. First, we introduce SQuID (Satellite Quantitative Intelligence Dataset), a benchmark of 2,000 satellite image Question-Answer pairs with both numerical range and categorical answers, designed to evaluate quantitative spatial reasoning. The dataset spans three difficulty tiers with annotations automatically generated from human labels and their learned variability. Second, we propose QVLM (Quantitative Vision-Language Model), a code-generation architecture that maintains pixel precision by decoupling language understanding from visual analysis. Instead of encoding images into embeddings, QVLM generates executable code that first calls a segmentation model to obtain pixel-level masks, then operates directly on these masks, preserving spatial indexing throughout the reasoning process. Our experiments show that QVLM using GPT-5 as coder achieves 42.0% accuracy on SQuID compared to 28.1% for a VLM prompted with image-question pairs. Our work reveals that, for quantitative spatial reasoning, architectural decoupling enables better accuracy on quantitative tasks.

Method: Uses Transformer Decoder as prediction head with learnable object queries and positional encodings for set prediction. Introduces Pyramid Token Fusion (PTF) to convert multi-scale radar features into unified token sequence.

Result: Achieves significant improvements over state-of-the-art radar-only baselines on RADDet dataset.

Conclusion: The Transformer-based approach with PTF effectively models long-range correlations in radar data and simplifies the detection pipeline by eliminating dense proposals and NMS post-processing.

Abstract: In this paper, we present a Transformer-based architecture for 3D radar object detection that uses a novel Transformer Decoder as the prediction head to directly regress 3D bounding boxes and class scores from radar feature representations. To bridge multi-scale radar features and the decoder, we propose Pyramid Token Fusion (PTF), a lightweight module that converts a feature pyramid into a unified, scale-aware token sequence. By formulating detection as a set prediction problem with learnable object queries and positional encodings, our design models long-range spatial-temporal correlations and cross-feature interactions. This approach eliminates dense proposal generation and heuristic post-processing such as extensive non-maximum suppression (NMS) tuning. We evaluate the proposed framework on the RADDet, where it achieves significant improvements over state-of-the-art radar-only baselines.

Method: The authors propose local and global explanation methods that generate explanations as logical formulas in monotone disjunctive-normal-form (MDNF) using human-recognizable primitive concepts. They also present an algorithm for multi-class explanations in the form of monotone explanation lists over primitive concepts.

Result: The explanations maintain high fidelity and coverage with respect to the black-box models they seek to explain, as demonstrated on challenging vision datasets, despite their simplicity and interpretability.

Conclusion: The proposed methods provide interpretable explanations for black-box neural networks using logical formulas over primitive concepts, achieving both transparency and high fidelity to the original models’ behavior.

Abstract: While deep neural networks are extremely effective at classifying images, they remain opaque and hard to interpret. We introduce local and global explanation methods for black-box models that generate explanations in terms of human-recognizable primitive concepts. Both the local explanations for a single image and the global explanations for a set of images are cast as logical formulas in monotone disjunctive-normal-form (MDNF), whose satisfaction guarantees that the model yields a high score on a given class. We also present an algorithm for explaining the classification of examples into multiple classes in the form of a monotone explanation list over primitive concepts. Despite their simplicity and interpretability we show that the explanations maintain high fidelity and coverage with respect to the blackbox models they seek to explain in challenging vision datasets.

Method: Used ResNet-18 trained on 500 clinician-labeled CCE images with Leighton-Rex scale classification, applied stratified K-fold cross-validation, implemented structured pruning for sparsity optimization, evaluated explainability with Grad-CAM, Grad-CAM++, Eigen-CAM, Ablation-CAM, and Random-CAM using ROAD method, and employed adaptive temperature scaling for model calibration on external datasets.

Result: Achieved 88% cross-validation accuracy with 79% sparsity through pruning (improved from 84% baseline), demonstrated effectiveness of pruning for efficiency without performance compromise, evaluated various explainability methods, and successfully calibrated pruned models for external datasets.

Conclusion: Pruning effectively improves model efficiency while maintaining accuracy for CCE cleansing quality prediction, explainability is crucial for clinical adoption, and adaptive calibration enables model generalization to external datasets, though challenges remain in evaluating cleansing quality and using ROAD method for this specific task.

Abstract: In this study, we explore the application of deep learning techniques for predicting cleansing quality in colon capsule endoscopy (CCE) images. Using a dataset of 500 images labeled by 14 clinicians on the Leighton-Rex scale (Poor, Fair, Good, and Excellent), a ResNet-18 model was trained for classification, leveraging stratified K-fold cross-validation to ensure robust performance. To optimize the model, structured pruning techniques were applied iteratively, achieving significant sparsity while maintaining high accuracy. Explainability of the pruned model was evaluated using Grad-CAM, Grad-CAM++, Eigen-CAM, Ablation-CAM, and Random-CAM, with the ROAD method employed for consistent evaluation. Our results indicate that for a pruned model, we can achieve a cross-validation accuracy of 88% with 79% sparsity, demonstrating the effectiveness of pruning in improving efficiency from 84% without compromising performance. We also highlight the challenges of evaluating cleansing quality of CCE images, emphasize the importance of explainability in clinical applications, and discuss the challenges associated with using the ROAD method for our task. Finally, we employ a variant of adaptive temperature scaling to calibrate the pruned models for an external dataset.

Method: Use a frozen diffusion backbone as a feature encoder, probe intermediate denoising features across layers and timesteps, and train linear classifiers for each feature pair. Evaluate in real-world plankton monitoring with controlled setups against supervised and self-supervised baselines.

Result: Frozen diffusion features are competitive with supervised baselines and outperform other self-supervised methods in both balanced and long-tailed settings. They maintain strong accuracy and Macro F1 under substantial distribution shifts in temporally and geographically shifted datasets.

Conclusion: Diffusion models can serve as effective general-purpose feature encoders for fine-grained recognition, offering strong performance even under distribution shifts, making them valuable for practical applications like environmental monitoring.

Abstract: Diffusion models have emerged as state-of-the-art generative methods for image synthesis, yet their potential as general-purpose feature encoders remains underexplored. Trained for denoising and generation without labels, they can be interpreted as self-supervised learners that capture both low- and high-level structure. We show that a frozen diffusion backbone enables strong fine-grained recognition by probing intermediate denoising features across layers and timesteps and training a linear classifier for each pair. We evaluate this in a real-world plankton-monitoring setting with practical impact, using controlled and comparable training setups against established supervised and self-supervised baselines. Frozen diffusion features are competitive with supervised baselines and outperform other self-supervised methods in both balanced and naturally long-tailed settings. Out-of-distribution evaluations on temporally and geographically shifted plankton datasets further show that frozen diffusion features maintain strong accuracy and Macro F1 under substantial distribution shift.

Method: Propose SGW-GAN, the first framework incorporating Sliced Gromov Wasserstein (SGW) distance into retinal image enhancement. SGW approximates the computationally expensive Gromov Wasserstein discrepancy via random projections, preserving relational fidelity while reducing computational cost.

Result: Experiments on public datasets show SGW-GAN produces visually compelling enhancements, achieves superior diabetic retinopathy grading performance, and reports the lowest GW discrepancy across disease labels, demonstrating both efficiency and clinical fidelity.

Conclusion: SGW-GAN effectively addresses the limitations of previous enhancement methods by preserving intra-class structure through SGW distance, making it both computationally efficient and clinically valuable for unpaired medical image enhancement tasks.

Abstract: Retinal fundus photography is indispensable for ophthalmic screening and diagnosis, yet image quality is often degraded by noise, artifacts, and uneven illumination. Recent GAN- and diffusion-based enhancement methods improve perceptual quality by aligning degraded images with high-quality distributions, but our analysis shows that this focus can distort intra-class geometry: clinically related samples become dispersed, disease-class boundaries blur, and downstream tasks such as grading or lesion detection are harmed. The Gromov Wasserstein (GW) discrepancy offers a principled solution by aligning distributions through internal pairwise distances, naturally preserving intra-class structure, but its high computational cost restricts practical use. To overcome this, we propose SGW-GAN, the first framework to incorporate Sliced GW (SGW) into retinal image enhancement. SGW approximates GW via random projections, retaining relational fidelity while greatly reducing cost. Experiments on public datasets show that SGW-GAN produces visually compelling enhancements, achieves superior diabetic retinopathy grading, and reports the lowest GW discrepancy across disease labels, demonstrating both efficiency and clinical fidelity for unpaired medical image enhancement.

Method: Systematic investigation of key architectural paradigms: sliding window-based dense feature extraction (WinCLIP), multi-stage feature alignment with learnable projections (AprilLab framework), and compositional prompt ensemble strategies. Evaluation across feature extraction mechanisms, text-visual alignment strategies, prompt engineering techniques, and computational efficiency.

Result: Rigorous experimentation on benchmarks (MVTec AD, VisA) comparing classification accuracy, segmentation precision, and inference efficiency. Provides foundational understanding of how and why VLMs succeed in anomaly detection.

Conclusion: Synthesizes practical insights for method selection, identifies current limitations, aims to facilitate informed adoption of VLM-based methods in industrial quality control, and guides future research directions.

Abstract: Vision-Language Models (VLMs), particularly CLIP, have revolutionized anomaly detection by enabling zero-shot and few-shot defect identification without extensive labeled datasets. By learning aligned representations of images and text, VLMs facilitate anomaly classification and segmentation through natural language descriptions of normal and abnormal states, eliminating traditional requirements for task-specific training or defect examples. This project presents a comprehensive analysis of VLM-based approaches for anomaly classification (AC) and anomaly segmentation (AS). We systematically investigate key architectural paradigms including sliding window-based dense feature extraction (WinCLIP), multi-stage feature alignment with learnable projections (AprilLab framework), and compositional prompt ensemble strategies. Our analysis evaluates these methods across critical dimensions: feature extraction mechanisms, text-visual alignment strategies, prompt engineering techniques, zero-shot versus few-shot trade-offs, computational efficiency, and cross-domain generalization. Through rigorous experimentation on benchmarks such as MVTec AD and VisA, we compare classification accuracy, segmentation precision, and inference efficiency. The primary contribution is a foundational understanding of how and why VLMs succeed in anomaly detection, synthesizing practical insights for method selection and identifying current limitations. This work aims to facilitate informed adoption of VLM-based methods in industrial quality control and guide future research directions.

Method: Develop a physics-grounded processing pipeline that: 1) maps event streams to estimates of per-pixel log-intensity and intensity derivatives, 2) embeds these measurements in a dynamic linear systems model with time-varying point spread function, and 3) enables inverse filtering directly from event data using frequency-domain Wiener deconvolution with known or parameterized dynamic transfer functions.

Result: Validated the approach in simulation for single and overlapping point sources under modulated defocus, and on real event data from a tunable-focus telescope imaging a star field. Demonstrated successful source localization and separability, showing the framework can effectively process event data for computational imaging tasks.

Conclusion: The proposed framework provides a practical bridge between event sensing and model-based computational imaging for dynamic optical systems, enabling traditional computational imaging techniques to work with the unique asynchronous, sparse data format of neuromorphic cameras.

Abstract: Event vision sensors (neuromorphic cameras) output sparse, asynchronous ON/OFF events triggered by log-intensity threshold crossings, enabling microsecond-scale sensing with high dynamic range and low data bandwidth. As a nonlinear system, this event representation does not readily integrate with the linear forward models that underpin most computational imaging and optical system design. We present a physics-grounded processing pipeline that maps event streams to estimates of per-pixel log-intensity and intensity derivatives, and embeds these measurements in a dynamic linear systems model with a time-varying point spread function. This enables inverse filtering directly from event data, using frequency-domain Wiener deconvolution with a known (or parameterised) dynamic transfer function. We validate the approach in simulation for single and overlapping point sources under modulated defocus, and on real event data from a tunable-focus telescope imaging a star field, demonstrating source localisation and separability. The proposed framework provides a practical bridge between event sensing and model-based computational imaging for dynamic optical systems.

Method: DIS2 paradigm shift: (1) principled missing information compensation, (2) class-specific modality contribution, (3) multi-resolution feature importance. Core DLKD reformulates disentanglement+distillation synergy. CFLM learns class-specific discriminative evidence. Hierarchical hybrid fusion uses multi-resolution features.

Result: Extensive experiments show DIS2 significantly outperforms state-of-the-art methods across benchmarks.

Conclusion: DIS2 successfully addresses RS multimodal challenges through active guided compensation rather than modality-shared dependence, with DLKD synergy and class-specific learning proving effective for heterogeneous RS data with missing modalities.

Abstract: The efficacy of multimodal learning in remote sensing (RS) is severely undermined by missing modalities. The challenge is exacerbated by the RS highly heterogeneous data and huge scale variation. Consequently, paradigms proven effective in other domains often fail when confronted with these unique data characteristics. Conventional disentanglement learning, which relies on significant feature overlap between modalities (modality-invariant), is insufficient for this heterogeneity. Similarly, knowledge distillation becomes an ill-posed mimicry task where a student fails to focus on the necessary compensatory knowledge, leaving the semantic gap unaddressed. Our work is therefore built upon three pillars uniquely designed for RS: (1) principled missing information compensation, (2) class-specific modality contribution, and (3) multi-resolution feature importance. We propose a novel method DIS2, a new paradigm shifting from modality-shared feature dependence and untargeted imitation to active, guided missing features compensation. Its core novelty lies in a reformulated synergy between disentanglement learning and knowledge distillation, termed DLKD. Compensatory features are explicitly captured which, when fused with the features of the available modality, approximate the ideal fused representation of the full-modality case. To address the class-specific challenge, our Classwise Feature Learning Module (CFLM) adaptively learn discriminative evidence for each target depending on signal availability. Both DLKD and CFLM are supported by a hierarchical hybrid fusion (HF) structure using features across resolutions to strengthen prediction. Extensive experiments validate that our proposed approach significantly outperforms state-of-the-art methods across benchmarks.

Method: Proposes GO-MLVTON with two key modules: 1) Garment Occlusion Learning module to learn occlusion relationships between garment layers, and 2) StableDiffusion-based Garment Morphing & Fitting module to deform and fit garments onto the human body.

Result: Extensive experiments demonstrate state-of-the-art performance. The method produces high-quality multi-layer try-on results with realistic deformation and layering.

Conclusion: GO-MLVTON successfully addresses the multi-layer VTON challenge, introduces a new MLG dataset for this task, and proposes a new evaluation metric (LACD) for layered appearance coherence.

Abstract: Existing Image-based virtual try-on (VTON) methods primarily focus on single-layer or multi-garment VTON, neglecting multi-layer VTON (ML-VTON), which involves dressing multiple layers of garments onto the human body with realistic deformation and layering to generate visually plausible outcomes. The main challenge lies in accurately modeling occlusion relationships between inner and outer garments to reduce interference from redundant inner garment features. To address this, we propose GO-MLVTON, the first multi-layer VTON method, introducing the Garment Occlusion Learning module to learn occlusion relationships and the StableDiffusion-based Garment Morphing & Fitting module to deform and fit garments onto the human body, producing high-quality multi-layer try-on results. Additionally, we present the MLG dataset for this task and propose a new metric named Layered Appearance Coherence Difference (LACD) for evaluation. Extensive experiments demonstrate the state-of-the-art performance of GO-MLVTON. Project page: https://upyuyang.github.io/go-mlvton/.

Method: Built on the Diverse Dermatology Images (DDI) dataset with 656 clinical images from 570 unique patients spanning Fitzpatrick skin types I-VI. Expert dermatologists used a hierarchical annotation schema with 22 main questions (single-choice, multi-choice, and open-ended) covering diagnosis, anatomic site, lesion morphology, distribution, surface features, color, image quality, plus open-ended narrative descriptions and summaries.

Result: Created DermaBench with approximately 14,474 VQA-style annotations. The benchmark is released as a metadata-only dataset to respect upstream licensing and is publicly available at Harvard Dataverse.

Conclusion: DermaBench addresses the gap in dermatology VQA evaluation by providing a comprehensive clinician-annotated benchmark that enables assessment of multimodal models’ visual understanding, language grounding, and clinical reasoning capabilities beyond simple classification tasks.

Abstract: Vision-language models (VLMs) are increasingly important in medical applications; however, their evaluation in dermatology remains limited by datasets that focus primarily on image-level classification tasks such as lesion recognition. While valuable for recognition, such datasets cannot assess the full visual understanding, language grounding, and clinical reasoning capabilities of multimodal models. Visual question answering (VQA) benchmarks are required to evaluate how models interpret dermatological images, reason over fine-grained morphology, and generate clinically meaningful descriptions. We introduce DermaBench, a clinician-annotated dermatology VQA benchmark built on the Diverse Dermatology Images (DDI) dataset. DermaBench comprises 656 clinical images from 570 unique patients spanning Fitzpatrick skin types I-VI. Using a hierarchical annotation schema with 22 main questions (single-choice, multi-choice, and open-ended), expert dermatologists annotated each image for diagnosis, anatomic site, lesion morphology, distribution, surface features, color, and image quality, together with open-ended narrative descriptions and summaries, yielding approximately 14.474 VQA-style annotations. DermaBench is released as a metadata-only dataset to respect upstream licensing and is publicly available at Harvard Dataverse.

Method: CARPE introduces vision-integration layers and a context-aware ensemble strategy that adaptively weights when to prioritize image representations versus relying on language model reasoning, enabling capture of various aspects of image representations.

Result: CARPE improves performance on image classification benchmarks and enhances results across various vision-language benchmarks, with consistent improvements in generalization across different task types.

Conclusion: CARPE is an effective, model-agnostic framework that can be integrated with most open-source LVLMs to enhance their vision capabilities while maintaining adaptability across diverse architectures.

Abstract: Recent advancements in Large Vision-Language Models (LVLMs) have pushed them closer to becoming general-purpose assistants. Despite their strong performance, LVLMs still struggle with vision-centric tasks such as image classification, underperforming compared to their base vision encoders, which are often CLIP-based models. To address this limitation, we propose Context-Aware Image Representation Prioritization via Ensemble (CARPE), a novel, model-agnostic framework which introduces vision-integration layers and a context-aware ensemble strategy to identify when to prioritize image representations or rely on the reasoning capabilities of the language model. This design enhances the model’s ability to adaptively weight visual and textual modalities and enables the model to capture various aspects of image representations, leading to consistent improvements in generalization across classification and vision-language benchmarks. Extensive experiments demonstrate that CARPE not only improves performance on image classification benchmarks but also enhances results across various vision-language benchmarks. Finally, CARPE is designed to be effectively integrated with most open-source LVLMs that consist of a vision encoder and a language model, ensuring its adaptability across diverse architectures.

Method: Created DiffFace-Edit dataset with over 2 million AI-generated fake images featuring edits across 8 facial regions with various editing combinations. Conducted comprehensive dataset analysis and proposed cross-domain evaluation combining IMDL methods.

Result: The dataset provides extensive coverage of fine-grained facial manipulations including single-region and multi-region edits. Analysis specifically examines the impact of detector-evasive samples on detection models.

Conclusion: DiffFace-Edit addresses critical gaps in AI-generated face datasets and enables research on detector-evasive samples, with the dataset being publicly available for further study.

Abstract: Generative models now produce imperceptible, fine-grained manipulated faces, posing significant privacy risks. However, existing AI-generated face datasets generally lack focus on samples with fine-grained regional manipulations. Furthermore, no researchers have yet studied the real impact of splice attacks, which occur between real and manipulated samples, on detectors. We refer to these as detector-evasive samples. Based on this, we introduce the DiffFace-Edit dataset, which has the following advantages: 1) It contains over two million AI-generated fake images. 2) It features edits across eight facial regions (e.g., eyes, nose) and includes a richer variety of editing combinations, such as single-region and multi-region edits. Additionally, we specifically analyze the impact of detector-evasive samples on detection models. We conduct a comprehensive analysis of the dataset and propose a cross-domain evaluation that combines IMDL methods. Dataset will be available at https://github.com/ywh1093/DiffFace-Edit.

Method: Created MIR-SafetyBench with 2,676 instances across 9 multi-image relation types, then evaluated 19 MLLMs to assess safety vulnerabilities in multi-image reasoning scenarios.

Result: Models with more advanced multi-image reasoning are paradoxically more vulnerable to safety issues. Many “safe” responses are superficial or evasive, and unsafe generations show lower attention entropy than safe ones.

Conclusion: There’s a concerning trade-off where improved multi-image reasoning capability increases safety risks, suggesting models may over-focus on task solving while neglecting safety constraints.

Abstract: As Multimodal Large Language Models (MLLMs) acquire stronger reasoning capabilities to handle complex, multi-image instructions, this advancement may pose new safety risks. We study this problem by introducing MIR-SafetyBench, the first benchmark focused on multi-image reasoning safety, which consists of 2,676 instances across a taxonomy of 9 multi-image relations. Our extensive evaluations on 19 MLLMs reveal a troubling trend: models with more advanced multi-image reasoning can be more vulnerable on MIR-SafetyBench. Beyond attack success rates, we find that many responses labeled as safe are superficial, often driven by misunderstanding or evasive, non-committal replies. We further observe that unsafe generations exhibit lower attention entropy than safe ones on average. This internal signature suggests a possible risk that models may over-focus on task solving while neglecting safety constraints. Our code and data are available at https://github.com/thu-coai/MIR-SafetyBench.

Method: Two key innovations: (1) Rollout Posterior Entropy metric for chart complexity + complexity-aware chart coder for diverse high-complexity charts; (2) Truth-anchored inverse QA synthesis with answer-first paradigm + consistency verification + model fail-rate filtering + CoT distillation.

Result: ChartVerse-8B achieves state-of-the-art performance, surpassing its teacher (Qwen3-VL-30B-A3B-Thinking) and rivaling the stronger Qwen3-VL-32B-Thinking model.

Conclusion: ChartVerse effectively addresses the data bottleneck in chart reasoning by synthesizing complex charts and rigorous reasoning data, enabling smaller models to achieve superior performance.

Abstract: Chart reasoning is a critical capability for Vision Language Models (VLMs). However, the development of open-source models is severely hindered by the lack of high-quality training data. Existing datasets suffer from a dual challenge: synthetic charts are often simplistic and repetitive, while the associated QA pairs are prone to hallucinations and lack the reasoning depth required for complex tasks. To bridge this gap, we propose ChartVerse, a scalable framework designed to synthesize complex charts and reliable reasoning data from scratch. (1) To address the bottleneck of simple patterns, we first introduce Rollout Posterior Entropy (RPE), a novel metric that quantifies chart complexity. Guided by RPE, we develop complexity-aware chart coder to autonomously synthesize diverse, high-complexity charts via executable programs. (2) To guarantee reasoning rigor, we develop truth-anchored inverse QA synthesis. Diverging from standard generation, we adopt an answer-first paradigm: we extract deterministic answers directly from the source code, generate questions conditional on these anchors, and enforce strict consistency verification. To further elevate difficulty and reasoning depth, we filter samples based on model fail-rate and distill high-quality Chain-of-Thought (CoT) reasoning. We curate ChartVerse-SFT-600K and ChartVerse-RL-40K using Qwen3-VL-30B-A3B-Thinking as the teacher. Experimental results demonstrate that ChartVerse-8B achieves state-of-the-art performance, notably surpassing its teacher and rivaling the stronger Qwen3-VL-32B-Thinking.

Method: Introduces Efficient visual Grounding language Models (EGM) that scale test-time computation by generating more tokens during inference. This is deployment-friendly as small models have cheaper per-token costs than large models.

Result: EGM-Qwen3-VL-8B achieves 91.4 IoU with 737ms latency (5.9x faster) vs Qwen3-VL-235B’s 90.5 IoU with 4320ms. Also improves amodal grounding (predicting visible+occluded parts) to match/exceed larger models.

Conclusion: Scaling test-time computation effectively bridges the gap between small and large VLMs for visual grounding, offering efficient deployment with comparable performance and faster inference.

Abstract: Visual grounding is an essential capability of Visual Language Models (VLMs) to understand the real physical world. Previous state-of-the-art grounding visual language models usually have large model sizes, making them heavy for deployment and slow for inference. However, we notice that the sizes of visual encoders are nearly the same for small and large VLMs and the major difference is the sizes of the language models. Small VLMs fall behind larger VLMs in grounding because of the difference in language understanding capability rather than visual information handling. To mitigate the gap, we introduce ‘Efficient visual Grounding language Models’ (EGM): a method to scale the test-time computation (#generated tokens). Scaling the test-time computation of a small model is deployment-friendly, and yields better end-to-end latency as the cost of each token is much cheaper compared to directly running a large model. On the RefCOCO benchmark, our EGM-Qwen3-VL-8B demonstrates 91.4 IoU with an average of 737ms (5.9x faster) latency while Qwen3-VL-235B demands 4,320ms to achieve 90.5 IoU. To validate our approach’s generality, we further set up a new amodal grounding setting that requires the model to predict both the visible and occluded parts of the objects. Experiments show our method can consistently and significantly improve the vanilla grounding and amodal grounding capabilities of small models to be on par with or outperform the larger models, thereby improving the efficiency for visual grounding.

Method: Attention-space Contrastive Guidance (ACG) operates within self-attention layers to construct both vision-language and language-only attention paths in a single forward pass. It uses orthogonalized correction to remove language-only aligned components and amplify visual contributions.

Result: ACG achieves state-of-the-art faithfulness and caption quality on CHAIR and POPE benchmarks while significantly reducing computational cost. It reduces latency by up to 2x compared to prior contrastive decoding methods requiring multiple forward passes.

Conclusion: ACG provides a principled and efficient alternative for hallucination mitigation in LVLMs by embedding contrastive guidance directly in attention-space representation contextualization, balancing computational efficiency with improved visual grounding.

Abstract: Hallucinations in large vision-language models (LVLMs) often arise when language priors dominate over visual evidence, causing object misidentification and visually inconsistent descriptions. We address this issue by framing hallucination mitigation as contrastive guidance, steering generation toward visually grounded and semantically faithful text. This approach regulates the model’s internal behavior by reducing over-dependence on language priors and contrasting visually grounded with language-only representations. We propose Attention-space Contrastive Guidance (ACG), a single-pass mechanism that operates within self-attention layers to construct both vision-language and language-only attention paths in a single forward computation. This integration enables computationally efficient guidance directly embedded in the model’s representation contextualization. To correct approximation bias introduced by the single-pass formulation, we further apply an orthogonalized correction that removes components aligned with the language-only path, selectively amplifying visual contributions. Experiments on the CHAIR and POPE benchmarks show that ACG achieves state-of-the-art faithfulness and caption quality while significantly reducing computational cost. Our method establishes a principled and efficient alternative, reducing latency by up to 2x compared to prior contrastive decoding methods that require multiple forward passes.

Method: Develops a face-voice association method that imposes maximum class separation among multimodal representations of different speakers as inductive bias. Combines this with losses for inter-class orthogonality.

Result: Achieves state-of-the-art performance on two face-voice association task formulations. Ablation study shows inductive bias is most effective when combined with inter-class orthogonality losses.

Conclusion: First work to demonstrate effectiveness of maximum class separation as inductive bias in multimodal learning, establishing a new paradigm for face-voice association.

Abstract: Face-voice association is widely studied in multimodal learning and is approached representing faces and voices with embeddings that are close for a same person and well separated from those of others. Previous work achieved this with loss functions. Recent advancements in classification have shown that the discriminative ability of embeddings can be strengthened by imposing maximum class separation as inductive bias. This technique has never been used in the domain of face-voice association, and this work aims at filling this gap. More specifically, we develop a method for face-voice association that imposes maximum class separation among multimodal representations of different speakers as an inductive bias. Through quantitative experiments we demonstrate the effectiveness of our approach, showing that it achieves SOTA performance on two task formulation of face-voice association. Furthermore, we carry out an ablation study to show that imposing inductive bias is most effective when combined with losses for inter-class orthogonality. To the best of our knowledge, this work is the first that applies and demonstrates the effectiveness of maximum class separation as an inductive bias in multimodal learning; it hence paves the way to establish a new paradigm.

Method: 1) Preserve semantic consistency by integrating entity-level representations across visual and auditory streams, 2) Organize content into a structured hierarchy (global summary, scene, segment, entity levels), 3) Employ agentic search mechanism for dynamic retrieval and reasoning across these layers.

Result: Achieves state-of-the-art with 84.1% overall accuracy on LVBench, with outstanding performance in challenging reasoning category (80.1%). Demonstrates good temporal coherence, entity consistency, and retrieval efficiency.

Conclusion: HAVEN’s structured, multimodal reasoning approach enables comprehensive and context-consistent understanding of long-form videos, highlighting the effectiveness of integrating audiovisual entity cohesion with hierarchical indexing and agentic search.

Abstract: Long video understanding presents significant challenges for vision-language models due to extremely long context windows. Existing solutions relying on naive chunking strategies with retrieval-augmented generation, typically suffer from information fragmentation and a loss of global coherence. We present HAVEN, a unified framework for long-video understanding that enables coherent and comprehensive reasoning by integrating audiovisual entity cohesion and hierarchical video indexing with agentic search. First, we preserve semantic consistency by integrating entity-level representations across visual and auditory streams, while organizing content into a structured hierarchy spanning global summary, scene, segment, and entity levels. Then we employ an agentic search mechanism to enable dynamic retrieval and reasoning across these layers, facilitating coherent narrative reconstruction and fine-grained entity tracking. Extensive experiments demonstrate that our method achieves good temporal coherence, entity consistency, and retrieval efficiency, establishing a new state-of-the-art with an overall accuracy of 84.1% on LVBench. Notably, it achieves outstanding performance in the challenging reasoning category, reaching 80.1%. These results highlight the effectiveness of structured, multimodal reasoning for comprehensive and context-consistent understanding of long-form videos.

Method: VIAFormer uses three key components: 1) Image Index for explicit 3D spatial grounding of 2D image tokens, 2) Correctional Flow objective that learns direct voxel-refinement trajectories, and 3) Hybrid Stream Transformer for robust cross-modal fusion between voxels and images.

Result: VIAFormer establishes new state-of-the-art performance in correcting both severe synthetic corruptions and realistic artifacts on voxel shapes obtained from powerful Vision Foundation Models. It also demonstrates practical utility as a reliable bridge in real-world 3D creation pipelines.

Conclusion: The proposed VIAFormer model effectively bridges multi-view images and 3D voxel refinement, paving the way for voxel-based methods to thrive in the era of large models and big data, with practical applications in real-world 3D creation workflows.

Abstract: We propose VIAFormer, a Voxel-Image Alignment Transformer model designed for Multi-view Conditioned Voxel Refinement–the task of repairing incomplete noisy voxels using calibrated multi-view images as guidance. Its effectiveness stems from a synergistic design: an Image Index that provides explicit 3D spatial grounding for 2D image tokens, a Correctional Flow objective that learns a direct voxel-refinement trajectory, and a Hybrid Stream Transformer that enables robust cross-modal fusion. Experiments show that VIAFormer establishes a new state of the art in correcting both severe synthetic corruptions and realistic artifacts on the voxel shape obtained from powerful Vision Foundation Models. Beyond benchmarking, we demonstrate VIAFormer as a practical and reliable bridge in real-world 3D creation pipelines, paving the way for voxel-based methods to thrive in large-model, big-data wave.

Method: Uses hierarchical sparse autoencoder with a foundation model to automatically extract concepts at various granularities, analyzes local co-occurrence dependencies to define concept relationships, and improves concept naming through these relations.

Result: Achieves competitive performance on classification and segmentation benchmarks while providing fine-grained, high-quality concept-based explanations.

Conclusion: Insight provides both strong task performance and interpretable, spatially-grounded concept explanations, addressing the opacity of current vision foundation models.

Abstract: Language-aligned vision foundation models perform strongly across diverse downstream tasks. Yet, their learned representations remain opaque, making interpreting their decision-making hard. Recent works decompose these representations into human-interpretable concepts, but provide poor spatial grounding and are limited to image classification tasks. In this work, we propose Insight, a language-aligned concept foundation model that provides fine-grained concepts, which are human-interpretable and spatially grounded in the input image. We leverage a hierarchical sparse autoencoder and a foundation model with strong semantic representations to automatically extract concepts at various granularities. Examining local co-occurrence dependencies of concepts allows us to define concept relationships. Through these relations we further improve concept naming and obtain richer explanations. On benchmark data, we show that Insight provides performance on classification and segmentation that is competitive with opaque foundation models while providing fine-grained, high quality concept-based explanations. Code is available at https://github.com/kawi19/Insight.

Method: Developed fusion architectures combining CNN with LSTM (CNN+CNN-LSTM) and CNN with DeiT Transformer (CNN+DeiT Transformer). Created custom dataset by capturing images of vegetables from fresh state until complete spoilage. Validated models on noisy images and integrated LIME for explainable AI visualization.

Result: Proposed fusion architectures outperformed several deep learning models (CNN, VGG16, ResNet50, Capsule Networks, DeiT Transformers). CNN+DeiT Transformer achieved F1-score of 0.98 for vegetable classification, 0.61 for spoilage detection, and MSE/SMAPE of 3.58/41.66% for spoilage forecasting.

Conclusion: Fusion architectures combining CNN with LSTM and DeiT transformer are effective for multi-task learning in agricultural supply chain applications, enabling simultaneous vegetable classification, spoilage detection, and shelf life forecasting with high accuracy and reliability.

Abstract: Food wastage is one of the critical challenges in the agricultural supply chain, and accurate and effective spoilage detection can help to reduce it. Further, it is highly important to forecast the spoilage information. This aids the longevity of the supply chain management in the agriculture field. This motivated us to propose fusion based architectures by combining CNN with LSTM and DeiT transformer for the following multi-tasks simultaneously: (i) vegetable classification, (ii) food spoilage detection, and (iii) shelf life forecasting. We developed a dataset by capturing images of vegetables from their fresh state until they were completely spoiled. From the experimental analysis it is concluded that the proposed fusion architectures CNN+CNN-LSTM and CNN+DeiT Transformer outperformed several deep learning models such as CNN, VGG16, ResNet50, Capsule Networks, and DeiT Transformers. Overall, CNN + DeiT Transformer yielded F1-score of 0.98 and 0.61 in vegetable classification and spoilage detection respectively and mean squared error (MSE) and symmetric mean absolute percentage error (SMAPE) of 3.58, and 41.66% respectively in spoilage forecasting. Further, the reliability of the fusion models was validated on noisy images and integrated with LIME to visualize the model decisions.

Method: ClaSP PE combines class-stratified querying to ensure coverage of underrepresented structures with log-scale power noising using a decaying schedule to enforce query diversity in early-stage AL and encourage exploitation later.

Result: In 24 experimental settings across four 3D biomedical datasets, ClaSP PE was the only method that generally outperformed improved random baselines with statistically significant gains in segmentation quality while remaining annotation efficient. It also robustly generalized to four previously unseen datasets without manual adaptation.

Conclusion: ClaSP PE provides the first reliable active learning solution for 3D biomedical segmentation that consistently outperforms random baselines in realistic scenarios, with open-source implementation and clear deployment guidelines for practical application.

Abstract: Active learning (AL) has the potential to drastically reduce annotation costs in 3D biomedical image segmentation, where expert labeling of volumetric data is both time-consuming and expensive. Yet, existing AL methods are unable to consistently outperform improved random sampling baselines adapted to 3D data, leaving the field without a reliable solution. We introduce Class-stratified Scheduled Power Predictive Entropy (ClaSP PE), a simple and effective query strategy that addresses two key limitations of standard uncertainty-based AL methods: class imbalance and redundancy in early selections. ClaSP PE combines class-stratified querying to ensure coverage of underrepresented structures and log-scale power noising with a decaying schedule to enforce query diversity in early-stage AL and encourage exploitation later. In our evaluation on 24 experimental settings using four 3D biomedical datasets within the comprehensive nnActive benchmark, ClaSP PE is the only method that generally outperforms improved random baselines in terms of both segmentation quality with statistically significant gains, whilst remaining annotation efficient. Furthermore, we explicitly simulate the real-world application by testing our method on four previously unseen datasets without manual adaptation, where all experiment parameters are set according to predefined guidelines. The results confirm that ClaSP PE robustly generalizes to novel tasks without requiring dataset-specific tuning. Within the nnActive framework, we present compelling evidence that an AL method can consistently outperform random baselines adapted to 3D segmentation, in terms of both performance and annotation efficiency in a realistic, close-to-production scenario. Our open-source implementation and clear deployment guidelines make it readily applicable in practice. Code is at https://github.com/MIC-DKFZ/nnActive.

Method: Proposes Dynamic Differential Linear Attention (DyDiLA) with three key components: dynamic projection module for token decoupling, dynamic measure kernel for better similarity measurement, and token differential operator for robust query-to-key retrieval. These are integrated into DyDi-LiT, a refined linear diffusion transformer.

Result: DyDi-LiT consistently outperforms current state-of-the-art models across multiple metrics, demonstrating strong practical potential for high-quality image generation with improved computational efficiency.

Conclusion: DyDiLA successfully addresses the oversmoothing problem in linear attention for diffusion transformers, enabling both computational efficiency and high generative performance, making it a promising approach for scalable image generation.

Abstract: Diffusion transformers (DiTs) have emerged as a powerful architecture for high-fidelity image generation, yet the quadratic cost of self-attention poses a major scalability bottleneck. To address this, linear attention mechanisms have been adopted to reduce computational cost; unfortunately, the resulting linear diffusion transformers (LiTs) models often come at the expense of generative performance, frequently producing over-smoothed attention weights that limit expressiveness. In this work, we introduce Dynamic Differential Linear Attention (DyDiLA), a novel linear attention formulation that enhances the effectiveness of LiTs by mitigating the oversmoothing issue and improving generation quality. Specifically, the novelty of DyDiLA lies in three key designs: (i) dynamic projection module, which facilitates the decoupling of token representations by learning with dynamically assigned knowledge; (ii) dynamic measure kernel, which provides a better similarity measurement to capture fine-grained semantic distinctions between tokens by dynamically assigning kernel functions for token processing; and (iii) token differential operator, which enables more robust query-to-key retrieval by calculating the differences between the tokens and their corresponding information redundancy produced by dynamic measure kernel. To capitalize on DyDiLA, we introduce a refined LiT, termed DyDi-LiT, that systematically incorporates our advancements. Extensive experiments show that DyDi-LiT consistently outperforms current state-of-the-art (SOTA) models across multiple metrics, underscoring its strong practical potential.

Method: Proposes OmniOVCD framework with Synergistic Fusion to Instance Decoupling (SFID) strategy. SFID leverages SAM 3’s decoupled output heads to fuse semantic, instance, and presence outputs to construct land-cover masks, then decomposes them into individual instance masks for change comparison, preserving category recognition accuracy and instance-level consistency.

Result: Achieves state-of-the-art performance on four public benchmarks: LEVIR-CD (67.2 IoU), WHU-CD (66.5 IoU), S2Looking (24.5 IoU), and SECOND (27.1 IoU), surpassing all previous methods.

Conclusion: OmniOVCD demonstrates that leveraging SAM 3’s integrated capabilities through the SFID strategy enables accurate open-vocabulary change detection without the instability issues of multi-model approaches, establishing a new benchmark for OVCD performance.

Abstract: Change Detection (CD) is a fundamental task in remote sensing. It monitors the evolution of land cover over time. Based on this, Open-Vocabulary Change Detection (OVCD) introduces a new requirement. It aims to reduce the reliance on predefined categories. Existing training-free OVCD methods mostly use CLIP to identify categories. These methods also need extra models like DINO to extract features. However, combining different models often causes problems in matching features and makes the system unstable. Recently, the Segment Anything Model 3 (SAM 3) is introduced. It integrates segmentation and identification capabilities within one promptable model, which offers new possibilities for the OVCD task. In this paper, we propose OmniOVCD, a standalone framework designed for OVCD. By leveraging the decoupled output heads of SAM 3, we propose a Synergistic Fusion to Instance Decoupling (SFID) strategy. SFID first fuses the semantic, instance, and presence outputs of SAM 3 to construct land-cover masks, and then decomposes them into individual instance masks for change comparison. This design preserves high accuracy in category recognition and maintains instance-level consistency across images. As a result, the model can generate accurate change masks. Experiments on four public benchmarks (LEVIR-CD, WHU-CD, S2Looking, and SECOND) demonstrate SOTA performance, achieving IoU scores of 67.2, 66.5, 24.5, and 27.1 (class-average), respectively, surpassing all previous methods.

Method: Provides unified perspective by framing visual puzzles through common abstraction, organizing existing benchmarks by reasoning mechanisms (inductive, analogical, algorithmic, deductive, geometric/spatial), and synthesizing empirical evidence across categories.

Result: Identifies consistent limitations in current models: brittle generalization, tight entanglement between perception and reasoning, and persistent gap between fluent explanations and faithful execution.

Conclusion: Visual puzzles should be viewed as diagnostic instruments rather than task formats; the survey outlines key directions for future benchmarks and reasoning-aware multimodal systems.

Abstract: Puzzles have long served as compact and revealing probes of human cognition, isolating abstraction, rule discovery, and systematic reasoning with minimal reliance on prior knowledge. Leveraging these properties, visual puzzles have recently emerged as a powerful diagnostic tool for evaluating the reasoning abilities of Large Vision-Language Models (LVLMs), offering controlled, verifiable alternatives to open-ended multimodal benchmarks. This survey provides a unified perspective of visual puzzle reasoning in LVLMs. We frame visual puzzles through a common abstraction and organize existing benchmarks by the reasoning mechanisms they target (inductive, analogical, algorithmic, deductive, and geometric/spatial), thereby linking puzzle design to the cognitive operations required for solving. Synthesizing empirical evidence across these categories, we identify consistent limitations in current models, including brittle generalization, tight entanglement between perception and reasoning, and a persistent gap between fluent explanations and faithful execution. By framing visual puzzles as diagnostic instruments rather than task formats, this survey elaborates on the state of LVLM reasoning and outlines key directions for future benchmarks and reasoning-aware multimodal systems.

Method: Three key components: 1) OSM-prior-driven geometric construction using OpenStreetMap semantic topology to directly generate metric-consistent TSDF; 2) Geometry-aware dynamic filtering with quad-modal constraint field (normal/height/depth consistency) to reject moving vehicles and transient occlusions; 3) Illumination-robust fusion in CIELAB space with adaptive L-channel weighting and depth-gradient suppression.

Result: Achieves 30+ FPS on entry-level GTX 1660, SSIM 0.87 (+16.0% improvement), 15x end-to-end speedup, and 83.3% GPU memory reduction compared to state-of-the-art 3D Gaussian Splatting that requires high-end GPUs (RTX 4090D). Tested on 68,000 m² real-world dataset and outputs explicit triangle meshes compatible with Unity/Unreal pipelines.

Conclusion: ParkingTwin successfully addresses the trilemma of parking-lot reconstruction by providing a training-free, lightweight system for online streaming 3D reconstruction that achieves real-time performance on entry-level hardware while maintaining high fidelity and compatibility with digital-twin pipelines.

Abstract: High-fidelity parking-lot digital twins provide essential priors for path planning, collision checking, and perception validation in Automated Valet Parking (AVP). Yet robot-oriented reconstruction faces a trilemma: sparse forward-facing views cause weak parallax and ill-posed geometry; dynamic occlusions and extreme lighting hinder stable texture fusion; and neural rendering typically needs expensive offline optimization, violating edge-side streaming constraints. We propose ParkingTwin, a training-free, lightweight system for online streaming 3D reconstruction. First, OSM-prior-driven geometric construction uses OpenStreetMap semantic topology to directly generate a metric-consistent TSDF, replacing blind geometric search with deterministic mapping and avoiding costly optimization. Second, geometry-aware dynamic filtering employs a quad-modal constraint field (normal/height/depth consistency) to reject moving vehicles and transient occlusions in real time. Third, illumination-robust fusion in CIELAB decouples luminance and chromaticity via adaptive L-channel weighting and depth-gradient suppression, reducing seams under abrupt lighting changes. ParkingTwin runs at 30+ FPS on an entry-level GTX 1660. On a 68,000 m^2 real-world dataset, it achieves SSIM 0.87 (+16.0%), delivers about 15x end-to-end speedup, and reduces GPU memory by 83.3% compared with state-of-the-art 3D Gaussian Splatting (3DGS) that typically requires high-end GPUs (RTX 4090D). The system outputs explicit triangle meshes compatible with Unity/Unreal digital-twin pipelines. Project page: https://mihoutao-liu.github.io/ParkingTwin/

Method: Proposed MVGD-Net with three novel modules: Cross-scale Multimodal Fusion Module (CMFM) integrates spatial features and optical flow maps; History Guided Attention Module (HGAM) and Temporal Cross Attention Module (TCAM) enhance temporal features; Temporal-Spatial Decoder (TSD) fuses spatial and temporal features to generate glass region masks.

Result: Extensive experiments show MVGD-Net outperforms relevant state-of-the-art methods. The authors also created a large-scale dataset with 312 diverse glass scenarios and 19,268 frames for training and evaluation.

Conclusion: Motion inconsistency is an effective cue for glass surface detection in videos. The proposed MVGD-Net successfully leverages this observation through innovative modules and achieves superior performance compared to existing methods.

Abstract: Glass surface ubiquitous in both daily life and professional environments presents a potential threat to vision-based systems, such as robot and drone navigation. To solve this challenge, most recent studies have shown significant interest in Video Glass Surface Detection (VGSD). We observe that objects in the reflection (or transmission) layer appear farther from the glass surfaces. Consequently, in video motion scenarios, the notable reflected (or transmitted) objects on the glass surface move slower than objects in non-glass regions within the same spatial plane, and this motion inconsistency can effectively reveal the presence of glass surfaces. Based on this observation, we propose a novel network, named MVGD-Net, for detecting glass surfaces in videos by leveraging motion inconsistency cues. Our MVGD-Net features three novel modules: the Cross-scale Multimodal Fusion Module (CMFM) that integrates extracted spatial features and estimated optical flow maps, the History Guided Attention Module (HGAM) and Temporal Cross Attention Module (TCAM), both of which further enhances temporal features. A Temporal-Spatial Decoder (TSD) is also introduced to fuse the spatial and temporal features for generating the glass region mask. Furthermore, for learning our network, we also propose a large-scale dataset, which comprises 312 diverse glass scenarios with a total of 19,268 frames. Extensive experiments demonstrate that our MVGD-Net outperforms relevant state-of-the-art methods.

Method: Proposes Glance-or-Gaze (GoG) framework with Selective Gaze mechanism that dynamically chooses between glancing at global context or gazing into high-value regions to filter irrelevant information before retrieval. Uses dual-stage training: 1) Reflective GoG Behavior Alignment via supervised fine-tuning, and 2) Complexity-Adaptive Reinforcement Learning for iterative reasoning on complex queries.

Result: Achieves state-of-the-art performance across six benchmarks. Ablation studies confirm both Selective Gaze and complexity-adaptive RL are essential for effective visual search.

Conclusion: GoG successfully shifts from passive perception to active visual planning, overcoming limitations of existing methods by filtering visual redundancy and enabling deep iterative reasoning for complex visual queries.

Abstract: Large Multimodal Models (LMMs) have achieved remarkable success in visual understanding, yet they struggle with knowledge-intensive queries involving long-tail entities or evolving information due to static parametric knowledge. Recent search-augmented approaches attempt to address this limitation, but existing methods rely on indiscriminate whole-image retrieval that introduces substantial visual redundancy and noise, and lack deep iterative reflection, limiting their effectiveness on complex visual queries. To overcome these challenges, we propose Glance-or-Gaze (GoG), a fully autonomous framework that shifts from passive perception to active visual planning. GoG introduces a Selective Gaze mechanism that dynamically chooses whether to glance at global context or gaze into high-value regions, filtering irrelevant information before retrieval. We design a dual-stage training strategy: Reflective GoG Behavior Alignment via supervised fine-tuning instills the fundamental GoG paradigm, while Complexity-Adaptive Reinforcement Learning further enhances the model’s capability to handle complex queries through iterative reasoning. Experiments across six benchmarks demonstrate state-of-the-art performance. Ablation studies confirm that both Selective Gaze and complexity-adaptive RL are essential for effective visual search. We will release our data and models for further exploration soon.

Method: Proposes Facial Spatiotemporal Graph (STGraph) representation that encodes facial color and structure using 3D facial mesh sequences. Introduces MeshPhys, a lightweight spatiotemporal graph convolutional network that operates on the STGraph to estimate physiological signals with surface-aligned processing.

Result: Achieves state-of-the-art or competitive performance across four benchmark datasets in both intra- and cross-dataset settings. Ablation studies show that constraining receptive fields to facial surface acts as a strong structural prior, and surface-aligned 3D-aware node features are critical for robust encoding.

Conclusion: The STGraph and MeshPhys constitute a novel, principled modeling paradigm for facial rPPG that enables robust, interpretable, and generalizable physiological signal estimation by explicitly aligning processing with the 3D facial surface.

Abstract: Facial remote photoplethysmography (rPPG) methods estimate physiological signals by modeling subtle color changes on the 3D facial surface over time. However, existing methods fail to explicitly align their receptive fields with the 3D facial surface-the spatial support of the rPPG signal. To address this, we propose the Facial Spatiotemporal Graph (STGraph), a novel representation that encodes facial color and structure using 3D facial mesh sequences-enabling surface-aligned spatiotemporal processing. We introduce MeshPhys, a lightweight spatiotemporal graph convolutional network that operates on the STGraph to estimate physiological signals. Across four benchmark datasets, MeshPhys achieves state-of-the-art or competitive performance in both intra- and cross-dataset settings. Ablation studies show that constraining the model’s receptive field to the facial surface acts as a strong structural prior, and that surface-aligned, 3D-aware node features are critical for robustly encoding facial surface color. Together, the STGraph and MeshPhys constitute a novel, principled modeling paradigm for facial rPPG, enabling robust, interpretable, and generalizable estimation. Code is available at https://samcantrill.github.io/facial-stgraph-rppg/ .

Method: Proposed History Injection mechanism for Transformer models (HiT) that maintains historical context from previous observations while reducing data storage by over 99% of original image size. Implemented within Prithvi-tiny foundation model for flood detection.

Result: HiT mechanism maintains detection accuracy compared to bitemporal baseline on STTORM-CD flood dataset. HiT-Prithvi model achieved 43 FPS on Jetson Orin Nano (representative onboard hardware for nanosats), enabling real-time processing.

Conclusion: Establishes practical framework for satellite-based continuous monitoring of natural disasters, supporting real-time hazard assessment without dependency on ground-based processing infrastructure. Code and models available open-source.

Abstract: Natural disaster monitoring through continuous satellite observation requires processing multi-temporal data under strict operational constraints. This paper addresses flood detection, a critical application for hazard management, by developing an onboard change detection system that operates within the memory and computational limits of small satellites. We propose History Injection mechanism for Transformer models (HiT), that maintains historical context from previous observations while reducing data storage by over 99% of original image size. Moreover, testing on the STTORM-CD flood dataset confirms that the HiT mechanism within the Prithvi-tiny foundation model maintains detection accuracy compared to the bitemporal baseline. The proposed HiT-Prithvi model achieved 43 FPS on Jetson Orin Nano, a representative onboard hardware used in nanosats. This work establishes a practical framework for satellite-based continuous monitoring of natural disasters, supporting real-time hazard assessment without dependency on ground-based processing infrastructure. Architecture as well as model checkpoints is available at https://github.com/zaitra/HiT-change-detection

Method: Introduces a universal, modular abstention framework with two key components: 1) an informed regularization term to guide abstention behavior, and 2) a flexible power-law-based auto-tuning algorithm for abstention penalty. The framework is integrated with three loss functions to create novel variants: GAC, SAC, and ADS.

Result: Experiments on CaDIS and DSAD medical datasets show the methods consistently and significantly outperform non-abstaining baselines, especially under high noise levels. The framework demonstrates versatility across different loss functions.

Conclusion: Enabling models to selectively ignore corrupted samples is a powerful and generalizable strategy for building more reliable segmentation models. The abstention mechanism effectively addresses label noise in medical image segmentation.

Abstract: Label noise is a critical problem in medical image segmentation, often arising from the inherent difficulty of manual annotation. Models trained on noisy data are prone to overfitting, which degrades their generalization performance. While a number of methods and strategies have been proposed to mitigate noisy labels in the segmentation domain, this area remains largely under-explored. The abstention mechanism has proven effective in classification tasks by enhancing the capabilities of Cross Entropy, yet its potential in segmentation remains unverified. In this paper, we address this gap by introducing a universal and modular abstention framework capable of enhancing the noise-robustness of a diverse range of loss functions. Our framework improves upon prior work with two key components: an informed regularization term to guide abstention behaviour, and a more flexible power-law-based auto-tuning algorithm for the abstention penalty. We demonstrate the framework’s versatility by systematically integrating it with three distinct loss functions to create three novel, noise-robust variants: GAC, SAC, and ADS. Experiments on the CaDIS and DSAD medical datasets show our methods consistently and significantly outperform their non-abstaining baselines, especially under high noise levels. This work establishes that enabling models to selectively ignore corrupted samples is a powerful and generalizable strategy for building more reliable segmentation models. Our code is publicly available at https://github.com/wemous/abstention-for-segmentation.

Method: Pairs a frozen pre-trained VLM with a lightweight encoding model, extracts hidden states from final tokens of each layer, and trains a simple encoder on pooled representations.

Result: Significantly advances state-of-the-art with substantial gains in Recall@1 (+27.23 and +69.59), shows robustness across VLM backbones, and exhibits strong zero-shot generalization.

Conclusion: PREGEN is an efficient and powerful CoVR framework that overcomes limitations of current methods, demonstrating effectiveness and semantic capabilities without VLM fine-tuning.

Abstract: Composed Video Retrieval (CoVR) aims to retrieve a video based on a query video and a modifying text. Current CoVR methods fail to fully exploit modern Vision-Language Models (VLMs), either using outdated architectures or requiring computationally expensive fine-tuning and slow caption generation. We introduce PREGEN (PRE GENeration extraction), an efficient and powerful CoVR framework that overcomes these limitations. Our approach uniquely pairs a frozen, pre-trained VLM with a lightweight encoding model, eliminating the need for any VLM fine-tuning. We feed the query video and modifying text into the VLM and extract the hidden state of the final token from each layer. A simple encoder is then trained on these pooled representations, creating a semantically rich and compact embedding for retrieval. PREGEN significantly advances the state of the art, surpassing all prior methods on standard CoVR benchmarks with substantial gains in Recall@1 of +27.23 and +69.59. Our method demonstrates robustness across different VLM backbones and exhibits strong zero-shot generalization to more complex textual modifications, highlighting its effectiveness and semantic capabilities.

Method: SVGFormer uses content-aware grouping to partition volumes into semantic supervoxel graphs, then employs a hierarchical encoder combining patch-level Transformer with supervoxel-level Graph Attention Network to model intra-region features and inter-regional dependencies.

Result: On BraTS dataset, classification model achieved F1-score of 0.875 and regression model achieved MAE of 0.028, demonstrating strong performance for both node-level classification and tumor proportion regression tasks.

Conclusion: Graph-based, encoder-only paradigm offers accurate and inherently interpretable alternative for 3D medical image representation with dual-scale explainability from patch to region level.

Abstract: Modern vision backbones for 3D medical imaging typically process dense voxel grids through parameter-heavy encoder-decoder structures, a design that allocates a significant portion of its parameters to spatial reconstruction rather than feature learning. Our approach introduces SVGFormer, a decoder-free pipeline built upon a content-aware grouping stage that partitions the volume into a semantic graph of supervoxels. Its hierarchical encoder learns rich node representations by combining a patch-level Transformer with a supervoxel-level Graph Attention Network, jointly modeling fine-grained intra-region features and broader inter-regional dependencies. This design concentrates all learnable capacity on feature encoding and provides inherent, dual-scale explainability from the patch to the region level. To validate the framework’s flexibility, we trained two specialized models on the BraTS dataset: one for node-level classification and one for tumor proportion regression. Both models achieved strong performance, with the classification model achieving a F1-score of 0.875 and the regression model a MAE of 0.028, confirming the encoder’s ability to learn discriminative and localized features. Our results establish that a graph-based, encoder-only paradigm offers an accurate and inherently interpretable alternative for 3D medical image representation.

Method: Extends Linear Discriminant Analysis into deep learning context with Colorspace Discriminant Analysis (CSDA), maximizing signed inter-class separability while minimizing intra-class variability using generalized discriminative loss. Introduces three alternative losses for stable end-to-end optimization.

Result: Experiments on wind turbine blade data show significant accuracy gains, demonstrating the importance of tailored preprocessing for domain-specific segmentation.

Conclusion: CSDA effectively improves segmentation by customizing color representation, highlighting the critical role of proper preprocessing in domain-specific computer vision tasks.

Abstract: Suboptimal color representation often hinders accurate image segmentation, yet many modern algorithms neglect this critical preprocessing step. This work presents a novel multidimensional nonlinear discriminant analysis algorithm, Colorspace Discriminant Analysis (CSDA), for improved segmentation. Extending Linear Discriminant Analysis into a deep learning context, CSDA customizes color representation by maximizing multidimensional signed inter-class separability while minimizing intra-class variability through a generalized discriminative loss. To ensure stable training, we introduce three alternative losses that enable end-to-end optimization of both the discriminative colorspace and segmentation process. Experiments on wind turbine blade data demonstrate significant accuracy gains, emphasizing the importance of tailored preprocessing in domain-specific segmentation.

Method: Introduces POCI-Diff framework with: 1) Joint enforcement of 3D geometric constraints and instance-level semantic binding in unified diffusion, 2) Blended Latent Diffusion for binding text descriptions to 3D bounding boxes, 3) Warping-free editing pipeline using regeneration instead of pixel deformation, 4) IP-Adapter conditioning for preserving object identity across edits.

Result: Produces high-quality images consistent with specified 3D layouts and edits, outperforms state-of-the-art methods in visual fidelity and layout adherence while eliminating warping-induced geometric artifacts.

Conclusion: POCI-Diff enables consistent and interactive 3D layout control for text-to-image generation with superior geometric preservation and editing coherence compared to existing approaches.

Abstract: We propose a diffusion-based approach for Text-to-Image (T2I) generation with consistent and interactive 3D layout control and editing. While prior methods improve spatial adherence using 2D cues or iterative copy-warp-paste strategies, they often distort object geometry and fail to preserve consistency across edits. To address these limitations, we introduce a framework for Positioning Objects Consistently and Interactively (POCI-Diff), a novel formulation for jointly enforcing 3D geometric constraints and instance-level semantic binding within a unified diffusion process. Our method enables explicit per-object semantic control by binding individual text descriptions to specific 3D bounding boxes through Blended Latent Diffusion, allowing one-shot synthesis of complex multi-object scenes. We further propose a warping-free generative editing pipeline that supports object insertion, removal, and transformation via regeneration rather than pixel deformation. To preserve object identity and consistency across edits, we condition the diffusion process on reference images using IP-Adapter, enabling coherent object appearance throughout interactive 3D editing while maintaining global scene coherence. Experimental results demonstrate that POCI-Diff produces high-quality images consistent with the specified 3D layouts and edits, outperforming state-of-the-art methods in both visual fidelity and layout adherence while eliminating warping-induced geometric artifacts.

Method: Directly learns per-pixel Gaussian representation from few input images, uses transformer-based encoder to fuse DINOv3 and Stable Diffusion VAE features. Extends Gaussian representations with per-Gaussian features and lightweight MLP-based dynamic network for real-time animation. Uses point maps from pre-trained large reconstruction model for geometry supervision.

Result: Significantly outperforms existing methods in both rendering quality and inference efficiency while supporting real-time dynamic avatar animation.

Conclusion: The proposed method enables efficient generation of high-fidelity Gaussian head avatars from minimal input with real-time animation capabilities, addressing scalability limitations of current approaches.

Abstract: Despite recent progress in 3D Gaussian-based head avatar modeling, efficiently generating high fidelity avatars remains a challenge. Current methods typically rely on extensive multi-view capture setups or monocular videos with per-identity optimization during inference, limiting their scalability and ease of use on unseen subjects. To overcome these efficiency drawbacks, we propose \OURS, a feed-forward method to generate high-quality Gaussian head avatars from only a few input images while supporting real-time animation. Our approach directly learns a per-pixel Gaussian representation from the input images, and aggregates multi-view information using a transformer-based encoder that fuses image features from both DINOv3 and Stable Diffusion VAE. For real-time animation, we extend the explicit Gaussian representations with per-Gaussian features and introduce a lightweight MLP-based dynamic network to predict 3D Gaussian deformations from expression codes. Furthermore, to enhance geometric smoothness of the 3D head, we employ point maps from a pre-trained large reconstruction model as geometry supervision. Experiments show that our approach significantly outperforms existing methods in both rendering quality and inference efficiency, while supporting real-time dynamic avatar animation.

Method: 1) Use deep feature extractor network to get representative video features without class/task info. 2) Progressively build series of deep clusters from extracted features. 3) Transfer knowledge by using previous task’s model as initial state for current task learning.

Result: Significantly outperforms other baselines on three standard video action recognition datasets (UCF101, HMDB51, Something-to-Something V2) in unsupervised setting.

Conclusion: Proposes effective unsupervised video CIL approach that learns without labels or task boundaries, demonstrating strong performance on benchmark datasets and addressing practical limitations of supervised methods.

Abstract: Unsupervised video class incremental learning (uVCIL) represents an important learning paradigm for learning video information without forgetting, and without considering any data labels. Prior approaches have focused on supervised class-incremental learning, relying on using the knowledge of labels and task boundaries, which is costly, requires human annotation, or is simply not a realistic option. In this paper, we propose a simple yet effective approach to address the uVCIL. We first consider a deep feature extractor network, providing a set of representative video features during each task without assuming any class or task information. We then progressively build a series of deep clusters from the extracted features. During the successive task learning, the model updated from the previous task is used as an initial state in order to transfer knowledge to the current learning task. We perform in-depth evaluations on three standard video action recognition datasets, including UCF101, HMDB51, and Something-to-Something V2, by ignoring the labels from the supervised setting. Our approach significantly outperforms other baselines on all datasets.

Method: Created DisasterVQA dataset with 1,395 real disaster images and 4,405 expert-curated question-answer pairs spanning floods, wildfires, earthquakes. Questions are grounded in humanitarian frameworks (FEMA ESF, OCHA MIRA) and cover binary, multiple-choice, and open-ended formats for situational awareness and operational decision-making.

Result: Benchmarked 7 state-of-the-art vision-language models showing performance variability across question types, disaster categories, regions, and humanitarian tasks. Models achieve high accuracy on binary questions but struggle with fine-grained quantitative reasoning, object counting, and context-sensitive interpretation, especially for underrepresented disaster scenarios.

Conclusion: DisasterVQA provides a challenging, practical benchmark to guide development of more robust and operationally meaningful vision-language models for disaster response, addressing current limitations in fine-grained reasoning and context sensitivity.

Abstract: Social media imagery provides a low-latency source of situational information during natural and human-induced disasters, enabling rapid damage assessment and response. While Visual Question Answering (VQA) has shown strong performance in general-purpose domains, its suitability for the complex and safety-critical reasoning required in disaster response remains unclear. We introduce DisasterVQA, a benchmark dataset designed for perception and reasoning in crisis contexts. DisasterVQA consists of 1,395 real-world images and 4,405 expert-curated question-answer pairs spanning diverse events such as floods, wildfires, and earthquakes. Grounded in humanitarian frameworks including FEMA ESF and OCHA MIRA, the dataset includes binary, multiple-choice, and open-ended questions covering situational awareness and operational decision-making tasks. We benchmark seven state-of-the-art vision-language models and find performance variability across question types, disaster categories, regions, and humanitarian tasks. Although models achieve high accuracy on binary questions, they struggle with fine-grained quantitative reasoning, object counting, and context-sensitive interpretation, particularly for underrepresented disaster scenarios. DisasterVQA provides a challenging and practical benchmark to guide the development of more robust and operationally meaningful vision-language models for disaster response. The dataset is publicly available at https://zenodo.org/records/18267770.

Method: Proposes a two-stream transformer architecture that extracts spatio-temporal information from both content (frames) and optical flow streams. The model uses self-attention mechanisms to capture relationships across the joint optical flow and temporal frame domains within transformer encoder blocks.

Result: The proposed methodology achieves excellent classification results on three well-known human activity video datasets, demonstrating the effectiveness of combining content and optical flow information in a transformer framework.

Conclusion: The two-stream transformer approach successfully leverages both appearance and motion information for video classification, showing that transformer networks can effectively process joint optical flow and temporal features for improved video understanding.

Abstract: Motion representation plays an important role in video understanding and has many applications including action recognition, robot and autonomous guidance or others. Lately, transformer networks, through their self-attention mechanism capabilities, have proved their efficiency in many applications. In this study, we introduce a new two-stream transformer video classifier, which extracts spatio-temporal information from content and optical flow representing movement information. The proposed model identifies self-attention features across the joint optical flow and temporal frame domain and represents their relationships within the transformer encoder mechanism. The experimental results show that our proposed methodology provides excellent classification results on three well-known video datasets of human activities.

Method: Introduces Deep Discriminant Analysis (DDA) with two stable loss functions that incorporate signed between-class variance, bound outputs with sigmoid, and convert multiplicative relationships to additive ones. Augments with probability loss to create Probabilistic DDA (PDDA) that minimizes class overlap in output distributions.

Result: PDDA produces highly confident predictions with reduced within-class variance. When applied to wind blade segmentation, it shows notable advances in performance and consistency, marking the first application of DDA to image segmentation.

Conclusion: PDDA effectively addresses limitations of traditional discriminant analysis by combining deep networks with stable optimization techniques, demonstrating practical value in wind energy maintenance applications through improved segmentation performance.

Abstract: Linear discriminant analysis improves class separability but struggles with non-linearly separable data. To overcome this, we introduce Deep Discriminant Analysis (DDA), which directly optimizes the Fisher criterion utilizing deep networks. To ensure stable training and avoid computational instabilities, we incorporate signed between-class variance, bound outputs with a sigmoid function, and convert multiplicative relationships into additive ones. We present two stable DDA loss functions and augment them with a probability loss, resulting in Probabilistic DDA (PDDA). PDDA effectively minimizes class overlap in output distributions, producing highly confident predictions with reduced within-class variance. When applied to wind blade segmentation, PDDA showcases notable advances in performance and consistency, critical for wind energy maintenance. To our knowledge, this is the first application of DDA to image segmentation.

Method: Leverages Segment Anything Model 2 (SAM2) foundation model with a custom threshold-based variant of the First Integer Neighbor Clustering Hierarchy (FINCH) algorithm for training-free multi-class object counting.

Result: Achieves competitive performance on FSC-147 and CARPK benchmarks, proposes synthetic multi-class dataset and F1 score as better evaluation metrics.

Conclusion: OCCAM demonstrates effective training-free multi-class object counting without supplementary information, offering a practical solution with publicly available code and dataset.

Abstract: Class-Agnostic object Counting (CAC) involves counting instances of objects from arbitrary classes within an image. Due to its practical importance, CAC has received increasing attention in recent years. Most existing methods assume a single object class per image, rely on extensive training of large deep learning models and address the problem by incorporating additional information, such as visual exemplars or text prompts. In this paper, we present OCCAM, the first training-free approach to CAC that operates without the need of any supplementary information. Moreover, our approach addresses the multi-class variant of the problem, as it is capable of counting the object instances in each and every class among arbitrary object classes within an image. We leverage Segment Anything Model 2 (SAM2), a foundation model, and a custom threshold-based variant of the First Integer Neighbor Clustering Hierarchy (FINCH) algorithm to achieve competitive performance on widely used benchmark datasets, FSC-147 and CARPK. We propose a synthetic multi-class dataset and F1 score as a more suitable evaluation metric. The code for our method and the proposed synthetic dataset will be made publicly available at https://mikespanak.github.io/OCCAM_counter.

Method: MTV jointly optimizes a shared backbone across three objectives: vision-language contrastive learning, self-supervised learning, and dense spatial supervision. To avoid manual annotations, it uses high-capacity “expert” models (Depth Anything V2 and OWLv2) to generate dense pseudo-labels at scale. The paper also systematically analyzes multi-task learning mechanics.

Result: MTV achieves “best-of-both-worlds” performance, significantly enhancing fine-grained spatial reasoning without compromising global semantic understanding. The framework demonstrates that multi-task learning with high-quality pseudo-supervision is scalable for creating more general visual encoders.

Conclusion: Multi-task learning that combines vision-language contrastive, self-supervised, and dense spatial objectives, enhanced by pseudo-labels from expert models, provides a scalable path toward more general and capable visual encoders that excel at both global semantics and local spatial reasoning.

Abstract: Current visual representation learning remains bifurcated: vision-language models (e.g., CLIP) excel at global semantic alignment but lack spatial precision, while self-supervised methods (e.g., MAE, DINO) capture intricate local structures yet struggle with high-level semantic context. We argue that these paradigms are fundamentally complementary and can be integrated into a principled multi-task framework, further enhanced by dense spatial supervision. We introduce MTV, a multi-task visual pretraining framework that jointly optimizes a shared backbone across vision-language contrastive, self-supervised, and dense spatial objectives. To mitigate the need for manual annotations, we leverage high-capacity “expert” models – such as Depth Anything V2 and OWLv2 – to synthesize dense, structured pseudo-labels at scale. Beyond the framework, we provide a systematic investigation into the mechanics of multi-task visual learning, analyzing: (i) the marginal gain of each objective, (ii) task synergies versus interference, and (iii) scaling behavior across varying data and model scales. Our results demonstrate that MTV achieves “best-of-both-worlds” performance, significantly enhancing fine-grained spatial reasoning without compromising global semantic understanding. Our findings suggest that multi-task learning, fueled by high-quality pseudo-supervision, is a scalable path toward more general visual encoders.

Method: MIRACLE uses hyperspherical embedding space fusion to integrate heterogeneous preoperative clinical and radiological data, combined with an interventional deep learning module that allows domain experts to adjust predictions based on clinical expertise.

Result: MIRACLE outperforms traditional machine learning models and contemporary LLM variants on the POC-L dataset of 3,094 lung cancer surgery patients from Roswell Park Comprehensive Cancer Center.

Conclusion: The MIRACLE architecture provides an effective solution for personalized and explainable postoperative risk management by combining multimodal data fusion with interpretable, clinically adjustable predictions.

Abstract: Postoperative complications remain a critical concern in clinical practice, adversely affecting patient outcomes and contributing to rising healthcare costs. We present MIRACLE, a deep learning architecture for prediction of risk of postoperative complications in lung cancer surgery by integrating preoperative clinical and radiological data. MIRACLE employs a hyperspherical embedding space fusion of heterogeneous inputs, enabling the extraction of robust, discriminative features from both structured clinical records and high-dimensional radiological images. To enhance transparency of prediction and clinical utility, we incorporate an interventional deep learning module in MIRACLE, that not only refines predictions but also provides interpretable and actionable insights, allowing domain experts to interactively adjust recommendations based on clinical expertise. We validate our approach on POC-L, a real-world dataset comprising 3,094 lung cancer patients who underwent surgery at Roswell Park Comprehensive Cancer Center. Our results demonstrate that MIRACLE outperforms various traditional machine learning models and contemporary large language models (LLM) variants alone, for personalized and explainable postoperative risk management.

Method: Augments deep two-sample tests with sample-level and feature-level explanations that reveal which individual samples and which input features drive statistically significant group differences. The framework highlights influential image regions and individual samples that contribute most to detected group differences.

Result: Applied to biomedical imaging data, the framework successfully identifies influential samples and highlights anatomically meaningful regions associated with disease-related variation, providing spatial and instance-wise insight into test decisions.

Conclusion: Bridges statistical inference and explainable AI, enabling interpretable, label-free population analysis in medical imaging by making deep two-sample tests explainable at both sample and feature levels.

Abstract: Deep neural two-sample tests have recently shown strong power for detecting distributional differences between groups, yet their black-box nature limits interpretability and practical adoption in biomedical analysis. Moreover, most existing post-hoc explainability methods rely on class labels, making them unsuitable for label-free statistical testing settings. We propose an explainable deep statistical testing framework that augments deep two-sample tests with sample-level and feature-level explanations, revealing which individual samples and which input features drive statistically significant group differences. Our method highlights which image regions and which individual samples contribute most to the detected group difference, providing spatial and instance-wise insight into the test’s decision. Applied to biomedical imaging data, the proposed framework identifies influential samples and highlights anatomically meaningful regions associated with disease-related variation. This work bridges statistical inference and explainable AI, enabling interpretable, label-free population analysis in medical imaging.

Method: The paper proposes using 2D rotation equivariance learned through data augmentation rather than equivariant-by-design architectures. This involves training models to be equivariant to image plane rotations through augmentation techniques.

Result: The approach improves model performance on human poses with rotations in the image plane, outperforming state-of-the-art equivariant-by-design methods on common HPE benchmarks.

Conclusion: Learning rotation equivariance through augmentation is more efficient and straightforward than equivariant-by-design methods, leading to better performance in monocular 3D human pose estimation.

Abstract: Estimating 3D from 2D is one of the central tasks in computer vision. In this work, we consider the monocular setting, i.e. single-view input, for 3D human pose estimation (HPE). Here, the task is to predict a 3D point set of human skeletal joints from a single 2D input image. While by definition this is an ill-posed problem, recent work has presented methods that solve it with up to several-centimetre error. Typically, these methods employ a two-step approach, where the first step is to detect the 2D skeletal joints in the input image, followed by the step of 2D-to-3D lifting. We find that common lifting models fail when encountering a rotated input. We argue that learning a single human pose along with its in-plane rotations is considerably easier and more geometrically grounded than directly learning a point-to-point mapping. Furthermore, our intuition is that endowing the model with the notion of rotation equivariance without explicitly constraining its parameter space should lead to a more straightforward learning process than one with equivariance by design. Utilising the common HPE benchmarks, we confirm that the 2D rotation equivariance per se improves the model performance on human poses akin to rotations in the image plane, and can be efficiently and straightforwardly learned by augmentation, outperforming state-of-the-art equivariant-by-design methods.

Method: 1) VideoMaMa: Uses pretrained video diffusion models to convert coarse segmentation masks into accurate alpha mattes. 2) Pseudo-labeling pipeline: Creates MA-V dataset with 50K+ real-world videos. 3) SAM2-Matte: Fine-tunes SAM2 on MA-V dataset for improved video matting.

Result: VideoMaMa shows strong zero-shot generalization to real videos despite synthetic-only training. SAM2-Matte outperforms models trained on existing matting datasets in robustness on in-the-wild videos. MA-V provides high-quality annotations for diverse video content.

Conclusion: Large-scale pseudo-labeled video matting datasets are crucial for progress. Generative priors and accessible segmentation cues enable scalable video matting research, bridging the gap between synthetic training and real-world application.

Abstract: Generalizing video matting models to real-world videos remains a significant challenge due to the scarcity of labeled data. To address this, we present Video Mask-to-Matte Model (VideoMaMa) that converts coarse segmentation masks into pixel accurate alpha mattes, by leveraging pretrained video diffusion models. VideoMaMa demonstrates strong zero-shot generalization to real-world footage, even though it is trained solely on synthetic data. Building on this capability, we develop a scalable pseudo-labeling pipeline for large-scale video matting and construct the Matting Anything in Video (MA-V) dataset, which offers high-quality matting annotations for more than 50K real-world videos spanning diverse scenes and motions. To validate the effectiveness of this dataset, we fine-tune the SAM2 model on MA-V to obtain SAM2-Matte, which outperforms the same model trained on existing matting datasets in terms of robustness on in-the-wild videos. These findings emphasize the importance of large-scale pseudo-labeled video matting and showcase how generative priors and accessible segmentation cues can drive scalable progress in video matting research.

Method: Proposes TrackletGPT, a language-like GPT framework that reintroduces sequential information using tracklets (granular sub-streamline segments). The method scales and refines GPT models for tractography segmentation, making it fully automatic and dataset-generalizable.

Result: TrackletGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores on both TractoInferno and HCP datasets, including in inter-dataset experiments.

Conclusion: TrackletGPT successfully addresses tract segmentation challenges by leveraging sequential information through tracklets, enabling superior performance and generalization across different datasets and conditions.

Abstract: White Matter Tract Segmentation is imperative for studying brain structural connectivity, neurological disorders and neurosurgery. This task remains complex, as tracts differ among themselves, across subjects and conditions, yet have similar 3D structure across hemispheres and subjects. To address these challenges, we propose TrackletGPT, a language-like GPT framework which reintroduces sequential information in tokens using tracklets. TrackletGPT generalises seamlessly across datasets, is fully automatic, and encodes granular sub-streamline segments, Tracklets, scaling and refining GPT models in Tractography Segmentation. Based on our experiments, TrackletGPT outperforms state-of-the-art methods on average DICE, Overlap and Overreach scores on TractoInferno and HCP datasets, even on inter-dataset experiments.

Method: Created VTONGuard dataset with 775,000+ real and synthetic try-on images covering diverse conditions (pose, background, garment styles). Conducted systematic evaluation of multiple detection paradigms under unified protocols. Proposed multi-task framework integrating auxiliary segmentation for boundary-aware feature learning.

Result: Revealed strengths/weaknesses of existing detection methods and highlighted cross-paradigm generalization challenges. The proposed multi-task framework achieved best overall performance on the VTONGuard benchmark.

Conclusion: VTONGuard enables fair comparisons, facilitates development of more robust detection models, and promotes safe, responsible deployment of virtual try-on technologies in practice.

Abstract: With the rapid advancement of generative AI, virtual try-on (VTON) systems are becoming increasingly common in e-commerce and digital entertainment. However, the growing realism of AI-generated try-on content raises pressing concerns about authenticity and responsible use. To address this, we present VTONGuard, a large-scale benchmark dataset containing over 775,000 real and synthetic try-on images. The dataset covers diverse real-world conditions, including variations in pose, background, and garment styles, and provides both authentic and manipulated examples. Based on this benchmark, we conduct a systematic evaluation of multiple detection paradigms under unified training and testing protocols. Our results reveal each method’s strengths and weaknesses and highlight the persistent challenge of cross-paradigm generalization. To further advance detection, we design a multi-task framework that integrates auxiliary segmentation to enhance boundary-aware feature learning, achieving the best overall performance on VTONGuard. We expect this benchmark to enable fair comparisons, facilitate the development of more robust detection models, and promote the safe and responsible deployment of VTON technologies in practice.

Method: DExTeR builds on Point-DETR, encoding single-point annotations as object queries. It uses class-guided deformable attention to guide attention sampling using point coordinates and class labels for class-specific feature extraction. CLICK-MoE (Class, Instance, and Common Knowledge Mixture of Experts) decouples class and instance representations to reduce confusion among adjacent/overlapping instances. Multi-point training strategy promotes prediction consistency across different point placements.

Result: DExTeR achieves state-of-the-art performance across three medical imaging datasets spanning endoscopy, chest X-rays, and endoscopic ultrasound, demonstrating its ability to reduce annotation costs while maintaining high detection accuracy in challenging medical scenarios.

Conclusion: DExTeR effectively addresses medical imaging challenges in point-to-box regression through transformer architecture with specialized attention mechanisms and representation decoupling, offering a practical solution for reducing annotation burden while preserving detection performance across diverse medical domains.

Abstract: Detecting anatomical landmarks in medical imaging is essential for diagnosis and intervention guidance. However, object detection models rely on costly bounding box annotations, limiting scalability. Weakly Semi-Supervised Object Detection (WSSOD) with point annotations proposes annotating each instance with a single point, minimizing annotation time while preserving localization signals. A Point-to-Box teacher model, trained on a small box-labeled subset, converts these point annotations into pseudo-box labels to train a student detector. Yet, medical imagery presents unique challenges, including overlapping anatomy, variable object sizes, and elusive structures, which hinder accurate bounding box inference. To overcome these challenges, we introduce DExTeR (DETR with Experts), a transformer-based Point-to-Box regressor tailored for medical imaging. Built upon Point-DETR, DExTeR encodes single-point annotations as object queries, refining feature extraction with the proposed class-guided deformable attention, which guides attention sampling using point coordinates and class labels to capture class-specific characteristics. To improve discrimination in complex structures, it introduces CLICK-MoE (CLass, Instance, and Common Knowledge Mixture of Experts), decoupling class and instance representations to reduce confusion among adjacent or overlapping instances. Finally, we implement a multi-point training strategy which promotes prediction consistency across different point placements, improving robustness to annotation variability. DExTeR achieves state-of-the-art performance across three datasets spanning different medical domains (endoscopy, chest X-rays, and endoscopic ultrasound) highlighting its potential to reduce annotation costs while maintaining high detection accuracy.

Method: STEC builds on Spatio-Temporal Frame Entropy (STFE) which measures per-frame spatial information via entropy-based structural complexity. STEC then evaluates sampled frames based on their temporal coverage and redundancy, jointly modeling spatial information strength, temporal dispersion, and non-redundancy.

Result: Experiments on MSR-VTT test-1k benchmark show STEC clearly differentiates common sampling strategies (random, uniform, content-aware methods). STEC also reveals robustness patterns across individual videos that aren’t captured by average performance alone.

Conclusion: STEC provides a principled, lightweight, task-agnostic diagnostic tool for analyzing frame sampling behavior under constrained budgets, offering practical value as a general-purpose evaluation metric for efficient video understanding pipelines.

Abstract: Frame sampling is a fundamental component in video understanding and video–language model pipelines, yet evaluating the quality of sampled frames remains challenging. Existing evaluation metrics primarily focus on perceptual quality or reconstruction fidelity, and are not designed to assess whether a set of sampled frames adequately captures informative and representative video content. We propose Spatio-Temporal Entropy Coverage (STEC), a simple and non-reference metric for evaluating the effectiveness of video frame sampling. STEC builds upon Spatio-Temporal Frame Entropy (STFE), which measures per-frame spatial information via entropy-based structural complexity, and evaluates sampled frames based on their temporal coverage and redundancy. By jointly modeling spatial information strength, temporal dispersion, and non-redundancy, STEC provides a principled and lightweight measure of sampling quality. Experiments on the MSR-VTT test-1k benchmark demonstrate that STEC clearly differentiates common sampling strategies, including random, uniform, and content-aware methods. We further show that STEC reveals robustness patterns across individual videos that are not captured by average performance alone, highlighting its practical value as a general-purpose evaluation tool for efficient video understanding. We emphasize that STEC is not designed to predict downstream task accuracy, but to provide a task-agnostic diagnostic signal for analyzing frame sampling behavior under constrained budgets.

Method: Developed a Unified Information Pipeline that standardizes heterogeneous datasets into comparable information flow, evaluated a fixed deployment-relevant detector under controlled cross-domain protocols, and benchmarked inference on low-cost edge hardware for operational feasibility.

Result: Found that structural factors (scene composition, object density, contextual redundancy) explain cross-domain performance loss more strongly than visual degradation like turbidity, identified “Context Collapse” failure mode in sparse scenes, and demonstrated practical sampling rates on edge hardware.

Conclusion: Shifts emphasis from image enhancement toward structure-aware reliability, providing a democratized tool for consistent marine ecosystem assessment that enables scalable, reliable monitoring across different underwater environments.

Abstract: Marine biodiversity monitoring requires scalability and reliability across complex underwater environments to support conservation and invasive-species management. Yet existing detection solutions often exhibit a pronounced deployment gap, with performance degrading sharply when transferred to new sites. This work establishes the foundational detection layer for a multi-year invasive species monitoring initiative targeting Arctic and Atlantic marine ecosystems. We address this challenge by developing a Unified Information Pipeline that standardises heterogeneous datasets into a comparable information flow and evaluates a fixed, deployment-relevant detector under controlled cross-domain protocols. Across multiple domains, we find that structural factors, such as scene composition, object density, and contextual redundancy, explain cross-domain performance loss more strongly than visual degradation such as turbidity, with sparse scenes inducing a characteristic “Context Collapse” failure mode. We further validate operational feasibility by benchmarking inference on low-cost edge hardware, showing that runtime optimisation enables practical sampling rates for remote monitoring. The results shift emphasis from image enhancement toward structure-aware reliability, providing a democratised tool for consistent marine ecosystem assessment.

Method: FantasyVLN encodes imagined visual tokens into a compact latent space using a pretrained Visual AutoRegressor during CoT reasoning training, and jointly learns from textual, visual, and multimodal CoT modes under a unified multi-CoT strategy. At inference, it performs direct instruction-to-action mapping while maintaining reasoning-aware representations.

Result: Extensive experiments on LH-VLN show the approach achieves reasoning-aware yet real-time navigation, improving success rates and efficiency while reducing inference latency by an order of magnitude compared to explicit CoT methods.

Conclusion: FantasyVLN provides a practical solution that preserves the interpretability and planning benefits of CoT reasoning while eliminating the computational overhead that made previous approaches impractical for real-time navigation.

Abstract: Achieving human-level performance in Vision-and-Language Navigation (VLN) requires an embodied agent to jointly understand multimodal instructions and visual-spatial context while reasoning over long action sequences. Recent works, such as NavCoT and NavGPT-2, demonstrate the potential of Chain-of-Thought (CoT) reasoning for improving interpretability and long-horizon planning. Moreover, multimodal extensions like OctoNav-R1 and CoT-VLA further validate CoT as a promising pathway toward human-like navigation reasoning. However, existing approaches face critical drawbacks: purely textual CoTs lack spatial grounding and easily overfit to sparse annotated reasoning steps, while multimodal CoTs incur severe token inflation by generating imagined visual observations, making real-time navigation impractical. In this work, we propose FantasyVLN, a unified implicit reasoning framework that preserves the benefits of CoT reasoning without explicit token overhead. Specifically, imagined visual tokens are encoded into a compact latent space using a pretrained Visual AutoRegressor (VAR) during CoT reasoning training, and the model jointly learns from textual, visual, and multimodal CoT modes under a unified multi-CoT strategy. At inference, our model performs direct instruction-to-action mapping while still enjoying reasoning-aware representations. Extensive experiments on LH-VLN show that our approach achieves reasoning-aware yet real-time navigation, improving success rates and efficiency while reducing inference latency by an order of magnitude compared to explicit CoT methods.

Method: Generalizes diffusion-based inverse single-image problem solvers (DPS, DMAP, DPPS, diffusion-based PnP/ADMM) for multi-image super-resolution MRI. Shows DPS likelihood correction allows exactly-separable gradient decomposition across independently acquired measurements, enabling MISR without constructing joint operators or modifying diffusion models.

Result: Demonstrates substantial gains over single-image super-resolution across 4×/8×/16× anisotropic degradations. Achieves state-of-the-art super-resolution of anisotropic MRI volumes and enables reconstruction of near-isotropic anatomy from routine 2D multi-slice acquisitions.

Conclusion: The proposed multi-image super-resolution approach successfully extends diffusion-based inverse problem solvers to handle multiple MRI measurements, providing significant improvements in reconstruction quality and enabling practical clinical applications from routine acquisitions.

Abstract: Diffusion models are the current state-of-the-art for solving inverse problems in imaging. Their impressive generative capability allows them to approximate sampling from a prior distribution, which alongside a known likelihood function permits posterior sampling without retraining the model. While recent methods have made strides in advancing the accuracy of posterior sampling, the majority focuses on single-image inverse problems. However, for modalities such as magnetic resonance imaging (MRI), it is common to acquire multiple complementary measurements, each low-resolution along a different axis. In this work, we generalize common diffusion-based inverse single-image problem solvers for multi-image super-resolution (MISR) MRI. We show that the DPS likelihood correction allows an exactly-separable gradient decomposition across independently acquired measurements, enabling MISR without constructing a joint operator, modifying the diffusion model, or increasing network function evaluations. We derive MISR versions of DPS, DMAP, DPPS, and diffusion-based PnP/ADMM, and demonstrate substantial gains over SISR across $4\times/8\times/16\times$ anisotropic degradations. Our results achieve state-of-the-art super-resolution of anisotropic MRI volumes and, critically, enable reconstruction of near-isotropic anatomy from routine 2D multi-slice acquisitions, which are otherwise highly degraded in orthogonal views.

Method: HuDA integrates human detection confidence for appearance quality and temporal prompt alignment score for motion realism, using off-the-shelf models without additional training. Applied via Group Reward Policy Optimization (GRPO) post-training.

Result: Outperforms specialized models finetuned with manual annotations, achieves 73% win-rate against state-of-the-art models like Wan 2.1, and improves generation quality beyond humans (animals, human-object interactions).

Conclusion: A simple, training-free reward model can significantly enhance video generation quality for complex motions, demonstrating effectiveness across various dynamic subjects beyond just humans.

Abstract: Video generation models have recently achieved impressive visual fidelity and temporal coherence. Yet, they continue to struggle with complex, non-rigid motions, especially when synthesizing humans performing dynamic actions such as sports, dance, etc. Generated videos often exhibit missing or extra limbs, distorted poses, or physically implausible actions. In this work, we propose a remarkably simple reward model, HuDA, to quantify and improve the human motion in generated videos. HuDA integrates human detection confidence for appearance quality, and a temporal prompt alignment score to capture motion realism. We show this simple reward function that leverages off-the-shelf models without any additional training, outperforms specialized models finetuned with manually annotated data. Using HuDA for Group Reward Policy Optimization (GRPO) post-training of video models, we significantly enhance video generation, especially when generating complex human motions, outperforming state-of-the-art models like Wan 2.1, with win-rate of 73%. Finally, we demonstrate that HuDA improves generation quality beyond just humans, for instance, significantly improving generation of animal videos and human-object interactions.

Method: The authors propose a novel offline estimation method that corrects annotations to follow physically feasible trajectories and achieve spatial and temporal consistency with sensor data. They define metrics for this problem and evaluate on Argoverse 2, MAN TruckScenes, and proprietary datasets.

Result: The approach increases annotation quality by more than 17% across datasets. Original annotations were misplaced by up to 2.5 m, with highly dynamic objects most affected. The impact of these errors on benchmarking is larger than typical improvements between state-of-the-art methods.

Conclusion: Accurate annotations are essential for correct interpretation of autonomous vehicle system performance. The discovered annotation errors significantly impact benchmarking, and the proposed correction method provides more reliable ground truth for training and evaluation.

Abstract: Accurate ground truth annotations are critical to supervised learning and evaluating the performance of autonomous vehicle systems. These vehicles are typically equipped with active sensors, such as LiDAR, which scan the environment in predefined patterns. 3D box annotation based on data from such sensors is challenging in dynamic scenarios, where objects are observed at different timestamps, hence different positions. Without proper handling of this phenomenon, systematic errors are prone to being introduced in the box annotations. Our work is the first to discover such annotation errors in widely used, publicly available datasets. Through our novel offline estimation method, we correct the annotations so that they follow physically feasible trajectories and achieve spatial and temporal consistency with the sensor data. For the first time, we define metrics for this problem; and we evaluate our method on the Argoverse 2, MAN TruckScenes, and our proprietary datasets. Our approach increases the quality of box annotations by more than 17% in these datasets. Furthermore, we quantify the annotation errors in them and find that the original annotations are misplaced by up to 2.5 m, with highly dynamic objects being the most affected. Finally, we test the impact of the errors in benchmarking and find that the impact is larger than the improvements that state-of-the-art methods typically achieve with respect to the previous state-of-the-art methods; showing that accurate annotations are essential for correct interpretation of performance. Our code is available at https://github.com/alexandre-justo-miro/annotation-correction-3D-boxes.

Method: FBL achieves sample balance through knowledge filling and knowledge sampling using edge-side generation models, with Knowledge Alignment Strategy to bridge synthetic-real data gap, and Knowledge Drop Strategy for regularization. Scales to complex scenarios with different client methods.

Result: Numerous experiments show FBL outperforms state-of-the-art baselines.

Conclusion: FBL effectively prevents client drift from the beginning through client-side sample balancing, offering a novel approach to non-iid federated learning challenges.

Abstract: Federated learning is a paradigm of joint learning in which clients collaborate by sharing model parameters instead of data. However, in the non-iid setting, the global model experiences client drift, which can seriously affect the final performance of the model. Previous methods tend to correct the global model that has already deviated based on the loss function or gradient, overlooking the impact of the client samples. In this paper, we rethink the role of the client side and propose Federated Balanced Learning, i.e., FBL, to prevent this issue from the beginning through sample balance on the client side. Technically, FBL allows unbalanced data on the client side to achieve sample balance through knowledge filling and knowledge sampling using edge-side generation models, under the limitation of a fixed number of data samples on clients. Furthermore, we design a Knowledge Alignment Strategy to bridge the gap between synthetic and real data, and a Knowledge Drop Strategy to regularize our method. Meanwhile, we scale our method to real and complex scenarios, allowing different clients to adopt various methods, and extend our framework to further improve performance. Numerous experiments show that our method outperforms state-of-the-art baselines. The code is released upon acceptance.

Method: 1) Construct WeatherQA, a novel multimodal reasoning benchmark in meteorology. 2) Propose Logically Consistent Reinforcement Fine-Tuning (LoCo-RFT) which introduces a logical consistency reward to resolve Self-Contradictory Reasoning (Self-Contra). 3) Develop Weather-R1, the first reasoning VLM with logical faithfulness in meteorology.

Result: Weather-R1 improves performance on WeatherQA by 9.8 percentage points over baseline, outperforming both Supervised Fine-Tuning and standard Reinforcement Fine-Tuning, and even surpassing the original Qwen2.5-VL-32B model.

Conclusion: The proposed LoCo-RFT method effectively addresses logical consistency issues in VLMs for meteorology, and Weather-R1 demonstrates superior performance, establishing a new standard for faithful reasoning in high-stakes meteorological applications.

Abstract: While Vision Language Models (VLMs) show advancing reasoning capabilities, their application in meteorology is constrained by a domain gap and a reasoning faithfulness gap. Specifically, mainstream Reinforcement Fine-Tuning (RFT) can induce Self-Contradictory Reasoning (Self-Contra), where the model’s reasoning contradicts its final answer, which is unacceptable in such a high-stakes domain. To address these challenges, we construct WeatherQA, a novel multimodal reasoning benchmark in meteorology. We also propose Logically Consistent Reinforcement Fine-Tuning (LoCo-RFT), which resolves Self-Contra by introducing a logical consistency reward. Furthermore, we introduce Weather-R1, the first reasoning VLM with logical faithfulness in meteorology, to the best of our knowledge. Experiments demonstrate that Weather-R1 improves performance on WeatherQA by 9.8 percentage points over the baseline, outperforming Supervised Fine-Tuning and RFT, and even surpassing the original Qwen2.5-VL-32B. These results highlight the effectiveness of our LoCo-RFT and the superiority of Weather-R1. Our benchmark and code are available at https://github.com/Marcowky/Weather-R1.

Method: MM-OOD uses multimodal large language models (MLLMs) with multi-round conversations for enhanced outlier detection. For near OOD tasks: directly feed ID images and text prompts to MLLMs. For far OOD tasks: use a sketch-generate-elaborate framework: 1) sketch outlier exposure with text prompts, 2) generate visual OOD samples, 3) elaborate using multimodal prompts.

Result: The method achieves significant improvements on widely used multimodal datasets like Food-101 and demonstrates scalability on ImageNet-1K.

Conclusion: MM-OOD effectively leverages multimodal reasoning capabilities of MLLMs to address limitations of text-only approaches, providing enhanced OOD detection performance across both near and far OOD scenarios.

Abstract: Out-of-Distribution (OOD) detection is a critical task that has garnered significant attention. The emergence of CLIP has spurred extensive research into zero-shot OOD detection, often employing a training-free approach. Current methods leverage expert knowledge from large language models (LLMs) to identify potential outliers. However, these approaches tend to over-rely on knowledge in the text space, neglecting the inherent challenges involved in detecting out-of-distribution samples in the image space. In this paper, we propose a novel pipeline, MM-OOD, which leverages the multimodal reasoning capabilities of MLLMs and their ability to conduct multi-round conversations for enhanced outlier detection. Our method is designed to improve performance in both near OOD and far OOD tasks. Specifically, (1) for near OOD tasks, we directly feed ID images and corresponding text prompts into MLLMs to identify potential outliers; and (2) for far OOD tasks, we introduce the sketch-generate-elaborate framework: first, we sketch outlier exposure using text prompts, then generate corresponding visual OOD samples, and finally elaborate by using multimodal prompts. Experiments demonstrate that our method achieves significant improvements on widely used multimodal datasets such as Food-101, while also validating its scalability on ImageNet-1K.

Method: CVSI uses three key components: (1) Visual Information Extraction - extracts global image features and converts images to pseudo tokens combined with modification text and likely added objects; (2) Semantic Information Extraction - generates multiple captions for reference images using pre-trained captioning models, then uses LLMs to generate modified captions and likely added objects; (3) Complementary Information Retrieval - integrates information from both query and database images to retrieve target images.

Result: Extensive experiments on three public datasets (CIRR, CIRCO, and FashionIQ) demonstrate that CVSI significantly outperforms existing state-of-the-art methods.

Conclusion: CVSI effectively addresses the limitations of existing ZS-CIR methods by providing a comprehensive approach that integrates complementary visual and semantic information for fine-grained composed image retrieval, showing superior performance across multiple benchmark datasets.

Abstract: Zero-shot composed image retrieval (ZS-CIR) is a rapidly growing area with significant practical applications, allowing users to retrieve a target image by providing a reference image and a relative caption describing the desired modifications. Existing ZS-CIR methods often struggle to capture fine-grained changes and integrate visual and semantic information effectively. They primarily rely on either transforming the multimodal query into a single text using image-to-text models or employing large language models for target image description generation, approaches that often fail to capture complementary visual information and complete semantic context. To address these limitations, we propose a novel Fine-Grained Zero-Shot Composed Image Retrieval method with Complementary Visual-Semantic Integration (CVSI). Specifically, CVSI leverages three key components: (1) Visual Information Extraction, which not only extracts global image features but also uses a pre-trained mapping network to convert the image into a pseudo token, combining it with the modification text and the objects most likely to be added. (2) Semantic Information Extraction, which involves using a pre-trained captioning model to generate multiple captions for the reference image, followed by leveraging an LLM to generate the modified captions and the objects most likely to be added. (3) Complementary Information Retrieval, which integrates information extracted from both the query and database images to retrieve the target image, enabling the system to efficiently handle retrieval queries in a variety of situations. Extensive experiments on three public datasets (e.g., CIRR, CIRCO, and FashionIQ) demonstrate that CVSI significantly outperforms existing state-of-the-art methods. Our code is available at https://github.com/yyc6631/CVSI.

Method: VERIDAH uses multiple classification heads combined with a weighted vertebra sequence prediction algorithm to handle enumeration anomalies. It works on arbitrary field-of-view images and can detect unusual vertebral counts.

Result: Significantly outperforms existing models: 98.30% vs 94.24% correct labeling on T2w MRI (p<0.001) and 99.18% vs 77.26% on CT (p<0.001). Correctly identifies thoracic anomalies in 87.80% (MRI) and 96.30% (CT) of cases, and lumbar anomalies in 94.48% (MRI) and 97.22% (CT) of cases.

Conclusion: VERIDAH effectively addresses the gap in automated vertebra labeling for enumeration anomalies, providing superior performance on both MRI and CT imaging, with potential to improve clinical assessment of spinal anomalies.

Abstract: The human spine commonly consists of seven cervical, twelve thoracic, and five lumbar vertebrae. However, enumeration anomalies may result in individuals having eleven or thirteen thoracic vertebrae and four or six lumbar vertebrae. Although the identification of enumeration anomalies has potential clinical implications for chronic back pain and operation planning, the thoracolumbar junction is often poorly assessed and rarely described in clinical reports. Additionally, even though multiple deep-learning-based vertebra labeling algorithms exist, there is a lack of methods to automatically label enumeration anomalies. Our work closes that gap by introducing “Vertebra Identification with Anomaly Handling” (VERIDAH), a novel vertebra labeling algorithm based on multiple classification heads combined with a weighted vertebra sequence prediction algorithm. We show that our approach surpasses existing models on T2w TSE sagittal (98.30% vs. 94.24% of subjects with all vertebrae correctly labeled, p < 0.001) and CT imaging (99.18% vs. 77.26% of subjects with all vertebrae correctly labeled, p < 0.001) and works in arbitrary field-of-view images. VERIDAH correctly labeled the presence 2 Möller et al. of thoracic enumeration anomalies in 87.80% and 96.30% of T2w and CT images, respectively, and lumbar enumeration anomalies in 94.48% and 97.22% for T2w and CT, respectively. Our code and models are available at: https://github.com/Hendrik-code/spineps.

Method: Propose rotation-equivariant VAE with Vector Neuron Vision Transformer (VN-ViT) encoder and rotation-equivariant conditional neural field decoder. Introduce novel SO(2)-equivariant fully connected layer to reduce equivariance from SO(3) to SO(2).

Result: SO(2)-equivariant layer outperforms standard Vector Neurons in SO(2)-equivariant model. VAE enables smoother interpolation and more well-behaved latent space compared to previous methods.

Conclusion: The proposed rotation-equivariant VAE effectively models natural illumination on spheres while preserving equivariance properties and providing better latent space characteristics.

Abstract: Inverse rendering is an ill-posed problem, but priors like illumination priors, can simplify it. Existing work either disregards the spherical and rotation-equivariant nature of illumination environments or does not provide a well-behaved latent space. We propose a rotation-equivariant variational autoencoder that models natural illumination on the sphere without relying on 2D projections. To preserve the SO(2)-equivariance of environment maps, we use a novel Vector Neuron Vision Transformer (VN-ViT) as encoder and a rotation-equivariant conditional neural field as decoder. In the encoder, we reduce the equivariance from SO(3) to SO(2) using a novel SO(2)-equivariant fully connected layer, an extension of Vector Neurons. We show that our SO(2)-equivariant fully connected layer outperforms standard Vector Neurons when used in our SO(2)-equivariant model. Compared to previous methods, our variational autoencoder enables smoother interpolation in latent space and offers a more well-behaved latent space.

Method: Two curriculum learning approaches using synthetic aerial-view data and real ground-view data: 1) Two-stage curriculum with direct fine-tuning, and 2) Progressive curriculum that expands the dataset in multiple stages before fine-tuning. Evaluated on REMAG dataset using SlowFast (CNN) and MViTv2 (Transformer) architectures.

Result: Combining both out-of-domain datasets outperforms single-domain training. Both curriculum strategies match top-1 accuracy of simple dataset combination while offering efficiency gains: two-step fine-tuning reduces iterations by 37% (SlowFast) and 30% (MViTv2); progressive approach further reduces by 9% (SlowFast) and 30% (MViTv2). Performance remains within 3% accuracy range.

Conclusion: Curriculum-based training enables efficient cross-view action recognition by strategically combining synthetic aerial and real ground data, maintaining comparable performance while significantly reducing training iterations.

Abstract: Despite significant progress in human action recognition, generalizing to diverse viewpoints remains a challenge. Most existing datasets are captured from ground-level perspectives, and models trained on them often struggle to transfer to drastically different domains such as aerial views. This paper examines how curriculum-based training strategies can improve generalization to unseen real aerial-view data without using any real aerial data during training. We explore curriculum learning for cross-view action recognition using two out-of-domain sources: synthetic aerial-view data and real ground-view data. Our results on the evaluation on order of training (fine-tuning on synthetic aerial data vs. real ground data) shows that fine-tuning on real ground data but differ in how they transition from synthetic to real. The first uses a two-stage curriculum with direct fine-tuning, while the second applies a progressive curriculum that expands the dataset in multiple stages before fine-tuning. We evaluate both methods on the REMAG dataset using SlowFast (CNN-based) and MViTv2 (Transformer-based) architectures. Results show that combining the two out-of-domain datasets clearly outperforms training on a single domain, whether real ground-view or synthetic aerial-view. Both curriculum strategies match the top-1 accuracy of simple dataset combination while offering efficiency gains. With the two-step fine-tuning method, SlowFast achieves up to a 37% reduction in iterations and MViTv2 up to a 30% reduction compared to simple combination. The multi-step progressive approach further reduces iterations, by up to 9% for SlowFast and 30% for MViTv2, relative to the two-step method. These findings demonstrate that curriculum-based training can maintain comparable performance (top-1 accuracy within 3% range) while improving training efficiency in cross-view action recognition.

Method: Interp3D uses generative priors with progressive alignment: 1) semantically aligned interpolation in condition space, 2) SLAT-guided structure interpolation for structural consistency, and 3) fine-grained texture fusion for appearance details. The framework is training-free.

Result: Evaluated on Interp3DData dataset with graded difficulty levels, Interp3D shows significant advantages over previous methods in fidelity, transition smoothness, and plausibility, confirmed by both quantitative metrics and human studies.

Conclusion: Interp3D successfully addresses textured 3D morphing challenges by jointly preserving geometric consistency and texture alignment through progressive alignment, enabling smooth and plausible transitions between 3D assets for animation, editing, and content creation applications.

Abstract: Textured 3D morphing seeks to generate smooth and plausible transitions between two 3D assets, preserving both structural coherence and fine-grained appearance. This ability is crucial not only for advancing 3D generation research but also for practical applications in animation, editing, and digital content creation. Existing approaches either operate directly on geometry, limiting them to shape-only morphing while neglecting textures, or extend 2D interpolation strategies into 3D, which often causes semantic ambiguity, structural misalignment, and texture blurring. These challenges underscore the necessity to jointly preserve geometric consistency, texture alignment, and robustness throughout the transition process. To address this, we propose Interp3D, a novel training-free framework for textured 3D morphing. It harnesses generative priors and adopts a progressive alignment principle to ensure both geometric fidelity and texture coherence. Starting from semantically aligned interpolation in condition space, Interp3D enforces structural consistency via SLAT (Structured Latent)-guided structure interpolation, and finally transfers appearance details through fine-grained texture fusion. For comprehensive evaluations, we construct a dedicated dataset, Interp3DData, with graded difficulty levels and assess generation results from fidelity, transition smoothness, and plausibility. Both quantitative metrics and human studies demonstrate the significant advantages of our proposed approach over previous methods. Source code is available at https://github.com/xiaolul2/Interp3D.

Method: PMCE constructs a nonparametric knowledge bank with visual statistics and CLIP-encoded class name embeddings. It retrieves relevant base classes for novel categories, aggregates prior information, and fuses with support prototypes via MAP update. Simultaneously uses frozen BLIP captioner for instance-level descriptions and a lightweight enhancer with consistency regularization to optimize both support prototypes and query features.

Result: Experiments on four benchmarks show PMCE consistently improves over strong baselines, achieving up to 7.71% absolute gain over the strongest semantic competitor on MiniImageNet in the 1-shot setting.

Conclusion: PMCE effectively leverages multi-granularity semantics and caption-guided enhancement to address prototype bias in few-shot learning, demonstrating significant performance improvements across benchmarks.

Abstract: Few-shot learning aims to identify novel categories from only a handful of labeled samples, where prototypes estimated from scarce data are often biased and generalize poorly. Semantic-based methods alleviate this by introducing coarse class-level information, but they are mostly applied on the support side, leaving query representations unchanged. In this paper, we present PMCE, a Probabilistic few-shot framework that leverages Multi-granularity semantics with Caption-guided Enhancement. PMCE constructs a nonparametric knowledge bank that stores visual statistics for each category as well as CLIP-encoded class name embeddings of the base classes. At meta-test time, the most relevant base classes are retrieved based on the similarities of class name embeddings for each novel category. These statistics are then aggregated into category-specific prior information and fused with the support set prototypes via a simple MAP update. Simultaneously, a frozen BLIP captioner provides label-free instance-level image descriptions, and a lightweight enhancer trained on base classes optimizes both support prototypes and query features under an inductive protocol with a consistency regularization to stabilize noisy captions. Experiments on four benchmarks show that PMCE consistently improves over strong baselines, achieving up to 7.71% absolute gain over the strongest semantic competitor on MiniImageNet in the 1-shot setting. Our code is available at https://anonymous.4open.science/r/PMCE-275D

Method: Grayscale Image Compression with Differentiable Logic Circuits (GIC-DLC) - a hardware-aware codec that trains lookup tables to combine neural network flexibility with Boolean operation efficiency.

Result: Outperforms traditional codecs in compression efficiency on grayscale benchmark datasets while enabling substantial reductions in energy consumption and latency.

Conclusion: Learned compression can be hardware-friendly, offering a promising direction for low-power image compression on edge devices.

Abstract: Neural image codecs achieve higher compression ratios than traditional hand-crafted methods such as PNG or JPEG-XL, but often incur substantial computational overhead, limiting their deployment on energy-constrained devices such as smartphones, cameras, and drones. We propose Grayscale Image Compression with Differentiable Logic Circuits (GIC-DLC), a hardware-aware codec where we train lookup tables to combine the flexibility of neural networks with the efficiency of Boolean operations. Experiments on grayscale benchmark datasets show that GIC-DLC outperforms traditional codecs in compression efficiency while allowing substantial reductions in energy consumption and latency. These results demonstrate that learned compression can be hardware-friendly, offering a promising direction for low-power image compression on edge devices.

Method: 1) Dual-Domain Detail Perception Module to handle high-resolution images without ViT limitations and store high-frequency details in Gaussians. 2) Feature-guided diffusion network to preserve high-frequency details during restoration. 3) Unified training strategy for joint optimization of ViT-based geometric backbone and diffusion-based refinement module.

Result: Experiments demonstrate superior generation quality across multiple datasets, showing the method can maintain high-fidelity novel view synthesis from sparse images.

Conclusion: The proposed framework successfully overcomes key limitations in current 3DGS methods by combining geometric priors from ViT backbones with diffusion-based refinement, enabling high-resolution processing and 3D-consistent detail preservation for novel view synthesis.

Abstract: We present a novel framework for high-fidelity novel view synthesis (NVS) from sparse images, addressing key limitations in recent feed-forward 3D Gaussian Splatting (3DGS) methods built on Vision Transformer (ViT) backbones. While ViT-based pipelines offer strong geometric priors, they are often constrained by low-resolution inputs due to computational costs. Moreover, existing generative enhancement methods tend to be 3D-agnostic, resulting in inconsistent structures across views, especially in unseen regions. To overcome these challenges, we design a Dual-Domain Detail Perception Module, which enables handling high-resolution images without being limited by the ViT backbone, and endows Gaussians with additional features to store high-frequency details. We develop a feature-guided diffusion network, which can preserve high-frequency details during the restoration process. We introduce a unified training strategy that enables joint optimization of the ViT-based geometric backbone and the diffusion-based refinement module. Experiments demonstrate that our method can maintain superior generation quality across multiple datasets.

Method: Proposes ASBA network with: 1) A-line ROI state space model to extract sparsely distributed flow features along depth axis, 2) B-line phase attention to capture long-range flow signals along lateral axis based on phase difference, and 3) Flow-aware weighted loss function prioritizing accurate reconstruction of flow signals.

Result: Extensive experiments on real animal data demonstrate that the proposed approach clearly outperforms existing state-of-the-art reconstruction methods for sparse ODT imaging.

Conclusion: ASBA enables high-fidelity ODT image reconstruction from highly sparse sampling, addressing limitations of current dense sampling practices while maintaining accurate blood flow signal reconstruction.

Abstract: Optical Doppler Tomography (ODT) is an emerging blood flow analysis technique. A 2D ODT image (B-scan) is generated by sequentially acquiring 1D depth-resolved raw A-scans (A-line) along the lateral axis (B-line), followed by Doppler phase-subtraction analysis. To ensure high-fidelity B-scan images, current practices rely on dense sampling, which prolongs scanning time, increases storage demands, and limits the capture of rapid blood flow dynamics. Recent studies have explored sparse sampling of raw A-scans to alleviate these limitations, but their effectiveness is hindered by the conservative sampling rates and the uniform modeling of flow and background signals. In this study, we introduce a novel blood flow-aware network, named ASBA (A-line ROI State space model and B-line phase Attention), to reconstruct ODT images from highly sparsely sampled raw A-scans. Specifically, we propose an A-line ROI state space model to extract sparsely distributed flow features along the A-line, and a B-line phase attention to capture long-range flow signals along each B-line based on phase difference. Moreover, we introduce a flow-aware weighted loss function that encourages the network to prioritize the accurate reconstruction of flow signals. Extensive experiments on real animal data demonstrate that the proposed approach clearly outperforms existing state-of-the-art reconstruction methods.

Method: Proposes self-supervised training using only LDCT images with step-wise blind-spot denoising mechanism that enforces conditional independence progressively. Adds Gaussian noise to LDCT images as regularization to mitigate overfitting.

Result: Extensive experiments on Mayo LDCT dataset show method consistently outperforms existing self-supervised approaches and achieves performance comparable to or better than several representative supervised denoising methods.

Conclusion: The proposed self-supervised method effectively addresses LDCT denoising without requiring paired NDCT data, achieving competitive performance with supervised methods while being more practical for clinical applications.

Abstract: Self-supervised learning is increasingly investigated for low-dose computed tomography (LDCT) image denoising, as it alleviates the dependence on paired normal-dose CT (NDCT) data, which are often difficult to acquire in clinical practice. In this paper, we propose a novel self-supervised training strategy that relies exclusively on LDCT images. We introduce a step-wise blind-spot denoising mechanism that enforces conditional independence in a progressive manner, enabling more fine-grained denoising learning. In addition, we add Gaussian noise to LDCT images, which acts as a regularization and mitigates overfitting. Extensive experiments on the Mayo LDCT dataset demonstrate that the proposed method consistently outperforms existing self-supervised approaches and achieves performance comparable to, or better than, several representative supervised denoising methods.

Method: Proposes IIR-VLM that integrates pre-trained ILR expert models as auxiliary visual encoders to provide specialized features. This enables VLMs to learn new instances in-context in a one-shot manner and leverage this knowledge for instance-aware visual understanding.

Result: Validated on existing instance personalization benchmarks and demonstrated superior ILR performance on a new challenging benchmark assessing ILR capabilities across varying difficulty and diverse categories (person, face, pet, general objects).

Conclusion: IIR-VLM effectively enhances VLMs for instance-level recognition by leveraging specialized ILR features, enabling practical applications requiring fine-grained instance discrimination without costly per-instance training.

Abstract: Instance-level recognition (ILR) concerns distinguishing individual instances from one another, with person re-identification as a prominent example. Despite the impressive visual perception capabilities of modern VLMs, we find their performance on ILR unsatisfactory, often dramatically underperforming domain-specific ILR models. This limitation hinders many practical application of VLMs, e.g. where recognizing familiar people and objects is crucial for effective visual understanding. Existing solutions typically learn to recognize instances one at a time using instance-specific datasets, which not only incur substantial data collection and training costs but also struggle with fine-grained discrimination. In this work, we propose IIR-VLM, a VLM enhanced for In-context Instance-level Recognition. We integrate pre-trained ILR expert models as auxiliary visual encoders to provide specialized features for learning diverse instances, which enables VLMs to learn new instances in-context in a one-shot manner. Further, IIR-VLM leverages this knowledge for instance-aware visual understanding. We validate IIR-VLM’s efficacy on existing instance personalization benchmarks. Finally, we demonstrate its superior ILR performance on a challenging new benchmark, which assesses ILR capabilities across varying difficulty and diverse categories, with person, face, pet and general objects as the instances at task.

Method: Uses a three-camera rig to capture synchronized videos, with a rig-aware Structure-from-Motion pipeline that handles wide-angle distortion and low-parallax scenes. Combines precise calibration, constrained matching with DISK features and LightGlue matcher, and Gaussian splatting for real-time rendering.

Result: Produces interactive 3D models that allow rotation, zooming, and slicing to detect issues in seconds. Achieves state-of-the-art quality through ablation studies showing essential design choices.

Conclusion: The system improves workplace safety by eliminating physical crawling under vehicles and enhances buyer confidence through detailed 3D visualization, with a specialized SfM pipeline overcoming challenging imaging conditions.

Abstract: Inspecting the undercarriage of used vehicles is a labor-intensive task that requires inspectors to crouch or crawl underneath each vehicle to thoroughly examine it. Additionally, online buyers rarely see undercarriage photos. We present an end-to-end pipeline that utilizes a three-camera rig to capture videos of the undercarriage as the vehicle drives over it, and produces an interactive 3D model of the undercarriage. The 3D model enables inspectors and customers to rotate, zoom, and slice through the undercarriage, allowing them to detect rust, leaks, or impact damage in seconds, thereby improving both workplace safety and buyer confidence. Our primary contribution is a rig-aware Structure-from-Motion (SfM) pipeline specifically designed to overcome the challenges of wide-angle lens distortion and low-parallax scenes. Our method overcomes the challenges of wide-angle lens distortion and low-parallax scenes by integrating precise camera calibration, synchronized video streams, and strong geometric priors from the camera rig. We use a constrained matching strategy with learned components, the DISK feature extractor, and the attention-based LightGlue matcher to generate high-quality sparse point clouds that are often unattainable with standard SfM pipelines. These point clouds seed the Gaussian splatting process to generate photorealistic undercarriage models that render in real-time. Our experiments and ablation studies demonstrate that our design choices are essential to achieve state-of-the-art quality.

Method: STAT encodes images into discrete codes with per-token keep probabilities, regularized to be monotonically decreasing and aligned with image-level complexity measures, producing length-adaptive 1D visual tokens.

Result: On ImageNet-1k, STAT-equipped causal AR models achieve competitive or superior generation quality compared to other probabilistic models, with favorable scaling behavior not seen in prior vanilla AR attempts.

Conclusion: STAT enables effective length-adaptive visual tokenization that is naturally compatible with causal AR models, addressing scaling limitations and improving visual generation performance.

Abstract: We present Soft Tail-dropping Adaptive Tokenizer (STAT), a 1D discrete visual tokenizer that adaptively chooses the number of output tokens per image according to its structural complexity and level of detail. STAT encodes an image into a sequence of discrete codes together with per-token keep probabilities. Beyond standard autoencoder objectives, we regularize these keep probabilities to be monotonically decreasing along the sequence and explicitly align their distribution with an image-level complexity measure. As a result, STAT produces length-adaptive 1D visual tokens that are naturally compatible with causal 1D autoregressive (AR) visual generative models. On ImageNet-1k, equipping vanilla causal AR models with STAT yields competitive or superior visual generation quality compared to other probabilistic model families, while also exhibiting favorable scaling behavior that has been elusive in prior vanilla AR visual generation attempts.

Method: OmniTransfer uses three key designs: Task-aware Positional Bias for adaptive reference video usage, Reference-decoupled Causal Learning to separate reference and target branches, and Task-adaptive Multimodal Alignment with multimodal semantic guidance for different tasks.

Result: Extensive experiments show OmniTransfer outperforms existing methods in appearance transfer (ID and style) and temporal transfer (camera movement and video effects), while matching pose-guided methods in motion transfer without using pose.

Conclusion: OmniTransfer establishes a new paradigm for flexible, high-fidelity video generation by fully exploiting spatio-temporal information and unifying various video transfer tasks.

Abstract: Videos convey richer information than images or text, capturing both spatial and temporal dynamics. However, most existing video customization methods rely on reference images or task-specific temporal priors, failing to fully exploit the rich spatio-temporal information inherent in videos, thereby limiting flexibility and generalization in video generation. To address these limitations, we propose OmniTransfer, a unified framework for spatio-temporal video transfer. It leverages multi-view information across frames to enhance appearance consistency and exploits temporal cues to enable fine-grained temporal control. To unify various video transfer tasks, OmniTransfer incorporates three key designs: Task-aware Positional Bias that adaptively leverages reference video information to improve temporal alignment or appearance consistency; Reference-decoupled Causal Learning separating reference and target branches to enable precise reference transfer while improving efficiency; and Task-adaptive Multimodal Alignment using multimodal semantic guidance to dynamically distinguish and tackle different tasks. Extensive experiments show that OmniTransfer outperforms existing methods in appearance (ID and style) and temporal transfer (camera movement and video effects), while matching pose-guided methods in motion transfer without using pose, establishing a new paradigm for flexible, high-fidelity video generation.

Method: Trained on large-scale high-quality distillation data; uses resume strategy for bounding box prediction during pretraining; refines with RLVR using IoU-based rewards; employs checkpoint averaging and task-arithmetic merging for robustness.

Result: Achieves state-of-the-art results on OlmOCR-Bench while being 9x smaller and substantially faster than prior best models; successfully predicts normalized bounding boxes for embedded images.

Conclusion: LightOnOCR-2-1B demonstrates that efficient, end-to-end multilingual document understanding is achievable without traditional OCR pipelines, with released models, datasets, and benchmarks under open licenses.

Abstract: We present \textbf{LightOnOCR-2-1B}, a 1B-parameter end-to-end multilingual vision–language model that converts document images (e.g., PDFs) into clean, naturally ordered text without brittle OCR pipelines. Trained on a large-scale, high-quality distillation mix with strong coverage of scans, French documents, and scientific PDFs, LightOnOCR-2 achieves state-of-the-art results on OlmOCR-Bench while being 9$\times$ smaller and substantially faster than prior best-performing models. We further extend the output format to predict normalized bounding boxes for embedded images, introducing localization during pretraining via a resume strategy and refining it with RLVR using IoU-based rewards. Finally, we improve robustness with checkpoint averaging and task-arithmetic merging. We release model checkpoints under Apache 2.0, and publicly release the dataset and \textbf{LightOnOCR-bbox-bench} evaluation under their respective licenses.

Method: The framework decomposes 4D synthesis into static 3D shape generation and motion reconstruction. It uses a canonical reference mesh to learn a compact motion latent representation and predicts per-frame vertex trajectories for complete, temporally coherent geometry. A scalable frame-wise transformer enables robustness to varying sequence lengths.

Result: Evaluations on standard benchmarks and a new dataset with accurate ground-truth geometry show that Motion 3-to-4 delivers superior fidelity and spatial consistency compared to prior work.

Conclusion: Motion 3-to-4 successfully addresses 4D synthesis challenges through decomposition into shape and motion components, achieving state-of-the-art performance in generating high-quality dynamic 3D objects from monocular video inputs.

Abstract: We present Motion 3-to-4, a feed-forward framework for synthesising high-quality 4D dynamic objects from a single monocular video and an optional 3D reference mesh. While recent advances have significantly improved 2D, video, and 3D content generation, 4D synthesis remains difficult due to limited training data and the inherent ambiguity of recovering geometry and motion from a monocular viewpoint. Motion 3-to-4 addresses these challenges by decomposing 4D synthesis into static 3D shape generation and motion reconstruction. Using a canonical reference mesh, our model learns a compact motion latent representation and predicts per-frame vertex trajectories to recover complete, temporally coherent geometry. A scalable frame-wise transformer further enables robustness to varying sequence lengths. Evaluations on both standard benchmarks and a new dataset with accurate ground-truth geometry show that Motion 3-to-4 delivers superior fidelity and spatial consistency compared to prior work. Project page is available at https://motion3-to-4.github.io/.

Method: Train as a hyper-network for implicit neural representation (INR) that maps images to model weights for fast reconstruction. Integrate with knowledge distillation to improve generalization and performance.

Result: Learns an unprecedented compressed embedding space with outstanding performance for various visual tasks. Competes with state-of-the-art results for image representation learning while enabling generative capabilities with high-quality tiny embeddings.

Conclusion: The model successfully unifies recognition and generation capabilities in a single framework, achieving competitive performance on representation learning tasks while maintaining generative abilities through compressed embeddings.

Abstract: Models for image representation learning are typically designed for either recognition or generation. Various forms of contrastive learning help models learn to convert images to embeddings that are useful for classification, detection, and segmentation. On the other hand, models can be trained to reconstruct images with pixel-wise, perceptual, and adversarial losses in order to learn a latent space that is useful for image generation. We seek to unify these two directions with a first-of-its-kind model that learns representations which are simultaneously useful for recognition and generation. We train our model as a hyper-network for implicit neural representation, which learns to map images to model weights for fast, accurate reconstruction. We further integrate our INR hyper-network with knowledge distillation to improve its generalization and performance. Beyond the novel training design, the model also learns an unprecedented compressed embedding space with outstanding performance for various visual tasks. The complete model competes with state-of-the-art results for image representation learning, while also enabling generative capabilities with its high-quality tiny embeddings. The code is available at https://github.com/tiktok/huvr.

Method: Proposes two novel methods: 1) postprocessing occupancy scores to identify uncertain regions, and 2) direct prediction of an uncertainty indicator. These extend any local spatial occupancy prediction method. Also creates a ShapeNet-derived dataset with realistic depth images and ground-truth uncertain region annotations.

Result: Direct uncertainty prediction is most accurate for uncertain region segmentation. Both novel methods outperform two baseline probabilistic shape completion approaches in both shape completion and uncertain region prediction. Avoiding predicted uncertain regions improves grasp quality across all tested methods.

Conclusion: The proposed methods effectively predict uncertain regions in shape completion, with direct uncertainty prediction being particularly accurate. These approaches enhance robotic manipulation by identifying areas of geometric uncertainty that should be avoided during grasp planning.

Abstract: Shape completion, i.e., predicting the complete geometry of an object from a partial observation, is highly relevant for several downstream tasks, most notably robotic manipulation. When basing planning or prediction of real grasps on object shape reconstruction, an indication of severe geometric uncertainty is indispensable. In particular, there can be an irreducible uncertainty in extended regions about the presence of entire object parts when given ambiguous object views. To treat this important case, we propose two novel methods for predicting such uncertain regions as straightforward extensions of any method for predicting local spatial occupancy, one through postprocessing occupancy scores, the other through direct prediction of an uncertainty indicator. We compare these methods together with two known approaches to probabilistic shape completion. Moreover, we generate a dataset, derived from ShapeNet, of realistically rendered depth images of object views with ground-truth annotations for the uncertain regions. We train on this dataset and test each method in shape completion and prediction of uncertain regions for known and novel object instances and on synthetic and real data. While direct uncertainty prediction is by far the most accurate in the segmentation of uncertain regions, both novel methods outperform the two baselines in shape completion and uncertain region prediction, and avoiding the predicted uncertain regions increases the quality of grasps for all tested methods.

Method: CFA decomposes relation triplet features into intrinsic (characteristics) and extrinsic (context) components, then uses two feature augmentation modules to enrich feature diversity by replacing or mixing up intrinsic/extrinsic features from other samples.

Result: CFA achieves state-of-the-art performance on the trade-off between different SGG metrics, demonstrating effectiveness as a model-agnostic debiasing solution.

Conclusion: CFA successfully mitigates bias in SGG by increasing triplet feature diversity through compositional feature augmentation, offering a novel perspective beyond traditional re-balancing methods.

Abstract: Scene Graph Generation (SGG) aims to detect all the visual relation triplets $<$\texttt{sub}, \texttt{pred}, \texttt{obj}$>$ in a given image. With the emergence of various advanced techniques for better utilizing both the intrinsic and extrinsic information in each relation triplet, SGG has achieved great progress over the recent years. However, due to the ubiquitous long-tailed predicate distributions, today’s SGG models are still easily biased to the head predicates. Currently, the most prevalent debiasing solutions for SGG are re-balancing methods, \eg, changing the distributions of original training samples. In this paper, we argue that all existing re-balancing strategies fail to increase the diversity of the relation triplet features of each predicate, which is critical for robust SGG. To this end, we propose a novel Compositional Feature Augmentation (\textbf{CFA}) strategy, which is the first unbiased SGG work to mitigate the bias issue from the perspective of increasing the diversity of triplet features. Specifically, we first decompose each relation triplet feature into two components: intrinsic feature and extrinsic feature, which correspond to the intrinsic characteristics and extrinsic contexts of a relation triplet, respectively. Then, we design two different feature augmentation modules to enrich the feature diversity of original relation triplets by replacing or mixing up either their intrinsic or extrinsic features from other samples. Due to its model-agnostic nature, CFA can be seamlessly incorporated into various SGG frameworks. Extensive ablations have shown that CFA achieves a new state-of-the-art performance on the trade-off between different metrics.

Method: Shadow Loss is a proxy-free, parameter-free objective that measures similarity via scalar projections onto the anchor direction. It reduces the loss-specific buffer from O(S·D) to O(S) while preserving the triplet structure. The method analyzes gradients, provides Lipschitz continuity bounds, and ensures stable optimization by penalizing trivial collapse.

Result: Shadow Loss consistently outperforms recent objectives (Triplet, Soft-Margin Triplet, Angular Triplet, SoftTriple, Multi-Similarity) across fine-grained retrieval (CUB-200, CARS196), large-scale product retrieval (Stanford Online Products, In-Shop Clothes), and standard/medical benchmarks (CIFAR-10/100, Tiny-ImageNet, HAM-10K, ODIR-5K). It converges in ≈1.5-2× fewer epochs and improves representation separability with higher silhouette scores.

Conclusion: Shadow Loss enables memory-linear training and faster convergence by decoupling discriminative power from embedding dimensionality and reusing batch dot-products. This makes deep metric learning practical on both edge and large-scale systems while maintaining or improving performance over existing methods.

Abstract: Deep metric learning objectives (e.g., triplet loss) require storing and comparing high-dimensional embeddings, making the per-batch loss buffer scale as $O(S\cdot D)$, where $S$ is the number of samples in a batch and $D$ is the feature dimension, thus limiting training on memory-constrained hardware. We propose Shadow Loss, a proxy-free, parameter-free objective that measures similarity via scalar projections onto the anchor direction, reducing the loss-specific buffer from $O(S\cdot D)$ to $O(S)$ while preserving the triplet structure. We analyze gradients, provide a Lipschitz continuity bound, and show that Shadow Loss penalizes trivial collapse for stable optimization. Across fine-grained retrieval (CUB-200, CARS196), large-scale product retrieval (Stanford Online Products, In-Shop Clothes), and standard/medical benchmarks (CIFAR-10/100, Tiny-ImageNet, HAM-10K, ODIR-5K), Shadow Loss consistently outperforms recent objectives (Triplet, Soft-Margin Triplet, Angular Triplet, SoftTriple, Multi-Similarity). It also converges in $\approx 1.5\text{-}2\times$ fewer epochs under identical backbones and mining. Furthermore, it improves representation separability as measured by higher silhouette scores. The design is architecture-agnostic and vectorized for efficient implementation. By decoupling discriminative power from embedding dimensionality and reusing batch dot-products, Shadow Loss enables memory-linear training and faster convergence, making deep metric learning practical on both edge and large-scale systems.

Method: 1) Tree-of-thought-powered expert navigator for fine-grained semantic parsing and zero-shot matching to suitable diffusion models via extensible prior-knowledge tree; 2) Advantage database updated with human-in-the-loop feedback to align model-selection with human preferences; 3) Fully decoupled agent architecture that activates optimal generative paths without retraining experts.

Result: Extensive experiments show DiffusionAgent retains high generation quality while significantly broadening prompt coverage, establishing new performance and generality benchmarks for multi-domain image synthesis.

Conclusion: DiffusionAgent bridges the gap between prompt comprehension and image synthesis through an agentic framework, enabling optimal model selection for open-domain prompts without retraining, setting a new standard for multi-domain image generation.

Abstract: In the accelerating era of human-instructed visual content creation, diffusion models have demonstrated remarkable generative potential. Yet their deployment is constrained by a dual bottleneck: semantic ambiguity in diverse prompts and the narrow specialization of individual models. A single diffusion architecture struggles to maintain optimal performance across heterogeneous prompts, while conventional “parse-then-call” pipelines artificially separate semantic understanding from generative execution. To bridge this gap, we introduce DiffusionAgent, a unified, language-model-driven agent that casts the entire “prompt comprehension-expert routing-image synthesis” loop into a agentic framework. Our contributions are three-fold: (1) a tree-of-thought-powered expert navigator that performs fine-grained semantic parsing and zero-shot matching to the most suitable diffusion model via an extensible prior-knowledge tree; (2) an advantage database updated with human-in-the-loop feedback, continually aligning model-selection policy with human aesthetic and semantic preferences; and (3) a fully decoupled agent architecture that activates the optimal generative path for open-domain prompts without retraining or fine-tuning any expert. Extensive experiments show that DiffusionAgent retains high generation quality while significantly broadening prompt coverage, establishing a new performance and generality benchmark for multi-domain image synthesis. The code is available at https://github.com/DiffusionAgent/DiffusionAgent

Method: Proposes CausalDA that discovers latent causal factors between variables and model decisions using causality perspective. Integrates CLIP for world knowledge to discover latent causal factors without supervision. Uses information bottleneck with theoretical guarantees.

Result: Achieves state-of-the-art results across distinct SFDA settings and source-free out-of-distribution generalization.

Conclusion: CausalDA provides a practical unified solution for SFDA that handles multiple scenarios without target domain assumptions, improving reliability and robustness through causal discovery.

Abstract: In the pursuit of transferring a source model to a target domain without access to the source training data, Source-Free Domain Adaptation (SFDA) has been extensively explored across various scenarios, including Closed-set, Open-set, Partial-set, and Generalized settings. Existing methods, focusing on specific scenarios, not only address a limited subset of challenges but also necessitate prior knowledge of the target domain, significantly limiting their practical utility and deployability. In light of these considerations, we introduce a more practical yet challenging problem, termed unified SFDA, which comprehensively incorporates all specific scenarios in a unified manner. In this paper, we propose a novel approach latent Causal factors discovery for unified SFDA (CausalDA). In contrast to previous alternatives that emphasize learning the statistical description of reality, we formulate CausalDA from a causality perspective. The objective is to uncover potential causality between latent variables and model decisions, enhancing the reliability and robustness of the learned model against domain shifts. To integrate extensive world knowledge, we leverage a pre-trained vision-language model such as CLIP. This aids in the formation and discovery of latent causal factors in the absence of supervision in the variation of distribution and semantics, coupled with a newly designed information bottleneck with theoretical guarantees. Extensive experiments demonstrate that CausalDA can achieve new state-of-the-art results in distinct SFDA settings, as well as source-free out-of-distribution generalization. Our code and data are available at https://github.com/tntek/CausalDA.

Method: Cost-effective synthetic data generation using high-fidelity 3D hand models with varied skin tones, genders, dynamic environments, realistic lighting, and diverse affective gesture animations to create balanced, expressive datasets.

Result: Models trained exclusively on Hi5 achieve comparable performance to human-annotated datasets, with superior robustness to occlusions and consistent accuracy across diverse skin tones, crucial for reliable affective gesture recognition.

Conclusion: Synthetic data effectively addresses limitations of existing datasets, enabling more inclusive, expressive, and reliable gesture recognition systems while achieving competitive pose estimation performance, with all resources publicly released.

Abstract: Hand pose estimation plays a vital role in capturing subtle nonverbal cues essential for understanding human affect. However, collecting diverse, expressive real-world data remains challenging due to labor-intensive manual annotation that often underrepresents demographic diversity and natural expressions. To address this issue, we introduce a cost-effective approach to generating synthetic data using high-fidelity 3D hand models and a wide range of affective hand poses. Our method includes varied skin tones, genders, dynamic environments, realistic lighting conditions, and diverse naturally occurring gesture animations. The resulting dataset, Hi5, contains 583,000 pose-annotated images, carefully balanced to reflect natural diversity and emotional expressiveness. Models trained exclusively on Hi5 achieve performance comparable to human-annotated datasets, exhibiting superior robustness to occlusions and consistent accuracy across diverse skin tones – which is crucial for reliably recognizing expressive gestures in affective computing applications. Our results demonstrate that synthetic data effectively addresses critical limitations of existing datasets, enabling more inclusive, expressive, and reliable gesture recognition systems while achieving competitive performance in pose estimation benchmarks. The Hi5 dataset, data synthesis pipeline, source code, and game engine project are publicly released to support further research in synthetic hand-gesture applications.

Method: 1) Noise-Filtered Tracklet Partition (NFTP) module to reduce feature bias from noisy tracking and partition tracklets into sub-tracklets. 2) Progressive clustering and merging of sub-tracklets using self-supervised signals from tracklet partition. 3) Class Smoothing Classification (CSC) loss for efficient model learning.

Result: Achieves state-of-the-art results on MARS and DukeMTMC-VideoReID datasets, comparable to advanced supervised methods.

Conclusion: SSR-C demonstrates effective unsupervised video person re-identification without any annotations, using self-supervised refined clustering with noise filtering and progressive pseudo-label generation.

Abstract: With rich temporal-spatial information, video-based person re-identification methods have shown broad prospects. Although tracklets can be easily obtained with ready-made tracking models, annotating identities is still expensive and impractical. Therefore, some video-based methods propose using only a few identity annotations or camera labels to facilitate feature learning. They also simply average the frame features of each tracklet, overlooking unexpected variations and inherent identity consistency within tracklets. In this paper, we propose the Self-Supervised Refined Clustering (SSR-C) framework without relying on any annotation or auxiliary information to promote unsupervised video person re-identification. Specifically, we first propose the Noise-Filtered Tracklet Partition (NFTP) module to reduce the feature bias of tracklets caused by noisy tracking results, and sequentially partition the noise-filtered tracklets into “sub-tracklets”. Then, we cluster and further merge sub-tracklets using the self-supervised signal from the tracklet partition, which is enhanced through a progressive strategy to generate reliable pseudo labels, facilitating intra-class cross-tracklet aggregation. Moreover, we propose the Class Smoothing Classification (CSC) loss to efficiently promote model learning. Extensive experiments on the MARS and DukeMTMC-VideoReID datasets demonstrate that our proposed SSR-C for unsupervised video person re-identification achieves state-of-the-art results and is comparable to advanced supervised methods. The code is available at https://github.com/Darylmeng/SSRC-Reid.

Method: 1) Analyze PDM fine-tuning through shortcut learning lens, showing adversarial perturbations cause latent-space misalignment in CLIP embeddings. 2) Propose systematic red-teaming framework: data purification using off-the-shelf image restoration to realign images, and contrastive decoupling learning with noise tokens to separate personalized concepts from spurious noise patterns.

Result: The framework demonstrates advantages over existing purification methods and shows robustness against adaptive perturbations. It provides a thorough evaluation framework for developing stronger protection against adversarial attacks on PDMs.

Conclusion: The study uncovers shortcut learning vulnerabilities in PDMs and provides an effective solution through data purification and contrastive decoupling learning. The proposed framework offers systematic protection against adversarial perturbations while maintaining image quality better than existing methods.

Abstract: Personalized diffusion models (PDMs) have become prominent for adapting pre-trained text-to-image models to generate images of specific subjects using minimal training data. However, PDMs are susceptible to minor adversarial perturbations, leading to significant degradation when fine-tuned on corrupted datasets. These vulnerabilities are exploited to create protective perturbations that prevent unauthorized image generation. Existing purification methods attempt to red-team the protective perturbation to break the protection but often over-purify images, resulting in information loss. In this work, we conduct an in-depth analysis of the fine-tuning process of PDMs through the lens of shortcut learning. We hypothesize and empirically demonstrate that adversarial perturbations induce a latent-space misalignment between images and their text prompts in the CLIP embedding space. This misalignment causes the model to erroneously associate noisy patterns with unique identifiers during fine-tuning, resulting in poor generalization. Based on these insights, we propose a systematic red-teaming framework that includes data purification and contrastive decoupling learning. We first employ off-the-shelf image restoration techniques to realign images with their original semantic content in latent space. Then, we introduce contrastive decoupling learning with noise tokens to decouple the learning of personalized concepts from spurious noise patterns. Our study not only uncovers shortcut learning vulnerabilities in PDMs but also provides a thorough evaluation framework for developing stronger protection. Our extensive evaluation demonstrates its advantages over existing purification methods and its robustness against adaptive perturbations.

Method: 1) Pre-train a neural network differentiable renderer to learn volume rendering priors from data; 2) Generalize these priors to map UDFs to depth images for multi-view RGB rendering; 3) Use auxiliary point sampling prior as ray-surface intersection indicator with novel sampling schemes; 4) Leverage pretrained prior as general surface refiner for Gaussian reconstruction methods.

Result: The learned volume rendering prior is unbiased, robust, scalable, 3D aware, and easy to learn. It also enhances other neural implicit representations like signed distance functions and occupancy.

Conclusion: Data-driven learned volume rendering priors overcome limitations of handcrafted differentiable renderers for UDFs, providing a more accurate and generalizable approach for 3D reconstruction from multi-view images.

Abstract: Unsigned distance functions (UDFs) have been a vital representation for open surfaces. With different differentiable renderers, current methods are able to train neural networks to infer a UDF by minimizing the rendering errors with the UDF to the multi-view ground truth. However, these differentiable renderers are mainly handcrafted, which makes them either biased on ray-surface intersections, or sensitive to unsigned distance outliers, or not scalable to large scenes. To resolve these issues, we present a novel differentiable renderer to infer UDFs more accurately. Instead of using handcrafted equations, our differentiable renderer is a neural network which is pre-trained in a data-driven manner. It learns how to render unsigned distances into depth images, leading to a prior knowledge, dubbed volume rendering priors. To infer a UDF for an unseen scene from multiple RGB images, we generalize the learned volume rendering priors to map inferred unsigned distances in alpha blending for RGB image rendering. To reduce the bias of sampling in UDF inference, we utilize an auxiliary point sampling prior as an indicator of ray-surface intersection, and propose novel schemes towards more accurate and uniform sampling near the zero-level sets. We also propose a new strategy that leverages our pretrained volume rendering prior to serve as a general surface refiner, which can be integrated with various Gaussian reconstruction methods to optimize the Gaussian distributions and refine geometric details. Our results show that the learned volume rendering prior is unbiased, robust, scalable, 3D aware, and more importantly, easy to learn. Further experiments show that the volume rendering prior is also a general strategy to enhance other neural implicit representations such as signed distance function and occupancy.

Method: Uses a bilateral encoder to decouple location and semantic feature extraction, preventing location branch from providing cues to semantic branch. Features are compressed for compactness. A cascaded feature fusion decoder progressively refines location features using semantic features, with refined location features as the sole basis for edge map generation to suppress noise and location errors.

Result: Achieves ODS of 0.838 on BSDS500, achieving state-of-the-art performance across multiple benchmarks, outperforming methods with higher computational costs and complex training strategies.

Conclusion: High-quality features are what really matters in edge detection, and encoder-decoder based detectors can achieve excellent performance without complex training strategies or huge computational costs.

Abstract: The performance of deep learning based edge detector has far exceeded that of humans, but the huge computational cost and complex training strategy hinder its further development and application. In this paper, we eliminate these complexities with a vanilla encoder-decoder based detector. Firstly, we design a bilateral encoder to decouple the extraction process of location features and semantic features. Since the location branch no longer provides cues for the semantic branch, the richness of features can be further compressed, which is the key to make our model more compact. We propose a cascaded feature fusion decoder, where the location features are progressively refined by semantic features. The refined location features are the only basis for generating the edge map. The coarse original location features and semantic features are avoided from direct contact with the final result. So the noise in the location features and the location error in the semantic features can be suppressed in the generated edge map. The proposed New Baseline for Edge Detection (NBED) achieves superior performance consistently across multiple edge detection benchmarks, even compared with those methods with huge computational cost and complex training strategy. The ODS of NBED on BSDS500 is 0.838, achieving state-of-the-art performance. Our study shows that what really matters in the current edge detection is high-quality features, and we can make the encoder-decoder based detector great again even without complex training strategies and huge computational cost. The code is available at https://github.com/Li-yachuan/NBED.

Method: Proposes RemoteDet-Mamba network with patch-level four-direction selective scanning fusion strategy that simultaneously learns unimodal local features and fuses cross-modal patch-level global semantic information, plus a lightweight fusion mechanism for decoupling densely packed targets.

Result: Achieves superior detection performance on DroneVehicle dataset compared to current mainstream methods while maintaining low parameter count and computational overhead.

Conclusion: RemoteDet-Mamba shows promising potential for practical UAV remote sensing applications by effectively addressing small object detection challenges with efficient computational design.

Abstract: Unmanned Aerial Vehicle (UAV) remote sensing, with its advantages of rapid information acquisition and low cost, has been widely applied in scenarios such as emergency response. However, due to the long imaging distance and complex imaging mechanisms, targets in remote sensing images often face challenges such as small object size, dense distribution, and low inter-class discriminability. To address these issues, this paper proposes a multi-modal remote sensing object detection network called RemoteDet-Mamba, which is based on a patch-level four-direction selective scanning fusion strategy. This method simultaneously learns unimodal local features and fuses cross-modal patch-level global semantic information, thereby enhancing the distinguishability of small objects and improving inter-class discrimination. Furthermore, the designed lightweight fusion mechanism effectively decouples densely packed targets while reducing computational complexity. Experimental results on the DroneVehicle dataset demonstrate that RemoteDet-Mamba achieves superior detection performance compared to current mainstream methods, while maintaining low parameter count and computational overhead, showing promising potential for practical applications.

Method: Efficient post-training strategy integrating LLMs into pretrained CLIP with caption-to-caption contrastive fine-tuning framework to address LLMs’ autoregressive nature and enhance discriminative quality of LLM outputs.

Result: Outperforms LoRA-based methods with nearly 4x faster training and superior performance; shows substantial improvements over SOTA models (CLIP, EVA02, SigLip2) across zero-shot multimodal retrieval, cross-lingual retrieval, and multimodal language model pretraining tasks.

Conclusion: LLM integration via efficient post-training with contrastive fine-tuning effectively enhances CLIP’s capabilities, demonstrating the value of leveraging LLMs’ text understanding for multimodal representation learning.

Abstract: CLIP is a foundational multimodal model that aligns image and text features into a shared representation space via contrastive learning on large-scale image-text pairs. Its effectiveness primarily stems from the use of natural language as rich supervision. Motivated by the remarkable advancements in large language models (LLMs), this work explores how LLMs’ superior text understanding and extensive open-world knowledge can enhance CLIP’s capability, especially for processing longer and more complex image captions. We propose an efficient post-training strategy that integrates LLMs into pretrained CLIP. To address the challenge posed by the autoregressive nature of LLMs, we introduce a caption-to-caption contrastive fine-tuning framework, significantly enhancing the discriminative quality of LLM outputs. Extensive experiments demonstrate that our approach outperforms LoRA-based methods, achieving nearly fourfold faster training with superior performance. Furthermore, we validate substantial improvements over state-of-the-art models such as CLIP, EVA02, and SigLip2 across various zero-shot multimodal retrieval tasks, cross-lingual retrieval tasks, and multimodal language model pretraining.

Method: 1) Automated data generation pipeline creates GalaxyEdit dataset for add/remove operations. 2) Fine-tune SD v1.5 on GalaxyEdit. 3) Enhance ControlNet-xs architecture with non-linear interaction layers based on Volterra filters for better communication between control network and U-Net.

Result: Fine-tuned SD v1.5 outperforms SOTA methods by 11.2% (add) and 26.1% (remove) in FID scores. Enhanced ControlNet-xs with Volterra filters outperforms original ControlNet-xs in both add/remove tasks and canny-guided generation.

Conclusion: Automated data generation enables large-scale image editing datasets, improving model performance on complex editing tasks. Enhanced lightweight adapters with Volterra filters improve communication for on-device editing scenarios.

Abstract: Training of large-scale text-to-image and image-to-image models requires a huge amount of annotated data. While text-to-image datasets are abundant, data available for instruction-based image-to-image tasks like object addition and removal is limited. This is because of the several challenges associated with the data generation process, such as, significant human effort, limited automation, suboptimal end-to-end models, data diversity constraints and high expenses. We propose an automated data generation pipeline aimed at alleviating such limitations, and introduce GalaxyEdit - a large-scale image editing dataset for add and remove operations. We fine-tune the SD v1.5 model on our dataset and find that our model can successfully handle a broader range of objects and complex editing instructions, outperforming state-of-the-art methods in FID scores by 11.2% and 26.1% for add and remove tasks respectively. Furthermore, in light of on-device usage scenarios, we expand our research to include task-specific lightweight adapters leveraging the ControlNet-xs architecture. While ControlNet-xs excels in canny and depth guided generation, we propose to improve the communication between the control network and U-Net for more intricate add and remove tasks. We achieve this by enhancing ControlNet-xs with non-linear interaction layers based on Volterra filters. Our approach outperforms ControlNet-xs in both add/remove and canny-guided image generation tasks, highlighting the effectiveness of the proposed enhancement.

Method: Uses 3D Gaussian Splatting as scene representation, transforms continuous dynamic estimation into exposure time estimation, introduces exposure regularization, multi-frame/multi-resolution consistency regularization, and blur-aware variable canonical Gaussians for large motion.

Result: Outperforms state-of-the-art 4D reconstruction methods on synthetic and real-world data across multiple tasks: novel-view synthesis, deblurring, frame interpolation, and video stabilization.

Conclusion: Deblur4DGS effectively handles motion blur in videos for 4D reconstruction and enables multiple video enhancement applications, with code publicly available.

Abstract: Recent 4D reconstruction methods have yielded impressive results but rely on sharp videos as supervision. However, motion blur often occurs in videos due to camera shake and object movement, while existing methods render blurry results when using such videos for reconstructing 4D models. Although a few approaches attempted to address the problem, they struggled to produce high-quality results, due to the inaccuracy in estimating continuous dynamic representations within the exposure time. Encouraged by recent works in 3D motion trajectory modeling using 3D Gaussian Splatting (3DGS), we take 3DGS as the scene representation manner, and propose Deblur4DGS to reconstruct a high-quality 4D model from blurry monocular video. Specifically, we transform continuous dynamic representations estimation within an exposure time into the exposure time estimation. Moreover, we introduce the exposure regularization term, multi-frame, and multi-resolution consistency regularization term to avoid trivial solutions. Furthermore, to better represent objects with large motion, we suggest blur-aware variable canonical Gaussians. Beyond novel-view synthesis, Deblur4DGS can be applied to improve blurry video from multiple perspectives, including deblurring, frame interpolation, and video stabilization. Extensive experiments in both synthetic and real-world data on the above four tasks show that Deblur4DGS outperforms state-of-the-art 4D reconstruction methods. The codes are available at https://github.com/ZcsrenlongZ/Deblur4DGS.

Method: Proposes a framework using dataset “fingerprints” - structured representations of feature distributions - to quantify task similarity. This enables secure knowledge transfer of neural architectures, pretraining, augmentation policies, and multi-task learning across tasks.

Result: Tested across 71 distinct tasks and 12 medical imaging modalities. The method outperforms traditional approaches for identifying relevant knowledge and facilitates collaborative model training without sharing raw data.

Conclusion: The framework fosters democratization of AI in medical imaging and could become a valuable tool for promoting faster scientific advancement by enabling secure knowledge transfer while respecting privacy constraints.

Abstract: The field of medical imaging AI is currently undergoing rapid transformations, with methodical research increasingly translated into clinical practice. Despite these successes, research suffers from knowledge silos, hindering collaboration and progress: Existing knowledge is scattered across publications and many details remain unpublished, while privacy regulations restrict data sharing. In the spirit of democratizing of AI, we propose a framework for secure knowledge transfer in the field of medical image analysis. The key to our approach is dataset “fingerprints”, structured representations of feature distributions, that enable quantification of task similarity. We tested our approach across 71 distinct tasks and 12 medical imaging modalities by transferring neural architectures, pretraining, augmentation policies, and multi-task learning. According to comprehensive analyses, our method outperforms traditional methods for identifying relevant knowledge and facilitates collaborative model training. Our framework fosters the democratization of AI in medical imaging and could become a valuable tool for promoting faster scientific advancement.

Method: SuperGSeg first uses neural 3D Gaussians to learn geometry, instance, and hierarchical segmentation features from multi-view images with 2D masks. These features create sparse super Gaussians (supergs) that facilitate lifting and distilling 2D language features into 3D space, enabling hierarchical scene understanding with moderate memory costs.

Result: Extensive experiments show SuperGSeg achieves remarkable performance on both open-vocabulary object selection and semantic segmentation tasks while maintaining moderate GPU memory usage.

Conclusion: SuperGSeg provides an effective solution for memory-efficient hierarchical scene understanding in 3D Gaussian Splatting by decoupling segmentation from language feature distillation and using sparse super Gaussians for feature lifting.

Abstract: 3D Gaussian Splatting has recently gained traction for its efficient training and real-time rendering. While its vanilla representation is mainly designed for view synthesis, recent works extended it to scene understanding with language features. However, storing additional high-dimensional features per Gaussian for semantic information is memory-intensive, which limits their ability to segment and interpret challenging scenes. To this end, we introduce SuperGSeg, a novel approach that fosters cohesive, context-aware hierarchical scene representation by disentangling segmentation and language field distillation. SuperGSeg first employs neural 3D Gaussians to learn geometry, instance and hierarchical segmentation features from multi-view images with the aid of off-the-shelf 2D masks. These features are then leveraged to create a sparse set of \acrlong{superg}s. \acrlong{superg}s facilitate the lifting and distillation of 2D language features into 3D space. They enable hierarchical scene understanding with high-dimensional language feature rendering at moderate GPU memory costs. Extensive experiments demonstrate that SuperGSeg achieves remarkable performance on both open-vocabulary object selection and semantic segmentation tasks.

Method: Created FaceXBench with 5,000 multimodal multiple-choice questions from 25 public datasets plus new FaceXAPI dataset, covering 14 tasks across 6 categories. Evaluated 26 open-source and 2 proprietary MLLMs using zero-shot, in-context task description, and chain-of-thought prompting settings.

Result: Current MLLMs, including advanced models like GPT-4o and GeminiPro 1.5, show significant room for improvement on complex face understanding tasks. The benchmark reveals unique challenges in face understanding that current models struggle with.

Conclusion: FaceXBench provides a crucial resource for developing MLLMs capable of sophisticated face understanding, highlighting the need for further research and improvement in this domain.

Abstract: Multimodal Large Language Models (MLLMs) demonstrate impressive problem-solving abilities across a wide range of tasks and domains. However, their capacity for face understanding has not been systematically studied. To address this gap, we introduce FaceXBench, a comprehensive benchmark designed to evaluate MLLMs on complex face understanding tasks. FaceXBench includes 5,000 multimodal multiple-choice questions derived from 25 public datasets and a newly created dataset, FaceXAPI. These questions cover 14 tasks across 6 broad categories, assessing MLLMs’ face understanding abilities in bias and fairness, face authentication, recognition, analysis, localization and tool retrieval. Using FaceXBench, we conduct an extensive evaluation of 26 open-source MLLMs alongside 2 proprietary models, revealing the unique challenges in complex face understanding tasks. We analyze the models across three evaluation settings: zero-shot, in-context task description, and chain-of-thought prompting. Our detailed analysis reveals that current MLLMs, including advanced models like GPT-4o, and GeminiPro 1.5, show significant room for improvement. We believe FaceXBench will be a crucial resource for developing MLLMs equipped to perform sophisticated face understanding. Code: https://github.com/Kartik-3004/facexbench

Method: Two key innovations: (1) cross-Inhibition Dynamic Potential (cIDP) as a self-adaptive mechanism for capturing motion cues across wide velocity spectrum, (2) Collaborative Directional Gradient Calculation (CDGC) strategy that enhances orienting accuracy while reducing computational overhead to one-eighth of previous isolated strategies.

Result: On RIST dataset, vSTMD and its feedback-facilitated variant vSTMD-F achieve relative F₁ gains of 30% and 58% over SOTA STMD approaches. Models show competitive orientation estimation vs. deep learning methods, with vSTMD being 60× faster than contemporary data-driven methods.

Conclusion: vSTMD demonstrates superiority of natural architecture for ET-target motion detection, offering high performance, wide velocity coverage, and real-time capability suitable for dynamic scenarios and complex backgrounds.

Abstract: Visual motion detection for extremely tiny (ET-) targets is challenging, due to their category-independent nature and the scarcity of visual cues, which often incapacitate mainstream feature-based models. Natural architectures with rich interpretability offer a promising alternative, where STMD architectures derived from insect visual STMD (Small Target Motion Detector) pathways have demonstrated their effectiveness. However, previous STMD models are constrained to a narrow velocity range, hindering their efficacy in real-world scenarios where targets exhibit diverse and unstable dynamics. To address this limitation, we present vSTMD, a learning-free model for motion detection of ET-targets at various velocities. Our key innovations include: (1) a cross-Inhibition Dynamic Potential (cIDP) that serves as a self-adaptive mechanism efficiently capturing motion cues across a wide velocity spectrum, and (2) the first Collaborative Directional Gradient Calculation (CDGC) strategy, which enhances orienting accuracy and robustness while reducing computational overhead to one-eighth of previously isolated strategies. Evaluated on the real-world dataset RIST, the proposed vSTMD and its feedback-facilitated variant vSTMD-F achieve relative $F_{1}$ gains of $30%$ and $58%$ over state-of-the-art (SOTA) STMD approaches, respectively. Furthermore, both models demonstrate competitive orientation estimation performance compared to SOTA deep learning-driven methods. Experiments also reveal the superiority of the natural architecture for ET-object motion detection - vSTMD is $60\times$ faster than contemporary data-driven methods, making it highly suitable for real-time applications in dynamic scenarios and complex backgrounds. Code is available at https://github.com/MingshuoXu/vSTMD.

Method: Introduces Coarse Voxel Queries Generator integrating radar geometric priors with image semantic features; Dual-Branch Temporal Encoder for parallel temporal feature processing in BEV and voxel spaces; Cross-Modal BEV-Voxel Fusion module with attention mechanisms and auxiliary tasks.

Result: Achieves state-of-the-art performance on OmniHD-Scenes, View-of-Delft (VoD), and TJ4DRadSet datasets, establishing new benchmarks for multi-modal 3D perception.

Conclusion: Doracamom enables comprehensive environmental perception through joint 3D object detection and semantic occupancy prediction using camera-radar fusion, with code and models to be publicly available.

Abstract: 3D object detection and occupancy prediction are critical tasks in autonomous driving, attracting significant attention. Despite the potential of recent vision-based methods, they encounter challenges under adverse conditions. Thus, integrating cameras with next-generation 4D imaging radar to achieve unified multi-task perception is highly significant, though research in this domain remains limited. In this paper, we propose Doracamom, the first framework that fuses multi-view cameras and 4D radar for joint 3D object detection and semantic occupancy prediction, enabling comprehensive environmental perception. Specifically, we introduce a novel Coarse Voxel Queries Generator that integrates geometric priors from 4D radar with semantic features from images to initialize voxel queries, establishing a robust foundation for subsequent Transformer-based refinement. To leverage temporal information, we design a Dual-Branch Temporal Encoder that processes multi-modal temporal features in parallel across BEV and voxel spaces, enabling comprehensive spatio-temporal representation learning. Furthermore, we propose a Cross-Modal BEV-Voxel Fusion module that adaptively fuses complementary features through attention mechanisms while employing auxiliary tasks to enhance feature quality. Extensive experiments on the OmniHD-Scenes, View-of-Delft (VoD), and TJ4DRadSet datasets demonstrate that Doracamom achieves state-of-the-art performance in both tasks, establishing new benchmarks for multi-modal 3D perception. Code and models will be publicly available.

Method: Proposed object-centric latent action learning framework using self-supervised object-centric pretraining to disentangle agent movement from distracting background dynamics, enabling LAPO to focus on task-relevant interactions.

Result: Method reduces negative effects of distractors by 50% across eight visually complex tasks in Distracting Control Suite (DCS) and Distracting MetaWorld (DMW), measured by average return and success rate.

Conclusion: Object-centric pretraining enables more robust proxy-action labels, better imitation learning, and efficient adaptation with few action-labeled trajectories in visually complex embodied AI tasks.

Abstract: Leveraging vast amounts of unlabeled internet video data for embodied AI is currently bottlenecked by the lack of action labels and the presence of action-correlated visual distractors. Although recent latent action policy optimization (LAPO) has shown promise in inferring proxy action labels from visual observations, its performance degrades significantly when distractors are present. To address this limitation, we propose a novel object-centric latent action learning framework that centers on objects rather than pixels. We leverage self-supervised object-centric pretraining to disentangle the movement of the agent and distracting background dynamics. This allows LAPO to focus on task-relevant interactions, resulting in more robust proxy-action labels, enabling better imitation learning and efficient adaptation of the agent with just a few action-labeled trajectories. We evaluated our method in eight visually complex tasks across the Distracting Control Suite (DCS) and Distracting MetaWorld (DMW). Our results show that object-centric pretraining mitigates the negative effects of distractors by 50%, as measured by downstream task performance: average return (DCS) and success rate (DMW).

Method: The study explores how Vision Transformers and Self-Organizing Maps can empower each other through synergistic integration. The approach aims to bridge the research gap by combining ViTs’ attention mechanisms with SOMs’ topological preservation capabilities.

Result: The integration of ViTs and SOMs leads to significantly improved performance in both unsupervised and supervised tasks. The synergistic enhancement demonstrates that these architectures can effectively complement each other’s strengths.

Conclusion: Self-Organizing Maps can effectively address Vision Transformers’ limitations on small datasets by providing needed inductive biases. The synergistic integration of ViTs and SOMs represents a promising direction for improving performance in data-limited scenarios.

Abstract: Vision Transformers (ViTs) have demonstrated exceptional performance in various vision tasks. However, they tend to underperform on smaller datasets due to their inherent lack of inductive biases. Current approaches address this limitation implicitly-often by pairing ViTs with pretext tasks or by distilling knowledge from convolutional neural networks (CNNs) to strengthen the prior. In contrast, Self-Organizing Maps (SOMs), a widely adopted self-supervised framework, are inherently structured to preserve topology and spatial organization, making them a promising candidate to directly address the limitations of ViTs in limited or small training datasets. Despite this potential, equipping SOMs with modern deep learning architectures remains largely unexplored. In this study, we conduct a novel exploration on how Vision Transformers (ViTs) and Self-Organizing Maps (SOMs) can empower each other, aiming to bridge this critical research gap. Our findings demonstrate that these architectures can synergistically enhance each other, leading to significantly improved performance in both unsupervised and supervised tasks. Code is publicly available on GitHub.

Method: Introduces Triplet-Based Referring Paradigm (TRP) that decouples concepts, decoding types, and targets using symbolic delimiters; creates VT-Instruct dataset with 100M+ multimodal samples across 25 task types with various visual prompts and units.

Result: REF-VLM outperforms other MLLMs across various standard benchmarks in both qualitative and quantitative experiments.

Conclusion: The proposed framework successfully addresses visual decoding challenges in MLLMs through structured representation learning and comprehensive multi-task training, with code, dataset, and demo to be publicly available.

Abstract: Multimodal Large Language Models (MLLMs) demonstrate robust zero-shot capabilities across diverse vision-language tasks after training on mega-scale datasets. However, dense prediction tasks, such as semantic segmentation and keypoint detection, pose significant challenges for MLLMs when represented solely as text outputs. Simultaneously, current MLLMs utilizing latent embeddings for visual task decoding generally demonstrate limited adaptability to both multi-task learning and multi-granularity scenarios. In this work, we present \textbf{REF-VLM}, an end-to-end framework for unified training of various visual decoding tasks. To address complex visual decoding scenarios, we introduce the \textbf{Triplet-Based Referring Paradigm (TRP)}, which explicitly decouples three critical dimensions in visual decoding tasks through a triplet structure: concepts, decoding types, and targets. TRP employs symbolic delimiters to enforce structured representation learning, enhancing the parsability and interpretability of model outputs. Additionally, we construct \textbf{Visual-Task Instruction Following Dataset (VT-Instruct)}, a large-scale multi-task dataset containing over 100 million multimodal dialogue samples across 25 task types. Beyond text inputs and outputs, VT-Instruct incorporates various visual prompts such as point, box, scribble, and mask, and generates outputs composed of text and visual units like box, keypoint, depth and mask. The combination of different visual prompts and visual units generates a wide variety of task types, expanding the applicability of REF-VLM significantly. Both qualitative and quantitative experiments demonstrate that our REF-VLM outperforms other MLLMs across a variety of standard benchmarks. The code, dataset, and demo will be publicly available.

Method: The paper investigates using MLLMs as unified evaluators for AI-generated videos, leveraging their visual perception and language understanding capabilities. They introduce UVE-Bench benchmark with videos from state-of-the-art VGMs and human preference annotations across 15 evaluation aspects. They evaluate 18 MLLMs on this benchmark and analyze key design choices affecting MLLM-driven evaluator performance.

Result: Advanced MLLMs (Qwen2VL-72B and InternVL2.5-78B) demonstrate promising unified evaluation ability, significantly surpassing existing specialized evaluation methods, though they still lag behind human evaluators. The analysis provides insights into design choices impacting MLLM-driven evaluator performance.

Conclusion: MLLMs show strong potential as unified evaluators for AI-generated videos, offering a scalable solution for comprehensive assessment. While not yet matching human performance, they significantly outperform specialized methods, and the UVE-Bench benchmark enables systematic evaluation of automatic metrics for video assessment.

Abstract: With the rapid growth of video generative models (VGMs), it is essential to develop reliable and comprehensive automatic metrics for AI-generated videos (AIGVs). Existing methods either use off-the-shelf models optimized for other tasks or rely on human assessment data to train specialized evaluators. These approaches are constrained to specific evaluation aspects and are difficult to scale with the increasing demands for finer-grained and more comprehensive evaluations. To address this issue, this work investigates the feasibility of using multimodal large language models (MLLMs) as a unified evaluator for AIGVs, leveraging their strong visual perception and language understanding capabilities. To evaluate the performance of automatic metrics in unified AIGV evaluation, we introduce a benchmark called UVE-Bench. UVE-Bench collects videos generated by state-of-the-art VGMs and provides pairwise human preference annotations across 15 evaluation aspects. Using UVE-Bench, we extensively evaluate 18 MLLMs. Our empirical results suggest that while advanced MLLMs (e.g., Qwen2VL-72B and InternVL2.5-78B) still lag behind human evaluators, they demonstrate promising ability in unified AIGV evaluation, significantly surpassing existing specialized evaluation methods. Additionally, we conduct an in-depth analysis of key design choices that impact the performance of MLLM-driven evaluators, offering valuable insights for future research on AIGV evaluation.

Method: Three-stage framework: (1) Semantic parsing using LLMs to interpret text into parameterized avatars and garment templates, (2) Geometry-aware garment generation with topology-preserving deformation and novel geometric losses, (3) Consistent texture synthesis using multi-view diffusion process optimized for garment texturing.

Result: Tailor outperforms state-of-the-art methods in fidelity, usability, and diversity through comprehensive quantitative and qualitative evaluations.

Conclusion: The framework bridges the gap in accessible 3D avatar generation with simulation-ready garments, offering a complete pipeline for CG-ready models that can be directly integrated into conventional CG workflows.

Abstract: Creating detailed 3D human avatars with fitted garments traditionally requires specialized expertise and labor-intensive workflows. While recent advances in generative AI have enabled text-to-3D human and clothing synthesis, existing methods fall short in offering accessible, integrated pipelines for generating CG-ready 3D avatars with physically compatible outfits; here we use the term CG-ready for models following a technical aesthetic common in computer graphics (CG) and adopt standard CG polygonal meshes and strands representations (rather than neural representations like NeRF and 3DGS) that can be directly integrated into conventional CG pipelines and support downstream tasks such as physical simulation. To bridge this gap, we introduce Tailor, an integrated text-to-3D framework that generates high-fidelity, customizable 3D avatars dressed in simulation-ready garments. Tailor consists of three stages. (1) Seman tic Parsing: we employ a large language model to interpret textual descriptions and translate them into parameterized human avatars and semantically matched garment templates. (2) Geometry-Aware Garment Generation: we propose topology-preserving deformation with novel geometric losses to generate body-aligned garments under text control. (3) Consistent Texture Synthesis: we propose a novel multi-view diffusion process optimized for garment texturing, which enforces view consistency, preserves photorealistic details, and optionally supports symmetric texture generation common in garments. Through comprehensive quantitative and qualitative evaluations, we demonstrate that Tailor outperforms state-of-the-art methods in fidelity, usability, and diversity. Our code will be released for academic use. Project page: https://human-tailor.github.io

Method: YOLO-LLTS introduces three key modules: HRFM-SOD for retaining information about distant/tiny signs, MFIA for feature interaction across different receptive fields, and PGFE for enhancing brightness, edges, contrast, and detail information. The method also includes creation of the CNTSSS dataset covering diverse nighttime scenarios.

Result: YOLO-LLTS achieves state-of-the-art performance with improvements of 2.7% mAP50 and 1.6% mAP50:95 on TT100K-night, 1.3% mAP50 and 1.9% mAP50:95 on CNTSSS, and 7.5% mAP50 and 9.8% mAP50:95 on GTSDB-night. Deployment on edge devices confirms real-time applicability.

Conclusion: YOLO-LLTS effectively addresses low-light traffic sign detection challenges through specialized modules and a comprehensive dataset, achieving superior performance while maintaining real-time capability for practical deployment in autonomous driving systems.

Abstract: Traffic sign detection is essential for autonomous driving and Advanced Driver Assistance Systems (ADAS). However, existing methods struggle to address the challenges of poor image quality and insufficient information under low-light conditions, leading to a decline in detection accuracy and affecting driving safety. To address this issue, we propose YOLO-LLTS, an end-to-end real-time traffic sign detection algorithm specifically designed for low-light environments. YOLO-LLTS introduces three main contributions: the HRFM-SOD module retains more information about distant or tiny traffic signs compared to traditional methods; the MFIA module interacts features with different receptive fields to improve information utilization; the PGFE module enhances detection accuracy by improving brightness, edges, contrast, and supplementing detail information. Additionally, we construct a new dataset, the Chinese Nighttime Traffic Sign Sample Set (CNTSSS), covering diverse nighttime scenarios. Experiments show that YOLO-LLTS achieves state-of-the-art performance, outperforming previous best methods by 2.7% mAP50 and 1.6% mAP50:95 on TT100K-night, 1.3% mAP50 and 1.9% mAP50:95 on CNTSSS, 7.5% mAP50 and 9.8% mAP50:95 on GTSDB-night, and superior results on CCTSDB2021. Deployment on edge devices confirms its real-time applicability and effectiveness. The code and the dataset are available at https://github.com/linzy88/YOLO-LLTS.

Method: Uses spatio-temporal significance pruning (removing 64% of primitives) with entropy-constrained spherical harmonics compression, plus a deep context model with intra-/inter-prediction and hyperprior for efficient latent embedding compression.

Result: Achieves over 120x compression, increases rendering FPS up to 20% compared to baseline 4DGS, and outperforms state-of-the-art frame-wise 3DGS compression methods.

Conclusion: Light4GS provides an effective lightweight storage-efficient dynamic 3DGS representation without sacrificing rendering quality, demonstrating superior compression through both intra- and inter-prediction methods.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as an efficient and high-fidelity paradigm for novel view synthesis. To adapt 3DGS for dynamic content, deformable 3DGS incorporates temporally deformable primitives with learnable latent embeddings to capture complex motions. Despite its impressive performance, the high-dimensional embeddings and vast number of primitives lead to substantial storage requirements. In this paper, we introduce a \textbf{Light}weight \textbf{4}D\textbf{GS} framework, called Light4GS, that employs significance pruning with a deep context model to provide a lightweight storage-efficient dynamic 3DGS representation. The proposed Light4GS is based on 4DGS that is a typical representation of deformable 3DGS. Specifically, our framework is built upon two core components: (1) a spatio-temporal significance pruning strategy that eliminates over 64% of the deformable primitives, followed by an entropy-constrained spherical harmonics compression applied to the remainder; and (2) a deep context model that integrates intra- and inter-prediction with hyperprior into a coarse-to-fine context structure to enable efficient multiscale latent embedding compression. Our approach achieves over 120x compression and increases rendering FPS up to 20% compared to the baseline 4DGS, and also superior to frame-wise state-of-the-art 3DGS compression methods, revealing the effectiveness of our Light4GS in terms of both intra- and inter-prediction methods without sacrificing rendering quality.

Method: PraNet-V2 introduces a Dual-Supervised Reverse Attention (DSRA) module with three key components: explicit background supervision, independent background modeling, and semantically enriched attention fusion. The framework can also be integrated with existing state-of-the-art semantic segmentation models to iteratively enhance foreground segmentation.

Result: PraNet-V2 demonstrates strong performance on four polyp segmentation datasets. When integrated with three state-of-the-art semantic segmentation models, it achieves up to 1.36% improvement in mean Dice score.

Conclusion: PraNet-V2 successfully addresses the limitations of PraNet-V1 by enabling multi-class segmentation through the novel DSRA module, making it a more versatile and effective framework for medical image segmentation tasks.

Abstract: Accurate medical image segmentation is essential for effective diagnosis and treatment. Previously, PraNet-V1 was proposed to enhance polyp segmentation by introducing a reverse attention (RA) module that utilizes background information. However, PraNet-V1 struggles with multi-class segmentation tasks. To address this limitation, we propose PraNet-V2, which, compared to PraNet-V1, effectively performs a broader range of tasks including multi-class segmentation. At the core of PraNet-V2 is the Dual-Supervised Reverse Attention (DSRA) module, which incorporates explicit background supervision, independent background modeling, and semantically enriched attention fusion. Our PraNet-V2 framework demonstrates strong performance on four polyp segmentation datasets. Additionally, by integrating DSRA to iteratively enhance foreground segmentation results in three state-of-the-art semantic segmentation models, we achieve up to a 1.36% improvement in mean Dice score. Code is available at: https://github.com/ai4colonoscopy/PraNet-V2/tree/main/binary_seg/jittor.

Method: 1) Created a curated dataset of spatially explicit prompts from LAION-400M with precise text-layout alignment; 2) Developed ESPLoRA, a flexible fine-tuning framework based on Low-Rank Adaptation; 3) Proposed refined evaluation metrics based on geometric constraints; 4) Introduced TORE algorithm to exploit spatial biases for improved consistency.

Result: The method outperforms CoMPaSS (current baseline framework) on spatial consistency benchmarks while maintaining generation time and output quality.

Conclusion: ESPLoRA effectively enhances spatial consistency in text-to-image generation without computational overhead, and the proposed metrics reveal spatial biases that can be strategically exploited for further improvements.

Abstract: Diffusion models have revolutionized text-to-image (T2I) synthesis, producing high-quality, photorealistic images. However, they still struggle to properly render the spatial relationships described in text prompts. To address the lack of spatial information in T2I generations, existing methods typically use external network conditioning and predefined layouts, resulting in higher computational costs and reduced flexibility. Our approach builds upon a curated dataset of spatially explicit prompts, meticulously extracted and synthesized from LAION-400M to ensure precise alignment between textual descriptions and spatial layouts. Alongside this dataset, we present ESPLoRA, a flexible fine-tuning framework based on Low-Rank Adaptation, specifically designed to enhance spatial consistency in generative models without increasing generation time or compromising the quality of the outputs. In addition to ESPLoRA, we propose refined evaluation metrics grounded in geometric constraints, capturing 3D spatial relations such as “in front of” or “behind”. These metrics also expose spatial biases in T2I models which, even when not fully mitigated, can be strategically exploited by our TORE algorithm to further improve the spatial consistency of generated images. Our method outperforms CoMPaSS, the current baseline framework, on spatial consistency benchmarks.

Method: Uses nnUNet architecture for multi-class segmentation of six plant structures (main root, lateral roots, seed, hypocotyl, leaves, petiole). Features dual graphical interfaces: Standard Interface for detailed architectural analysis with gravitropic response parameters, and Screening Interface for high-throughput automated tracking. Integrates Functional Principal Component Analysis for temporal pattern comparison. Maintains modular low-cost hardware while enhancing software capabilities.

Result: Demonstrated significant accuracy improvements in segmentation. Successfully applied to multi-species analysis (Arabidopsis thaliana and Solanum lycopersicum). Validated through three use cases: circadian growth pattern characterization, gravitropic response analysis in transgenic plants, and high-throughput etiolation screening across multiple genotypes. The system enables easy retraining and incorporation of additional training data without requiring machine learning expertise.

Conclusion: ChronoRoot 2.0 dramatically improves accessibility to sophisticated temporal plant phenotyping through intuitive graphical interfaces and expanded analytical capabilities while maintaining the low-cost, modular hardware advantages of its predecessor, making it a valuable open-source platform for plant researchers.

Abstract: Plant developmental plasticity, particularly in root system architecture, is fundamental to understanding adaptability and agricultural sustainability. ChronoRoot 2.0 builds upon established low-cost hardware while significantly enhancing software capabilities and usability. The system employs nnUNet architecture for multi-class segmentation, demonstrating significant accuracy improvements while simultaneously tracking six distinct plant structures encompassing root, shoot, and seed components: main root, lateral roots, seed, hypocotyl, leaves, and petiole. This architecture enables easy retraining and incorporation of additional training data without requiring machine learning expertise. The platform introduces dual specialized graphical interfaces: a Standard Interface for detailed architectural analysis with novel gravitropic response parameters, and a Screening Interface enabling high-throughput analysis of multiple plants through automated tracking. Functional Principal Component Analysis integration enables discovery of novel phenotypic parameters through temporal pattern comparison. We demonstrate multi-species analysis, with Arabidopsis thaliana and Solanum lycopersicum, both morphologically distinct plant species. Three use cases in Arabidopsis thaliana and validation with tomato seedlings demonstrate enhanced capabilities: circadian growth pattern characterization, gravitropic response analysis in transgenic plants, and high-throughput etiolation screening across multiple genotypes.ChronoRoot 2.0 maintains the low-cost, modular hardware advantages of its predecessor while dramatically improving accessibility through intuitive graphical interfaces and expanded analytical capabilities. The open-source platform makes sophisticated temporal plant phenotyping more accessible to researchers without computational expertise.

Method: Proposes Fine-grained Spatial-Temporal Perception (FGSTP) algorithm: 1) Constructs correlation volume to capture motion information between consecutive frames, 2) Progressively refines object-level features using previous outputs, 3) Uses decoder to optimize boundary segmentation. Also creates GasVid dataset with manual labeling.

Result: Experimental results on GasVid dataset show FGSTP excels in segmenting non-rigid objects like gas leaks, generating the most accurate masks compared to other state-of-the-art models.

Conclusion: FGSTP provides an effective solution for gas leak segmentation by capturing motion clues and refining features, addressing the challenges of concealed appearances and random shapes. The GasVid dataset also fills a gap in labeled data for this important safety application.

Abstract: Gas leaks pose significant risks to human health and the environment. Despite long-standing concerns, there are limited methods that can efficiently and accurately detect and segment leaks due to their concealed appearance and random shapes. In this paper, we propose a Fine-grained Spatial-Temporal Perception (FGSTP) algorithm for gas leak segmentation. FGSTP captures critical motion clues across frames and integrates them with refined object features in an end-to-end network. Specifically, we first construct a correlation volume to capture motion information between consecutive frames. Then, the fine-grained perception progressively refines the object-level features using previous outputs. Finally, a decoder is employed to optimize boundary segmentation. Because there is no highly precise labeled dataset for gas leak segmentation, we manually label a gas leak video dataset, GasVid. Experimental results on GasVid demonstrate that our model excels in segmenting non-rigid objects such as gas leaks, generating the most accurate mask compared to other state-of-the-art (SOTA) models.

Method: Conducted first large-scale benchmarking study with over 200 combinations of model architectures, datasets, and training strategies across three datasets (FaultSeg3D, CRACKS, Thebe). Systematically assessed pretraining, fine-tuning, and joint training under varying domain shifts, complemented with novel fault characteristic descriptor analysis.

Result: Common fine-tuning practices cause catastrophic forgetting, especially with disjoint datasets; larger models like Segformer are more robust; domain adaptation outperforms fine-tuning for large shifts but underperforms for similar domains; models absorb structural biases from training data.

Conclusion: Established robust experimental baseline providing insights into tradeoffs in fault delineation workflows and highlighting directions for building more generalizable and interpretable seismic interpretation models.

Abstract: Machine learning has taken a critical role in seismic interpretation workflows, especially in fault delineation tasks. However, despite the recent proliferation of pretrained models and synthetic datasets, the field still lacks a systematic understanding of the generalizability limits of these models across seismic data representing diverse geologic, acquisition and processing settings. Distributional shifts between data sources, limitations in fine-tuning strategies and labeled data accessibility, and inconsistent evaluation protocols all remain major roadblocks to deploying reliable models in real-world exploration. In this paper, we present the first large-scale benchmarking study explicitly designed to provide guidelines for domain shift strategies in seismic interpretation. Our benchmark spans over 200 combinations of model architectures, datasets and training strategies, across three datasets (synthetic and real) including FaultSeg3D, CRACKS, and Thebe. We systematically assess pretraining, fine-tuning, and joint training under varying domain shifts. Our analysis shows that common fine-tuning practices can lead to catastrophic forgetting, especially when source and target datasets are disjoint, and that larger models such as Segformer are more robust than smaller architectures. We also find that domain adaptation methods outperform fine-tuning when shifts are large, yet underperform when domains are similar. Finally, we complement segmentation metrics with a novel analysis based on fault characteristic descriptors, revealing how models absorb structural biases from training datasets. Overall, we establish a robust experimental baseline that provides insights into tradeoffs in current fault delineation workflows and highlights directions for building more generalizable and interpretable models.

Method: SM3D introduces two key components: 1) Geometric Spectral Compensator (GSC) to counteract low-pass bias by explicitly injecting graph-guided high-frequency components through local Laplacian analysis, restoring structural sensitivity; 2) Semantic Coherence Refiner (SCR) to rectify semantic drift through frequency-aware channel recalibration, implemented via two pathways: exact Laplacian eigendecomposition (SCR-L) and linear-complexity Chebyshev polynomial approximation (SCR-C) for computational efficiency.

Result: Extensive experiments demonstrate state-of-the-art performance: 96.0% accuracy on ModelNet40 and 86.5% mIoU on ShapeNetPart, validating effectiveness in mitigating spectral low-pass bias and semantic dilution.

Conclusion: SM3D successfully addresses the spectral limitations of SSMs for point cloud understanding by jointly preserving geometric fidelity and semantic consistency through spectral-aware compensation mechanisms, achieving superior performance on benchmark datasets.

Abstract: Point clouds are a fundamental 3D data representation that underpins various computer vision tasks. Recently, Mamba has demonstrated strong potential for 3D point cloud understanding. However, existing approaches primarily focus on point serialization, overlooking a more fundamental limitation: State Space Models (SSMs) inherently exhibit a spectral low-pass bias arising from their recursive formulation. In serialized point clouds, this bias is particularly detrimental, as it suppresses high-frequency geometric structures and progressively dilutes semantic discriminability across deep layers. To address these limitations, we propose SM3D, a spectral-aware framework designed to jointly preserve geometric fidelity and semantic consistency. First, a Geometric Spectral Compensator (GSC) is introduced to counteract the low-pass bias by explicitly injecting graph-guided high-frequency components through local Laplacian analysis, thereby restoring structural sensitivity. Second, we design a Semantic Coherence Refiner (SCR) to rectify semantic drift through frequency-aware channel recalibration. To balance theoretical precision and computational efficiency, SCR is instantiated via two pathways: an exact Laplacian eigendecomposition (SCR-L) and a linear-complexity Chebyshev polynomial approximation (SCR-C). Extensive experiments demonstrate that SM3D achieves state-of-the-art performance, including 96.0% accuracy on ModelNet40 and 86.5% mIoU on ShapeNetPart, validating its effectiveness in mitigating spectral low-pass bias and semantic dilution (Code: https://github.com/L1277471578/SM3D).

Method: FUSS introduces novel federation strategies that promote global consistency in feature and prototype space, jointly optimizing local segmentation heads and shared semantic centroids across distributed clients in a fully decentralized, label-free manner.

Result: Experiments on benchmark and real-world datasets (binary and multi-class segmentation) show FUSS consistently outperforms local-only client trainings and extensions of classical FL algorithms under varying client data distributions.

Conclusion: FUSS successfully enables federated unsupervised semantic segmentation, demonstrating the feasibility of decentralized label-free training while maintaining performance across heterogeneous data distributions, with full reproducibility through publicly available code and scripts.

Abstract: This work explores the application of Federated Learning (FL) to Unsupervised Semantic image Segmentation (USS). Recent USS methods extract pixel-level features using frozen visual foundation models and refine them through self-supervised objectives that encourage semantic grouping. These features are then grouped to semantic clusters to produce segmentation masks. Extending these ideas to federated settings requires feature representation and cluster centroid alignment across distributed clients, an inherently difficult task under heterogeneous data distributions in the absence of supervision. To address this, we propose FUSS (Federated Unsupervised image Semantic Segmentation) which is, to our knowledge, the first framework to enable fully decentralized, label-free semantic segmentation training. FUSS introduces novel federation strategies that promote global consistency in feature and prototype space, jointly optimizing local segmentation heads and shared semantic centroids. Experiments on both benchmark and real-world datasets, including binary and multi-class segmentation tasks, show that FUSS consistently outperforms local-only client trainings as well as extensions of classical FL algorithms under varying client data distributions. To fully support reproducibility, the source code, data partitioning scripts, and implementation details are publicly available at: https://github.com/evanchar/FUSS

Method: Created a comprehensive multi-model multi-generator benchmark with deepfakes from leading academic and commercial models. Includes carefully constructed protocols to assess generalization under distribution shifts in identity and generator characteristics. Benchmarked diverse detection methods (CNNs, vision transformers, temporal models) and performed error analysis using Grad-CAM visualizations.

Result: Developed TalkingHeadBench dataset hosted on Hugging Face with open access to all data splits and protocols. The benchmark evaluates state-of-the-art detectors against the most advanced generators and analyzes their robustness and generalization capabilities.

Conclusion: TalkingHeadBench aims to accelerate research towards more robust and generalizable detection models to address rapidly evolving generative techniques in talking-head deepfakes.

Abstract: The rapid advancement of talking-head deepfake generation fueled by advanced generative models has elevated the realism of synthetic videos to a level that poses substantial risks in domains such as media, politics, and finance. However, current benchmarks for deepfake talking-head detection fail to reflect this progress, relying on outdated generators and offering limited insight into model robustness and generalization. We introduce TalkingHeadBench, a comprehensive multi-model multi-generator benchmark and curated dataset designed to evaluate the performance of state-of-the-art detectors on the most advanced generators. Our dataset includes deepfakes synthesized by leading academic and commercial models and features carefully constructed protocols to assess generalization under distribution shifts in identity and generator characteristics. We benchmark a diverse set of existing detection methods, including CNNs, vision transformers, and temporal models, and analyze their robustness and generalization capabilities. In addition, we provide error analysis using Grad-CAM visualizations to expose common failure modes and detector biases. TalkingHeadBench is hosted on https://huggingface.co/datasets/luchaoqi/TalkingHeadBench with open access to all data splits and protocols. Our benchmark aims to accelerate research towards more robust and generalizable detection models in the face of rapidly evolving generative techniques.

Method: Two-stage framework: 1) Transform raw video into language-based representations using multisensory inputs (clip captions, audio/subtitles), 2) Feed language descriptions into powerful reasoning LLMs. Uses Adaptive Context Reduction to handle long-context inputs by dynamically determining temporal sampling granularity.

Result: Achieves best-reported results on Video-MME (long), Video-MMMU (comprehension), Video-MMLU, CGBench, and EgoLife benchmarks. Shows that strong reasoning LLMs can effectively aggregate multisensory information from video, speech, and audio for complex reasoning tasks.

Conclusion: SILVR demonstrates that simple, modular, training-free approaches can significantly enhance video reasoning capabilities in MLLMs by leveraging powerful text-based LLMs, enabling complex temporal, causal, long-context, and knowledge acquisition reasoning in video understanding.

Abstract: Recent advances in test-time optimization have led to remarkable reasoning capabilities in Large Language Models (LLMs), enabling them to solve highly complex problems in math and coding. However, the reasoning capabilities of multimodal LLMs (MLLMs) still significantly lag, especially for complex video-language tasks. To address this issue, we present SILVR, a Simple Language-based Video Reasoning framework that decomposes complex video understanding into two stages. In the first stage, SILVR transforms raw video into language-based representations using multisensory inputs, such as short clip captions and audio/speech subtitles. In the second stage, language descriptions are fed into a powerful reasoning LLM to solve complex video-language understanding tasks. To handle long-context multisensory inputs, we use an Adaptive Context Reduction scheme, which dynamically determines the temporal granularity with which to sample the tokens. Our simple, modular, and training-free video reasoning framework achieves the best-reported results on Video-MME (long), Video-MMMU (comprehension), Video-MMLU, CGBench, and EgoLife. Furthermore, our empirical study focused on video reasoning capabilities shows that, despite not being explicitly trained on video, strong reasoning LLMs can effectively aggregate multisensory input information from video, speech, and audio for complex temporal, causal, long-context, and knowledge acquisition reasoning tasks in video. More details can be found at https://sites.google.com/cs.unc.edu/silvr.

Method: Introduced CulturalFrames benchmark with 983 prompts spanning 10 countries and 5 socio-cultural domains. Generated 3637 images using 4 state-of-the-art T2I models and collected over 10k detailed human annotations to evaluate both explicit (stated) and implicit (unstated, implied) cultural expectations.

Result: Across models and countries, cultural expectations were missed an average of 44% of the time. Explicit expectations were missed at 68% rate, while implicit expectation failures averaged 49%. Existing T2I evaluation metrics showed poor correlation with human judgments of cultural alignment.

Conclusion: The study exposes critical gaps in T2I models’ cultural representation, provides a concrete testbed (CulturalFrames), and outlines actionable directions for developing culturally informed models and metrics to improve global usability.

Abstract: The increasing ubiquity of text-to-image (T2I) models as tools for visual content generation raises concerns about their ability to accurately represent diverse cultural contexts – where missed cues can stereotype communities and undermine usability. In this work, we present the first study to systematically quantify the alignment of T2I models and evaluation metrics with respect to both explicit (stated) as well as implicit (unstated, implied by the prompt’s cultural context) cultural expectations. To this end, we introduce CulturalFrames, a novel benchmark designed for rigorous human evaluation of cultural representation in visual generations. Spanning 10 countries and 5 socio-cultural domains, CulturalFrames comprises 983 prompts, 3637 corresponding images generated by 4 state-of-the-art T2I models, and over 10k detailed human annotations. We find that across models and countries, cultural expectations are missed an average of 44% of the time. Among these failures, explicit expectations are missed at a surprisingly high average rate of 68%, while implicit expectation failures are also significant, averaging 49%. Furthermore, we show that existing T2I evaluation metrics correlate poorly with human judgments of cultural alignment, irrespective of their internal reasoning. Collectively, our findings expose critical gaps, provide a concrete testbed, and outline actionable directions for developing culturally informed T2I models and metrics that improve global usability.

Method: Developed StructuralGeo simulation engine to generate unlimited synthetic 3D lithological models, then trained both unconditional and conditional generative flow-matching models with 3D attention U-Net architecture.

Result: Created a foundation model that reconstructs multiple plausible 3D geological scenarios from surface topography and sparse borehole data, capturing structures like layers, faults, folds, and dikes.

Conclusion: The combination of geological simulation and generative AI provides a flexible prior for probabilistic modeling, regional fine-tuning, and AI-based regularization in traditional geophysical inversion workflows.

Abstract: Reconstructing the structural geology and mineral composition of the first few kilometers of the Earth’s subsurface from sparse or indirect surface observations remains a long-standing challenge with critical applications in mineral exploration, geohazard assessment, and geotechnical engineering. This inherently ill-posed problem is often addressed by classical geophysical inversion methods, which typically yield a single maximum-likelihood model that fails to capture the full range of plausible geology. The adoption of modern deep learning methods has been limited by the lack of large 3D training datasets. We address this gap with \textit{StructuralGeo}, a geological simulation engine that mimics eons of tectonic, magmatic, and sedimentary processes to generate a virtually limitless supply of realistic synthetic 3D lithological models. Using this dataset, we train both unconditional and conditional generative flow-matching models with a 3D attention U-Net architecture. The resulting foundation model can reconstruct multiple plausible 3D scenarios from surface topography and sparse borehole data, depicting structures such as layers, faults, folds, and dikes. By sampling many reconstructions from the same observations, we introduce a probabilistic framework for estimating the size and extent of subsurface features. While the realism of the output is bounded by the fidelity of the training data to true geology, this combination of simulation and generative AI functions offers a flexible prior for probabilistic modeling, regional fine-tuning, and use as an AI-based regularizer in traditional geophysical inversion workflows.

Method: Pretrained and fine-tuned models on three dataset scales (PhilEO Globe: 0.5TB, FastTOM: 2TB, MajorTOM: 23TB), evaluated three architectural families (Geo-Aware U-Net/CNN, ViT-UPerNet/Transformer, Mamba/State-Space Model) across 44M to 300M parameters, benchmarked on PhilEO Bench tasks.

Result: CNN-based models remain competitive in low-shot settings (200M Geo-Aware U-Net outperforms larger architectures on regression), but ViT-UPerNet achieves best performance when scaling to multi-terabyte datasets (especially for semantic segmentation on 23TB MajorTOM). Mamba shows efficiency advantages but needs more large-scale pretraining to match CNNs/ViTs.

Conclusion: Dataset scale significantly impacts optimal architecture choice: CNNs excel in low-data regimes, Transformers dominate at large scale, and Mamba shows promise for efficiency. The work provides scaling law insights and releases code/models/dataset for reproducibility in GeoSpatial Foundation Models research.

Abstract: Foundation Models (FMs) have achieved state-of-the-art performance across domains by leveraging large-scale pretraining. In Earth Observation (EO), the availability of petabyte-scale satellite archives has recently enabled the development of GeoSpatial Foundation Models (GFMs). Yet, fundamental questions remain regarding how dataset size, model architecture, and size interact to determine downstream performance. In this work, we systematically explore this design space by pretraining and fine-tuning models on three dataset scales: PhilEO Globe (0.5TB), FastTOM (2TB, introduced here), and MajorTOM (23TB). We evaluate three architectural families: Geo-Aware U-Net (CNN), ViT-UPerNet (Transformer), and Mamba (State-Space Model); across model sizes ranging from 44M to 300M parameters. All models are benchmarked on the PhilEO Bench, covering: road density and building density regression, and land cover segmentation, and are compared against existing GFMs such as TerraMind and Prithvi-EO-2.0. Our results show that CNN-based models remain highly competitive in low-shot settings, with a 200M-parameter Geo-Aware U-Net outperforming larger architectures on regression tasks. However, when scaling to multi-terabyte datasets, ViT-UPerNet achieves the best performance, particularly for semantic segmentation on MajorTOM (23TB). Finally, we provide the first extensive evaluation of Mamba models in EO, highlighting their potential efficiency advantages, though further large-scale pretraining is required to fully match CNNs and ViTs. All code, pretrained models, and the FastTOM dataset are released publicly, enabling reproducibility and further exploration of scaling laws for GFMs.

Method: Combines existing supervised learning backdoor literature targeting face detectors, face antispoofing, and face feature extractors. Analyzes 20 pipeline configurations and 15 attack scenarios holistically to demonstrate system-level vulnerability.

Result: Reveals that an attacker only needs a single backdoored model to compromise an entire face recognition system, demonstrating significant system-level vulnerability.

Conclusion: Discusses the impact of such attacks and proposes best practices and countermeasures for stakeholders to address these security vulnerabilities in face recognition systems.

Abstract: The widespread deployment of Deep Learning-based Face Recognition Systems raises many security concerns. While prior research has identified backdoor vulnerabilities on isolated components, Backdoor Attacks on real-world, unconstrained pipelines remain underexplored. This SoK paper presents the first comprehensive system-level analysis and measurement of the impact of Backdoor Attacks on fully-fledged Face Recognition Systems. We combine the existing Supervised Learning backdoor literature targeting face detectors, face antispoofing, and face feature extractors to demonstrate a system-level vulnerability. By analyzing 20 pipeline configurations and 15 attack scenarios in a holistic manner, we reveal that an attacker only needs a single backdoored model to compromise an entire Face Recognition System. Finally, we discuss the impact of such attacks and propose best practices and countermeasures for stakeholders.

Method: Systematic research methodology to identify state-of-the-art works, organized in taxonomy-based format covering input modalities, task types, and application domains. Covers three primary VHGR tasks: static gesture recognition, isolated dynamic gestures, and continuous gesture recognition.

Result: Provides comprehensive overview of VHGR field including architectural trends, learning strategies, commonly used datasets, and standard performance metrics to support experimental evaluation of future methods.

Conclusion: Identifies major challenges in VHGR (both general computer vision issues and domain-specific obstacles) and outlines promising directions for future research, serving as a useful guideline for researchers in the field.

Abstract: The rapid evolution of deep learning (DL) models and the ever-increasing size of available datasets have raised the interest of the research community in the always important field of visual hand gesture recognition (VHGR), and delivered a wide range of applications, such as sign language understanding and human-computer interaction using cameras. Despite the large volume of research works in the field, a structured and complete survey on VHGR is still missing, leaving researchers to navigate through hundreds of papers in order to find the right combination of data, model, and approach for each task. The current survey aims to fill this gap by presenting a comprehensive overview of this computer vision field. With a systematic research methodology that identifies the state-of-the-art works and a structured presentation of the various methods, datasets, and evaluation metrics, this review aims to constitute a useful guideline for researchers, helping them to choose the right strategy for handling a VHGR task. Starting with the methodology used to locate the related literature, the survey identifies and organizes the key VHGR approaches in a taxonomy-based format, and presents the various dimensions that affect the final method choice, such as input modality, task type, and application domain. The state-of-the-art techniques are grouped across three primary VHGR tasks: static gesture recognition, isolated dynamic gestures, and continuous gesture recognition. For each task, the architectural trends and learning strategies are listed. To support the experimental evaluation of future methods in the field, the study reviews commonly used datasets and presents the standard performance metrics. Our survey concludes by identifying the major challenges in VHGR, including both general computer vision issues and domain-specific obstacles, and outlines promising directions for future research.

Method: SGPMIL uses Sparse Gaussian Processes to learn posterior distributions over attention scores, introducing feature scaling in the SGP predictive mean function for faster training and improved efficiency.

Result: Extensive experiments on digital pathology datasets show SGPMIL preserves competitive bag-level performance while significantly improving instance-level prediction quality and interpretability under uncertainty.

Conclusion: SGPMIL provides a principled probabilistic framework for uncertainty-aware MIL that enhances reliability and interpretability of instance relevance predictions in digital pathology applications.

Abstract: Multiple Instance Learning (MIL) offers a natural solution for settings where only coarse, bag-level labels are available, without having access to instance-level annotations. This is usually the case in digital pathology, which consists of gigapixel-sized images. While deterministic attention-based MIL approaches achieve strong bag-level performance, they often overlook the uncertainty inherent in instance relevance. In this paper, we address the lack of uncertainty quantification in instance-level attention scores by introducing SGPMIL, a new probabilistic attention-based MIL framework grounded in Sparse Gaussian Processes (SGP). By learning a posterior distribution over attention scores, SGPMIL enables principled uncertainty estimation, resulting in more reliable and calibrated instance relevance maps. Our approach not only preserves competitive bag-level performance but also significantly improves the quality and interpretability of instance-level predictions under uncertainty. SGPMIL extends prior work by introducing feature scaling in the SGP predictive mean function, leading to faster training, improved efficiency, and enhanced instance-level performance. Extensive experiments on multiple well-established digital pathology datasets highlight the effectiveness of our approach across both bag- and instance-level evaluations. Our code is available at https://github.com/mandlos/SGPMIL.

Method: Proposes a paired image generation method that trains an extra diffusion guider for conditional diffusion models, enabling generation of both DBT slices and corresponding mass lesion masks without requiring external conditions.

Result: The method improves generation quality without external conditions and generates paired DBT slices with lesion masks. When incorporated into supervised training for mass lesion segmentation, it enhances downstream task performance by alleviating annotated data shortages.

Conclusion: The proposed paired image generation approach effectively addresses both the quality limitation in lesion area generation and the lack of corresponding annotations in existing diffusion-based augmentation methods, improving downstream segmentation performance for breast cancer screening.

Abstract: The segmentation of mass lesions in digital breast tomosynthesis (DBT) images is very significant for the early screening of breast cancer. However, the high-density breast tissue often leads to high concealment of the mass lesions, which makes manual annotation difficult and time-consuming. As a result, there is a lack of annotated data for model training. Diffusion models are commonly used for data augmentation, but the existing methods face two challenges. First, due to the high concealment of lesions, it is difficult for the model to learn the features of the lesion area. This leads to the low generation quality of the lesion areas, thus limiting the quality of the generated images. Second, existing methods can only generate images and cannot generate corresponding annotations, which restricts the usability of the generated images in supervised training. In this work, we propose a paired image generation method. The method does not require external conditions and can achieve the generation of paired images by training an extra diffusion guider for the conditional diffusion model. During the experimental phase, we generated paired DBT slices and mass lesion masks. Then, we incorporated them into the supervised training process of the mass lesion segmentation task. The experimental results show that our method can improve the generation quality without external conditions. Moreover, it contributes to alleviating the shortage of annotated data, thus enhancing the performance of downstream tasks. The source code is available at https://github.com/zhanghx1320/PIG.

Method: Organized the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) sub-challenge at MICCAI 2024 EndoVis challenge. Created a novel multi-center dataset of 13 full-length laparoscopic cholecystectomy videos from 3 institutions with unified annotations for three interrelated tasks: surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation.

Result: The dataset enables joint investigation of instrument localization and procedural context within the same data while supporting integration of temporal information across entire procedures. Results are reported following BIAS guidelines for biomedical image analysis challenges.

Conclusion: The PhaKIR sub-challenge advances the field by providing a unique benchmark for developing temporally aware, context-driven methods in RAMIS and offers a high-quality resource for future research in surgical scene understanding.

Abstract: Reliable recognition and localization of surgical instruments in endoscopic video recordings are foundational for a wide range of applications in computer- and robot-assisted minimally invasive surgery (RAMIS), including surgical training, skill assessment, and autonomous assistance. However, robust performance under real-world conditions remains a significant challenge. Incorporating surgical context - such as the current procedural phase - has emerged as a promising strategy to improve robustness and interpretability. To address these challenges, we organized the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) sub-challenge as part of the Endoscopic Vision (EndoVis) challenge at MICCAI 2024. We introduced a novel, multi-center dataset comprising thirteen full-length laparoscopic cholecystectomy videos collected from three distinct medical institutions, with unified annotations for three interrelated tasks: surgical phase recognition, instrument keypoint estimation, and instrument instance segmentation. Unlike existing datasets, ours enables joint investigation of instrument localization and procedural context within the same data while supporting the integration of temporal information across entire procedures. We report results and findings in accordance with the BIAS guidelines for biomedical image analysis challenges. The PhaKIR sub-challenge advances the field by providing a unique benchmark for developing temporally aware, context-driven methods in RAMIS and offers a high-quality resource to support future research in surgical scene understanding.

Method: Wukong leverages two key insights about diffusion models: (1) early denoising steps define semantic layout, and (2) cross-attention layers align text and image regions. It uses a transformer-based framework that analyzes intermediate outputs from early denoising steps and reuses U-Net’s pre-trained cross-attention parameters, operating within the diffusion process for early detection.

Result: Wukong significantly outperforms text-based safeguards and achieves comparable accuracy to image filters while offering much greater efficiency. The authors also introduced a new dataset with prompts, seeds, and image-specific NSFW labels for evaluation.

Conclusion: Wukong provides an effective solution for NSFW detection in T2I generation by working within the diffusion process, enabling early detection without waiting for full image generation, balancing accuracy and efficiency better than existing approaches.

Abstract: Text-to-Image (T2I) generation is a popular AI-generated content (AIGC) technology enabling diverse and creative image synthesis. However, some outputs may contain Not Safe For Work (NSFW) content (e.g., violence), violating community guidelines. Detecting NSFW content efficiently and accurately, known as external safeguarding, is essential. Existing external safeguards fall into two types: text filters, which analyze user prompts but overlook T2I model-specific variations and are prone to adversarial attacks; and image filters, which analyze final generated images but are computationally costly and introduce latency. Diffusion models, the foundation of modern T2I systems like Stable Diffusion, generate images through iterative denoising using a U-Net architecture with ResNet and Transformer blocks. We observe that: (1) early denoising steps define the semantic layout of the image, and (2) cross-attention layers in U-Net are crucial for aligning text and image regions. Based on these insights, we propose Wukong, a transformer-based NSFW detection framework that leverages intermediate outputs from early denoising steps and reuses U-Net’s pre-trained cross-attention parameters. Wukong operates within the diffusion process, enabling early detection without waiting for full image generation. We also introduce a new dataset containing prompts, seeds, and image-specific NSFW labels, and evaluate Wukong on this and two public benchmarks. Results show that Wukong significantly outperforms text-based safeguards and achieves comparable accuracy of image filters, while offering much greater efficiency.

Method: Hierarchical Cascade Framework (HCF) uses direct latent-space transformations across network nodes, with policy-driven quantization control and edge quantization principle based on differential entropy analysis.

Result: HCF achieves up to 0.6dB PSNR gains, outperforms successive compression by up to 5.56% BD-Rate on CLIC while saving up to 97.8% FLOPs, 96.5% GPU memory, and 90.0% execution time. Also beats progressive methods by up to 12.64% BD-Rate on Kodak.

Conclusion: HCF provides an efficient solution for distributed multi-stage image compression with superior rate-distortion performance, computational efficiency, and retraining-free cross-quality adaptation capabilities.

Abstract: Distributed multi-stage image compression – where visual content traverses multiple processing nodes under varying quality requirements – poses challenges. Progressive methods enable bitstream truncation but underutilize available compute resources; successive compression repeats costly pixel-domain operations and suffers cumulative quality loss and inefficiency; fixed-parameter models lack post-encoding flexibility. In this work, we developed the Hierarchical Cascade Framework (HCF) that achieves high rate-distortion performance and better computational efficiency through direct latent-space transformations across network nodes in distributed multi-stage image compression systems. Under HCF, we introduced policy-driven quantization control to optimize rate-distortion trade-offs, and established the edge quantization principle through differential entropy analysis. The configuration based on this principle demonstrates up to 0.6dB PSNR gains over other configurations. When comprehensively evaluated on the Kodak, CLIC, and CLIC2020-mobile datasets, HCF outperforms successive-compression methods by up to 5.56% BD-Rate in PSNR on CLIC, while saving up to 97.8% FLOPs, 96.5% GPU memory, and 90.0% execution time. It also outperforms state-of-the-art progressive compression methods by up to 12.64% BD-Rate on Kodak and enables retraining-free cross-quality adaptation with 7.13-10.87% BD-Rate reductions on CLIC2020-mobile.

Method: Construct two ultra-small backbone families (<1M parameters) using scaling laws from standard object recognition architectures. Systematically study ImageNet pretraining effect on downstream infrared object detection tasks across three datasets.

Result: ImageNet pre-training is still useful for small models, but beyond a certain capacity threshold, it offers diminishing returns for out-of-distribution detection robustness. Too small models work well for in-domain problems but are brittle when conditions differ.

Conclusion: Practitioners should still use pre-training and avoid too small models when possible, as while small models work for in-domain problems, they lack robustness when working conditions change.

Abstract: Many real-world applications require recognition models that are robust to different operational conditions and modalities, but at the same time run on small embedded devices, with limited hardware. While for normal size models, pre-training is known to be very beneficial in accuracy and robustness, for small models, that can be employed for embedded and edge devices, its effect is not clear. In this work, we investigate the effect of ImageNet pretraining on increasingly small backbone architectures (ultra-small models, with less than 1M parameters) with respect to robustness in downstream object detection tasks in the infrared visual modality. Using scaling laws derived from standard object recognition architectures, we construct two ultra-small backbone families and systematically study their performance. Our experiments on three different datasets reveal that while ImageNet pre-training is still useful, beyond a certain capacity threshold, it offers diminishing returns in terms of out-of-distribution detection robustness. Therefore, we advise practitioners to still use pre-training and, when possible avoid too small models as while they might work well for in-domain problems, they are brittle when working conditions are different.

Method: HierarchicalPrune combines three techniques: (1) Hierarchical Position Pruning that removes less essential later blocks based on functional hierarchies, (2) Positional Weight Preservation that protects early model portions essential for semantic structure, and (3) Sensitivity-Guided Distillation that adjusts knowledge-transfer intensity based on block-wise sensitivity variations.

Result: With INT4 weight quantization, HierarchicalPrune achieves 77.5-80.4% memory reduction (e.g., 15.8 GB to 3.2 GB) and 27.9-38.0% latency reduction on server/consumer GPUs, with minimal quality drops (2.6% in GenEval, 7% in HPSv2). User study with 85 participants shows perceptual quality comparable to original model.

Conclusion: HierarchicalPrune successfully compresses billion-scale diffusion models for on-device inference while preserving output quality, significantly outperforming prior compression methods through its hierarchical approach to model pruning.

Abstract: State-of-the-art text-to-image diffusion models (DMs) achieve remarkable quality, yet their massive parameter scale (8-11B) poses significant challenges for inferences on resource-constrained devices. In this paper, we present HierarchicalPrune, a novel compression framework grounded in a key observation: DM blocks exhibit distinct functional hierarchies, where early blocks establish semantic structures while later blocks handle texture refinements. HierarchicalPrune synergistically combines three techniques: (1) Hierarchical Position Pruning, which identifies and removes less essential later blocks based on position hierarchy; (2) Positional Weight Preservation, which systematically protects early model portions that are essential for semantic structural integrity; and (3) Sensitivity-Guided Distillation, which adjusts knowledge-transfer intensity based on our discovery of block-wise sensitivity variations. As a result, our framework brings billion-scale diffusion models into a range more suitable for on-device inference, while preserving the quality of the output images. Specifically, combined with INT4 weight quantisation, HierarchicalPrune achieves 77.5-80.4% memory footprint reduction (e.g., from 15.8 GB to 3.2 GB) and 27.9-38.0% latency reduction, measured on server and consumer grade GPUs, with the minimum drop of 2.6% in GenEval score and 7% in HPSv2 score compared to the original model. Finally, our comprehensive user study with 85 participants demonstrates that HierarchicalPrune maintains perceptual quality comparable to the original model while significantly outperforming prior works.

Method: Propose DocReward, a document reward model trained under a textual-quality-agnostic framework. Construct DocPair dataset of 117K paired documents across 32 domains and 267 document types, each containing high- and low-professionalism documents with identical content but different structure/style. Train using Bradley-Terry loss to score documents and penalize predictions that contradict annotated rankings.

Result: DocReward outperforms GPT-5 by 14.6 percentage points in accuracy on a manually annotated benchmark. Extrinsic reinforcement learning experiments validate its effectiveness in guiding professional document generation.

Conclusion: DocReward successfully addresses the gap in evaluating document professionalism beyond textual quality, providing an effective reward model for guiding agentic workflows in professional document generation with improved structural and stylistic quality.

Abstract: Recent advances in agentic workflows have enabled the automation of tasks such as professional document generation. However, they primarily focus on textual quality, neglecting visual structure and style, which are crucial for readability and engagement. This gap stems mainly from a lack of effective reward models capable of guiding agents toward producing documents with high structural and stylistic professionalism. To address this, we propose DocReward, a document reward model that evaluates documents based on their structure and style. The model is trained under a textual-quality-agnostic framework to assess professionalism without being influenced by textual quality. To achieve this, we construct a multi-domain dataset DocPair of 117K paired documents, covering 32 domains and 267 document types, each comprising a high- and low-professionalism document with identical content but different structure and style. This setup enables the model to evaluate professionalism comprehensively and independently of textual quality. DocReward is trained using the Bradley-Terry loss to score documents, penalizing predictions that contradict the annotated ranking. On a manually annotated benchmark, DocReward outperforms GPT-5 by 14.6 percentage points in accuracy. Extrinsic RL experiments further validate its effectiveness in guiding professional document generation.

Method: Proposes a joint learning framework that extracts image components with diffusion models to represent clean signals, uses these components for translation, and employs a time-dependent translation network for complex mapping, enabling end-to-end joint learning.

Result: Extensive experiments on RGB↔RGB and cross-modality tasks (RGB↔Edge, RGB↔Semantics, RGB↔Depth) show better generative performance than state-of-the-art methods.

Conclusion: The joint learning approach enables global optimization of both diffusion and translation processes, improving optimality and achieving enhanced fidelity and structural consistency in cross-domain image translation.

Abstract: We introduce a diffusion-based cross-domain image translator in the absence of paired training data. Unlike GAN-based methods, our approach integrates diffusion models to learn the image translation process, allowing for more coverable modeling of the data distribution and performance improvement of the cross-domain translation. However, incorporating the translation process within the diffusion process is still challenging since the two processes are not aligned exactly, i.e., the diffusion process is applied to the noisy signal while the translation process is conducted on the clean signal. As a result, recent diffusion-based studies employ separate training or shallow integration to learn the two processes, yet this may cause the local minimal of the translation optimization, constraining the effectiveness of diffusion models. To address the problem, we propose a novel joint learning framework that aligns the diffusion and the translation process, thereby improving the global optimality. Specifically, we propose to extract the image components with diffusion models to represent the clean signal and employ the translation process with the image components, enabling an end-to-end joint learning manner. On the other hand, we introduce a time-dependent translation network to learn the complex translation mapping, resulting in effective translation learning and significant performance improvement. Benefiting from the design of joint learning, our method enables global optimization of both processes, enhancing the optimality and achieving improved fidelity and structural consistency. We have conducted extensive experiments on RGB$\leftrightarrow$RGB and diverse cross-modality translation tasks including RGB$\leftrightarrow$Edge, RGB$\leftrightarrow$Semantics and RGB$\leftrightarrow$Depth, showcasing better generative performances than the state of the arts.

Method: LaRe combines visual refocusing with rich latent representations for iterative reasoning in latent space, plus a semantic augmentation training strategy with joint alignment and reconstruction objectives to enhance semantic structure.

Result: LaRe improves average accuracy by 9.4% over baselines, reduces inference tokens by 16.5%, and when scaled to 7B-parameter LLM backbone, achieves SOTA-comparable performance while outperforming larger-scale models on most benchmarks.

Conclusion: LaRe effectively addresses multimodal reasoning limitations by enabling iterative latent space reasoning with visual refocusing, achieving superior performance with computational efficiency.

Abstract: Chain of Thought (CoT) reasoning enhances logical performance by decomposing complex tasks, yet its multimodal extension faces a trade-off. The existing Thinking with Images paradigm is limited by the modality gap between vision and language, which hinders reliable extraction of reasoning relevant information from high dimensional visual data. Recent latent space reasoning method provides stronger multimodal representations, but it often lacks the ability to refocus on visual inputs and suffers from limited interpretability. To address these issues, we propose \underline{La}tent \underline{Re}focusing (LaRe), a novel multimodal reasoning paradigm that combines visual refocusing with rich latent representations, enabling iterative reasoning within the latent space. We further design a semantic augmentation training strategy that enhances the semantic structure of the latent space through joint alignment and reconstruction objectives. Experimental evaluations demonstrate that LaRe improves average accuracy by 9.4% compared to existing baselines while reducing the number of tokens required for inference by 16.5%. When scaled to a 7B-parameter Large Language Model backbone, LaRe achieves performance comparable to state-of-the-art models and outperforms larger-scale models on almost all benchmarks. Code and checkpoints will be released later.

Method: TransMiter is a lightweight adapter that captures the knowledge gap between pre-trained and fine-tuned VLMs in an unsupervised manner. Once trained, this knowledge can be transferred across different models without backpropagation, requiring only a few layers.

Result: TransMiter effectively transfers adaptation knowledge while preserving generalization abilities across VLMs of different sizes and architectures. With minimal labeled data, it often surpasses fine-tuned stronger models with marginal training cost.

Conclusion: TransMiter provides an efficient, model-agnostic solution for transferring adaptation knowledge between VLMs without backpropagation, enabling cost-effective enhancement of larger models using knowledge from smaller ones.

Abstract: Vision-Language Models (VLMs) have been widely used in various visual recognition tasks due to their remarkable generalization capabilities. As these models grow in size and complexity, fine-tuning becomes costly, emphasizing the need to reuse adaptation knowledge from ‘weaker’ models to efficiently enhance ‘stronger’ ones. However, existing adaptation transfer methods exhibit limited transferability across models due to their model-specific design and high computational demands. To tackle this, we propose Transferable Model-agnostic adapter (TransMiter), a light-weight adapter that improves vision-language models ‘without backpropagation’. TransMiter captures the knowledge gap between pre-trained and fine-tuned VLMs, in an ‘unsupervised’ manner. Once trained, this knowledge can be seamlessly transferred across different models without the need for backpropagation. Moreover, TransMiter consists of only a few layers, inducing a negligible additional inference cost. Notably, supplementing the process with a few labeled data further yields additional performance gain, often surpassing a fine-tuned stronger model, with a marginal training cost. Experimental results and analyses demonstrate that TransMiter effectively and efficiently transfers adaptation knowledge while preserving generalization abilities across VLMs of different sizes and architectures in visual recognition tasks.

Method: Calibration Attention (CalAttn) predicts sample-specific temperature from the [CLS] token and backpropagates calibration gradients into the transformer backbone. This enables token-conditioned uncertainty modulation with minimal overhead (<0.1% additional parameters) while reshaping the uncertainty structure of representations.

Result: Across multiple datasets with ViT/DeiT/Swin backbones, CalAttn consistently improves calibration while preserving accuracy, achieving relative ECE reductions of 3.7% to 77.7% over strong baselines across diverse training objectives.

Conclusion: Treating calibration as a representation-level problem is a practical and effective direction for trustworthy uncertainty estimation in transformers, moving beyond post-hoc logit adjustments to reshape underlying uncertainty structure.

Abstract: Most calibration methods operate at the logit level, implicitly assuming that miscalibration can be corrected without changing the underlying representation. We challenge this assumption and propose \textbf{Calibration Attention (CalAttn)}, a \emph{representation-aware} calibration module for vision transformers that couples instance-wise temperature scaling to transformer token geometry under a proper scoring objective. CalAttn predicts a sample-specific temperature from the \texttt{[CLS]} token and backpropagates calibration gradients into the backbone, thereby reshaping the uncertainty structure of the representation rather than post-hoc adjusting confidence. This yields \emph{token-conditioned uncertainty modulation} with negligible overhead ((<0.1%) additional parameters). Across multiple datasets with ViT/DeiT/Swin backbones, CalAttn consistently improves calibration while preserving accuracy, achieving relative ECE reductions of (3.7%) to (77.7%) over strong baselines across diverse training objectives. Our results indicate that treating calibration as a representation-level problem is a practical and effective direction for trustworthy uncertainty estimation in transformers. Code: https://github.com/EagleAdelaide/CalibrationAttention-CalAttn-

Method: Systematic survey approach reviewing RL-based methods across visual content generation domains. Examines RL evolution from classical control to general-purpose optimization tool, and its integration into image, video, and 3D/4D generation. Analyzes RL’s dual role as both fine-tuning mechanism and structural component for aligning generation with complex high-level goals.

Result: Recent advances demonstrate RL’s effectiveness in enhancing controllability, consistency, and human alignment across generative tasks. The survey provides comprehensive overview of RL applications in visual content generation, showing how RL addresses limitations of traditional surrogate objectives.

Conclusion: RL offers powerful framework for optimizing visual content generation beyond traditional surrogate objectives. The paper concludes with open challenges and future research directions at the intersection of RL and generative modeling, highlighting RL’s potential as both fine-tuning tool and structural component for complex generation tasks.

Abstract: Generative models have made significant progress in synthesizing visual content, including images, videos, and 3D/4D structures. However, they are typically trained with surrogate objectives such as likelihood or reconstruction loss, which often misalign with perceptual quality, semantic accuracy, or physical realism. Reinforcement learning (RL) offers a principled framework for optimizing non-differentiable, preference-driven, and temporally structured objectives. Recent advances demonstrate its effectiveness in enhancing controllability, consistency, and human alignment across generative tasks. This survey provides a systematic overview of RL-based methods for visual content generation. We review the evolution of RL from classical control to its role as a general-purpose optimization tool, and examine its integration into image, video, and 3D/4D generation. Across these domains, RL serves not only as a fine-tuning mechanism but also as a structural component for aligning generation with complex, high-level goals. We conclude with open challenges and future research directions at the intersection of RL and generative modeling.

Method: Created STRIDE-QA dataset from 100 hours of multi-sensor driving data in Tokyo, with dense automatically generated annotations (3D bounding boxes, segmentation masks, multi-object tracks) and three novel QA tasks for object-centric and ego-centric reasoning.

Result: Existing VLMs struggle significantly (near-zero scores on prediction consistency), while VLMs fine-tuned on STRIDE-QA achieve 55% success in spatial localization and 28% consistency in future motion prediction.

Conclusion: STRIDE-QA establishes a comprehensive foundation for developing more reliable VLMs for safety-critical autonomous systems by addressing the spatiotemporal reasoning gap in current vision-language models.

Abstract: Vision-Language Models (VLMs) have been applied to autonomous driving to support decision-making in complex real-world scenarios. However, their training on static, web-sourced image-text pairs fundamentally limits the precise spatiotemporal reasoning required to understand and predict dynamic traffic scenes. We address this critical gap with STRIDE-QA, a large-scale visual question answering (VQA) dataset for physically grounded reasoning from an ego-centric perspective. Constructed from 100 hours of multi-sensor driving data in Tokyo, capturing diverse and challenging conditions, STRIDE-QA is the largest VQA dataset for spatiotemporal reasoning in urban driving, offering 16M QA pairs over 270K frames. Grounded by dense, automatically generated annotations including 3D bounding boxes, segmentation masks, and multi-object tracks, the dataset uniquely supports both object-centric and ego-centric reasoning through three novel QA tasks that require spatial localization and temporal prediction. Our benchmarks demonstrate that existing VLMs struggle significantly, with near-zero scores on prediction consistency. In contrast, VLMs fine-tuned on STRIDE-QA exhibit dramatic performance gains, achieving 55% success in spatial localization and 28% consistency in future motion prediction, compared to near-zero scores from general-purpose VLMs. Therefore, STRIDE-QA establishes a comprehensive foundation for developing more reliable VLMs for safety-critical autonomous systems.

Method: Proposes SafePatch, a structurally isolated safety module with a trainable clone of the base model’s encoder. Uses a strictly aligned counterfactual safety dataset (ACS) for differential supervision training to enable interpretable safety rectification.

Result: Achieves robust unsafe suppression (7% unsafe on I2P benchmark) while preserving image quality and semantic alignment across nudity and multi-category benchmarks and recent adversarial prompt attacks.

Conclusion: SafePatch demonstrates that external safety rectification is an effective paradigm that overcomes the Safety Tax limitation, providing robust safety without compromising generation quality through interpretable, structurally isolated intervention.

Abstract: Text-to-image (T2I) generative models have achieved remarkable visual fidelity, yet remain vulnerable to generating unsafe content. Existing safety defenses typically intervene internally within the generative model, but suffer from severe concept entanglement, leading to degradation of benign generation quality, a trade-off we term the Safety Tax. To overcome this limitation, we advocate a paradigm shift from destructive internal editing to external safety rectification. Following this principle, we propose SafePatch, a structurally isolated safety module that performs external, interpretable rectification without modifying the base model. The core backbone of SafePatch is architecturally instantiated as a trainable clone of the base model’s encoder, allowing it to inherit rich semantic priors and maintain representation consistency. To enable interpretable safety rectification, we construct a strictly aligned counterfactual safety dataset (ACS) for differential supervision training. Across nudity and multi-category benchmarks and recent adversarial prompt attacks, SafePatch achieves robust unsafe suppression (7% unsafe on I2P) while preserving image quality and semantic alignment.

Method: Learns a high-success-rate distribution conditioned on target prompts by explicitly modeling the signal component during denoising, providing fine-grained control. Training-free and compatible with diffusion and flow matching architectures, with support for additional conditioning like bounding boxes.

Result: Extensive experiments show the approach outperforms current state-of-the-art methods in text-to-image alignment.

Conclusion: The proposed method effectively addresses alignment issues in text-to-image generation through a novel distribution learning approach that maintains fidelity while avoiding common artifacts, with code publicly available.

Abstract: State-of-the-art text-to-image models produce visually impressive results but often struggle with precise alignment to text prompts, leading to missing critical elements or unintended blending of distinct concepts. We propose a novel approach that learns a high-success-rate distribution conditioned on a target prompt, ensuring that generated images faithfully reflect the corresponding prompts. Our method explicitly models the signal component during the denoising process, offering fine-grained control that mitigates over-optimization and out-of-distribution artifacts. Moreover, our framework is training-free and seamlessly integrates with both existing diffusion and flow matching architectures. It also supports additional conditioning modalities – such as bounding boxes – for enhanced spatial alignment. Extensive experiments demonstrate that our approach outperforms current state-of-the-art methods. The code is available at https://github.com/grimalPaul/gsn-factory.

Method: 1) MedForget benchmark: Models hospital data as nested structure for fine-grained evaluation across retain/forget splits. 2) CHIP method: Training-free, hierarchy-aware multimodal unlearning that removes target-specific weight subspaces while preserving sibling-shared information.

Result: Existing methods struggle with hierarchy-aware forgetting. CHIP achieves highest forget-retain performance gap across all hierarchy levels while maintaining competitive downstream utility compared to existing methods.

Conclusion: MedForget provides a practical, HIPAA-aligned benchmark for structured multimodal unlearning in medical data, and CHIP offers an effective general solution for hierarchy-aware forgetting that balances deletion with utility.

Abstract: Pretrained Multimodal Large Language Models (MLLMs) are increasingly used in sensitive domains such as medical AI, where privacy regulations like HIPAA and GDPR require specific removal of individuals’ or institutions’ data. This motivates machine unlearning, which aims to remove the influence of target data from a trained model. However, existing unlearning benchmarks fail to reflect the hierarchical and multimodal structure of real-world medical data, limiting their ability to properly evaluate unlearning in practice. Therefore, we introduce MedForget, a hierarchy-aware multimodal unlearning benchmark that models hospital data as a nested structure, enabling fine-grained evaluation of multimodal unlearning across retain and forget splits. Experiments with current unlearning methods show that existing approaches struggle to achieve effective hierarchy-aware forgetting without degrading downstream medical utility. To address this limitation, we propose Cross-modal Hierarchy-Informed Projection for unlearning (CHIP), a training-free, hierarchy-aware multimodal unlearning method that deletes information by selectively removing target-specific weight subspaces while preserving sibling-shared information. Experiments show that CHIP achieves the highest forget-retain performance gap across all hierarchy levels while maintaining competitive downstream utility compared to existing methods. Overall, MedForget provides a practical, HIPAA-aligned benchmark for evaluating structured multimodal unlearning for medical data, and CHIP offers an effective and general solution for hierarchy-aware forgetting that balances deletion with utility.

Method: Proposes sensitivity-regularized fine-tuning framework that: 1) probes tangent space of pre-trained weights to measure prior sensitivities for preserving generalization, 2) characterizes transfer sensitivities during tuning for adaptability and stability, and 3) incorporates these sensitivities as unified regularization terms.

Result: Method achieves superior performance, surpassing state-of-the-art techniques across various multi-modality tracking benchmarks, demonstrating enhanced transferability across modalities.

Conclusion: The sensitivity-regularized fine-tuning framework effectively addresses the plasticity-stability dilemma in multi-modality tracker adaptation by leveraging intrinsic parameter sensitivities, offering a principled approach for cross-modal transfer learning.

Abstract: This paper tackles the critical challenge of optimizing multi-modality trackers by effectively adapting pre-trained models for RGB data. Existing fine-tuning paradigms oscillate between excessive freedom and over-restriction, both leading to a suboptimal plasticity-stability trade-off. To mitigate this dilemma, we propose a novel sensitivity-regularized fine-tuning framework, which delicately refines the learning process by incorporating intrinsic parameter sensitivities. Through a comprehensive investigation of the transition from pre-trained to multi-modal contexts, we identify that parameters sensitive to pivotal foundational patterns and cross-domain shifts are the primary drivers of this issue. Specifically, we first probe the tangent space of pre-trained weights to measure and orient prior sensitivities, dedicated to preserving generalization. Subsequently, we characterize transfer sensitivities during the tuning phase, emphasizing adaptability and stability. By incorporating these sensitivities as unified regularization terms, our method significantly enhances the transferability across modalities. Extensive experiments showcase the superior performance of our method, surpassing current state-of-the-art techniques across various multi-modality tracking benchmarks. The source code and models will be publicly available at https://github.com/zhiwen-xdu/SRTrack.

Method: Benchmark DINOv3 across common medical vision tasks (2D/3D classification, segmentation, registration) on various medical imaging modalities, systematically analyzing scalability by varying model sizes and input image resolutions.

Result: DINOv3 shows impressive performance and establishes a formidable new baseline, even outperforming medical-specific foundation models like BiomedCLIP and CT-Net on several tasks. However, features degrade in highly specialized domains (WSIs, EM, PET), and scaling laws don’t consistently apply - performance doesn’t reliably increase with larger models or finer resolutions.

Conclusion: DINOv3 serves as a strong baseline whose powerful visual features can act as a robust prior for multiple medical tasks, opening promising directions like leveraging its features for multiview consistency in 3D reconstruction.

Abstract: The advent of large-scale vision foundation models, pre-trained on diverse natural images, has marked a paradigm shift in computer vision. However, how the frontier vision foundation models’ efficacies transfer to specialised domains such as medical imaging remains an open question. This report investigates whether DINOv3, a state-of-the-art self-supervised vision transformer (ViT) pre-trained on natural images, can directly serve as a powerful, unified encoder for medical vision tasks without domain-specific fine-tuning. To answer this, we benchmark DINOv3 across common medical vision tasks, including 2D and 3D classification, segmentation, and registration on a wide range of medical imaging modalities. We systematically analyse its scalability by varying model sizes and input image resolutions. Our findings reveal that DINOv3 shows impressive performance and establishes a formidable new baseline. Remarkably, it can even outperform medical-specific foundation models like BiomedCLIP and CT-Net on several tasks, despite being trained solely on natural images. However, we identify clear limitations: The model’s features degrade in scenarios requiring deep domain specialisation, such as in whole-slide images (WSIs), electron microscopy (EM), and positron emission tomography (PET). Furthermore, we observe that DINOv3 does not consistently follow the scaling law in the medical domain. Its performance does not reliably increase with larger models or finer feature resolutions, showing diverse scaling behaviours across tasks. Overall, our work establishes DINOv3 as a strong baseline, whose powerful visual features can serve as a robust prior for multiple medical tasks. This opens promising future directions, such as leveraging its features to enforce multiview consistency in 3D reconstruction.

Method: Introduces Stochastic Generative Diffusion Fusion (SGDF) with multiple generative mechanism for multi-view features, robust to low-quality data. Builds Generative Diffusion Contrastive Network (GDCN) on top of SGDF.

Result: Extensive experiments show GDCN achieves state-of-the-art results in deep multi-view clustering tasks.

Conclusion: The proposed SGDF method effectively addresses low-quality data issues in multi-view fusion, and GDCN demonstrates superior performance in deep MVC applications.

Abstract: In recent years, Multi-View Clustering (MVC) has been significantly advanced under the influence of deep learning. By integrating heterogeneous data from multiple views, MVC enhances clustering analysis, making multi-view fusion critical to clustering performance. However, there is a problem of low-quality data in multi-view fusion. This problem primarily arises from two reasons: 1) Certain views are contaminated by noisy data. 2) Some views suffer from missing data. This paper proposes a novel Stochastic Generative Diffusion Fusion (SGDF) method to address this problem. SGDF leverages a multiple generative mechanism for the multi-view feature of each sample. It is robust to low-quality data. Building on SGDF, we further present the Generative Diffusion Contrastive Network (GDCN). Extensive experiments show that GDCN achieves the state-of-the-art results in deep MVC tasks. The source code is publicly available at https://github.com/HackerHyper/GDCN.

Method: Proposes RIS-FUSION, a cascaded framework that unifies fusion and RIS through joint optimization. Core component is LangGatedFusion module that injects textual features into the fusion backbone to enhance semantic alignment. Also introduces MM-RIS, a large-scale benchmark with 12.5k training and 3.5k testing triplets (infrared-visible image pairs, segmentation masks, and referring expressions).

Result: Extensive experiments show RIS-FUSION achieves state-of-the-art performance, outperforming existing methods by over 11% in mIoU (mean Intersection over Union).

Conclusion: The proposed RIS-FUSION framework successfully bridges text-driven image fusion with referring image segmentation through joint optimization, demonstrating superior performance and providing a new benchmark (MM-RIS) for multimodal referring image segmentation tasks.

Abstract: Text-driven infrared and visible image fusion has gained attention for enabling natural language to guide the fusion process. However, existing methods lack a goal-aligned task to supervise and evaluate how effectively the input text contributes to the fusion outcome. We observe that referring image segmentation (RIS) and text-driven fusion share a common objective: highlighting the object referred to by the text. Motivated by this, we propose RIS-FUSION, a cascaded framework that unifies fusion and RIS through joint optimization. At its core is the LangGatedFusion module, which injects textual features into the fusion backbone to enhance semantic alignment. To support multimodal referring image segmentation task, we introduce MM-RIS, a large-scale benchmark with 12.5k training and 3.5k testing triplets, each consisting of an infrared-visible image pair, a segmentation mask, and a referring expression. Extensive experiments show that RIS-FUSION achieves state-of-the-art performance, outperforming existing methods by over 11% in mIoU. Code and dataset will be released at https://github.com/SijuMa2003/RIS-FUSION.

Method: Leverages inpainting ability of latent diffusion models with adaptive attribute-guidance module that applies gradient correction during reverse denoising to align facial attributes with synthesized target. Supports localized anonymization where users specify unchanged facial regions.

Result: Extensive experiments on CelebA-HQ and FFHQ datasets show method outperforms state-of-the-art approaches while requiring no additional model training.

Conclusion: Proposed framework provides effective portrait anonymization with complete control over process, maintaining utility for computer vision tasks while protecting personal identities.

Abstract: The growing use of portrait images in computer vision highlights the need to protect personal identities. At the same time, anonymized images must remain useful for downstream computer vision tasks. In this work, we propose a unified framework that leverages the inpainting ability of latent diffusion models to generate realistic anonymized images. Unlike prior approaches, we have complete control over the anonymization process by designing an adaptive attribute-guidance module that applies gradient correction during the reverse denoising process, aligning the facial attributes of the generated image with those of the synthesized target image. Our framework also supports localized anonymization, allowing users to specify which facial regions are left unchanged. Extensive experiments conducted on the public CelebA-HQ and FFHQ datasets show that our method outperforms state-of-the-art approaches while requiring no additional model training. The source code is available on our page.

Method: GeLoc3r uses Geometric Consistency Regularization (GCR) during training: generates dense 3D-2D correspondences using ground-truth depth, weights them with a FusionTransformer to learn correspondence importance, computes geometrically-consistent poses via weighted RANSAC, and uses this as a consistency loss to transfer geometric knowledge into the regression network.

Result: Significant improvements over ReLoc3R: 40.45% vs. 34.85% AUC@5° on CO3Dv2 (16% relative improvement), 68.66% vs. 66.70% on RealEstate10K, and 50.45% vs. 49.60% on MegaDepth1500, while maintaining fast inference speed.

Conclusion: GeLoc3r represents a paradigm shift by teaching geometric consistency during training rather than enforcing it at inference, achieving both the speed of regression methods and the geometric understanding of correspondence methods.

Abstract: Prior ReLoc3R achieves breakthrough performance with fast 25ms inference and state-of-the-art regression accuracy, yet our analysis reveals subtle geometric inconsistencies in its internal representations that prevent reaching the precision ceiling of correspondence-based methods like MASt3R (which require 300ms per pair). In this work, we present GeLoc3r, a novel approach to relative camera pose estimation that enhances pose regression methods through Geometric Consistency Regularization (GCR). GeLoc3r overcomes the speed-accuracy dilemma by training regression networks to produce geometrically consistent poses without inference-time geometric computation. During training, GeLoc3r leverages ground-truth depth to generate dense 3D-2D correspondences, weights them using a FusionTransformer that learns correspondence importance, and computes geometrically-consistent poses via weighted RANSAC. This creates a consistency loss that transfers geometric knowledge into the regression network. Unlike FAR method which requires both regression and geometric solving at inference, GeLoc3r only uses the enhanced regression head at test time, maintaining ReLoc3R’s fast speed and approaching MASt3R’s high accuracy. On challenging benchmarks, GeLoc3r consistently outperforms ReLoc3R, achieving significant improvements including 40.45% vs. 34.85% AUC@5° on the CO3Dv2 dataset (16% relative improvement), 68.66% vs. 66.70% AUC@5° on RealEstate10K, and 50.45% vs. 49.60% on MegaDepth1500. By teaching geometric consistency during training rather than enforcing it at inference, GeLoc3r represents a paradigm shift in how neural networks learn camera geometry, achieving both the speed of regression and the geometric understanding of correspondence methods.

Method: Created TennisTV benchmark modeling rallies as temporal-ordered stroke sequences, using automated pipelines for filtering and question generation, covering 8 tasks from stroke to rally level.

Result: Evaluation of 17 MLLMs revealed two key insights: frame-sampling density needs task-specific balancing, and improving temporal grounding is essential for stronger reasoning.

Conclusion: TennisTV provides the first systematic assessment of tennis video understanding and identifies critical areas for MLLM improvement in fast-paced sports analysis.

Abstract: Multimodal large language models (MLLMs) excel at general video understanding but struggle with fast, high-frequency sports like tennis, where rally clips are short yet information-dense. To systematically evaluate MLLMs in this challenging domain, we present TennisTV, the first and most comprehensive benchmark for tennis video understanding. TennisTV models each rally as a temporal-ordered sequence of consecutive stroke events, using automated pipelines for filtering and question generation. It covers 8 tasks from the stroke level to the rally level and includes 2527 human-verified questions. Evaluating 17 representative MLLMs, we provide the first systematic assessment of tennis video understanding. Results yield two key insights: (i) frame-sampling density should be tailored and balanced across tasks, and (ii) improving temporal grounding is essential for stronger reasoning.

Method: BDGF framework with adaptive modality masking for balanced DDPM pre-training, hierarchical diffusion-guided feature extraction using CNN/Mamba/transformer networks, and mutual learning strategy for inter-branch collaboration.

Result: Superior classification performance demonstrated on four multimodal remote sensing datasets.

Conclusion: BDGF effectively addresses modality imbalance and leverages diffusion features to guide complementary multimodal feature extraction for improved land-cover classification.

Abstract: Deep learning-based techniques for the analysis of multimodal remote sensing data have become popular due to their ability to effectively integrate complementary spatial, spectral, and structural information from different sensors. Recently, denoising diffusion probabilistic models (DDPMs) have attracted attention in the remote sensing community due to their powerful ability to capture robust and complex spatial-spectral distributions. However, pre-training multimodal DDPMs may result in modality imbalance, and effectively leveraging diffusion features to guide complementary diversity feature extraction remains an open question. To address these issues, this paper proposes a balanced diffusion-guided fusion (BDGF) framework that leverages multimodal diffusion features to guide a multi-branch network for land-cover classification. Specifically, we propose an adaptive modality masking strategy to encourage the DDPMs to obtain a modality-balanced rather than spectral image-dominated data distribution. Subsequently, these diffusion features hierarchically guide feature extraction among CNN, Mamba, and transformer networks by integrating feature fusion, group channel attention, and cross-attention mechanisms. Finally, a mutual learning strategy is developed to enhance inter-branch collaboration by aligning the probability entropy and feature similarity of individual subnetworks. Extensive experiments on four multimodal remote sensing datasets demonstrate that the proposed method achieves superior classification performance. The code is available at https://github.com/HaoLiu-XDU/BDGF.

Method: GenView++ introduces two synergistic innovations: (1) multi-source adaptive view generation that synthesizes diverse yet semantically coherent views by dynamically modulating generative parameters across image-conditioned, text-conditioned, and image-text-conditioned strategies; (2) quality-driven contrastive learning that assesses each pair’s semantic alignment and diversity to dynamically reweight their training contribution.

Result: Extensive experiments show GenView++ improves MoCov2 by +2.5% on ImageNet linear classification for vision representation learning. For vision-language learning, it raises average zero-shot classification accuracy by +12.31% over CLIP and +5.31% over SLIP across ten datasets, and improves Flickr30k text retrieval R@5 by +3.2%.

Conclusion: GenView++ effectively addresses both construction and learning limitations in contrastive learning through its unified framework, demonstrating significant improvements across vision and vision-language tasks.

Abstract: The success of contrastive learning depends on the construction and utilization of high-quality positive pairs. However, current methods face critical limitations on two fronts: on the construction side, both handcrafted and generative augmentations often suffer from limited diversity and risk semantic corruption; on the learning side, the absence of a quality assessment mechanism leads to suboptimal supervision where all pairs are treated equally. To tackle these challenges, we propose GenView++, a unified framework that addresses both fronts by introducing two synergistic innovations. To improve pair construction, GenView++ introduces a multi-source adaptive view generation mechanism to synthesize diverse yet semantically coherent views by dynamically modulating generative parameters across image-conditioned, text-conditioned, and image-text-conditioned strategies. Second, a quality-driven contrastive learning mechanism assesses each pair’s semantic alignment and diversity to dynamically reweight their training contribution, prioritizing high-quality pairs while suppressing redundant or misaligned pairs. Extensive experiments demonstrate the effectiveness of GenView++ across both vision and vision-language tasks. For vision representation learning, it improves MoCov2 by +2.5% on ImageNet linear classification. For vision-language learning, it raises the average zero-shot classification accuracy by +12.31% over CLIP and +5.31% over SLIP across ten datasets, and further improves Flickr30k text retrieval R@5 by +3.2%.

Method: Leverages IP-Adapter’s automatically generated masks during inference to segment subjects, then applies these masks in a second pass to restrict image tokens to the subject area, allowing text prompts to attend to the background.

Result: The method produces images that accurately depict personalized subjects while definitively matching text prompts, showing high prompt and source image alignment compared to other test-time personalization methods.

Conclusion: The proposed masking approach effectively addresses the text-prompt ignoring problem in personalization, validated by user studies showing end-user appreciation for the improved results.

Abstract: Personalizing diffusion models allows users to generate new images that incorporate a given subject, allowing more control than a text prompt. These models often suffer somewhat when they end up just recreating the subject image and ignoring the text prompt. We observe that one popular method for personalization, IP-Adapter, automatically generates masks that segment the subject from the background during inference. We propose to use this automatically generated mask on a second pass to mask the image tokens, thus restricting them to the subject, not the background, allowing the text prompt to attend to the rest of the image. For text prompts describing locations and places, this produces images that accurately depict the subject while definitively matching the prompt. We compare our method to a few other test time personalization methods, and find our method displays high prompt and source image alignment. We also perform a user study to validate whether end users would appreciate our method. Code available at https://github.com/jamesBaker361/monkey

Method: Systematically classifies image-to-video transfer learning techniques into two main categories (frozen features and adapted features) with numerous subcategories, and analyzes their applications across various video-text learning tasks from fine-grained to coarse-grained settings.

Result: Provides comprehensive experimental analysis of different transfer learning paradigms’ efficacy on downstream video understanding tasks, establishing a structured roadmap for advancing video-text learning based on existing image-language foundation models.

Conclusion: Identifies prevailing challenges and highlights promising future research directions for image-to-video transfer learning, offering a comprehensive overview to inspire advancement in this rapidly evolving domain.

Abstract: Image-Language Foundation Models (ILFMs) have demonstrated remarkable success in vision-language understanding, providing transferable multimodal representations that generalize across diverse downstream image-based tasks. The advancement of video-text research has spurred growing interest in extending image-based models to the video domain. This paradigm, termed as image-to-video transfer learning, effectively mitigates the substantial data and computational demands compared to training video-language models from scratch while achieves comparable or even stronger model performance. This survey provides the first comprehensive review of this emerging field, which begins by summarizing the widely used ILFMs and their capabilities. We then systematically classify existing image-to-video transfer learning techniques into two broad root categories (frozen features and adapted features), along with numerous fine-grained subcategories, based on the paradigm for transferring image understanding capability to video tasks. Building upon the task-specific nature of image-to-video transfer, this survey methodically elaborates these strategies and details their applications across a spectrum of video-text learning tasks, ranging from fine-grained settings (e.g., spatio-temporal video grounding) to coarse-grained ones (e.g., video question answering). We further present a detailed experimental analysis to investigate the efficacy of different image-to-video transfer learning paradigms on a range of downstream video understanding tasks. Finally, we identify prevailing challenges and highlight promising directions for future research. By offering a comprehensive and structured overview, this survey aims to establish a structured roadmap for advancing video-text learning based on existing ILFM, and to inspire future research directions in this rapidly evolving domain. Github repository is available.

Method: Introduces SketchSem3D benchmark with two subsets (Sketch-based SemanticKITTI and KITTI-360), and proposes Cylinder Mamba Diffusion (CymbaDiff) model that imposes structured spatial ordering, captures cylindrical continuity and vertical hierarchy.

Result: CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization on SketchSem3D benchmark.

Conclusion: SketchSem3D enables standardized evaluation for 3D outdoor scene generation, while CymbaDiff significantly enhances spatial coherence through structured spatial modeling.

Abstract: Outdoor 3D semantic scene generation produces realistic and semantically rich environments for applications such as urban simulation and autonomous driving. However, advances in this direction are constrained by the absence of publicly available, well-annotated datasets. We introduce SketchSem3D, the first large-scale benchmark for generating 3D outdoor semantic scenes from abstract freehand sketches and pseudo-labeled annotations of satellite images. SketchSem3D includes two subsets, Sketch-based SemanticKITTI and Sketch-based KITTI-360 (containing LiDAR voxels along with their corresponding sketches and annotated satellite images), to enable standardized, rigorous, and diverse evaluations. We also propose Cylinder Mamba Diffusion (CymbaDiff) that significantly enhances spatial coherence in outdoor 3D scene generation. CymbaDiff imposes structured spatial ordering, explicitly captures cylindrical continuity and vertical hierarchy, and preserves both physical neighborhood relationships and global context within the generated scenes. Extensive experiments on SketchSem3D demonstrate that CymbaDiff achieves superior semantic consistency, spatial realism, and cross-dataset generalization. The code and dataset will be available at https://github.com/Lillian-research-hub/CymbaDiff

Method: 1) Create ground-truth masks by fusing outputs from 7 keypoint detectors and 2 edge detectors to extract diverse visual cues. 2) Train a lightweight ESPNet model using these fused masks as labels, enabling semantic focus on images while producing dense keypoints.

Result: DeepDetect surpasses other detectors on Oxford, HPatches, and Middlebury datasets with maximum values of: 0.5143 (average keypoint density), 0.9582 (average repeatability), 338,118 (correct matches), and 842,045 (voxels in stereo 3D reconstruction).

Conclusion: DeepDetect successfully unifies classical detector strengths through deep learning, creating an intelligent, all-in-one dense detector that adapts well to diverse and visually degraded conditions while maintaining high performance metrics.

Abstract: Keypoint detection is the foundation of many computer vision tasks, including image registration, structure-from-motion, 3D reconstruction, visual odometry, and SLAM. Traditional detectors (SIFT, ORB, BRISK, FAST, etc.) and learning-based methods (SuperPoint, R2D2, QuadNet, LIFT, etc.) have shown strong performance gains yet suffer from key limitations: sensitivity to photometric changes, low keypoint density and repeatability, limited adaptability to challenging scenes, and lack of semantic understanding, often failing to prioritize visually important regions. We present DeepDetect, an intelligent, all-in-one, dense detector that unifies the strengths of classical detectors using deep learning. Firstly, we create ground-truth masks by fusing outputs of 7 keypoint and 2 edge detectors, extracting diverse visual cues from corners and blobs to prominent edges and textures in the images. Afterwards, a lightweight and efficient model: ESPNet, is trained using fused masks as labels, enabling DeepDetect to focus semantically on images while producing highly dense keypoints, that are adaptable to diverse and visually degraded conditions. Evaluations on Oxford, HPatches, and Middlebury datasets demonstrate that DeepDetect surpasses other detectors achieving maximum values of 0.5143 (average keypoint density), 0.9582 (average repeatability), 338,118 (correct matches), and 842,045 (voxels in stereo 3D reconstruction).

Method: DINO-CV: Self-supervised cross-view pre-training framework based on knowledge distillation using DEMs from high-resolution airborne LiDAR. Learns invariant geometric/geomorphic features across DEM-derived views (Multi-directional Hillshade and Visualization for Archaeological Topography) to address vegetation occlusion and data scarcity.

Result: Achieved 68.6% mIoU on test areas at Budj Bim Cultural Landscape (UNESCO site). Maintained 63.8% mIoU when fine-tuned with only 10% labeled data, demonstrating data-efficient performance.

Conclusion: Self-supervised learning on high-resolution DEM derivatives enables large-scale automated mapping of cultural heritage in complex vegetated environments. Approach offers scalable solution for environmental monitoring and heritage preservation in inaccessible/sensitive regions beyond archaeology.

Abstract: Historic dry-stone walls hold significant cultural and environmental importance, serving as historical markers and contributing to ecosystem preservation and wildfire management during dry seasons in Australia. However, many of these stone structures in remote or vegetated landscapes remain undocumented due to limited accessibility and the high cost of manual mapping. Deep learning-based segmentation offers a scalable approach for automated mapping of such features, but challenges remain: 1.the visual occlusion of low-lying dry-stone walls by dense vegetation and 2.the scarcity of labeled training data. This study presents DINO-CV, a self-supervised cross-view pre-training framework based on knowledge distillation, designed for accurate and data-efficient mapping of dry-stone walls using Digital Elevation Models (DEMs) derived from high-resolution airborne LiDAR. By learning invariant geometric and geomorphic features across DEM-derived views, (i.e., Multi-directional Hillshade and Visualization for Archaeological Topography), DINO-CV addresses the occlusion by vegetation and data scarcity challenges. Applied to the Budj Bim Cultural Landscape at Victoria, Australia, a UNESCO World Heritage site, the approach achieves a mean Intersection over Union (mIoU) of 68.6% on test areas and maintains 63.8% mIoU when fine-tuned with only 10% labeled data. These results demonstrate the potential of self-supervised learning on high-resolution DEM derivatives for large-scale, automated mapping of cultural heritage features in complex and vegetated environments. Beyond archaeology, this approach offers a scalable solution for environmental monitoring and heritage preservation across inaccessible or environmentally sensitive regions.

Method: The authors introduce a framework for dimensionality reduction and use it to trace the evolution of DR techniques across the remote sensing data value chain. They synthesize trends and offer future perspectives.

Result: The paper organizes the diverse DR landscape in remote sensing, identifies a shift from single-task models to unified representations, and provides a framework for understanding DR evolution in this domain.

Conclusion: The future of DR in remote sensing involves robust and interpretable techniques, with potential for bridging classical DR methods with modern representation learning in the foundation model era.

Abstract: Earth observation involves collecting, analyzing, and processing an ever-growing mass of data. This planetary data is crucial for addressing relevant societal, economic, and environmental challenges, ranging from environmental monitoring to urban planning and disaster management. However, its high dimensionality entails significant feature redundancy and computational overhead limiting the effectiveness of machine learning models. Dimensionality reduction (DR) techniques, specifically feature extraction, address these challenges by preserving essential data properties while reducing redundancy and enhancing tasks in Remote Sensing (RS). The landscape of DR for RS is a diverse, disorganized, and rapidly evolving field. We offer a practical guide for this landscape by introducing a framework of DR. Using this framework, we trace the evolution of DR across the data value chain in RS. Finally, we synthesize these trends and offer perspectives for the future of DR in RS by first characterizing this shift from single-task models to unified representations, then identifying two perspectives in the foundation model era: the need for robust and interpretable DR and the potential of bridging classical DR with modern representation learning.

Method: Created POVFNDB benchmark with 10 tasks, 15 evaluation dimensions, and 36,240 human-annotated QA pairs. Developed POVFND-CoT framework to construct process-oriented chain-of-thought data, then fine-tuned Qwen2.5VL-7B-Instruct model on this data.

Result: Comprehensive evaluation of proprietary and open-source MLLMs on the benchmark. Fine-tuned Qwen2.5VL-7B-Instruct achieved state-of-the-art performance on video fake news detection tasks.

Conclusion: POVFNDB provides a systematic process-oriented benchmark for evaluating MLLMs’ perception, understanding, and reasoning in video fake news detection, with the proposed fine-tuning approach establishing strong baseline performance.

Abstract: The advent of multi-modal large language models (MLLMs) has greatly advanced research on video fake news detection (VFND) tasks. Existing benchmarks typically focus on the detection accuracy, while failing to provide fine-grained assessments for the entire detection process. To address these limitations, we introduce {POVFNDB (Process-oriented Video Fake News Detection Benchmark)}, a process-oriented benchmark comprising 10 tasks designed to systematically evaluate MLLMs’ perception, understanding, and reasoning capabilities in VFND. This benchmark contains \textit{36,240} human-annotated question-answer (QA) in structured or open-ended formats, spanning 15 distinct evaluation dimensions that characterize different aspects of the video fake news detection process. Using POVFNDB, we conduct comprehensive evaluations on both proprietary and open-source MLLMs. Moreover, we establish a strong benchmark baseline by fine-tuning Qwen2.5VL-7B-Instruct on process-oriented chain-of-thought data constructed with our proposed POVFND-CoT framework, achieving state-of-the-art performance on VFND.

Method: Comparative analysis of UNet-based architectures with different backbones: ResNet, Transformer-based, and State-space (Mamba) backbones initialized with pretrained weights. Attention mechanisms were introduced, with CBAM found optimal. ResNetUNet3+ with CBAM was the proposed model.

Result: ResNet-based models showed superior sample efficiency despite modern architectures’ theoretical advantages. ResNetUNet3+ with CBAM achieved highest performance: Dice 0.755, IoU 0.662, lowest HD95 of 77.911. While mean Dice improvement wasn’t statistically significant (p > 0.05), the model showed greater stability (lower std) and higher specificity (0.926).

Conclusion: Classical ResNet architectures enhanced with modern attention modules (CBAM) provide robust, statistically comparable alternatives to emerging methods for liver tumor segmentation. CNN inductive biases remain advantageous for generalizing on limited medical data, offering stable clinical practice solutions.

Abstract: Segmentation of liver structures in multi-phase contrast-enhanced computed tomography (CECT) plays a crucial role in computer-aided diagnosis and treatment planning. In this study, we investigate the performance of UNet-based architectures for liver tumor segmentation, evaluating ResNet, Transformer-based, and State-space (Mamba) backbones initialized with pretrained weights. Our comparative analysis reveals that despite the theoretical advantages of modern architectures in modeling long-range dependencies, ResNet-based models demonstrated superior sample efficiency on this dataset. This suggests that the inherent inductive biases of Convolutional Neural Networks (CNNs) remain advantageous for generalizing on limited medical data compared to data-hungry alternatives. To further improve segmentation quality, we introduce attention mechanisms into the backbone, finding that the Convolutional Block Attention Module (CBAM) yields the optimal configuration. The ResNetUNet3+ with CBAM achieved the highest nominal performance with a Dice score of 0.755 and IoU of 0.662, while also delivering the most precise boundary delineation (lowest HD95 of 77.911). Critically, while statistical testing indicated that the improvement in mean Dice score was not significant (p > 0.05) compared to the baseline, the proposed model exhibited greater stability (lower standard deviation) and higher specificity (0.926). These findings demonstrate that classical ResNet architectures, when enhanced with modern attention modules, provide a robust and statistically comparable alternative to emerging methods, offering a stable direction for liver tumor segmentation in clinical practice.

Method: Evaluated three GFMs (Prithvi 2.0, Clay V1.5, DOFA, and UViT) against traditional models (TransNorm, U-Net, Attention U-Net) using PlanetScope, Sentinel-1, and Sentinel-2 data. Conducted leave-one-region-out cross-validation across five regions and 19 sites, plus few-shot experiments with limited training data.

Result: GFMs show competitive performance with only 2-5% variation between best and worst models. Clay performs best on PlanetScope (0.79 mIoU) and Sentinel-2 (0.70), while Prithvi leads on Sentinel-1 (0.57). Clay shows 4% improvement over U-Net in cross-validation, better detail retention, and achieves 0.64 mIoU with just five training images. Clay is 3x faster than Prithvi and 2x faster than DOFA due to smaller model size (26M vs 650M/410M parameters).

Conclusion: GFMs offer small to moderate improvements in flood mapping accuracy at lower computational cost and labeling effort compared to traditional U-Net, with Clay emerging as the most practical choice due to its balance of performance, efficiency, and data efficiency.

Abstract: Geo-Foundational Models (GFMs) enable fast and reliable extraction of spatiotemporal information from satellite imagery, improving flood inundation mapping by leveraging location and time embeddings. Despite their potential, it remains unclear whether GFMs outperform traditional models like U-Net. A systematic comparison across sensors and data availability scenarios is still lacking, which is an essential step to guide end-users in model selection. To address this, we evaluate three GFMs, Prithvi 2.0, Clay V1.5, DOFA, and UViT (a Prithvi variant), against TransNorm, U-Net, and Attention U-Net using PlanetScope, Sentinel-1, and Sentinel-2. We observe competitive performance among all GFMs, with only 2-5% variation between the best and worst models across sensors. Clay outperforms others on PlanetScope (0.79 mIoU) and Sentinel-2 (0.70), while Prithvi leads on Sentinel-1 (0.57). In leave-one-region-out cross-validation across five regions, Clay shows slightly better performance across all sensors (mIoU: 0.72(0.04), 0.66(0.07), 0.51(0.08)) compared to Prithvi (0.70(0.05), 0.64(0.09), 0.49(0.13)) and DOFA (0.67(0.07), 0.64(0.04), 0.49(0.09)) for PlanetScope, Sentinel-2, and Sentinel-1, respectively. Across all 19 sites, leave-one-region-out cross-validation reveals a 4% improvement by Clay compared to U-Net. Visual inspection highlights Clay’s superior ability to retain fine details. Few-shot experiments show Clay achieves 0.64 mIoU on PlanetScope with just five training images, outperforming Prithvi (0.24) and DOFA (0.35). In terms of computational time, Clay is a better choice due to its smaller model size (26M parameters), making it ~3x faster than Prithvi (650M) and 2x faster than DOFA (410M). Contrary to previous findings, our results suggest GFMs offer small to moderate improvements in flood mapping accuracy at lower computational cost and labeling effort compared to traditional U-Net.

Method: A part-aware bottom-up group reasoning framework that: 1) detects individuals and enhances their features using part-aware cues, 2) infers group configuration by associating individuals via similarity-based reasoning that considers both spatial relations and subtle social cues signaling interactions.

Result: Outperforms prior methods on the NVI dataset, achieving new state-of-the-art results. Additional validation on the Café dataset demonstrates generalizability to group activity understanding.

Conclusion: The proposed part-aware bottom-up approach effectively captures fine-grained social cues and explicitly models interpersonal relations, leading to more accurate social interaction detection and group inference compared to existing holistic methods.

Abstract: Social interactions often emerge from subtle, fine-grained cues such as facial expressions, gaze, and gestures. However, existing methods for social interaction detection overlook such nuanced cues and primarily rely on holistic representations of individuals. Moreover, they directly detect social groups without explicitly modeling the underlying interactions between individuals. These drawbacks limit their ability to capture localized social signals and introduce ambiguity when group configurations should be inferred from social interactions grounded in nuanced cues. In this work, we propose a part-aware bottom-up group reasoning framework for fine-grained social interaction detection. The proposed method infers social groups and their interactions using body part features and their interpersonal relations. Our model first detects individuals and enhances their features using part-aware cues, and then infers group configuration by associating individuals via similarity-based reasoning, which considers not only spatial relations but also subtle social cues that signal interactions, leading to more accurate group inference. Experiments on the NVI dataset demonstrate that our method outperforms prior methods, achieving the new state of the art, while additional results on the Café dataset further validate its generalizability to group activity understanding.

Method: Proposes Semantic-Consistent Bidirectional Contrastive Hashing (SCBCH) with two modules: 1) Cross-modal Semantic-Consistent Classification (CSCC) estimates sample reliability using cross-modal semantic consistency to reduce noisy label impact; 2) Bidirectional Soft Contrastive Hashing (BSCH) dynamically generates soft contrastive sample pairs based on multi-label semantic overlap for adaptive contrastive learning.

Result: Extensive experiments on four widely-used cross-modal retrieval benchmarks show the method effectively handles noisy multi-label conditions and consistently outperforms state-of-the-art approaches.

Conclusion: SCBCH provides a robust solution for cross-modal retrieval in noisy multi-label scenarios by addressing both label noise and partial semantic overlap issues through semantic consistency and adaptive contrastive learning.

Abstract: Cross-modal hashing (CMH) facilitates efficient retrieval across different modalities (e.g., image and text) by encoding data into compact binary representations. While recent methods have achieved remarkable performance, they often rely heavily on fully annotated datasets, which are costly and labor-intensive to obtain. In real-world scenarios, particularly in multi-label datasets, label noise is prevalent and severely degrades retrieval performance. Moreover, existing CMH approaches typically overlook the partial semantic overlaps inherent in multi-label data, limiting their robustness and generalization. To tackle these challenges, we propose a novel framework named Semantic-Consistent Bidirectional Contrastive Hashing (SCBCH). The framework comprises two complementary modules: (1) Cross-modal Semantic-Consistent Classification (CSCC), which leverages cross-modal semantic consistency to estimate sample reliability and reduce the impact of noisy labels; (2) Bidirectional Soft Contrastive Hashing (BSCH), which dynamically generates soft contrastive sample pairs based on multi-label semantic overlap, enabling adaptive contrastive learning between semantically similar and dissimilar samples across modalities. Extensive experiments on four widely-used cross-modal retrieval benchmarks validate the effectiveness and robustness of our method, consistently outperforming state-of-the-art approaches under noisy multi-label conditions.

Method: Adapted Denoising Diffusion Probabilistic Models and Autoregressive Causal Transformers for generative shape modeling and completion tasks, with thorough quantitative evaluation including baseline discriminative model and extensive ablation study.

Result: 1) Diffusion model with continuous latents outperforms both discriminative model and autoregressive approach, achieving state-of-the-art performance on multi-modal shape completion from single noisy depth images under realistic conditions. 2) When compared on same discrete latent space, autoregressive model can match or exceed diffusion performance on these tasks.

Conclusion: Both diffusion models and autoregressive transformers show strong performance for 3D shape tasks, with diffusion excelling on continuous representations and autoregressive models performing competitively on discrete latent spaces, providing guidance for model selection based on representation type.

Abstract: While generative models have seen significant adoption across a wide range of data modalities, including 3D data, a consensus on which model is best suited for which task has yet to be reached. Further, conditional information such as text and images to steer the generation process are frequently employed, whereas others, like partial 3D data, have not been thoroughly evaluated. In this work, we compare two of the most promising generative models–Denoising Diffusion Probabilistic Models and Autoregressive Causal Transformers–which we adapt for the tasks of generative shape modeling and completion. We conduct a thorough quantitative evaluation and comparison of both tasks, including a baseline discriminative model and an extensive ablation study. Our results show that (1) the diffusion model with continuous latents outperforms both the discriminative model and the autoregressive approach and delivers state-of-the-art performance on multi-modal shape completion from a single, noisy depth image under realistic conditions and (2) when compared on the same discrete latent space, the autoregressive model can match or exceed diffusion performance on these tasks.

Method: Proposes Current-Centric Contextual 3D Fusion (C3DFusion) module that generates hidden region-aware 3D feature geometry by explicitly aligning 3D-lifted point features from current and historical frames. Uses two complementary techniques: historical context blurring (suppresses noise from inaccurately warped historical features by attenuating their scale) and current-centric feature densification (enhances current point features by increasing their volumetric contribution).

Result: Significantly outperforms state-of-the-art methods on SemanticKITTI and SSCBench-KITTI-360 datasets. Shows robust generalization with notable performance gains when applied to other baseline models.

Conclusion: C3DFusion effectively addresses the limitation of reconstructing out-of-frame areas in 3D SSC by leveraging temporal information through explicit feature alignment and complementary fusion techniques, demonstrating strong effectiveness and generalization capability.

Abstract: Recent camera-based 3D semantic scene completion (SSC) methods have increasingly explored leveraging temporal cues to enrich the features of the current frame. However, while these approaches primarily focus on enhancing in-frame regions, they often struggle to reconstruct critical out-of-frame areas near the sides of the ego-vehicle, although previous frames commonly contain valuable contextual information about these unseen regions. To address this limitation, we propose the Current-Centric Contextual 3D Fusion (C3DFusion) module, which generates hidden region-aware 3D feature geometry by explicitly aligning 3D-lifted point features from both current and historical frames. C3DFusion performs enhanced temporal fusion through two complementary techniques-historical context blurring and current-centric feature densification-which suppress noise from inaccurately warped historical point features by attenuating their scale, and enhance current point features by increasing their volumetric contribution. Simply integrated into standard SSC architectures, C3DFusion demonstrates strong effectiveness, significantly outperforming state-of-the-art methods on the SemanticKITTI and SSCBench-KITTI-360 datasets. Furthermore, it exhibits robust generalization, achieving notable performance gains when applied to other baseline models.

Method: Created ManipBench with 450K+ manipulated images from 25 state-of-the-art editing models across 12 categories, with 100K images annotated with bounding boxes, judgment cues, and textual explanations. Developed ManipShield using Multimodal Large Language Model with contrastive LoRA fine-tuning and task-specific decoders for unified detection, localization, and explanation.

Result: ManipShield achieves state-of-the-art performance on ManipBench and public datasets, exhibits strong generalization to unseen manipulation models, and provides interpretable detection with bounding boxes and explanations.

Conclusion: ManipBench addresses limitations of existing benchmarks with comprehensive coverage and interpretability, while ManipShield demonstrates effective unified manipulation detection, localization, and explanation capabilities with strong generalization.

Abstract: With the rapid advancement of generative models, powerful image editing methods now enable diverse and highly realistic image manipulations that far surpass traditional deepfake techniques, posing new challenges for manipulation detection. Existing image manipulation detection and localization (IMDL) benchmarks suffer from limited content diversity, narrow generative-model coverage, and insufficient interpretability, which hinders the generalization and explanation capabilities of current manipulation detection methods. To address these limitations, we introduce \textbf{ManipBench}, a large-scale benchmark for image manipulation detection and localization focusing on AI-edited images. ManipBench contains over 450K manipulated images produced by 25 state-of-the-art image editing models across 12 manipulation categories, among which 100K images are further annotated with bounding boxes, judgment cues, and textual explanations to support interpretable detection. Building upon ManipBench, we propose \textbf{ManipShield}, an all-in-one model based on a Multimodal Large Language Model (MLLM) that leverages contrastive LoRA fine-tuning and task-specific decoders to achieve unified image manipulation detection, localization, and explanation. Extensive experiments on ManipBench and several public datasets demonstrate that ManipShield achieves state-of-the-art performance and exhibits strong generality to unseen manipulation models. Both ManipBench and ManipShield will be released upon publication.

Method: Four-stage automated pipeline: (1) meta-annotation collection (object/frame/scene captions), (2) scene graph construction with relation correction for proximal object relations, (3) discriminative object referring for exclusive compact descriptions, (4) multi-task data generation for diverse dialogues. Uses rule-based constraints with 2D MLLMs and LLMs for controllable, scalable generation.

Result: Produces Disc3D dataset with over 2 million samples across 25K hybrid 3D scenes, covering scene/view/object captioning, visual grounding, and five object-centric QA tasks. Training with Disc3D yields consistent, significant improvements on public benchmarks and Disc3D-QA tasks.

Conclusion: The fully automated pipeline enables cost-effective generation of unambiguous, high-quality 3D dialogue data, systematically addressing dataset limitations and advancing 3D MLLM capabilities without human intervention.

Abstract: 3D Multi-modal Large Language Models (MLLMs) still lag behind their 2D peers, largely because large-scale, high-quality 3D scene-dialogue datasets remain scarce. Prior efforts hinge on expensive human annotation and leave two key ambiguities unresolved: viewpoint ambiguity, where spatial language presumes unknown camera poses, and object referring ambiguity, where non-exclusive descriptions blur the line between targets and distractors. We therefore present a fully automated pipeline that converts raw 3D scans into unambiguous, high-quality dialogue data at a fraction of the previous cost. By synergizing rule-based constraints with 2D MLLMs and LLMs, the pipeline enables controllable, scalable generation without human intervention. The pipeline comprises four stages: (1) meta-annotation collection harvesting object-, frame-, and scene-level captions, (2) scene graph construction with relation correction to capture proximal object relations, (3) discriminative object referring that generates exclusive and compact descriptions, and (4) multi-task data generation synthesizing diverse dialogues. Our pipeline systematically mitigates inherent flaws in source datasets and produces the final Disc3D dataset, over 2 million samples in 25K hybrid 3D scenes, spanning scene, view, and object captioning, visual grounding, and five object-centric QA tasks. Extensive experiments demonstrate that training with Disc3D yields consistent, significant improvements on both public benchmarks and our multifaceted Disc3D-QA tasks. Code, data, and models will be publicly available.

Method: Developed Vidi2 model with fine-grained spatio-temporal grounding (identifying timestamps and bounding boxes), temporal retrieval, and video QA capabilities. Introduced VUE-STG benchmark for STG evaluation and upgraded VUE-TR to VUE-TR-V2. Later released Vidi2.5 with enhanced STG and Vidi2.5-Think for complex plot reasoning, with VUE-PLOT benchmark for evaluation.

Result: Vidi2 substantially outperforms leading proprietary systems (Gemini 3 Pro Preview, GPT-5) on VUE-TR-V2 and VUE-STG benchmarks. Vidi2.5 offers stronger STG capability and better TR/Video QA performance. Vidi2.5-Think outperforms Gemini 3 Pro Preview on fine-grained character understanding with comparable performance on complex plot reasoning. Demonstrated effectiveness on real-world video editing planning.

Conclusion: The Vidi models represent significant advancement in video understanding capabilities, particularly in spatio-temporal grounding and multimodal reasoning, outperforming state-of-the-art proprietary systems while introducing comprehensive benchmarks for evaluation across multiple video understanding tasks.

Abstract: Video has emerged as the primary medium for communication and creativity on the Internet, driving strong demand for scalable, high-quality video production. Vidi models continue to evolve toward next-generation video creation and have achieved state-of-the-art performance in multimodal temporal retrieval (TR). In its second release, Vidi2 advances video understanding with fine-grained spatio-temporal grounding (STG) and extends its capability to video question answering (Video QA), enabling comprehensive multimodal reasoning. Given a text query, Vidi2 can identify not only the corresponding timestamps but also the bounding boxes of target objects within the output time ranges. To enable comprehensive evaluation of STG, we introduce a new benchmark, VUE-STG, which offers critical improvements over existing STG datasets. In addition, we upgrade the previous VUE-TR benchmark to VUE-TR-V2, achieving a more balanced duration and query distribution. Remarkably, the Vidi2 model substantially outperforms leading proprietary systems, such as Gemini 3 Pro Preview and GPT-5, on both VUE-TR-V2 and VUE-STG, while achieving competitive results with popular open-source models with similar scale on video QA benchmarks. The latest Vidi2.5 offers significantly stronger STG capability and slightly better TR and Video QA performance over Vidi2. This update also introduces a Vidi2.5-Think model to handle plot understanding with complex plot reasoning. To comprehensively evaluate the performance of plot understanding, we propose VUE-PLOT benchmark with two tracks, Character and Reasoning. Notably, Vidi2.5-Think outperforms Gemini 3 Pro Preview on fine-grained character understanding with comparable performance on complex plot reasoning. Furthermore, we demonstrate the effectiveness of Vidi2.5 on a challenging real-world application, video editing planning.

Method: Introduces Visual LIF (VLIF) neuron to overcome spatial contextual limitations, plus Spiking Decomposition and Enhancement Module and lightweight Spiking Multi-scale Unit for hierarchical multi-scale representation learning.

Result: Extensive experiments across five benchmark deraining datasets show significant outperformance over state-of-the-art SNN-based deraining methods with only 13% of their energy consumption.

Conclusion: Establishes foundation for deploying SNNs in high-performance, energy-efficient low-level vision tasks, demonstrating practical viability of SNNs for real-world computer vision applications.

Abstract: Biologically plausible and energy-efficient frameworks such as Spiking Neural Networks (SNNs) have not been sufficiently explored in low-level vision tasks. Taking image deraining as an example, this study addresses the representation of the inherent high-pass characteristics of spiking neurons, specifically in image deraining and innovatively proposes the Visual LIF (VLIF) neuron, overcoming the obstacle of lacking spatial contextual understanding present in traditional spiking neurons. To tackle the limitation of frequency-domain saturation inherent in conventional spiking neurons, we leverage the proposed VLIF to introduce the Spiking Decomposition and Enhancement Module and the lightweight Spiking Multi-scale Unit for hierarchical multi-scale representation learning. Extensive experiments across five benchmark deraining datasets demonstrate that our approach significantly outperforms state-of-the-art SNN-based deraining methods, achieving this superior performance with only 13% of their energy consumption. These findings establish a solid foundation for deploying SNNs in high-performance, energy-efficient low-level vision tasks.

Method: Develops Context-measure based on a probabilistic pixel-aware correlation framework that incorporates spatial dependencies and pixel-wise camouflage quantification to better align with human perception.

Result: Extensive experiments across three challenging camouflaged object segmentation datasets show Context-measure delivers more reliability than existing context-independent metrics.

Conclusion: Context-measure provides a foundational evaluation benchmark for various computer vision applications involving camouflaged patterns in agricultural, industrial, and medical scenarios.

Abstract: Camouflage is primarily context-dependent yet current metrics for camouflaged scenarios overlook this critical factor. Instead, these metrics are originally designed for evaluating general or salient objects, with an inherent assumption of uncorrelated spatial context. In this paper, we propose a new contextualized evaluation paradigm, Context-measure, built upon a probabilistic pixel-aware correlation framework. By incorporating spatial dependencies and pixel-wise camouflage quantification, our measure better aligns with human perception. Extensive experiments across three challenging camouflaged object segmentation datasets show that Context-measure delivers more reliability than existing context-independent metrics. Our measure can provide a foundational evaluation benchmark for various computer vision applications involving camouflaged patterns, such as agricultural, industrial, and medical scenarios. Code is available at https://github.com/pursuitxi/Context-measure.

Method: Proposes a pose hypothesis selection algorithm that integrates predictions from foundation models, jointly refines them through learned render-and-compare for spatial/temporal/pixel alignment, and reasons about intricate contacts with physical constraints.

Result: Outperforms prior methods by 38% on in-distribution datasets and 36% on unseen datasets in reconstruction error, generalizes beyond training categories, and works zero-shot on in-the-wild internet videos.

Conclusion: CARI4D enables category-agnostic 4D human-object interaction reconstruction from monocular RGB videos with superior performance and generalization, making it applicable to real-world scenarios without category restrictions.

Abstract: Accurate capture of human-object interaction from ubiquitous sensors like RGB cameras is important for applications in human understanding, gaming, and robot learning. However, inferring 4D interactions from a single RGB view is highly challenging due to the unknown object and human information, depth ambiguity, occlusion, and complex motion, which hinder consistent 3D and temporal reconstruction. Previous methods simplify the setup by assuming ground truth object template or constraining to a limited set of object categories. We present CARI4D, the first category-agnostic method that reconstructs spatially and temporarily consistent 4D human-object interaction at metric scale from monocular RGB videos. To this end, we propose a pose hypothesis selection algorithm that robustly integrates the individual predictions from foundation models, jointly refine them through a learned render-and-compare paradigm to ensure spatial, temporal and pixel alignment, and finally reasoning about intricate contacts for further refinement satisfying physical constraints. Experiments show that our method outperforms prior art by 38% on in-distribution dataset and 36% on unseen dataset in terms of reconstruction error. Our model generalizes beyond the training categories and thus can be applied zero-shot to in-the-wild internet videos. Our code and pretrained models will be publicly released.

Method: Three core innovations: (1) Phase-Aware Cross-Attention (PACA) decomposes prompts into global base and temporally-anchored phase blocks for precise temporal-semantic alignment; (2) Progressive Audio-Visual Alignment aligns modality influence with hierarchical feature learning; (3) Two-stage training strategy establishes audio-visual correspondence first, then injects action control through fine-tuning.

Result: Extensive experiments demonstrate that ActAvatar significantly outperforms state-of-the-art methods in both action control and visual quality.

Conclusion: ActAvatar provides a framework for phase-level precision action control through textual guidance, effectively addressing key challenges in talking avatar generation while maintaining audio-visual alignment and text-following capabilities.

Abstract: Despite significant advances in talking avatar generation, existing methods face critical challenges: insufficient text-following capability for diverse actions, lack of temporal alignment between actions and audio content, and dependency on additional control signals such as pose skeletons. We present ActAvatar, a framework that achieves phase-level precision in action control through textual guidance by capturing both action semantics and temporal context. Our approach introduces three core innovations: (1) Phase-Aware Cross-Attention (PACA), which decomposes prompts into a global base block and temporally-anchored phase blocks, enabling the model to concentrate on phase-relevant tokens for precise temporal-semantic alignment; (2) Progressive Audio-Visual Alignment, which aligns modality influence with the hierarchical feature learning process-early layers prioritize text for establishing action structure while deeper layers emphasize audio for refining lip movements, preventing modality interference; (3) A two-stage training strategy that first establishes robust audio-visual correspondence on diverse data, then injects action control through fine-tuning on structured annotations, maintaining both audio-visual alignment and the model’s text-following capabilities. Extensive experiments demonstrate that ActAvatar significantly outperforms state-of-the-art methods in both action control and visual quality.

Method: Two-stage framework: 1) Debate-Enhanced Pseudo Labeling with adaptive entropy-driven point sampling and multi-agent debate mechanism to enhance SAM for COD; 2) FADeNet with progressive fusion of multi-level frequency-aware features and dynamic reweighting of supervision strength to alleviate scribble bias.

Result: Significantly narrows the gap between weakly and fully supervised COD, achieving state-of-the-art performance on multiple benchmarks.

Conclusion: D³ETOR effectively addresses key limitations in WSCOD by jointly exploiting pseudo masks and scribble semantics through innovative debate mechanisms and frequency-aware debiasing, advancing the field toward practical weakly-supervised solutions.

Abstract: Weakly-Supervised Camouflaged Object Detection (WSCOD) aims to locate and segment objects that are visually concealed within their surrounding scenes, relying solely on sparse supervision such as scribble annotations. Despite recent progress, existing WSCOD methods still lag far behind fully supervised ones due to two major limitations: (1) the pseudo masks generated by general-purpose segmentation models (e.g., SAM) and filtered via rules are often unreliable, as these models lack the task-specific semantic understanding required for effective pseudo labeling in COD; and (2) the neglect of inherent annotation bias in scribbles, which hinders the model from capturing the global structure of camouflaged objects. To overcome these challenges, we propose ${D}^{3}$ETOR, a two-stage WSCOD framework consisting of Debate-Enhanced Pseudo Labeling and Frequency-Aware Progressive Debiasing. In the first stage, we introduce an adaptive entropy-driven point sampling method and a multi-agent debate mechanism to enhance the capability of SAM for COD, improving the interpretability and precision of pseudo masks. In the second stage, we design FADeNet, which progressively fuses multi-level frequency-aware features to balance global semantic understanding with local detail modeling, while dynamically reweighting supervision strength across regions to alleviate scribble bias. By jointly exploiting the supervision signals from both the pseudo masks and scribble semantics, ${D}^{3}$ETOR significantly narrows the gap between weakly and fully supervised COD, achieving state-of-the-art performance on multiple benchmarks.

Method: InstructMoLE framework uses Instruction-Guided Routing (IGR) that derives a global routing signal from user instructions to select a single coherent expert council applied uniformly across all input tokens. Also introduces output-space orthogonality loss to promote expert functional diversity and prevent representational collapse.

Result: Extensive experiments show InstructMoLE significantly outperforms existing LoRA adapters and MoLE variants across challenging multi-conditional generation benchmarks, enabling superior compositional control and fidelity to user intent.

Conclusion: InstructMoLE presents a robust and generalizable framework for instruction-driven fine-tuning of generative models, addressing limitations of local routing by using global instruction-guided routing to preserve semantic and structural integrity in complex image generation tasks.

Abstract: Parameter-Efficient Fine-Tuning of Diffusion Transformers (DiTs) for diverse, multi-conditional tasks often suffers from task interference when using monolithic adapters like LoRA. The Mixture of Low-rank Experts (MoLE) architecture offers a modular solution, but its potential is usually limited by routing policies that operate at a token level. Such local routing can conflict with the global nature of user instructions, leading to artifacts like spatial fragmentation and semantic drift in complex image generation tasks. To address these limitations, we introduce InstructMoLE, a novel framework that employs an Instruction-Guided Mixture of Low-Rank Experts. Instead of per-token routing, InstructMoLE utilizes a global routing signal, Instruction-Guided Routing (IGR), derived from the user’s comprehensive instruction. This ensures that a single, coherently chosen expert council is applied uniformly across all input tokens, preserving the global semantics and structural integrity of the generation process. To complement this, we introduce an output-space orthogonality loss, which promotes expert functional diversity and mitigates representational collapse. Extensive experiments demonstrate that InstructMoLE significantly outperforms existing LoRA adapters and MoLE variants across challenging multi-conditional generation benchmarks. Our work presents a robust and generalizable framework for instruction-driven fine-tuning of generative models, enabling superior compositional control and fidelity to user intent.

Method: Two-stage adaptive design: (1) lightweight temporal extractor captures global context and ranks data windows by saliency, (2) 4D hierarchical encoder (Hiera-JEPA) learns fine-grained voxel-level representations only from top-k selected windows while masking ~70% of patches.

Result: Achieves state-of-the-art performance across seven public benchmarks while requiring only 4k pre-training sessions and ~30% GPU memory compared to traditional voxel-level methods.

Conclusion: SLIM-Brain successfully addresses the data- and training-efficiency bottlenecks in fMRI foundation models, enabling effective atlas-free modeling with practical computational requirements.

Abstract: Foundation models are emerging as a powerful paradigm for fMRI analysis, but current approaches face a dual bottleneck of data- and training-efficiency. Atlas-based methods aggregate voxel signals into fixed regions of interest, reducing data dimensionality but discarding fine-grained spatial details, and requiring extremely large cohorts to train effectively as general-purpose foundation models. Atlas-free methods, on the other hand, operate directly on voxel-level information - preserving spatial fidelity but are prohibitively memory- and compute-intensive, making large-scale pre-training infeasible. We introduce SLIM-Brain (Sample-efficient, Low-memory fMRI Foundation Model for Human Brain), a new atlas-free foundation model that simultaneously improves both data- and training-efficiency. SLIM-Brain adopts a two-stage adaptive design: (i) a lightweight temporal extractor captures global context across full sequences and ranks data windows by saliency, and (ii) a 4D hierarchical encoder (Hiera-JEPA) learns fine-grained voxel-level representations only from the top-$k$ selected windows, while deleting about 70% masked patches. Extensive experiments across seven public benchmarks show that SLIM-Brain establishes new state-of-the-art performance on diverse tasks, while requiring only 4 thousand pre-training sessions and approximately 30% of GPU memory comparing to traditional voxel-level methods.

Method: Created RxnBench with two tasks: Single-Figure QA (1,525 questions from 305 reaction schemes) testing visual perception and mechanistic reasoning, and Full-Document QA (108 articles) requiring cross-modal integration of text, schemes, and tables.

Result: MLLMs excel at extracting explicit text but struggle with deep chemical logic and precise structural recognition. Models with inference-time reasoning outperform standard architectures, but none achieve 50% accuracy on Full-Document QA.

Conclusion: There’s an urgent need for domain-specific visual encoders and stronger reasoning engines to advance autonomous AI chemists, as current MLLMs have critical capability gaps in chemical reaction understanding.

Abstract: The integration of Multimodal Large Language Models (MLLMs) into chemistry promises to revolutionize scientific discovery, yet their ability to comprehend the dense, graphical language of reactions within authentic literature remains underexplored. Here, we introduce RxnBench, a multi-tiered benchmark designed to rigorously evaluate MLLMs on chemical reaction understanding from scientific PDFs. RxnBench comprises two tasks: Single-Figure QA (SF-QA), which tests fine-grained visual perception and mechanistic reasoning using 1,525 questions derived from 305 curated reaction schemes, and Full-Document QA (FD-QA), which challenges models to synthesize information from 108 articles, requiring cross-modal integration of text, schemes, and tables. Our evaluation of MLLMs reveals a critical capability gap: while models excel at extracting explicit text, they struggle with deep chemical logic and precise structural recognition. Notably, models with inference-time reasoning significantly outperform standard architectures, yet none achieve 50% accuracy on FD-QA. These findings underscore the urgent need for domain-specific visual encoders and stronger reasoning engines to advance autonomous AI chemists.

Method: 1) Self-Augmented point clouds via triangulation between training and extrapolated views for coverage estimation; 2) Residual learning strategy for 3D Gaussian Splatting with uncertainty-driven filtering and dropout/hard-negative-mining sampling; 3) Physically guided view selection.

Result: Outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness for active view selection tasks.

Conclusion: SA-ResGS provides a comprehensive solution for uncertainty-aware active scene reconstruction with improved training stability, better uncertainty estimation, and implicit unbiasing of uncertainty quantification.

Abstract: We propose Self-Augmented Residual 3D Gaussian Splatting (SA-ResGS), a novel framework to stabilize uncertainty quantification and enhancing uncertainty-aware supervision in next-best-view (NBV) selection for active scene reconstruction. SA-ResGS improves both the reliability of uncertainty estimates and their effectiveness for supervision by generating Self-Augmented point clouds (SA-Points) via triangulation between a training view and a rasterized extrapolated view, enabling efficient scene coverage estimation. While improving scene coverage through physically guided view selection, SA-ResGS also addresses the challenge of under-supervised Gaussians, exacerbated by sparse and wide-baseline views, by introducing the first residual learning strategy tailored for 3D Gaussian Splatting. This targeted supervision enhances gradient flow in high-uncertainty Gaussians by combining uncertainty-driven filtering with dropout- and hard-negative-mining-inspired sampling. Our contributions are threefold: (1) a physically grounded view selection strategy that promotes efficient and uniform scene coverage; (2) an uncertainty-aware residual supervision scheme that amplifies learning signals for weakly contributing Gaussians, improving training stability and uncertainty estimation across scenes with diverse camera distributions; (3) an implicit unbiasing of uncertainty quantification as a consequence of constrained view selection and residual supervision, which together mitigate conflicting effects of wide-baseline exploration and sparse-view ambiguity in NBV planning. Experiments on active view selection demonstrate that SA-ResGS outperforms state-of-the-art baselines in both reconstruction quality and view selection robustness.

Method: Proposes IDESplat with iterative depth probability boosting: 1) Depth Probability Boosting Unit (DPBU) that integrates epipolar attention maps from cascaded warp operations multiplicatively, 2) Stacking multiple DPBUs to iteratively refine depth candidates, and 3) Progressive refinement of depth maps for accurate Gaussian mean prediction.

Result: Achieves outstanding reconstruction quality and state-of-the-art performance on RealEstate10K, ACID, and DL3DV datasets. On RE10K: outperforms DepthSplat by 0.33 dB PSNR with only 10.7% parameters and 70% memory. On DTU cross-dataset: improves PSNR by 2.95 dB over DepthSplat, demonstrating strong generalization.

Conclusion: IDESplat’s iterative depth probability boosting approach effectively addresses the limitations of single-warp methods, enabling more accurate Gaussian mean prediction and superior 3D reconstruction with real-time efficiency and strong generalization capabilities.

Abstract: Generalizable 3D Gaussian Splatting aims to directly predict Gaussian parameters using a feed-forward network for scene reconstruction. Among these parameters, Gaussian means are particularly difficult to predict, so depth is usually estimated first and then unprojected to obtain the Gaussian sphere centers. Existing methods typically rely solely on a single warp to estimate depth probability, which hinders their ability to fully leverage cross-view geometric cues, resulting in unstable and coarse depth maps. To address this limitation, we propose IDESplat, which iteratively applies warp operations to boost depth probability estimation for accurate Gaussian mean prediction. First, to eliminate the inherent instability of a single warp, we introduce a Depth Probability Boosting Unit (DPBU) that integrates epipolar attention maps produced by cascading warp operations in a multiplicative manner. Next, we construct an iterative depth estimation process by stacking multiple DPBUs, progressively identifying potential depth candidates with high likelihood. As IDESplat iteratively boosts depth probability estimates and updates the depth candidates, the depth map is gradually refined, resulting in accurate Gaussian means. We conduct experiments on RealEstate10K, ACID, and DL3DV. IDESplat achieves outstanding reconstruction quality and state-of-the-art performance with real-time efficiency. On RE10K, it outperforms DepthSplat by 0.33 dB in PSNR, using only 10.7% of the parameters and 70% of the memory. Additionally, our IDESplat improves PSNR by 2.95 dB over DepthSplat on the DTU dataset in cross-dataset experiments, demonstrating its strong generalization ability.

Method: Uses dense correspondence-guided pre-registration to initialize Gaussians with accurate geometry, constructs 3D Context Proposals via cross-view clustering, fuses global features onto Gaussians during direct registration, and maintains semantic coherence without additional optimization beyond standard reconstruction.

Result: Achieves strong open-vocabulary 3DGS understanding with semantic attachment completed in about five minutes per scene, which is two times faster than state-of-the-art methods.

Conclusion: ProFuse provides an efficient, context-aware framework for open-vocabulary 3D scene understanding that enhances consistency and cohesion while adding minimal computational overhead and requiring no fine-tuning.

Abstract: We present ProFuse, an efficient context-aware framework for open-vocabulary 3D scene understanding with 3D Gaussian Splatting (3DGS). The pipeline enhances cross-view consistency and intra-mask cohesion within a direct registration setup, adding minimal overhead and requiring no render-supervised fine-tuning. Instead of relying on a pretrained 3DGS scene, we introduce a dense correspondence-guided pre-registration phase that initializes Gaussians with accurate geometry while jointly constructing 3D Context Proposals via cross-view clustering. Each proposal carries a global feature obtained through weighted aggregation of member embeddings, and this feature is fused onto Gaussians during direct registration to maintain per-primitive language coherence across views. With associations established in advance, semantic fusion requires no additional optimization beyond standard reconstruction, and the model retains geometric refinement without densification. ProFuse achieves strong open-vocabulary 3DGS understanding while completing semantic attachment in about five minutes per scene, which is two times faster than SOTA. Additional details are available at our project page https://chiou1203.github.io/ProFuse/.

Method: 1) Trinocular setup with horizontally/vertically translated virtual viewpoints for view consistency; 2) Synthetic epipolar depth priors from virtual viewpoints as self-supervised regularizers; 3) Depth-aware alpha adjustment to modulate Gaussian opacity during early training based on depth.

Result: OceanSplat substantially outperforms existing methods for both scene reconstruction and restoration in scattering media, demonstrating effectiveness on real-world underwater and simulated scenes.

Conclusion: The approach successfully disentangles 3D Gaussians from scattering medium through geometric constraints, enabling accurate scene structure representation and significant reduction of floating artifacts in underwater environments.

Abstract: We introduce OceanSplat, a novel 3D Gaussian Splatting-based approach for high-fidelity underwater scene reconstruction. To overcome multi-view inconsistencies caused by scattering media, we design a trinocular setup for each camera pose by rendering from horizontally and vertically translated virtual viewpoints, enforcing view consistency to facilitate spatial optimization of 3D Gaussians. Furthermore, we derive synthetic epipolar depth priors from the virtual viewpoints, which serve as self-supervised depth regularizers to compensate for the limited geometric cues in degraded underwater scenes. We also propose a depth-aware alpha adjustment that modulates the opacity of 3D Gaussians during early training based on their depth along the viewing direction, deterring the formation of medium-induced primitives. Our approach promotes the disentanglement of 3D Gaussians from the scattering medium through effective geometric constraints, enabling accurate representation of scene structure and significantly reducing floating artifacts. Experiments on real-world underwater and simulated scenes demonstrate that OceanSplat substantially outperforms existing methods for both scene reconstruction and restoration in scattering media.

Method: Developed FlyPose, a lightweight top-down human pose estimation pipeline trained on multiple datasets, with deployment on Jetson Orin AGX for onboard UAV inference.

Result: Achieved 6.8 mAP improvement in person detection across multiple datasets and 16.3 mAP improvement in 2D pose estimation on UAV-Human dataset, with ~20ms inference latency.

Conclusion: FlyPose enables real-time human pose estimation from UAVs, addressing aerial perspective challenges, and includes the release of FlyPose-104 dataset for difficult aerial scenarios.

Abstract: Unmanned Aerial Vehicles (UAVs) are increasingly deployed in close proximity to humans for applications such as parcel delivery, traffic monitoring, disaster response and infrastructure inspections. Ensuring safe and reliable operation in these human-populated environments demands accurate perception of human poses and actions from an aerial viewpoint. This perspective challenges existing methods with low resolution, steep viewing angles and (self-)occlusion, especially if the application demands realtime feasibile models. We train and deploy FlyPose, a lightweight top-down human pose estimation pipeline for aerial imagery. Through multi-dataset training, we achieve an average improvement of 6.8 mAP in person detection across the test-sets of Manipal-UAV, VisDrone, HIT-UAV as well as our custom dataset. For 2D human pose estimation we report an improvement of 16.3 mAP on the challenging UAV-Human dataset. FlyPose runs with an inference latency of ~20 milliseconds including preprocessing on a Jetson Orin AGX Developer Kit and is deployed onboard a quadrotor UAV during flight experiments. We also publish FlyPose-104, a small but challenging aerial human pose estimation dataset, that includes manual annotations from difficult aerial perspectives: https://github.com/farooqhassaan/FlyPose.