Method: Two-stage approach: RapidFuzz pre-filtering followed by LLM verification on short snippets. Evaluated six modern LLMs with optimized prompt design (query before transcript, compact formatting) and reasoning budgets.

Result: Improved fuzzy match accuracy by up to 50 points while halving latency and reducing cost per correct result by up to 96%. With reasoning budgets, accuracy increased from 37% to 77% for weak setups and above 90% for strong ones. Robust across transcript lengths and domains.

Conclusion: The “Assisted Fuzzy” approach effectively solves the timestamp retrieval problem for non-verbatim quotes, with prompt design being more critical than model choice. The method maintains high performance across various transcript characteristics while significantly reducing computational costs.

Abstract: Traditional fuzzy matching often fails when searching for quotes that are semantically identical but syntactically different across documents-a common issue when aligning official written records with speech-to-text transcripts. We introduce TimeStampEval, a benchmark for retrieving precise millisecond timestamps from long transcripts given non-verbatim quotes. Our simple two-stage method dramatically improves retrieval accuracy while cutting inference costs by over 90%. The motivating use case is an automated long-form podcast that assembles Congressional Record clips into AI-hosted narration. The technical challenge: given a sentence-timestamped transcript and a target quote that may differ due to transcription or editorial drift, return exact start and end boundaries. Standard algorithms handle verbatim text but break under fuzzier variants. Evaluating six modern LLMs on a 2,800-sentence (120k-token) transcript revealed four key findings. (1) Prompt design matters more than model choice: placing the query before the transcript and using compact formatting improved accuracy by 3-20 points while reducing token count by 30-40%. (2) Off-by-one errors form a distinct category, showing models understand the task but misplace boundaries. (3) A modest reasoning budget (600-850 tokens) raises accuracy from 37% to 77% for weak setups and to above 90% for strong ones. (4) Our “Assisted Fuzzy” approach-RapidFuzz pre-filtering followed by LLM verification on short snippets-improves fuzzy match accuracy by up to 50 points while halving latency and reducing cost per correct result by up to 96%. Extended tests on ten transcripts (50k-900k tokens, 1989-2025) confirm robustness to transcript length, vocabulary drift, and domain change, maintaining 95-100% rejection accuracy for absent targets.

Method: Uses reinforcement learning to train models for deeper and more frequent agent-environment interactions, leveraging environment feedback and external information acquisition to correct errors and refine trajectories.

Result: The 72B variant achieves 81.9% on GAIA, 37.7% on HLE, 47.1% on BrowseComp, and 55.6% on BrowseComp-ZH, surpassing previous open-source agents and approaching commercial counterparts like GPT-5-high.

Conclusion: Interaction scaling represents a third critical dimension for building next-generation research agents, complementing model capacity and context windows, with performance improving predictably as interaction depth increases.

Abstract: We present MiroThinker v1.0, an open-source research agent designed to advance tool-augmented reasoning and information-seeking capabilities. Unlike previous agents that only scale up model size or context length, MiroThinker explores interaction scaling at the model level, systematically training the model to handle deeper and more frequent agent-environment interactions as a third dimension of performance improvement. Unlike LLM test-time scaling, which operates in isolation and risks degradation with longer reasoning chains, interactive scaling leverages environment feedback and external information acquisition to correct errors and refine trajectories. Through reinforcement learning, the model achieves efficient interaction scaling: with a 256K context window, it can perform up to 600 tool calls per task, enabling sustained multi-turn reasoning and complex real-world research workflows. Across four representative benchmarks-GAIA, HLE, BrowseComp, and BrowseComp-ZH-the 72B variant achieves up to 81.9%, 37.7%, 47.1%, and 55.6% accuracy respectively, surpassing previous open-source agents and approaching commercial counterparts such as GPT-5-high. Our analysis reveals that MiroThinker benefits from interactive scaling consistently: research performance improves predictably as the model engages in deeper and more frequent agent-environment interactions, demonstrating that interaction depth exhibits scaling behaviors analogous to model size and context length. These findings establish interaction scaling as a third critical dimension for building next-generation open research agents, complementing model capacity and context windows.

Method: Analyzes definitions of reasoning in NLP literature and proposes viewing transformer-based LMs as implementing implicit finite-order Markov kernels that map contexts to token distributions, where reasoning-like outputs emerge from statistical regularities.

Result: Shows that LMs produce reasoning-like behavior through statistical pattern matching rather than explicit reasoning mechanisms, explaining why they lack logical consistency guarantees.

Conclusion: The distinction between statistical pattern matching and genuine reasoning is fundamental for evaluating epistemic uncertainty in LMs, and calls for more precise descriptions of computational processes in NLP research.

Abstract: Language models (LMs) are said to be exhibiting reasoning, but what does this entail? We assess definitions of reasoning and how key papers in the field of natural language processing (NLP) use the notion and argue that the definitions provided are not consistent with how LMs are trained, process information, and generate new tokens. To illustrate this incommensurability we assume the view that transformer-based LMs implement an \textit{implicit} finite-order Markov kernel mapping contexts to conditional token distributions. In this view, reasoning-like outputs correspond to statistical regularities and approximate statistical invariances in the learned kernel rather than the implementation of explicit logical mechanisms. This view is illustrative of the claim that LMs are “statistical pattern matchers”" and not genuine reasoners and provides a perspective that clarifies why reasoning-like outputs arise in LMs without any guarantees of logical consistency. This distinction is fundamental to how epistemic uncertainty is evaluated in LMs. We invite a discussion on the importance of how the computational processes of the systems we build and analyze in NLP research are described.

Method: Evaluated seven open-weight models ranging from 0.6B to 70B parameters on hydropower licensing documentation, analyzing performance scaling and validation effectiveness.

Result: Identified a 14B parameter threshold where validation transitions from ineffective (F1 < 0.15) to viable (F1 = 0.64). Consumer-deployable models achieve 64% F1, smaller models plateau at 51%, and large-scale models approach 77% F1 but require enterprise infrastructure. Found systematic hallucination patterns in smaller models.

Conclusion: Established the first comprehensive resource-performance mapping for open-weight information extraction in regulatory contexts, enabling evidence-based model selection. Results provide immediate value for hydropower compliance and insights into parameter scaling effects that generalize across information extraction tasks.

Abstract: Information extraction from regulatory documents using large language models presents critical trade-offs between performance and computational resources. We evaluated seven open-weight models (0.6B-70B parameters) on hydropower licensing documentation to provide empirical deployment guidance. Our analysis identified a pronounced 14B parameter threshold where validation methods transition from ineffective (F1 $<$ 0.15) to viable (F1 = 0.64). Consumer-deployable models achieve 64% F1 through appropriate validation, while smaller models plateau at 51%. Large-scale models approach 77% F1 but require enterprise infrastructure. We identified systematic hallucination patterns where perfect recall indicates extraction failure rather than success in smaller models. Our findings establish the first comprehensive resource-performance mapping for open-weight information extraction in regulatory contexts, enabling evidence-based model selection. These results provide immediate value for hydropower compliance while contributing insights into parameter scaling effects that generalize across information extraction tasks.

Method: Used a simple LLM-based autoformalizer to verify LLM-generated outputs against natural language requirements through two experiments: checking logical equivalence of differently-worded requirements and identifying logical inconsistencies.

Result: The autoformalizer successfully identified logical equivalence between differently-worded requirements and detected logical inconsistencies between NL requirements and LLM-generated outputs.

Conclusion: Autoformalization shows significant potential for ensuring fidelity and logical consistency of LLM-generated outputs, laying foundation for future extensive studies in this application.

Abstract: Autoformalization, the process of translating informal statements into formal logic, has gained renewed interest with the emergence of powerful Large Language Models (LLMs). While LLMs show promise in generating structured outputs from natural language (NL), such as Gherkin Scenarios from NL feature requirements, there’s currently no formal method to verify if these outputs are accurate. This paper takes a preliminary step toward addressing this gap by exploring the use of a simple LLM-based autoformalizer to verify LLM-generated outputs against a small set of natural language requirements. We conducted two distinct experiments. In the first one, the autoformalizer successfully identified that two differently-worded NL requirements were logically equivalent, demonstrating the pipeline’s potential for consistency checks. In the second, the autoformalizer was used to identify a logical inconsistency between a given NL requirement and an LLM-generated output, highlighting its utility as a formal verification tool. Our findings, while limited, suggest that autoformalization holds significant potential for ensuring the fidelity and logical consistency of LLM-generated outputs, laying a crucial foundation for future, more extensive studies into this novel application.

Method: Uses dictionary-based sentiment analysis with a custom lexicon built from NRC-VAD dataset scores, and applies Wards hierarchical clustering to group similar sentiment plots.

Result: Experimental evaluation shows the analysis is helpful for consumers and readers when selecting narratives or stories.

Conclusion: The proposed framework effectively analyzes movie scripts through sentiment arcs and character context, providing valuable insights for narrative selection.

Abstract: Story understanding and analysis have long been challenging areas within Natural Language Understanding. Automated narrative analysis requires deep computational semantic representations along with syntactic processing. Moreover, the large volume of narrative data demands automated semantic analysis and computational learning rather than manual analytical approaches. In this paper, we propose a framework that analyzes the sentiment arcs of movie scripts and performs extended analysis related to the context of the characters involved. The framework enables the extraction of high-level and low-level concepts conveyed through the narrative. Using dictionary-based sentiment analysis, our approach applies a custom lexicon built with the LabMTsimple storylab module. The custom lexicon is based on the Valence, Arousal, and Dominance scores from the NRC-VAD dataset. Furthermore, the framework advances the analysis by clustering similar sentiment plots using Wards hierarchical clustering technique. Experimental evaluation on a movie dataset shows that the resulting analysis is helpful to consumers and readers when selecting a narrative or story.

Method: Created annotated corpus of 6,393 radiology reports; compared traditional ML classifiers (LR, SVM, Longformer) with fine-tuned Llama3-8B and generative LLMs (GPT-4o, GPT-OSS-20B) using baseline and task-optimized configurations with refined prompts.

Result: GPT-4o (Advanced) achieved best performance (F1=0.832), GPT-OSS-20B (Advanced) close second (F1=0.828), LR and SVM also strong (F1=0.776, 0.775); high inter-annotator agreement (F1=0.846).

Conclusion: While optimized LLMs approach human-level performance, interpretable and resource-efficient traditional models remain valuable baselines for radiology follow-up adherence detection.

Abstract: Large language models (LLMs) have shown considerable promise in clinical natural language processing, yet few domain-specific datasets exist to rigorously evaluate their performance on radiology tasks. In this work, we introduce an annotated corpus of 6,393 radiology reports from 586 patients, each labeled for follow-up imaging status, to support the development and benchmarking of follow-up adherence detection systems. Using this corpus, we systematically compared traditional machine-learning classifiers, including logistic regression (LR), support vector machines (SVM), Longformer, and a fully fine-tuned Llama3-8B-Instruct, with recent generative LLMs. To evaluate generative LLMs, we tested GPT-4o and the open-source GPT-OSS-20B under two configurations: a baseline (Base) and a task-optimized (Advanced) setting that focused inputs on metadata, recommendation sentences, and their surrounding context. A refined prompt for GPT-OSS-20B further improved reasoning accuracy. Performance was assessed using precision, recall, and F1 scores with 95% confidence intervals estimated via non-parametric bootstrapping. Inter-annotator agreement was high (F1 = 0.846). GPT-4o (Advanced) achieved the best performance (F1 = 0.832), followed closely by GPT-OSS-20B (Advanced; F1 = 0.828). LR and SVM also performed strongly (F1 = 0.776 and 0.775), underscoring that while LLMs approach human-level agreement through prompt optimization, interpretable and resource-efficient models remain valuable baselines.

Method: Created MedPT corpus through multi-stage curation with hybrid quantitative-qualitative analysis, LLM-driven annotation classifying questions into 7 semantic types, and filtering noise while contextually enriching ambiguous queries.

Result: Achieved 94% F1-score on 20-class medical specialty routing task using fine-tuned 1.7B parameter model, with error analysis showing misclassifications reflect genuine clinical ambiguities rather than random errors.

Conclusion: MedPT enables development of equitable, accurate, and culturally-aware medical technologies for Portuguese-speaking populations, proving the dataset’s semantic richness and utility for specialized medical AI applications.

Abstract: While large language models (LLMs) show transformative potential in healthcare, their development remains focused on high-resource languages, creating a critical barrier for others as simple translation fails to capture unique clinical and cultural nuances, such as endemic diseases. To address this, we introduce MedPT, the first large-scale, real-world corpus for Brazilian Portuguese, comprising 384,095 authentic question-answer pairs from patient-doctor interactions. The dataset underwent a meticulous multi-stage curation protocol, using a hybrid quantitative-qualitative analysis to filter noise and contextually enrich thousands of ambiguous queries. We further augmented the corpus via LLM-driven annotation, classifying questions into seven semantic types to capture user intent. Our analysis reveals its thematic breadth (3,200 topics) and unique linguistic properties, like the natural asymmetry in patient-doctor communication. To validate its utility, we benchmark a medical specialty routing task: fine-tuning a 1.7B parameter model achieves an outstanding 94% F1-score on a 20-class setup. Furthermore, our qualitative error analysis shows misclassifications are not random but reflect genuine clinical ambiguities (e.g., between comorbid conditions), proving the dataset’s deep semantic richness. We publicly release MedPT to foster the development of more equitable, accurate, and culturally-aware medical technologies for the Portuguese-speaking world.

Method: Leverage large language models to convert clinical free-text into structured, task-specific question-answer pairs before predictive modeling.

Result: Substantially enhances transparency and controllability with only modest performance reduction (2-3% AUC drop) on ICU mortality prediction compared to direct fine-tuning.

Conclusion: ClinStructor provides a foundation for building reliable, interpretable, and generalizable machine learning models in clinical environments.

Abstract: Clinical notes contain valuable, context-rich information, but their unstructured format introduces several challenges, including unintended biases (e.g., gender or racial bias), and poor generalization across clinical settings (e.g., models trained on one EHR system may perform poorly on another due to format differences) and poor interpretability. To address these issues, we present ClinStructor, a pipeline that leverages large language models (LLMs) to convert clinical free-text into structured, task-specific question-answer pairs prior to predictive modeling. Our method substantially enhances transparency and controllability and only leads to a modest reduction in predictive performance (a 2-3% drop in AUC), compared to direct fine-tuning, on the ICU mortality prediction task. ClinStructor lays a strong foundation for building reliable, interpretable, and generalizable machine learning models in clinical environments.

Method: Applied supervised fine-tuning and reinforcement learning to GPT-2, restructuring input formats to process context and emotional states simultaneously, using multi-component reward functions aligned with therapist responses and annotated emotions.

Result: Reinforcement learning outperformed baseline GPT-2 across all metrics: BLEU (0.0111), ROUGE-1 (0.1397), ROUGE-2 (0.0213), ROUGE-L (0.1317), METEOR (0.0581), and achieved 99.34% emotion accuracy vs 66.96% for baseline.

Conclusion: Reinforcement learning effectively develops therapeutic dialogue systems that can serve as valuable assistive tools for therapists while maintaining essential human clinical oversight.

Abstract: Mental health illness represents a substantial global socioeconomic burden, with COVID-19 further exacerbating accessibility challenges and driving increased demand for telehealth mental health support. While large language models (LLMs) offer promising solutions through 24/7 availability and non-judgmental interactions, pre-trained models often lack the contextual and emotional awareness necessary for appropriate therapeutic responses. This paper investigated the application of supervised fine-tuning (SFT) and reinforcement learning (RL) techniques to enhance GPT-2’s capacity for therapeutic dialogue generation. The methodology restructured input formats to enable simultaneous processing of contextual information and emotional states alongside user input, employing a multi-component reward function that aligned model outputs with professional therapist responses and annotated emotions. Results demonstrated improvements through reinforcement learning over baseline GPT-2 across multiple evaluation metrics: BLEU (0.0111), ROUGE-1 (0.1397), ROUGE-2 (0.0213), ROUGE-L (0.1317), and METEOR (0.0581). LLM evaluation confirmed high contextual relevance and professionalism, while reinforcement learning achieved 99.34% emotion accuracy compared to 66.96% for baseline GPT-2. These findings demonstrate reinforcement learning’s effectiveness in developing therapeutic dialogue systems that can serve as valuable assistive tools for therapists while maintaining essential human clinical oversight.

Method: Uses additive LLMs where inputs are semantically meaningful components (e.g., note sections, intake form fields) and predicts outcomes as the sum of each component’s contribution.

Result: Achieves performance comparable to conventional LLM classifiers while providing interpretable component-level risk curves and enabling quality-assurance checks.

Conclusion: CALM improves trust in clinical text classification by making model predictions interpretable at both patient and population levels, revealing clinically meaningful patterns.

Abstract: Large Language Models have advanced clinical text classification, but their opaque predictions remain a critical barrier to practical adoption in research and clinical settings where investigators and physicians need to understand which parts of a patient’s record drive risk signals. To address this challenge, we introduce \textbf{CALM}, short for \textbf{Classification with Additive Large Language Models}, an interpretable framework for semi-structured text where inputs are composed of semantically meaningful components, such as sections of an admission note or question-answer fields from an intake form. CALM predicts outcomes as the additive sum of each component’s contribution, making these contributions part of the forward computation itself and enabling faithful explanations at both the patient and population level. The additive structure also enables clear visualizations, such as component-level risk curves similar to those used in generalized additive models, making the learned relationships easier to inspect and communicate. Although CALM expects semi-structured inputs, many clinical documents already have this form, and similar structure can often be automatically extracted from free-text notes. CALM achieves performance comparable to conventional LLM classifiers while improving trust, supporting quality-assurance checks, and revealing clinically meaningful patterns during model development and auditing.

Method: Created BEA-Large (255h spontaneous speech) and BEA-Dialogue (85h natural conversations) from unprocessed portions of Hungarian BEA corpus, with detailed metadata and speaker-independent partitions.

Result: Fine-tuned Fast Conformer achieved 14.18% WER on spontaneous speech and 4.8% on repeated speech; diarization error rates between 13.05%-18.26%.

Conclusion: Conversational ASR remains challenging due to disfluencies, overlaps, and informal speech; released datasets provide framework for developing spontaneous speech benchmarks in other languages.

Abstract: The advancement of automatic speech recognition (ASR) has been largely enhanced by extensive datasets in high-resource languages, while languages such as Hungarian remain underrepresented due to limited spontaneous and conversational corpora. To address this gap, we introduce two new datasets – BEA-Large and BEA-Dialogue – constructed from the previously unprocessed portions of the Hungarian speech corpus named BEA. BEA-Large extends BEA-Base with 255 hours of spontaneous speech from 433 speakers, enriched with detailed segment-level metadata. BEA-Dialogue, comprising 85 hours of spontaneous conversations, is a Hungarian speech corpus featuring natural dialogues partitioned into speaker-independent subsets, supporting research in conversational ASR and speaker diarization. We establish reproducible baselines on these datasets using publicly available ASR models, with the fine-tuned Fast Conformer model achieving word error rates as low as 14.18% on spontaneous and 4.8% on repeated speech. Diarization experiments yield diarization error rates between 13.05% and 18.26%, providing reference points for future improvements. The results highlight the persistent difficulty of conversational ASR, particularly due to disfluencies, overlaps, and informal speech patterns. By releasing these datasets and baselines, we aim to advance Hungarian speech technology and offer a methodological framework for developing spontaneous and conversational benchmarks in other languages.

Method: Introduce Indirect Data Engagement (InData) dataset with data analysis questions at three difficulty levels (Easy, Medium, Hard) to benchmark LLMs’ multi-step tool-based reasoning capabilities.

Result: Benchmarked 15 open-source LLMs showing high accuracy on Easy tasks (97.3%) but significant performance drop on Hard tasks (69.6%), indicating lack of robust multi-step reasoning.

Conclusion: Current LLMs lack strong multi-step tool-based reasoning abilities, and InData enables development and evaluation of improved models for secure data analysis.

Abstract: Large language model agents for data analysis typically generate and execute code directly on databases. However, when applied to sensitive data, this approach poses significant security risks. To address this issue, we propose a security-motivated alternative: restrict LLMs from direct code generation and data access, and require them to interact with data exclusively through a predefined set of secure, verified tools. Although recent tool-use benchmarks exist, they primarily target tool selection and simple execution rather than the compositional, multi-step reasoning needed for complex data analysis. To reduce this gap, we introduce Indirect Data Engagement (InData), a dataset designed to assess LLMs’ multi-step tool-based reasoning ability. InData includes data analysis questions at three difficulty levels–Easy, Medium, and Hard–capturing increasing reasoning complexity. We benchmark 15 open-source LLMs on InData and find that while large models (e.g., gpt-oss-120b) achieve high accuracy on Easy tasks (97.3%), performance drops sharply on Hard tasks (69.6%). These results show that current LLMs still lack robust multi-step tool-based reasoning ability. With InData, we take a step toward enabling the development and evaluation of LLMs with stronger multi-step tool-use capabilities. We will publicly release the dataset and code.

Method: Proposed LLM-KAT evaluation procedure for measuring knowledge attachment, and introduced entity anonymization technique to encourage better utilization of external knowledge.

Result: Experiments on OpenDialKG dataset show improved knowledge attachment in LLMs when using the proposed approach.

Conclusion: The simple entity anonymization technique effectively helps LLMs better leverage external knowledge in knowledge graph-based dialogue generation tasks.

Abstract: Knowledge graph-based dialogue generation (KG-DG) is a challenging task requiring models to effectively incorporate external knowledge into conversational responses. While large language models (LLMs) have achieved impressive results across various NLP tasks, their ability to utilize external knowledge in KG-DG remains under-explored. We observe that LLMs often rely on internal knowledge, leading to detachment from provided knowledge graphs, even when they are given a flawlessly retrieved knowledge graph. First, we introduce LLM-KAT, an evaluation procedure for measuring knowledge attachment in generated responses. Second, we propose a simple yet effective entity anonymization technique to encourage LLMs to better leverage external knowledge. Experiments on the OpenDialKG dataset demonstrate that our approach improves LLMs’ attachment on external knowledge.

Method: First studied a simplified theoretical setting to characterize scaling behavior of miscalibration with respect to dataset size, then empirically measured miscalibration in language models ranging from 0.5B to 70B parameters.

Result: Found that miscalibration scaling behavior depends on power law exponent of data distribution - for exponents close to 1, scaling exponent is close to 0, meaning miscalibration improves very slowly with scale. Empirical results matched theoretical predictions.

Conclusion: Larger models accumulate error at similar rates as smaller ones, explaining why we use similar truncation levels. However, the paper proves it’s theoretically possible to reduce entropy while preserving log loss using a black box that can predict future entropy of text.

Abstract: We study the problem of entropy calibration, which asks whether a language model’s entropy over generations matches its log loss on human text. Past work found that models are miscalibrated, with entropy per step increasing (and text quality decreasing) as generations grow longer. This error accumulation is a fundamental problem in autoregressive models, and the standard solution is to truncate the distribution, which improves text quality at the cost of diversity. In this paper, we ask: is miscalibration likely to improve with scale, and is it theoretically possible to calibrate without tradeoffs? To build intuition, we first study a simplified theoretical setting to characterize the scaling behavior of miscalibration with respect to dataset size. We find that the scaling behavior depends on the power law exponent of the data distribution – in particular, for a power law exponent close to 1, the scaling exponent is close to 0, meaning that miscalibration improves very slowly with scale. Next, we measure miscalibration empirically in language models ranging from 0.5B to 70B parameters. We find that the observed scaling behavior is similar to what is predicted by the simplified setting: our fitted scaling exponents for text are close to 0, meaning that larger models accumulate error at a similar rate as smaller ones. This scaling (or, lack thereof) provides one explanation for why we sample from larger models with similar amounts of truncation as smaller models, even though the larger models are of higher quality. However, truncation is not a satisfying solution because it comes at the cost of increased log loss. In theory, is it even possible to reduce entropy while preserving log loss? We prove that it is possible, if we assume access to a black box which can fit models to predict the future entropy of text.

Method: Three-stage framework: 1) Generate NER-oriented Chain of Thought (CoT) datasets with reasoning chains, 2) Tune NER model to generate rationales before final answers, 3) Enhance reasoning process using comprehensive reward signals for verifiable extractions.

Result: Achieves competitive performance overall and SOTA in zero-shot settings, outperforming GPT-4 by 12.3 percentage points on F1 score. Demonstrates impressive cognitive ability and potential for reasoning-oriented information extraction.

Conclusion: The reasoning framework successfully addresses cognitive shortcutting in NER by enabling explicit, verifiable reasoning, showing great promise for advancing research in reasoning-oriented information extraction.

Abstract: Generative LLMs typically improve Named Entity Recognition (NER) performance through instruction tuning. They excel at generating entities by semantic pattern matching but lack an explicit, verifiable reasoning mechanism. This “cognitive shortcutting” leads to suboptimal performance and brittle generalization, especially in zero-shot and lowresource scenarios where reasoning from limited contextual cues is crucial. To address this issue, a reasoning framework is proposed for NER, which shifts the extraction paradigm from implicit pattern matching to explicit reasoning. This framework consists of three stages: Chain of Thought (CoT) generation, CoT tuning, and reasoning enhancement. First, a dataset annotated with NER-oriented CoTs is generated, which contain task-relevant reasoning chains. Then, they are used to tune the NER model to generate coherent rationales before deriving the final answer. Finally, a reasoning enhancement stage is implemented to optimize the reasoning process using a comprehensive reward signal. This stage ensures explicit and verifiable extractions. Experiments show that ReasoningNER demonstrates impressive cognitive ability in the NER task, achieving competitive performance. In zero-shot settings, it achieves state-of-the-art (SOTA) performance, outperforming GPT-4 by 12.3 percentage points on the F1 score. Analytical results also demonstrate its great potential to advance research in reasoningoriented information extraction. Our codes are available at https://github.com/HuiResearch/ReasoningIE.

Method: Systematically perturbing reasoning chains and manipulating delivery tones in vision language models, then analyzing how reasoning errors impact user trust and error detection ability.

Result: Users equate trust with outcome agreement (sustaining reliance despite flawed reasoning), and confident tones suppress error detection while maintaining reliance (delivery style overrides correctness).

Conclusion: CoT explanations can simultaneously clarify and mislead, highlighting the need for NLP systems to provide explanations that encourage scrutiny and critical thinking rather than blind trust.

Abstract: Explanations are often promoted as tools for transparency, but they can also foster confirmation bias; users may assume reasoning is correct whenever outputs appear acceptable. We study this double-edged role of Chain-of-Thought (CoT) explanations in multimodal moral scenarios by systematically perturbing reasoning chains and manipulating delivery tones. Specifically, we analyze reasoning errors in vision language models (VLMs) and how they impact user trust and the ability to detect errors. Our findings reveal two key effects: (1) users often equate trust with outcome agreement, sustaining reliance even when reasoning is flawed, and (2) the confident tone suppresses error detection while maintaining reliance, showing that delivery styles can override correctness. These results highlight how CoT explanations can simultaneously clarify and mislead, underscoring the need for NLP systems to provide explanations that encourage scrutiny and critical thinking rather than blind trust. All code will be released publicly.

Method: The authors introduce benchmarks with realistic situational contexts requiring culturally grounded reasoning, and use four complementary metrics (Coverage, Specificity, Connotation, and Coherence) alongside standard Exact Match to capture different dimensions of response quality.

Result: Empirical analysis shows that traditional ’thin’ evaluation systematically overestimates cultural competence and produces unstable assessments with high variance, while their ’thick’ evaluation exposes differences in reasoning depth, reduces variance, and provides more stable, interpretable signals.

Conclusion: Thick evaluation with realistic contexts and multiple metrics provides more accurate and stable assessment of cultural competence in LLMs compared to traditional evaluation methods.

Abstract: Large language models (LLMs) are increasingly deployed in culturally diverse environments, yet existing evaluations of cultural competence remain limited. Existing methods focus on de-contextualized correctness or forced-choice judgments, overlooking the need for cultural understanding and reasoning required for appropriate responses. To address this gap, we introduce a set of benchmarks that, instead of directly probing abstract norms or isolated statements, present models with realistic situational contexts that require culturally grounded reasoning. In addition to the standard Exact Match metric, we introduce four complementary metrics (Coverage, Specificity, Connotation, and Coherence) to capture different dimensions of model’s response quality. Empirical analysis across frontier models reveals that thin evaluation systematically overestimates cultural competence and produces unstable assessments with high variance. In contrast, thick evaluation exposes differences in reasoning depth, reduces variance, and provides more stable, interpretable signals of cultural understanding.

Method: First apply backtranslation using synthetic data from monolingual Japanese corpora, then fine-tune on genuine small parallel dataset, and finally combine both by augmenting small dataset with BT examples before fine-tuning.

Result: Baseline COMET=0.460 → BT alone=0.468 → FT alone=0.589 → Combined BT+FT=0.597, showing significant improvement over individual techniques.

Conclusion: The synergistic combination of backtranslation and targeted fine-tuning offers a lightweight yet powerful strategy for improving low-resource language pairs even with limited training data.

Abstract: In this paper, we explore the effectiveness of combining fine-tuning and backtranslation on a small Japanese corpus for neural machine translation. Starting from a baseline English{\textrightarrow}Japanese model (COMET = 0.460), we first apply backtranslation (BT) using synthetic data generated from monolingual Japanese corpora, yielding a modest increase (COMET = 0.468). Next, we fine-tune (FT) the model on a genuine small parallel dataset drawn from diverse Japanese news and literary corpora, achieving a substantial jump to COMET = 0.589 when using Mistral 7B. Finally, we integrate both backtranslation and fine-tuning{ – }first augmenting the small dataset with BT generated examples, then adapting via FT{ – }which further boosts performance to COMET = 0.597. These results demonstrate that, even with limited training data, the synergistic use of backtranslation and targeted fine-tuning on Japanese corpora can significantly enhance translation quality, outperforming each technique in isolation. This approach offers a lightweight yet powerful strategy for improving low-resource language pairs.

Method: Developed LLMLagBench to systematically evaluate LLMs’ knowledge of recent events and identify the earliest probable temporal boundaries of their training data.

Result: Applied the benchmark to evaluate various LLMs with both declared and undeclared training cutoffs, with reliability assessed through manual validation and comparison with publicly available pretraining information.

Conclusion: LLMLagBench provides a systematic approach to identify LLMs’ temporal knowledge limitations, helping address the problem of outdated information being used in reasoning tasks.

Abstract: Large Language Models (LLMs) are pretrained on textual data up to a specific temporal cutoff. This creates a strict knowledge boundary beyond which models cannot provide accurate information without querying external sources. More subtly, when this limitation is unknown or ignored, LLMs may inadvertently blend outdated time-sensitive information with general knowledge during reasoning tasks, potentially compromising response accuracy. We introduce LLMLagBench, an LLM freshness benchmark, as a systematic approach for identifying the earliest probable temporal boundaries of an LLM’s training data by evaluating its knowledge of recent events. We then apply this benchmark to evaluate a large set of LLMs, including models with both explicitly declared and undeclared training cutoffs. The reliability of the benchmark is assessed by manual validation and comparison with publicly released information about LLM pretraining.

Method: PRISM model derives user personas from historical data, aligns textual/visual cues via Chain-of-Thought reasoning, and uses mutual task reinforcement for joint stance detection and response generation.

Result: Experiments on U-MStance dataset show PRISM achieves significant performance gains over strong baselines in multimodal conversational stance detection.

Conclusion: User-centric and context-grounded multimodal reasoning is essential for realistic stance understanding in social media conversations.

Abstract: The rapid proliferation of multimodal social media content has driven research in Multimodal Conversational Stance Detection (MCSD), which aims to interpret users’ attitudes toward specific targets within complex discussions. However, existing studies remain limited by: 1) pseudo-multimodality, where visual cues appear only in source posts while comments are treated as text-only, misaligning with real-world multimodal interactions; and 2) user homogeneity, where diverse users are treated uniformly, neglecting personal traits that shape stance expression. To address these issues, we introduce U-MStance, the first user-centric MCSD dataset, containing over 40k annotated comments across six real-world targets. We further propose PRISM, a Persona-Reasoned multImodal Stance Model for MCSD. PRISM first derives longitudinal user personas from historical posts and comments to capture individual traits, then aligns textual and visual cues within conversational context via Chain-of-Thought to bridge semantic and pragmatic gaps across modalities. Finally, a mutual task reinforcement mechanism is employed to jointly optimize stance detection and stance-aware response generation for bidirectional knowledge transfer. Experiments on U-MStance demonstrate that PRISM yields significant gains over strong baselines, underscoring the effectiveness of user-centric and context-grounded multimodal reasoning for realistic stance understanding.

Method: Constructed TeleSalesCorpus dataset, then proposed AI-Salesman with dual-stage architecture: training stage uses Bayesian-supervised reinforcement learning from noisy dialogues, and inference stage uses Dynamic Outline-Guided Agent (DOGA) with pre-built script library for turn-by-turn strategic guidance.

Result: AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, demonstrating effectiveness in complex persuasive scenarios.

Conclusion: The proposed framework successfully addresses challenges in goal-driven persuasive dialogue by combining robust training with dynamic inference guidance, showing promising results for real-world sales applications.

Abstract: Goal-driven persuasive dialogue, exemplified by applications like telemarketing, requires sophisticated multi-turn planning and strict factual faithfulness, which remains a significant challenge for even state-of-the-art Large Language Models (LLMs). A lack of task-specific data often limits previous works, and direct LLM application suffers from strategic brittleness and factual hallucination. In this paper, we first construct and release TeleSalesCorpus, the first real-world-grounded dialogue dataset for this domain. We then propose AI-Salesman, a novel framework featuring a dual-stage architecture. For the training stage, we design a Bayesian-supervised reinforcement learning algorithm that learns robust sales strategies from noisy dialogues. For the inference stage, we introduce the Dynamic Outline-Guided Agent (DOGA), which leverages a pre-built script library to provide dynamic, turn-by-turn strategic guidance. Moreover, we design a comprehensive evaluation framework that combines fine-grained metrics for key sales skills with the LLM-as-a-Judge paradigm. Experimental results demonstrate that our proposed AI-Salesman significantly outperforms baseline models in both automatic metrics and comprehensive human evaluations, showcasing its effectiveness in complex persuasive scenarios.

Method: Uses a pixel-level Grounding LLM with reasoning and referring segmentation capabilities. Includes an innovative pipeline for generating instruction-tuning data (R-Instruct) with rich-context descriptions, grounding masks, and hard negative samples.

Result: Outperforms prior complex frameworks and achieves SOTA on R^2-HalBench benchmark, rivaling GPT-4o’s capabilities. Surpasses prior methods in pixel-level grounding with over 10% improvement.

Conclusion: VBackChecker provides an effective reference-free hallucination detection framework with interpretability that handles rich-context scenarios well, demonstrating superior performance compared to existing methods.

Abstract: Multimodal Large Language Models (MLLMs) have unlocked powerful cross-modal capabilities, but still significantly suffer from hallucinations. As such, accurate detection of hallucinations in MLLMs is imperative for ensuring their reliability in practical applications. To this end, guided by the principle of “Seeing is Believing”, we introduce VBackChecker, a novel reference-free hallucination detection framework that verifies the consistency of MLLMgenerated responses with visual inputs, by leveraging a pixellevel Grounding LLM equipped with reasoning and referring segmentation capabilities. This reference-free framework not only effectively handles rich-context scenarios, but also offers interpretability. To facilitate this, an innovative pipeline is accordingly designed for generating instruction-tuning data (R-Instruct), featuring rich-context descriptions, grounding masks, and hard negative samples. We further establish R^2 -HalBench, a new hallucination benchmark for MLLMs, which, unlike previous benchmarks, encompasses real-world, rich-context descriptions from 18 MLLMs with high-quality annotations, spanning diverse object-, attribute, and relationship-level details. VBackChecker outperforms prior complex frameworks and achieves state-of-the-art performance on R^2 -HalBench, even rivaling GPT-4o’s capabilities in hallucination detection. It also surpasses prior methods in the pixel-level grounding task, achieving over a 10% improvement. All codes, data, and models are available at https://github.com/PinxueGuo/VBackChecker.

Method: Uses a frozen, asymmetric critique LLM that retrospectively evaluates each turn using privileged information from full trajectory and gold answers, converting assessments into stable, dense rewards to guide policy improvement.

Result: Experimental results across diverse multi-hop reasoning benchmarks show CriticSearch consistently outperforms existing baselines with faster convergence, improved training stability, and higher performance.

Conclusion: CriticSearch effectively addresses sparse reward problems in search agent training through fine-grained credit assignment, demonstrating superior performance and stability in complex reasoning tasks.

Abstract: Tool-Integrated Reasoning (TIR) with search engines enables large language models to iteratively retrieve up-to-date external knowledge, enhancing adaptability and generalization in complex question-answering tasks. However, existing search agent pipelines typically depend on reinforcement learning based optimization, which often suffers from sparse outcome rewards, leading to inefficient exploration and unstable training. We introduce CriticSearch, a fine-grained credit-assignment framework that supplies dense, turn-level feedback via a retrospective critic mechanism. During training, a frozen, asymmetric critique LLM retrospectively evaluates each turn using privileged information from the full trajectory and gold answers, converting these assessments into stable, dense rewards that guide policy improvement. Experimental results across diverse multi-hop reasoning benchmarks demonstrate that CriticSearch consistently outperforms existing baselines, achieving faster convergence, improved training stability, and higher performance.

Method: Decomposes entity recognition into type-level judgment by lightweight managers and span-level extraction by specialized experts, with KeyInfo retrievers injecting semantically aligned few-shot exemplars during inference.

Result: Outperforms recent baselines in most domains on CrossNER, MIT-Movie, MIT-Restaurant, and a new multi-domain customer-service dataset.

Conclusion: MME-RAG provides a scalable and interpretable solution for adaptive dialogue understanding, with hierarchical decomposition and KeyInfo-guided retrieval being key to robustness and cross-domain generalization.

Abstract: Fine-grained entity recognition is crucial for reasoning and decision-making in task-oriented dialogues, yet current large language models (LLMs) continue to face challenges in domain adaptation and retrieval controllability. We introduce MME-RAG, a Multi-Manager-Expert Retrieval-Augmented Generation framework that decomposes entity recognition into two coordinated stages: type-level judgment by lightweight managers and span-level extraction by specialized experts. Each expert is supported by a KeyInfo retriever that injects semantically aligned, few-shot exemplars during inference, enabling precise and domain-adaptive extraction without additional training. Experiments on CrossNER, MIT-Movie, MIT-Restaurant, and our newly constructed multi-domain customer-service dataset demonstrate that MME-RAG performs better than recent baselines in most domains. Ablation studies further show that both the hierarchical decomposition and KeyInfo-guided retrieval are key drivers of robustness and cross-domain generalization, establishing MME-RAG as a scalable and interpretable solution for adaptive dialogue understanding.

Method: Uses internal consistency checks by probing factual statements within generated text and comparing responses across the same LLM and different LLMs, without external knowledge bases.

Result: Achieves higher accuracy than existing baselines while using fewer resources, validated on multiple datasets covering factual text generation and open generation tasks.

Conclusion: CONFACTCHECK provides an efficient and effective solution for hallucination detection in constrained settings where model fine-tuning or external knowledge bases are not available.

Abstract: Large language models (LLMs), despite their remarkable text generation capabilities, often hallucinate and generate text that is factually incorrect and not grounded in real-world knowledge. This poses serious risks in domains like healthcare, finance, and customer support. A typical way to use LLMs is via the APIs provided by LLM vendors where there is no access to model weights or options to fine-tune the model. Existing methods to detect hallucinations in such settings where the model access is restricted or constrained by resources typically require making multiple LLM API calls, increasing latency and API cost. We introduce CONFACTCHECK, an efficient hallucination detection approach that does not leverage any external knowledge base and works on the simple intuition that responses to factual probes within the generated text should be consistent within a single LLM and across different LLMs. Rigorous empirical evaluation on multiple datasets that cover both the generation of factual texts and the open generation shows that CONFACTCHECK can detect hallucinated facts efficiently using fewer resources and achieves higher accuracy scores compared to existing baselines that operate under similar conditions. Our code is available here.

Method: Integrates contrastive learning (SimCLR) and gloss-based distillation to learn Vietnamese contextualized embeddings, and creates ViConWSD - the first large-scale synthetic dataset for Vietnamese semantic evaluation.

Result: Outperforms baselines on WSD (F1=0.87), achieves competitive performance on ViCon (AP=0.88) and ViSim-400 (Spearman’s rho=0.60).

Conclusion: ViConBERT effectively models both discrete senses and graded semantic relations in Vietnamese, providing a strong foundation for Vietnamese semantic understanding.

Abstract: Recent advances in contextualized word embeddings have greatly improved semantic tasks such as Word Sense Disambiguation (WSD) and contextual similarity, but most progress has been limited to high-resource languages like English. Vietnamese, in contrast, still lacks robust models and evaluation resources for fine-grained semantic understanding. In this paper, we present ViConBERT, a novel framework for learning Vietnamese contextualized embeddings that integrates contrastive learning (SimCLR) and gloss-based distillation to better capture word meaning. We also introduce ViConWSD, the first large-scale synthetic dataset for evaluating semantic understanding in Vietnamese, covering both WSD and contextual similarity. Experimental results show that ViConBERT outperforms strong baselines on WSD (F1 = 0.87) and achieves competitive performance on ViCon (AP = 0.88) and ViSim-400 (Spearman’s rho = 0.60), demonstrating its effectiveness in modeling both discrete senses and graded semantic relations. Our code, models, and data are available at https://github.com/tkhangg0910/ViConBERT

Method: Comprehensive benchmarking of 25 models, Textgrad-based meta-prompt optimization for gpt-4.1-mini, and post-training Qwen3-4B with SFT and GRPO for dual objectives of compression rate adherence and task performance.

Result: Cmprsr model demonstrates superiority over extractive and vanilla abstractive compression across all compression rates on lengthy inputs and short prompts, with fine control over cost-quality trade-off.

Conclusion: The proposed compression paradigm and Cmprsr model effectively address LLM cost issues while maintaining performance, showing generalizability across input lengths and domains.

Abstract: Motivated by the high costs of using black-box Large Language Models (LLMs), we introduce a novel prompt compression paradigm, under which we use smaller LLMs to compress inputs for the larger ones. We present the first comprehensive LLM-as-a-compressor benchmark spanning 25 open- and closed-source models, which reveals significant disparity in models’ compression ability in terms of (i) preserving semantically important information (ii) following the user-provided compression rate (CR). We further improve the performance of gpt-4.1-mini, the best overall vanilla compressor, with Textgrad-based compression meta-prompt optimization. We also identify the most promising open-source vanilla LLM - Qwen3-4B - and post-train it with a combination of supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), pursuing the dual objective of CR adherence and maximizing the downstream task performance. We call the resulting model Cmprsr and demonstrate its superiority over both extractive and vanilla abstractive compression across the entire range of compression rates on lengthy inputs from MeetingBank and LongBench as well as short prompts from GSM8k. The latter highlights Cmprsr’s generalizability across varying input lengths and domains. Moreover, Cmprsr closely follows the requested compression rate, offering fine control over the cost-quality trade-off.

Method: Engineered a light pipeline that transforms existing abstractive gold standard summaries into corresponding extractive versions, preserving expert opinions from the original summaries.

Result: Augmented seven existing case summarization datasets with extractive summaries and performed extensive evaluation covering structural, lexical, semantic dimensions and domain-level information.

Conclusion: The enriched datasets will be released publicly to advance automatic legal document summarization research, providing valuable resources for the community.

Abstract: Summarization of legal judgments poses a heavy cognitive burden on law practitioners due to the complexity of the language, context-sensitive legal jargon, and the length of the document. Therefore, the automatic summarization of legal documents has attracted serious attention from natural language processing researchers. Since the abstractive summaries of legal documents generated by deep neural methods remain prone to the risk of misrepresenting nuanced legal jargon or overlooking key contextual details, we envisage a rising trend toward the use of extractive case summarizers. Given the high cost of human annotation for gold standard extractive summaries, we engineer a light and transparent pipeline that leverages existing abstractive gold standard summaries to create the corresponding extractive gold standard versions. The approach ensures that the experts` opinions ensconced in the original gold standard abstractive summaries are carried over to the transformed extractive summaries. We aim to augment seven existing case summarization datasets, which include abstractive summaries, by incorporating corresponding extractive summaries and create an enriched data resource for case summarization research community. To ensure the quality of the augmented extractive summaries, we perform an extensive comparative evaluation with the original abstractive gold standard summaries covering structural, lexical, and semantic dimensions. We also compare the domain-level information of the two summaries. We commit to release the augmented datasets in the public domain for use by the research community and believe that the resource will offer opportunities to advance the field of automatic summarization of legal documents.

Method: Collected Japanese quiz data with questions, answers, and human correct response rates; prompted LLMs to answer under various settings and compared performance.

Result: LLMs perform worse than humans on quizzes not covered by Wikipedia entries and those requiring numerical answers.

Conclusion: LLMs have different difficulty patterns than humans, struggling with knowledge gaps and numerical reasoning.

Abstract: LLMs have achieved performance that surpasses humans in many NLP tasks. However, it remains unclear whether problems that are difficult for humans are also difficult for LLMs. This study investigates how the difficulty of quizzes in a buzzer setting differs between LLMs and humans. Specifically, we first collect Japanese quiz data including questions, answers, and correct response rate of humans, then prompted LLMs to answer the quizzes under several settings, and compare their correct answer rate to that of humans from two analytical perspectives. The experimental results showed that, compared to humans, LLMs struggle more with quizzes whose correct answers are not covered by Wikipedia entries, and also have difficulty with questions that require numerical answers.

Method: Two experiments: (1) varying distractor text types (semantic, syntactic, repetition) after negation to measure rebound strength; (2) testing polarity separation between neutral and negative framings. Also includes circuit tracing analysis to identify neural mechanisms.

Result: Rebound consistently occurs immediately after negation, intensifies with longer or semantic distractors, while repetition supports suppression. Stronger polarity separation correlates with more persistent rebound. Circuit analysis shows middle-layer attention heads amplify forbidden tokens while early layers suppress.

Conclusion: The findings link cognitive predictions of ironic rebound with mechanistic insights into long-context interference in LLMs, and the authors release ReboundBench dataset for future research.

Abstract: Negation instructions such as ‘do not mention $X$’ can paradoxically increase the accessibility of $X$ in human thought, a phenomenon known as ironic rebound. Large language models (LLMs) face the same challenge: suppressing a concept requires internally activating it, which may prime rebound instead of avoidance. We investigated this tension with two experiments. \textbf{(1) Load & content}: after a negation instruction, we vary distractor text (semantic, syntactic, repetition) and measure rebound strength. \textbf{(2) Polarity separation}: We test whether models distinguish neutral from negative framings of the same concept and whether this separation predicts rebound persistence. Results show that rebound consistently arises immediately after negation and intensifies with longer or semantic distractors, while repetition supports suppression. Stronger polarity separation correlates with more persistent rebound. Together, these findings, complemented by a circuit tracing analysis that identifies sparse middle-layer attention heads amplifying forbidden tokens while early layers suppress, link cognitive predictions of ironic rebound with mechanistic insights into long-context interference. To support future work, we release ReboundBench, a dataset of $5,000$ systematically varied negation prompts designed to probe rebound in LLMs.

Method: Created ILAKKANAM benchmark with 820 manually curated questions from Sri Lankan school Tamil exams, annotated by linguists across 5 linguistic categories and factual knowledge, covering Grades 1-13. Evaluated both closed-source and open-source LLMs using standardized framework.

Result: Gemini 2.5 achieved highest overall performance; open-source models lagged. All models performed well on lower-grade questions but declined as linguistic complexity increased. No strong correlation found between overall performance and ability to identify linguistic categories.

Conclusion: LLMs’ performance in Tamil appears driven by exposure rather than genuine linguistic understanding, highlighting the need for better linguistic grounding in low-resource languages.

Abstract: Large Language Models (LLMs) have shown strong generalization across tasks in high-resource languages; however, their linguistic competence in low-resource and morphologically rich languages such as Tamil remains largely unexplored. Existing multilingual benchmarks often rely on translated English datasets, failing to capture the linguistic and cultural nuances of the target language. To address this gap, we introduce ILAKKANAM, the first Tamil-specific linguistic evaluation benchmark manually curated using 820 questions from Sri Lankan school-level Tamil subject examination papers. Each question is annotated by trained linguists under five linguistic categories and a factual knowledge category, spanning Grades 1–13 to ensure broad linguistic coverage. We evaluate both closed-source and open-source LLMs using a standardized evaluation framework. Our results show that Gemini 2.5 achieves the highest overall performance, while open-source models lag behind, highlighting the gap in linguistic grounding. Category- and grade-wise analyses reveal that all models perform well on lower-grade questions but show a clear decline as linguistic complexity increases. Further, no strong correlation is observed between a model’s overall performance and its ability to identify linguistic categories, suggesting that performance may be driven by exposure rather than genuine understanding.

Method: Constructed MRMBench with six probing tasks for different preference dimensions, and introduced inference-time probing to identify dimensions used during reward prediction and assess prediction confidence.

Result: MRMBench strongly correlates with LLM alignment performance, revealing that reward models often struggle with multi-dimensional preferences, and inference-time probing provides reliable confidence assessment.

Conclusion: MRMBench serves as a reliable benchmark for developing advanced reward models, highlighting the need for multi-objective optimization in reward modeling, while inference-time probing improves interpretability and alignment performance.

Abstract: Previous methods evaluate reward models by testing them on a fixed pairwise ranking test set, but they typically do not provide performance information on each preference dimension. In this work, we address the evaluation challenge of reward models by probing preference representations. To confirm the effectiveness of this evaluation method, we construct a Multi-dimensional Reward Model Benchmark (MRMBench), a collection of six probing tasks for different preference dimensions. We design it to favor and encourage reward models that better capture preferences across different dimensions. Furthermore, we introduce an analysis method, inference-time probing, which identifies the dimensions used during the reward prediction and enhances its interpretability. Through extensive experiments, we find that MRMBench strongly correlates with the alignment performance of large language models (LLMs), making it a reliable reference for developing advanced reward models. Our analysis of MRMBench evaluation results reveals that reward models often struggle to capture preferences across multiple dimensions, highlighting the potential of multi-objective optimization in reward modeling. Additionally, our findings show that the proposed inference-time probing method offers a reliable metric for assessing the confidence of reward predictions, which ultimately improves the alignment of LLMs.

Method: Proposed SerenQA framework with serendipity metric (relevance, novelty, surprise), expert-annotated benchmark from Clinical Knowledge Graph, and structured pipeline with three subtasks: knowledge retrieval, subgraph reasoning, and serendipity exploration.

Result: State-of-the-art LLMs perform well on retrieval but struggle to identify genuinely surprising and valuable discoveries, showing significant room for improvement in serendipity detection.

Conclusion: The study highlights the gap in LLMs’ ability to uncover unexpected insights and provides a framework for evaluating and improving serendipity-aware KGQA systems.

Abstract: Large Language Models (LLMs) have greatly advanced knowledge graph question answering (KGQA), yet existing systems are typically optimized for returning highly relevant but predictable answers. A missing yet desired capacity is to exploit LLMs to suggest surprise and novel (“serendipitious”) answers. In this paper, we formally define the serendipity-aware KGQA task and propose the SerenQA framework to evaluate LLMs’ ability to uncover unexpected insights in scientific KGQA tasks. SerenQA includes a rigorous serendipity metric based on relevance, novelty, and surprise, along with an expert-annotated benchmark derived from the Clinical Knowledge Graph, focused on drug repurposing. Additionally, it features a structured evaluation pipeline encompassing three subtasks: knowledge retrieval, subgraph reasoning, and serendipity exploration. Our experiments reveal that while state-of-the-art LLMs perform well on retrieval, they still struggle to identify genuinely surprising and valuable discoveries, underscoring a significant room for future improvements. Our curated resources and extended version are released at: https://cwru-db-group.github.io/serenQA.

Method: Built on 2B-parameter Granite-3.3-2B-Instruct model with instruction tuning on ~1.4M training instances. Uses two specialized components: ContentFilter trained on MLCommons hazard taxonomy and JailbreakFilter trained with curriculum learning covering 60 attack types.

Result: Achieves state-of-the-art safety performance on public and proprietary benchmarks while remaining lightweight. Provides multi-class safety predictions with binary confidence scores for improved interpretability.

Conclusion: SGuard-v1 offers an effective, lightweight safety solution for LLMs that reduces deployment overhead and improves interpretability, released under Apache-2.0 License to enable further research and deployment.

Abstract: We present SGuard-v1, a lightweight safety guardrail for Large Language Models (LLMs), which comprises two specialized models to detect harmful content and screen adversarial prompts in human-AI conversational settings. The first component, ContentFilter, is trained to identify safety risks in LLM prompts and responses in accordance with the MLCommons hazard taxonomy, a comprehensive framework for trust and safety assessment of AI. The second component, JailbreakFilter, is trained with a carefully designed curriculum over integrated datasets and findings from prior work on adversarial prompting, covering 60 major attack types while mitigating false-unsafe classification. SGuard-v1 is built on the 2B-parameter Granite-3.3-2B-Instruct model that supports 12 languages. We curate approximately 1.4 million training instances from both collected and synthesized data and perform instruction tuning on the base model, distributing the curated data across the two component according to their designated functions. Through extensive evaluation on public and proprietary safety benchmarks, SGuard-v1 achieves state-of-the-art safety performance while remaining lightweight, thereby reducing deployment overhead. SGuard-v1 also improves interpretability for downstream use by providing multi-class safety predictions and their binary confidence scores. We release the SGuard-v1 under the Apache-2.0 License to enable further research and practical deployment in AI safety.

Method: Defines 9 question templates covering explicit syntactical and implicit contextual roles for nouns, producing interpretable QA pairs that integrate with QA-SRL for unified sentence decomposition.

Result: Achieves near-complete coverage of AMR’s noun arguments while surfacing additional contextual relations. Combined with QA-SRL, yields over 130% higher granularity than FactScore and DecompScore.

Conclusion: QA-Noun complements the broader QA-based semantic framework, providing a comprehensive and scalable approach to fine-grained semantic decomposition for cross-text alignment.

Abstract: Decomposing sentences into fine-grained meaning units is increasingly used to model semantic alignment. While QA-based semantic approaches have shown effectiveness for representing predicate-argument relations, they have so far left noun-centered semantics largely unaddressed. We introduce QA-Noun, a QA-based framework for capturing noun-centered semantic relations. QA-Noun defines nine question templates that cover both explicit syntactical and implicit contextual roles for nouns, producing interpretable QA pairs that complement verbal QA-SRL. We release detailed guidelines, a dataset of over 2,000 annotated noun mentions, and a trained model integrated with QA-SRL to yield a unified decomposition of sentence meaning into individual, highly fine-grained, facts. Evaluation shows that QA-Noun achieves near-complete coverage of AMR’s noun arguments while surfacing additional contextually implied relations, and that combining QA-Noun with QA-SRL yields over 130% higher granularity than recent fact-based decomposition methods such as FactScore and DecompScore. QA-Noun thus complements the broader QA-based semantic framework, forming a comprehensive and scalable approach to fine-grained semantic decomposition for cross-text alignment.

Method: Uses intent-driven routing to domain-specific extraction templates, supervised fine-tuning, and reinforcement learning-based implicit extraction for concise, coherent knowledge integration.

Result: Outperforms existing methods on six public benchmarks and real-world NowNewsQA across three backbone models, showing strong generalization across domains and long-text tasks.

Conclusion: TAdaRAG effectively addresses RAG limitations through task-adaptive knowledge graph construction, demonstrating practical effectiveness and strong generalization capabilities.

Abstract: Retrieval-Augmented Generation (RAG) improves large language models by retrieving external knowledge, often truncated into smaller chunks due to the input context window, which leads to information loss, resulting in response hallucinations and broken reasoning chains. Moreover, traditional RAG retrieves unstructured knowledge, introducing irrelevant details that hinder accurate reasoning. To address these issues, we propose TAdaRAG, a novel RAG framework for on-the-fly task-adaptive knowledge graph construction from external sources. Specifically, we design an intent-driven routing mechanism to a domain-specific extraction template, followed by supervised fine-tuning and a reinforcement learning-based implicit extraction mechanism, ensuring concise, coherent, and non-redundant knowledge integration. Evaluations on six public benchmarks and a real-world business benchmark (NowNewsQA) across three backbone models demonstrate that TAdaRAG outperforms existing methods across diverse domains and long-text tasks, highlighting its strong generalization and practical effectiveness.

Method: Uses counterfactual data augmentation with two types of response pairs: length-divergent pairs with similar content, and content-divergent pairs of similar length, to train reward models to assess content quality independently of verbosity.

Result: Empirical evaluations show reduced length bias in reward assignment and more concise, content-focused outputs from policy models.

Conclusion: The approach effectively reduces length bias and improves robustness and content sensitivity of reward modeling in RLHF pipelines.

Abstract: Reinforcement learning from human feedback (RLHF) is widely used to align large language models (LLMs) with human preferences. However, RLHF-trained reward models often exhibit length bias – a systematic tendency to favor longer responses by conflating verbosity with quality. We propose a causal framework for analyzing and mitigating length bias in RLHF reward modeling. Central to our approach is a counterfactual data augmentation method that generates response pairs designed to isolate content quality from verbosity. These counterfactual examples are then used to train the reward model, enabling it to assess responses based on content quality independently of verbosity. Specifically, we construct (1) length-divergent pairs with similar content and (2) content-divergent pairs of similar length. Empirical evaluations show that our method reduces length bias in reward assignment and leads to more concise, content-focused outputs from the policy model. These findings demonstrate that the proposed approach effectively reduces length bias and improves the robustness and content sensitivity of reward modeling in RLHF pipelines.

Method: 1) Developed a web-style GUI as front-end, 2) Created automated script to convert dialogue states and system actions into GUI operation instructions, 3) Collected web page snapshots with corresponding operation instructions, 4) Proposed MATE multimodal baseline model.

Result: Created MMWOZ dataset with GUI interactions and established baseline performance using the MATE model for multimodal task-oriented dialogue.

Conclusion: The work successfully addresses the practical gap in task-oriented dialogue systems by introducing multimodal capabilities and provides a foundation for developing more realistic dialogue agents that can interact with GUI interfaces.

Abstract: Task-oriented dialogue systems have garnered significant attention due to their conversational ability to accomplish goals, such as booking airline tickets for users. Traditionally, task-oriented dialogue systems are conceptualized as intelligent agents that interact with users using natural language and have access to customized back-end APIs. However, in real-world scenarios, the widespread presence of front-end Graphical User Interfaces (GUIs) and the absence of customized back-end APIs create a significant gap for traditional task-oriented dialogue systems in practical applications. In this paper, to bridge the gap, we collect MMWOZ, a new multimodal dialogue dataset that is extended from MultiWOZ 2.3 dataset. Specifically, we begin by developing a web-style GUI to serve as the front-end. Next, we devise an automated script to convert the dialogue states and system actions from the original dataset into operation instructions for the GUI. Lastly, we collect snapshots of the web pages along with their corresponding operation instructions. In addition, we propose a novel multimodal model called MATE (Multimodal Agent for Task-oriEnted dialogue) as the baseline model for the MMWOZ dataset. Furthermore, we conduct comprehensive experimental analysis using MATE to investigate the construction of a practical multimodal agent for task-oriented dialogue.

Method: GAPO extends GRPO by computing rewards over the group as a whole, enabling learning from group-level properties like diversity and coverage. Uses frequency-aware reward function to encourage uniform sampling over valid completions.

Result: GAPO-trained models produce valid and more diverse responses. Improves response diversity without compromising accuracy on standard benchmarks (GSM8K, MATH, HumanEval, MMLU-Pro).

Conclusion: GAPO effectively addresses mode collapse in LLMs by leveraging group-level optimization, enhancing diversity while maintaining performance across various tasks.

Abstract: Large Language Models (LLMs) often suffer from mode collapse, repeatedly generating the same few completions even when many valid answers exist, limiting their diversity across a wide range of tasks. We introduce Group-Aware Policy Optimization (GAPO), a simple extension of the recent and popular Group Relative Policy Optimization (GRPO) that computes rewards over the group as a whole. GAPO enables learning from the group-level properties such as diversity and coverage. We demonstrate GAPO using a frequency-aware reward function that encourages uniform sampling over valid LLM completions, and show that GAPO-trained models produce valid and more diverse model responses. Beyond this setup, GAPO generalizes to open-ended prompts and improves response diversity without compromising accuracy on standard LLM benchmarks (GSM8K, MATH, HumanEval, MMLU-Pro). Our code will be made publicly available.

Method: Built on Qwen2.5-7B architecture with dynamic-capacity MoE design, Omni-Modality 3D RoPE for cross-modality alignment, progressive training with iterative GSPO-DPO reinforcement, and trained on 75B tokens of multimodal data with special speech/image generation tokens.

Result: Achieves SOTA or highly competitive performance on 85 benchmarks, surpassing Qwen2.5-Omni on over 50 of 76 benchmarks, with key strengths in video understanding (+7%), omnimodal understanding (+7%), audiovisual reasoning (+4%), speech processing (4.2% WER reduction), and image processing/generation.

Conclusion: Uni-MoE 2.0 demonstrates that carefully designed MoE architectures with progressive training strategies can achieve state-of-the-art multimodal performance with significantly less training data, making high-quality omnimodal AI more accessible through open-source release.

Abstract: We present Uni-MoE 2.0 from the Lychee family. As a fully open-source omnimodal large model (OLM), it substantially advances Lychee’s Uni-MoE series in language-centric multimodal understanding, reasoning, and generating. Based on the Qwen2.5-7B dense architecture, we build Uni-MoE-2.0-Omni from scratch through three core contributions: dynamic-capacity Mixture-of-Experts (MoE) design, a progressive training strategy enhanced with an iterative reinforcement strategy, and a carefully curated multimodal data matching technique. It is capable of omnimodal understanding, as well as generating images, text, and speech. Architecturally, our new MoE framework balances computational efficiency and capability for 10 cross-modal inputs using shared, routed, and null experts, while our Omni-Modality 3D RoPE ensures spatio-temporal cross-modality alignment in the self-attention layer. For training, following cross-modal pretraining, we use a progressive supervised fine-tuning strategy that activates modality-specific experts and is enhanced by balanced data composition and an iterative GSPO-DPO method to stabilise RL training and improve reasoning. Data-wise, the base model, trained on approximately 75B tokens of open-source multimodal data, is equipped with special speech and image generation tokens, allowing it to learn these generative tasks by conditioning its outputs on linguistic cues. Extensive evaluation across 85 benchmarks demonstrates that our model achieves SOTA or highly competitive performance against leading OLMs, surpassing Qwen2.5-Omni (trained with 1.2T tokens) on over 50 of 76 benchmarks. Key strengths include video understanding (+7% avg. of 8), omnimodallity understanding (+7% avg. of 4), and audiovisual reasoning (+4%). It also advances long-form speech processing (reducing WER by 4.2%) and leads in low-level image processing and controllable generation across 5 metrics.

Method: Proposed NOTAM semantic parsing task emphasizing semantic inference and aviation domain knowledge integration. Constructed Knots dataset with 12,347 expert-annotated NOTAMs using multi-agent collaborative framework. Evaluated prompt-engineering strategies and model-adaptation techniques.

Result: Achieved substantial improvements in aviation text understanding and processing through systematic evaluation of various parsing approaches. Demonstrated effectiveness of the proposed semantic parsing method.

Conclusion: The approach successfully addresses the gap in deep semantic understanding of NOTAMs and provides valuable insights for automated NOTAM analysis systems, with publicly available code for further research.

Abstract: Notice to Air Missions (NOTAMs) serve as a critical channel for disseminating key flight safety information, yet their complex linguistic structures and implicit reasoning pose significant challenges for automated parsing. Existing research mainly focuses on surface-level tasks such as classification and named entity recognition, lacking deep semantic understanding. To address this gap, we propose NOTAM semantic parsing, a task emphasizing semantic inference and the integration of aviation domain knowledge to produce structured, inference-rich outputs. To support this task, we construct Knots (Knowledge and NOTAM Semantics), a high-quality dataset of 12,347 expert-annotated NOTAMs covering 194 Flight Information Regions, enhanced through a multi-agent collaborative framework for comprehensive field discovery. We systematically evaluate a wide range of prompt-engineering strategies and model-adaptation techniques, achieving substantial improvements in aviation text understanding and processing. Our experimental results demonstrate the effectiveness of the proposed approach and offer valuable insights for automated NOTAM analysis systems. Our code is available at: https://github.com/Estrellajer/Knots.

Method: Proposes Reason-KE++ framework combining SFT with RL and stage-aware reward mechanism that provides dense supervision for intermediate reasoning steps like decomposition and sub-answer correctness.

Result: Achieves new SOTA of 95.48% on MQUAKE-CF-3k (+5.28% improvement), demonstrating that naive outcome-only RL collapses reasoning integrity while process-aware alignment boosts performance.

Conclusion: For complex multi-hop reasoning tasks, aligning the reasoning process is essential for building trustworthy LLMs, as process-level faithfulness prevents factual hallucinations and ensures sound reasoning.

Abstract: Aligning Large Language Models (LLMs) to be faithful to new knowledge in complex, multi-hop reasoning tasks is a critical, yet unsolved, challenge. We find that SFT-based methods, e.g., Reason-KE, while state-of-the-art, suffer from a “faithfulness gap”: they optimize for format mimicry rather than sound reasoning. This gap enables the LLM’s powerful parametric priors to override new contextual facts, resulting in critical factual hallucinations (e.g., incorrectly reasoning “Houston” from “NASA” despite an explicit edit). To solve this core LLM alignment problem, we propose Reason-KE++, an SFT+RL framework that instills process-level faithfulness. Its core is a Stage-aware Reward mechanism that provides dense supervision for intermediate reasoning steps (e.g., Decomposition, Sub-answer Correctness). Crucially, we identify that naive outcome-only RL is a deceptive trap for LLM alignment: it collapses reasoning integrity (e.g., 19.00% Hop acc) while superficially boosting final accuracy. Our process-aware framework sets a new SOTA of 95.48% on MQUAKE-CF-3k (+5.28%), demonstrating that for complex tasks, aligning the reasoning process is essential for building trustworthy LLMs.

Method: Three-component model: conformer-based encoder from self-supervised pre-training, causal transformer decoder with relative attention, and unit-based neural vocoder. Synthetic data generation pipeline using LLM translation and zero-shot TTS.

Result: Achieved 4.6 ASR BLEU improvement on Persian-English CVSS corpus with synthetic data, increasing available parallel speech by 6x.

Conclusion: Combining self-supervised pre-training, discrete speech units, and synthetic parallel data is effective for improving direct S2ST in low-resource language pairs.

Abstract: Direct speech-to-speech translation (S2ST), in which all components are trained jointly, is an attractive alternative to cascaded systems because it offers a simpler pipeline and lower inference latency. However, direct S2ST models require large amounts of parallel speech data in the source and target languages, which are rarely available for low-resource languages such as Persian. This paper presents a direct S2ST system for translating Persian speech into English speech, as well as a pipeline for synthetic parallel Persian-English speech generation. The model comprises three components: (1) a conformer-based encoder, initialized from self-supervised pre-training, maps source speech to high-level acoustic representations; (2) a causal transformer decoder with relative position multi-head attention translates these representations into discrete target speech units; (3) a unit-based neural vocoder generates waveforms from the predicted discrete units. To mitigate the data scarcity problem, we construct a new Persian-English parallel speech corpus by translating Persian speech transcriptions into English using a large language model and then synthesizing the corresponding English speech with a state-of-the-art zero-shot text-to-speech system. The resulting corpus increases the amount of available parallel speech by roughly a factor of six. On the Persian-English portion of the CVSS corpus, the proposed model achieves improvement of 4.6 ASR BLEU with the synthetic data over direct baselines. These results indicate that combining self-supervised pre-training, discrete speech units, and synthetic parallel data is effective for improving direct S2ST in low-resource language pairs such as Persian-English

Method: Multi-agent system that autonomously engineers, evolves, and executes novel code-based attack algorithms with a code-level self-correction loop for iterative rewriting of attack logic.

Result: Achieved 85.5% Attack Success Rate against Claude-Sonnet-4.5 and generated significantly more diverse attacks than existing methods.

Conclusion: EvoSynth establishes a new state-of-the-art and opens a new direction for evolutionary synthesis of jailbreak methods.

Abstract: Automated red teaming frameworks for Large Language Models (LLMs) have become increasingly sophisticated, yet they share a fundamental limitation: their jailbreak logic is confined to selecting, combining, or refining pre-existing attack strategies. This binds their creativity and leaves them unable to autonomously invent entirely new attack mechanisms. To overcome this gap, we introduce \textbf{EvoSynth}, an autonomous framework that shifts the paradigm from attack planning to the evolutionary synthesis of jailbreak methods. Instead of refining prompts, EvoSynth employs a multi-agent system to autonomously engineer, evolve, and execute novel, code-based attack algorithms. Crucially, it features a code-level self-correction loop, allowing it to iteratively rewrite its own attack logic in response to failure. Through extensive experiments, we demonstrate that EvoSynth not only establishes a new state-of-the-art by achieving an 85.5% Attack Success Rate (ASR) against highly robust models like Claude-Sonnet-4.5, but also generates attacks that are significantly more diverse than those from existing methods. We release our framework to facilitate future research in this new direction of evolutionary synthesis of jailbreak methods. Code is available at: https://github.com/dongdongunique/EvoSynth.

Method: AFM assigns each past message one of three fidelity levels (FULL, COMPRESSED, PLACEHOLDER) based on semantic similarity to current query, recency weighting, and importance classification, packing messages chronologically under token budget constraints.

Result: In safety benchmarks with peanut allergy scenarios, AFM retains allergy information across conversations, matches naive replay’s safety performance, and reduces average token usage by 66%.

Conclusion: AFM enables significant cost reduction in LLM inference without sacrificing safety or factual continuity, with modular implementation available for OpenAI-compatible APIs.

Abstract: Large language models (LLMs) are increasingly deployed in multi-turn dialogue settings, but their behavior is still bottlenecked by fixed context windows and naive memory strategies. Replaying the full conversation at every turn is simple but expensive, while static summarization or recency-only heuristics often erase safety-critical user details. We present Adaptive Focus Memory (AFM), a dynamic context manager that assigns each past message one of three fidelity levels – FULL, COMPRESSED, or PLACEHOLDER – based on semantic similarity to the current query, half-life recency weighting, and importance classification. AFM packs messages chronologically under a strict token budget, preferring high fidelity for the most relevant turns while aiming to preserve a cheap trace of the dialogue. In a safety-oriented benchmark involving a user with a severe peanut allergy planning a trip to Thailand, AFM retains the allergy across both short and medium-length conversations, matches the safety performance of naive replay, and cuts average token usage by 66% relative to a replay baseline. We release a modular Python implementation of AFM designed for OpenAI-compatible APIs and offline operation, enabling practitioners to reduce inference cost without sacrificing safety or factual continuity in the evaluated scenario.

Method: Systematic empirical evaluation of set membership queries across prompt phrasing, semantic structure, element ordering, and model choice using large-scale controlled experiments.

Result: LLM performance on elementary set membership tasks is consistently brittle and unpredictable across all dimensions, indicating fragmented and convoluted understanding of basic set concepts.

Conclusion: The simplicity of set membership problems enables comprehensive mapping of LLM failure modes, making this approach a valuable methodology for general LLM evaluation.

Abstract: Large language models (LLMs) achieve superhuman performance on complex reasoning tasks, yet often fail on much simpler problems, raising concerns about their reliability and interpretability. We investigate this paradox through a focused study with two key design features: simplicity, to expose basic failure modes, and scale, to enable comprehensive controlled experiments. We focus on set membership queries – among the most fundamental forms of reasoning – using tasks like Is apple an element of the set \{pear, plum, apple, raspberry\}?''. We conduct a systematic empirical evaluation across prompt phrasing, semantic structure, element ordering, and model choice. Our large-scale analysis reveals that LLM performance on this elementary task is consistently brittle, and unpredictable across all dimensions, suggesting that the models' understanding’’ of the set concept is fragmented and convoluted at best. Our work demonstrates that the large-scale experiments enabled by the simplicity of the problem allow us to map and analyze the failure modes comprehensively, making this approach a valuable methodology for LLM evaluation in general.

Method: Train a small GPT-style transformer on character-level corpus, track vocabulary usage evolution (average word length, correct/incorrect words, vocabulary diversity), and apply Poisson/sub-Poisson statistics to quantify word connections and reorganization.

Result: Reveals distinct transition points during training not visible in standard loss/validation curves but detectable through vocabulary- and statistics-based probes, showing phase transitions occur even in modest models.

Conclusion: Phase-transition reorganizations are a general feature of language model training, observable in small models, detectable in linear space, and occurring early as coherence emerges, providing new insights into nonlinear training dynamics.

Abstract: Phase transitions have been proposed as the origin of emergent abilities in large language models (LLMs), where new capabilities appear abruptly once models surpass critical thresholds of scale. Prior work, such as that of Wei et al., demonstrated these phenomena under model and data scaling, with transitions revealed after applying a log scale to training compute. In this work, we ask three complementary questions: (1) Are phase transitions unique to large models, or can they also be observed in small transformer-based language models? (2) Can such transitions be detected directly in linear training space, rather than only after log rescaling? and (3) Can these transitions emerge at early stages of training? To investigate, we train a small GPT-style transformer on a character-level corpus and analyze the evolution of vocabulary usage throughout training. We track the average word length, the number of correct versus incorrect words, and shifts in vocabulary diversity. Building on these measures, we apply Poisson and sub-Poisson statistics to quantify how words connect and reorganize. This combined analysis reveals a distinct transition point during training. Notably, these transitions are not apparent in standard loss or validation curves, but become visible through our vocabulary- and statistics-based probes. Our findings suggest that phase-transition reorganizations are a general feature of language model training, observable even in modest models, detectable directly in linear training space, and occurring surprisingly early as coherence emerges. This perspective provides new insight into the nonlinear dynamics of language model training and underscores the importance of tailored metrics for uncovering phase transition behaviors

Method: Adding interruptions (control sentences) to user input approximately every x tokens, which can be generalized to Chain-of-Thought processes to prevent scheming behavior.

Result: The paper proposes this method but does not provide experimental results in the abstract.

Conclusion: Interruptions offer a potential scalable solution to strengthen LLM alignment against jailbreaking as input length increases.

Abstract: Current Large Language Model alignment research mostly focuses on improving model robustness against adversarial attacks and misbehavior by training on examples and prompting. Research has shown that LLM jailbreak probability increases with the size of the user input or conversation length. There is a lack of appropriate research into means of strengthening alignment which also scale with user input length. We propose interruptions as a possible solution to this problem. Interruptions are control sentences added to the user input approximately every x tokens for some arbitrary x. We suggest that this can be generalized to the Chain-of-Thought process to prevent scheming.

Method: Evaluated LLM robustness by generating paraphrased natural language statements from MiniF2F and Lean 4 ProofNet benchmarks, measuring semantic and compilation validity across two modern LLMs through cross-evaluation.

Result: Found significant performance variability across paraphrased inputs, demonstrating that minor shifts in natural language statements can substantially impact model outputs.

Conclusion: LLMs remain sensitive to paraphrased natural language inputs in autoformalization tasks, highlighting robustness challenges despite their impressive capabilities.

Abstract: Large Language Models (LLMs) have recently emerged as powerful tools for autoformalization. Despite their impressive performance, these models can still struggle to produce grounded and verifiable formalizations. Recent work in text-to-SQL, has revealed that LLMs can be sensitive to paraphrased natural language (NL) inputs, even when high degrees of semantic fidelity are preserved (Safarzadeh, Oroojlooyjadid, and Roth 2025). In this paper, we investigate this claim in the autoformalization domain. Specifically, we evaluate the robustness of LLMs generating formal proofs with semantically similar paraphrased NL statements by measuring semantic and compilation validity. Using the formal benchmarks MiniF2F (Zheng, Han, and Polu 2021) and Lean 4 version of ProofNet (Xin et al. 2024), and two modern LLMs, we generate paraphrased natural language statements and cross-evaluate these statements across both models. The results of this paper reveal performance variability across paraphrased inputs, demonstrating that minor shifts in NL statements can significantly impact model outputs.

Method: Built from 1.74M PubMed Central articles across 2,744 journals, using bibliometric indicators, collaboration features, and a reproducible three-stage LLM pipeline for AI engagement extraction.

Result: Journals with higher collaboration intensity achieve greater citation impact, and AI engagement has become an increasingly strong correlate of journal prestige, especially in quartile rankings.

Conclusion: BioMedJImpact serves as both a comprehensive dataset and validated methodological framework for content-aware scientometric analysis of scientific impact and innovation dynamics in biomedicine.

Abstract: Assessing journal impact is central to scholarly communication, yet existing open resources rarely capture how collaboration structures and artificial intelligence (AI) research jointly shape venue prestige in biomedicine. We present BioMedJImpact, a large-scale, biomedical-oriented dataset designed to advance journal-level analysis of scientific impact and AI engagement. Built from 1.74 million PubMed Central articles across 2,744 journals, BioMedJImpact integrates bibliometric indicators, collaboration features, and LLM-derived semantic indicators for AI engagement. Specifically, the AI engagement feature is extracted through a reproducible three-stage LLM pipeline that we propose. Using this dataset, we analyze how collaboration intensity and AI engagement jointly influence scientific impact across pre- and post-pandemic periods (2016-2019, 2020-2023). Two consistent trends emerge: journals with higher collaboration intensity, particularly those with larger and more diverse author teams, tend to achieve greater citation impact, and AI engagement has become an increasingly strong correlate of journal prestige, especially in quartile rankings. To further validate the three-stage LLM pipeline we proposed for deriving the AI engagement feature, we conduct human evaluation, confirming substantial agreement in AI relevance detection and consistent subfield classification. Together, these contributions demonstrate that BioMedJImpact serves as both a comprehensive dataset capturing the intersection of biomedicine and AI, and a validated methodological framework enabling scalable, content-aware scientometric analysis of scientific impact and innovation dynamics. Code is available at https://github.com/JonathanWry/BioMedJImpact.

Method: Used attribution patching to identify causally influential components, derived emotional expression vectors from activation differences between contrastive text pairs (positive vs. negative emotional examples), and applied these vectors to conversational prompts.

Result: Steered responses showed increased positive sentiment (joy, trust) and more frequent first-person pronoun usage, indicating greater personal engagement and enhanced emotional characteristics.

Conclusion: The approach provides a precise and interpretable framework for emotional steering in conversational AI, offering new directions for studying and implementing human-like emotional expression in language models.

Abstract: Large Language Models (LLMs) demonstrate increasing conversational fluency, yet instilling them with nuanced, human-like emotional expression remains a significant challenge. Current alignment techniques often address surface-level output or require extensive fine-tuning. This paper demonstrates that targeted activation engineering can steer LLaMA 3.1-8B to exhibit more human-like emotional nuances. We first employ attribution patching to identify causally influential components, to find a key intervention locus by observing activation patterns during diagnostic conversational tasks. We then derive emotional expression vectors from the difference in the activations generated by contrastive text pairs (positive vs. negative examples of target emotions). Applying these vectors to new conversational prompts significantly enhances emotional characteristics: steered responses show increased positive sentiment (e.g., joy, trust) and more frequent first-person pronoun usage, indicative of greater personal engagement. Our findings offer a precise and interpretable framework and new directions for the study of conversational AI.

Method: Defined a new stability measure combining accuracy and consistency, used LDA’s generative properties to create corpora with ground truth topics, and ran LDA 50 times on generated corpora to analyze output variability.

Result: LDA can correctly determine the underlying number of topics but is more internally consistent than accurate - multiple reruns return similar topics that don’t match the true underlying topics.

Conclusion: LDA demonstrates internal consistency across runs but fails to recover true topics, suggesting it may capture patterns that are consistent but not necessarily meaningful or accurate representations of the underlying content.

Abstract: Topic modelling in Natural Language Processing uncovers hidden topics in large, unlabelled text datasets. It is widely applied in fields such as information retrieval, content summarisation, and trend analysis across various disciplines. However, probabilistic topic models can produce different results when rerun due to their stochastic nature, leading to inconsistencies in latent topics. Factors like corpus shuffling, rare text removal, and document elimination contribute to these variations. This instability affects replicability, reliability, and interpretation, raising concerns about whether topic models capture meaningful topics or just noise. To address these problems, we defined a new stability measure that incorporates accuracy and consistency and uses the generative properties of LDA to generate a new corpus with ground truth. These generated corpora are run through LDA 50 times to determine the variability in the output. We show that LDA can correctly determine the underlying number of topics in the documents. We also find that LDA is more internally consistent, as the multiple reruns return similar topics; however, these topics are not the true topics.

Method: Used span-corruption pretraining and instruction-style fine-tuning on EEG report polishing, paragraph summarization, and terminology question answering from the Harvard EEG Database.

Result: Achieves lower perplexity, higher extraction/summarization accuracy, better label efficiency, and improved robustness to negation and factual hallucination compared to general models of same scale.

Conclusion: NeuroLex bridges biomedical text modeling and brain-computer interface applications, providing a foundation for interpretable and language-driven neural decoding.

Abstract: Clinical electroencephalogram (EEG) reports encode domain-specific linguistic conventions that general-purpose language models (LMs) fail to capture. We introduce NeuroLex, a lightweight domain-adaptive language model trained purely on EEG report text from the Harvard Electroencephalography Database. Unlike existing biomedical LMs, NeuroLex is tailored to the linguistic and diagnostic characteristics of EEG reporting, enabling it to serve as both an independent textual model and a decoder backbone for multimodal EEG-language systems. Using span-corruption pretraining and instruction-style fine-tuning on report polishing, paragraph summarization, and terminology question answering, NeuroLex learns the syntax and reasoning patterns characteristic of EEG interpretation. Comprehensive evaluations show that it achieves lower perplexity, higher extraction and summarization accuracy, better label efficiency, and improved robustness to negation and factual hallucination compared with general models of the same scale. With an EEG-aware linguistic backbone, NeuroLex bridges biomedical text modeling and brain-computer interface applications, offering a foundation for interpretable and language-driven neural decoding.

Method: Systematic review approach analyzing MCoT from three aspects: CoT paradigms, post-training stage, and inference stage, while examining underlying mechanisms and organizing existing research systematically.

Result: Comprehensive analysis of MCoT’s theoretical foundations, methodological approaches, evaluation benchmarks, application scenarios, and identification of current challenges in the field.

Conclusion: MCoT shows promise for enhancing multimodal reasoning capabilities but faces challenges that require future research directions to address current limitations and advance the field.

Abstract: With the remarkable success of Multimodal Large Language Models (MLLMs) in perception tasks, enhancing their complex reasoning capabilities has emerged as a critical research focus. Existing models still suffer from challenges such as opaque reasoning paths and insufficient generalization ability. Chain-of-Thought (CoT) reasoning, which has demonstrated significant efficacy in language models by enhancing reasoning transparency and output interpretability, holds promise for improving model reasoning capabilities when extended to the multimodal domain. This paper provides a systematic review centered on “Multimodal Chain-of-Thought” (MCoT). First, it analyzes the background and theoretical motivations for its inception from the perspectives of technical evolution and task demands. Then, it introduces mainstream MCoT methods from three aspects: CoT paradigms, the post-training stage, and the inference stage, while also analyzing their underlying mechanisms. Furthermore, the paper summarizes existing evaluation benchmarks and metrics, and discusses the application scenarios of MCoT. Finally, it analyzes the challenges currently facing MCoT and provides an outlook on its future research directions.

Method: Developed and compared three transformer architectures (BERT, GPT-2, DeBERTa) for binary and multiclass hope classification tasks.

Result: BERT achieved best performance (84.49% binary, 72.03% multiclass accuracy) with lowest computational cost. GPT-2 had lowest accuracy but excelled at sarcasm detection (92.46% recall).

Conclusion: Architectural suitability may outweigh model size for specialized emotion detection tasks, providing a framework for computational hope analysis.

Abstract: This paper presents a transformer-based approach for classifying hope expressions in text. We developed and compared three architectures (BERT, GPT-2, and DeBERTa) for both binary classification (Hope vs. Not Hope) and multiclass categorization (five hope-related categories). Our initial BERT implementation achieved 83.65% binary and 74.87% multiclass accuracy. In the extended comparison, BERT demonstrated superior performance (84.49% binary, 72.03% multiclass accuracy) while requiring significantly fewer computational resources (443s vs. 704s training time) than newer architectures. GPT-2 showed lowest overall accuracy (79.34% binary, 71.29% multiclass), while DeBERTa achieved moderate results (80.70% binary, 71.56% multiclass) but at substantially higher computational cost (947s for multiclass training). Error analysis revealed architecture-specific strengths in detecting nuanced hope expressions, with GPT-2 excelling at sarcasm detection (92.46% recall). This study provides a framework for computational analysis of hope, with applications in mental health and social media analysis, while demonstrating that architectural suitability may outweigh model size for specialized emotion detection tasks.

Method: Systematic algorithm audit of 1,508 baby care and pregnancy queries using a robust evaluation framework assessing answer consistency, relevance, medical safeguards, source categories, and sentiment alignment.

Result: 33% inconsistency between AIO and FS on same page; high relevance but critically lacking medical safeguards (11% AIO, 7% FS); health/wellness websites dominate sources but FS often link to commercial sources.

Conclusion: Concerning gaps in AI-mediated health information quality demonstrate need for stronger controls; methodology provides transferable framework for auditing AI systems in high-stakes domains impacting user well-being.

Abstract: Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no control over their presentation. Through a systematic algorithm audit of 1,508 real baby care and pregnancy-related queries, we evaluate the quality and consistency of these information displays. Our robust evaluation framework assesses multiple quality dimensions, including answer consistency, relevance, presence of medical safeguards, source categories, and sentiment alignment. Our results reveal concerning gaps in information consistency, with information in AIO and FS displayed on the same search result page being inconsistent with each other in 33% of cases. Despite high relevance scores, both features critically lack medical safeguards (present in just 11% of AIO and 7% of FS responses). While health and wellness websites dominate source categories for both, AIO and FS, FS also often link to commercial sources. These findings have important implications for public health information access and demonstrate the need for stronger quality controls in AI-mediated health information. Our methodology provides a transferable framework for auditing AI systems across high-stakes domains where information quality directly impacts user well-being.

Method: Created Visual Room 2.0 benchmark with 350 multi-modal samples and 2,100 questions across three hierarchical levels (low, middle, high) covering perception tasks (attribute recognition to scene understanding) and cognition tasks (textual entailment to causal/social reasoning).

Result: Evaluation of 10 SoTA MLLMs showed: (1) 8.0% higher perceptual than cognitive competence, (2) cognition not causally dependent on perception-based reasoning, (3) cognition scales with model size but perception doesn’t consistently improve with larger variants.

Conclusion: The work operationalizes ‘Seeing ≠ Understanding’ as a testable hypothesis and provides a new paradigm for evaluating MLLMs from perceptual processing to cognitive reasoning.

Abstract: Can multi-modal large language models (MLLMs) truly understand what they can see? Extending Searle’s Chinese Room into the multi-modal domain, this paper proposes the Visual Room argument: MLLMs may describe every visual detail precisely yet fail to comprehend the underlying emotions and intentions, namely seeing is not understanding. Building on this, we introduce \textit{Visual Room} 2.0, a hierarchical benchmark for evaluating perception-cognition alignment of MLLMs. We model human perceptive and cognitive processes across three levels: low, middle, and high, covering 17 representative tasks. The perception component ranges from attribute recognition to scene understanding, while the cognition component extends from textual entailment to causal and social reasoning. The dataset contains 350 multi-modal samples, each with six progressive questions (2,100 in total) spanning perception to cognition. Evaluating 10 state-of-the-art (SoTA) MLLMs, we highlight three key findings: (1) MLLMs exhibit stronger perceptual competence than cognitive ability (8.0%$\uparrow$); (2) cognition appears not causally dependent on perception-based reasoning; and (3) cognition scales with model size, but perception does not consistently improve with larger variants. This work operationalizes Seeing $\ne$ Understanding as a testable hypothesis, offering a new paradigm from perceptual processing to cognitive reasoning in MLLMs. Our dataset is available at https://huggingface.co/datasets/LHK2003/PCBench.

Method: HCNR identifies honesty-critical neurons that govern expression and restores them to pre-trained state while harmonizing with task-oriented neurons using Hessian-guided compensation.

Result: Recovers 33.25% of compromised honesty across four QA tasks and five LLM families, with at least 2.23x speedup and over 10x less data compared to baselines.

Conclusion: HCNR offers a practical solution for trustworthy LLM deployment by surgically repairing suppressed honesty expression capacity rather than assuming deep corruption.

Abstract: The honesty of Large Language Models (LLMs) is increasingly important for safe deployment in high-stakes domains. However, this crucial trait is severely undermined by supervised fine-tuning (SFT), a common technique for model specialization. Existing recovery methods rely on data-intensive global parameter adjustments, implicitly assuming that SFT deeply corrupts the models’ ability to recognize their knowledge boundaries. However, we observe that fine-tuned LLMs still preserve this ability; what is damaged is their capacity to faithfully express that awareness. Building on this, we propose Honesty-Critical Neurons Restoration (HCNR) to surgically repair this suppressed capacity. HCNR identifies and restores key expression-governing neurons to their pre-trained state while harmonizing them with task-oriented neurons via Hessian-guided compensation. Experiments on four QA tasks and five LLM families demonstrate that HCNR effectively recovers 33.25% of the compromised honesty while achieving at least 2.23x speedup with over 10x less data compared to baseline methods, offering a practical solution for trustworthy LLM deployment.

Method: Created benchmark with 6,000 questions from authoritative sources covering 42 topics in 6 economically relevant domains. Measures Omniscience Index (-100 to 100) that jointly evaluates factual recall while penalizing hallucinations and rewarding abstention when uncertain.

Result: Claude 4.1 Opus achieved highest score (4.8), one of only three models scoring above zero. Performance varies by domain with different labs leading across the six domains, revealing persistent factuality and calibration weaknesses in frontier models.

Conclusion: Models should be chosen based on specific domain requirements rather than general performance for knowledge-intensive tasks, due to significant performance variability across domains.

Abstract: Existing language model evaluations primarily measure general capabilities, yet reliable use of these models across a range of domains demands factual accuracy and recognition of knowledge gaps. We introduce AA-Omniscience, a benchmark designed to measure both factual recall and knowledge calibration across 6,000 questions. Questions are derived from authoritative academic and industry sources, and cover 42 economically relevant topics within six different domains. The evaluation measures a model’s Omniscience Index, a bounded metric (-100 to 100) measuring factual recall that jointly penalizes hallucinations and rewards abstention when uncertain, with 0 equating to a model that answers questions correctly as much as it does incorrectly. Among evaluated models, Claude 4.1 Opus attains the highest score (4.8), making it one of only three models to score above zero. These results reveal persistent factuality and calibration weaknesses across frontier models. Performance also varies by domain, with the models from three different research labs leading across the six domains. This performance variability suggests models should be chosen according to the demands of the use case rather than general performance for tasks where knowledge is important.

Method: Evaluates traditional embedding alignment techniques, novel multilingual models, and combined alignment techniques on BLI tasks across high-resource and low-resource languages. Proposes stem-based BLI approach and vocabulary pruning technique to address limitations of conventional word-based BLI.

Result: Found that BLI doesn’t always measure true alignment accurately. Combined embedding alignment techniques generally perform better, while multilingual embeddings excel mainly in low-resource language scenarios. Language family relationships impact performance.

Conclusion: Multilingual embeddings aren’t universally superior - their advantage is context-dependent, particularly strong for low-resource languages. Proposed novel evaluation methods provide more accurate assessment of embedding space alignment.

Abstract: Sans a dwindling number of monolingual embedding studies originating predominantly from the low-resource domains, it is evident that multilingual embedding has become the de facto choice due to its adaptability to the usage of code-mixed languages, granting the ability to process multilingual documents in a language-agnostic manner, as well as removing the difficult task of aligning monolingual embeddings. But is this victory complete? Are the multilingual models better than aligned monolingual models in every aspect? Can the higher computational cost of multilingual models always be justified? Or is there a compromise between the two extremes? Bilingual Lexicon Induction is one of the most widely used metrics in terms of evaluating the degree of alignment between two embedding spaces. In this study, we explore the strengths and limitations of BLI as a measure to evaluate the degree of alignment of two embedding spaces. Further, we evaluate how well traditional embedding alignment techniques, novel multilingual models, and combined alignment techniques perform BLI tasks in the contexts of both high-resource and low-resource languages. In addition to that, we investigate the impact of the language families to which the pairs of languages belong. We identify that BLI does not measure the true degree of alignment in some cases and we propose solutions for them. We propose a novel stem-based BLI approach to evaluate two aligned embedding spaces that take into account the inflected nature of languages as opposed to the prevalent word-based BLI techniques. Further, we introduce a vocabulary pruning technique that is more informative in showing the degree of the alignment, especially performing BLI on multilingual embedding models. Often, combined embedding alignment techniques perform better while in certain cases multilingual embeddings perform better (mainly low-resource language cases).

Method: Three-stage framework: 1) Continuous pre-training on mathematical corpus with novel data tasks including CoT-augmented state prediction, 2) Supervised Fine-tuning in expert iteration loop, 3) Group Relative Policy Optimization for challenging problems.

Result: Achieves 37.0% average pass rate (pass@32), solves 27 problems on PutnamBench, and achieves 24.0% on CombiBench, demonstrating state-of-the-art performance among similarly-sized models.

Conclusion: The diverse training data and progressively refined training pipeline effectively enhances formal reasoning capabilities of lightweight LLMs.

Abstract: Large Language Models (LLMs) have shown significant promise in automated theorem proving, yet progress is often constrained by the scarcity of diverse and high-quality formal language data. To address this issue, we introduce Spark-Prover-X1, a 7B parameter model trained via an three-stage framework designed to unlock the reasoning potential of more accessible and moderately-sized LLMs. The first stage infuses deep knowledge through continuous pre-training on a broad mathematical corpus, enhanced by a suite of novel data tasks. Key innovation is a “CoT-augmented state prediction” task to achieve fine-grained reasoning. The second stage employs Supervised Fine-tuning (SFT) within an expert iteration loop to specialize both the Spark-Prover-X1-7B and Spark-Formalizer-X1-7B models. Finally, a targeted round of Group Relative Policy Optimization (GRPO) is applied to sharpen the prover’s capabilities on the most challenging problems. To facilitate robust evaluation, particularly on problems from real-world examinations, we also introduce ExamFormal-Bench, a new benchmark dataset of 402 formal problems. Experimental results demonstrate that Spark-Prover-X1-7B achieves state-of-the-art performance among similarly-sized open-source models, attaining a 37.0% average pass rate (pass@32). It shows exceptional performance on difficult competition benchmarks, notably solving 27 problems on PutnamBench (pass@32) and achieving 24.0% on CombiBench (pass@32). Our work validates that this diverse training data and progressively refined training pipeline provides an effective path for enhancing the formal reasoning capabilities of lightweight LLMs. Both Spark-Prover-X1-7B and Spark-Formalizer-X1-7B, along with the ExamFormal-Bench dataset, are made publicly available at:https://www.modelscope.cn/organization/iflytek, https://gitcode.com/ifly_opensource.

Method: Compiled 5 publicly available discourse tasks across different levels (lexicon, sentential, documental) with 52 datasets, including discourse parsing, temporal relation extraction, discourse particle disambiguation, and multilingual discourse relation classification.

Result: State-of-the-art models show strong performance in arithmetic temporal reasoning but struggle with full document reasoning and subtle semantic/discourse phenomena like rhetorical relation recognition.

Conclusion: Current LLMs have limitations in comprehensive discourse understanding, particularly with document-level reasoning and nuanced discourse phenomena, highlighting areas for future improvement.

Abstract: We introduce BeDiscovER (Benchmark of Discourse Understanding in the Era of Reasoning Language Models), an up-to-date, comprehensive suite for evaluating the discourse-level knowledge of modern LLMs. BeDiscovER compiles 5 publicly available discourse tasks across discourse lexicon, (multi-)sentential, and documental levels, with in total 52 individual datasets. It covers both extensively studied tasks such as discourse parsing and temporal relation extraction, as well as some novel challenges such as discourse particle disambiguation (e.g., ``just’’), and also aggregates a shared task on Discourse Relation Parsing and Treebanking for multilingual and multi-framework discourse relation classification. We evaluate open-source LLMs: Qwen3 series, DeepSeek-R1, and frontier model such as GPT-5-mini on BeDiscovER, and find that state-of-the-art models exhibit strong performance in arithmetic aspect of temporal reasoning, but they struggle with full document reasoning and some subtle semantic and discourse phenomena, such as rhetorical relation recognition.

Method: Systematic evaluation of contemporary LLMs using a golden standard dataset of 150 published RCTs across medical specialties, under zero-shot settings. Primary outcome was macro-averaged F1-score for three-class classification.

Result: Overall performance was modest with top models Gemini-2.5-Flash and DeepSeek-R1 achieving macro F1 scores of 0.634. Models performed well on compliant items (F1 > 0.850) but poorly on non-compliant and not applicable items (F1 < 0.400). GPT-4o underperformed with F1-score of 0.521.

Conclusion: LLMs show potential as preliminary screening assistants for identifying well-reported items but cannot reliably detect reporting omissions or methodological flaws, making them unsuitable for replacing human expertise in trial quality appraisal.

Abstract: The Consolidated Standards of Reporting Trials statement is the global benchmark for transparent and high-quality reporting of randomized controlled trials. Manual verification of CONSORT adherence is a laborious, time-intensive process that constitutes a significant bottleneck in peer review and evidence synthesis. This study aimed to systematically evaluate the accuracy and reliability of contemporary LLMs in identifying the adherence of published RCTs to the CONSORT 2010 statement under a zero-shot setting. We constructed a golden standard dataset of 150 published RCTs spanning diverse medical specialties. The primary outcome was the macro-averaged F1-score for the three-class classification task, supplemented by item-wise performance metrics and qualitative error analysis. Overall model performance was modest. The top-performing models, Gemini-2.5-Flash and DeepSeek-R1, achieved nearly identical macro F1 scores of 0.634 and Cohen’s Kappa coefficients of 0.280 and 0.282, respectively, indicating only fair agreement with expert consensus. A striking performance disparity was observed across classes: while most models could identify compliant items with high accuracy (F1 score > 0.850), they struggled profoundly with identifying non-compliant and not applicable items, where F1 scores rarely exceeded 0.400. Notably, some high-profile models like GPT-4o underperformed, achieving a macro F1-score of only 0.521. LLMs show potential as preliminary screening assistants for CONSORT checks, capably identifying well-reported items. However, their current inability to reliably detect reporting omissions or methodological flaws makes them unsuitable for replacing human expertise in the critical appraisal of trial quality.

Method: AEC decomposes event extraction into specialized subtasks handled by dedicated LLM agents: retrieval, planning, coding, and verification. Event schemas are represented as executable class definitions for deterministic validation and iterative refinement.

Result: Experiments across five diverse domains and six LLMs show AEC consistently outperforms prior zero-shot baselines, demonstrating improved precision, completeness, and schema consistency in event extraction.

Conclusion: Treating event extraction as a structured code-generation process through collaborative agent workflows enables LLMs to achieve precise, complete, and schema-consistent extractions in zero-shot settings.

Abstract: Zero-shot event extraction (ZSEE) remains a significant challenge for large language models (LLMs) due to the need for complex reasoning and domain-specific understanding. Direct prompting often yields incomplete or structurally invalid outputs–such as misclassified triggers, missing arguments, and schema violations. To address these limitations, we present Agent-Event-Coder (AEC), a novel multi-agent framework that treats event extraction like software engineering: as a structured, iterative code-generation process. AEC decomposes ZSEE into specialized subtasks–retrieval, planning, coding, and verification–each handled by a dedicated LLM agent. Event schemas are represented as executable class definitions, enabling deterministic validation and precise feedback via a verification agent. This programming-inspired approach allows for systematic disambiguation and schema enforcement through iterative refinement. By leveraging collaborative agent workflows, AEC enables LLMs to produce precise, complete, and schema-consistent extractions in zero-shot settings. Experiments across five diverse domains and six LLMs demonstrate that AEC consistently outperforms prior zero-shot baselines, showcasing the power of treating event extraction like code generation. The code and data are released on https://github.com/UESTC-GQJ/Agent-Event-Coder.

Method: Implemented and evaluated two models - ConvLSTM (recurrent) and Vanilla Transformer (attention-based) on Azerbaijani Sign Language Dataset (AzSLD) and Word-Level American Sign Language (WLASL) dataset.

Result: Vanilla Transformer consistently outperformed ConvLSTM in both Top-1 and Top-5 accuracy across datasets, achieving 76.8% Top-1 accuracy on AzSLD and 88.3% on WLASL. ConvLSTM was more computationally efficient but less accurate.

Conclusion: Transformers excel in accuracy and signer independence while ConvLSTM offers computational efficiency and better temporal modeling, providing guidance for architecture selection based on application requirements and resource constraints.

Abstract: This study presents a systematic comparative analysis of recurrent and attention-based neural architectures for isolated sign language recognition. We implement and evaluate two representative models-ConvLSTM and Vanilla Transformer-on the Azerbaijani Sign Language Dataset (AzSLD) and the Word-Level American Sign Language (WLASL) dataset. Our results demonstrate that the attention-based Vanilla Transformer consistently outperforms the recurrent ConvLSTM in both Top-1 and Top-5 accuracy across datasets, achieving up to 76.8% Top-1 accuracy on AzSLD and 88.3% on WLASL. The ConvLSTM, while more computationally efficient, lags in recognition accuracy, particularly on smaller datasets. These findings highlight the complementary strengths of each paradigm: the Transformer excels in overall accuracy and signer independence, whereas the ConvLSTM offers advantages in computational efficiency and temporal modeling. The study provides a nuanced analysis of these trade-offs, offering guidance for architecture selection in sign language recognition systems depending on application requirements and resource constraints.

Method: Use LLM-generated predictions on unlabeled data with grammar competency rubric prompts as pseudo labels, train transformer model with noise-handling framework.

Result: Model achieves high accuracy in grammar competency estimation; LLM choice and clean-to-noisy sample ratio critically affect performance and stability.

Conclusion: Framework enables scalable, low-resource grammar assessment systems with robust and interpretable performance.

Abstract: Grammar competency estimation is essential for assessing linguistic proficiency in both written and spoken language; however, the spoken modality presents additional challenges due to its spontaneous, unstructured, and disfluent nature. Developing accurate grammar scoring models further requires extensive expert annotation, making large-scale data creation impractical. To address these limitations, we propose a zero-shot grammar competency estimation framework that leverages unlabeled data and Large Language Models (LLMs) without relying on manual labels. During training, we employ LLM-generated predictions on unlabeled data by using grammar competency rubric-based prompts. These predictions, treated as pseudo labels, are utilized to train a transformer-based model through a novel training framework designed to handle label noise effectively. We show that the choice of LLM for pseudo-label generation critically affects model performance and that the ratio of clean-to-noisy samples during training strongly influences stability and accuracy. Finally, a qualitative analysis of error intensity and score prediction confirms the robustness and interpretability of our approach. Experimental results demonstrate the efficacy of our approach in estimating grammar competency scores with high accuracy, paving the way for scalable, low-resource grammar assessment systems.

Method: Created 20,000-row manually annotated Bangla corpus, benchmarked with multilingual LLMs (few-shot prompting) and fine-tuned encoder models (BanglaBERT).

Result: LLMs achieved up to 82.68% accuracy, while fine-tuned BanglaBERT achieved highest accuracy of 84.78% and F1 score of 0.677.

Conclusion: Establishes linguistically-informed baseline for semantic-preserving text normalization in Bangla, with fine-tuning proving superior to LLM prompting.

Abstract: Automatic Speech Recognition (ASR) transcripts, especially in low-resource languages like Bangla, contain a critical ambiguity: word-word repetitions can be either Repetition Disfluency (unintentional ASR error/hesitation) or Morphological Reduplication (a deliberate grammatical construct). Standard disfluency correction fails by erroneously deleting valid linguistic information. To solve this, we introduce the first publicly available, 20,000-row Bangla corpus, manually annotated to explicitly distinguish between these two phenomena in noisy ASR transcripts. We benchmark this novel resource using two paradigms: state-of-the-art multilingual Large Language Models (LLMs) and task-specific fine-tuning of encoder models. LLMs achieve competitive performance (up to 82.68% accuracy) with few-shot prompting. However, fine-tuning proves superior, with the language-specific BanglaBERT model achieving the highest accuracy of 84.78% and an F1 score of 0.677. This establishes a strong, linguistically-informed baseline and provides essential data for developing sophisticated, semantic-preserving text normalization systems for Bangla.

Method: Developed TCM-5CEval benchmark with 5 dimensions: Core Knowledge, Classical Literacy, Clinical Decision-making, Chinese Materia Medica, and Clinical Non-pharmacological Therapy. Evaluated 15 prominent LLMs and conducted permutation-based consistency testing to assess reasoning stability.

Result: Significant performance disparities among models, with deepseek_r1 and gemini_2_5_pro as top performers. Models proficient in foundational knowledge but struggle with classical text interpretation. Permutation testing revealed widespread fragility - all models showed substantial performance degradation with varied question option ordering, indicating positional bias sensitivity.

Conclusion: TCM-5CEval provides detailed diagnostic capabilities for LLMs in TCM and exposes fundamental weaknesses in reasoning stability. The benchmark is available on Medbench platform to promote standardized comparison and further research.

Abstract: Large language models (LLMs) have demonstrated exceptional capabilities in general domains, yet their application in highly specialized and culturally-rich fields like Traditional Chinese Medicine (TCM) requires rigorous and nuanced evaluation. Building upon prior foundational work such as TCM-3CEval, which highlighted systemic knowledge gaps and the importance of cultural-contextual alignment, we introduce TCM-5CEval, a more granular and comprehensive benchmark. TCM-5CEval is designed to assess LLMs across five critical dimensions: (1) Core Knowledge (TCM-Exam), (2) Classical Literacy (TCM-LitQA), (3) Clinical Decision-making (TCM-MRCD), (4) Chinese Materia Medica (TCM-CMM), and (5) Clinical Non-pharmacological Therapy (TCM-ClinNPT). We conducted a thorough evaluation of fifteen prominent LLMs, revealing significant performance disparities and identifying top-performing models like deepseek_r1 and gemini_2_5_pro. Our findings show that while models exhibit proficiency in recalling foundational knowledge, they struggle with the interpretative complexities of classical texts. Critically, permutation-based consistency testing reveals widespread fragilities in model inference. All evaluated models, including the highest-scoring ones, displayed a substantial performance degradation when faced with varied question option ordering, indicating a pervasive sensitivity to positional bias and a lack of robust understanding. TCM-5CEval not only provides a more detailed diagnostic tool for LLM capabilities in TCM but aldso exposes fundamental weaknesses in their reasoning stability. To promote further research and standardized comparison, TCM-5CEval has been uploaded to the Medbench platform, joining its predecessor in the “In-depth Challenge for Comprehensive TCM Abilities” special track.

Method: Analyze statistics of token replacements in pivot sentences that yield identical translations, calculating probabilities of token substitutions while preserving translation output to estimate translation entropy.

Result: Translation entropy can be measured and enhanced along decoder blocks; method allows quantitative ranking of translators and reveals asymmetric mutual translation entropy; multi-token replacement shows multiplicative degeneracy effect.

Conclusion: Translation entropy is established as a measurable property enabling objective benchmarking of artificial translators, with validation on MarianMT, T5-Base and NLLB-200 systems.

Abstract: The translation of written language has been known since the 3rd century BC; however, its necessity has become increasingly common in the information age. Today, many translators exist, based on encoder-decoder deep architectures, nevertheless, no quantitative objective methods are available to assess their performance, likely because the entropy of even a single language remains unknown. This study presents a quantitative method for estimating translation entropy, with the following key finding. Given a translator, several sentences that differ by only one selected token of a given pivot sentence yield identical translations. Analyzing the statistics of this phenomenon across an ensemble of such sentences, consisting each of a pivot selected token, yields the probabilities of replacing this specific token with others while preserving the translation. These probabilities constitute the entropy of the selected token, and the average across all selected pivot tokens provides an estimate of the translator’s overall translation entropy, which is enhanced along the decoder blocks. This entropic measure allows for the quantitative ranking of several publicly available translators and reveals whether mutual translation entropy is symmetric. Extending the proposed method to include the replacement of two tokens in a given pivot sentence demonstrates a multiplicative effect, where translation degeneracy is proportional to the product of the degeneracies of the two tokens. These findings establish translation entropy as a measurable property and objective benchmarking of artificial translators. Results are based on MarianMT, T5-Base and NLLB-200 translators.

Method: Tested multiple LLMs (GPT-3.5, GPT-4, GPT-4o, Gemini 1.0 Pro, Llama 2/3, Mixtral 8x7B, airoboros 70B, RoLlama 2 7B) using comprehensive corpus with various prompt templates from zero-shot to multi-shot instructions.

Result: GPT-4o achieves high diacritic restoration accuracy, consistently surpassing baseline, while Meta’s Llama family exhibits wider variability.

Conclusion: Model architecture, training data, and prompt design significantly impact diacritic restoration performance, outlining directions for improving NLP tools for diacritic-rich languages.

Abstract: Automatic diacritic restoration is crucial for text processing in languages with rich diacritical marks, such as Romanian. This study evaluates the performance of several large language models (LLMs) in restoring diacritics in Romanian texts. Using a comprehensive corpus, we tested models including OpenAI’s GPT-3.5, GPT-4, GPT-4o, Google’s Gemini 1.0 Pro, Meta’s Llama 2 and Llama 3, MistralAI’s Mixtral 8x7B Instruct, airoboros 70B, and OpenLLM-Ro’s RoLlama 2 7B, under multiple prompt templates ranging from zero-shot to complex multi-shot instructions. Results show that models such as GPT-4o achieve high diacritic restoration accuracy, consistently surpassing a neutral echo baseline, while others, including Meta’s Llama family, exhibit wider variability. These findings highlight the impact of model architecture, training data, and prompt design on diacritic restoration performance and outline promising directions for improving NLP tools for diacritic-rich languages.

Method: Created a dataset of 4k+ English words with spectrogram/waveform figures, tested VLMs through multiple-choice tasks with phonemic/graphemic transcription prediction using distractor transcriptions based on phonemic edit distance.

Result: Both zero-shot and finetuned VLMs rarely performed above chance level, indicating poor understanding of speech representations.

Conclusion: VLMs require specific parametric knowledge for interpreting speech figures, not just paired multimodal samples, highlighting limitations in their phonetic analysis capabilities.

Abstract: With the rise of Large Language Models (LLMs) and their vision-enabled counterparts (VLMs), numerous works have investigated their capabilities in tasks that fuse the modalities of vision and language. In this work, we benchmark the extent to which VLMs are able to act as highly-trained phoneticians, interpreting spectrograms and waveforms of speech. To do this, we synthesise a novel dataset containing 4k+ English words spoken in isolation alongside stylistically consistent spectrogram and waveform figures. We test the ability of VLMs to understand these representations of speech through a multiple-choice task whereby models must predict the correct phonemic or graphemic transcription of a spoken word when presented amongst 3 distractor transcriptions that have been selected based on their phonemic edit distance to the ground truth. We observe that both zero-shot and finetuned models rarely perform above chance, demonstrating the requirement for specific parametric knowledge of how to interpret such figures, rather than paired samples alone.

Method: Identify weakly-correlated benchmark categories, select expert models for each category cluster, and apply non-uniform weighted averaging to combine them.

Result: Improves performance and robustness across multiple domains including multilingual capabilities, tool calling, and math; achieves state-of-the-art results on Berkeley Function Calling Leaderboard.

Conclusion: SoCE provides a principled approach to model souping that outperforms uniform averaging by leveraging category-specific expertise through optimized weighted combination.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, but their training remains resource- and time-intensive, requiring massive compute power and careful orchestration of training procedures. Model souping-the practice of averaging weights from multiple models of the same architecture-has emerged as a promising pre- and post-training technique that can enhance performance without expensive retraining. In this paper, we introduce Soup Of Category Experts (SoCE), a principled approach for model souping that utilizes benchmark composition to identify optimal model candidates and applies non-uniform weighted averaging to maximize performance. Contrary to previous uniform-averaging approaches, our method leverages the observation that benchmark categories often exhibit low inter-correlations in model performance. SoCE identifies “expert” models for each weakly-correlated category cluster and combines them using optimized weighted averaging rather than uniform weights. We demonstrate that the proposed method improves performance and robustness across multiple domains, including multilingual capabilities, tool calling, and math and achieves state-of-the-art results on the Berkeley Function Calling Leaderboard.

Method: RegionMarker defines trigger regions in low-dimensional space using a secret dimensionality reduction matrix, randomly selects these regions, and injects watermarks into text embeddings associated with the regions. The approach uses text embeddings themselves as watermarks.

Result: Extensive experiments on various datasets demonstrate that RegionMarker effectively resists different attack methods, including paraphrasing and dimension-perturbation attacks, making it difficult for watermark removal attacks to evade detection.

Conclusion: RegionMarker provides comprehensive copyright protection for EaaS by being resilient to multiple attack types through its region-triggered semantic watermarking approach, thereby protecting model providers from economic losses.

Abstract: Embedding-as-a-Service (EaaS) is an effective and convenient deployment solution for addressing various NLP tasks. Nevertheless, recent research has shown that EaaS is vulnerable to model extraction attacks, which could lead to significant economic losses for model providers. For copyright protection, existing methods inject watermark embeddings into text embeddings and use them to detect copyright infringement. However, current watermarking methods often resist only a subset of attacks and fail to provide \textit{comprehensive} protection. To this end, we present the region-triggered semantic watermarking framework called RegionMarker, which defines trigger regions within a low-dimensional space and injects watermarks into text embeddings associated with these regions. By utilizing a secret dimensionality reduction matrix to project onto this subspace and randomly selecting trigger regions, RegionMarker makes it difficult for watermark removal attacks to evade detection. Furthermore, by embedding watermarks across the entire trigger region and using the text embedding as the watermark, RegionMarker is resilient to both paraphrasing and dimension-perturbation attacks. Extensive experiments on various datasets show that RegionMarker is effective in resisting different attack methods, thereby protecting the copyright of EaaS.

Method: Created multi-dialect dataset from 538 sentiment-balanced hotel reviews, originally in MSA and translated into Saudi and Moroccan dialects, validated by native speakers.

Result: Over 40 teams registered, 12 submitted systems; top system achieved F1 score of 0.81.

Conclusion: Demonstrates feasibility but ongoing challenges in sentiment analysis across Arabic dialects for real-world customer experience applications.

Abstract: The hospitality industry in the Arab world increasingly relies on customer feedback to shape services, driving the need for advanced Arabic sentiment analysis tools. To address this challenge, the Sentiment Analysis on Arabic Dialects in the Hospitality Domain shared task focuses on Sentiment Detection in Arabic Dialects. This task leverages a multi-dialect, manually curated dataset derived from hotel reviews originally written in Modern Standard Arabic (MSA) and translated into Saudi and Moroccan (Darija) dialects. The dataset consists of 538 sentiment-balanced reviews spanning positive, neutral, and negative categories. Translations were validated by native speakers to ensure dialectal accuracy and sentiment preservation. This resource supports the development of dialect-aware NLP systems for real-world applications in customer experience analysis. More than 40 teams have registered for the shared task, with 12 submitting systems during the evaluation phase. The top-performing system achieved an F1 score of 0.81, demonstrating the feasibility and ongoing challenges of sentiment analysis across Arabic dialects.

Method: Conducted controlled PEFT/LoRA study across multiple open-weight LLM families and sizes, fine-tuning each model on single task-language source and measuring transfer as percentage-point change versus baseline when evaluated on all other task-language target pairs.

Result: Uncovered two consistent patterns: 1) pronounced on-task vs. off-task asymmetry with positive Matched-Task (Cross-Language) transfer but collateral degradation in off-task transfer, 2) stable donor-recipient structure across languages and tasks (hub donors vs. brittle recipients).

Conclusion: Findings have implications for risk-aware fine-tuning and model specialization, highlighting the need to consider transfer effects when optimizing LLMs for specific tasks or languages.

Abstract: Large language models (LLMs) perform strongly across tasks and languages, yet how improvements in one task or language affect other tasks and languages and their combinations remains poorly understood. We conduct a controlled PEFT/LoRA study across multiple open-weight LLM families and sizes, treating task and language as transfer axes while conditioning on model family and size; we fine-tune each model on a single task-language source and measure transfer as the percentage-point change versus its baseline score when evaluated on all other task-language target pairs. We decompose transfer into (i) Matched-Task (Cross-Language), (ii) Matched-Language (Cross-Task), and (iii) Cross-Task (Cross-Language) regimes. We uncover two consistent general patterns. First, a pronounced on-task vs. off-task asymmetry: Matched-Task (Cross-Language) transfer is reliably positive, whereas off-task transfer often incurs collateral degradation. Second, a stable donor-recipient structure across languages and tasks (hub donors vs. brittle recipients). We outline implications for risk-aware fine-tuning and model specialisation.

Method: Developed PEDIASBench evaluation framework assessing LLMs across three dimensions: basic knowledge application, dynamic diagnosis/treatment capability, and pediatric medical safety/ethics. Evaluated 12 models across 19 pediatric subspecialties and 211 diseases.

Result: Top models achieved >90% accuracy on basic knowledge but performance declined ~15% with complexity. Models struggled with integrative reasoning, real-time patient adaptation, and humanistic sensitivity. DeepSeek-R1 scored highest in case reasoning (0.58), Qwen2.5-72B best in ethics/safety (92.05%).

Conclusion: Current LLMs cannot independently perform pediatric care but show promise for decision support, education, and communication. Future development should focus on multimodal integration and clinical feedback loops to enhance safety and human-AI collaboration.

Abstract: With the rapid rise of large language models (LLMs) in medicine, a key question is whether they can function as competent pediatricians in real-world clinical settings. We developed PEDIASBench, a systematic evaluation framework centered on a knowledge-system framework and tailored to realistic clinical environments. PEDIASBench assesses LLMs across three dimensions: application of basic knowledge, dynamic diagnosis and treatment capability, and pediatric medical safety and medical ethics. We evaluated 12 representative models released over the past two years, including GPT-4o, Qwen3-235B-A22B, and DeepSeek-V3, covering 19 pediatric subspecialties and 211 prototypical diseases. State-of-the-art models performed well on foundational knowledge, with Qwen3-235B-A22B achieving over 90% accuracy on licensing-level questions, but performance declined ~15% as task complexity increased, revealing limitations in complex reasoning. Multiple-choice assessments highlighted weaknesses in integrative reasoning and knowledge recall. In dynamic diagnosis and treatment scenarios, DeepSeek-R1 scored highest in case reasoning (mean 0.58), yet most models struggled to adapt to real-time patient changes. On pediatric medical ethics and safety tasks, Qwen2.5-72B performed best (accuracy 92.05%), though humanistic sensitivity remained limited. These findings indicate that pediatric LLMs are constrained by limited dynamic decision-making and underdeveloped humanistic care. Future development should focus on multimodal integration and a clinical feedback-model iteration loop to enhance safety, interpretability, and human-AI collaboration. While current LLMs cannot independently perform pediatric care, they hold promise for decision support, medical education, and patient communication, laying the groundwork for a safe, trustworthy, and collaborative intelligent pediatric healthcare system.

Method: Developed a multi-step LLM-based synthesis pipeline to create PAL-Set dataset, and proposed H²Memory - a hierarchical and heterogeneous memory framework with retrieval-augmented generation for personalized response generation.

Result: Comprehensive experiments on PAL-Bench and external datasets demonstrate the effectiveness of the proposed memory framework in improving personalized service-oriented interactions.

Conclusion: PAL-Bench provides a valuable benchmark for evaluating personalization capabilities, and H²Memory framework effectively addresses the gaps in capturing long-term user characteristics for personalized dialogue assistance.

Abstract: With the rise of smart personal devices, service-oriented human-agent interactions have become increasingly prevalent. This trend highlights the need for personalized dialogue assistants that can understand user-specific traits to accurately interpret requirements and tailor responses to individual preferences. However, existing approaches often overlook the complexities of long-term interactions and fail to capture users’ subjective characteristics. To address these gaps, we present PAL-Bench, a new benchmark designed to evaluate the personalization capabilities of service-oriented assistants in long-term user-agent interactions. In the absence of available real-world data, we develop a multi-step LLM-based synthesis pipeline, which is further verified and refined by human annotators. This process yields PAL-Set, the first Chinese dataset comprising multi-session user logs and dialogue histories, which serves as the foundation for PAL-Bench. Furthermore, to improve personalized service-oriented interactions, we propose H$^2$Memory, a hierarchical and heterogeneous memory framework that incorporates retrieval-augmented generation to improve personalized response generation. Comprehensive experiments on both our PAL-Bench and an external dataset demonstrate the effectiveness of the proposed memory framework.

Method: Proposes a two-parameter logarithmic model E(x) = a * ln(1 + b * x) calibrated from tolerance points, supported by psychophysical evidence (Weber-Fechner law, Cognitive Load Theory) showing error perception diminishes logarithmically with scale.

Result: Empirical data from three enterprise environments confirms acceptable error counts grow logarithmically with sample size. The model improves interpretability, fairness, and inter-rater reliability while maintaining compatibility with existing evaluation workflows.

Conclusion: The logarithmic scoring model advances translation quality evaluation toward more accurate, scalable assessment that better reflects human perception, providing a stronger foundation for both human and AI-generated text evaluation.

Abstract: Analytic Translation Quality Evaluation (TQE), based on Multidimensional Quality Metrics (MQM), traditionally uses a linear error-to-penalty scale calibrated to a reference sample of 1000-2000 words. However, linear extrapolation biases judgment on samples of different sizes, over-penalizing short samples and under-penalizing long ones, producing misalignment with expert intuition. Building on the Multi-Range framework, this paper presents a calibrated, non-linear scoring model that better reflects how human content consumers perceive translation quality across samples of varying length. Empirical data from three large-scale enterprise environments shows that acceptable error counts grow logarithmically, not linearly, with sample size. Psychophysical and cognitive evidence, including the Weber-Fechner law and Cognitive Load Theory, supports this premise by explaining why the perceptual impact of additional errors diminishes while the cognitive burden grows with scale. We propose a two-parameter model E(x) = a * ln(1 + b * x), a, b > 0, anchored to a reference tolerance and calibrated from two tolerance points using a one-dimensional root-finding step. The model yields an explicit interval within which the linear approximation stays within +/-20 percent relative error and integrates into existing evaluation workflows with only a dynamic tolerance function added. The approach improves interpretability, fairness, and inter-rater reliability across both human and AI-generated translations. By operationalizing a perceptually valid scoring paradigm, it advances translation quality evaluation toward more accurate and scalable assessment. The model also provides a stronger basis for AI-based document-level evaluation aligned with human judgment. Implementation considerations for CAT/LQA systems and implications for human and AI-generated text evaluation are discussed.

Method: Developed specific guidelines for annotating obfuscated sentiment, annotated over 100 financial reports, benchmarked various text classification models, and conducted an event study to evaluate impact on stock prices.

Result: Demonstrated strong performance in sentiment classification and found that market reactions are selectively influenced by specific aspects within the reports.

Conclusion: The findings highlight the complexity of sentiment analysis in financial texts and emphasize the importance of addressing obfuscated language to accurately assess market sentiment.

Abstract: Understanding sentiment in financial documents is crucial for gaining insights into market behavior. These reports often contain obfuscated language designed to present a positive or neutral outlook, even when underlying conditions may be less favorable. This paper presents a novel approach using Aspect-Based Sentiment Analysis (ABSA) to decode obfuscated sentiment in Thai financial annual reports. We develop specific guidelines for annotating obfuscated sentiment in these texts and annotate more than one hundred financial reports. We then benchmark various text classification models on this annotated dataset, demonstrating strong performance in sentiment classification. Additionally, we conduct an event study to evaluate the real-world implications of our sentiment analysis on stock prices. Our results suggest that market reactions are selectively influenced by specific aspects within the reports. Our findings underscore the complexity of sentiment analysis in financial texts and highlight the importance of addressing obfuscated language to accurately assess market sentiment.

Method: Developed computational framework using LLMs to automate qualitative annotation based on expert codebook, evaluated against human experts across multiple narratives and codes.

Result: LLMs achieved near-human-expert performance with average F1 score of 0.80 across 8 narratives and 14 codes, successfully analyzed 22 stories and political speeches.

Conclusion: LLM-assisted annotation shows strong potential for scalable narrative analysis in civic storytelling, though limitations exist that require future research.

Abstract: Public Narratives (PNs) are key tools for leadership development and civic mobilization, yet their systematic analysis remains challenging due to their subjective interpretation and the high cost of expert annotation. In this work, we propose a novel computational framework that leverages large language models (LLMs) to automate the qualitative annotation of public narratives. Using a codebook we co-developed with subject-matter experts, we evaluate LLM performance against that of expert annotators. Our work reveals that LLMs can achieve near-human-expert performance, achieving an average F1 score of 0.80 across 8 narratives and 14 codes. We then extend our analysis to empirically explore how PN framework elements manifest across a larger dataset of 22 stories. Lastly, we extrapolate our analysis to a set of political speeches, establishing a novel lens in which to analyze political rhetoric in civic spaces. This study demonstrates the potential of LLM-assisted annotation for scalable narrative analysis and highlights key limitations and directions for future research in computational civic storytelling.

Method: Developed a taxonomy for text-to-SQL classification across multiple dimensions, then used this taxonomy with LLMs to synthesize a new dataset (SQL-Synth) through a guided pipeline.

Result: SQL-Synth exhibits greater diversity and coverage than existing benchmarks, and experiments show current LLMs perform poorly on it but can be substantially improved through fine-tuning.

Conclusion: The taxonomy enables comprehensive dataset analysis and LLM performance evaluation, and can guide construction of training data to improve text-to-SQL capabilities.

Abstract: Text-to-SQL datasets are essential for training and evaluating text-to-SQL models, but existing datasets often suffer from limited coverage and fail to capture the diversity of real-world applications. To address this, we propose a novel taxonomy for text-to-SQL classification based on dimensions including core intents, statement types, syntax structures, and key actions. Using this taxonomy, we evaluate widely used public text-to-SQL datasets (e.g., Spider and Bird) and reveal limitations in their coverage and diversity. We then introduce a taxonomy-guided dataset synthesis pipeline, yielding a new dataset named SQL-Synth. This approach combines the taxonomy with Large Language Models (LLMs) to ensure the dataset reflects the breadth and complexity of real-world text-to-SQL applications. Extensive analysis and experimental results validate the effectiveness of our taxonomy, as SQL-Synth exhibits greater diversity and coverage compared to existing benchmarks. Moreover, we uncover that existing LLMs typically fall short in adequately capturing the full range of scenarios, resulting in limited performance on SQL-Synth. However, fine-tuning can substantially improve their performance in these scenarios. The proposed taxonomy has significant potential impact, as it not only enables comprehensive analysis of datasets and the performance of different LLMs, but also guides the construction of training data for LLMs.

Method: O-Mem uses active user profiling to dynamically extract and update user characteristics and event records from proactive interactions. It supports hierarchical retrieval of persona attributes and topic-related context for adaptive personalized responses.

Result: O-Mem achieves 51.76% on LoCoMo benchmark (3% improvement over LangMem) and 62.99% on PERSONAMEM (3.5% improvement over A-Mem). It also boosts token and interaction response time efficiency compared to previous frameworks.

Conclusion: O-Mem demonstrates promising directions for developing efficient and human-like personalized AI assistants by addressing key limitations in memory systems for long-term interactions.

Abstract: Recent advancements in LLM-powered agents have demonstrated significant potential in generating human-like responses; however, they continue to face challenges in maintaining long-term interactions within complex environments, primarily due to limitations in contextual consistency and dynamic personalization. Existing memory systems often depend on semantic grouping prior to retrieval, which can overlook semantically irrelevant yet critical user information and introduce retrieval noise. In this report, we propose the initial design of O-Mem, a novel memory framework based on active user profiling that dynamically extracts and updates user characteristics and event records from their proactive interactions with agents. O-Mem supports hierarchical retrieval of persona attributes and topic-related context, enabling more adaptive and coherent personalized responses. O-Mem achieves 51.76% on the public LoCoMo benchmark, a nearly 3% improvement upon LangMem,the previous state-of-the-art, and it achieves 62.99% on PERSONAMEM, a 3.5% improvement upon A-Mem,the previous state-of-the-art. O-Mem also boosts token and interaction response time efficiency compared to previous memory frameworks. Our work opens up promising directions for developing efficient and human-like personalized AI assistants in the future.

Method: Using large language models (LLMs) to translate machine-learned lexical cues into human-understandable language phenomena that differentiate deceptive from genuine reviews.

Result: The language phenomena obtained are empirically grounded in data, generalizable across similar domains, and more predictive than phenomena from LLMs’ prior knowledge or in-context learning.

Conclusion: These language phenomena can help people critically assess online review credibility where deception detection classifiers are unavailable.

Abstract: Deceptive reviews mislead consumers, harm businesses, and undermine trust in online marketplaces. Machine learning classifiers can learn from large amounts of training examples to effectively distinguish deceptive reviews from genuine ones. However, the distinguishing features learned by these classifiers are often subtle, fragmented, and difficult for humans to interpret. In this work, we explore using large language models (LLMs) to translate machine-learned lexical cues into human-understandable language phenomena that can differentiate deceptive reviews from genuine ones. We show that language phenomena obtained in this manner are empirically grounded in data, generalizable across similar domains, and more predictive than phenomena either in LLMs’ prior knowledge or obtained through in-context learning. These language phenomena have the potential to aid people in critically assessing the credibility of online reviews in environments where deception detection classifiers are unavailable.

Method: TAI framework with translation module using Odds Ratio Preference Alignment Algorithm and image generation module using semantic graphs to capture metaphors and meanings.

Result: Experimental evaluation shows TAI Diffusion outperforms baselines in poem image generation, and introduces MorphoVerse Dataset with 1,570 poems across 21 Indian languages.

Conclusion: The work enhances accessibility of Indian poetry globally, supporting SDG goals for quality education and reduced inequalities through improved translation and visual comprehension.

Abstract: Indian poetry, known for its linguistic complexity and deep cultural resonance, has a rich and varied heritage spanning thousands of years. However, its layered meanings, cultural allusions, and sophisticated grammatical constructions often pose challenges for comprehension, especially for non-native speakers or readers unfamiliar with its context and language. Despite its cultural significance, existing works on poetry have largely overlooked Indian language poems. In this paper, we propose the Translation and Image Generation (TAI) framework, leveraging Large Language Models (LLMs) and Latent Diffusion Models through appropriate prompt tuning. Our framework supports the United Nations Sustainable Development Goals of Quality Education (SDG 4) and Reduced Inequalities (SDG 10) by enhancing the accessibility of culturally rich Indian-language poetry to a global audience. It includes (1) a translation module that uses an Odds Ratio Preference Alignment Algorithm to accurately translate morphologically rich poetry into English, and (2) an image generation module that employs a semantic graph to capture tokens, dependencies, and semantic relationships between metaphors and their meanings, to create visually meaningful representations of Indian poems. Our comprehensive experimental evaluation, including both human and quantitative assessments, demonstrates the superiority of TAI Diffusion in poem image generation tasks, outperforming strong baselines. To further address the scarcity of resources for Indian-language poetry, we introduce the Morphologically Rich Indian Language Poems MorphoVerse Dataset, comprising 1,570 poems across 21 low-resource Indian languages. By addressing the gap in poetry translation and visual comprehension, this work aims to broaden accessibility and enrich the reader’s experience.

Method: HAPO tracks history states (minimum length of previous correct responses) and uses a novel length reward function to incentivize discovering more concise correct solutions, combined with correctness reward for joint optimization.

Result: HAPO-trained models achieved 33-59% length reduction on math benchmarks with only 2-5% accuracy drops, demonstrating effective induction of concise reasoning abilities.

Conclusion: HAPO successfully enables LLMs to progressively generate more efficient solutions by leveraging historical information, balancing correctness and conciseness through its reward structure.

Abstract: While scaling the length of responses at test-time has been shown to markedly improve the reasoning abilities and performance of large language models (LLMs), it often results in verbose outputs and increases inference cost. Prior approaches for efficient test-time scaling, typically using universal budget constraints or query-level length optimization, do not leverage historical information from previous encounters with the same problem during training. We hypothesize that this limits their ability to progressively make solutions more concise over time. To address this, we present History-Aware Policy Optimization (HAPO), which keeps track of a history state (e.g., the minimum length over previously generated correct responses) for each problem. HAPO employs a novel length reward function based on this history state to incentivize the discovery of correct solutions that are more concise than those previously found. Crucially, this reward structure avoids overly penalizing shorter incorrect responses with the goal of facilitating exploration towards more efficient solutions. By combining this length reward with a correctness reward, HAPO jointly optimizes for correctness and efficiency. We use HAPO to train DeepSeek-R1-Distill-Qwen-1.5B, DeepScaleR-1.5B-Preview, and Qwen-2.5-1.5B-Instruct, and evaluate HAPO on several math benchmarks that span various difficulty levels. Experiment results demonstrate that HAPO effectively induces LLMs’ concise reasoning abilities, producing length reductions of 33-59% with accuracy drops of only 2-5%.

Method: Developed Lang1 models (100M-7B parameters) pretrained on specialized corpus combining 80B clinical tokens from EHRs and 627B internet tokens, evaluated using ReMedE benchmark with 668,331 EHR notes across five critical healthcare tasks.

Result: After finetuning, Lang1-1B outperformed finetuned generalist models up to 70x larger and zero-shot models up to 671x larger, improving AUROC by 3.64%-6.75% and 1.66%-23.66% respectively. Models showed cross-task scaling and effective transfer to out-of-distribution settings.

Conclusion: Effective healthcare AI requires in-domain pretraining, supervised finetuning, and real-world evaluation beyond proxy benchmarks, demonstrating specialized LLMs can compete with generalist models in specialized healthcare tasks.

Abstract: Hospitals and healthcare systems rely on operational decisions that determine patient flow, cost, and quality of care. Despite strong performance on medical knowledge and conversational benchmarks, foundation models trained on general text may lack the specialized knowledge required for these operational decisions. We introduce Lang1, a family of models (100M-7B parameters) pretrained on a specialized corpus blending 80B clinical tokens from NYU Langone Health’s EHRs and 627B tokens from the internet. To rigorously evaluate Lang1 in real-world settings, we developed the REalistic Medical Evaluation (ReMedE), a benchmark derived from 668,331 EHR notes that evaluates five critical tasks: 30-day readmission prediction, 30-day mortality prediction, length of stay, comorbidity coding, and predicting insurance claims denial. In zero-shot settings, both general-purpose and specialized models underperform on four of five tasks (36.6%-71.7% AUROC), with mortality prediction being an exception. After finetuning, Lang1-1B outperforms finetuned generalist models up to 70x larger and zero-shot models up to 671x larger, improving AUROC by 3.64%-6.75% and 1.66%-23.66% respectively. We also observed cross-task scaling with joint finetuning on multiple tasks leading to improvement on other tasks. Lang1-1B effectively transfers to out-of-distribution settings, including other clinical tasks and an external health system. Our findings suggest that predictive capabilities for hospital operations require explicit supervised finetuning, and that this finetuning process is made more efficient by in-domain pretraining on EHR. Our findings support the emerging view that specialized LLMs can compete with generalist models in specialized tasks, and show that effective healthcare systems AI requires the combination of in-domain pretraining, supervised finetuning, and real-world evaluation beyond proxy benchmarks.

Method: Use distance and reward margin to quantify response differences, combine them into DCRM metric, and propose best-of-N² pairing to select high-DCRM response pairs.

Result: Higher DCRM correlates with better learning outcomes. The proposed method produces datasets that improve performance on AlpacaEval, MT-Bench, and Arena-Hard benchmarks.

Conclusion: DCRM effectively measures preference dataset quality, and selecting high-DCRM pairs through best-of-N² pairing can enhance preference optimization performance.

Abstract: Recent research has attempted to associate preference optimization (PO) performance with the underlying preference datasets. In this work, our observation is that the differences between the preferred response $y^+$ and dispreferred response $y^-$ influence what LLMs can learn, which may not match the desirable differences to learn. Therefore, we use distance and reward margin to quantify these differences, and combine them to get Distance Calibrated Reward Margin (DCRM), a metric that measures the quality of a response pair for PO. Intuitively, DCRM encourages minimal noisy differences and maximal desired differences. With this, we study 3 types of commonly used preference datasets, classified along two axes: the source of the responses and the preference labeling function. We establish a general correlation between higher DCRM of the training set and better learning outcome. Inspired by this, we propose a best-of-$N^2$ pairing method that selects response pairs with the highest DCRM. Empirically, in various settings, our method produces training datasets that can further improve models’ performance on AlpacaEval, MT-Bench, and Arena-Hard over the existing training sets.

Method: The approach defines six modal operators based on an agent’s system of ontic rules that determine expected and accepted timelines. The logical theory uses branching time models with evaluation contexts including time flow models, ontic rules, timelines, and instants.

Result: The paper successfully formalizes the six notions of necessity and possibility, provides their logical semantics in branching time, and develops a complete axiomatic system for reasoning about these temporal modalities.

Conclusion: The framework offers a comprehensive logical treatment of temporal and historical modalities from a linguistic perspective, enabling formal reasoning about different types of necessity and possibility in branching time structures.

Abstract: In this paper, we do three kinds of work. First, we recognize four notions of necessity and two notions of possibility related to time flow, namely strong/weak historical/temporal necessities, as well as historical/temporal possibilities, which are motivated more from a linguistic perspective than from a philosophical one. Strong/weak historical necessities and historical possibility typically concern the possible futures of the present world, and strong/weak temporal necessities and temporal possibility concern possible timelines of alternatives of the present world. Second, we provide our approach to the six notions and present a logical theory of them in branching time. Our approach to the six notions is as follows. The agent has a system of ontic rules that determine expected timelines. She treats some ontic rules as undefeatable, determining accepted timelines. The domains of strong/weak historical necessities, respectively, consist of accepted and expected timelines passing through the present moment, and historical possibility is the dual of strong historical necessity. The domains of strong/weak temporal necessities, respectively, consist of accepted and expected timelines, and temporal possibility is the dual of strong temporal necessity. The logical theory has six operators: a last-moment operator, a next-moment operator, and four operators for the four notions of necessity. Formulas’ evaluation contexts consist of a tree-like model representing a time flow, a context representing the agent’s system of ontic rules, a timeline, and an instant. Third, we offer an axiomatic system for the logical theory and show its soundness and completeness.

Method: Applied LLMs to SimulMT using existing incremental-decoding methods with a new RALCP algorithm for latency reduction, tested on Llama2-7b-chat model across nine languages from MUST-C dataset.

Result: LLM outperforms dedicated MT models in both BLEU and LAAL metrics, showing advantages in tuning efficiency and robustness, though computational cost remains a significant obstacle.

Conclusion: LLMs show promise for SimulMT with superior performance and robustness, but computational efficiency needs improvement for practical applications.

Abstract: Real-world simultaneous machine translation (SimulMT) systems face more challenges than just the quality-latency trade-off. They also need to address issues related to robustness with noisy input, processing long contexts, and flexibility for knowledge injection. These challenges demand models with strong language understanding and generation capabilities which may not often equipped by dedicated MT models. In this paper, we investigate the possibility of applying Large Language Models (LLM) to SimulMT tasks by using existing incremental-decoding methods with a newly proposed RALCP algorithm for latency reduction. We conducted experiments using the \texttt{Llama2-7b-chat} model on nine different languages from the MUST-C dataset. The results show that LLM outperforms dedicated MT models in terms of BLEU and LAAL metrics. Further analysis indicates that LLM has advantages in terms of tuning efficiency and robustness. However, it is important to note that the computational cost of LLM remains a significant obstacle to its application in SimulMT.

Method: Created a dataset of 32,500 sentences covering Bangla, Banglish, and English from five regional dialects. Proposed two novel models: DialectBanglaT5 for dialect-to-standard Bangla translation and DialectBanglaBERT for dialect region classification.

Result: DialectBanglaT5 achieved highest BLEU score of 71.93, METEOR of 0.8503, and lowest WER of 0.1470 and CER of 0.0791 on Mymensingh dialect. DialectBanglaBERT achieved 89.02% overall region classification accuracy with F1-scores of 0.9241 for Chittagong and 0.8736 for Mymensingh.

Conclusion: This is the first large-scale investigation of Bangla regional dialect translation and region detection. The proposed models demonstrate the potential of dialect-specific modeling and set new benchmarks for low-resource and dialect-rich language research.

Abstract: The Bangla linguistic variety is a fascinating mix of regional dialects that contributes to the cultural diversity of the Bangla-speaking community. Despite extensive study into translating Bangla to English, English to Bangla, and Banglish to Bangla in the past, there has been a noticeable gap in translating Bangla regional dialects into standard Bangla. In this study, we set out to fill this gap by creating a collection of 32,500 sentences, encompassing Bangla, Banglish, and English, representing five regional Bangla dialects. Our aim is to translate these regional dialects into standard Bangla and detect regions accurately. To tackle the translation and region detection tasks, we propose two novel models: DialectBanglaT5 for translating regional dialects into standard Bangla and DialectBanglaBERT for identifying the dialect’s region of origin. DialectBanglaT5 demonstrates superior performance across all dialects, achieving the highest BLEU score of 71.93, METEOR of 0.8503, and the lowest WER of 0.1470 and CER of 0.0791 on the Mymensingh dialect. It also achieves strong ROUGE scores across all dialects, indicating both accuracy and fluency in capturing dialectal nuances. In parallel, DialectBanglaBERT achieves an overall region classification accuracy of 89.02%, with notable F1-scores of 0.9241 for Chittagong and 0.8736 for Mymensingh, confirming its effectiveness in handling regional linguistic variation. This is the first large-scale investigation focused on Bangla regional dialect translation and region detection. Our proposed models highlight the potential of dialect-specific modeling and set a new benchmark for future research in low-resource and dialect-rich language settings.

Method: Multi-turn-dialogue-based decoding framework for conversational simultaneous machine translation using LLMs like Llama2-7b-chat.

Result: Achieves superior translation quality compared to specialized SimulMT models while maintaining comparable computational latency on two benchmarks.

Conclusion: The conversational framework successfully enhances LLM inference efficiency for SimulMT, balancing quality and latency effectively.

Abstract: Simultaneous machine translation (SimulMT) presents a challenging trade-off between translation quality and latency. Recent studies have shown that LLMs can achieve good performance in SimulMT tasks. However, this often comes at the expense of high inference cost and latency. In this paper, we propose a conversational SimulMT framework to enhance the inference efficiency of LLM-based SimulMT through multi-turn-dialogue-based decoding. Our experiments with Llama2-7b-chat on two SimulMT benchmarks demonstrate the superiority of LLM in translation quality while achieving comparable computational latency to specialized SimulMT models.

Method: Uses attribute-guided generation and group checking for diversity; code-based mathematical assessment and retrieval-augmented generation for accuracy; user-specified constraints for customization.

Result: Produces superior quality datasets that enhance LLM benchmarking and improve capabilities in agent-oriented abilities and reasoning through data augmentation.

Conclusion: DataGen effectively addresses key challenges in synthetic data generation and supports dynamic benchmarking while enhancing LLM performance across multiple domains.

Abstract: Large Language Models (LLMs) such as GPT-4 and Llama3 have significantly impacted various fields by enabling high-quality synthetic data generation and reducing dependence on expensive human-generated datasets. Despite this, challenges remain in the areas of generalization, controllability, diversity, and truthfulness within the existing generative frameworks. To address these challenges, this paper presents DataGen, a comprehensive LLM-powered framework designed to produce diverse, accurate, and highly controllable datasets. DataGen is adaptable, supporting all types of text datasets and enhancing the generative process through innovative mechanisms. To augment data diversity, DataGen incorporates an attribute-guided generation module and a group checking feature. For accuracy, it employs a code-based mathematical assessment for label verification alongside a retrieval-augmented generation technique for factual validation. The framework also allows for user-specified constraints, enabling customization of the data generation process to suit particular requirements. Extensive experiments demonstrate the superior quality of data generated by DataGen, and each module within DataGen plays a critical role in this enhancement. Additionally, DataGen is applied in two practical scenarios: benchmarking LLMs and data augmentation. The results indicate that DataGen effectively supports dynamic and evolving benchmarking and that data augmentation improves LLM capabilities in various domains, including agent-oriented abilities and reasoning skills.

Method: Introduces ProFuser that evaluates model advantage through both cross entropy during training and inference outputs, with progressive transition from inference mode to training mode.

Result: Fused three models (Vicuna-7B-v1.5, Llama-2-7B-Chat, MPT-7B-8K-Chat) and demonstrated improved performance in knowledge, reasoning, and safety compared to baseline methods.

Conclusion: ProFuser effectively enhances model fusion by incorporating both training and inference modes, providing a more comprehensive approach to model advantage assessment.

Abstract: While fusing the capacities and advantages of various large language models offers a pathway to construct more powerful and versatile models, a fundamental challenge is to properly select advantageous model during training. Existing fusion methods primarily focus on the training mode that uses cross entropy on ground truth in a teacher-forcing setup to measure a model’s advantage, which may provide limited insight towards model advantage. In this paper, we introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes. Our method evaluates model advantage not only through cross entropy during training but also by considering inference outputs, providing a more comprehensive assessment. To combine the two modes effectively, we introduce ProFuser to progressively transition from inference mode to training mode. To validate ProFuser’s effectiveness, we fused three models, including Vicuna-7B-v1.5, Llama-2-7B-Chat, and MPT-7B-8K-Chat, and demonstrated the improved performance in knowledge, reasoning, and safety compared to baseline methods.

Method: Conducted comprehensive empirical study comparing different calibration languages for pruning across diverse languages, tasks, models, and state-of-the-art pruning techniques. Analyzed latent subspaces, pruning masks, and individual neurons within pruned models.

Result: Calibration on target language effectively retains perplexity and yields high signal-to-noise ratios but does not consistently improve downstream task performance. Analysis reveals pruning preserves dominant language-specific features but loses nuanced language-agnostic features crucial for knowledge retention and reasoning.

Conclusion: Current pruning approaches have limitations - while they effectively preserve language-specific information, this is insufficient to counteract the loss of crucial language-agnostic features needed for knowledge and reasoning capabilities.

Abstract: Recent advances in large language model (LLM) pruning have shown state-of-the-art (SotA) compression results in post-training and retraining-free settings while maintaining high predictive performance. However, previous research mainly considered calibrating based on English text, despite the multilingual nature of modern LLMs and their frequent use in non-English languages. This analysis paper conducts an in-depth investigation of the performance and internal representation changes associated with pruning multilingual language models for monolingual applications. We present the first comprehensive empirical study, comparing different calibration languages for pruning multilingual models across diverse languages, tasks, models, and SotA pruning techniques. We further analyze the latent subspaces, pruning masks, and individual neurons within pruned models. Our results reveal that while calibration on the target language effectively retains perplexity and yields high signal-to-noise ratios, it does not consistently improve downstream task performance. Further analysis of internal representations at three different levels highlights broader limitations of current pruning approaches: While they effectively preserve dominant information like language-specific features, this is insufficient to counteract the loss of nuanced, language-agnostic features that are crucial for knowledge retention and reasoning.

Method: Developed a dataset with adversarial noise variations, applied standardized robustness metrics, and tested transformer-based QA models.

Result: Experiments revealed robustness vulnerabilities and provided insights into model performance under realistic noisy conditions.

Conclusion: The study highlights significant robustness issues in QA models and offers a framework for evaluating model resilience to adversarial text perturbations.

Abstract: Contextual question-answering models are susceptible to adversarial perturbations to input context, commonly observed in real-world scenarios. These adversarial noises are designed to degrade the performance of the model by distorting the textual input. We introduce a unique dataset that incorporates seven distinct types of adversarial noise into the context, each applied at five different intensity levels on the SQuAD dataset. To quantify the robustness, we utilize robustness metrics providing a standardized measure for assessing model performance across varying noise types and levels. Experiments on transformer-based question-answering models reveal robustness vulnerabilities and important insights into the model’s performance in realistic textual input.

Method: Replaced linear mappings in Discriminative Lexicon Model with deep dense neural networks (DDL), comparing performance across English, Dutch, Estonian, and Taiwan Mandarin datasets, and testing frequency-informed variants.

Result: DDL outperforms linear methods for large datasets (English/Dutch) and pseudo-morphological words, but underperforms for Estonian/Mandarin. Frequency-informed DDL (FIDDL) beats frequency-informed linear mappings (FIL), but linear methods are better for incremental learning.

Conclusion: Both linear and deep mappings are currently informative for understanding language, with each having specific strengths depending on dataset size, language characteristics, and learning requirements.

Abstract: Recently, deep learning models have increasingly been used in cognitive modelling of language. This study asks whether deep learning can help us to better understand the learning problem that needs to be solved by speakers, above and beyond linear methods. We utilise the Discriminative Lexicon Model introduced by Baayen and colleagues, which models comprehension and production with mappings between numeric form and meaning vectors. While so far, these mappings have been linear (Linear Discriminative Learning, LDL), in the present study we replace them with deep dense neural networks (Deep Discriminative Learning, DDL). We find that DDL affords more accurate mappings for large and diverse datasets from English and Dutch, but not necessarily for Estonian and Taiwan Mandarin. DDL outperforms LDL in particular for words with pseudo-morphological structure such as chol+er. Applied to average reaction times, we find that DDL is outperformed by frequency-informed linear mappings (FIL). However, DDL trained in a frequency-informed way (‘frequency-informed’ deep learning, FIDDL) substantially outperforms FIL. Finally, while linear mappings can very effectively be updated from trial-to-trial to model incremental lexical learning, deep mappings cannot do so as effectively. At present, both linear and deep mappings are informative for understanding language.

Method: PROFILE framework analyzes factor-level preference alignment across summarization, instruction-following, and document-based QA tasks through automated measurement of generation vs. discrimination capabilities.

Result: Significant discrepancy found: LLMs show poor factor-level alignment in text generation but strong alignment in discrimination tasks, revealing a generation-discrimination gap that can be exploited for improvement.

Conclusion: Factor-level analysis is valuable for identifying hidden misalignments, and the generation-discrimination gap provides practical opportunities to enhance LLM-human preference alignment through methods like fine-tuning with self-guidance.

Abstract: Large language models (LLMs) often exhibit tendencies that diverge from human preferences, such as favoring certain writing styles or producing overly verbose outputs. While crucial for improvement, identifying the factors driving these misalignments remains challenging due to existing evaluation methods’ reliance on coarse-grained comparisons and lack of explainability. To address this, we introduce PROFILE, an automated framework to uncover and measure factor-level preference alignment of humans and LLMs. Using PROFILE, we analyze preference alignment across three key tasks: summarization, instruction-following, and document-based QA. We find a significant discrepancy: while LLMs show poor factor-level alignment with human preferences when generating texts, they demonstrate strong alignment in discrimination tasks. We demonstrate how leveraging the identified generation-discrimination gap can be used to improve LLM alignment through multiple approaches, including fine-tuning with self-guidance. Our work highlights the value of factor-level analysis for identifying hidden misalignments and provides a practical framework for improving LLM-human preference alignment.

Method: Leveraged NLG and automated answer grading literature to automate customer support team decision-making. Tested on binary questions and instructional questions, defining correctness based on how the support team makes decisions.

Result: The approach can identify wrong messages in 55% of cases, demonstrating potential for automatically assessing when chatbots provide incorrect or misleading answers.

Conclusion: The work contributes a definition and metrics for assessing correctness, plus suggestions to improve correctness regarding regional language and question type, showing promise for automated quality assessment of LLM-generated customer support responses.

Abstract: Companies support their customers using live chats and chatbots to gain their loyalty. AFAS is a Dutch company aiming to leverage the opportunity large language models (LLMs) offer to answer customer queries with minimal to no input from its customer support team. Adding to its complexity, it is unclear what makes a response correct, and that too in Dutch. Further, with minimal data available for training, the challenge is to identify whether an answer generated by a large language model is correct and do it on the fly. This study is the first to define the correctness of a response based on how the support team at AFAS makes decisions. It leverages literature on natural language generation and automated answer grading systems to automate the decision-making of the customer support team. We investigated questions requiring a binary response (e.g., Would it be possible to adjust tax rates manually?) or instructions (e.g., How would I adjust tax rate manually?) to test how close our automated approach reaches support rating. Our approach can identify wrong messages in 55% of the cases. This work demonstrates the potential for automatically assessing when our chatbot may provide incorrect or misleading answers. Specifically, we contribute (1) a definition and metrics for assessing correctness, and (2) suggestions to improve correctness with respect to regional language and question type.

Method: Comprehensive survey of 100 papers, categorizing TDA applications in NLP into theoretical approaches (topological explanations of linguistic phenomena) and non-theoretical approaches (TDA combined with ML features using various numerical representations).

Result: Identified and categorized existing research efforts, showing TDA’s potential for NLP despite limited adoption. Created taxonomy of approaches and compiled resources.

Conclusion: TDA shows promise for NLP but faces challenges and unresolved questions. The field remains niche but has dedicated research community exploring its applications.

Abstract: The surge of data available on the Internet has led to the adoption of various computational methods to analyze and extract valuable insights from this wealth of information. Among these, the field of Machine Learning (ML) has thrived by leveraging data to extract meaningful insights. However, ML techniques face notable challenges when dealing with real-world data, often due to issues of imbalance, noise, insufficient labeling, and high dimensionality. To address these limitations, some researchers advocate for the adoption of Topological Data Analysis (TDA), a statistical approach that discerningly captures the intrinsic shape of data despite noise. Despite its potential, TDA has not gained as much traction within the Natural Language Processing (NLP) domain compared to structurally distinct areas like computer vision. Nevertheless, a dedicated community of researchers has been exploring the application of TDA in NLP, yielding 100 papers we comprehensively survey in this paper. Our findings categorize these efforts into theoretical and non-theoretical approaches. Theoretical approaches aim to explain linguistic phenomena from a topological viewpoint, while non-theoretical approaches merge TDA with ML features, utilizing diverse numerical representation techniques. We conclude by exploring the challenges and unresolved questions that persist in this niche field. Resources and a list of papers on this topic can be found at: https://github.com/AdaUchendu/AwesomeTDA4NLP.

Method: The survey systematically categorizes world models into two primary functions: (1) constructing internal representations to understand world mechanisms, and (2) predicting future states for simulation and decision-making guidance. It examines current progress and explores applications across various domains.

Result: The review presents a comprehensive taxonomy of world models, analyzes their applications in key domains (generative games, autonomous driving, robotics, social simulacra), and provides a curated collection of representative papers with code repositories.

Conclusion: The survey identifies key challenges in world model development and outlines potential future research directions, emphasizing the importance of world models for advancing artificial general intelligence.

Abstract: The concept of world models has garnered significant attention due to advancements in multimodal large language models such as GPT-4 and video generation models such as Sora, which are central to the pursuit of artificial general intelligence. This survey offers a comprehensive review of the literature on world models. Generally, world models are regarded as tools for either understanding the present state of the world or predicting its future dynamics. This review presents a systematic categorization of world models, emphasizing two primary functions: (1) constructing internal representations to understand the mechanisms of the world, and (2) predicting future states to simulate and guide decision-making. Initially, we examine the current progress in these two categories. We then explore the application of world models in key domains, including generative games, autonomous driving, robotics, and social simulacra, with a focus on how each domain utilizes these aspects. Finally, we outline key challenges and provide insights into potential future research directions. We summarize the representative papers along with their code repositories in https://github.com/tsinghua-fib-lab/World-Model.

Method: Interactive human-in-the-loop system (HITL-GAT) with three customized adversarial text generation methods, demonstrated through a Tibetan script case study.

Result: Established the first adversarial robustness benchmark for Tibetan script, providing a reference for other lower-resourced languages.

Conclusion: HITL-GAT effectively addresses challenges in adversarial text generation for lower-resourced languages and enables creation of sustainable robustness benchmarks.

Abstract: DNN-based language models excel across various NLP tasks but remain highly vulnerable to textual adversarial attacks. While adversarial text generation is crucial for NLP security, explainability, evaluation, and data augmentation, related work remains overwhelmingly English-centric, leaving the problem of constructing high-quality and sustainable adversarial robustness benchmarks for lower-resourced languages both difficult and understudied. First, method customization for lower-resourced languages is complicated due to linguistic differences and limited resources. Second, automated attacks are prone to generating invalid or ambiguous adversarial texts. Last but not least, language models continuously evolve and may be immune to parts of previously generated adversarial texts. To address these challenges, we introduce HITL-GAT, an interactive system based on a general approach to human-in-the-loop generation of adversarial texts. Additionally, we demonstrate the utility of HITL-GAT through a case study on Tibetan script, employing three customized adversarial text generation methods and establishing its first adversarial robustness benchmark, providing a valuable reference for other lower-resourced languages.

Method: Retrieves key relational paths from indexing graph, converts them to text, uses flow-based pruning to reduce redundancy, and employs path-based prompting for LLMs.

Result: Consistently outperforms state-of-the-art baselines across six datasets and five evaluation dimensions.

Conclusion: PathRAG effectively reduces redundant information and guides LLMs to generate more logical and coherent responses through path-based retrieval and prompting.

Abstract: Retrieval-augmented generation (RAG) improves the response quality of large language models (LLMs) by retrieving knowledge from external databases. Typical RAG approaches split the text database into chunks, organizing them in a flat structure for efficient searches. To better capture the inherent dependencies and structured relationships across the text database, researchers propose to organize textual information into an indexing graph, known asgraph-based RAG. However, we argue that the limitation of current graph-based RAG methods lies in the redundancy of the retrieved information, rather than its insufficiency. Moreover, previous methods use a flat structure to organize retrieved information within the prompts, leading to suboptimal performance. To overcome these limitations, we propose PathRAG, which retrieves key relational paths from the indexing graph, and converts these paths into textual form for prompting LLMs. Specifically, PathRAG effectively reduces redundant information with flow-based pruning, while guiding LLMs to generate more logical and coherent responses with path-based prompting. Experimental results show that PathRAG consistently outperforms state-of-the-art baselines across six datasets and five evaluation dimensions. The code is available at the following link: https://github.com/BUPT-GAMMA/PathRAG

Method: Joint evidence selection and answer generation framework optimized through preference alignment with curated pairwise feedback, dynamically identifying relevant content while suppressing noise.

Result: Ext2Gen substantially enhances generation robustness and outperforms methods using independent compression models like Recomp, CompAct, and EXIT, with additional benefits from improved retrieval techniques.

Conclusion: Generation-side enhancements can address limitations that retrieval alone cannot overcome, producing more accurate and faithful answers even under noisy retrieval conditions.

Abstract: Retrieval-augmented generation (RAG) enhances LLMs with external knowledge, yet generation remains vulnerable to retrieval-induced noise and uncertain placement of relevant chunks, often causing hallucinations. We present Ext2Gen, an extract-then-generate framework that strengthens LLMs via joint evidence selection and answer generation, dynamically identifying query-relevant content while suppressing noise, thereby removing the need for any independent pre-generation compression module. Optimized through preference alignment with well-curated pairwise feedback, Ext2Gen produces accurate and faithful answers even under noisy or imprecise retrieval. Experiments demonstrate that it substantially enhances the robustness of the generation backbone and yields greater performance gains than methods relying on independent compression models, e.g., Recomp, CompAct, EXIT). It further benefits from improved retrieval techniques such as query rewriting, underscoring that generation-side enhancements address limitations that retrieval alone cannot overcome.

Method: Train encoder-decoder transformer VLMs (Florence-2 and Gemma-2 based, 0.4B-11.2B params) on synthetic dataset with image-aligned translations, using language prefixes to enable captioning emergence.

Result: Captioning emerges in languages only seen in translation tasks, following scaling laws based on model multilinguality, size, and seen samples; competitive performance achieved on downstream tasks.

Conclusion: Cross-lingual transfer enables zero-shot capabilities in VLMs, with scaling laws governing indirect learning of unseen task-language pairs.

Abstract: Cross-lingual, cross-task transfer is challenged by task-specific data scarcity, which becomes more severe as language support grows and is further amplified in vision-language models (VLMs). We investigate multilingual generalization in encoder-decoder transformer VLMs to enable zero-shot image captioning in languages encountered only in the translation task. In this setting, the encoder must learn to generate generalizable, task-aware latent vision representations to instruct the decoder via inserted cross-attention layers. To analyze scaling behavior, we train Florence-2 based and Gemma-2 based models (0.4B to 11.2B parameters) on a synthetic dataset using varying compute budgets. While all languages in the dataset have image-aligned translations, only a subset of them include image captions. Notably, we show that captioning can emerge using a language prefix, even when this language only appears in the translation task. We find that indirect learning of unseen task-language pairs adheres to scaling laws that are governed by the multilinguality of the model, model size, and seen training samples. Finally, we demonstrate that the scaling laws extend to downstream tasks, achieving competitive performance through fine-tuning in multimodal machine translation (Multi30K, CoMMuTE), lexical disambiguation (CoMMuTE), and image captioning (Multi30K, XM3600, COCO Karpathy).

Method: Proposed a multimodal solution explanation task and created ME2 benchmark with 1,000 math problems annotated with visual keypoints and corresponding explanatory text that references visual elements.

Result: Current models struggle to identify visual keypoints and open-source models face notable difficulties in generating keypoint-based explanations, revealing significant gaps in mathematical visual grounding and visually grounded reasoning.

Conclusion: The multimodal solution explanation task and ME2 dataset will catalyze research on LLMs in education and promote their use as effective, explanation-oriented AI tutors by addressing visual reasoning capabilities.

Abstract: With the rapid advancement of mathematical reasoning capabilities in Large Language Models (LLMs), AI systems are increasingly being adopted in educational settings to support students’ comprehension of problem-solving processes. However, a critical component remains underexplored in current LLM-generated explanations: multimodal explanation. In real-world instructional contexts, human tutors routinely employ visual aids, such as diagrams, markings, and highlights, to enhance conceptual clarity. To bridge this gap, we introduce the multimodal solution explanation task, designed to evaluate whether models can identify visual keypoints, such as auxiliary lines, points, angles, and generate explanations that incorporate these key elements essential for understanding. To evaluate model performance on this task, we propose ME2, a multimodal benchmark consisting of 1,000 math problems annotated with visual keypoints and corresponding explanatory text that references those elements. Our empirical results show that current models struggle to identify visual keypoints. In the task of generating keypoint-based explanations, open-source models also face notable difficulties. This highlights a significant gap in current LLMs’ ability to perform mathematical visual grounding, engage in visually grounded reasoning, and provide explanations in educational contexts. We expect that the multimodal solution explanation task and the ME2 dataset will catalyze further research on LLMs in education and promote their use as effective, explanation-oriented AI tutors.

Method: Created TathyaNyaya dataset from Supreme Court and High Court judgments focusing on factual statements, and developed FactLegalLlama by instruction-tuning LLaMa-3-8B on this dataset. Used transformers for binary judgment prediction combined with FactLegalLlama for explanation generation.

Result: TathyaNyaya surpasses existing datasets in scale and diversity, establishing benchmarks for explainable AI in legal analysis. The framework enhances predictive performance and interpretability through factual precision and domain-specific tuning.

Conclusion: TathyaNyaya and FactLegalLlama serve as foundational resources for AI-assisted legal decision-making, emphasizing the importance of factual precision and domain-specific approaches for transparency and interpretability in legal AI systems.

Abstract: In the landscape of Fact-based Judgment Prediction and Explanation (FJPE), reliance on factual data is essential for developing robust and realistic AI-driven decision-making tools. This paper introduces TathyaNyaya, the largest annotated dataset for FJPE tailored to the Indian legal context, encompassing judgments from the Supreme Court of India and various High Courts. Derived from the Hindi terms “Tathya” (fact) and “Nyaya” (justice), the TathyaNyaya dataset is uniquely designed to focus on factual statements rather than complete legal texts, reflecting real-world judicial processes where factual data drives outcomes. Complementing this dataset, we present FactLegalLlama, an instruction-tuned variant of the LLaMa-3-8B Large Language Model (LLM), optimized for generating high-quality explanations in FJPE tasks. Finetuned on the factual data in TathyaNyaya, FactLegalLlama integrates predictive accuracy with coherent, contextually relevant explanations, addressing the critical need for transparency and interpretability in AI-assisted legal systems. Our methodology combines transformers for binary judgment prediction with FactLegalLlama for explanation generation, creating a robust framework for advancing FJPE in the Indian legal domain. TathyaNyaya not only surpasses existing datasets in scale and diversity but also establishes a benchmark for building explainable AI systems in legal analysis. The findings underscore the importance of factual precision and domain-specific tuning in enhancing predictive performance and interpretability, positioning TathyaNyaya and FactLegalLlama as foundational resources for AI-assisted legal decision-making.

Method: The proposed Swin-VIB framework integrates a pipeline of variational information bottleneck models to adapt the retrieved information difference, facilitating robust response generation in conflicting contexts.

Result: Extensive experiments show Swin-VIB outperforms all competitive baselines in multiple-choice task accuracy and improves EM values in open-ended QA tasks by at least 11.14%.

Conclusion: The framework effectively addresses knowledge conflicts in LLMs by leveraging information-theoretic principles to reduce uncertainty and improve response reliability.

Abstract: The proliferation of large language models (LLMs) has significantly advanced intelligent systems. Unfortunately, LLMs often face knowledge conflicts between internal memory and retrieved external information, arising from misinformation, biases, or outdated knowledge. These conflicts undermine response reliability and introduce uncertainty in decision-making. In this work, we analyze how LLMs navigate knowledge conflicts from an information-theoretic perspective and reveal that when conflicting and supplementary information exhibit significant differences, LLMs confidently resolve their preferences and alleviate the uncertainty during their response generation. When this difference is ambiguous, LLMs experience considerable uncertainty about their generation. Based on this insight, we propose Swin-VIB, a novel framework that integrates a pipeline of variational information bottleneck models to adapt the retrieved information difference, facilitating robust response generation of LLMs even in conflicting contexts. Extensive experiments confirm our theoretical analysis and demonstrate the performance of Swin-VIB. Notably, Swin-VIB outperforms all competitive baselines in terms of the accuracy of the multiple-choice task, while improving the EM values in the open-ended QA task by at least 11.14%.

Method: Three-component architecture: Generator creates content, Evaluator assesses quality using metrics, Reflector provides improvement feedback. Built on structured dataset of 4,287 medical papers decomposed into research components.

Result: Significant improvements over conventional approaches (9.91% average gain with DeepSeek V3, comparable with GPT-4o Mini). Human evaluation shows quality comparable to human-authored papers, especially strong in synthesizing future research directions.

Conclusion: FRAME provides a robust foundation for automated medical research paper generation that maintains academic standards and can efficiently assist medical research.

Abstract: The automation of scientific research through large language models (LLMs) presents significant opportunities but faces critical challenges in knowledge synthesis and quality assurance. We introduce Feedback-Refined Agent Methodology (FRAME), a novel framework that enhances medical paper generation through iterative refinement and structured feedback. Our approach comprises three key innovations: (1) A structured dataset construction method that decomposes 4,287 medical papers into essential research components through iterative refinement; (2) A tripartite architecture integrating Generator, Evaluator, and Reflector agents that progressively improve content quality through metric-driven feedback; and (3) A comprehensive evaluation framework that combines statistical metrics with human-grounded benchmarks. Experimental results demonstrate FRAME’s effectiveness, achieving significant improvements over conventional approaches across multiple models (9.91% average gain with DeepSeek V3, comparable improvements with GPT-4o Mini) and evaluation dimensions. Human evaluation confirms that FRAME-generated papers achieve quality comparable to human-authored works, with particular strength in synthesizing future research directions. The results demonstrated our work could efficiently assist medical research by building a robust foundation for automated medical research paper generation while maintaining rigorous academic standards.

Method: Developed a large tagged parallel dataset (taggedPBC) with POS-tagged text data from diverse languages, validated against SOTA taggers and hand-tagged corpora, and introduced the N1 ratio measure for word order analysis.

Result: The dataset shows high accuracy correlation with existing taggers and hand-tagged corpora. The N1 ratio correlates with expert word order determinations, enabling accurate classification of intransitive word order using Gaussian Naive Bayes.

Conclusion: taggedPBC represents a significant advancement for corpus-based crosslinguistic research, available via GitHub for collaboration, though further expansion is needed.

Abstract: Existing datasets available for crosslinguistic investigations have tended to focus on large amounts of data for a small group of languages or a small amount of data for a large number of languages. This means that claims based on these datasets are limited in what they reveal about universal properties of the human language faculty. While this has begun to change through the efforts of projects seeking to develop tagged corpora for a large number of languages, such efforts are still constrained by limits on resources. The current paper reports on a large tagged parallel dataset which has been developed to partially address this issue. The taggedPBC contains POS-tagged parallel text data from more than 1,940 languages, representing 155 language families and 78 isolates, dwarfing previously available resources. The accuracy of particular tags in this dataset is shown to correlate well with both existing SOTA taggers for high-resource languages (SpaCy, Trankit) as well as hand-tagged corpora (Universal Dependencies Treebanks). Additionally, a novel measure derived from this dataset, the N1 ratio, correlates with expert determinations of intransitive word order in three typological databases (WALS, Grambank, Autotyp) such that a Gaussian Naive Bayes classifier trained on this feature can accurately identify basic intransitive word order for languages not in those databases. While much work is still needed to expand and develop this dataset, the taggedPBC is an important step to enable corpus-based crosslinguistic investigations, and is made available for research and collaboration via GitHub.

Method: Proposed PDRR framework with four stages: Predict (question type), Decompose (into structured triples), Retrieve (from KBs), and Reason (using LLM as agent to complete triples).

Result: Experimental results show PDRR consistently outperforms existing methods across various LLM backbones and achieves superior performance on both chain-structured and non-chain complex questions.

Conclusion: The PDRR framework effectively addresses limitations of both LLM-only and chain-based approaches by incorporating planning and logical structuring through semantic parsing-inspired decomposition.

Abstract: Knowledge Base Question Answering (KBQA) aims to answer natural language questions using structured knowledge from KBs. While LLM-only approaches offer generalization, they suffer from outdated knowledge, hallucinations, and lack of transparency. Chain-based KG-RAG methods address these issues by incorporating external KBs, but are limited to simple chain-structured questions due to the absence of planning and logical structuring. Inspired by semantic parsing methods, we propose PDRR: a four-stage framework consisting of Predict, Decompose, Retrieve, and Reason. Our method first predicts the question type and decomposes the question into structured triples. Then retrieves relevant information from KBs and guides the LLM as an agent to reason over and complete the decomposed triples. Experimental results demonstrate that PDRR consistently outperforms existing methods across various LLM backbones and achieves superior performance on both chain-structured and non-chain complex questions.

Method: Introduced FRAME benchmark with diverse score distributions, defined visual biases, and tested LVLM judges’ vulnerability by injecting these biases into the benchmark. Also examined combined biases, pairwise evaluations, and prompt-based mitigation strategies.

Result: All tested LVLM judges showed vulnerability across all domains, consistently inflating scores for manipulated images. Combining multiple biases amplified effects, and visual biases persisted despite mitigation attempts.

Conclusion: Current LVLM evaluation systems have significant vulnerability to visual manipulations, highlighting the urgent need for more robust LVLM judges that can resist adversarial visual biases.

Abstract: Recently, large vision-language models (LVLMs) have emerged as the preferred tools for judging text-image alignment, yet their robustness along the visual modality remains underexplored. This work is the first study to address a key research question: Can adversarial visual manipulations systematically fool LVLM judges into assigning unfairly inflated scores? We define potential image induced biases within the context of T2I evaluation and examine how these biases affect the evaluations of LVLM judges. Moreover, we introduce a novel, fine-grained, multi-domain meta-evaluation benchmark named FRAME, which is deliberately constructed to exhibit diverse score distributions. By introducing the defined biases into the benchmark, we reveal that all tested LVLM judges exhibit vulnerability across all domains, consistently inflating scores for manipulated images. Further analysis reveals that combining multiple biases amplifies their effects, and pairwise evaluations are similarly susceptible. Moreover, we observe that visual biases persist under prompt-based mitigation strategies, highlighting the vulnerability of current LVLM evaluation systems and underscoring the urgent need for more robust LVLM judges.

Method: Parameter-efficient fine-tuning using newly curated dataset of 20k doctor-patient Q&A pairs and 60% of 90M-token crawled corpus from Persian medical magazines on aya-expanse-8b baseline model.

Result: Fine-tuned model achieved improved medical question answering accuracy, successfully passed Iranian Basic Medical Science Entrance Exam (which baseline failed), and improved Persian-translated MMLU accuracy by average 2.67%.

Conclusion: Open-access online data can effectively enrich small language models for medical fields in low-resource languages, providing practical solutions for resource-constrained environments.

Abstract: The rapid advancement of language models has demonstrated the potential of artificial intelligence in the healthcare industry. However, small language models struggle with specialized domains in low-resource languages like Persian. While numerous medical-domain websites exist in Persian, no curated dataset or corpus has been available making ours the first of its kind. This study introduces a newly curated dataset comprising 20k doctor-patient Q&A pairs and 60% of a 90-million-token crawled corpus from medical magazines. Using a parameter-efficient fine-tuning approach, we enhanced the medical knowledge of the baseline model, aya-expanse-8b. Benchmark evaluations demonstrate that the fine-tuned model achieves improved accuracy in medical question answering and successfully passed the Iranian Basic Medical Science Entrance Exam (IBSEE) in September 2023, which the baseline model did not. Additionally, the fine-tuned model improved Persian-translated MMLU accuracy by an average of 2.67%. This work highlights the potential of leveraging open-access online data to enrich small language models in medical fields, providing a novel solution for Persian medical AI applications suitable for resource-constrained environments. Future research could explore multimodal input to further enhance performance.

Method: Adversarial Scenario Extrapolation (ASE) framework that leverages Chain-of-Thought reasoning to guide LLMs through self-generative processes of contemplating potential adversarial scenarios and formulating defensive strategies before responding to user queries.

Result: Achieves near-zero jailbreak attack success rates and minimal toxicity, reduces outright rejections to <4%, outperforms six state-of-the-art defenses in robustness-seamlessness trade-offs, with 92-99% accuracy on adversarial Q&A and 4-10x lower bias scores.

Conclusion: ASE transforms adversarial perception into an intrinsic cognitive process, setting a new paradigm for secure and natural human-AI interaction by simultaneously enhancing robustness and seamlessness.

Abstract: Large Language Models (LLMs) exhibit impressive capabilities, but remain susceptible to a growing spectrum of safety risks, including jailbreaks, toxic content, hallucinations, and bias. Existing defenses often address only a single threat type or resort to rigid outright rejection, sacrificing user experience and failing to generalize across diverse and novel attacks. This paper introduces Adversarial Scenario Extrapolation (ASE), a novel inference-time computation framework that leverages Chain-of-Thought (CoT) reasoning to simultaneously enhance LLM robustness and seamlessness. ASE guides the LLM through a self-generative process of contemplating potential adversarial scenarios and formulating defensive strategies before generating a response to the user query. Comprehensive evaluation on four adversarial benchmarks with four latest LLMs shows that ASE achieves near-zero jailbreak attack success rates and minimal toxicity, while slashing outright rejections to <4%. ASE outperforms six state-of-the-art defenses in robustness-seamlessness trade-offs, with 92-99% accuracy on adversarial Q&A and 4-10x lower bias scores. By transforming adversarial perception into an intrinsic cognitive process, ASE sets a new paradigm for secure and natural human-AI interaction.

Method: Created SCRum-9 dataset with 7,516 tweets across 9 languages, annotated by native speakers with confidence scores. Benchmarked 5 LLMs and 2 MLMs using ICL and fine-tuning. Explored multilingual synthetic data generation from LLMs for fine-tuning smaller models.

Result: LLMs with weak ICL performance can generate valuable synthetic data that enables small MLMs to outperform zero-shot ICL in LLMs. Model predictions often match human annotators’ second-choice labels rather than diverging from human judgments.

Conclusion: SCRum-9 provides a valuable resource for multilingual rumour analysis research. Synthetic data from LLMs can effectively boost performance of smaller models, and model predictions show alignment with human uncertainty patterns in ambiguous cases.

Abstract: We introduce SCRum-9, the largest multilingual Stance Classification dataset for Rumour analysis in 9 languages, containing 7,516 tweets from X. SCRum-9 goes beyond existing stance classification datasets by covering more languages, linking examples to more fact-checked claims (2.1k), and including confidence-related annotations from multiple annotators to account for intra- and inter-annotator variability. Annotations were made by at least two native speakers per language, totalling more than 405 hours of annotation and 8,150 dollars in compensation. Further, SCRum-9 is used to benchmark five large language models (LLMs) and two multilingual masked language models (MLMs) in In-Context Learning (ICL) and fine-tuning setups. This paper also innovates by exploring the use of multilingual synthetic data for rumour stance classification, showing that even LLMs with weak ICL performance can produce valuable synthetic data for fine-tuning small MLMs, enabling them to achieve higher performance than zero-shot ICL in LLMs. Finally, we examine the relationship between model predictions and human uncertainty on ambiguous cases finding that model predictions often match the second-choice labels assigned by annotators, rather than diverging entirely from human judgments. SCRum-9 is publicly released to the research community with potential to foster further research on multilingual analysis of misleading narratives on social media.

Method: Proposes \method with an extensible toolkit (web search, forgery detection, consistency analysis) and MCTS with multi-source verification. Uses greedy search to select relevant tools, then MCTS with dual rewards for balanced exploration-exploitation.

Result: Extensive experiments show \method consistently outperforms existing baselines on challenging mixed-source multimodal misinformation benchmarks.

Conclusion: \method demonstrates strong potential as a training-free detector for complex multimodal misinformation, with effective tree search mechanisms and tool usage validated through ablation studies.

Abstract: Real-world multimodal misinformation often arises from mixed forgery sources, requiring dynamic reasoning and adaptive verification. However, existing methods mainly rely on static pipelines and limited tool usage, limiting their ability to handle such complexity and diversity. To address this challenge, we propose \method, a novel misinformation detection agent that incorporates an extensible toolkit with Monte Carlo Tree Search (MCTS). The toolkit consists of modular tools such as web search, forgery detection, and consistency analysis. Each tool is described using standardized templates, enabling seamless integration and future expansion. To avoid inefficiency from using all tools simultaneously, a greedy search-based selector is proposed to identify a task-relevant subset. This subset then serves as the action space for MCTS to dynamically collect evidence and perform multi-source verification. To better align MCTS with the multi-source nature of misinformation detection, \method~ extends traditional MCTS with multi-source verification, which decomposes the task into coordinated subtasks targeting different forgery sources. A dual reward mechanism containing a reasoning trajectory score and a confidence score is further proposed to encourage a balance between exploration across mixed forgery sources and exploitation for more reliable evidence. We conduct ablation studies to confirm the effectiveness of the tree search mechanism and tool usage. Extensive experiments further show that \method~ consistently outperforms existing baselines on challenging mixed-source multimodal misinformation benchmarks, demonstrating its strong potential as a training-free detector.

Method: LAQ generates low-cost pseudo lookahead queries to serve as observation windows for importance estimation, enabling more accurate KV cache eviction aligned with real inference scenarios.

Result: Experimental results on LongBench and Needle-in-a-Haystack benchmarks show LAQ outperforms existing methods across various budget levels, achieving 1-4 point improvement on LongBench under limited cache budget.

Conclusion: LAQ provides a more consistent and accurate KV cache eviction framework that is complementary to existing approaches and can be flexibly combined for further improvements.

Abstract: Large language models (LLMs) rely on key-value cache (KV cache) to accelerate decoding by reducing redundant computations. However, the KV cache memory usage grows substantially with longer text sequences, posing challenges for efficient deployment. Existing KV cache eviction methods prune tokens using prefilling-stage attention scores, causing inconsistency with actual inference queries, especially under tight memory budgets. In this paper, we propose Lookahead Q-Cache (LAQ), a novel eviction framework that generates low-cost pseudo lookahead queries to better approximate the true decoding-stage queries. By using these lookahead queries as the observation window for importance estimation, LAQ achieves more consistent and accurate KV cache eviction aligned with real inference scenarios. Experimental results on LongBench and Needle-in-a-Haystack benchmarks show that LAQ outperforms existing methods across various budget levels, achieving a 1 $\sim$ 4 point improvement on LongBench under limited cache budget. Moreover, LAQ is complementary to existing approaches and can be flexibly combined to yield further improvements.

Method: Leverages the observation that language models assign most probability mass to few tokens. Designs a temporal difference learning framework that operates on a reduced action space (vocabulary subset) rather than the full vocabulary.

Result: Demonstrates how practical algorithms can be derived from this framework and shows resulting performance improvements in distillation.

Conclusion: The proposed temporal difference-based distillation framework effectively exploits teacher model sparsity to create more efficient distillation methods with improved performance.

Abstract: Large language models have led to significant progress across many NLP tasks, although their massive sizes often incur substantial computational costs. Distillation has become a common practice to compress these large and highly capable models into smaller, more efficient ones. Many existing language model distillation methods can be viewed as behavior cloning from the perspective of imitation learning or inverse reinforcement learning. This viewpoint has inspired subsequent studies that leverage (inverse) reinforcement learning techniques, including variations of behavior cloning and temporal difference learning methods. Rather than proposing yet another specific temporal difference method, we introduce a general framework for temporal difference-based distillation by exploiting the distributional sparsity of the teacher model. Specifically, it is often observed that language models assign most probability mass to a small subset of tokens. Motivated by this observation, we design a temporal difference learning framework that operates on a reduced action space (a subset of vocabulary), and demonstrate how practical algorithms can be derived and the resulting performance improvements.

Method: Uses retrieval-augmented generation (RAG) to dynamically incorporate relevant knowledge for precise classification.

Result: Outperforms traditional fine-tuning, zero-shot, and few-shot methods in both in-domain and out-of-domain scenarios on real-world datasets.

Conclusion: Demonstrates potential for real-world deployment in adaptive and large-scale intent classification systems.

Abstract: Accurate intent classification is critical for efficient routing in customer service, ensuring customers are connected with the most suitable agents while reducing handling times and operational costs. However, as companies expand their product lines, intent classification faces scalability challenges due to the increasing number of intents and variations in taxonomy across different verticals. In this paper, we introduce REIC, a Retrieval-augmented generation Enhanced Intent Classification approach, which addresses these challenges effectively. REIC leverages retrieval-augmented generation (RAG) to dynamically incorporate relevant knowledge, enabling precise classification without the need for frequent retraining. Through extensive experiments on real-world datasets, we demonstrate that REIC outperforms traditional fine-tuning, zero-shot, and few-shot methods in large-scale customer service settings. Our results highlight its effectiveness in both in-domain and out-of-domain scenarios, demonstrating its potential for real-world deployment in adaptive and large-scale intent classification systems.

Method: Two-phase approach: 1) Incrementally process input chunks with compressed KV cache, cross-layer context embeddings, and early exit strategy; 2) Identify essential tokens via similarity matching and selectively recompute KV cache.

Result: Achieves over 52% and 34% performance gains on RULER and BABILong at 1M context length, outperforms baselines on Infinite-Bench, RepoEval, and MM-NIAH, reduces inference time by 30% and peak memory usage by 5%.

Conclusion: REFORM demonstrates both efficiency and superior performance across diverse tasks and domains, providing an effective solution for long-context processing challenges.

Abstract: As large language models increasingly gain popularity in real-world applications, processing extremely long contexts, often exceeding the model’s pre-trained context limits, has emerged as a critical challenge. While existing approaches to efficient long-context processing show promise, recurrent compression-based methods struggle with information preservation, whereas random access approaches require substantial memory resources. We introduce REFORM, a novel inference framework that efficiently handles long contexts through a two-phase approach. First, it incrementally processes input chunks while maintaining a compressed KV cache, constructs cross-layer context embeddings, and utilizes early exit strategy for improved efficiency. Second, it identifies and gathers essential tokens via similarity matching and selectively recomputes the KV cache. Compared to baselines, REFORM achieves over 52% and 34% performance gains on RULER and BABILong respectively at 1M context length. It also outperforms baselines on Infinite-Bench, RepoEval, and MM-NIAH, demonstrating flexibility across diverse tasks and domains. Additionally, REFORM reduces inference time by 30% and peak memory usage by 5%, achieving both efficiency and superior performance.

Method: Introduces Formal Description of Visualization (FDV) as structured textual representation for charts, and a 4-stage framework: researching, exemplar report textualization, planning, and multimodal report generation.

Result: Achieves 82% overall win rate over baseline using Claude 3.7 Sonnet model, with evaluation on MultimodalReportBench containing 100 diverse topics and 5 dedicated metrics.

Conclusion: The proposed framework effectively addresses the challenge of generating high-quality multimodal reports with integrated visualizations, demonstrating significant improvement over text-only approaches.

Abstract: Visualizations play a crucial part in effective communication of concepts and information. Recent advances in reasoning and retrieval augmented generation have enabled Large Language Models (LLMs) to perform deep research and generate comprehensive reports. Despite its progress, existing deep research frameworks primarily focus on generating text-only content, leaving the automated generation of interleaved texts and visualizations underexplored. This novel task poses key challenges in designing informative visualizations and effectively integrating them with text reports. To address these challenges, we propose Formal Description of Visualization (FDV), a structured textual representation of charts that enables LLMs to learn from and generate diverse, high-quality visualizations. Building on this representation, we introduce Multimodal DeepResearcher, an agentic framework that decomposes the task into four stages: (1) researching, (2) exemplar report textualization, (3) planning, and (4) multimodal report generation. For the evaluation of generated multimodal reports, we develop MultimodalReportBench, which contains 100 diverse topics served as inputs along with 5 dedicated metrics. Extensive experiments across models and evaluation methods demonstrate the effectiveness of Multimodal DeepResearcher. Notably, utilizing the same Claude 3.7 Sonnet model, Multimodal DeepResearcher achieves an 82% overall win rate over the baseline method.

Method: Uses stochastic adaptive N-gram drafting with memory-efficient logit-based N-gram module, Gumbel-Top-K sampling, and data-driven tree construction to predict tokens without separate draft models.

Result: Reduces inference latency by 60-65% compared to standard autoregressive decoding while maintaining accuracy across multiple reasoning tasks (AIME-2024, GPQA-Diamond, LiveCodeBench).

Conclusion: STAND provides a plug-and-play solution that outperforms state-of-the-art speculative decoding methods and can be applied to any existing language model without additional training.

Abstract: Language models have demonstrated remarkable capabilities in reasoning tasks through test-time scaling techniques like best-of-N sampling and tree search. However, these approaches often demand substantial computational resources, creating a critical trade-off between performance and efficiency. We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach that exploits the inherent redundancy in reasoning trajectories to achieve significant acceleration without compromising accuracy. Our analysis shows that reasoning paths frequently reuse similar reasoning patterns, enabling efficient model-free token prediction without requiring separate draft models. By introducing stochastic drafting and preserving probabilistic information through a memory-efficient logit-based N-gram module, combined with optimized Gumbel-Top-K sampling and data-driven tree construction, STAND significantly improves token acceptance rates. Extensive evaluations across multiple models and reasoning tasks (AIME-2024, GPQA-Diamond, and LiveCodeBench) demonstrate that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding while maintaining accuracy. Furthermore, STAND consistently outperforms state-of-the-art speculative decoding methods across diverse inference patterns, including single-trajectory decoding, batch decoding, and test-time tree search. As a model-free approach, STAND can be applied to any existing language model without additional training, making it a powerful plug-and-play solution for accelerating language model reasoning.

Method: RACE extracts essential reasoning steps and computes four diagnostic signals: inter-sample consistency of reasoning traces, entropy-based answer uncertainty, semantic alignment between reasoning and answers, and internal coherence of reasoning.

Result: Experiments across datasets and different LLMs show that RACE outperforms existing hallucination detection baselines.

Conclusion: RACE provides a robust and generalizable solution for evaluating Large Reasoning Models by detecting hallucinations in their reasoning traces.

Abstract: Large Reasoning Models (LRMs) extend large language models with explicit, multi-step reasoning traces to enhance transparency and performance on complex tasks. However, these reasoning traces can be redundant or logically inconsistent, becoming a new and hard-to-detect source of hallucination. Existing hallucination detection methods focus primarily on answer-level uncertainty and often fail to detect hallucinations or logical inconsistencies arising from the model’s reasoning trace. This oversight is particularly problematic for LRMs, where the explicit thinking trace is not only an important support to the model’s decision-making process but also a key source of potential hallucination. To this end, we propose RACE (Reasoning and Answer Consistency Evaluation), a novel framework specifically tailored for hallucination detection in LRMs. RACE operates by extracting essential reasoning steps and computing four diagnostic signals: inter-sample consistency of reasoning traces, entropy-based answer uncertainty, semantic alignment between reasoning and answers, and internal coherence of reasoning. The joint utilization of these signals makes RACE a more robust detector of hallucinations in LRMs. Experiments across datasets and different LLMs demonstrate that RACE outperforms existing hallucination detection baselines, offering a robust and generalizable solution for evaluating LRMs. The source code is available at https://github.com/bebr2/RACE

Method: MAPLE uses 4 specialized agents: Solver (ReAct reasoning), Checker (answer verification), Reflector (error diagnosis and strategy correction), and Archiver (long-term memory management for experience reuse and evolution).

Result: Experiments on WiKiTQ and TabFact show significant improvements over existing methods, achieving state-of-the-art performance across multiple LLM backbones.

Conclusion: The MAPLE framework successfully mimics human problem-solving through multi-agent collaboration with feedback loops and long-term memory, effectively addressing limitations of current LLM approaches for complex table-based QA.

Abstract: Table-based question answering requires complex reasoning capabilities that current LLMs struggle to achieve with single-pass inference. Existing approaches, such as Chain-of-Thought reasoning and question decomposition, lack error detection mechanisms and discard problem-solving experiences, contrasting sharply with how humans tackle such problems. In this paper, we propose MAPLE (Multi-agent Adaptive Planning with Long-term mEmory), a novel framework that mimics human problem-solving through specialized cognitive agents working in a feedback-driven loop. MAPLE integrates 4 key components: (1) a Solver using the ReAct paradigm for reasoning, (2) a Checker for answer verification, (3) a Reflector for error diagnosis and strategy correction, and (4) an Archiver managing long-term memory for experience reuse and evolution. Experiments on WiKiTQ and TabFact demonstrate significant improvements over existing methods, achieving state-of-the-art performance across multiple LLM backbones.

Method: Proposes two strategies: 1) Contrastive reasoning feedback comparing embeddings against strong/weak baselines, and 2) Residual embedding refinement that stabilizes updates by integrating current and historical gradients.

Result: Achieved 5% accuracy gain on MathQA without additional training, with extensive experiments on five reasoning benchmarks demonstrating effectiveness.

Conclusion: The proposed framework effectively refines latent reasoning trajectories, enabling improved reasoning accuracy through controlled embedding updates without explicit token generation.

Abstract: Reasoning is a key component of language understanding in Large Language Models. While Chain-of-Thought prompting enhances performance via explicit intermediate steps, it suffers from sufficient token overhead and a fixed reasoning trajectory, preventing step-wise refinement. Recent advances in latent reasoning address these limitations by refining internal reasoning processes directly in the model’s latent space, without producing explicit outputs. However, a key challenge remains: how to effectively update reasoning embeddings during post-training to guide the model toward more accurate solutions. To overcome this challenge, we propose a lightweight post-training framework that refines latent reasoning trajectories using two novel strategies: 1) Contrastive reasoning feedback, which compares reasoning embeddings against strong and weak baselines to infer effective update directions via embedding enhancement; 2) Residual embedding refinement, which stabilizes updates by progressively integrating current and historical gradients, enabling fast yet controlled convergence. Extensive experiments and case studies are conducted on five reasoning benchmarks to demonstrate the effectiveness of the proposed framework. Notably, a 5% accuracy gain on MathQA without additional training.

Method: Created ToxSyn via a controllable four-stage pipeline that generates Portuguese hate speech data with discourse-type annotations (capturing rhetorical strategies like sarcasm/dehumanization) and systematically includes non-toxic counterexamples about minorities.

Result: Experiments revealed catastrophic mutual generalization failure: models trained on social media fail on minority-specific contexts, and vice-versa. Summary metrics like Macro F1 mask model failures and are unreliable indicators of true performance.

Conclusion: Social media hate speech detection and minority-specific hate speech detection are distinct tasks. ToxSyn is publicly released to advance synthetic data generation and benchmark progress in hate speech detection for low- and mid-resource languages.

Abstract: The development of robust hate speech detection systems remains limited by the lack of large-scale, fine-grained training data, especially for languages beyond English. Existing corpora typically rely on coarse toxic/non-toxic labels, and the few that capture hate directed at specific minority groups critically lack the non-toxic counterexamples (i.e., benign text about minorities) required to distinguish genuine hate from mere discussion. We introduce ToxSyn, the first Portuguese large-scale corpus explicitly designed for multi-label hate speech detection across nine protected minority groups. Generated via a controllable four-stage pipeline, ToxSyn includes discourse-type annotations to capture rhetorical strategies of toxic language, such as sarcasm or dehumanization. Crucially, it systematically includes the non-toxic counterexamples absent in all other public datasets. Our experiments reveal a catastrophic, mutual generalization failure between social-media domains and ToxSyn: models trained on social media struggle to generalize to minority-specific contexts, and vice-versa. This finding indicates they are distinct tasks and exposes summary metrics like Macro F1 can be unreliable indicators of true model behavior, as they completely mask model failure. We publicly release ToxSyn at HuggingFace to foster reproducible research on synthetic data generation and benchmark progress in hate-speech detection for low- and mid-resource languages.

Method: Pairs a ranker with an autoregressive neural processor to select and contextualize only the most relevant tokens, decoupling sequence length from context width.

Result: Avey performs comparably to Transformer on standard short-range NLP benchmarks and significantly outperforms it on long-range dependency modeling tasks.

Conclusion: Avey provides an effective alternative to both attention-based and recurrent architectures, enabling efficient processing of arbitrarily long sequences while maintaining strong performance.

Abstract: The Transformer has become the de facto standard for modern language models owing to its parallelizable training and effective autoregressive decoding. However, its fixed context window and the quadratic time and memory costs of its self-attention mechanism remain central bottlenecks. These constraints have revived interest in recurrent architectures that scale linearly with sequence length, but at the cost of reduced parallelism. In this paper, we introduce Avey, a new foundational architecture that breaks away from both attention and recurrence. Avey pairs a ranker with an autoregressive neural processor to select and contextualize only the most relevant tokens for any given token. Specifically, it decouples sequence length from context width, thus enabling effective and efficient processing of arbitrarily long sequences. Results show that Avey compares favorably to the Transformer across a variety of standard short-range NLP benchmarks, while significantly outperforming it on tasks requiring long-range dependency modeling.

Method: Propose RATTENTION - local attention integrated with specialized linear attention mechanism to capture information from out-of-window tokens, implemented with custom Pallas kernels.

Result: RATTENTION with 512 window size consistently matches full-attention performance across diverse settings, achieves superior Pareto tradeoff between performance and efficiency, and shows enhanced long-context performance on RULER benchmark.

Conclusion: RATTENTION shifts the Pareto frontier for local-global attention models, enabling significant efficiency gains without compromising performance, with open-sourced implementation for further research.

Abstract: Local-global attention models have recently emerged as compelling alternatives to standard Transformers, promising improvements in both training and inference efficiency. However, the crucial choice of window size presents a Pareto tradeoff: larger windows maintain performance akin to full attention but offer minimal efficiency gains in short-context scenarios, while smaller windows can lead to performance degradation. Current models, such as Gemma2 and Mistral, adopt conservative window sizes (e.g., 4096 out of an 8192 pretraining length) to preserve performance. This work investigates strategies to shift this Pareto frontier, enabling local-global models to achieve efficiency gains even in short-context regimes. Our core motivation is to address the intrinsic limitation of local attention – its complete disregard for tokens outside the defined window. We explore RATTENTION, a variant of local attention integrated with a specialized linear attention mechanism designed to capture information from these out-of-window tokens. Pretraining experiments at the 3B and 12B scales demonstrate that RATTENTION achieves a superior Pareto tradeoff between performance and efficiency. As a sweet spot, RATTENTION with a window size of just 512 consistently matches the performance of full-attention models across diverse settings. Furthermore, the recurrent nature inherent in the linear attention component of RATTENTION contributes to enhanced long-context performance, as validated on the RULER benchmark. Crucially, these improvements do not compromise training efficiency; thanks to a specialized kernel implementation and the reduced window size, RATTENTION maintains training speeds comparable to existing state-of-the-art approaches. We open-sourced our Pallas kernels along with model codes to facilitate further research effort.

Method: Introduces emergent local memory - a continuous-learning completely-parallel content-addressable memory encoding global order through self-organizing dynamics.

Result: The system demonstrates the ability to produce human language without data by exploiting its own self-organizing dynamics, showing that words arise as a side-effect of emergent symbolic order.

Conclusion: This work answers essential questions about the existence and origin of human language data, revealing that human language patterns reflect a universal subregular mechanism of word formation.

Abstract: We introduce a novel paradigm of emergent local memory. It is a continuous-learning completely-parallel content-addressable memory encoding global order. It demonstrates how local constraints on uncoordinated learning can produce topologically protected memories realizing emergent symbolic order. It is therefore a neuro-symbolic bridge. It further has the ability to produce human language without data, by exploiting its own self-organizing dynamics. It teaches us that words arise as a side-effect of emergent symbolic order, and that human language patterns at all structural levels reflect a universal mechanism of word formation (which is subregular). This work answers essential questions about the existence & origin of all the human language data.

Method: Introduced sAwMIL (Sparse-Aware Multiple-Instance Learning), a multiclass probing framework combining multiple-instance learning with conformal prediction, using internal LLM activations to classify statements as true, false, or neither.

Result: Evaluation across 16 LLMs showed: (1) common probing methods are unreliable and sometimes worse than zero-shot prompting; (2) truth and falsehood are encoded asymmetrically; (3) LLMs encode a third distinct signal beyond true/false.

Conclusion: Existing probing methods have fundamental flaws, and sAwMIL provides a more reliable framework for understanding how LLMs encode knowledge, revealing complex asymmetric encoding patterns with a third distinct signal type.

Abstract: The public often attributes human-like qualities to large language models (LLMs) and assumes they “know” certain things. In reality, LLMs encode information retained during training as internal probabilistic knowledge. This study examines existing methods for probing the veracity of that knowledge and identifies several flawed underlying assumptions. To address these flaws, we introduce sAwMIL (Sparse-Aware Multiple-Instance Learning), a multiclass probing framework that combines multiple-instance learning with conformal prediction. sAwMIL leverages internal activations of LLMs to classify statements as true, false, or neither. We evaluate sAwMIL across 16 open-source LLMs, including default and chat-based variants, on three new curated datasets. Our results show that (1) common probing methods fail to provide a reliable and transferable veracity direction and, in some settings, perform worse than zero-shot prompting; (2) truth and falsehood are not encoded symmetrically; and (3) LLMs encode a third type of signal that is distinct from both true and false.

Method: Two-stage training framework with multi-query parallelism that enables LLMs to adaptively leverage internal and external knowledge during reasoning, transitioning from single-query to parallel processing.

Result: Outperforms strongest baseline by up to 13.7% on seven QA benchmarks and decreases inference time by 11.1%.

Conclusion: Multi-query parallelism in RAG-R1 effectively enhances reasoning robustness while significantly reducing inference latency compared to single-query approaches.

Abstract: Large Language Models (LLMs), despite their remarkable capabilities, are prone to generating hallucinated or outdated content due to their static internal knowledge. While Retrieval-Augmented Generation (RAG) integrated with Reinforcement Learning (RL) offers a solution, these methods are fundamentally constrained by a single-query mode, leading to prohibitive latency and inherent brittleness. To overcome these limitations, we introduce RAG-R1, a novel two-stage training framework centered around multi-query parallelism. Our framework enables LLMs to adaptively leverage internal and external knowledge during the reasoning process while transitioning from the single-query mode to multi-query parallelism. This architectural shift bolsters reasoning robustness while significantly reducing inference latency. Extensive experiments on seven question-answering benchmarks confirm the superiority of our method, which outperforms the strongest baseline by up to 13.7% and decreases inference time by 11.1%.

Method: Proposes CognitiveAttack framework using supervised fine-tuning and reinforcement learning to generate prompts embedding optimized combinations of cognitive biases, systematically leveraging both individual and combined biases.

Result: Achieved 60.1% attack success rate vs 31.6% for SOTA black-box method PAP, exposing significant vulnerabilities across 30 diverse LLMs, especially in open-source models.

Conclusion: Multi-bias interactions represent a powerful attack vector, highlighting critical limitations in current defense mechanisms and paving the way for more robust, human-aligned AI systems through interdisciplinary cognitive science-LLM safety integration.

Abstract: Large Language Models (LLMs) demonstrate impressive capabilities across a wide range of tasks, yet their safety mechanisms remain susceptible to adversarial attacks that exploit cognitive biases – systematic deviations from rational judgment. Unlike prior jailbreaking approaches focused on prompt engineering or algorithmic manipulation, this work highlights the overlooked power of multi-bias interactions in undermining LLM safeguards. We propose CognitiveAttack, a novel red-teaming framework that systematically leverages both individual and combined cognitive biases. By integrating supervised fine-tuning and reinforcement learning, CognitiveAttack generates prompts that embed optimized bias combinations, effectively bypassing safety protocols while maintaining high attack success rates. Experimental results reveal significant vulnerabilities across 30 diverse LLMs, particularly in open-source models. CognitiveAttack achieves a substantially higher attack success rate compared to the SOTA black-box method PAP (60.1% vs. 31.6%), exposing critical limitations in current defense mechanisms. These findings highlight multi-bias interactions as a powerful yet underexplored attack vector. This work introduces a novel interdisciplinary perspective by bridging cognitive science and LLM safety, paving the way for more robust and human-aligned AI systems.

Method: Amplified language-specific neurons across 18 languages using three models trained in different languages, evaluated with Language Steering Shift (LSS) score and tested on downstream tasks including commonsense reasoning, knowledge, and translation.

Result: Optimal amplification factors effectively steer outputs toward target languages, improving self-language performance in some cases but generally degrading cross-language results on downstream tasks.

Conclusion: Language-specific neuron amplification benefits low-resource languages but provides limited advantage for cross-lingual transfer, highlighting their role in multilingual behavior.

Abstract: Language-specific neurons in LLMs that strongly correlate with individual languages have been shown to influence model behavior by deactivating them. However, their role in amplification remains underexplored. This work investigates the effect of amplifying language-specific neurons through interventions across 18 languages, including low-resource ones, using three models primarily trained in different languages. We compare amplification factors by their effectiveness in steering to the target language using a proposed Language Steering Shift (LSS) evaluation score, then evaluate it on downstream tasks: commonsense reasoning (XCOPA, XWinograd), knowledge (Include), and translation (FLORES). The optimal amplification factors effectively steer output toward nearly all tested languages. Intervention using this factor on downstream tasks improves self-language performance in some cases but generally degrades cross-language results. These findings highlight the effect of language-specific neurons in multilingual behavior, where amplification can be beneficial especially for low-resource languages, but provides limited advantage for cross-lingual transfer.

Method: Proposed NyayaRAG framework that provides models with factual case descriptions, relevant legal statutes, and semantically retrieved prior cases using a domain-specific pipeline for the Indian legal system.

Result: Augmenting factual inputs with structured legal knowledge significantly improves both predictive accuracy and explanation quality across various input configurations, as measured by standard metrics and LLM-based evaluators.

Conclusion: Incorporating statutory provisions and judicial precedents through RAG framework is essential for realistic legal judgment prediction in common law systems like India, leading to more accurate and explainable outcomes.

Abstract: Legal Judgment Prediction (LJP) has emerged as a key area in AI for law, aiming to automate judicial outcome forecasting and enhance interpretability in legal reasoning. While previous approaches in the Indian context have relied on internal case content such as facts, issues, and reasoning, they often overlook a core element of common law systems, which is reliance on statutory provisions and judicial precedents. In this work, we propose NyayaRAG, a Retrieval-Augmented Generation (RAG) framework that simulates realistic courtroom scenarios by providing models with factual case descriptions, relevant legal statutes, and semantically retrieved prior cases. NyayaRAG evaluates the effectiveness of these combined inputs in predicting court decisions and generating legal explanations using a domain-specific pipeline tailored to the Indian legal system. We assess performance across various input configurations using both standard lexical and semantic metrics as well as LLM-based evaluators such as G-Eval. Our results show that augmenting factual inputs with structured legal knowledge significantly improves both predictive accuracy and explanation quality.

Method: Proposed Political Neuron Localization through Activation Contrasting (PNLAC) to identify general and topic-specific political neurons. Developed InhibitFT, an inhibition-based fine-tuning method that selectively inhibits neurons to reduce cross-topic generalization.

Result: Identified two types of political neurons across four models and datasets. InhibitFT reduces cross-topic stance generalization by 20% on average while preserving topic-specific performance. Only 5% of neurons need inhibition for effective mitigation.

Conclusion: The study reveals internal mechanisms of political stance generalization in LLMs and provides an effective mitigation method. InhibitFT successfully reduces unintended cross-topic generalization while maintaining model performance on target topics.

Abstract: Fine-tuning Large Language Models on a political topic will significantly manipulate their political stance on various issues and unintentionally affect their stance on unrelated topics. While previous studies have proposed this issue, there is still a lack of understanding regarding the internal representations of these stances and the mechanisms that lead to unintended cross-topic generalization. In this paper, we systematically explore the internal mechanisms underlying this phenomenon from a neuron-level perspective and how to mitigate the cross-topic generalization of political fine-tuning. Firstly, we propose Political Neuron Localization through Activation Contrasting (PNLAC) to identify two distinct types of political neurons: general political neurons, which govern stance across multiple political topics, and topic-specific neurons} that affect the model’s political stance on individual topics. We find the existence of these political neuron types across four models and datasets through activation patching experiments. Leveraging these insights, we introduce InhibitFT, an inhibition-based fine-tuning method, effectively mitigating the cross-topic stance generalization. Experimental results demonstrate the robustness of identified neuron types across various models and datasets, and show that InhibitFT significantly reduces the cross-topic stance generalization by 20% on average, while preserving topic-specific performance. Moreover, we demonstrate that selectively inhibiting only 5% of neurons is sufficient to effectively mitigate the cross-topic stance generalization.

Method: Compared various LLM-based methods with three professors on True/False questions in Neural Networks and Machine Learning. Used direct prompting with Gemini 2.5 and supervised learning with LLM uncertainties.

Result: Professors had limited ability to distinguish easy vs difficult questions. Gemini 2.5 outperformed professors, and supervised learning with LLM uncertainties achieved even better results using minimal training data.

Conclusion: Supervised learning using LLM uncertainty can help professors better estimate exam question difficulty, improving assessment quality.

Abstract: Estimating the difficulty of exam questions is essential for developing good exams, but professors are not always good at this task. We compare various Large Language Model-based methods with three professors in their ability to estimate what percentage of students will give correct answers on True/False exam questions in the areas of Neural Networks and Machine Learning. Our results show that the professors have limited ability to distinguish between easy and difficult questions and that they are outperformed by directly asking Gemini 2.5 to solve this task. Yet, we obtained even better results using uncertainties of the LLMs solving the questions in a supervised learning setting, using only 42 training samples. We conclude that supervised learning using LLM uncertainty can help professors better estimate the difficulty of exam questions, improving the quality of assessment.

Method: Certainty-Guided Reflection Suppression (CGRS) dynamically suppresses reflection trigger generation when the model shows high confidence in its current response, preventing redundant reflection cycles without model retraining or architectural changes.

Result: CGRS reduces token usage by 18.5% to 41.9% across four reasoning benchmarks while preserving accuracy, achieving optimal balance between length reduction and performance across various model architectures and scales (4B to 32B parameters).

Conclusion: CGRS is an effective, model-agnostic method that mitigates overthinking in LRLMs while maintaining reasoning quality, offering practical value for efficient reasoning without requiring retraining or architectural modifications.

Abstract: Recent Large Reasoning Language Models (LRLMs) employ long chain-of-thought reasoning with complex reflection behaviors, typically signaled by specific trigger words (e.g., “Wait” and “Alternatively”) to enhance performance. However, these reflection behaviors can lead to the overthinking problem where the generation of redundant reasoning steps that unnecessarily increase token usage, raise inference costs, and reduce practical utility. In this paper, we propose Certainty-Guided Reflection Suppression (CGRS), a novel method that mitigates overthinking in LRLMs while maintaining reasoning accuracy. CGRS operates by dynamically suppressing the model’s generation of reflection triggers when it exhibits high confidence in its current response, thereby preventing redundant reflection cycles without compromising output quality. Our approach is model-agnostic, requires no retraining or architectural modifications, and can be integrated seamlessly with existing autoregressive generation pipelines. Extensive experiments across four reasoning benchmarks (i.e., AIME24, AMC23, MATH500, and GPQA-D) demonstrate CGRS’s effectiveness: it reduces token usage by an average of 18.5% to 41.9% while preserving accuracy. It also achieves the optimal balance between length reduction and performance compared to state-of-the-art baselines. These results hold consistently across model architectures (e.g., DeepSeek-R1-Distill series, QwQ-32B, and Qwen3 family) and scales (4B to 32B parameters), highlighting CGRS’s practical value for efficient reasoning.

Method: Decomposes queries into subproblems, constructs dependency DAGs, linearizes via topological sort, and applies graph/context pruning to reduce redundant retrieval and token costs.

Result: Extensive experiments show LogicRAG achieves superior performance and efficiency compared to state-of-the-art baselines.

Conclusion: LogicRAG provides an effective alternative to pre-built graph methods by dynamically extracting reasoning structures, enabling adaptive retrieval with reduced costs and improved accuracy.

Abstract: Large language models (LLMs) often suffer from hallucination, generating factually incorrect statements when handling questions beyond their knowledge and perception. Retrieval-augmented generation (RAG) addresses this by retrieving query-relevant contexts from knowledge bases to support LLM reasoning. Recent advances leverage pre-constructed graphs to capture the relational connections among distributed documents, showing remarkable performance in complex tasks. However, existing Graph-based RAG (GraphRAG) methods rely on a costly process to transform the corpus into a graph, introducing overwhelming token cost and update latency. Moreover, real-world queries vary in type and complexity, requiring different logic structures for accurate reasoning. The pre-built graph may not align with these required structures, resulting in ineffective knowledge retrieval. To this end, we propose a $\textbf{Logic}$-aware $\textbf{R}etrieval$-$\textbf{A}$ugmented $\textbf{G}$eneration framework ($\textbf{LogicRAG}$) that dynamically extracts reasoning structures at inference time to guide adaptive retrieval without any pre-built graph. LogicRAG begins by decomposing the input query into a set of subproblems and constructing a directed acyclic graph (DAG) to model the logical dependencies among them. To support coherent multi-step reasoning, LogicRAG then linearizes the graph using topological sort, so that subproblems can be addressed in a logically consistent order. Besides, LogicRAG applies graph pruning to reduce redundant retrieval and uses context pruning to filter irrelevant context, significantly reducing the overall token cost. Extensive experiments demonstrate that LogicRAG achieves both superior performance and efficiency compared to state-of-the-art baselines.

Method: Proposed SceneJailEval with scenario-adaptive multi-dimensional framework, a novel 14-scenario dataset with rich jailbreak variants and regional cases, and robust extensibility for customized or emerging scenarios.

Result: Achieved F1 score of 0.917 on full-scenario dataset (+6% over SOTA) and 0.995 on JBB benchmark (+3% over SOTA), breaking through accuracy bottleneck in heterogeneous scenarios.

Conclusion: SceneJailEval successfully addresses the critical ‘one-size-fits-all’ limitation of existing methods and establishes superiority in jailbreak evaluation across diverse scenarios.

Abstract: Accurate jailbreak evaluation is critical for LLM red team testing and jailbreak research. Mainstream methods rely on binary classification (string matching, toxic text classifiers, and LLM-based methods), outputting only “yes/no” labels without quantifying harm severity. Emerged multi-dimensional frameworks (e.g., Security Violation, Relative Truthfulness and Informativeness) use unified evaluation standards across scenarios, leading to scenario-specific mismatches (e.g., “Relative Truthfulness” is irrelevant to “hate speech”), undermining evaluation accuracy. To address these, we propose SceneJailEval, with key contributions: (1) A pioneering scenario-adaptive multi-dimensional framework for jailbreak evaluation, overcoming the critical “one-size-fits-all” limitation of existing multi-dimensional methods, and boasting robust extensibility to seamlessly adapt to customized or emerging scenarios. (2) A novel 14-scenario dataset featuring rich jailbreak variants and regional cases, addressing the long-standing gap in high-quality, comprehensive benchmarks for scenario-adaptive evaluation. (3) SceneJailEval delivers state-of-the-art performance with an F1 score of 0.917 on our full-scenario dataset (+6% over SOTA) and 0.995 on JBB (+3% over SOTA), breaking through the accuracy bottleneck of existing evaluation methods in heterogeneous scenarios and solidifying its superiority.

Method: Transform textual reviews into knowledge graphs using (subject, predicate, object) triples with sentiment scores, then use graph embeddings (Node2Vec) and sentiment features with machine learning classifiers for rating prediction.

Result: Performs similar to state-of-the-art models but with lower computational cost, achieves comparable performance to LLMs, outperforms traditional NLP baselines on agreement-based metrics like Cohen’s Kappa.

Conclusion: Graph-based representations offer advantages in interpretability, visual exploration, and RAG integration potential, laying groundwork for future research with advanced graph neural networks and fine-tuned LLM extraction methods.

Abstract: In the hospitality industry, understanding the factors that drive customer review ratings is critical for improving guest satisfaction and business performance. This work proposes ReviewGraph for Review Rating Prediction (RRP), a novel framework that transforms textual customer reviews into knowledge graphs by extracting (subject, predicate, object) triples and associating sentiment scores. Using graph embeddings (Node2Vec) and sentiment features, the framework predicts review rating scores through machine learning classifiers. We compare ReviewGraph performance with traditional NLP baselines (such as Bag of Words, TF-IDF, and Word2Vec) and large language models (LLMs), evaluating them in the HotelRec dataset. In comparison to the state of the art literature, our proposed model performs similar to their best performing model but with lower computational cost (without ensemble). While ReviewGraph achieves comparable predictive performance to LLMs and outperforms baselines on agreement-based metrics such as Cohen’s Kappa, it offers additional advantages in interpretability, visual exploration, and potential integration into Retrieval-Augmented Generation (RAG) systems. This work highlights the potential of graph-based representations for enhancing review analytics and lays the groundwork for future research integrating advanced graph neural networks and fine-tuned LLM-based extraction methods. We will share ReviewGraph output and platform open-sourced on our GitHub page https://github.com/aaronlifenghan/ReviewGraph

Method: Proposed Prefix INjection Guard (PING) - an iterative approach that alternates between generating candidate natural language prefixes and selecting those that optimize both task performance and refusal behavior for harmful requests.

Result: PING significantly enhances safety of fine-tuned LLM agents without sacrificing effectiveness, outperforming existing prompting approaches across web navigation and code generation benchmarks. Analysis shows prefix tokens are crucial for behavior modification.

Conclusion: PING provides an effective method to maintain safety alignment in agentic LLMs during fine-tuning, using natural language prefixes to guide refusal behavior while preserving task performance.

Abstract: Beyond simple text generation, Large Language Models (LLMs) have evolved into agentic systems capable of planning and interacting with external tools to solve complex tasks. This evolution involves fine-tuning LLMs on agent-specific tasks to enhance their proficiency. However, safety concerns are frequently overlooked during this fine-tuning process. In this work, we show that aligned LLMs can become unintentionally misaligned, leading to a higher likelihood of executing harmful tasks and a reduced tendency to refuse them when fine-tuned to execute agentic tasks. To address these safety challenges, we propose Prefix INjection Guard (PING), a simple yet effective method that prepends automatically generated natural language prefixes to agent responses, guiding them to refuse harmful requests while preserving performance on benign tasks. Specifically, we introduce an iterative approach that alternates between (1) generating candidate prefixes and (2) selecting those that optimize both task performance and refusal behavior. Experimental results demonstrate that PING significantly enhances the safety of fine-tuned LLM agents without sacrificing their effectiveness. PING consistently outperforms existing prompting approaches across diverse benchmarks in both web navigation and code generation tasks. Our analysis of internal hidden states via linear probes reveals that prefix tokens are crucial for behavior modification, explaining the performance gains. WARNING: This paper contains contents that are unethical or offensive in nature.

Method: Proposes ESFP-RM: a two-stage reward model using explanation-based slot framework with masked language models, leveraging contextual explanations for better prediction.

Result: ESFP-RM delivers more stable and generalizable reward signals compared to generative reward models in both RLHF and out-of-distribution scenarios.

Conclusion: Reframing reward modeling as NLI and using MLMs with contextual explanations provides a promising path for building superior, more generalizable reward models.

Abstract: The emergence of LM-based judging reward modeling, represented by generative reward models, has successfully made reinforcement learning from AI feedback (RLAIF) efficient and scalable. To further advance this paradigm, we propose a core insight: this form of reward modeling shares fundamental formal consistency with natural language inference (NLI), a core task in natural language understanding. This reframed perspective points to a key path for building superior reward models: scaling the model’s comprehension boundaries. Pursuing this path, exploratory experiments on NLI tasks demonstrate that the slot prediction masked language models (MLMs) incorporating contextual explanations achieve significantly better performance compared to mainstream autoregressive models. Based on this key finding, we propose ESFP-RM, a two-stage LM-based judging reward model that utilizes an explanation based slot framework for prediction to fully leverage the advantages of MLMs. Extensive experiments demonstrate that in both reinforcement learning from human feedback (RLHF) and out-of-distribution (OOD) scenarios, the ESFP-RM framework delivers more stable and generalizable reward signals compared to generative reward models.

Method: Ranked Preference Reinforcement Optimization (RPRO) combines reinforcement learning with preference-driven reasoning refinement, using task-adaptive reasoning templates, probabilistic evaluation, groupwise ranking optimization based on Bradley-Terry model, and KL-divergence regularization.

Result: Experiments on PubMedQA, MedQA-USMLE, and FEMH clinical dataset show consistent improvements over baselines, with 2B-parameter model outperforming larger 7B-20B models including medical-specialized variants.

Conclusion: Combining preference optimization with quality-driven refinement provides a scalable and clinically grounded approach to building more reliable medical LLMs.

Abstract: Medical question answering requires advanced reasoning that integrates domain knowledge with logical inference. However, existing large language models (LLMs) often generate reasoning chains that lack factual accuracy and clinical reliability. We propose Ranked Preference Reinforcement Optimization (RPRO), a novel framework that combines reinforcement learning with preference-driven reasoning refinement to enhance clinical chain-of-thought (CoT) performance. RPRO distinguishes itself from prior approaches by employing task-adaptive reasoning templates and a probabilistic evaluation mechanism that aligns model outputs with established clinical workflows, while automatically identifying and correcting low-quality reasoning chains. Unlike traditional pairwise preference methods, RPRO introduces a groupwise ranking optimization based on the Bradley–Terry model and incorporates KL-divergence regularization for stable training. Experiments on PubMedQA, MedQA-USMLE, and a real-world clinical dataset from Far Eastern Memorial Hospital (FEMH) demonstrate consistent improvements over strong baselines. Remarkably, our 2B-parameter model outperforms much larger 7B–20B models, including medical-specialized variants. These findings demonstrate that combining preference optimization with quality-driven refinement provides a scalable and clinically grounded approach to building more reliable medical LLMs.

Method: Proposed a self-training approach that leverages unlabeled data to elicit reward reasoning, developing GRAM-R² - a generative reward model trained to produce preference labels and accompanying reward rationales.

Result: GRAM-R² consistently outperforms strong discriminative and generative baselines in experiments on response ranking, task adaptation, and reinforcement learning from human feedback.

Conclusion: GRAM-R² serves as an effective foundation model for reward reasoning that can be applied to various tasks with minimal fine-tuning, supporting downstream applications like response ranking and task-specific reward tuning.

Abstract: Significant progress in reward modeling over recent years has been driven by a paradigm shift from task-specific designs towards generalist reward models. Despite this trend, developing effective reward models remains a fundamental challenge: the heavy reliance on large-scale labeled preference data. Pre-training on abundant unlabeled data offers a promising direction, but existing approaches fall short of instilling explicit reasoning into reward models. To bridge this gap, we propose a self-training approach that leverages unlabeled data to elicit reward reasoning in reward models. Based on this approach, we develop GRAM-R$^2$, a generative reward model trained to produce not only preference labels but also accompanying reward rationales. GRAM-R$^2$ can serve as a foundation model for reward reasoning and can be applied to a wide range of tasks with minimal or no additional fine-tuning. It can support downstream applications such as response ranking and task-specific reward tuning. Experiments on response ranking, task adaptation, and reinforcement learning from human feedback demonstrate that GRAM-R$^2$ consistently delivers strong performance, outperforming several strong discriminative and generative baselines.

Method: Used a hybrid AI-human framework with iterative expert feedback to refine AI-driven, multi-criteria filtering for constructing high-quality benchmark data spanning 13 specialties, 8 error types, 4 writing styles, and 5 difficulty levels.

Result: Evaluation of 20 leading LLMs shows they can often determine if text contains errors but struggle with precise error localization, with top performers falling short of human performance. Models exhibit ‘over-criticism’ - misidentifying correct information as erroneous.

Conclusion: MedFact highlights significant challenges in deploying medical LLMs and provides essential resources to develop factually reliable medical AI systems, revealing that advanced reasoning techniques can exacerbate fact-checking errors.

Abstract: Deploying Large Language Models (LLMs) in medical applications requires fact-checking capabilities to ensure patient safety and regulatory compliance. We introduce MedFact, a challenging Chinese medical fact-checking benchmark with 2,116 expert-annotated instances from diverse real-world texts, spanning 13 specialties, 8 error types, 4 writing styles, and 5 difficulty levels. Construction uses a hybrid AI-human framework where iterative expert feedback refines AI-driven, multi-criteria filtering to ensure high quality and difficulty. We evaluate 20 leading LLMs on veracity classification and error localization, and results show models often determine if text contains errors but struggle to localize them precisely, with top performers falling short of human performance. Our analysis reveals the “over-criticism” phenomenon, a tendency for models to misidentify correct information as erroneous, which can be exacerbated by advanced reasoning techniques such as multi-agent collaboration and inference-time scaling. MedFact highlights the challenges of deploying medical LLMs and provides resources to develop factually reliable medical AI systems.

Method: Three-stage framework: (1) CRAS metric to diagnose role adherence across four dimensions, (2) attention drift analysis to localize conflict resolution in middle layers, (3) SAIL surgical alignment using LoRA on focal layers with token-weighted DPO optimization.

Result: Improves instruction hierarchy compliance by +5.60% on AutoGen with MedQA benchmark without full-model finetuning.

Conclusion: Surgical alignment on localized layers effectively addresses hierarchical compliance failures in multi-agent systems while being more efficient than full-model finetuning.

Abstract: Large Language Model (LLM)-powered multi-agent systems (MAS) have rapidly advanced collaborative reasoning, tool use, and role-specialized coordination in complex tasks. However, reliability-critical deployment remains hindered by a systemic failure mode: hierarchical compliance under instruction conflicts (system-user, peer-peer), where agents misprioritize system-level rules in the presence of competing demands. Moreover, widely used macro-level metrics (e.g., pass@k) obscure these micro-level violations and offer little actionable guidance for remedy. In this work, we present a full-stack, three-stage framework: (1) Diagnose - Contextualized Role Adherence Score (CRAS), a query-wise, context-aware scoring metric that decomposes role adherence into four measurable dimensions; (2) Localize - attention drift analysis revealing that instruction conflicts are resolved by attention heads that are largely concentrated in middle layers; (3) Align - Surgical Alignment of Instruction Layers (SAIL), which installs LoRA only on the localized focal layers and optimizes a token-weighted DPO-style preference objective that credits tokens by their focal attentional contribution. Across standard benchmarks and MAS frameworks, our surgical approach improves instruction hierarchy compliance (e.g., +5.60% with AutoGen on MedQA) without full-model finetuning.

Method: TARE alternates between inner adversarial search with hard paraphrases and outer robust selection. ATARE extends this with anisotropic weights to shape semantic neighborhoods and adaptive radius balancing exploration and fidelity.

Result: The methods outperform accuracy-only prompt search by creating prompts that preserve accuracy under paraphrasing while remaining computationally practical across diverse tasks.

Conclusion: Addressing textual sharpness through semantic neighborhood robustness leads to more stable and reliable prompt optimization that maintains performance under semantic variations.

Abstract: The performance of Large Language Models (LLMs) hinges on carefully engineered prompts. However, prevailing prompt optimization methods, ranging from heuristic edits and reinforcement learning to evolutionary search, primarily target point-wise accuracy. They seldom enforce paraphrase invariance or searching stability, and therefore cannot remedy this brittleness in practice. Automated prompt search remains brittle: small, semantically preserving paraphrases often cause large performance swings. We identify this brittleness as the textual sharpness of the prompt landscape. In this work, we provide the first formal treatment of textual sharpness in the discrete, semantic space of prompts, together with an operational robustness criterion over a semantic neighborhood; the design is black-box or API-only, requiring no gradients to update the model’s parameters. Then we introduce TARE (Textual Sharpness-Aware Evolving), a derivative-free framework that alternates between an inner, sampling-based adversarial search that stresses a prompt with hard paraphrases and an outer, robust selection that prefers candidates whose neighborhoods remain strong. We further propose ATARE, which learns anisotropic weights to shape the semantic neighborhood and adapts its radius over time to balance exploration and fidelity. Diverse tasks evaluate our methods, whose design for minimizing textual sharpness gap leads to prompts that preserve accuracy under paraphrasing, outperforming accuracy-only prompt search while remaining computationally practical.

Method: PET reframes preference prediction as inferring dynamic probability distributions over stable preference clusters using logit-probing and generative classification techniques, enabling transparent preference learning.

Result: PET improves ranking quality by up to 40% in NDCG on public benchmarks and outperforms state-of-the-art production models by 7 times in NDCG for long-tail content on real-world short-video platform data.

Conclusion: PET transforms user profiling from direct preference list generation to transparent distributional preference mapping, enabling more explainable, fair, and diverse personalization systems.

Abstract: Understanding how user preference evolves over time is a fundamental challenge central to modern digital ecosystems, for which Large Language Models (LLMs) are an increasingly prominent and popular approach due to their ability to comprehend the rich semantic context within behavioral data. A common practice is to use LLMs to predict a user’s next action by directly generating a ranked list of preferred items. Although effective for short-term prediction, the end-to-end generation paradigm inherently limits personalization. Its opaque decision-making process obscures holistic user profiling and exacerbates popularity bias. To address these limitations, we propose Preference Evolution Tracking (PET), a framework that reframes the task as inferring a dynamic probability distribution over a stable and interpretable lattice of preference clusters. By applying logit-probing and generative classification techniques, PET infers a user’s preference as a probability distribution, enabling transparent preference learning. On public benchmarks (Yelp, MovieLens), PET improves ranking quality by up to 40% in NDCG over direct generation baselines. On a large-scale, real-world dataset from a short-video platform, it excels at ranking long-tail contents, significantly outperforming a SOTA production model by 7 times in the NDCG score. Ultimately, PET transforms the user profile model from direct preference list generation to a transparent distributional preference mapping, paving the way for more explainable, fair, and diverse personalization systems.

Method: Proposed noise synthesis framework and new metrics to evaluate REAL-MT robustness systematically. Instantiated REAL-MT with Qwen-series models (standard LLMs and large reasoning models) and evaluated on idiomatic translation across language pairs under synthesized noise.

Result: Low-resource language pairs degrade more severely under noise than high-resource ones, often producing nonsensical translations. LRMs show no improvement in error correction and are more susceptible to noise, rationalizing incorrect contexts. Attention shifts away from source idiom to noisy content while confidence increases despite declining accuracy.

Conclusion: Current approaches have limitations, revealing a fundamental trade-off between robustness and clean-context performance. Highlights need for self-verifying integration mechanisms to improve REAL-MT reliability.

Abstract: \textbf{RE}trieval-\textbf{A}ugmented \textbf{L}LM-based \textbf{M}achine \textbf{T}ranslation (REAL-MT) shows promise for knowledge-intensive tasks like idiomatic translation, but its reliability under noisy retrieval contexts remains poorly understood despite this being a common challenge in real-world deployment. To address this gap, we propose a noise synthesis framework and new metrics to evaluate the robustness of REAL-MT systematically. Using this framework, we instantiate REAL-MT with Qwen-series models, including standard LLMs and large reasoning models (LRMs) with enhanced reasoning, and evaluate their performance on idiomatic translation across high-, medium-, and low-resource language pairs under synthesized noise. Our results show that low-resource language pairs, which rely more heavily on retrieved context, degrade more severely under noise than high-resource ones and often produce nonsensical translations. Although LRMs possess enhanced reasoning capabilities, they show no improvement in error correction and are even more susceptible to noise, tending to rationalize incorrect contexts. We find that this stems from an attention shift away from the source idiom to noisy content, while confidence increases despite declining accuracy, indicating poor calibration. To mitigate these issues, we investigate training-free and fine-tuning strategies, which improve robustness at the cost of performance in clean contexts, revealing a fundamental trade-off. Our findings highlight the limitations of current approaches, underscoring the need for self-verifying integration mechanisms.

Method: Created a dataset of 200 politically significant Bangla news articles labeled for three stances, and conducted comprehensive evaluation of 28 proprietary and open-source LLMs using diagnostic analyses.

Result: LLMs showed strong performance in detecting government-critique content (F1 up to 0.83) but struggled significantly with neutral articles (F1 as low as 0.00), with tendency to over-predict government-leaning stances.

Conclusion: The dataset provides foundation for advancing Bangla stance detection research and offers insights for improving LLM performance in low-resource languages, highlighting challenges in detecting neutral content.

Abstract: Detecting media bias is crucial, specifically in the South Asian region. Despite this, annotated datasets and computational studies for Bangla political bias research remain scarce. Crucially because, political stance detection in Bangla news requires understanding of linguistic cues, cultural context, subtle biases, rhetorical strategies, code-switching, implicit sentiment, and socio-political background. To address this, we introduce the first benchmark dataset of 200 politically significant and highly debated Bangla news articles, labeled for government-leaning, government-critique, and neutral stances, alongside diagnostic analyses for evaluating large language models (LLMs). Our comprehensive evaluation of 28 proprietary and open-source LLMs shows strong performance in detecting government-critique content (F1 up to 0.83) but substantial difficulty with neutral articles (F1 as low as 0.00). Models also tend to over-predict government-leaning stances, often misinterpreting ambiguous narratives. This dataset and its associated diagnostics provide a foundation for advancing stance detection in Bangla media research and offer insights for improving LLM performance in low-resource languages.

Method: Analyzed 710 projects with paired snapshots, parsing governance text into actors, rules, actions, and objects, then measuring change using entropy (evenness), richness (diversity), and Jensen Shannon divergence (drift).

Result: Projects define more roles and actions over time with more even distribution, while rule composition remains stable. Governance grows by expanding and balancing participation categories without major shifts in prescriptive force.

Conclusion: The study provides a reproducible baseline for evaluating whether future AI-mediated workflows concentrate or redistribute authority in open source governance.

Abstract: We study how open source communities describe participation and control through version controlled governance documents. Using a corpus of 710 projects with paired snapshots, we parse text into actors, rules, actions, and objects, then group them and measure change with entropy for evenness, richness for diversity, and Jensen Shannon divergence for drift. Projects define more roles and more actions over time, and these are distributed more evenly, while the composition of rules remains stable. These findings indicate that governance grows by expanding and balancing categories of participation without major shifts in prescriptive force. The analysis provides a reproducible baseline for evaluating whether future AI mediated workflows concentrate or redistribute authority.

Method: Web scraping and structured parsing of Macedonian recipes, with normalization of heterogeneous ingredient descriptions (units, quantities, descriptors), followed by exploratory analysis using Pointwise Mutual Information and Lift score to identify ingredient co-occurrence patterns.

Result: Created a new Macedonian recipe dataset and identified distinctive ingredient combinations that characterize Macedonian cuisine through frequency and co-occurrence analysis.

Conclusion: The dataset provides a valuable resource for studying food culture in underrepresented languages and reveals unique patterns of Macedonian culinary tradition.

Abstract: Computational gastronomy increasingly relies on diverse, high-quality recipe datasets to capture regional culinary traditions. Although there are large-scale collections for major languages, Macedonian recipes remain under-represented in digital research. In this work, we present the first systematic effort to construct a Macedonian recipe dataset through web scraping and structured parsing. We address challenges in processing heterogeneous ingredient descriptions, including unit, quantity, and descriptor normalization. An exploratory analysis of ingredient frequency and co-occurrence patterns, using measures such as Pointwise Mutual Information and Lift score, highlights distinctive ingredient combinations that characterize Macedonian cuisine. The resulting dataset contributes a new resource for studying food culture in underrepresented languages and offers insights into the unique patterns of Macedonian culinary tradition.

Method: Established a benchmark with 150 scenarios across Economy, Healthcare, Education, Social Interaction, and Entertainment domains, with over 1,000 samples. Evaluated intrinsic patterns (egoistic vs sycophantic behaviors) and extrinsic factors (neutral conditions, reward-based incentivization, coercive pressures) with multi-turn interaction loops.

Result: Extensive experiments reveal critical vulnerabilities in LLMs and LRMs, particularly amplified deception under reinforcement dynamics, showing models lack robust resistance to manipulative contextual cues.

Conclusion: Current models urgently need advanced safeguards against various deception behaviors, as demonstrated by their vulnerability to manipulative contextual cues and reinforcement dynamics.

Abstract: Despite the remarkable advances of Large Language Models (LLMs) across diverse cognitive tasks, the rapid enhancement of these capabilities also introduces emergent deceptive behaviors that may induce severe risks in high-stakes deployments. More critically, the characterization of deception across realistic real-world scenarios remains underexplored. To bridge this gap, we establish DeceptionBench, the first benchmark that systematically evaluates how deceptive tendencies manifest across different societal domains, what their intrinsic behavioral patterns are, and how extrinsic factors affect them. Specifically, on the static count, the benchmark encompasses 150 meticulously designed scenarios in five domains, i.e., Economy, Healthcare, Education, Social Interaction, and Entertainment, with over 1,000 samples, providing sufficient empirical foundations for deception analysis. On the intrinsic dimension, we explore whether models exhibit self-interested egoistic tendencies or sycophantic behaviors that prioritize user appeasement. On the extrinsic dimension, we investigate how contextual factors modulate deceptive outputs under neutral conditions, reward-based incentivization, and coercive pressures. Moreover, we incorporate sustained multi-turn interaction loops to construct a more realistic simulation of real-world feedback dynamics. Extensive experiments across LLMs and Large Reasoning Models (LRMs) reveal critical vulnerabilities, particularly amplified deception under reinforcement dynamics, demonstrating that current models lack robust resistance to manipulative contextual cues and the urgent need for advanced safeguards against various deception behaviors. Code and resources are publicly available at https://github.com/Aries-iai/DeceptionBench.

Method: Integrates synthetic dialogue generation with dynamically constructed rubrics that serve as adaptive guides for incremental RL, using rubric-driven feedback and general-purpose LLMs as judges without task-specific fine-tuning.

Result: Applied to Qwen3-4B-Instruct model, ORBIT raises HealthBench-Hard score from 7.0 to 27.5 using only 2k training samples, achieving SOTA performance for models at this scale and competing with strongest open-source baselines.

Conclusion: Rubric-guided RL consistently improves consultation quality across diverse medical scenarios and generalizes to other domains like instruction-following, highlighting the generality of rubric-based feedback.

Abstract: Reinforcement learning has powered many of the recent breakthroughs in large language models, especially for tasks where rewards can be computed automatically, such as code generation. However, these methods deteriorate in open-ended domains like medical consultation, where feedback is inherently ambiguous, highly context-dependent, and cannot be reduced to a reliable scalar signal. In such settings, RL must either rely on supervision-intensive reward models that often fail to generalize, or it falls into pathological behaviors such as reward hacking - an especially troubling risk for high-stakes medical dialogue. To address these limitations, we introduce ORBIT, an open-ended rubric-based incremental training framework for high-stakes medical dialogue. ORBIT integrates synthetic dialogue generation with dynamically constructed rubrics that serve as adaptive guides for incremental RL. Instead of relying on external medical knowledge bases or handcrafted rule sets, ORBIT uses rubric-driven feedback to steer the learning process. Its judge component can be instantiated with general-purpose instruction-following LLMs, removing the need for any task-specific fine-tuning. Applied to the Qwen3-4B-Instruct model, ORBIT raises the HealthBench-Hard score from 7.0 to 27.5 using only 2k training samples, achieving SOTA performance for models at this scale. With larger rubric datasets, ORBIT-trained models further compete with the strongest open-source baselines on HealthBench-Hard. Our analysis shows that rubric-guided RL consistently improves consultation quality across diverse medical scenarios. We also apply such rubric generation and training pipeline to InfoBench, where ORBIT enhances instruction-following performance, highlighting the generality of rubric-based feedback.

Method: CoCT paradigm where LLMs first produce concept tags (emotions, strategies, topics) then generate detailed content following these concepts, implementing hierarchical thinking.

Result: CoCT outperforms baselines including self-refine, ECoT, SoT and RAG in daily and emotional support conversations across in-domain and out-of-domain concept settings, validated by automatic, human and LLM-based evaluations.

Conclusion: CoCT provides a potential solution for LLM prompting paradigm that can be applied to a wider scope of tasks beyond traditional reasoning domains.

Abstract: Chain-of-Thought (CoT) is widely applied to enhance the LLM capability in math, coding and reasoning tasks. However, its performance is limited for open-domain tasks, when there are no clearly defined reasoning steps or logical transitions. To mitigate such challenges, we propose a new prompt-based paradigm called Chain of Conceptual Thoughts (CoCT), which suggests the LLM first to produce the tag of concepts, then complete the detailed content following the concept. To encourage this hierarchical way of thinking, we implement the concepts with emotions, strategies and topics. We experiment with this paradigm in daily and emotional support conversations, covering tasks with both in-domain and out-of-domain concept settings. Automatic, human, and LLM-based evaluations reveal that CoCT surpasses several prompt-based baselines such as self-refine, ECoT, SoT and RAG, suggesting a potential solution of LLM prompting paradigm for a wider scope of tasks.

Method: Constructed from 3 real-world time-stamped corpora (AWS updates, Azure changes, WHO disease outbreaks), identifies natural knowledge evolution and generates questions with gold answers tailored to different LLM knowledge cut-off dates.

Result: Evaluation of 12 LLMs across 3 knowledge probing formats shows significant performance drops of up to 31% on evolveQA compared to static knowledge questions.

Conclusion: EvolveQA effectively reveals LLMs’ limitations in handling temporally evolving knowledge and provides a fair evaluation framework for different knowledge cut-off dates.

Abstract: LLMs often fail to handle temporal knowledge conflicts–contradictions arising when facts evolve over time within their training data. Existing studies evaluate this phenomenon through benchmarks built on structured knowledge bases like Wikidata, but they focus on widely-covered, easily-memorized popular entities and lack the dynamic structure needed to fairly evaluate LLMs with different knowledge cut-off dates. We introduce evolveQA, a benchmark specifically designed to evaluate LLMs on temporally evolving knowledge, constructed from 3 real-world, time-stamped corpora: AWS updates, Azure changes, and WHO disease outbreak reports. Our framework identifies naturally occurring knowledge evolution and generates questions with gold answers tailored to different LLM knowledge cut-off dates. Through extensive evaluation of 12 open and closed-source LLMs across 3 knowledge probing formats, we demonstrate significant performance drops of up to 31% on evolveQA compared to static knowledge questions.

Method: Uses a propose-and-verify procedure where student proposes tokens that are verified by teacher (greedy Top-k or non-greedy Spec-k variants). Accepted tokens receive full loss while rejected tokens are masked/down-weighted.

Result: Consistently improves strong baselines and achieves state-of-the-art results for small models across instruction following, mathematical reasoning, code generation, and VLM settings without architectural changes.

Conclusion: SelecTKD provides an effective plug-and-play distillation framework that shifts focus from measuring divergence to selective learning, stabilizing optimization and improving performance across diverse tasks.

Abstract: Knowledge distillation (KD) is a standard route to compress Large Language Models (LLMs) into compact students, yet most pipelines uniformly apply token-wise loss regardless of teacher confidence. This indiscriminate supervision amplifies noisy, high-entropy signals and is especially harmful under large teacher-student capacity gaps. We introduce SelecTKD, a plug-and-play Selective Token-Weighted distillation framework that shifts the focus from “how to measure divergence” to “where to apply learning”. At each step, the student proposes tokens that are verified by the teacher through a robust propose-and-verify procedure with two variants: greedy Top-k and non-greedy Spec-k. Accepted tokens receive full loss, while rejected tokens are masked or down-weighted. This objective-agnostic design works with on- and off-policy data, induces an implicit curriculum quantified by Token Acceptance Rate (TAR), and stabilizes optimization. Across instruction following, mathematical reasoning, code generation, and a VLM setting, SelecTKD consistently improves strong baselines and achieves state-of-the-art results for small models without architectural changes or extra reference models.

Method: Systematic review of 180+ papers with novel taxonomy categorizing unlearning methods by intervention phase (parameter modification vs. selection), and multidimensional analysis of evaluation paradigms including 18 benchmarks and 10 knowledge memorization metric categories.

Result: Provides comprehensive framework for understanding LLM unlearning methods and evaluation approaches, enabling deeper insights and comparative analysis across different strategies.

Conclusion: The survey advances LLM unlearning field and secure AI development by establishing systematic categorization and evaluation framework, while identifying current challenges and future directions.

Abstract: Large Language Models (LLMs) demonstrate remarkable capabilities, but their training on massive corpora poses significant risks from memorized sensitive information. To mitigate these issues and align with legal standards, unlearning has emerged as a critical technique to selectively erase specific knowledge from LLMs without compromising their overall performance. This survey provides a systematic review of over 180 papers on LLM unlearning published since 2021. First, it introduces a novel taxonomy that categorizes unlearning methods based on the phase in the LLM pipeline of the intervention. This framework further distinguishes between parameter modification and parameter selection strategies, thus enabling deeper insights and more informed comparative analysis. Second, it offers a multidimensional analysis of evaluation paradigms. For datasets, we compare 18 existing benchmarks from the perspectives of task format, content, and experimental paradigms to offer actionable guidance. For metrics, we move beyond mere enumeration by dividing knowledge memorization metrics into 10 categories to analyze their advantages and applicability, while also reviewing metrics for model utility, robustness, and efficiency. By discussing current challenges and future directions, this survey aims to advance the field of LLM unlearning and the development of secure AI systems.

Method: MPG framework under decoding-time combination paradigm that leverages implicit density ratios in single-dimensional models, implemented via Speculative Chunk-level based Rejection sampling (SCR) which generates responses in chunks and validates them parallelly.

Result: Experiments on MBTI personality and Role-Playing show improvements up to 16%-18% in multi-personality generation effectiveness.

Conclusion: The proposed MPG framework provides an efficient and flexible solution for multi-personality generation in LLMs without requiring retraining or external models, achieving significant performance improvements.

Abstract: Multi-personality generation for LLMs, enabling simultaneous embodiment of multiple personalization attributes, is a fundamental challenge. Existing retraining-based approaches are costly and poorly scalable, while decoding-time methods often rely on external models or heuristics, limiting flexibility and robustness. In this paper, we propose a novel Multi-Personality Generation (MPG) framework under the decoding-time combination paradigm. It flexibly controls multi-personality without relying on scarce multi-dimensional models or extra training, leveraging implicit density ratios in single-dimensional models as a “free lunch” to reformulate the task as sampling from a target strategy aggregating these ratios. To implement MPG efficiently, we design Speculative Chunk-level based Rejection sampling (SCR), which generates responses in chunks and parallelly validates them via estimated thresholds within a sliding window. This significantly reduces computational overhead while maintaining high-quality generation. Experiments on MBTI personality and Role-Playing demonstrate the effectiveness of MPG, showing improvements up to 16%-18%. Code and data are available at https://github.com/Libra117/MPG .

Method: Proposed Silenced Bias Benchmark (SBB) that uses activation steering to reduce model refusals during question-answer evaluations, allowing detection of silenced biases in the latent space without relying on prompt manipulation.

Result: The approach reveals an alarming distinction between models’ direct refusal responses and their underlying fairness issues, exposing biases that were effectively concealed by safety alignment.

Conclusion: SBB provides a scalable fairness evaluation framework that can expand to new demographic groups and subjects, encouraging development of fair models beyond the masking effects of alignment training.

Abstract: Safety-aligned large language models (LLMs) are becoming increasingly widespread, especially in sensitive applications where fairness is essential and biased outputs can cause significant harm. However, evaluating the fairness of models is a complex challenge, and approaches that do so typically utilize standard question-answer (QA) styled schemes. Such methods often overlook deeper issues by interpreting the model’s refusal responses as positive fairness measurements, which creates a false sense of fairness. In this work, we introduce the concept of silenced biases, which are unfair preferences encoded within models’ latent space and are effectively concealed by safety-alignment. Previous approaches that considered similar indirect biases often relied on prompt manipulation or handcrafted implicit queries, which present limited scalability and risk contaminating the evaluation process with additional biases. We propose the Silenced Bias Benchmark (SBB), which aims to uncover these biases by employing activation steering to reduce model refusals during QA. SBB supports easy expansion to new demographic groups and subjects, presenting a fairness evaluation framework that encourages the future development of fair models and tools beyond the masking effects of alignment training. We demonstrate our approach over multiple LLMs, where our findings expose an alarming distinction between models’ direct responses and their underlying fairness issues.

Method: Evaluated several LLMs on a curated dataset of food recipes using zero-shot and few-shot settings, with metrics including Kendall’s Tau, NLCS, and NED to capture different aspects of ordering quality.

Result: Model performance declines with increasing sequence length and greater step displacement (more severe shuffling), highlighting limitations in procedural reasoning.

Conclusion: Current LLMs have significant limitations in procedural reasoning tasks, particularly when dealing with longer sequences and more disordered inputs, indicating areas for future improvement.

Abstract: Reasoning over procedural sequences, where the order of steps directly impacts outcomes, is a critical capability for large language models (LLMs). In this work, we study the task of reconstructing globally ordered sequences from shuffled procedural steps, using a curated dataset of food recipes, a domain where correct sequencing is essential for task success. We evaluate several LLMs under zero-shot and few-shot settings and present a comprehensive evaluation framework that adapts established metrics from ranking and sequence alignment. These include Kendall’s Tau, Normalized Longest Common Subsequence (NLCS), and Normalized Edit Distance (NED), which capture complementary aspects of ordering quality. Our analysis shows that model performance declines with increasing sequence length, reflecting the added complexity of longer procedures. We also find that greater step displacement in the input, corresponding to more severe shuffling, leads to further degradation. These findings highlight the limitations of current LLMs in procedural reasoning, especially with longer and more disordered inputs.

Method: SCD uses Error Prompt Mechanism (EPM) to detect errors and provide customized messages during inference, plus two-stage distillation to transfer capabilities from large to small LLMs.

Result: SCD achieves best performance on 5 benchmarks with 3 structured data types using 8B models, closely approaching GPT4 on some datasets. EPM also helps large LLMs surpass SOTA results.

Conclusion: SCD effectively improves small LLMs’ structured data QA capabilities through self-correction distillation and error detection mechanisms.

Abstract: Structured data question answering (QA), including table QA, Knowledge Graph (KG) QA, and temporal KG QA, is a pivotal research area. Advances in large language models (LLMs) have driven significant progress in unified structural QA frameworks like TrustUQA. However, these frameworks face challenges when applied to small-scale LLMs since small-scale LLMs are prone to errors in generating structured queries. To improve the structured data QA ability of small-scale LLMs, we propose a self-correction distillation (SCD) method. In SCD, an error prompt mechanism (EPM) is designed to detect errors and provide customized error messages during inference, and a two-stage distillation strategy is designed to transfer large-scale LLMs’ query-generation and error-correction capabilities to small-scale LLM. Experiments across 5 benchmarks with 3 structured data types demonstrate that our SCD achieves the best performance and superior generalization on small-scale LLM (8B) compared to other distillation methods, and closely approaches the performance of GPT4 on some datasets. Furthermore, large-scale LLMs equipped with EPM surpass the state-of-the-art results on most datasets.

Method: Created VocalBench-zh, an ability-level divided evaluation suite with 10 well-crafted subsets and over 10K high-quality instances, covering 12 user-oriented characters adapted to Mandarin context.

Result: Evaluation of 14 mainstream models revealed common challenges for current approaches and highlighted the need for new insights into next-generation speech interactive systems.

Conclusion: VocalBench-zh provides a systematic evaluation framework for Mandarin speech-to-speech models, enabling better model comparison and development of improved speech interaction systems.

Abstract: The development of multi-modal large language models (LLMs) leads to intelligent approaches capable of speech interactions. As one of the most widely spoken languages globally, Mandarin is supported by most models to enhance their applicability and reach. However, the scarcity of comprehensive speech-to-speech (S2S) benchmarks in Mandarin contexts impedes systematic evaluation for developers and hinders fair model comparison for users. In this work, we propose VocalBench-zh, an ability-level divided evaluation suite adapted to Mandarin context consisting of 10 well-crafted subsets and over 10K high-quality instances, covering 12 user-oriented characters. The evaluation experiment on 14 mainstream models reveals the common challenges for current routes, and highlights the need for new insights into next-generation speech interactive systems. The evaluation codes and datasets will be available at https://github.com/SJTU-OmniAgent/VocalBench-zh.

Method: Built a super-learner system using meta-learning to integrate five major LLMs, learning each model’s specific capabilities to leverage collective diagnostic power.

Result: Individual LLMs achieved 58-65% accuracy, while the super-learner reached 70% accuracy, with at least one LLM achieving 85% accuracy on specific cases.

Conclusion: Meta-learning ensemble approaches can significantly enhance diagnostic accuracy in emergency medicine by combining the strengths of multiple LLMs, outperforming both individual models and human doctors.

Abstract: Medical decision-support and advising systems are critical for emergency physicians to quickly and accurately assess patients’ conditions and make diagnosis. Artificial Intelligence (AI) has emerged as a transformative force in healthcare in recent years and Large Language Models (LLMs) have been employed in various fields of medical decision-support systems. We studied responses of a group of different LLMs to real cases in emergency medicine. The results of our study on five most renown LLMs showed significant differences in capabilities of Large Language Models for diagnostics acute diseases in medical emergencies with accuracy ranging between 58% and 65%. This accuracy significantly exceeds the reported accuracy of human doctors. We built a super-learner MEDAS (Medical Emergency Diagnostic Advising System) of five major LLMs - Gemini, Llama, Grok, GPT, and Claude). The super-learner produces higher diagnostic accuracy, 70%, even with a quite basic meta-learner. However, at least one of the integrated LLMs in the same super-learner produces 85% correct diagnoses. The super-learner integrates a cluster of LLMs using a meta-learner capable of learning different capabilities of each LLM to leverage diagnostic accuracy of the model by collective capabilities of all LLMs in the cluster. The results of our study showed that aggregated diagnostic accuracy provided by a meta-learning approach exceeds that of any individual LLM, suggesting that the super-learner can take advantage of the combined knowledge of the medical datasets used to train the group of LLMs.

Method: Two-phase framework: generation phase pairs LLM with required attribute classifiers using weighted KL-divergence; optimization phase uses energy function with classifier scores and penalty terms for iterative conflict resolution.

Result: Significantly outperforms baselines in attribute accuracy, linguistic fluency, output diversity while reducing toxicity, across 17 attribute dimensions.

Conclusion: C³TG provides an effective, flexible solution for multi-dimensional text attribute control that requires no costly model modifications.

Abstract: Recent advancements in large language models (LLMs) have demonstrated remarkable text generation capabilities. However, controlling specific attributes of generated text remains challenging without architectural modifications or extensive fine-tuning. Current methods typically toggle a single, basic attribute but struggle with precise multi-attribute control. In scenarios where attribute requirements conflict, existing methods lack coordination mechanisms, causing interference between desired attributes. Furthermore, these methods fail to incorporate iterative optimization processes in the controlled generation pipeline. To address these limitations, we propose Conflict-aware, Composite, and Collaborative Controlled Text Generation (C$^3$TG), a two-phase framework for fine-grained, multi-dimensional text attribute control. During generation, C$^3$TG selectively pairs the LLM with the required attribute classifiers from the 17 available dimensions and employs weighted KL-divergence to adjust token probabilities. The optimization phase then leverages an energy function combining classifier scores and penalty terms to resolve attribute conflicts through iterative feedback, enabling precise control over multiple dimensions simultaneously while preserving natural text flow. Experiments show that C$^3$TG significantly outperforms baselines across multiple metrics including attribute accuracy, linguistic fluency, and output diversity, while simultaneously reducing toxicity. These results establish C$^3$TG as an effective and flexible solution for multi-dimensional text attribute control that requires no costly model modifications.

Method: Proposes Adaptive Margin-attached Preference Optimization (AMaPO) with instance-wise adaptive margins refined by Z-normalization and exponential scaling to amplify gradients for misranked samples and suppress them for correct ones.

Result: AMaPO achieves better ranking accuracy and superior downstream alignment performance compared to existing methods, and targeted analysis confirms it successfully mitigates overfitting and underfitting issues.

Conclusion: AMaPO provides a principled solution to the fundamental Overfitting-Underfitting Dilemma in offline preference optimization, offering improved performance through dynamic gradient reallocation.

Abstract: Offline preference optimization offers a simpler and more stable alternative to RLHF for aligning language models. However, their effectiveness is critically dependent on ranking accuracy, a metric where further gains are highly impactful. This limitation arises from a fundamental problem that we identify and formalize as the Overfitting-Underfitting Dilemma: current margin designs cause models to apply excessive, wasteful gradients to correctly ranked samples (overfitting) while providing insufficient corrective signals for misranked ones (underfitting). To resolve this dilemma, we propose Adaptive Margin-attached Preference Optimization (AMaPO), a simple yet principled algorithm. AMaPO employs an instance-wise adaptive margin, refined by Z-normalization and exponential scaling, which dynamically reallocates learning effort by amplifying gradients for misranked samples and suppressing them for correct ones. Extensive experiments on widely used benchmarks demonstrate that AMaPO not only achieves better ranking accuracy and superior downstream alignment performance, but targeted analysis also confirms that it successfully mitigates the core overfitting and underfitting issues.

Method: Created LEX-ICON dataset with 8,052 words from 4 languages and 2,930 pseudo-words, analyzed MLLMs’ layer-wise processing using phoneme-level attention scores across 25 semantic dimensions for both text and audio inputs.

Result: MLLMs demonstrate phonetic intuitions consistent with linguistic research and show phonosemantic attention patterns that emphasize iconic phonemes across multiple semantic dimensions.

Conclusion: This work bridges AI and cognitive linguistics, providing the first large-scale quantitative analysis of phonetic iconicity in MLLMs’ interpretability, revealing their ability to capture sound-meaning associations.

Abstract: Sound symbolism is a linguistic concept that refers to non-arbitrary associations between phonetic forms and their meanings. We suggest that this can be a compelling probe into how Multimodal Large Language Models (MLLMs) interpret auditory information in human languages. We investigate MLLMs’ performance on phonetic iconicity across textual (orthographic and IPA) and auditory forms of inputs with up to 25 semantic dimensions (e.g., sharp vs. round), observing models’ layer-wise information processing by measuring phoneme-level attention fraction scores. To this end, we present LEX-ICON, an extensive mimetic word dataset consisting of 8,052 words from four natural languages (English, French, Japanese, and Korean) and 2,930 systematically constructed pseudo-words, annotated with semantic features applied across both text and audio modalities. Our key findings demonstrate (1) MLLMs’ phonetic intuitions that align with existing linguistic research across multiple semantic dimensions and (2) phonosemantic attention patterns that highlight models’ focus on iconic phonemes. These results bridge domains of artificial intelligence and cognitive linguistics, providing the first large-scale, quantitative analyses of phonetic iconicity in terms of MLLMs’ interpretability.

Method: Constructed large-scale synthetic dataset using 5 generation techniques, explored grounding in documents/personas/topics, compared translation vs native generation, and developed modular quality evaluation pipeline with script/language detection, metadata checks, n-gram analysis, and perplexity filtering.

Result: Created BhashaKritika dataset (540B tokens), identified key trade-offs in generation strategies, and established best practices for multilingual corpus construction through empirical model runs.

Conclusion: The study provides a systematic framework for generating and evaluating synthetic multilingual pretraining data, highlighting effective strategies for low-resource language settings and enabling robust quality control across diverse linguistic contexts.

Abstract: In the context of pretraining of Large Language Models (LLMs), synthetic data has emerged as an alternative for generating high-quality pretraining data at scale. This is particularly beneficial in low-resource language settings where the benefits of recent LLMs have been unevenly distributed across languages. In this work, we present a systematic study on the generation and evaluation of synthetic multilingual pretraining data for Indic languages, where we construct a large-scale synthetic dataset BhashaKritika, comprising 540B tokens using 5 different techniques for 10 languages. We explore the impact of grounding generation in documents, personas, and topics. We analyze how language choice, both in the prompt instructions and document grounding, affects data quality, and we compare translations of English content with native generation in Indic languages. To support scalable and language-sensitive evaluation, we introduce a modular quality evaluation pipeline that integrates script and language detection, metadata consistency checks, n-gram repetition analysis, and perplexity-based filtering using KenLM models. Our framework enables robust quality control across diverse scripts and linguistic contexts. Empirical results through model runs reveal key trade-offs in generation strategies and highlight best practices for constructing effective multilingual corpora.

Method: Uses high-resolution image processing, TrOCR for text recognition, RoBERTa-based sentiment analysis with entropy fusion, five-model voting mechanism, and unsupervised anomaly detection.

Result: Developed a robust framework that generates a numerical Stress Index from handwritten examination scripts.

Conclusion: The system presents an innovative data-driven approach in academic forensics for quantifying psychological stress through handwriting analysis.

Abstract: This research explores the fusion of graphology and artificial intelligence to quantify psychological stress levels in students by analyzing their handwritten examination scripts. By leveraging Optical Character Recognition and transformer based sentiment analysis models, we present a data driven approach that transcends traditional grading systems, offering deeper insights into cognitive and emotional states during examinations. The system integrates high resolution image processing, TrOCR, and sentiment entropy fusion using RoBERTa based models to generate a numerical Stress Index. Our method achieves robustness through a five model voting mechanism and unsupervised anomaly detection, making it an innovative framework in academic forensics.

Method: Used onboard vehicle sensors and SVM classifier for real-time pothole detection.

Result: Achieved 98.1% accuracy on data collected from 2km local road containing 26 potholes.

Conclusion: Proposed method enables effective large-scale pothole management through real-time detection using vehicle sensors.

Abstract: Road conditions play an important role in our everyday commute. With the proliferating number of vehicles on the road each year, it has become necessary to access the road conditions very frequently, this would ensure that the traffic also flows smoothly. Even the smallest crack in the road could be easily be chipped into a large pothole due to changing surface temperatures of the road and from the force of vehicles riding over it. In this paper, we have addressed how we could better identify these potholes in realtime with the help of onboard sensors in vehicles so that the data could be useful for analysis and better management of potholes on a large scale. For the implementation, we used an SVM classifier to detect potholes, we achieved 98.1% accuracy based on data collected from a local road for about 2 km which had 26 potholes distributed along the road. Code is available at: https://github.com/aswathselvam/Potholes

Method: Developed annotated ultra-high-resolution remote sensing dataset with 15 habitat categories. Proposed DWFF-Net using frozen-parameter DINOv3 encoder, data-level adaptive dynamic weighting for feature fusion, dynamic weight computation network in decoder, and hybrid loss function.

Result: Achieved mIoU of 0.6979 and F1-score of 0.8049 on constructed dataset, outperforming baseline by 0.021 and 0.0161 respectively. Improved IoU for micro-habitat categories like field ridges.

Conclusion: Established habitat identification framework enabling sub-meter precision habitat mapping at low cost, providing technical support for fine-grained habitat monitoring in cultivated landscapes.

Abstract: Addressing the current lack of a standardized habitat classification system for cultivated land ecosystems, incomplete coverage of habitat types, and the inability of existing models to effectively integrate semantic and texture features-resulting in insufficient segmentation accuracy and blurred boundaries for multi-scale habitats (e.g., large-scale field plots and micro-habitats)-this study developed a comprehensively annotated ultra-high-resolution remote sensing image dataset encompassing 15 categories of cultivated land system habitats. Furthermore, we propose a Dynamic-Weighted Feature Fusion Network (DWFF-Net). The encoder of this model utilizes a frozen-parameter DINOv3 to extract foundational features. By analyzing the relationships between different category images and feature maps, we introduce a data-level adaptive dynamic weighting strategy for feature fusion. The decoder incorporates a dynamic weight computation network to achieve thorough integration of multi-layer features, and a hybrid loss function is adopted to optimize model training. Experimental results on the constructed dataset demonstrate that the proposed model achieves a mean Intersection over Union (mIoU) of 0.6979 and an F1-score of 0.8049, outperforming the baseline network by 0.021 and 0.0161, respectively. Ablation studies further confirm the complementary nature of multi-layer feature fusion, which effectively improves the IoU for micro-habitat categories such as field ridges. This study establishes a habitat identification framework for cultivated land systems based on adaptive multi-layer feature fusion, enabling sub-meter precision habitat mapping at a low cost and providing robust technical support for fine-grained habitat monitoring in cultivated landscapes.

Method: Proposes CalMRL which leverages modality priors and connections to model missing modality imputation at representation level. Uses bi-step learning with closed-form solution for posterior distribution of shared latents to resolve optimization challenges.

Result: Extensive experiments demonstrate CalMRL’s superiority in handling missing modalities. The method provides new flexibility to utilize data with missing modalities that was previously unusable.

Conclusion: CalMRL effectively mitigates the anchor shift problem in multimodal representation learning with missing modalities, offering theoretical guarantees and practical improvements for real-world datasets with incomplete modalities.

Abstract: Multimodal representation learning harmonizes distinct modalities by aligning them into a unified latent space. Recent research generalizes traditional cross-modal alignment to produce enhanced multimodal synergy but requires all modalities to be present for a common instance, making it challenging to utilize prevalent datasets with missing modalities. We provide theoretical insights into this issue from an anchor shift perspective. Observed modalities are aligned with a local anchor that deviates from the optimal one when all modalities are present, resulting in an inevitable shift. To address this, we propose CalMRL for multimodal representation learning to calibrate incomplete alignments caused by missing modalities. Specifically, CalMRL leverages the priors and the inherent connections among modalities to model the imputation for the missing ones at the representation level. To resolve the optimization dilemma, we employ a bi-step learning method with the closed-form solution of the posterior distribution of shared latents. We validate its mitigation of anchor shift and convergence with theoretical guidance. By equipping the calibrated alignment with the existing advanced method, we offer new flexibility to absorb data with missing modalities, which is originally unattainable. Extensive experiments and comprehensive analyses demonstrate the superiority of CalMRL. Our code, model checkpoints, and evaluation raw data will be publicly available.

Method: Combines edge-aware geodesic distance learning with iterative Fast Marching refinement, adaptive prototype extraction using spatially-weighted aggregation, and adaptive parameter learning that adjusts to different organ characteristics.

Result: Extensive experiments show improvements over state-of-the-art methods, with reduced boundary errors while maintaining computational efficiency.

Conclusion: AGENet is highly suitable for clinical applications requiring precise segmentation with limited annotated data, leveraging predictable geometric patterns in medical structures.

Abstract: Medical image segmentation requires large annotated datasets, creating a significant bottleneck for clinical applications. While few-shot segmentation methods can learn from minimal examples, existing approaches demonstrate suboptimal performance in precise boundary delineation for medical images, particularly when anatomically similar regions appear without sufficient spatial context. We propose AGENet (Adaptive Geodesic Edge-aware Network), a novel framework that incorporates spatial relationships through edge-aware geodesic distance learning. Our key insight is that medical structures follow predictable geometric patterns that can guide prototype extraction even with limited training data. Unlike methods relying on complex architectural components or heavy neural networks, our approach leverages computationally lightweight geometric modeling. The framework combines three main components: (1) An edge-aware geodesic distance learning module that respects anatomical boundaries through iterative Fast Marching refinement, (2) adaptive prototype extraction that captures both global structure and local boundary details via spatially-weighted aggregation, and (3) adaptive parameter learning that automatically adjusts to different organ characteristics. Extensive experiments across diverse medical imaging datasets demonstrate improvements over state-of-the-art methods. Notably, our method reduces boundary errors compared to existing approaches while maintaining computational efficiency, making it highly suitable for clinical applications requiring precise segmentation with limited annotated data.

Method: Uses Prototype-Enhanced Registers Attention (ProERA) and Dual Relative Positional Encoding (DRPE) for feature extraction without pre-training, plus Language-Guided Prototype Embedding (LGPE) to leverage textual information for few-shot learning and enable zero-shot inference.

Result: Outperforms state-of-the-art methods by 5.68% on S3DIS and 3.82% on ScanNet benchmarks.

Conclusion: The proposed pre-training-free approach effectively addresses limitations of existing methods by better utilizing support set information and enabling both few-shot and zero-shot segmentation capabilities.

Abstract: Recent approaches for few-shot 3D point cloud semantic segmentation typically require a two-stage learning process, i.e., a pre-training stage followed by a few-shot training stage. While effective, these methods face overreliance on pre-training, which hinders model flexibility and adaptability. Some models tried to avoid pre-training yet failed to capture ample information. In addition, current approaches focus on visual information in the support set and neglect or do not fully exploit other useful data, such as textual annotations. This inadequate utilization of support information impairs the performance of the model and restricts its zero-shot ability. To address these limitations, we present a novel pre-training-free network, named Efficient Point Cloud Semantic Segmentation for Few- and Zero-shot scenarios. Our EPSegFZ incorporates three key components. A Prototype-Enhanced Registers Attention (ProERA) module and a Dual Relative Positional Encoding (DRPE)-based cross-attention mechanism for improved feature extraction and accurate query-prototype correspondence construction without pre-training. A Language-Guided Prototype Embedding (LGPE) module that effectively leverages textual information from the support set to improve few-shot performance and enable zero-shot inference. Extensive experiments show that our method outperforms the state-of-the-art method by 5.68% and 3.82% on the S3DIS and ScanNet benchmarks, respectively.

Method: TASA features a task-aware 2D affordance detection module to identify manipulable points from language and visual inputs, guiding task-relevant view selection. It then uses a 3D affordance refinement module to integrate 2D semantic priors with local 3D geometry.

Result: Experiments on SceneFun3D demonstrate that TASA significantly outperforms baselines in both accuracy and efficiency for scene-level affordance segmentation.

Conclusion: TASA effectively addresses limitations of existing methods by jointly leveraging 2D semantic cues and 3D geometric reasoning, achieving superior performance in 3D scene-level affordance understanding from natural language instructions.

Abstract: Understanding 3D scene-level affordances from natural language instructions is essential for enabling embodied agents to interact meaningfully in complex environments. However, this task remains challenging due to the need for semantic reasoning and spatial grounding. Existing methods mainly focus on object-level affordances or merely lift 2D predictions to 3D, neglecting rich geometric structure information in point clouds and incurring high computational costs. To address these limitations, we introduce Task-Aware 3D Scene-level Affordance segmentation (TASA), a novel geometry-optimized framework that jointly leverages 2D semantic cues and 3D geometric reasoning in a coarse-to-fine manner. To improve the affordance detection efficiency, TASA features a task-aware 2D affordance detection module to identify manipulable points from language and visual inputs, guiding the selection of task-relevant views. To fully exploit 3D geometric information, a 3D affordance refinement module is proposed to integrate 2D semantic priors with local 3D geometry, resulting in accurate and spatially coherent 3D affordance masks. Experiments on SceneFun3D demonstrate that TASA significantly outperforms the baselines in both accuracy and efficiency in scene-level affordance segmentation.

Method: Systematically curated SenseNova-SI-8M dataset with eight million diverse data samples under rigorous spatial capability taxonomy, built upon established multimodal foundations including Qwen3-VL, InternVL3, and Bagel models.

Result: Achieved unprecedented performance: 68.7% on VSI-Bench, 43.3% on MMSI, 85.6% on MindCube, 54.6% on ViewSpatial, and 50.1% on SITE, while maintaining 84.9% on MMBench-En for general multimodal understanding.

Conclusion: Demonstrated successful cultivation of spatial intelligence through systematic data scaling, showing emergent generalization capabilities and validating downstream application potential. The project is ongoing with publicly released models to facilitate further research.

Abstract: Despite remarkable progress, multimodal foundation models still exhibit surprising deficiencies in spatial intelligence. In this work, we explore scaling up multimodal foundation models to cultivate spatial intelligence within the SenseNova-SI family, built upon established multimodal foundations including visual understanding models (i.e., Qwen3-VL and InternVL3) and unified understanding and generation models (i.e., Bagel). We take a principled approach to constructing high-performing and robust spatial intelligence by systematically curating SenseNova-SI-8M: eight million diverse data samples under a rigorous taxonomy of spatial capabilities. SenseNova-SI demonstrates unprecedented performance across a broad range of spatial intelligence benchmarks: 68.7% on VSI-Bench, 43.3% on MMSI, 85.6% on MindCube, 54.6% on ViewSpatial, and 50.1% on SITE, while maintaining strong general multimodal understanding (e.g., 84.9% on MMBench-En). More importantly, we analyze the impact of data scaling, discuss early signs of emergent generalization capabilities enabled by diverse data training, analyze the risk of overfitting and language shortcuts, present a preliminary study on spatial chain-of-thought reasoning, and validate the potential downstream application. SenseNova-SI is an ongoing project, and this report will be updated continuously. All newly trained multimodal foundation models are publicly released to facilitate further research in this direction.

Method: Proposed LE-CapsNet as a light, enhanced variant of CapsNet with optimized structure to reduce parameters and improve performance.

Result: LE-CapsNet achieves 76.73% accuracy on CIFAR-10 with only 3.8M weights and 4x faster inference than CapsNet. It also achieves 94.3% accuracy on AffNIST (vs CapsNet’s 90.52%), showing better robustness to affine transformations.

Conclusion: LE-CapsNet successfully addresses CapsNet’s limitations by providing a more efficient, accurate, and robust alternative that maintains the benefits of capsule networks while significantly improving performance.

Abstract: Capsule Network (CapsNet) classifier has several advantages over CNNs, including better detection of images containing overlapping categories and higher accuracy on transformed images. Despite the advantages, CapsNet is slow due to its different structure. In addition, CapsNet is resource-hungry, includes many parameters and lags in accuracy compared to CNNs. In this work, we propose LE-CapsNet as a light, enhanced and more accurate variant of CapsNet. Using 3.8M weights, LECapsNet obtains 76.73% accuracy on the CIFAR-10 dataset while performing inference 4x faster than CapsNet. In addition, our proposed network is more robust at detecting images with affine transformations compared to CapsNet. We achieve 94.3% accuracy on the AffNIST dataset (compared to CapsNet 90.52%).

Method: Target-Balanced Score Distillation (TBSD) formulates generation as a multi-objective optimization problem and introduces an adaptive strategy that effectively resolves the trade-off by properly utilizing Target Negative Prompts.

Result: Extensive experiments demonstrate that TBSD significantly outperforms existing state-of-the-art methods, yielding 3D assets with high-fidelity textures and geometrically accurate shape.

Conclusion: TBSD successfully resolves the fundamental trade-off in SDS variants by adaptively balancing texture realism and shape preservation through multi-objective optimization.

Abstract: Score Distillation Sampling (SDS) enables 3D asset generation by distilling priors from pretrained 2D text-to-image diffusion models, but vanilla SDS suffers from over-saturation and over-smoothing. To mitigate this issue, recent variants have incorporated negative prompts. However, these methods face a critical trade-off: limited texture optimization, or significant texture gains with shape distortion. In this work, we first conduct a systematic analysis and reveal that this trade-off is fundamentally governed by the utilization of the negative prompts, where Target Negative Prompts (TNP) that embed target information in the negative prompts dramatically enhancing texture realism and fidelity but inducing shape distortions. Informed by this key insight, we introduce the Target-Balanced Score Distillation (TBSD). It formulates generation as a multi-objective optimization problem and introduces an adaptive strategy that effectively resolves the aforementioned trade-off. Extensive experiments demonstrate that TBSD significantly outperforms existing state-of-the-art methods, yielding 3D assets with high-fidelity textures and geometrically accurate shape.

Method: Uses a MicroNAS-inspired framework that treats rank selection as a global search problem, employing fast accuracy estimators to evaluate candidate tensor decompositions (like Tucker factorization) under memory and accuracy constraints.

Result: Achieved 8x compression of ResNet-18 on ImageNet with <4% accuracy drop; 2x compression of YOLOv5s on COCO with no accuracy loss; 2x compression of YOLOv5n with 2.5% accuracy drop. Introduced STResNet family with competitive performance.

Conclusion: CompressNAS provides an effective framework for globally optimizing CNN compression through tensor decomposition, enabling significant model size reduction while maintaining competitive accuracy for deployment on resource-constrained devices.

Abstract: Deep Convolutional Neural Networks (CNNs) are increasingly difficult to deploy on microcontrollers (MCUs) and lightweight NPUs (Neural Processing Units) due to their growing size and compute demands. Low-rank tensor decomposition, such as Tucker factorization, is a promising way to reduce parameters and operations with reasonable accuracy loss. However, existing approaches select ranks locally and often ignore global trade-offs between compression and accuracy. We introduce CompressNAS, a MicroNAS-inspired framework that treats rank selection as a global search problem. CompressNAS employs a fast accuracy estimator to evaluate candidate decompositions, enabling efficient yet exhaustive rank exploration under memory and accuracy constraints. In ImageNet, CompressNAS compresses ResNet-18 by 8x with less than 4% accuracy drop; on COCO, we achieve 2x compression of YOLOv5s without any accuracy drop and 2x compression of YOLOv5n with a 2.5% drop. Finally, we present a new family of compressed models, STResNet, with competitive performance compared to other efficient models.

Method: MIRROR uses dedicated encoders for each modality, a modality alignment module for integration, a modality retention module to preserve unique attributes, and a style clustering module to reduce redundancy and enhance disease-relevant information through clustering space modeling.

Result: Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis demonstrate MIRROR’s superior performance in constructing comprehensive oncological feature representations.

Conclusion: MIRROR effectively integrates histopathology and transcriptomics data while maintaining modality-specific fidelity, benefiting cancer diagnosis through comprehensive feature representations.

Abstract: Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease. Multi-modal self-supervised learning has demonstrated remarkable potential in learning pathological representations by integrating diverse data sources. Conventional multi-modal integration methods primarily emphasize modality alignment, while paying insufficient attention to retaining the modality-specific structures. However, unlike conventional scenarios where multi-modal inputs share highly overlapping features, histopathology and transcriptomics exhibit pronounced heterogeneity, offering orthogonal yet complementary insights. Histopathology provides morphological and spatial context, elucidating tissue architecture and cellular topology, whereas transcriptomics delineates molecular signatures through gene expression patterns. This inherent disparity introduces a major challenge in aligning them while maintaining modality-specific fidelity. To address these challenges, we present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention. MIRROR employs dedicated encoders to extract comprehensive features for each modality, which is further complemented by a modality alignment module to achieve seamless integration between phenotype patterns and molecular profiles. Furthermore, a modality retention module safeguards unique attributes from each modality, while a style clustering module mitigates redundancy and enhances disease-relevant information by modeling and aligning consistent pathological signatures within a clustering space. Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR’s superior performance, demonstrating its effectiveness in constructing comprehensive oncological feature representations and benefiting the cancer diagnosis.

Method: Uses two adaptation modes: lightweight token-prompt retrieval from shared memory for resource-limited UAVs, and gradient-free sparse visual prompt optimization via CMA-ES for resource-massive UAVs, with activation-statistic detection and cross-UAV knowledge sharing.

Result: Significantly improves segmentation accuracy and robustness over static models and state-of-the-art TTA baselines on UAVid and VDD benchmarks, with real-world validation under diverse weather conditions.

Conclusion: Provides a practical path to resilient, communication-efficient perception for low-altitude UAV networks through collaborative adaptation with minimal bandwidth overhead.

Abstract: Low-altitude Unmanned Aerial Vehicle (UAV) networks rely on robust semantic segmentation as a foundational enabler for distributed sensing-communication-control co-design across heterogeneous agents within the network. However, segmentation foundation models deteriorate quickly under weather, lighting, and viewpoint drift. Resource-limited UAVs cannot run gradient-based test-time adaptation, while resource-massive UAVs adapt independently, wasting shared experience. To address these challenges, we propose AdaptFly, a prompt-guided test-time adaptation framework that adjusts segmentation models without weight updates. AdaptFly features two complementary adaptation modes. For resource-limited UAVs, it employs lightweight token-prompt retrieval from a shared global memory. For resource-massive UAVs, it uses gradient-free sparse visual prompt optimization via Covariance Matrix Adaptation Evolution Strategy. An activation-statistic detector triggers adaptation, while cross-UAV knowledge pool consolidates prompt knowledge and enables fleet-wide collaboration with negligible bandwidth overhead. Extensive experiments on UAVid and VDD benchmarks, along with real-world UAV deployments under diverse weather conditions, demonstrate that AdaptFly significantly improves segmentation accuracy and robustness over static models and state-of-the-art TTA baselines. The results highlight a practical path to resilient, communication-efficient perception in the emerging low-altitude economy.

Method: Uses masked autoencoder with novel masking strategy based on human eye blind spot knowledge, mimicking how the brain fills visual gaps. The pretrained encoder is then used in contrastive learning-based video-text model for word-referent mapping.

Result: The proposed biologically plausible masking strategy is at least as effective as standard random masking for learning word-referent mappings from cross-situational and temporally extended episodes.

Conclusion: Biologically inspired masking strategies can be equally effective as standard approaches while being more biologically justified, offering promising directions for developmental AI systems.

Abstract: Typically, children start to learn their first words between 6 and 9 months, linking spoken utterances to their visual referents. Without prior knowledge, a word encountered for the first time can be interpreted in countless ways; it might refer to any of the objects in the environment, their components, or attributes. Using longitudinal, egocentric, and ecologically valid data from the experience of one child, in this work, we propose a self-supervised and biologically plausible strategy to learn strong visual representations. Our masked autoencoder-based visual backbone incorporates knowledge about the blind spot in human eyes to define a novel masking strategy. This mask and reconstruct approach attempts to mimic the way the human brain fills the gaps in the eyes’ field of view. This represents a significant shift from standard random masking strategies, which are difficult to justify from a biological perspective. The pretrained encoder is utilized in a contrastive learning-based video-text model capable of acquiring word-referent mappings. Extensive evaluation suggests that the proposed biologically plausible masking strategy is at least as effective as random masking for learning word-referent mappings from cross-situational and temporally extended episodes.

Method: Uses Graph Convolutional Network encoder based on Kolmogorov-Arnold Networks to capture nonlinear dependencies, spot-feature-pair contrastive learning for cross-modal alignment, and dynamic expert routing mechanism to adaptively select informative modalities per spot.

Result: GROVER outperforms state-of-the-art baselines on real-world spatial omics datasets, demonstrating robust and reliable multimodal integration performance.

Conclusion: GROVER provides an effective solution for adaptive integration of spatial multi-omics data, successfully addressing key challenges in multimodal fusion through its novel architectural components.

Abstract: Effectively modeling multimodal spatial omics data is critical for understanding tissue complexity and underlying biological mechanisms. While spatial transcriptomics, proteomics, and epigenomics capture molecular features, they lack pathological morphological context. Integrating these omics with histopathological images is therefore essential for comprehensive disease tissue analysis. However, substantial heterogeneity across omics, imaging, and spatial modalities poses significant challenges. Naive fusion of semantically distinct sources often leads to ambiguous representations. Additionally, the resolution mismatch between high-resolution histology images and lower-resolution sequencing spots complicates spatial alignment. Biological perturbations during sample preparation further distort modality-specific signals, hindering accurate integration. To address these challenges, we propose Graph-guided Representation of Omics and Vision with Expert Regulation for Adaptive Spatial Multi-omics Fusion (GROVER), a novel framework for adaptive integration of spatial multi-omics data. GROVER leverages a Graph Convolutional Network encoder based on Kolmogorov-Arnold Networks to capture the nonlinear dependencies between each modality and its associated spatial structure, thereby producing expressive, modality-specific embeddings. To align these representations, we introduce a spot-feature-pair contrastive learning strategy that explicitly optimizes the correspondence across modalities at each spot. Furthermore, we design a dynamic expert routing mechanism that adaptively selects informative modalities for each spot while suppressing noisy or low-quality inputs. Experiments on real-world spatial omics datasets demonstrate that GROVER outperforms state-of-the-art baselines, providing a robust and reliable solution for multimodal integration.

Method: Propose HSI-Detect, a two-stage pipeline that reconstructs 31-channel hyperspectral images from standard RGB input and performs detection in the hyperspectral domain.

Result: Evaluation on FaceForensics++ dataset shows consistent improvements over RGB-only baselines, with hyperspectral imaging amplifying manipulation artifacts that are weak or invisible in RGB.

Conclusion: Spectral-domain mapping shows promise for Deepfake detection by expanding input representation into denser spectral bands to reveal hidden manipulation artifacts.

Abstract: Modern generative and diffusion models produce highly realistic images that can mislead human perception and even sophisticated automated detection systems. Most detection methods operate in RGB space and thus analyze only three spectral channels. We propose HSI-Detect, a two-stage pipeline that reconstructs a 31-channel hyperspectral image from a standard RGB input and performs detection in the hyperspectral domain. Expanding the input representation into denser spectral bands amplifies manipulation artifacts that are often weak or invisible in the RGB domain, particularly in specific frequency bands. We evaluate HSI-Detect across FaceForensics++ dataset and show the consistent improvements over RGB-only baselines, illustrating the promise of spectral-domain mapping for Deepfake detection.

Method: Analyzed two symmetry-aware metrics (Procrustes Matching and Hard Gromov Wasserstein), showed they’re not bi-Lipschitz equivalent, modified standard invariant networks to achieve bi-Lipschitz guarantees.

Result: Standard invariant networks are not bi-Lipschitz with respect to PM metric, but modified versions can obtain bi-Lipschitz guarantees and show advantages in correspondence tasks.

Conclusion: Proposed bi-Lipschitz invariant models outperform standard invariant models for finding correspondences between 3D point clouds.

Abstract: Many neural networks for point clouds are, by design, invariant to the symmetries of this datatype: permutations and rigid motions. The purpose of this paper is to examine whether such networks preserve natural symmetry aware distances on the point cloud spaces, through the notion of bi-Lipschitz equivalence. This inquiry is motivated by recent work in the Equivariant learning literature which highlights the advantages of bi-Lipschitz models in other scenarios. We consider two symmetry aware metrics on point clouds: (a) The Procrustes Matching (PM) metric and (b) Hard Gromov Wasserstien distances. We show that these two distances themselves are not bi-Lipschitz equivalent, and as a corollary deduce that popular invariant networks for point clouds are not bi-Lipschitz with respect to the PM metric. We then show how these networks can be modified so that they do obtain bi-Lipschitz guarantees. Finally, we provide initial experiments showing the advantage of the proposed bi-Lipschitz model over standard invariant models, for the tasks of finding correspondences between 3D point clouds.

Method: Multi-agent system with: 1) multimodal concept generator that mines visual concepts from training images, 2) symbol discovery conditioned on visual concepts, 3) LLM reasoner that composes symbols into first-order rules, 4) vision verifier that quantifies symbol presence during inference.

Result: Outperforms state-of-the-art neurosymbolic baselines by average 5% on five benchmarks (including medical imaging and underrepresented datasets), reduces hallucinated symbols in rules by up to 50%.

Conclusion: The system successfully reinstates visual grounding while maintaining transparent reasoning, providing explicit reasoning pathways and reducing hallucinations in vision-language tasks.

Abstract: Modern vision-language models (VLMs) deliver impressive predictive accuracy yet offer little insight into ‘why’ a decision is reached, frequently hallucinating facts, particularly when encountering out-of-distribution data. Neurosymbolic frameworks address this by pairing black-box perception with interpretable symbolic reasoning, but current methods extract their symbols solely from task labels, leaving them weakly grounded in the underlying visual data. In this paper, we introduce a multi-agent system - Concept-RuleNet that reinstates visual grounding while retaining transparent reasoning. Specifically, a multimodal concept generator first mines discriminative visual concepts directly from a representative subset of training images. Next, these visual concepts are utilized to condition symbol discovery, anchoring the generations in real image statistics and mitigating label bias. Subsequently, symbols are composed into executable first-order rules by a large language model reasoner agent - yielding interpretable neurosymbolic rules. Finally, during inference, a vision verifier agent quantifies the degree of presence of each symbol and triggers rule execution in tandem with outputs of black-box neural models, predictions with explicit reasoning pathways. Experiments on five benchmarks, including two challenging medical-imaging tasks and three underrepresented natural-image datasets, show that our system augments state-of-the-art neurosymbolic baselines by an average of 5% while also reducing the occurrence of hallucinated symbols in rules by up to 50%.

Method: Uses implicit sparse attention mechanism that selects primary components/dimensions instead of attending to entire sequences. Applied to encoder-decoder ANN architectures for synthetic image generation, specifically tested on face recognition with makeup and occlusion datasets.

Result: Achieved significant reduction in bottleneck size while increasing variability of limited original datasets for face recognition tasks involving makeup and occlusion.

Conclusion: Batch Transformers with dimension-wise attention provide an efficient alternative to traditional Transformers, enabling better performance on data-limited tasks through improved feature selection and reduced computational requirements.

Abstract: A novel Transformer variation architecture is proposed in the implicit sparse style. Unlike “traditional” Transformers, instead of attention to sequential or batch entities in their entirety of whole dimensionality, in the proposed Batch Transformers, attention to the “important” dimensions (primary components) is implemented. In such a way, the “important” dimensions or feature selection allows for a significant reduction of the bottleneck size in the encoder-decoder ANN architectures. The proposed architecture is tested on the synthetic image generation for the face recognition task in the case of the makeup and occlusion data set, allowing for increased variability of the limited original data set.

Method: Uses reflective reinforcement learning to orchestrate a registry of pretrained experts, handles prompts through dynamic task decomposition, and supervises alignment via structured feedback from a vision-language model critic, casting image synthesis as a Markov Decision Process.

Result: Outperforms baselines including frontier models across industry-standard and custom benchmarks in alignment, fidelity, and aesthetics, and is consistently preferred in human evaluations.

Conclusion: Reinforcement learning can enable AI systems to autonomously decompose, reorder, and combine visual models, advancing towards general-purpose visual assistants.

Abstract: Recent advances in text-to-image generation have produced strong single-shot models, yet no individual system reliably executes the long, compositional prompts typical of creative workflows. We introduce Image-POSER, a reflective reinforcement learning framework that (i) orchestrates a diverse registry of pretrained text-to-image and image-to-image experts, (ii) handles long-form prompts end-to-end through dynamic task decomposition, and (iii) supervises alignment at each step via structured feedback from a vision-language model critic. By casting image synthesis and editing as a Markov Decision Process, we learn non-trivial expert pipelines that adaptively combine strengths across models. Experiments show that Image-POSER outperforms baselines, including frontier models, across industry-standard and custom benchmarks in alignment, fidelity, and aesthetics, and is consistently preferred in human evaluations. These results highlight that reinforcement learning can endow AI systems with the capacity to autonomously decompose, reorder, and combine visual models, moving towards general-purpose visual assistants.

Method: Uses a ground-truth-primed memory and burn-in anchor loss for stable initialization, with a single lightweight temporal-attention layer that refines embeddings across frames for fixed GPU memory usage.

Result: Achieves 76.3 AUC and 53.7 FPS (AMP, 4.3 GB VRAM) on Mini-LaSOT benchmark, outperforming transformer baselines like TrackFormer and MOTRv2 under fast motion, scale change, and occlusion.

Conclusion: SOTFormer demonstrates effective unification of detection, tracking, and prediction with constant memory requirements and real-time performance, providing robust handling of challenging scenarios.

Abstract: Accurate single-object tracking and short-term motion forecasting remain challenging under occlusion, scale variation, and temporal drift, which disrupt the temporal coherence required for real-time perception. We introduce \textbf{SOTFormer}, a minimal constant-memory temporal transformer that unifies object detection, tracking, and short-horizon trajectory prediction within a single end-to-end framework. Unlike prior models with recurrent or stacked temporal encoders, SOTFormer achieves stable identity propagation through a ground-truth-primed memory and a burn-in anchor loss that explicitly stabilizes initialization. A single lightweight temporal-attention layer refines embeddings across frames, enabling real-time inference with fixed GPU memory. On the Mini-LaSOT (20%) benchmark, SOTFormer attains 76.3 AUC and 53.7 FPS (AMP, 4.3 GB VRAM), outperforming transformer baselines such as TrackFormer and MOTRv2 under fast motion, scale change, and occlusion.

Method: Proposes MP-GFormer, a dynamic graph transformer that integrates evolving 3D geometric representations through attention mechanism using StereoLithography surface meshes and boundary representation for initial designs.

Result: Achieves 24% improvement in accuracy for main operation predictions and 36% improvement for sub-operation predictions compared to state-of-the-art approaches on synthesized dataset.

Conclusion: Integrating 3D geometric information into dynamic graph learning significantly enhances machining operation sequence prediction accuracy, demonstrating the importance of domain-aware geometric representations.

Abstract: Machining process planning (MP) is inherently complex due to structural and geometrical dependencies among part features and machining operations. A key challenge lies in capturing dynamic interdependencies that evolve with distinct part geometries as operations are performed. Machine learning has been applied to address challenges in MP, such as operation selection and machining sequence prediction. Dynamic graph learning (DGL) has been widely used to model dynamic systems, thanks to its ability to integrate spatio-temporal relationships. However, in MP, while existing DGL approaches can capture these dependencies, they fail to incorporate three-dimensional (3D) geometric information of parts and thus lack domain awareness in predicting machining operation sequences. To address this limitation, we propose MP-GFormer, a 3D-geometry-aware dynamic graph transformer that integrates evolving 3D geometric representations into DGL through an attention mechanism to predict machining operation sequences. Our approach leverages StereoLithography surface meshes representing the 3D geometry of a part after each machining operation, with the boundary representation method used for the initial 3D designs. We evaluate MP-GFormer on a synthesized dataset and demonstrate that the method achieves improvements of 24% and 36% in accuracy for main and sub-operation predictions, respectively, compared to state-of-the-art approaches.

Method: Introduces Chromatic-Aware Coordinate Transformation module to learn image-adaptive color space, isolating fringing into a dedicated dimension, then uses 5D Look-Up Table for color correction. Created PF-Synth dataset for training.

Result: Extensive experiments on synthetic and real-world datasets demonstrate state-of-the-art performance in purple fringing removal.

Conclusion: DCA-LUT provides an effective deep learning solution for purple fringing removal that outperforms traditional methods.

Abstract: Purple fringing, a persistent artifact caused by Longitudinal Chromatic Aberration (LCA) in camera lenses, has long degraded the clarity and realism of digital imaging. Traditional solutions rely on complex and expensive apochromatic (APO) lens hardware and the extraction of handcrafted features, ignoring the data-driven approach. To fill this gap, we introduce DCA-LUT, the first deep learning framework for purple fringing removal. Inspired by the physical root of the problem, the spatial misalignment of RGB color channels due to lens dispersion, we introduce a novel Chromatic-Aware Coordinate Transformation (CA-CT) module, learning an image-adaptive color space to decouple and isolate fringing into a dedicated dimension. This targeted separation allows the network to learn a precise ``purple fringe channel", which then guides the accurate restoration of the luminance channel. The final color correction is performed by a learned 5D Look-Up Table (5D LUT), enabling efficient and powerful% non-linear color mapping. To enable robust training and fair evaluation, we constructed a large-scale synthetic purple fringing dataset (PF-Synth). Extensive experiments in synthetic and real-world datasets demonstrate that our method achieves state-of-the-art performance in purple fringing removal.

Method: Two-stage approach: (1) redistribute task-relevant information across layers via L2-regularized optimization, (2) inject structured perturbations to misalign task subspaces and break curvature compatibility in the loss landscape.

Result: Extensive experiments on vision (ViT-L-14) and language (Llama2, Gemma2, Mistral) models show MergeGuard reduces merged model accuracy by up to 90% with less than 1.5% performance loss on the protected model.

Conclusion: MergeGuard effectively protects models from unauthorized merging by reshaping parameter geometry to cause destructive interference in merged models while maintaining original model functionality.

Abstract: The rapid proliferation of pretrained models and open repositories has made model merging a convenient yet risky practice, allowing free-riders to combine fine-tuned models into a new multi-capability model without authorization. Such unauthorized model merging not only violates intellectual property rights but also undermines model ownership and accountability. To address this issue, we present MergeGuard, a proactive dual-stage weight protection framework that disrupts merging compatibility while maintaining task fidelity. In the first stage, we redistribute task-relevant information across layers via L2-regularized optimization, ensuring that important gradients are evenly dispersed. In the second stage, we inject structured perturbations to misalign task subspaces, breaking curvature compatibility in the loss landscape. Together, these stages reshape the model’s parameter geometry such that merged models collapse into destructive interference while the protected model remains fully functional. Extensive experiments on both vision (ViT-L-14) and language (Llama2, Gemma2, Mistral) models demonstrate that MergeGuard reduces merged model accuracy by up to 90% with less than 1.5% performance loss on the protected model.

Method: Built on MedSigLIP with Feature-wise Linear Modulation (FiLM) for injecting textual priors, combined with multi-scale pooling (global, local, texture-aware) through separate regression heads fused by lightweight MLP, trained with pairwise ranking loss.

Result: Achieved PLCC = 0.9575, SROCC = 0.9561, and KROCC = 0.8301 on LDCTIQA2023 with only 1,000 training images, surpassing top-ranked published challenge submissions.

Conclusion: The prompt-guided approach demonstrates effectiveness in medical image quality assessment, enabling superior performance with limited training data through clinical intent conditioning.

Abstract: We propose a prompt-conditioned framework built on MedSigLIP that injects textual priors via Feature-wise Linear Modulation (FiLM) and multi-scale pooling. Text prompts condition patch-token features on clinical intent, enabling data-efficient learning and rapid adaptation. The architecture combines global, local, and texture-aware pooling through separate regression heads fused by a lightweight MLP, trained with pairwise ranking loss. Evaluated on the LDCTIQA2023 (a public LDCT quality assessment challenge) with 1,000 training images, we achieve PLCC = 0.9575, SROCC = 0.9561, and KROCC = 0.8301, surpassing the top-ranked published challenge submissions and demonstrating the effectiveness of our prompt-guided approach.

Method: Proposed FocusSDF loss function based on signed distance functions that adaptively assigns higher weights to pixels closer to lesion/organ boundaries, making the network boundary-aware.

Result: Extensive evaluations across 5 state-of-the-art models including MedSAM, using 4 distance-based loss functions on diverse medical datasets (cerebral aneurysm, stroke, liver, breast tumor) consistently showed FocusSDF’s superior performance.

Conclusion: FocusSDF effectively addresses boundary preservation challenges in medical image segmentation and demonstrates consistent superiority over existing distance transform based loss functions.

Abstract: Segmentation of medical images constitutes an essential component of medical image analysis, providing the foundation for precise diagnosis and efficient therapeutic interventions in clinical practices. Despite substantial progress, most segmentation models do not explicitly encode boundary information; as a result, making boundary preservation a persistent challenge in medical image segmentation. To address this challenge, we introduce FocusSDF, a novel loss function based on the signed distance functions (SDFs), which redirects the network to concentrate on boundary regions by adaptively assigning higher weights to pixels closer to the lesion or organ boundary, effectively making it boundary aware. To rigorously validate FocusSDF, we perform extensive evaluations against five state-of-the-art medical image segmentation models, including the foundation model MedSAM, using four distance-based loss functions across diverse datasets covering cerebral aneurysm, stroke, liver, and breast tumor segmentation tasks spanning multiple imaging modalities. The experimental results consistently demonstrate the superior performance of FocusSDF over existing distance transform based loss functions.

Method: Compared baseline model trained on real imagery with 5 zero-shot and 5 few-shot models incorporating progressively more synthetic imagery. Zero-shot used only synthetic data, while few-shot combined real and synthetic images.

Result: Zero-shot models showed improved detection performance with added synthetic imagery, plateauing beyond 100% supplementation. Few-shot models achieved better recall and slightly higher accuracy with synthetic-real combinations, though not statistically significant.

Conclusion: Synthetic imagery enables accurate object detection models for wildlife monitoring when real data is scarce, allowing monitoring of rare/inaccessible species and increased monitoring frequency without requiring initial real data.

Abstract: Accurate population estimates are essential for wildlife management, providing critical insights into species abundance and distribution. Traditional survey methods, including visual aerial counts and GNSS telemetry tracking, are widely used to monitor muskox populations in Arctic regions. These approaches are resource intensive and constrained by logistical challenges. Advances in remote sensing, artificial intelligence, and high resolution aerial imagery offer promising alternatives for wildlife detection. Yet, the effectiveness of deep learning object detection models (ODMs) is often limited by small datasets, making it challenging to train robust ODMs for sparsely distributed species like muskoxen. This study investigates the integration of synthetic imagery (SI) to supplement limited training data and improve muskox detection in zero shot (ZS) and few-shot (FS) settings. We compared a baseline model trained on real imagery with 5 ZS and 5 FS models that incorporated progressively more SI in the training set. For the ZS models, where no real images were included in the training set, adding SI improved detection performance. As more SI were added, performance in precision, recall and F1 score increased, but eventually plateaued, suggesting diminishing returns when SI exceeded 100% of the baseline model training dataset. For FS models, combining real and SI led to better recall and slightly higher overall accuracy compared to using real images alone, though these improvements were not statistically significant. Our findings demonstrate the potential of SI to train accurate ODMs when data is scarce, offering important perspectives for wildlife monitoring by enabling rare or inaccessible species to be monitored and to increase monitoring frequency. This approach could be used to initiate ODMs without real data and refine it as real images are acquired over time.

Method: Developed Harpia - a CUDA-based processing library with strict memory control, native chunked execution, GPU-accelerated filtering, annotation, and quantification tools for handling datasets exceeding single-GPU memory capacity.

Result: Significant improvements in processing speed, memory efficiency, and scalability compared to NVIDIA cuCIM and scikit-image frameworks. Enables reliable operation on large datasets with interactive human-in-the-loop interface.

Conclusion: Harpia’s efficient GPU resource management and interactive interface make it particularly suitable for collaborative scientific imaging workflows in shared HPC infrastructures.

Abstract: High-resolution volumetric imaging techniques, such as X-ray tomography and advanced microscopy, generate increasingly large datasets that challenge existing tools for efficient processing, segmentation, and interactive exploration. This work introduces new capabilities to Annotat3D through Harpia, a new CUDA-based processing library designed to support scalable, interactive segmentation workflows for large 3D datasets in high-performance computing (HPC) and remote-access environments. Harpia features strict memory control, native chunked execution, and a suite of GPU-accelerated filtering, annotation, and quantification tools, enabling reliable operation on datasets exceeding single-GPU memory capacity. Experimental results demonstrate significant improvements in processing speed, memory efficiency, and scalability compared to widely used frameworks such as NVIDIA cuCIM and scikit-image. The system’s interactive, human-in-the-loop interface, combined with efficient GPU resource management, makes it particularly suitable for collaborative scientific imaging workflows in shared HPC infrastructures.

Method: Proposes C3Net with dual-pathway decoder: Edge Refinement Pathway uses gradient-initialized Edge Enhancement Modules for boundary recovery, and Contextual Localization Pathway employs Image-based Context Guidance for intrinsic saliency suppression. An Attentive Fusion Module combines both pathways via spatial gating.

Result: Achieves state-of-the-art performance with S-measures of 0.898 on COD10K, 0.904 on CAMO, and 0.913 on NC4K datasets while maintaining efficient processing.

Conclusion: Complex, multifaceted detection challenges require architectural innovation with specialized components working synergistically for comprehensive coverage beyond isolated improvements.

Abstract: Camouflaged object detection identifies objects that blend seamlessly with their surroundings through similar colors, textures, and patterns. This task challenges both traditional segmentation methods and modern foundation models, which fail dramatically on camouflaged objects. We identify six fundamental challenges in COD: Intrinsic Similarity, Edge Disruption, Extreme Scale Variation, Environmental Complexities, Contextual Dependencies, and Salient-Camouflaged Object Disambiguation. These challenges frequently co-occur and compound the difficulty of detection, requiring comprehensive architectural solutions. We propose C3Net, which addresses all challenges through a specialized dual-pathway decoder architecture. The Edge Refinement Pathway employs gradient-initialized Edge Enhancement Modules to recover precise boundaries from early features. The Contextual Localization Pathway utilizes our novel Image-based Context Guidance mechanism to achieve intrinsic saliency suppression without external models. An Attentive Fusion Module synergistically combines the two pathways via spatial gating. C3Net achieves state-of-the-art performance with S-measures of 0.898 on COD10K, 0.904 on CAMO, and 0.913 on NC4K, while maintaining efficient processing. C3Net demonstrates that complex, multifaceted detection challenges require architectural innovation, with specialized components working synergistically to achieve comprehensive coverage beyond isolated improvements. Code, model weights, and results are available at https://github.com/Baber-Jan/C3Net.

Method: Adapted DSPy framework for automated prompt optimization, implementing pipelines for 5 medical imaging tasks across radiology, gastroenterology, and dermatology, evaluating 10 open-source VLMs with 4 prompt optimization techniques.

Result: Optimized pipelines achieved 53% median relative improvement over zero-shot baselines, with largest gains from 300% to 3,400% on tasks with low zero-shot performance. Approach is scalable, preserves data privacy, and works with open-source models.

Conclusion: Automated prompt optimization has substantial potential for medical AI systems, enabling significant performance gains while reducing dependence on manual prompt design, allowing clinicians to focus on patient care.

Abstract: Vision-language foundation models (VLMs) show promise for diverse imaging tasks but often underperform on medical benchmarks. Prior efforts to improve performance include model finetuning, which requires large domain-specific datasets and significant compute, or manual prompt engineering, which is hard to generalize and often inaccessible to medical institutions seeking to deploy these tools. These challenges motivate interest in approaches that draw on a model’s embedded knowledge while abstracting away dependence on human-designed prompts to enable scalable, weight-agnostic performance improvements. To explore this, we adapt the Declarative Self-improving Python (DSPy) framework for structured automated prompt optimization in medical vision-language systems through a comprehensive, formal evaluation. We implement prompting pipelines for five medical imaging tasks across radiology, gastroenterology, and dermatology, evaluating 10 open-source VLMs with four prompt optimization techniques. Optimized pipelines achieved a median relative improvement of 53% over zero-shot prompting baselines, with the largest gains ranging from 300% to 3,400% on tasks where zero-shot performance is low. These results highlight the substantial potential of applying automated prompt optimization to medical AI systems, demonstrating significant gains for vision-based applications requiring accurate clinical image interpretation. By reducing dependence on prompt design to elicit intended outputs, these techniques allow clinicians to focus on patient care and clinical decision-making. Furthermore, our experiments offer scalability and preserve data privacy, demonstrating performance improvement on open-source VLMs. We publicly release our evaluation pipelines to support reproducible research on specialized medical tasks, available at https://github.com/DaneshjouLab/prompt-triage-lab.

Method: Uses Pyramid Vision Transformer backbone for multi-scale feature extraction, Attention-Based Scale Integration Units for selective feature merging, and recursive-feedback decoding with Multi-Granularity Fusion Units for precise object detection.

Result: Achieves state-of-the-art results on two benchmark datasets and ranks second on two other datasets, successfully detecting small and multiple camouflaged objects.

Conclusion: The proposed MSRNet effectively addresses challenges in camouflaged object detection through multi-scale learning and recursive feature optimization, demonstrating improved performance in complex scenarios.

Abstract: Camouflaged object detection is an emerging and challenging computer vision task that requires identifying and segmenting objects that blend seamlessly into their environments due to high similarity in color, texture, and size. This task is further complicated by low-light conditions, partial occlusion, small object size, intricate background patterns, and multiple objects. While many sophisticated methods have been proposed for this task, current methods still struggle to precisely detect camouflaged objects in complex scenarios, especially with small and multiple objects, indicating room for improvement. We propose a Multi-Scale Recursive Network that extracts multi-scale features via a Pyramid Vision Transformer backbone and combines them via specialized Attention-Based Scale Integration Units, enabling selective feature merging. For more precise object detection, our decoder recursively refines features by incorporating Multi-Granularity Fusion Units. A novel recursive-feedback decoding strategy is developed to enhance global context understanding, helping the model overcome the challenges in this task. By jointly leveraging multi-scale learning and recursive feature optimization, our proposed method achieves performance gains, successfully detecting small and multiple camouflaged objects. Our model achieves state-of-the-art results on two benchmark datasets for camouflaged object detection and ranks second on the remaining two. Our codes, model weights, and results are available at \href{https://github.com/linaagh98/MSRNet}{https://github.com/linaagh98/MSRNet}.

Method: Dual-path architecture with intelligent routing (low missingness to MICE, complex patterns to GAIN with temporal analysis), cross-path attention fusion using missingness-aware embeddings, and end-to-end joint optimization.

Result: State-of-the-art performance with RMSE of 0.108 (vs. baselines’ 0.119-0.152) and AUROC of 0.812 for mortality prediction on MIMIC-III and multimodal benchmarks.

Conclusion: PI-NAIM provides a unified solution for real-world scenarios with incomplete data, enabling seamless integration into vision pipelines handling missing modalities or corrupted inputs.

Abstract: Medical imaging and multi-modal clinical settings often face the challange of missing modality in their diagnostic pipelines. Existing imputation methods either lack representational capacity or are computationally expensive. We propose PI-NAIM, a novel dual-path architecture that dynamically routes samples to optimized imputation approaches based on missingness complexity. Our framework integrates: (1) intelligent path routing that directs low missingness samples to efficient statistical imputation (MICE) and complex patterns to powerful neural networks (GAIN with temporal analysis); (2) cross-path attention fusion that leverages missingness-aware embeddings to intelligently combine both branches; and (3) end-to-end joint optimization of imputation accuracy and downstream task performance. Extensive experiments on MIMIC-III and multimodal benchmarks demonstrate state-of-the-art performance, achieving RMSE of 0.108 (vs. baselines’ 0.119-0.152) and substantial gains in downstream tasks with an AUROC of 0.812 for mortality prediction. PI-NAIM’s modular design enables seamless integration into vision pipelines handling incomplete sensor measurements, missing modalities, or corrupted inputs, providing a unified solution for real-world scenario. The code is publicly available at https://github.com/AfifaKhaled/PI-NAIM-Path-Integrated-Neural-Adaptive-Imputation-Model

Method: QTSplus uses cross-attention scoring, predicts instance-specific retention budgets based on query complexity, and selects top-n tokens with differentiable straight-through estimator during training and hard gate at inference, plus a small re-encoder for temporal order preservation.

Result: Achieves 89% vision stream compression and 28% latency reduction on long videos, with near-parity accuracy overall and significant improvements (+20.5 and +5.6 points) on TempCompass direction and order accuracies compared to original Qwen models.

Conclusion: QTSplus is an effective, general mechanism for scaling MLLMs to real-world long-video scenarios while preserving task-relevant evidence, demonstrating practical efficiency gains without sacrificing performance.

Abstract: Despite the recent advances in the video understanding ability of multimodal large language models (MLLMs), long video understanding remains a challenge. One of the main issues is that the number of vision tokens grows linearly with video length, which causes an explosion in attention cost, memory, and latency. To solve this challenge, we present Query-aware Token Selector (\textbf{QTSplus}), a lightweight yet powerful visual token selection module that serves as an information gate between the vision encoder and LLMs. Given a text query and video tokens, QTSplus dynamically selects the most important visual evidence for the input text query by (i) scoring visual tokens via cross-attention, (ii) \emph{predicting} an instance-specific retention budget based on the complexity of the query, and (iii) \emph{selecting} Top-$n$ tokens with a differentiable straight-through estimator during training and a hard gate at inference. Furthermore, a small re-encoder preserves temporal order using absolute time information, enabling second-level localization while maintaining global coverage. Integrated into Qwen2.5-VL, QTSplus compresses the vision stream by up to \textbf{89%} and reduces end-to-end latency by \textbf{28%} on long videos. The evaluation on eight long video understanding benchmarks shows near-parity accuracy overall when compared with the original Qwen models and outperforms the original model by \textbf{+20.5} and \textbf{+5.6} points respectively on TempCompass direction and order accuracies. These results show that QTSplus is an effective, general mechanism for scaling MLLMs to real-world long-video scenarios while preserving task-relevant evidence. We will make all code, data, and trained models’ weights publicly available.

Method: Proposed an event-guided diffusion model that maps sparse HDR event features (edges, corners) into diffusion latent space. This provides precise structural guidance during generation to transfer HDR information from events to frames.

Result: Achieved state-of-the-art results on two benchmarks and a newly collected drone dataset in heavy haze (AQI = 341) with synchronized RGB and event sensors.

Conclusion: Event cameras combined with diffusion models effectively address dehazing challenges by leveraging HDR structural cues, improving visual realism and reducing semantic drift in hazy conditions.

Abstract: Clear imaging under hazy conditions is a critical task. Prior-based and neural methods have improved results. However, they operate on RGB frames, which suffer from limited dynamic range. Therefore, dehazing remains ill-posed and can erase structure and illumination details. To address this, we use event cameras for dehazing for the \textbf{first time}. Event cameras offer much higher HDR ($120 dBvs.60 dB$) and microsecond latency, therefore they suit hazy scenes. In practice, transferring HDR cues from events to frames is hard because real paired data are scarce. To tackle this, we propose an event-guided diffusion model that utilizes the strong generative priors of diffusion models to reconstruct clear images from hazy inputs by effectively transferring HDR information from events. Specifically, we design an event-guided module that maps sparse HDR event features, \textit{e.g.,} edges, corners, into the diffusion latent space. This clear conditioning provides precise structural guidance during generation, improves visual realism, and reduces semantic drift. For real-world evaluation, we collect a drone dataset in heavy haze (AQI = 341) with synchronized RGB and event sensors. Experiments on two benchmarks and our dataset achieve state-of-the-art results.

Method: Evaluated three U-Net architectures: baseline BEGL-UNet, Attention-Residual BEGL-UNet with residual blocks and gated attention, and Spatial Channel Attention BEGL-UNet using CBAM modules. All used BEGL loss combining binary cross-entropy with Gaussian edge enhancement. Tested on Poço da Bebidinha Archaeological Complex images with 5-fold cross-validation.

Result: Attention-Residual BEGL-UNet achieved best performance: Dice Score 0.710, validation loss 0.067, recall 0.854. Spatial Channel Attention BEGL-UNet: DSC 0.707, recall 0.857. Baseline BEGL-UNet: DSC 0.690. Attention mechanisms improved DSC by 2.5-2.9% over baseline.

Conclusion: Attention mechanisms significantly enhance semantic segmentation performance for archaeological heritage preservation, with Attention-Residual BEGL-UNet being the most effective architecture for rock art petroglyph segmentation.

Abstract: This study presents a comparative analysis of three U-Net-based architectures for semantic segmentation of rock art petroglyphs from Brazilian archaeological sites. The investigated architectures were: (1) BEGL-UNet with Border-Enhanced Gaussian Loss function; (2) Attention-Residual BEGL-UNet, incorporating residual blocks and gated attention mechanisms; and (3) Spatial Channel Attention BEGL-UNet, which employs spatial-channel attention modules based on Convolutional Block Attention Module. All implementations employed the BEGL loss function combining binary cross-entropy with Gaussian edge enhancement. Experiments were conducted on images from the Poço da Bebidinha Archaeological Complex, Piauí, Brazil, using 5-fold cross-validation. Among the architectures, Attention-Residual BEGL-UNet achieved the best overall performance with Dice Score of 0.710, validation loss of 0.067, and highest recall of 0.854. Spatial Channel Attention BEGL-UNet obtained comparable performance with DSC of 0.707 and recall of 0.857. The baseline BEGL-UNet registered DSC of 0.690. These results demonstrate the effectiveness of attention mechanisms for archaeological heritage digital preservation, with Dice Score improvements of 2.5-2.9% over the baseline.

Method: Model fine-grained glomerular subtyping as few-shot problem; systematically evaluate pathology-specialized and general-purpose VLMs; analyze classification performance and representation geometry including feature alignment and subtype separability.

Result: Pathology-specialized VLMs with vanilla fine-tuning are most effective, achieving substantial gains in discrimination and calibration with only 4-8 examples per subtype. Discrimination between positive/negative examples is as important as image-text alignment.

Conclusion: Supervision level and adaptation strategy jointly shape diagnostic performance and multimodal structure, providing guidance for model selection, adaptation strategies, and annotation investment in clinical settings.

Abstract: Fine-grained glomerular subtyping is central to kidney biopsy interpretation, but clinically valuable labels are scarce and difficult to obtain. Existing computational pathology approaches instead tend to evaluate coarse diseased classification under full supervision with image-only models, so it remains unclear how vision-language models (VLMs) should be adapted for clinically meaningful subtyping under data constraints. In this work, we model fine-grained glomerular subtyping as a clinically realistic few-shot problem and systematically evaluate both pathology-specialized and general-purpose vision-language models under this setting. We assess not only classification performance (accuracy, AUC, F1) but also the geometry of the learned representations, examining feature alignment between image and text embeddings and the separability of glomerular subtypes. By jointly analyzing shot count, model architecture and domain knowledge, and adaptation strategy, this study provides guidance for future model selection and training under real clinical data constraints. Our results indicate that pathology-specialized vision-language backbones, when paired with the vanilla fine-tuning, are the most effective starting point. Even with only 4-8 labeled examples per glomeruli subtype, these models begin to capture distinctions and show substantial gains in discrimination and calibration, though additional supervision continues to yield incremental improvements. We also find that the discrimination between positive and negative examples is as important as image-text alignment. Overall, our results show that supervision level and adaptation strategy jointly shape both diagnostic performance and multimodal structure, providing guidance for model selection, adaptation strategies, and annotation investment.

Method: Proposes Dual-Line Inference pipeline with identity-semantic separation, Identity Adaptive Fusion strategy for deferred fusion, and Identity Aggregation Prepending module to replace random initializations.

Result: Achieves stable and effective performance in IPPG tasks beyond facial close-ups, enabling efficient generation without manual masking or fine-tuning.

Conclusion: The method breaks facial close-up constraints, facilitates film-level character-scene creation, and provides richer personalized generation capabilities as a plug-and-play component.

Abstract: Identity-Preserving Personalized Generation (IPPG) has advanced film production and artistic creation, yet existing approaches overemphasize facial regions, resulting in outputs dominated by facial close-ups.These methods suffer from weak visual narrativity and poor semantic consistency under complex text prompts, with the core limitation rooted in identity (ID) feature embeddings undermining the semantic expressiveness of generative models. To address these issues, this paper presents an IPPG method that breaks the constraint of facial close-ups, achieving synergistic optimization of identity fidelity and scene semantic creation. Specifically, we design a Dual-Line Inference (DLI) pipeline with identity-semantic separation, resolving the representation conflict between ID and semantics inherent in traditional single-path architectures. Further, we propose an Identity Adaptive Fusion (IdAF) strategy that defers ID-semantic fusion to the noise prediction stage, integrating adaptive attention fusion and noise decision masking to avoid ID embedding interference on semantics without manual masking. Finally, an Identity Aggregation Prepending (IdAP) module is introduced to aggregate ID information and replace random initializations, further enhancing identity preservation. Experimental results validate that our method achieves stable and effective performance in IPPG tasks beyond facial close-ups, enabling efficient generation without manual masking or fine-tuning. As a plug-and-play component, it can be rapidly deployed in existing IPPG frameworks, addressing the over-reliance on facial close-ups, facilitating film-level character-scene creation, and providing richer personalized generation capabilities for related domains.

Method: The authors conduct empirical studies revealing three dynamic patterns of transferability, propose a Concentric Decay Model (CDM) to explain these patterns, and develop Dynamic Parameter Optimization (DPO) based on the rise-then-fall pattern.

Result: Comprehensive experiments show that DPO significantly improves transferability across different surrogate models, iterations, and tasks while reducing computational complexity.

Conclusion: The proposed DPO method effectively addresses the limitations of existing transformation-based attacks by providing an efficient parameter optimization approach that enhances transferability with reduced computational overhead.

Abstract: Despite their wide application, the vulnerabilities of deep neural networks raise societal concerns. Among them, transformation-based attacks have demonstrated notable success in transfer attacks. However, existing attacks suffer from blind spots in parameter optimization, limiting their full potential. Specifically, (1) prior work generally considers low-iteration settings, yet attacks perform quite differently at higher iterations, so characterizing overall performance based only on low-iteration results is misleading. (2) Existing attacks use uniform parameters for different surrogate models, iterations, and tasks, which greatly impairs transferability. (3) Traditional transformation parameter optimization relies on grid search. For n parameters with m steps each, the complexity is O(mn). Large computational overhead limits further optimization of parameters. To address these limitations, we conduct an empirical study with various transformations as baselines, revealing three dynamic patterns of transferability with respect to parameter strength. We further propose a novel Concentric Decay Model (CDM) to effectively explain these patterns. Building on these insights, we propose an efficient Dynamic Parameter Optimization (DPO) based on the rise-then-fall pattern, reducing the complexity to O(nlogm). Comprehensive experiments on existing transformation-based attacks across different surrogate models, iterations, and tasks demonstrate that our DPO can significantly improve transferability.

Method: Coarse stage: Human-in-the-Loop Bootstrapping with SAM for robust segmentation. Fine stage: Convert 2D segmentation to 1D regression by sampling groove-normal profiles and refining with lightweight MLP.

Result: Outperforms previous approaches in segmentation accuracy and metrology precision while requiring less supervision.

Conclusion: LithoSeg offers promising prospects for real-world semiconductor manufacturing applications with improved precision and reduced supervision requirements.

Abstract: Accurate segmentation and measurement of lithography scanning electron microscope (SEM) images are crucial for ensuring precise process control, optimizing device performance, and advancing semiconductor manufacturing yield. Lithography segmentation requires pixel-level delineation of groove contours and consistent performance across diverse pattern geometries and process window. However, existing methods often lack the necessary precision and robustness, limiting their practical applicability. To overcome this challenge, we propose LithoSeg, a coarse-to-fine network tailored for lithography segmentation. In the coarse stage, we introduce a Human-in-the-Loop Bootstrapping scheme for the Segment Anything Model (SAM) to attain robustness with minimal supervision. In the subsequent fine stage, we recast 2D segmentation as 1D regression problem by sampling groove-normal profiles using the coarse mask and performing point-wise refinement with a lightweight MLP. LithoSeg outperforms previous approaches in both segmentation accuracy and metrology precision while requiring less supervision, offering promising prospects for real-world applications.

Method: SIT-ADDA-Auto integrates shallow-layer adversarial alignment with predictive uncertainty to automatically select adaptation depth, adapting only the earliest convolutional layers while freezing deeper layers without requiring target labels.

Result: The method improves reconstruction and downstream segmentation across exposure/illumination shifts, cross-instrument transfer, and multiple stains, outperforming full-encoder adaptation and non-adversarial baselines with reduced semantic feature drift.

Conclusion: Provides a design rule for label-free adaptation in microscopy and a practical recipe for field settings, demonstrating that adapting only early layers yields reliable transfer while preserving semantic representations.

Abstract: Deep learning is transforming microscopy, yet models often fail when applied to images from new instruments or acquisition settings. Conventional adversarial domain adaptation (ADDA) retrains entire networks, often disrupting learned semantic representations. Here, we overturn this paradigm by showing that adapting only the earliest convolutional layers, while freezing deeper layers, yields reliable transfer. Building on this principle, we introduce Subnetwork Image Translation ADDA with automatic depth selection (SIT-ADDA-Auto), a self-configuring framework that integrates shallow-layer adversarial alignment with predictive uncertainty to automatically select adaptation depth without target labels. We demonstrate robustness via multi-metric evaluation, blinded expert assessment, and uncertainty-depth ablations. Across exposure and illumination shifts, cross-instrument transfer, and multiple stains, SIT-ADDA improves reconstruction and downstream segmentation over full-encoder adaptation and non-adversarial baselines, with reduced drift of semantic features. Our results provide a design rule for label-free adaptation in microscopy and a recipe for field settings; the code is publicly available.

Method: Four synchronized cameras with YOLOv11 segmentation for vehicle detection, homography transformation to bird’s-eye view, and novel pixel-level PET algorithm without fixed cells, running on NVIDIA Jetson AGX Xavier edge devices.

Result: The framework identifies high-risk regions with sub-second precision, achieves 2.68 FPS for 800x800 pixel heatmaps, and provides fine-grained hazard visualization accurate to 3.3 sq-cm.

Conclusion: Validates feasibility of decentralized vision-based PET analysis for intelligent transportation systems, offering replicable methodology for high-resolution, real-time, and scalable intersection safety evaluation.

Abstract: Traffic safety analysis at signalized intersections is vital for reducing vehicle and pedestrian collisions, yet traditional crash-based studies are limited by data sparsity and latency. This paper presents a novel multi-camera computer vision framework for real-time safety assessment through Post-Encroachment Time (PET) computation, demonstrated at the intersection of H Street and Broadway in Chula Vista, California. Four synchronized cameras provide continuous visual coverage, with each frame processed on NVIDIA Jetson AGX Xavier devices using YOLOv11 segmentation for vehicle detection. Detected vehicle polygons are transformed into a unified bird’s-eye map using homography matrices, enabling alignment across overlapping camera views. A novel pixel-level PET algorithm measures vehicle position without reliance on fixed cells, allowing fine-grained hazard visualization via dynamic heatmaps, accurate to 3.3 sq-cm. Timestamped vehicle and PET data is stored in an SQL database for long-term monitoring. Results over various time intervals demonstrate the framework’s ability to identify high-risk regions with sub-second precision and real-time throughput on edge devices, producing data for an 800 x 800 pixel logarithmic heatmap at an average of 2.68 FPS. This study validates the feasibility of decentralized vision-based PET analysis for intelligent transportation systems, offering a replicable methodology for high-resolution, real-time, and scalable intersection safety evaluation.

Method: Two-stage LIHE framework: 1) Referential Decoupling predicts target count and decomposes complex expressions into sub-expressions, 2) Referent Grounding localizes sub-expressions using HEMix hybrid similarity module combining Euclidean proximity and hyperbolic geometry.

Result: LIHE establishes the first effective weakly supervised WGREC baseline on gRefCOCO and Ref-ZOM datasets, while HEMix improves IoU@0.5 by up to 2.5% on standard REC benchmarks.

Conclusion: The proposed LIHE framework successfully addresses the challenges of WGREC by handling variable referent counts and preventing semantic collapse through hybrid Euclidean-hyperbolic similarity modeling.

Abstract: Existing Weakly-Supervised Referring Expression Comprehension (WREC) methods, while effective, are fundamentally limited by a one-to-one mapping assumption, hindering their ability to handle expressions corresponding to zero or multiple targets in realistic scenarios. To bridge this gap, we introduce the Weakly-Supervised Generalized Referring Expression Comprehension task (WGREC), a more practical paradigm that handles expressions with variable numbers of referents. However, extending WREC to WGREC presents two fundamental challenges: supervisory signal ambiguity, where weak image-level supervision is insufficient for training a model to infer the correct number and identity of referents, and semantic representation collapse, where standard Euclidean similarity forces hierarchically-related concepts into non-discriminative clusters, blurring categorical boundaries. To tackle these challenges, we propose a novel WGREC framework named Linguistic Instance-Split Hyperbolic-Euclidean (LIHE), which operates in two stages. The first stage, Referential Decoupling, predicts the number of target objects and decomposes the complex expression into simpler sub-expressions. The second stage, Referent Grounding, then localizes these sub-expressions using HEMix, our innovative hybrid similarity module that synergistically combines the precise alignment capabilities of Euclidean proximity with the hierarchical modeling strengths of hyperbolic geometry. This hybrid approach effectively prevents semantic collapse while preserving fine-grained distinctions between related concepts. Extensive experiments demonstrate LIHE establishes the first effective weakly supervised WGREC baseline on gRefCOCO and Ref-ZOM, while HEMix achieves consistent improvements on standard REC benchmarks, improving IoU@0.5 by up to 2.5%. The code is available at https://anonymous.4open.science/r/LIHE.

Method: Introduces Null-Space Diffusion Distillation (NSDD), a single-pass student model that distills the null-space component of iterative DDNM+ solver, conditioned on lensless measurement and range-space anchor.

Result: NSDD achieves near-teacher perceptual quality (second-best LPIPS), outperforms DPS and classical baselines, and is the second fastest method behind Wiener while preserving measurement consistency.

Conclusion: NSDD provides a practical path toward fast, ground-truth-free, photorealistic lensless imaging by separating range-space enforcement from null-space diffusion-prior updates.

Abstract: State-of-the-art photorealistic reconstructions for lensless cameras often rely on paired lensless-lensed supervision, which can bias models due to lens-lensless domain mismatch. To avoid this, ground-truth-free diffusion priors are attractive; however, generic formulations tuned for conventional inverse problems often break under the noisy, highly multiplexed, and ill-posed lensless deconvolution setting. We observe that methods which separate range-space enforcement from null-space diffusion-prior updates yield stable, realistic reconstructions. Building on this, we introduce Null-Space Diffusion Distillation (NSDD): a single-pass student that distills the null-space component of an iterative DDNM+ solver, conditioned on the lensless measurement and on a range-space anchor. NSDD preserves measurement consistency and achieves photorealistic results without paired supervision at a fraction of the runtime and memory. On Lensless-FFHQ and PhlatCam, NSDD is the second fastest, behind Wiener, and achieves near-teacher perceptual quality (second-best LPIPS, below DDNM+), outperforming DPS and classical convex baselines. These results suggest a practical path toward fast, ground-truth-free, photorealistic lensless imaging.

Method: Created VL-SurgPT dataset with 754 tissue tracking clips (17,171 points) and 154 instrument tracking clips (7 instrument types), established benchmarks with 8 tracking methods, and proposed TG-SurgPT - a text-guided approach that leverages semantic descriptions.

Result: Incorporating point status information significantly improves tracking accuracy and reliability, especially in adverse visual scenarios where conventional vision-only methods struggle.

Conclusion: Bridging visual and linguistic modalities enables development of context-aware tracking systems crucial for advancing computer-assisted surgery applications that maintain performance under challenging intraoperative conditions.

Abstract: Accurate point tracking in surgical environments remains challenging due to complex visual conditions, including smoke occlusion, specular reflections, and tissue deformation. While existing surgical tracking datasets provide coordinate information, they lack the semantic context necessary to understand tracking failure mechanisms. We introduce VL-SurgPT, the first large-scale multimodal dataset that bridges visual tracking with textual descriptions of point status in surgical scenes. The dataset comprises 908 in vivo video clips, including 754 for tissue tracking (17,171 annotated points across five challenging scenarios) and 154 for instrument tracking (covering seven instrument types with detailed keypoint annotations). We establish comprehensive benchmarks using eight state-of-the-art tracking methods and propose TG-SurgPT, a text-guided tracking approach that leverages semantic descriptions to improve robustness in visually challenging conditions. Experimental results demonstrate that incorporating point status information significantly improves tracking accuracy and reliability, particularly in adverse visual scenarios where conventional vision-only methods struggle. By bridging visual and linguistic modalities, VL-SurgPT enables the development of context-aware tracking systems crucial for advancing computer-assisted surgery applications that can maintain performance even under challenging intraoperative conditions.

Method: Proposes GCAgent framework with Schematic and Narrative Episodic Memory that structurally models events and their causal/temporal relations. Operates in Perception-Action-Reflection cycle using Memory Manager for context-aware inference.

Result: Achieves up to 23.5% accuracy improvement on Video-MME Long split over baseline, 73.4% accuracy on Long split, and highest overall average (71.9%) on Video-MME benchmark among 7B-scale MLLMs.

Conclusion: The agent-based reasoning paradigm with structured memory effectively enables cognitively-inspired long-video understanding, establishing new state-of-the-art performance.

Abstract: Long-video understanding remains a significant challenge for Multimodal Large Language Models (MLLMs) due to inherent token limitations and the complexity of capturing long-term temporal dependencies. Existing methods often fail to capture the global context and complex event relationships necessary for deep video reasoning. To address this, we introduce GCAgent, a novel Global-Context-Aware Agent framework that achieves comprehensive long-video understanding. Our core innovation is the Schematic and Narrative Episodic Memory. This memory structurally models events and their causal and temporal relations into a concise, organized context, fundamentally resolving the long-term dependency problem. Operating in a multi-stage Perception-Action-Reflection cycle, our GCAgent utilizes a Memory Manager to retrieve relevant episodic context for robust, context-aware inference. Extensive experiments confirm that GCAgent significantly enhances long-video understanding, achieving up to 23.5% accuracy improvement on the Video-MME Long split over a strong MLLM baseline. Furthermore, our framework establishes state-of-the-art performance among comparable 7B-scale MLLMs, achieving 73.4% accuracy on the Long split and the highest overall average (71.9%) on the Video-MME benchmark, validating our agent-based reasoning paradigm and structured memory for cognitively-inspired long-video understanding.

Method: Two key components: 1) Joint visual-physical cue learning for comprehensive representation learning, 2) Candidate pose aggregation that refines multiple diffusion-generated poses using both visual and physical predictions.

Result: Significantly outperforms state-of-the-art approaches in both pose accuracy and physical plausibility through extensive experiments.

Conclusion: The proposed framework successfully achieves visually consistent and physically plausible hand-object pose estimation by integrating visual and physical cues in an end-to-end trainable manner.

Abstract: Estimating the 3D poses of hands and objects from a single RGB image is a fundamental yet challenging problem, with broad applications in augmented reality and human-computer interaction. Existing methods largely rely on visual cues alone, often producing results that violate physical constraints such as interpenetration or non-contact. Recent efforts to incorporate physics reasoning typically depend on post-optimization or non-differentiable physics engines, which compromise visual consistency and end-to-end trainability. To overcome these limitations, we propose a novel framework that jointly integrates visual and physical cues for hand-object pose estimation. This integration is achieved through two key ideas: 1) joint visual-physical cue learning: The model is trained to extract 2D visual cues and 3D physical cues, thereby enabling more comprehensive representation learning for hand-object interactions; 2) candidate pose aggregation: A novel refinement process that aggregates multiple diffusion-generated candidate poses by leveraging both visual and physical predictions, yielding a final estimate that is visually consistent and physically plausible. Extensive experiments demonstrate that our method significantly outperforms existing state-of-the-art approaches in both pose accuracy and physical plausibility.

Method: Proposes KA-MIG framework with three knowledge graphs (co-occurrence, semantic similarity, position-token incompatibility), a graph-aware encoder for token/position representations, and lightweight fusion with existing MIG methods.

Result: Experimental results show improved performance for class-conditional image generation on ImageNet compared to existing MIG methods.

Conclusion: Incorporating explicit token-level semantic dependency knowledge as priors effectively enhances the model’s ability to capture semantic dependencies and improves generation quality in masked image generation.

Abstract: Masked image generation (MIG) has demonstrated remarkable efficiency and high-fidelity images by enabling parallel token prediction. Existing methods typically rely solely on the model itself to learn semantic dependencies among visual token sequences. However, directly learning such semantic dependencies from data is challenging because the individual tokens lack clear semantic meanings, and these sequences are usually long. To address this limitation, we propose a novel Knowledge-Augmented Masked Image Generation framework, named KA-MIG, which introduces explicit knowledge of token-level semantic dependencies (\emph{i.e.}, extracted from the training data) as priors to learn richer representations for improving performance. In particular, we explore and identify three types of advantageous token knowledge graphs, including two positive and one negative graphs (\emph{i.e.}, the co-occurrence graph, the semantic similarity graph, and the position-token incompatibility graph). Based on three prior knowledge graphs, we design a graph-aware encoder to learn token and position-aware representations. After that, a lightweight fusion mechanism is introduced to integrate these enriched representations into the existing MIG methods. Resorting to such prior knowledge, our method effectively enhances the model’s ability to capture semantic dependencies, leading to improved generation quality. Experimental results demonstrate that our method improves upon existing MIG for class-conditional image generation on ImageNet.

Method: Uses MLLMs and diffusion models to generate scene-specific reference gallery, integrates external information via Reference-Guided Feature Enhancement module, trains decoder to predict Gaussian primitives, and refines them with Texture-Aware Density Control based on internal texture richness.

Result: Outperforms existing methods on RealEstate10K, ACID, and DTU datasets, demonstrating strong cross-dataset and cross-resolution generalization capabilities.

Conclusion: SRSplat effectively addresses the texture detail recovery problem in 3D reconstruction from sparse LR images through joint external reference and internal texture optimization.

Abstract: Feed-forward 3D reconstruction from sparse, low-resolution (LR) images is a crucial capability for real-world applications, such as autonomous driving and embodied AI. However, existing methods often fail to recover fine texture details. This limitation stems from the inherent lack of high-frequency information in LR inputs. To address this, we propose \textbf{SRSplat}, a feed-forward framework that reconstructs high-resolution 3D scenes from only a few LR views. Our main insight is to compensate for the deficiency of texture information by jointly leveraging external high-quality reference images and internal texture cues. We first construct a scene-specific reference gallery, generated for each scene using Multimodal Large Language Models (MLLMs) and diffusion models. To integrate this external information, we introduce the \textit{Reference-Guided Feature Enhancement (RGFE)} module, which aligns and fuses features from the LR input images and their reference twin image. Subsequently, we train a decoder to predict the Gaussian primitives using the multi-view fused feature obtained from \textit{RGFE}. To further refine predicted Gaussian primitives, we introduce \textit{Texture-Aware Density Control (TADC)}, which adaptively adjusts Gaussian density based on the internal texture richness of the LR inputs. Extensive experiments demonstrate that our SRSplat outperforms existing methods on various datasets, including RealEstate10K, ACID, and DTU, and exhibits strong cross-dataset and cross-resolution generalization capabilities.

Method: Proposed Federated Stain Distribution Alignment (FedSDA) that uses diffusion models and stain separation to align client stain distributions with a target distribution, while circumventing privacy risks by avoiding training diffusion models on raw data.

Result: FedSDA effectively improves baseline methods that focus on mitigating model update disparities and outperforms other non-IID data distribution approaches. Extensive experiments demonstrate its effectiveness.

Conclusion: FedSDA provides valuable practical insights for computational pathology by successfully addressing non-IID data issues through stain distribution alignment in federated learning frameworks.

Abstract: Federated learning (FL) has shown success in collaboratively training a model among decentralized data resources without directly sharing privacy-sensitive training data. Despite recent advances, non-IID (non-independent and identically distributed) data poses an inevitable challenge that hinders the use of FL. In this work, we address the issue of non-IID histopathological images with feature distribution shifts from an intuitive perspective that has only received limited attention. Specifically, we address this issue from the perspective of data distribution by solely adjusting the data distributions of all clients. Building on the success of diffusion models in fitting data distributions and leveraging stain separation to extract the pivotal features that are closely related to the non-IID properties of histopathological images, we propose a Federated Stain Distribution Alignment (FedSDA) method. FedSDA aligns the stain distribution of each client with a target distribution in an FL framework to mitigate distribution shifts among clients. Furthermore, considering that training diffusion models on raw data in FL has been shown to be susceptible to privacy leakage risks, we circumvent this problem while still effectively achieving alignment. Extensive experimental results show that FedSDA is not only effective in improving baselines that focus on mitigating disparities across clients’ model updates but also outperforms baselines that address the non-IID data issues from the perspective of data distribution. We show that FedSDA provides valuable and practical insights for the computational pathology community.

Method: Propose DCMM-Transformer that uses a Degree-Corrected Mixed-Membership model as an additive bias in self-attention, enabling differentiable and interpretable incorporation of community structure and degree heterogeneity without binary sampling or multiplicative masking.

Result: Comprehensive experiments across diverse medical imaging datasets (brain, chest, breast, ocular) demonstrate superior performance and generalizability. Learned group structure produces anatomically meaningful and semantically coherent attention maps.

Conclusion: DCMM-Transformer effectively incorporates anatomical structure in medical image analysis through a differentiable approach, achieving better performance while enhancing interpretability with structured attention modulation.

Abstract: Medical images exhibit latent anatomical groupings, such as organs, tissues, and pathological regions, that standard Vision Transformers (ViTs) fail to exploit. While recent work like SBM-Transformer attempts to incorporate such structures through stochastic binary masking, they suffer from non-differentiability, training instability, and the inability to model complex community structure. We present DCMM-Transformer, a novel ViT architecture for medical image analysis that incorporates a Degree-Corrected Mixed-Membership (DCMM) model as an additive bias in self-attention. Unlike prior approaches that rely on multiplicative masking and binary sampling, our method introduces community structure and degree heterogeneity in a fully differentiable and interpretable manner. Comprehensive experiments across diverse medical imaging datasets, including brain, chest, breast, and ocular modalities, demonstrate the superior performance and generalizability of the proposed approach. Furthermore, the learned group structure and structured attention modulation substantially enhance interpretability by yielding attention maps that are anatomically meaningful and semantically coherent.

Method: Uses attention-based module with Spatial Feature Transform (SFT) modulation to align high-resolution queries from low-level features with semantically enriched low-resolution keys, trained at low upsampling ratios.

Result: Effectively recovers fine-grained spatial details and consistently outperforms existing feature upsampling methods across diverse downstream tasks.

Conclusion: JAFAR provides a flexible and effective solution for feature upsampling that generalizes well from low to high scales without requiring high-resolution supervision.

Abstract: Foundation Vision Encoders have become essential for a wide range of dense vision tasks. However, their low-resolution spatial feature outputs necessitate feature upsampling to produce the high-resolution modalities required for downstream tasks. In this work, we introduce JAFAR, a lightweight and flexible feature upsampler that enhances the spatial resolution of visual features from any Foundation Vision Encoder to an arbitrary target resolution. JAFAR employs an attention-based module designed to promote semantic alignment between high-resolution queries, derived from low-level image features, and semantically enriched low-resolution keys, using Spatial Feature Transform (SFT) modulation. Notably, despite the absence of high-resolution supervision, we demonstrate that learning at low upsampling ratios and resolutions generalizes remarkably well to significantly higher output scales. Extensive experiments show that JAFAR effectively recovers fine-grained spatial details and consistently outperforms existing feature upsampling methods across a diverse set of downstream tasks. Project page at https://jafar-upsampler.github.io

Method: Two-stage progressive training: initial transfer learning with standard augmentations followed by fine-tuning with advanced affine and deepfake-specific augmentations, using DeiT’s knowledge distillation to capture subtle manipulation artifacts.

Result: Achieved 98.71% accuracy after stage one and 99.22% accuracy with AUROC of 0.9997 after stage two, outperforming latest OpenForensics baselines on 190,335 images.

Conclusion: The approach provides practical benchmarks for facial deepfake detection and demonstrates the effectiveness of progressive training with specialized augmentations.

Abstract: Deepfakes are major threats to the integrity of digital media. We propose DeiTFake, a DeiT-based transformer and a novel two-stage progressive training strategy with increasing augmentation complexity. The approach applies an initial transfer-learning phase with standard augmentations followed by a fine-tuning phase using advanced affine and deepfake-specific augmentations. DeiT’s knowledge distillation model captures subtle manipulation artifacts, increasing robustness of the detection model. Trained on the OpenForensics dataset (190,335 images), DeiTFake achieves 98.71% accuracy after stage one and 99.22% accuracy with an AUROC of 0.9997, after stage two, outperforming the latest OpenForensics baselines. We analyze augmentation impact and training schedules, and provide practical benchmarks for facial deepfake detection.

Method: Two-stage approach: 1) View-Aware Adversarial Bridging (VAAB) for view-invariant features and robust pseudo-labels, 2) Heterogeneous Graph Filtering Calibration (HGFC) with dual inter-view structure graphs for reliable view correspondence.

Result: Achieves state-of-the-art unsupervised performance: +10.63% AP on University-1652 and +16.73% AP on SUES-200 for Satellite→Drone matching, even surpassing supervised baselines.

Conclusion: UniABG effectively addresses unsupervised CVGL challenges through adversarial bridging and graph-based calibration, demonstrating strong performance without annotation requirements.

Abstract: Cross-view geo-localization (CVGL) matches query images ($\textit{e.g.}$, drone) to geographically corresponding opposite-view imagery ($\textit{e.g.}$, satellite). While supervised methods achieve strong performance, their reliance on extensive pairwise annotations limits scalability. Unsupervised alternatives avoid annotation costs but suffer from noisy pseudo-labels due to intrinsic cross-view domain gaps. To address these limitations, we propose $\textit{UniABG}$, a novel dual-stage unsupervised cross-view geo-localization framework integrating adversarial view bridging with graph-based correspondence calibration. Our approach first employs View-Aware Adversarial Bridging (VAAB) to model view-invariant features and enhance pseudo-label robustness. Subsequently, Heterogeneous Graph Filtering Calibration (HGFC) refines cross-view associations by constructing dual inter-view structure graphs, achieving reliable view correspondence. Extensive experiments demonstrate state-of-the-art unsupervised performance, showing that UniABG improves Satellite $\rightarrow$ Drone AP by +10.63% on University-1652 and +16.73% on SUES-200, even surpassing supervised baselines. The source code is available at https://github.com/chenqi142/UniABG

Method: Three main innovations: 1) PipeSP for pipelined sequence parallelism across GPUs, 2) DeDiVAE to decouple diffusion and VAE modules into separate GPU groups for pipelined execution, 3) Attention co-processing (Aco) to better utilize VAE group GPU resources.

Result: Integrated into OpenSoraPlan and HunyuanVideo frameworks, PipeDiT achieves 1.06x to 4.02x speedups over baseline systems across various resolution and timestep configurations on 8-GPU systems.

Conclusion: PipeDiT effectively addresses the inference latency and memory consumption challenges in DiT-based video generation, enabling more practical deployment of state-of-the-art video generation models.

Abstract: Video generation has been advancing rapidly, and diffusion transformer (DiT) based models have demonstrated remark- able capabilities. However, their practical deployment is of- ten hindered by slow inference speeds and high memory con- sumption. In this paper, we propose a novel pipelining frame- work named PipeDiT to accelerate video generation, which is equipped with three main innovations. First, we design a pipelining algorithm (PipeSP) for sequence parallelism (SP) to enable the computation of latent generation and commu- nication among multiple GPUs to be pipelined, thus reduc- ing inference latency. Second, we propose DeDiVAE to de- couple the diffusion module and the variational autoencoder (VAE) module into two GPU groups, whose executions can also be pipelined to reduce memory consumption and infer- ence latency. Third, to better utilize the GPU resources in the VAE group, we propose an attention co-processing (Aco) method to further reduce the overall video generation latency. We integrate our PipeDiT into both OpenSoraPlan and Hun- yuanVideo, two state-of-the-art open-source video generation frameworks, and conduct extensive experiments on two 8- GPU systems. Experimental results show that, under many common resolution and timestep configurations, our PipeDiT achieves 1.06x to 4.02x speedups over OpenSoraPlan and HunyuanVideo.

Method: Transforms GPS trajectories into movement-semantics features, segments them into patches, uses intra- and inter-patch attentions to encode local and global patterns, and employs curvature-guided augmentation that preserves informative segments while masking redundant ones.

Result: Outperforms state-of-the-art methods with mean ranks close to ideal value of 1 at similarity search tasks, improvements up to 20.3% at heuristic approximation, and reduces inference latency by up to 43.4%.

Conclusion: MovSemCL effectively addresses key limitations in trajectory similarity computation through hierarchical representation, efficient encoding, and physically plausible augmentations, achieving superior performance with reduced computational costs.

Abstract: Trajectory similarity computation is fundamental functionality that is used for, e.g., clustering, prediction, and anomaly detection. However, existing learning-based methods exhibit three key limitations: (1) insufficient modeling of trajectory semantics and hierarchy, lacking both movement dynamics extraction and multi-scale structural representation; (2) high computational costs due to point-wise encoding; and (3) use of physically implausible augmentations that distort trajectory semantics. To address these issues, we propose MovSemCL, a movement-semantics contrastive learning framework for trajectory similarity computation. MovSemCL first transforms raw GPS trajectories into movement-semantics features and then segments them into patches. Next, MovSemCL employs intra- and inter-patch attentions to encode local as well as global trajectory patterns, enabling efficient hierarchical representation and reducing computational costs. Moreover, MovSemCL includes a curvature-guided augmentation strategy that preserves informative segments (e.g., turns and intersections) and masks redundant ones, generating physically plausible augmented views. Experiments on real-world datasets show that MovSemCL is capable of outperforming state-of-the-art methods, achieving mean ranks close to the ideal value of 1 at similarity search tasks and improvements by up to 20.3% at heuristic approximation, while reducing inference latency by up to 43.4%.

Method: Two-stage framework: 1) VG-Align distills visual pathway into audio pathway by aligning token distributions, 2) EmoAdapter with Emotion Enhancer and Supervisor injects emotion-sensitive residuals and applies emotion supervision.

Result: Achieves state-of-the-art results on ArtEmoBenchmark, outperforming audio-only, visual-only, and audio-visual baselines. Ablations show proposed components are complementary.

Conclusion: The framework enables efficient audio-visual emotion understanding with limited audio pretraining, demonstrating effectiveness for cross-modal emotion comprehension in artistic contexts.

Abstract: Emotion understanding is critical for making Large Language Models (LLMs) more general, reliable, and aligned with humans. Art conveys emotion through the joint design of visual and auditory elements, yet most prior work is human-centered or single-modality, overlooking the emotion intentionally expressed by the artwork. Meanwhile, current Audio-Visual Language Models (AVLMs) typically require large-scale audio pretraining to endow Visual Language Models (VLMs) with hearing, which limits scalability. We present Vision Anchored Audio-Visual Emotion LLM (VAEmotionLLM), a two-stage framework that teaches a VLM to hear by seeing with limited audio pretraining and to understand emotion across modalities. In Stage 1, Vision-Guided Audio Alignment (VG-Align) distills the frozen visual pathway into a new audio pathway by aligning next-token distributions of the shared LLM on synchronized audio-video clips, enabling hearing without a large audio dataset. In Stage 2, a lightweight Cross-Modal Emotion Adapter (EmoAdapter), composed of the Emotion Enhancer and the Emotion Supervisor, injects emotion-sensitive residuals and applies emotion supervision to enhance cross-modal emotion understanding. We also construct ArtEmoBenchmark, an art-centric emotion benchmark that evaluates content and emotion understanding under audio-only, visual-only, and audio-visual inputs. VAEmotionLLM achieves state-of-the-art results on ArtEmoBenchmark, outperforming audio-only, visual-only, and audio-visual baselines. Ablations show that the proposed components are complementary.

Method: Uses text embeddings as prototype priors, multimodal prompts for adaptive refinement, dual-constrained quantization space with compactness/separation regularization, and Gumbel-Softmax for differentiable discretization.

Result: Superior performance demonstrated on ModelNet40 and ScanObjectNN datasets, showing effectiveness in encoding both geometric and semantic information.

Conclusion: The proposed framework successfully addresses limitations of current quantization methods by leveraging multimodal alignment and adaptive prompting, achieving robust hybrid representations for point cloud analysis.

Abstract: Vector quantization has emerged as a powerful tool in large-scale multimodal models, unifying heterogeneous representations through discrete token encoding. However, its effectiveness hinges on robust codebook design. Current prototype-based approaches relying on trainable vectors or clustered centroids fall short in representativeness and interpretability, even as multimodal alignment demonstrates its promise in vision-language models. To address these limitations, we propose a simple multimodal prompting-driven quantization framework for point cloud analysis. Our methodology is built upon two core insights: 1) Text embeddings from pre-trained models inherently encode visual semantics through many-to-one contrastive alignment, naturally serving as robust prototype priors; and 2) Multimodal prompts enable adaptive refinement of these prototypes, effectively mitigating vision-language semantic gaps. The framework introduces a dual-constrained quantization space, enforced by compactness and separation regularization, which seamlessly integrates visual and prototype features, resulting in hybrid representations that jointly encode geometric and semantic information. Furthermore, we employ Gumbel-Softmax relaxation to achieve differentiable discretization while maintaining quantization sparsity. Extensive experiments on the ModelNet40 and ScanObjectNN datasets clearly demonstrate the superior effectiveness of the proposed method.

Method: Modified ResNet-101 architecture integrated with probabilistic reasoning to simulate label dependencies and uncertainties in multilabel scenarios.

Result: Achieved 0.794 mAP on COCO-2014, outperforming ResNet-SRN (0.771) and Vision Transformer baselines (0.785), with improved precision-recall scores.

Conclusion: Integrating probabilistic reasoning into deep learning models effectively addresses multilabel classification challenges and achieves state-of-the-art performance.

Abstract: Multilabel image categorization has drawn interest recently because of its numerous computer vision applications. The proposed work introduces a novel method for classifying multilabel images using the COCO-2014 dataset and a modified ResNet-101 architecture. By simulating label dependencies and uncertainties, the approach uses probabilistic reasoning to improve prediction accuracy. Extensive tests show that the model outperforms earlier techniques and approaches to state-of-the-art outcomes in multilabel categorization. The work also thoroughly assesses the model’s performance using metrics like precision-recall score and achieves 0.794 mAP on COCO-2014, outperforming ResNet-SRN (0.771) and Vision Transformer baselines (0.785). The novelty of the work lies in integrating probabilistic reasoning into deep learning models to effectively address the challenges presented by multilabel scenarios.

Method: Introduces SemanticStitch framework with semantic priors for foreground objects and a novel loss function that emphasizes semantic integrity of salient objects to enhance stitching quality.

Result: Experimental results show substantial improvements over traditional techniques, with the method providing robust support for practical applications as validated on two specialized real-world datasets.

Conclusion: SemanticStitch effectively addresses semantic disruption issues in image stitching by incorporating semantic awareness, significantly improving visual coherence and foreground object preservation compared to traditional approaches.

Abstract: Image stitching often faces challenges due to varying capture angles, positional differences, and object movements, leading to misalignments and visual discrepancies. Traditional seam carving methods neglect semantic information, causing disruptions in foreground continuity. We introduce SemanticStitch, a deep learning-based framework that incorporates semantic priors of foreground objects to preserve their integrity and enhance visual coherence. Our approach includes a novel loss function that emphasizes the semantic integrity of salient objects, significantly improving stitching quality. We also present two specialized real-world datasets to evaluate our method’s effectiveness. Experimental results demonstrate substantial improvements over traditional techniques, providing robust support for practical applications.

Method: Hierarchical layer-grouped prompt tuning where: (i) layers in same group share similar prompts adjusted by position encoding to preserve pre-trained model relationships, (ii) single task-specific root prompt generates sub-prompts for each layer group to enhance synergy and reduce independence.

Result: Extensive experiments across four benchmarks show favorable performance compared to several state-of-the-art continual learning methods.

Conclusion: The proposed hierarchical layer-grouped prompt tuning method effectively improves model stability and reduces catastrophic forgetting in continual learning scenarios.

Abstract: Prompt-based continual learning methods fine-tune only a small set of additional learnable parameters while keeping the pre-trained model’s parameters frozen. It enables efficient adaptation to new tasks while mitigating the risk of catastrophic forgetting. These methods typically attach one independent task-specific prompt to each layer of pre-trained models to locally modulate its features, ensuring that the layer’s representation aligns with the requirements of the new task. However, although introducing learnable prompts independently at each layer provides high flexibility for adapting to new tasks, this overly flexible tuning could make certain layers susceptible to unnecessary updates. As all prompts till the current task are added together as a final prompt for all seen tasks, the model may easily overwrite feature representations essential to previous tasks, which increases the risk of catastrophic forgetting. To address this issue, we propose a novel hierarchical layer-grouped prompt tuning method for continual learning. It improves model stability in two ways: (i) Layers in the same group share roughly the same prompts, which are adjusted by position encoding. This helps preserve the intrinsic feature relationships and propagation pathways of the pre-trained model within each group. (ii) It utilizes a single task-specific root prompt to learn to generate sub-prompts for each layer group. In this way, all sub-prompts are conditioned on the same root prompt, enhancing their synergy and reducing independence. Extensive experiments across four benchmarks demonstrate that our method achieves favorable performance compared with several state-of-the-art methods.

Method: PACE uses two core modules: ST-DSM (which densifies spike-based features and performs spatiotemporal matching of amplitude and phase) and PEQ-N (a plug-and-play probabilistic integer quantizer compatible with standard event-frame pipelines).

Result: PACE outperforms existing methods on DVS-Gesture, CIFAR10-DVS, and N-MNIST datasets, achieving 84.4% accuracy on N-MNIST (85% of full training set performance) while reducing training time by 50× and storage cost by 6000×.

Conclusion: PACE enables minute-scale SNN training and efficient edge deployment by creating compact dataset surrogates that maintain high performance while dramatically reducing computational and storage requirements.

Abstract: Event cameras sense brightness changes and output binary asynchronous event streams, attracting increasing attention. Their bio-inspired dynamics align well with spiking neural networks (SNNs), offering a promising energy-efficient alternative to conventional vision systems. However, SNNs remain costly to train due to temporal coding, which limits their practical deployment. To alleviate the high training cost of SNNs, we introduce \textbf{PACE} (Phase-Aligned Condensation for Events), the first dataset distillation framework to SNNs and event-based vision. PACE distills a large training dataset into a compact synthetic one that enables fast SNN training, which is achieved by two core modules: \textbf{ST-DSM} and \textbf{PEQ-N}. ST-DSM uses residual membrane potentials to densify spike-based features (SDR) and to perform fine-grained spatiotemporal matching of amplitude and phase (ST-SM), while PEQ-N provides a plug-and-play straight through probabilistic integer quantizer compatible with standard event-frame pipelines. Across DVS-Gesture, CIFAR10-DVS, and N-MNIST datasets, PACE outperforms existing coreset selection and dataset distillation baselines, with particularly strong gains on dynamic event streams and at low or moderate IPC. Specifically, on N-MNIST, it achieves (84.4%) accuracy, about (85%) of the full training set performance, while reducing training time by more than (50\times) and storage cost by (6000\times), yielding compact surrogates that enable minute-scale SNN training and efficient edge deployment.

Method: SpikeNM uses semi-structured N:M pruning with M-way basis-logit parameterization and differentiable top-k sampler to linearize complexity. It also employs eligibility-inspired distillation (EID) to align mask probabilities with spiking dynamics using temporally accumulated credits as soft targets.

Result: At 2:4 sparsity, SpikeNM maintains and even improves accuracy across mainstream datasets while producing hardware-amenable sparse patterns that complement intrinsic spike sparsity.

Conclusion: SpikeNM successfully bridges the gap between unstructured and structured pruning for SNNs, providing a hardware-friendly semi-structured approach that preserves accuracy and enables more aggressive sparsification through innovative parameterization and neuroscience-inspired techniques.

Abstract: Brain-inspired Spiking neural networks (SNNs) promise energy-efficient intelligence via event-driven, sparse computation, but deeper architectures inflate parameters and computational cost, hindering their edge deployment. Recent progress in SNN pruning helps alleviate this burden, yet existing efforts fall into only two families: \emph{unstructured} pruning, which attains high sparsity but is difficult to accelerate on general hardware, and \emph{structured} pruning, which eases deployment but lack flexibility and often degrades accuracy at matched sparsity. In this work, we introduce \textbf{SpikeNM}, the first SNN-oriented \emph{semi-structured} (N{:}M) pruning framework that learns sparse SNNs \emph{from scratch}, enforcing \emph{at most (N)} non-zeros per (M)-weight block. To avoid the combinatorial space complexity (\sum_{k=1}^{N}\binom{M}{k}) growing exponentially with (M), SpikeNM adopts an (M)-way basis-logit parameterization with a differentiable top-(k) sampler, \emph{linearizing} per-block complexity to (\mathcal O(M)) and enabling more aggressive sparsification. Further inspired by neuroscience, we propose \emph{eligibility-inspired distillation} (EID), which converts temporally accumulated credits into block-wise soft targets to align mask probabilities with spiking dynamics, reducing sampling variance and stabilizing search under high sparsity. Experiments show that at (2{:}4) sparsity, SpikeNM maintains and even with gains across main-stream datasets, while yielding hardware-amenable patterns that complement intrinsic spike sparsity.

Method: DINOv3-Guided Cross Fusion (DGCF) integrates frozen self-supervised DINOv3 Transformer with trainable CNN encoder-decoder, using cross fusion module and Multi-Level DINOv3 Perceptual loss.

Result: Achieved state-of-the-art performance on SynthRAD2023 pelvic dataset for MRI→CT and CBCT→CT translation tasks in terms of MS-SSIM, PSNR and segmentation-based metrics.

Conclusion: First work to employ DINOv3 representations for medical image translation, demonstrating potential of self-supervised Transformer guidance for semantic-aware CT synthesis.

Abstract: Generating synthetic CT images from CBCT or MRI has a potential for efficient radiation dose planning and adaptive radiotherapy. However, existing CNN-based models lack global semantic understanding, while Transformers often overfit small medical datasets due to high model capacity and weak inductive bias. To address these limitations, we propose a DINOv3-Guided Cross Fusion (DGCF) framework that integrates a frozen self-supervised DINOv3 Transformer with a trainable CNN encoder-decoder. It hierarchically fuses global representation of Transformer and local features of CNN via a learnable cross fusion module, achieving balanced local appearance and contextual representation. Furthermore, we introduce a Multi-Level DINOv3 Perceptual (MLDP) loss that encourages semantic similarity between synthetic CT and the ground truth in DINOv3’s feature space. Experiments on the SynthRAD2023 pelvic dataset demonstrate that DGCF achieved state-of-the-art performance in terms of MS-SSIM, PSNR and segmentation-based metrics on both MRI$\rightarrow$CT and CBCT$\rightarrow$CT translation tasks. To the best of our knowledge, this is the first work to employ DINOv3 representations for medical image translation, highlighting the potential of self-supervised Transformer guidance for semantic-aware CT synthesis. The code is available at https://github.com/HiLab-git/DGCF.

Method: Introduces ada-BOV tokens that adaptively absorb denoised preceding frames via adaptive layer norm modulation, plus a refinement strategy that decouples sampling trajectory length from attention window constraints, and a disturbance-augmented training noise schedule.

Result: Achieves compelling qualitative and quantitative results across multiple metrics, demonstrating improved global consistency and local dynamics in long video generation.

Conclusion: The proposed ada-BOV tokens and refinement strategy effectively address limitations in existing autoregressive video diffusion models, enabling high-quality long video generation with better consistency and motion dynamics.

Abstract: Recent advancements in diffusion-based video generation have produced impressive and high-fidelity short videos. To extend these successes to generate coherent long videos, most video diffusion models (VDMs) generate videos in an autoregressive manner, i.e., generating subsequent frames conditioned on previous ones. There are generally two primary paradigms: chunk-based extension and stream denoising. The former directly concatenates previous clean frames as conditioning, suffering from denoising latency and error accumulation. The latter maintains the denoising sequence with monotonically increasing noise levels. In each denoising iteration, one clean frame is produced while a new pure noise is simultaneously appended, enabling live-stream sampling. However, it struggles with fragile consistency and poor motion dynamics. In this paper, we propose Adaptive Begin-of-Video Tokens (ada-BOV) for autoregressive VDMs. The BOV tokens are special learnable embeddings on VDMs. They adaptively absorb denoised preceding frames via an adaptive-layer-norm-like modulation. This design preserves the global consistency while allowing for flexible conditioning in dynamic scenarios. To ensure the quality of local dynamics essential in modulating BOV tokens, we further propose a refinement strategy for stream denoising. It decouples the sampling trajectory length from the attention window size constraint, leading to improved local guidance and overall imaging quality. We also propose a disturbance-augmented training noise schedule, which balances the convergence speed with model robustness for the stream denoising. Extensive experiments demonstrate that our method achieves compelling qualitative and quantitative results across multiple metrics.

Method: Propose Subset-Selected Counterfactual Augmentation (SS-CA) using Counterfactual LIMA to identify minimal spatial region sets whose removal alters predictions. Replace identified regions with natural background and train jointly on augmented and original samples.

Result: Extensive experiments on ImageNet variants show SS-CA improves in-distribution generalization and achieves superior performance on out-of-distribution benchmarks (ImageNet-R, ImageNet-S). Models also exhibit enhanced generalization under noise perturbations.

Conclusion: SS-CA effectively uses interpretability insights to correct model deficiencies, improving both performance and robustness by mitigating incomplete causal learning through targeted counterfactual augmentation.

Abstract: In current visual model training, models often rely on only limited sufficient causes for their predictions, which makes them sensitive to distribution shifts or the absence of key features. Attribution methods can accurately identify a model’s critical regions. However, masking these areas to create counterfactuals often causes the model to misclassify the target, while humans can still easily recognize it. This divergence highlights that the model’s learned dependencies may not be sufficiently causal. To address this issue, we propose Subset-Selected Counterfactual Augmentation (SS-CA), which integrates counterfactual explanations directly into the training process for targeted intervention. Building on the subset-selection-based LIMA attribution method, we develop Counterfactual LIMA to identify minimal spatial region sets whose removal can selectively alter model predictions. Leveraging these attributions, we introduce a data augmentation strategy that replaces the identified regions with natural background, and we train the model jointly on both augmented and original samples to mitigate incomplete causal learning. Extensive experiments across multiple ImageNet variants show that SS-CA improves generalization on in-distribution (ID) test data and achieves superior performance on out-of-distribution (OOD) benchmarks such as ImageNet-R and ImageNet-S. Under perturbations including noise, models trained with SS-CA also exhibit enhanced generalization, demonstrating that our approach effectively uses interpretability insights to correct model deficiencies and improve both performance and robustness.

Method: Extends SPOTER paradigm with cultural-specific preprocessing, uses a compact four-layer transformer encoder with optimized learnable positional encodings, and employs curriculum learning for better generalization and faster convergence.

Result: Achieved 97.92% Top-1 validation accuracy on BdSLW60 benchmark, representing 22.82% improvement over Bi-LSTM baseline, with reduced parameters, lower FLOPs, and higher FPS.

Conclusion: BdSL-SPOTER provides a practical framework for real-world accessibility applications and serves as a scalable model for other low-resource regional sign languages.

Abstract: We introduce BdSL-SPOTER, a pose-based transformer framework for accurate and efficient recognition of Bengali Sign Language (BdSL). BdSL-SPOTER extends the SPOTER paradigm with cultural specific preprocessing and a compact four-layer transformer encoder featuring optimized learnable positional encodings, while employing curriculum learning to enhance generalization on limited data and accelerate convergence. On the BdSLW60 benchmark, it achieves 97.92% Top-1 validation accuracy, representing a 22.82% improvement over the Bi-LSTM baseline, all while keeping computational costs low. With its reduced number of parameters, lower FLOPs, and higher FPS, BdSL-SPOTER provides a practical framework for real-world accessibility applications and serves as a scalable model for other low-resource regional sign languages.

Method: Multi-task deep learning model trained on building footprint/height data paired with quarterly PlanetScope satellite images, predicting at 37.6-meter resolution.

Result: Achieves 85-88% F1 score on validation, 0.96 trend-consistency score, and captures quarterly settlement changes efficiently.

Conclusion: TEMPO provides cost-effective, temporally stable building monitoring essential for global resilience and adaptation efforts.

Abstract: We present TEMPO, a global, temporally resolved dataset of building density and height derived from high-resolution satellite imagery using deep learning models. We pair building footprint and height data from existing datasets with quarterly PlanetScope basemap satellite images to train a multi-task deep learning model that predicts building density and building height at a 37.6-meter per pixel resolution. We apply this model to global PlanetScope basemaps from Q1 2018 through Q2 2025 to create global, temporal maps of building density and height. We validate these maps by comparing against existing building footprint datasets. Our estimates achieve an F1 score between 85% and 88% on different hand-labeled subsets, and are temporally stable, with a 0.96 five-year trend-consistency score. TEMPO captures quarterly changes in built settlements at a fraction of the computational cost of comparable approaches, unlocking large-scale monitoring of development patterns and climate impacts essential for global resilience and adaptation efforts.

Method: Proposes DFF-Adapter with lightweight multi-head LoRA modules integrated into every transformer block of DINOv2, enabling multi-task learning for both authenticity detection and fine-grained manipulation type classification with shared knowledge propagation.

Result: Achieves detection accuracy comparable to or surpassing current state-of-the-art methods while using only 3.5M trainable parameters, demonstrating parameter efficiency.

Conclusion: The proposed DFF-Adapter effectively addresses deepfake detection by leveraging fine-grained manipulation classification to enhance artifact sensitivity, providing an efficient and powerful solution for information integrity protection.

Abstract: The proliferation of sophisticated deepfakes poses significant threats to information integrity. While DINOv2 shows promise for detection, existing fine-tuning approaches treat it as generic binary classification, overlooking distinct artifacts inherent to different deepfake methods. To address this, we propose a DeepFake Fine-Grained Adapter (DFF-Adapter) for DINOv2. Our method incorporates lightweight multi-head LoRA modules into every transformer block, enabling efficient backbone adaptation. DFF-Adapter simultaneously addresses authenticity detection and fine-grained manipulation type classification, where classifying forgery methods enhances artifact sensitivity. We introduce a shared branch propagating fine-grained manipulation cues to the authenticity head. This enables multi-task cooperative optimization, explicitly enhancing authenticity discrimination with manipulation-specific knowledge. Utilizing only 3.5M trainable parameters, our parameter-efficient approach achieves detection accuracy comparable to or even surpassing that of current complex state-of-the-art methods.

Method: Proposed MEMR-Seg task for multi-round entity-level reasoning segmentation. Built MR-MedSeg dataset with 177K multi-round medical segmentation dialogues. Developed MediRound baseline model with a lightweight Judgment & Correction Mechanism to mitigate error propagation in the chain-like multi-round pipeline.

Result: Experimental results show the method effectively addresses the MEMR-Seg task and outperforms conventional medical referring segmentation methods.

Conclusion: MEMR-Seg enables interactive multi-round reasoning for medical image segmentation, providing a more flexible and reasoning-based approach compared to existing single-round methods.

Abstract: Despite the progress in medical image segmentation, most existing methods remain task-specific and lack interactivity. Although recent text-prompt-based segmentation approaches enhance user-driven and reasoning-based segmentation, they remain confined to single-round dialogues and fail to perform multi-round reasoning. In this work, we introduce Multi-Round Entity-Level Medical Reasoning Segmentation (MEMR-Seg), a new task that requires generating segmentation masks through multi-round queries with entity-level reasoning. To support this task, we construct MR-MedSeg, a large-scale dataset of 177K multi-round medical segmentation dialogues, featuring entity-based reasoning across rounds. Furthermore, we propose MediRound, an effective baseline model designed for multi-round medical reasoning segmentation. To mitigate the inherent error propagation in the chain-like pipeline of multi-round segmentation, we introduce a lightweight yet effective Judgment & Correction Mechanism during model inference. Experimental results demonstrate that our method effectively addresses the MEMR-Seg task and outperforms conventional medical referring segmentation methods.

Method: Jointly models radar target detection and motion estimation in unified architecture using self-supervised loss functions guided by Doppler shifts and echo intensity for spatial and motion consistency.

Result: Outperforms radar-based decoupled motion perception pipelines and achieves reliable motion perception across diverse weather and illumination conditions on public datasets.

Conclusion: RadarMP enhances perception capabilities for full-scenario autonomous driving systems by providing precise 3D scene motion perception in challenging weather conditions.

Abstract: Accurate 3D scene motion perception significantly enhances the safety and reliability of an autonomous driving system. Benefiting from its all-weather operational capability and unique perceptual properties, 4D mmWave radar has emerged as an essential component in advanced autonomous driving. However, sparse and noisy radar points often lead to imprecise motion perception, leaving autonomous vehicles with limited sensing capabilities when optical sensors degrade under adverse weather conditions. In this paper, we propose RadarMP, a novel method for precise 3D scene motion perception using low-level radar echo signals from two consecutive frames. Unlike existing methods that separate radar target detection and motion estimation, RadarMP jointly models both tasks in a unified architecture, enabling consistent radar point cloud generation and pointwise 3D scene flow prediction. Tailored to radar characteristics, we design specialized self-supervised loss functions guided by Doppler shifts and echo intensity, effectively supervising spatial and motion consistency without explicit annotations. Extensive experiments on the public dataset demonstrate that RadarMP achieves reliable motion perception across diverse weather and illumination conditions, outperforming radar-based decoupled motion perception pipelines and enhancing perception capabilities for full-scenario autonomous driving systems.

Method: OAD-Promoter has three components: Object-concentrated Example Generation (OEG) for global captions and object samples, Memory Knowledge Assistance (MKA) for retrieving relevant knowledge from stored examples, and OAD Prompt for optimized LLM inference.

Result: Experiments show OAD-Promoter significantly improves LLM-based VQA performance in few-shot or zero-shot settings, achieving new state-of-the-art results.

Conclusion: OAD-Promoter effectively addresses language bias and domain-shift challenges in LLM-based VQA, enhancing reliability and generalization capabilities.

Abstract: Large Language Models (LLMs) have become a crucial tool in Visual Question Answering (VQA) for handling knowledge-intensive questions in few-shot or zero-shot scenarios. However, their reliance on massive training datasets often causes them to inherit language biases during the acquisition of knowledge. This limitation imposes two key constraints on existing methods: (1) LLM predictions become less reliable due to bias exploitation, and (2) despite strong knowledge reasoning capabilities, LLMs still struggle with out-of-distribution (OOD) generalization. To address these issues, we propose Object Attribute Description Promoter (OAD-Promoter), a novel approach for enhancing LLM-based VQA by mitigating language bias and improving domain-shift robustness. OAD-Promoter comprises three components: the Object-concentrated Example Generation (OEG) module, the Memory Knowledge Assistance (MKA) module, and the OAD Prompt. The OEG module generates global captions and object-concentrated samples, jointly enhancing visual information input to the LLM and mitigating bias through complementary global and regional visual cues. The MKA module assists the LLM in handling OOD samples by retrieving relevant knowledge from stored examples to support questions from unseen domains. Finally, the OAD Prompt integrates the outputs of the preceding modules to optimize LLM inference. Experiments demonstrate that OAD-Promoter significantly improves the performance of LLM-based VQA methods in few-shot or zero-shot settings, achieving new state-of-the-art results.

Method: Trained models from SNNTorch are translated to compact C representation with static, cache-friendly data layouts and preallocation. Sparse spiking activity is exploited to prune inactive neurons and synapses.

Result: Achieved ~10x speedups on desktop CPU with additional gains from pruning, large memory reductions enabling microcontroller deployment (Arduino Portenta H7), and functional parity with Python baseline on N-MNIST and ST-MNIST datasets.

Conclusion: SNNs can be executed efficiently on conventional embedded platforms when paired with an optimized runtime and spike-driven model compression.

Abstract: Spiking neural networks (SNNs) communicate via discrete spikes in time rather than continuous activations. Their event-driven nature offers advantages for temporal processing and energy efficiency on resource-constrained hardware, but training and deployment remain challenging. We present a lightweight C-based runtime for SNN inference on edge devices and optimizations that reduce latency and memory without sacrificing accuracy. Trained models exported from SNNTorch are translated to a compact C representation; static, cache-friendly data layouts and preallocation avoid interpreter and allocation overheads. We further exploit sparse spiking activity to prune inactive neurons and synapses, shrinking computation in upstream convolutional layers. Experiments on N-MNIST and ST-MNIST show functional parity with the Python baseline while achieving ~10 speedups on desktop CPU and additional gains with pruning, together with large memory reductions that enable microcontroller deployment (Arduino Portenta H7). Results indicate that SNNs can be executed efficiently on conventional embedded platforms when paired with an optimized runtime and spike-driven model compression. Code: https://github.com/karol-jurzec/snn-generator/

Method: Developed a dataset of 157K visual QA instances with fact-level citations to multimodal documents, created fine-grained automatic metrics for informativeness, groundedness, and fluency, and evaluated various prompting methods and LVLMs with multimodal RAG.

Result: LVLMs with multimodal RAG generate more informative and fluent answers than unimodal RAG, but show weaker groundedness for image documents. There’s a trade-off between informativeness and groundedness across prompting methods.

Conclusion: Mitigating contextual bias in interpreting image documents is crucial for future research in multimodal source attribution systems.

Abstract: Source attribution aims to enhance the reliability of AI-generated answers by including references for each statement, helping users validate the provided answers. However, existing work has primarily focused on text-only scenario and largely overlooked the role of multimodality. We introduce MAVIS, the first benchmark designed to evaluate multimodal source attribution systems that understand user intent behind visual questions, retrieve multimodal evidence, and generate long-form answers with citations. Our dataset comprises 157K visual QA instances, where each answer is annotated with fact-level citations referring to multimodal documents. We develop fine-grained automatic metrics along three dimensions of informativeness, groundedness, and fluency, and demonstrate their strong correlation with human judgments. Our key findings are threefold: (1) LVLMs with multimodal RAG generate more informative and fluent answers than unimodal RAG, but they exhibit weaker groundedness for image documents than for text documents, a gap amplified in multimodal settings. (2) Given the same multimodal documents, there is a trade-off between informativeness and groundedness across different prompting methods. (3) Our proposed method highlights mitigating contextual bias in interpreting image documents as a crucial direction for future research. The dataset and experimental code are available at https://github.com/seokwon99/MAVIS

Method: Proposes TMKT with Time-step Mixup (TSM) strategy that interpolates RGB and DVS inputs at various time steps. Uses two modality-aware objectives: Modality Aware Guidance (MAG) for per-frame supervision and Mixup Ratio Perception (MRP) for sequence-level mix ratio estimation.

Result: TMKT enables smoother knowledge transfer, mitigates modality mismatch during training, and achieves superior performance in spiking image classification tasks across diverse benchmarks and multiple SNN backbones.

Conclusion: The proposed TMKT framework with Time-step Mixup strategy effectively bridges the modality gap between RGB and DVS, providing stable optimization and improved performance for spiking neural networks in visual intelligence tasks.

Abstract: The integration of event cameras and spiking neural networks (SNNs) promises energy-efficient visual intelligence, yet scarce event data and the sparsity of DVS outputs hinder effective training. Prior knowledge transfers from RGB to DVS often underperform because the distribution gap between modalities is substantial. In this work, we present Time-step Mixup Knowledge Transfer (TMKT), a cross-modal training framework with a probabilistic Time-step Mixup (TSM) strategy. TSM exploits the asynchronous nature of SNNs by interpolating RGB and DVS inputs at various time steps to produce a smooth curriculum within each sequence, which reduces gradient variance and stabilizes optimization with theoretical analysis. To employ auxiliary supervision from TSM, TMKT introduces two lightweight modality-aware objectives, Modality Aware Guidance (MAG) for per-frame source supervision and Mixup Ratio Perception (MRP) for sequence-level mix ratio estimation, which explicitly align temporal features with the mixing schedule. TMKT enables smoother knowledge transfer, helps mitigate modality mismatch during training, and achieves superior performance in spiking image classification tasks. Extensive experiments across diverse benchmarks and multiple SNN backbones, together with ablations, demonstrate the effectiveness of our method.

Method: Uses Frequency-Interactive Attention with two key components: Frequency Representation Interaction (FRI) module for cross-domain alignment via frequency component exchange, and Feature Injection (FIJ) module that incorporates source-side queries, keys, values, and text embeddings into target branch’s cross-attention.

Result: Achieves high-fidelity editing at low computational cost (~6s per 512*512 image on RTX 4090), outperforms existing methods in visual quality, background fidelity, and controllability, and successfully extends to medical applications for surgical image augmentation.

Conclusion: FIA-Edit provides an efficient and effective solution for text-guided image editing that preserves source structure while enabling precise semantic edits, with demonstrated value in both general and clinical applications.

Abstract: Text-guided image editing has advanced rapidly with the rise of diffusion models. While flow-based inversion-free methods offer high efficiency by avoiding latent inversion, they often fail to effectively integrate source information, leading to poor background preservation, spatial inconsistencies, and over-editing due to the lack of effective integration of source information. In this paper, we present FIA-Edit, a novel inversion-free framework that achieves high-fidelity and semantically precise edits through a Frequency-Interactive Attention. Specifically, we design two key components: (1) a Frequency Representation Interaction (FRI) module that enhances cross-domain alignment by exchanging frequency components between source and target features within self-attention, and (2) a Feature Injection (FIJ) module that explicitly incorporates source-side queries, keys, values, and text embeddings into the target branch’s cross-attention to preserve structure and semantics. Comprehensive and extensive experiments demonstrate that FIA-Edit supports high-fidelity editing at low computational cost (~6s per 512 * 512 image on an RTX 4090) and consistently outperforms existing methods across diverse tasks in visual quality, background fidelity, and controllability. Furthermore, we are the first to extend text-guided image editing to clinical applications. By synthesizing anatomically coherent hemorrhage variations in surgical images, FIA-Edit opens new opportunities for medical data augmentation and delivers significant gains in downstream bleeding classification. Our project is available at: https://github.com/kk42yy/FIA-Edit.

Method: Proposes Center-Reassigned Hashing (CRH) with dynamic hash center reassignment from a preset codebook, joint optimization of hash function, and a multi-head mechanism to enhance representational capacity of hash centers.

Result: Extensive experiments on three benchmarks show CRH learns semantically meaningful hash centers and outperforms state-of-the-art deep hashing methods in retrieval tasks.

Conclusion: CRH provides an effective end-to-end solution for deep hashing that dynamically adapts hash centers to data distribution while capturing richer semantic structures through multi-head mechanisms.

Abstract: Hash center-based deep hashing methods improve upon pairwise or triplet-based approaches by assigning fixed hash centers to each class as learning targets, thereby avoiding the inefficiency of local similarity optimization. However, random center initialization often disregards inter-class semantic relationships. While existing two-stage methods mitigate this by first refining hash centers with semantics and then training the hash function, they introduce additional complexity, computational overhead, and suboptimal performance due to stage-wise discrepancies. To address these limitations, we propose $\textbf{Center-Reassigned Hashing (CRH)}$, an end-to-end framework that $\textbf{dynamically reassigns hash centers}$ from a preset codebook while jointly optimizing the hash function. Unlike previous methods, CRH adapts hash centers to the data distribution $\textbf{without explicit center optimization phases}$, enabling seamless integration of semantic relationships into the learning process. Furthermore, $\textbf{a multi-head mechanism}$ enhances the representational capacity of hash centers, capturing richer semantic structures. Extensive experiments on three benchmarks demonstrate that CRH learns semantically meaningful hash centers and outperforms state-of-the-art deep hashing methods in retrieval tasks.

Method: PGNet: multi-stage framework with dual-feature encoding, coarse scaffold synthesis, and hierarchical correction via Completion-by-Correction paradigm using pretrained image-to-3D shape priors.

Result: Achieves 23.5% improvement in Chamfer Distance and 7.1% improvement in F-score on ShapeNetViPC dataset compared to state-of-the-art baselines.

Conclusion: Completion-by-Correction paradigm enables structurally consistent and observation-aligned reconstruction by shifting from unconstrained synthesis to guided refinement with topologically complete shape priors.

Abstract: Point cloud completion aims to reconstruct complete 3D shapes from partial observations, which is a challenging problem due to severe occlusions and missing geometry. Despite recent advances in multimodal techniques that leverage complementary RGB images to compensate for missing geometry, most methods still follow a Completion-by-Inpainting paradigm, synthesizing missing structures from fused latent features. We empirically show that this paradigm often results in structural inconsistencies and topological artifacts due to limited geometric and semantic constraints. To address this, we rethink the task and propose a more robust paradigm, termed Completion-by-Correction, which begins with a topologically complete shape prior generated by a pretrained image-to-3D model and performs feature-space correction to align it with the partial observation. This paradigm shifts completion from unconstrained synthesis to guided refinement, enabling structurally consistent and observation-aligned reconstruction. Building upon this paradigm, we introduce PGNet, a multi-stage framework that conducts dual-feature encoding to ground the generative prior, synthesizes a coarse yet structurally aligned scaffold, and progressively refines geometric details via hierarchical correction. Experiments on the ShapeNetViPC dataset demonstrate the superiority of PGNet over state-of-the-art baselines in terms of average Chamfer Distance (-23.5%) and F-score (+7.1%).

Method: MixAR uses factorized formulation with discrete tokens as prior guidance for continuous AR prediction. It explores three mixture strategies: DC-SA (self-attention), DC-CA (cross-attention), and DC-Mix (replacing mask tokens with discrete counterparts). Also introduces TI-Mix to bridge training-inference distribution gaps.

Result: DC-Mix strategy achieves favorable balance between computational efficiency and generation fidelity, and TI-Mix consistently improves performance by aligning training and generation distributions.

Conclusion: MixAR successfully addresses challenges in continuous autoregressive modeling by leveraging discrete tokens as guidance, with DC-Mix and TI-Mix proving particularly effective for improving generation quality while maintaining efficiency.

Abstract: Autoregressive (AR) approaches, which represent images as sequences of discrete tokens from a finite codebook, have achieved remarkable success in image generation. However, the quantization process and the limited codebook size inevitably discard fine-grained information, placing bottlenecks on fidelity. Motivated by this limitation, recent studies have explored autoregressive modeling in continuous latent spaces, which offers higher generation quality. Yet, unlike discrete tokens constrained by a fixed codebook, continuous representations lie in a vast and unstructured space, posing significant challenges for efficient autoregressive modeling. To address these challenges, we introduce MixAR, a novel framework that leverages mixture training paradigms to inject discrete tokens as prior guidance for continuous AR modeling. MixAR is a factorized formulation that leverages discrete tokens as prior guidance for continuous autoregressive prediction. We investigate several discrete-continuous mixture strategies, including self-attention (DC-SA), cross-attention (DC-CA), and a simple approach (DC-Mix) that replaces homogeneous mask tokens with informative discrete counterparts. Moreover, to bridge the gap between ground-truth training tokens and inference tokens produced by the pre-trained AR model, we propose Training-Inference Mixture (TI-Mix) to achieve consistent training and generation distributions. In our experiments, we demonstrate a favorable balance of the DC-Mix strategy between computational efficiency and generation fidelity, and consistent improvement of TI-Mix.

Method: Proposes MMRINet with linear-complexity Mamba state-space models replacing quadratic attention, Dual-Path Feature Refinement modules for feature diversity, and Progressive Feature Aggregation for multi-scale fusion.

Result: Achieves average Dice score of 0.752 and average HD95 of 12.23 in BraTS-Lighthouse SSA 2025 with only ~2.5M parameters.

Conclusion: MMRINet demonstrates efficient and accurate brain tumor segmentation suitable for low-resource clinical environments.

Abstract: Automated brain tumor segmentation in multi-parametric MRI remains challenging in resource-constrained settings where deep 3D networks are computationally prohibitive. We propose MMRINet, a lightweight architecture that replaces quadratic-complexity attention with linear-complexity Mamba state-space models for efficient volumetric context modeling. Novel Dual-Path Feature Refinement (DPFR) modules maximize feature diversity without additional data requirements, while Progressive Feature Aggregation (PFA) enables effective multi-scale fusion. In the BraTS-Lighthouse SSA 2025, our model achieves strong performance with an average Dice score of (0.752) and an average HD95 of (12.23) with only ~2.5M parameters, demonstrating efficient and accurate segmentation suitable for low-resource clinical environments. Our GitHub repository can be accessed here: github.com/BioMedIA-MBZUAI/MMRINet.

Method: Two-phase approach: 1) Learn view-invariant and action-discriminative features using contrastive learning on multi-view data within a single modality, 2) Perform domain adaptation to new modalities using information bottleneck loss without requiring labeled data from the new domain.

Result: Improves top-1 accuracy on RGB video data by almost 50% compared to supervised contrastive learning-based cross-view methods, and outperforms unsupervised domain adaptation-only methods by up to 5% using the same video transformer backbone.

Conclusion: The proposed joint framework effectively addresses both cross-view generalization and domain adaptation challenges, enabling more robust and scalable deployment of driver monitoring systems across diverse vehicle configurations.

Abstract: Driver distraction remains a leading cause of road traffic accidents, contributing to thousands of fatalities annually across the globe. While deep learning-based driver activity recognition methods have shown promise in detecting such distractions, their effectiveness in real-world deployments is hindered by two critical challenges: variations in camera viewpoints (cross-view) and domain shifts such as change in sensor modality or environment. Existing methods typically address either cross-view generalization or unsupervised domain adaptation in isolation, leaving a gap in the robust and scalable deployment of models across diverse vehicle configurations. In this work, we propose a novel two-phase cross-view, cross-modal unsupervised domain adaptation framework that addresses these challenges jointly on real-time driver monitoring data. In the first phase, we learn view-invariant and action-discriminative features within a single modality using contrastive learning on multi-view data. In the second phase, we perform domain adaptation to a new modality using information bottleneck loss without requiring any labeled data from the new domain. We evaluate our approach using state-of-the art video transformers (Video Swin, MViT) and multi modal driver activity dataset called Drive&Act, demonstrating that our joint framework improves top-1 accuracy on RGB video data by almost 50% compared to a supervised contrastive learning-based cross-view method, and outperforms unsupervised domain adaptation-only methods by up to 5%, using the same video transformer backbone.

Method: Uses Dual Style Randomization (DSR) for diverse style simulation, Hierarchical Semantic Mining (HSM) with multi-scale superpixels for granularity learning, and Prototype Confidence-modulated Thresholding (PCMT) for segmentation ambiguity reduction.

Result: Extensive experiments on four target domain datasets demonstrate state-of-the-art performance.

Conclusion: The proposed HSL framework effectively addresses segmentation granularity gaps and enhances semantic discriminability in cross-domain few-shot segmentation.

Abstract: Cross-domain Few-shot Segmentation (CD-FSS) aims to segment novel classes from target domains that are not involved in training and have significantly different data distributions from the source domain, using only a few annotated samples, and recent years have witnessed significant progress on this task. However, existing CD-FSS methods primarily focus on style gaps between source and target domains while ignoring segmentation granularity gaps, resulting in insufficient semantic discriminability for novel classes in target domains. Therefore, we propose a Hierarchical Semantic Learning (HSL) framework to tackle this problem. Specifically, we introduce a Dual Style Randomization (DSR) module and a Hierarchical Semantic Mining (HSM) module to learn hierarchical semantic features, thereby enhancing the model’s ability to recognize semantics at varying granularities. DSR simulates target domain data with diverse foreground-background style differences and overall style variations through foreground and global style randomization respectively, while HSM leverages multi-scale superpixels to guide the model to mine intra-class consistency and inter-class distinction at different granularities. Additionally, we also propose a Prototype Confidence-modulated Thresholding (PCMT) module to mitigate segmentation ambiguity when foreground and background are excessively similar. Extensive experiments are conducted on four popular target domain datasets, and the results demonstrate that our method achieves state-of-the-art performance.

Method: Three adaptive mechanisms: (1) lazy-active query classification, (2) head-level dynamic KV budget allocation based on flattest head, (3) KV cache slimming via selective visual KV fetching.

Result: Matches full attention performance while achieving 2.7x speedup during prefill and 2.4x memory reduction during decoding.

Conclusion: OmniSparse effectively bridges training-inference gap and enables fine-grained sparse attention for long-video MLLMs with significant acceleration and memory efficiency.

Abstract: Existing sparse attention methods primarily target inference-time acceleration by selecting critical tokens under predefined sparsity patterns. However, they often fail to bridge the training-inference gap and lack the capacity for fine-grained token selection across multiple dimensions such as queries, key-values (KV), and heads, leading to suboptimal performance and limited acceleration gains. In this paper, we introduce OmniSparse, a training-aware fine-grained sparse attention framework for long-video MLLMs, which operates in both training and inference with dynamic token budget allocation. Specifically, OmniSparse contains three adaptive and complementary mechanisms: (1) query selection via lazy-active classification, retaining active queries that capture broad semantic similarity while discarding most lazy ones that focus on limited local context and exhibit high functional redundancy; (2) KV selection with head-level dynamic budget allocation, where a shared budget is determined based on the flattest head and applied uniformly across all heads to ensure attention recall; and (3) KV cache slimming to reduce head-level redundancy by selectively fetching visual KV cache according to the head-level decoding query pattern. Experimental results show that OmniSparse matches the performance of full attention while achieving up to 2.7x speedup during prefill and 2.4x memory reduction during decoding.

Method: Assign learnable spatial shifting parameters to each view and adjust views toward spatially consistent targets guided by reconstructed mesh, with input view as extra constraint for optimization to enhance robustness to non-frontal angles.

Result: Consistently achieves leading results in both geometric and texture evaluation metrics across flexible input viewpoints, with more complete geometric details and clean textures.

Conclusion: LSS3D effectively handles multi-view inconsistencies and non-frontal input views through learnable spatial shifting, providing high-quality 3D generation and contributing a comprehensive quantitative evaluation pipeline to the community.

Abstract: Recently, multi-view diffusion-based 3D generation methods have gained significant attention. However, these methods often suffer from shape and texture misalignment across generated multi-view images, leading to low-quality 3D generation results, such as incomplete geometric details and textural ghosting. Some methods are mainly optimized for the frontal perspective and exhibit poor robustness to oblique perspective inputs. In this paper, to tackle the above challenges, we propose a high-quality image-to-3D approach, named LSS3D, with learnable spatial shifting to explicitly and effectively handle the multiview inconsistencies and non-frontal input view. Specifically, we assign learnable spatial shifting parameters to each view, and adjust each view towards a spatially consistent target, guided by the reconstructed mesh, resulting in high-quality 3D generation with more complete geometric details and clean textures. Besides, we include the input view as an extra constraint for the optimization, further enhancing robustness to non-frontal input angles, especially for elevated viewpoint inputs. We also provide a comprehensive quantitative evaluation pipeline that can contribute to the community in performance comparisons. Extensive experiments demonstrate that our method consistently achieves leading results in both geometric and texture evaluation metrics across more flexible input viewpoints.

Method: Uses multi-view geometry extraction with depth/normal maps and segmentation masks, decoupled geometry-enhanced attention, adaptive learning strategy, iterative refinement, and dynamic geometry intensity adjustment.

Result: Generates images that are both consistent across views and rich in detail, improving overall image quality and detail preservation.

Conclusion: The proposed model effectively addresses cross-view consistency and detail quality issues in multi-view image generation through geometric guidance and adaptive mechanisms.

Abstract: Multi-view image generation holds significant application value in computer vision, particularly in domains like 3D reconstruction, virtual reality, and augmented reality. Most existing methods, which rely on extending single images, face notable computational challenges in maintaining cross-view consistency and generating high-resolution outputs. To address these issues, we propose the Geometry-guided Multi-View Diffusion Model, which incorporates mechanisms for extracting multi-view geometric information and adjusting the intensity of geometric features to generate images that are both consistent across views and rich in detail. Specifically, we design a multi-view geometry information extraction module that leverages depth maps, normal maps, and foreground segmentation masks to construct a shared geometric structure, ensuring shape and structural consistency across different views. To enhance consistency and detail restoration during generation, we develop a decoupled geometry-enhanced attention mechanism that strengthens feature focus on key geometric details, thereby improving overall image quality and detail preservation. Furthermore, we apply an adaptive learning strategy that fine-tunes the model to better capture spatial relationships and visual coherence between the generated views, ensuring realistic results. Our model also incorporates an iterative refinement process that progressively improves the output quality through multiple stages of image generation. Finally, a dynamic geometry information intensity adjustment mechanism is proposed to adaptively regulate the influence of geometric data, optimizing overall quality while ensuring the naturalness of generated images. More details can be found on the project page: https://github.com/SobeyMIL/GeoMVD.com.

Method: Leverages YOLOv8 for object detection and EasyOCR for license plate recognition, trained on custom annotated dataset with data augmentation. Uses Streamlit-based interface for real-time monitoring and includes advanced image preprocessing for challenging conditions.

Result: Achieves overall precision of 0.9147, recall of 0.886, mAP@50 of 0.843, and mAP@50:95 of 0.503, demonstrating strong detection capability under strict IoU thresholds.

Conclusion: Presents a practical and effective automated solution for traffic rule enforcement with considerations for real-world deployment, particularly innovative for detecting rear-view mirror absence on motorcycles.

Abstract: Road safety is a critical global concern, with manual enforcement of helmet laws and vehicle safety standards (e.g., rear-view mirror presence) being resource-intensive and inconsistent. This paper presents an AI-powered system to automate traffic violation detection, significantly enhancing enforcement efficiency and road safety. The system leverages YOLOv8 for robust object detection and EasyOCR for license plate recognition. Trained on a custom dataset of annotated images (augmented for diversity), it identifies helmet non-compliance, the absence of rear-view mirrors on motorcycles, an innovative contribution to automated checks, and extracts vehicle registration numbers. A Streamlit-based interface facilitates real-time monitoring and violation logging. Advanced image preprocessing enhances license plate recognition, particularly under challenging conditions. Based on evaluation results, the model achieves an overall precision of 0.9147, a recall of 0.886, and a mean Average Precision (mAP@50) of 0.843. The mAP@50 95 of 0.503 further indicates strong detection capability under stricter IoU thresholds. This work demonstrates a practical and effective solution for automated traffic rule enforcement, with considerations for real-world deployment discussed.

Method: Uses a learnable token-wise router with ε-greedy training strategy that sparsely selects top-k hidden states, creating denoising timestep- and input-dependent interactions between modalities.

Result: Achieves state-of-the-art results in text-to-image generation and editing with only 3B to 5B parameters, matching or surpassing models up to 4× larger.

Conclusion: MoS establishes a flexible and compute-efficient paradigm for scaling multimodal diffusion models through sparse, state-based modality fusion.

Abstract: We introduce MoS (Mixture of States), a novel fusion paradigm for multimodal diffusion models that merges modalities using flexible, state-based interactions. The core of MoS is a learnable, token-wise router that creates denoising timestep- and input-dependent interactions between modalities’ hidden states, precisely aligning token-level features with the diffusion trajectory. This router sparsely selects the top-$k$ hidden states and is trained with an $ε$-greedy strategy, efficiently selecting contextual features with minimal learnable parameters and negligible computational overhead. We validate our design with text-to-image generation (MoS-Image) and editing (MoS-Editing), which achieve state-of-the-art results. With only 3B to 5B parameters, our models match or surpass counterparts up to $4\times$ larger. These findings establish MoS as a flexible and compute-efficient paradigm for scaling multimodal diffusion models.

Method: Uses semantic-aware positive pair mining with adaptive normalization, text-conditioned sparse attention pooling for localized alignment, and hard-negative aware contrastive loss with adaptive reweighting.

Result: Achieves state-of-the-art performance on five medical imaging benchmarks for classification, detection, and segmentation tasks.

Conclusion: FaNe effectively addresses false negatives and enables fine-grained cross-modal alignment, demonstrating superior performance in medical vision-language pre-training.

Abstract: Medical vision-language pre-training (VLP) offers significant potential for advancing medical image understanding by leveraging paired image-report data. However, existing methods are limited by Fa}lse Negatives (FaNe) induced by semantically similar texts and insufficient fine-grained cross-modal alignment. To address these limitations, we propose FaNe, a semantic-enhanced VLP framework. To mitigate false negatives, we introduce a semantic-aware positive pair mining strategy based on text-text similarity with adaptive normalization. Furthermore, we design a text-conditioned sparse attention pooling module to enable fine-grained image-text alignment through localized visual representations guided by textual cues. To strengthen intra-modal discrimination, we develop a hard-negative aware contrastive loss that adaptively reweights semantically similar negatives. Extensive experiments on five downstream medical imaging benchmarks demonstrate that FaNe achieves state-of-the-art performance across image classification, object detection, and semantic segmentation, validating the effectiveness of our framework.

Method: Fine-tuned MiniGPT-v2 model using cascaded training for pancreas detection, tumor classification, and tumor detection with multimodal prompts combining questions and CT scans from NIH, MSD, and AbdomenCT-1k datasets.

Result: Achieved IoU of 0.595/0.550 for pancreas detection, 87.6% accuracy for cancer classification, and multi-organ detection with liver IoU 0.8399, but lower tumor detection IoU of 0.168.

Conclusion: Promising solution for pancreas tumor classification but needs improvement in detection tasks, especially for pancreas tumors.

Abstract: Problem: Pancreas radiological imaging is challenging due to the small size, blurred boundaries, and variability of shape and position of the organ among patients. Goal: In this work we present MiniGPT-Pancreas, a Multimodal Large Language Model (MLLM), as an interactive chatbot to support clinicians in pancreas cancer diagnosis by integrating visual and textual information. Methods: MiniGPT-v2, a general-purpose MLLM, was fine-tuned in a cascaded way for pancreas detection, tumor classification, and tumor detection with multimodal prompts combining questions and computed tomography scans from the National Institute of Health (NIH), and Medical Segmentation Decathlon (MSD) datasets. The AbdomenCT-1k dataset was used to detect the liver, spleen, kidney, and pancreas. Results: MiniGPT-Pancreas achieved an Intersection over Union (IoU) of 0.595 and 0.550 for the detection of pancreas on NIH and MSD datasets, respectively. For the pancreas cancer classification task on the MSD dataset, accuracy, precision, and recall were 0.876, 0.874, and 0.878, respectively. When evaluating MiniGPT-Pancreas on the AbdomenCT-1k dataset for multi-organ detection, the IoU was 0.8399 for the liver, 0.722 for the kidney, 0.705 for the spleen, and 0.497 for the pancreas. For the pancreas tumor detection task, the IoU score was 0.168 on the MSD dataset. Conclusions: MiniGPT-Pancreas represents a promising solution to support clinicians in the classification of pancreas images with pancreas tumors. Future research is needed to improve the score on the detection task, especially for pancreas tumors.

Method: Spectral Representation Filtering (SRF) identifies hallucination modes through eigendecomposition of covariance differences between truthful and hallucinatory caption features, then applies a soft spectral filter to attenuate these modes in deeper VLM layers.

Result: SRF consistently reduces hallucination rates across LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2 models on MSCOCO, POPE-VQA, and other benchmarks, achieving state-of-the-art faithfulness without degrading caption quality.

Conclusion: SRF provides an effective post-hoc solution for hallucination suppression that requires no training, incurs zero inference overhead, and works across multiple VLM families while preserving semantic fidelity.

Abstract: Vision-language models (VLMs) frequently produce hallucinations in the form of descriptions of objects, attributes, or relations that do not exist in the image due to over-reliance on language priors and imprecise cross-modal grounding. We introduce Spectral Representation Filtering (SRF), a lightweight, training-free method to suppress such hallucinations by analyzing and correcting the covariance structure of the model’s representations. SRF identifies low-rank hallucination modes through eigendecomposition of the covariance of the differences between features collected for truthful and hallucinatory captions, revealing structured biases in the feature space. A soft spectral filter then attenuates these modes in the feed-forward projection weights of deeper vLLM layers, equalizing feature variance while preserving semantic fidelity. Unlike decoding or retraining-based approaches, SRF operates entirely post-hoc, incurs zero inference overhead, and requires no architectural modifications. Across three families of VLMs (LLaVA-1.5, MiniGPT-4, and mPLUG-Owl2), SRF consistently reduces hallucination rates on MSCOCO, POPE-VQA, and other visual tasks benchmarks, achieving state-of-the-art faithfulness without degrading caption quality.

Method: DHMI clusters an auxiliary dataset to derive semantic hash centers as surrogate anchors, then uses surrogate-guided denoising optimization with a novel attack metric (fusing classification consistency and hash proximity) to dynamically select candidate samples. A cluster of surrogate models guides refinement to generate high-fidelity, semantically consistent images.

Result: Experiments on multiple datasets show DHMI successfully reconstructs high-resolution, high-quality images even under the most challenging black-box setting. It outperforms existing state-of-the-art model inversion attacks in black-box scenarios.

Conclusion: DHMI demonstrates practical efficacy and confirms the critical privacy risks inherent in deep hashing systems, highlighting the need for better privacy protection in deep hashing applications.

Abstract: Deep hashing improves retrieval efficiency through compact binary codes, yet it introduces severe and often overlooked privacy risks. The ability to reconstruct original training data from hash codes could lead to serious threats such as biometric forgery and privacy breaches. However, model inversion attacks specifically targeting deep hashing models remain unexplored, leaving their security implications unexamined. This research gap stems from the inaccessibility of genuine training hash codes and the highly discrete Hamming space, which prevents existing methods from adapting to deep hashing. To address these challenges, we propose DHMI, the first diffusion-based model inversion framework designed for deep hashing. DHMI first clusters an auxiliary dataset to derive semantic hash centers as surrogate anchors. It then introduces a surrogate-guided denoising optimization method that leverages a novel attack metric (fusing classification consistency and hash proximity) to dynamically select candidate samples. A cluster of surrogate models guides the refinement of these candidates, ensuring the generation of high-fidelity and semantically consistent images. Experiments on multiple datasets demonstrate that DHMI successfully reconstructs high-resolution, high-quality images even under the most challenging black-box setting, where no training hash codes are available. Our method outperforms the existing state-of-the-art model inversion attacks in black-box scenarios, confirming both its practical efficacy and the critical privacy risks inherent in deep hashing systems.

Method: Re-engineered core modules: ffmpeg for fast keyframe extraction, Vintern-1B-v3.5 for multilingual OCR, faster-whisper for real-time ASR, and lightweight vision-language models for efficient question answering. Also redesigned UI for better responsiveness and workflow.

Result: Retrieval time reduced by up to 75%, with increased accuracy and user satisfaction. System proved competitive for large-scale video search.

Conclusion: Fusionista2.0 successfully delivers a fast, accurate, and user-friendly video retrieval system optimized for time-constrained scenarios like VBS.

Abstract: The Video Browser Showdown (VBS) challenges systems to deliver accurate results under strict time constraints. To meet this demand, we present Fusionista2.0, a streamlined video retrieval system optimized for speed and usability. All core modules were re-engineered for efficiency: preprocessing now relies on ffmpeg for fast keyframe extraction, optical character recognition uses Vintern-1B-v3.5 for robust multilingual text recognition, and automatic speech recognition employs faster-whisper for real-time transcription. For question answering, lightweight vision-language models provide quick responses without the heavy cost of large models. Beyond these technical upgrades, Fusionista2.0 introduces a redesigned user interface with improved responsiveness, accessibility, and workflow efficiency, enabling even non-expert users to retrieve relevant content rapidly. Evaluations demonstrate that retrieval time was reduced by up to 75% while accuracy and user satisfaction both increased, confirming Fusionista2.0 as a competitive and user-friendly system for large-scale video search.

Method: Two-stage approach: Stage 1 learns Disease-Aware Semantic Tokens (DASTs) via cross-attention and multi-label classification with contrastive learning; Stage 2 uses Disease-Visual Attention Fusion (DVAF) and Dual-Modal Similarity Retrieval (DMSR) to integrate disease representations and retrieve relevant exemplars.

Result: Achieves state-of-the-art performance on CheXpert Plus, IU X-ray, and MIMIC-CXR datasets with significant improvements in clinical accuracy and linguistic quality.

Conclusion: The proposed disease-aware framework effectively addresses limitations in existing chest X-ray report generation methods by enhancing disease awareness and vision-language alignment.

Abstract: Radiology report generation from chest X-rays is an important task in artificial intelligence with the potential to greatly reduce radiologists’ workload and shorten patient wait times. Despite recent advances, existing approaches often lack sufficient disease-awareness in visual representations and adequate vision-language alignment to meet the specialized requirements of medical image analysis. As a result, these models usually overlook critical pathological features on chest X-rays and struggle to generate clinically accurate reports. To address these limitations, we propose a novel dual-stage disease-aware framework for chest X-ray report generation. In Stage1, our model learns Disease-Aware Semantic Tokens (DASTs) corresponding to specific pathology categories through cross-attention mechanisms and multi-label classification, while simultaneously aligning vision and language representations via contrastive learning. In Stage2, we introduce a Disease-Visual Attention Fusion (DVAF) module to integrate disease-aware representations with visual features, along with a Dual-Modal Similarity Retrieval (DMSR) mechanism that combines visual and disease-specific similarities to retrieve relevant exemplars, providing contextual guidance during report generation. Extensive experiments on benchmark datasets (i.e., CheXpert Plus, IU X-ray, and MIMIC-CXR) demonstrate that our disease-aware framework achieves state-of-the-art performance in chest X-ray report generation, with significant improvements in clinical accuracy and linguistic quality.

Method: Created CrossVid benchmark with hierarchical tasks covering four high-level dimensions and ten specific tasks, providing diverse question formats (single-choice, multiple-choice, open-ended) to comprehensively evaluate spatial-temporal reasoning in cross-video contexts.

Result: Gemini-2.5-Pro performed best with 50.4% average accuracy, but most MLLMs struggle with cross-video reasoning due to inability to integrate or compare evidence across multiple videos.

Conclusion: CrossVid reveals significant limitations in current MLLMs’ cross-video reasoning capabilities and provides a foundation for future improvements in this area.

Abstract: Cross-Video Reasoning (CVR) presents a significant challenge in video understanding, which requires simultaneous understanding of multiple videos to aggregate and compare information across groups of videos. Most existing video understanding benchmarks focus on single-video analysis, failing to assess the ability of multimodal large language models (MLLMs) to simultaneously reason over various videos. Recent benchmarks evaluate MLLMs’ capabilities on multi-view videos that capture different perspectives of the same scene. However, their limited tasks hinder a thorough assessment of MLLMs in diverse real-world CVR scenarios. To this end, we introduce CrossVid, the first benchmark designed to comprehensively evaluate MLLMs’ spatial-temporal reasoning ability in cross-video contexts. Firstly, CrossVid encompasses a wide spectrum of hierarchical tasks, comprising four high-level dimensions and ten specific tasks, thereby closely reflecting the complex and varied nature of real-world video understanding. Secondly, CrossVid provides 5,331 videos, along with 9,015 challenging question-answering pairs, spanning single-choice, multiple-choice, and open-ended question formats. Through extensive experiments on various open-source and closed-source MLLMs, we observe that Gemini-2.5-Pro performs best on CrossVid, achieving an average accuracy of 50.4%. Notably, our in-depth case study demonstrates that most current MLLMs struggle with CVR tasks, primarily due to their inability to integrate or compare evidence distributed across multiple videos for reasoning. These insights highlight the potential of CrossVid to guide future advancements in enhancing MLLMs’ CVR capabilities.

Method: Proposed ZoomEarth framework with adaptive cropping-zooming using Region-Guided reward, trained via supervised fine-tuning and Group Relative Policy Optimization on the LRS-GRO benchmark dataset containing 17 question types across global, region, and object levels.

Result: Achieves state-of-the-art performance on LRS-GRO and zero-shot performance on three public UHR remote sensing benchmarks. Can be integrated with downstream models for cloud removal, denoising, segmentation, and image editing.

Conclusion: ZoomEarth demonstrates strong versatility and extensibility through active perception paradigm, enabling effective processing of ultra-high-resolution remote sensing images with seamless integration into various downstream applications.

Abstract: Ultra-high-resolution (UHR) remote sensing (RS) images offer rich fine-grained information but also present challenges in effective processing. Existing dynamic resolution and token pruning methods are constrained by a passive perception paradigm, suffering from increased redundancy when obtaining finer visual inputs. In this work, we explore a new active perception paradigm that enables models to revisit information-rich regions. First, we present LRS-GRO, a large-scale benchmark dataset tailored for active perception in UHR RS processing, encompassing 17 question types across global, region, and object levels, annotated via a semi-automatic pipeline. Building on LRS-GRO, we propose ZoomEarth, an adaptive cropping-zooming framework with a novel Region-Guided reward that provides fine-grained guidance. Trained via supervised fine-tuning (SFT) and Group Relative Policy Optimization (GRPO), ZoomEarth achieves state-of-the-art performance on LRS-GRO and, in the zero-shot setting, on three public UHR remote sensing benchmarks. Furthermore, ZoomEarth can be seamlessly integrated with downstream models for tasks such as cloud removal, denoising, segmentation, and image editing through simple tool interfaces, demonstrating strong versatility and extensibility.

Method: Proposes TM-UNet with multi-scale token-memory (MSTM) blocks that transform 2D features into token sequences using spatial scanning and matrix memory cells to selectively retain contextual information with exponential gating and parallel pooling.

Result: TM-UNet outperforms state-of-the-art methods across diverse medical segmentation tasks with substantially reduced computation cost.

Conclusion: The proposed token-memory mechanism enables efficient global reasoning for medical segmentation with linear complexity, making it suitable for clinical deployment.

Abstract: Medical image segmentation is essential for clinical diagnosis and treatment planning. Although transformer-based methods have achieved remarkable results, their high computational cost hinders clinical deployment. To address this issue, we propose TM-UNet, a novel lightweight framework that integrates token sequence modeling with an efficient memory mechanism for efficient medical segmentation. Specifically, we introduce a multi-scale token-memory (MSTM) block that transforms 2D spatial features into token sequences through strategic spatial scanning, leveraging matrix memory cells to selectively retain and propagate discriminative contextual information across tokens. This novel token-memory mechanism acts as a dynamic knowledge store that captures long-range dependencies with linear complexity, enabling efficient global reasoning without redundant computation. Our MSTM block further incorporates exponential gating to identify token effectiveness and multi-scale contextual extraction via parallel pooling operations, enabling hierarchical representation learning without computational overhead. Extensive experiments demonstrate that TM-UNet outperforms state-of-the-art methods across diverse medical segmentation tasks with substantially reduced computation cost. The code is available at https://github.com/xq141839/TM-UNet.

Method: Uses decider tokens from previous denoising steps to build importance maps, maintains most salient tokens, and merges redundant ones through similarity-based aggregation. Features dynamic merge ratios that vary with each denoising step.

Result: Extensive experiments show D³ToM accelerates inference while preserving competitive performance under equivalent computational budgets.

Conclusion: D³ToM provides an effective plug-and-play solution for accelerating Diffusion MLLMs through dynamic token merging without altering model parameters.

Abstract: Diffusion-based multimodal large language models (Diffusion MLLMs) have recently demonstrated impressive non-autoregressive generative capabilities across vision-and-language tasks. However, Diffusion MLLMs exhibit substantially slower inference than autoregressive models: Each denoising step employs full bidirectional self-attention over the entire sequence, resulting in cubic decoding complexity that becomes computationally impractical with thousands of visual tokens. To address this challenge, we propose D$^{3}$ToM, a Decider-guided dynamic token merging method that dynamically merges redundant visual tokens at different denoising steps to accelerate inference in Diffusion MLLMs. At each denoising step, D$^{3}$ToM uses decider tokens-the tokens generated in the previous denoising step-to build an importance map over all visual tokens. Then it maintains a proportion of the most salient tokens and merges the remainder through similarity-based aggregation. This plug-and-play module integrates into a single transformer layer, physically shortening the visual token sequence for all subsequent layers without altering model parameters. Moreover, D$^{3}$ToM employs a merge ratio that dynamically varies with each denoising step, aligns with the native decoding process of Diffusion MLLMs, achieving superior performance under equivalent computational budgets. Extensive experiments show that D$^{3}$ToM accelerates inference while preserving competitive performance. The code is released at https://github.com/bcmi/D3ToM-Diffusion-MLLM.

Method: Uses a custom 3D calibration target with planes for LiDARs, ChArUco patterns for RGB cameras, and active LED patterns for event cameras, enabling simultaneous perception by all three modalities in a single calibration process.

Result: Validated through extensive experiments on custom autonomous driving dataset, demonstrating accurate and robust calibration performance across all three sensor types.

Conclusion: The proposed framework successfully enables one-shot joint extrinsic calibration of multi-modal sensor systems, providing a practical solution for complex autonomous driving vision systems where precise sensor alignment is critical.

Abstract: We present a novel multi-modal extrinsic calibration framework designed to simultaneously estimate the relative poses between event cameras, LiDARs, and RGB cameras, with particular focus on the challenging event camera calibration. Core of our approach is a novel 3D calibration target, specifically designed and constructed to be concurrently perceived by all three sensing modalities. The target encodes features in planes, ChArUco, and active LED patterns, each tailored to the unique characteristics of LiDARs, RGB cameras, and event cameras respectively. This unique design enables a one-shot, joint extrinsic calibration process, in contrast to existing approaches that typically rely on separate, pairwise calibrations. Our calibration pipeline is designed to accurately calibrate complex vision systems in the context of autonomous driving, where precise multi-sensor alignment is critical. We validate our approach through an extensive experimental evaluation on a custom built dataset, recorded with an advanced autonomous driving sensor setup, confirming the accuracy and robustness of our method.

Method: Proposes Frequency Recalibration (FreRec) with two components: Statistical High-frequency Replacement (SHR) for rough alignment and Reconstructive High-frequency Mapping (RHM) for quality enhancement and detail reconstruction.

Result: Extensive experiments on brain MRIs, chest X-rays, and fundus images show FreRec significantly improves medical image classification performance compared to uncalibrated AI-synthesized samples.

Conclusion: FreRec is an effective standalone post-processing method compatible with any generative model that reduces frequency distributional discrepancy and enhances GDA reliability in medical applications.

Abstract: Developing Medical AI relies on large datasets and easily suffers from data scarcity. Generative data augmentation (GDA) using AI generative models offers a solution to synthesize realistic medical images. However, the bias in GDA is often underestimated in medical domains, with concerns about the risk of introducing detrimental features generated by AI and harming downstream tasks. This paper identifies the frequency misalignment between real and synthesized images as one of the key factors underlying unreliable GDA and proposes the Frequency Recalibration (FreRec) method to reduce the frequency distributional discrepancy and thus improve GDA. FreRec involves (1) Statistical High-frequency Replacement (SHR) to roughly align high-frequency components and (2) Reconstructive High-frequency Mapping (RHM) to enhance image quality and reconstruct high-frequency details. Extensive experiments were conducted in various medical datasets, including brain MRIs, chest X-rays, and fundus images. The results show that FreRec significantly improves downstream medical image classification performance compared to uncalibrated AI-synthesized samples. FreRec is a standalone post-processing step that is compatible with any generative model and can integrate seamlessly with common medical GDA pipelines.

Method: Introduces a controllable LiDAR generation model using diffusion priors conditioned on coarsely extrapolated rendering to produce extra geometry-consistent scans, with an effective distillation mechanism for expansive reconstruction.

Result: Achieves state-of-the-art performance on multiple public datasets for both interpolated and extrapolated viewpoints, surpassing existing GS and NeRF-based methods.

Conclusion: LiDAR-GS++ enables real-time, high-fidelity LiDAR re-simulation with global geometric consistency while preserving detailed scene surfaces, addressing limitations of single-scan reconstruction.

Abstract: Recent GS-based rendering has made significant progress for LiDAR, surpassing Neural Radiance Fields (NeRF) in both quality and speed. However, these methods exhibit artifacts in extrapolated novel view synthesis due to the incomplete reconstruction from single traversal scans. To address this limitation, we present LiDAR-GS++, a LiDAR Gaussian Splatting reconstruction method enhanced by diffusion priors for real-time and high-fidelity re-simulation on public urban roads. Specifically, we introduce a controllable LiDAR generation model conditioned on coarsely extrapolated rendering to produce extra geometry-consistent scans and employ an effective distillation mechanism for expansive reconstruction. By extending reconstruction to under-fitted regions, our approach ensures global geometric consistency for extrapolative novel views while preserving detailed scene surfaces captured by sensors. Experiments on multiple public datasets demonstrate that LiDAR-GS++ achieves state-of-the-art performance for both interpolated and extrapolated viewpoints, surpassing existing GS and NeRF-based methods.

Method: An audio-driven online annotation platform where annotators narrate observations aloud while synchronously linking spoken phrases to image regions or 3D scene parts. The platform incorporates speech-to-text transcription and region-of-attention marking.

Result: Created a human-annotated multimodal dataset with 3,531 images, 898 3D scenes, 7,460 3D objects, and audio-aligned dense annotations in 20 languages. Models trained on this dataset showed 5% improvement in multilingual, 47% in cultural alignment, and 54% in 3D spatial capabilities.

Conclusion: DenseAnnotate provides a feasible approach for creating high-quality dense annotations that significantly enhance model performance across multiple domains, making it valuable for future vision-language research and various multimodal tasks.

Abstract: With the rapid adoption of multimodal large language models (MLLMs) across diverse applications, there is a pressing need for task-centered, high-quality training data. A key limitation of current training datasets is their reliance on sparse annotations mined from the Internet or entered via manual typing that capture only a fraction of an image’s visual content. Dense annotations are more valuable but remain scarce. Traditional text-based annotation pipelines are poorly suited for creating dense annotations: typing limits expressiveness, slows annotation speed, and underrepresents nuanced visual features, especially in specialized areas such as multicultural imagery and 3D asset annotation. In this paper, we present DenseAnnotate, an audio-driven online annotation platform that enables efficient creation of dense, fine-grained annotations for images and 3D assets. Annotators narrate observations aloud while synchronously linking spoken phrases to image regions or 3D scene parts. Our platform incorporates speech-to-text transcription and region-of-attention marking. To demonstrate the effectiveness of DenseAnnotate, we conducted case studies involving over 1,000 annotators across two domains: culturally diverse images and 3D scenes. We curate a human-annotated multi-modal dataset of 3,531 images, 898 3D scenes, and 7,460 3D objects, with audio-aligned dense annotations in 20 languages, including 8,746 image captions, 2,000 scene captions, and 19,000 object captions. Models trained on this dataset exhibit improvements of 5% in multilingual, 47% in cultural alignment, and 54% in 3D spatial capabilities. Our results show that our platform offers a feasible approach for future vision-language research and can be applied to various tasks and diverse types of data.

Method: Support-Exemplar-Query (SEQ) learning paradigm that structures training data into temporally coherent trajectories, enabling learning of class-specific temporal prototypes and sequence alignment via differentiable soft-DTW loss with multi-term objective for semantic consistency and temporal smoothness.

Result: Effective in both static and temporal tasks: enhances performance on fine-grained and ultra-fine-grained image classification, and delivers precise, temporally consistent predictions in video anomaly detection.

Conclusion: The approach bridges static and temporal learning in a modular, data-efficient manner using only simple classifiers on pre-extracted features, introducing temporal inductive bias through loss design alone.

Abstract: Real-world visual data rarely presents as isolated, static instances. Instead, it often evolves gradually over time through variations in pose, lighting, object state, or scene context. However, conventional classifiers are typically trained under the assumption of temporal independence, limiting their ability to capture such dynamics. We propose a simple yet effective framework that equips standard feedforward classifiers with temporal reasoning, all without modifying model architectures or introducing recurrent modules. At the heart of our approach is a novel Support-Exemplar-Query (SEQ) learning paradigm, which structures training data into temporally coherent trajectories. These trajectories enable the model to learn class-specific temporal prototypes and align prediction sequences via a differentiable soft-DTW loss. A multi-term objective further promotes semantic consistency and temporal smoothness. By interpreting input sequences as evolving feature trajectories, our method introduces a strong temporal inductive bias through loss design alone. This proves highly effective in both static and temporal tasks: it enhances performance on fine-grained and ultra-fine-grained image classification, and delivers precise, temporally consistent predictions in video anomaly detection. Despite its simplicity, our approach bridges static and temporal learning in a modular and data-efficient manner, requiring only a simple classifier on top of pre-extracted features.

Method: Uses LLM-driven agent workflow to extract design constraints, encodes them into grid-based representation, and applies coarse-to-fine integer programming optimization with corridor connectivity, room accessibility, and spatial exclusivity constraints.

Result: Significantly outperforms existing two-stage design pipelines in solution quality and achieves notable computational efficiency through the coarse-to-fine optimization strategy.

Conclusion: The joint optimization framework effectively combines LLM interpretation with mathematical optimization to produce high-quality interior designs efficiently.

Abstract: We present a novel framework for automated interior design that combines large language models (LLMs) with grid-based integer programming to jointly optimize room layout and furniture placement. Given a textual prompt, the LLM-driven agent workflow extracts structured design constraints related to room configurations and furniture arrangements. These constraints are encoded into a unified grid-based representation inspired by ``Modulor". Our formulation accounts for key design requirements, including corridor connectivity, room accessibility, spatial exclusivity, and user-specified preferences. To improve computational efficiency, we adopt a coarse-to-fine optimization strategy that begins with a low-resolution grid to solve a simplified problem and guides the solution at the full resolution. Experimental results across diverse scenarios demonstrate that our joint optimization approach significantly outperforms existing two-stage design pipelines in solution quality, and achieves notable computational efficiency through the coarse-to-fine strategy.

Method: Models negation as a subspace in joint embedding space using spherical caps around embeddings, scoring images by proximity to affirmative and distance from negated concepts.

Result: 30% average improvement on negation understanding across retrieval, MCQ, and text-to-image tasks, closing gap between affirmative and negated prompts.

Conclusion: Proposed training-free method effectively handles negation in VLMs while preserving zero-shot capabilities that fine-tuned models lose.

Abstract: Vision-Language Models (VLMs) struggle with negation. Given a prompt like “retrieve (or generate) a street scene without pedestrians,” they often fail to respect the “not.” Existing methods address this limitation by fine-tuning on large negation datasets, but such retraining often compromises the model’s zero-shot performance on affirmative prompts. We show that the embedding space of VLMs, such as CLIP, can be divided into semantically consistent subspaces. Based on this property, we propose a training-free framework that models negation as a subspace in the joint embedding space rather than a single point (Figure 1). To find the matching image for a caption such as “A but not N,” we construct two spherical caps around the embeddings of A and N, and we score images by the central direction of the region that is close to A and far from N. Across retrieval, MCQ, and text-to-image tasks, our method improves negation understanding by about 30% on average over prior methods. It closes the gap between affirmative and negated prompts while preserving the zero-shot performance that fine-tuned models fail to maintain. Code will be released upon publication.

Method: Back-projecting vehicle detections from infrastructure cameras to real-world 3D ground plane coordinates, with both single-camera and multi-camera weak fusion approaches.

Result: Back-projection yields more accurate trajectory classification and turning movement counts than image plane analysis, with multi-camera fusion achieving even higher accuracy.

Conclusion: Traffic should be analyzed on the ground plane rather than the image plane for more accurate turning movement counts.

Abstract: Accurate turning movement counts at intersections are important for signal control, traffic management and urban planning. Computer vision systems for automatic turning movement counts typically rely on visual analysis in the image plane of an infrastructure camera. Here we explore potential advantages of back-projecting vehicles detected in one or more infrastructure cameras to the ground plane for analysis in real-world 3D coordinates. For single-camera systems we find that back-projection yields more accurate trajectory classification and turning movement counts. We further show that even higher accuracy can be achieved through weak fusion of back-projected detections from multiple cameras. These results suggeest that traffic should be analyzed on the ground plane, not the image plane

Method: Proposes CLAReSNet with multi-scale convolutional stem with deep residual blocks and enhanced CBAM for spatial features, followed by spectral encoder combining bidirectional RNNs with Multi-Scale Spectral Latent Attention (MSLA) that reduces complexity from O(T²D) to O(Tlog(T)D) via adaptive latent token allocation.

Result: Achieved state-of-the-art performance on Indian Pines (99.71% accuracy) and Salinas (99.96% accuracy) datasets, significantly surpassing HybridSN, SSRN, and SpectralFormer. Learned embeddings show superior inter-class separability and compact intra-class clustering.

Conclusion: CLAReSNet effectively addresses HSI classification challenges under limited samples and severe class imbalance through its hybrid architecture that integrates multi-scale convolutional extraction with transformer-style attention via adaptive latent bottleneck.

Abstract: Hyperspectral image (HSI) classification faces critical challenges, including high spectral dimensionality, complex spectral-spatial correlations, and limited training samples with severe class imbalance. While CNNs excel at local feature extraction and transformers capture long-range dependencies, their isolated application yields suboptimal results due to quadratic complexity and insufficient inductive biases. We propose CLAReSNet (Convolutional Latent Attention Residual Spectral Network), a hybrid architecture that integrates multi-scale convolutional extraction with transformer-style attention via an adaptive latent bottleneck. The model employs a multi-scale convolutional stem with deep residual blocks and an enhanced Convolutional Block Attention Module for hierarchical spatial features, followed by spectral encoder layers combining bidirectional RNNs (LSTM/GRU) with Multi-Scale Spectral Latent Attention (MSLA). MSLA reduces complexity from $\mathcal{O}(T^2D)$ to $\mathcal{O}(T\log(T)D)$ by adaptive latent token allocation (8-64 tokens) that scales logarithmically with the sequence length. Hierarchical cross-attention fusion dynamically aggregates multi-level representations for robust classification. Experiments conducted on the Indian Pines and Salinas datasets show state-of-the-art performance, achieving overall accuracies of 99.71% and 99.96%, significantly surpassing HybridSN, SSRN, and SpectralFormer. The learned embeddings exhibit superior inter-class separability and compact intra-class clustering, validating CLAReSNet’s effectiveness under limited samples and severe class imbalance.

Method: Created a benchmark with ~3,000 annotated triplets from various image generation models and MLLMs as detectors, using MLLMs as reward models to judge explanation quality.

Result: Best reward model scored 88.76% on the benchmark, while human inter-annotator agreement reached 98.30%, showing a significant performance gap between MLLMs and humans.

Conclusion: There remains a visible gap between current MLLMs’ reasoning abilities and human-level performance in judging explanations for AI-generated image detection, with common pitfalls identified.

Abstract: Conventional, classification-based AI-generated image detection methods cannot explain why an image is considered real or AI-generated in a way a human expert would, which reduces the trustworthiness and persuasiveness of these detection tools for real-world applications. Leveraging Multimodal Large Language Models (MLLMs) has recently become a trending solution to this issue. Further, to evaluate the quality of generated explanations, a common approach is to adopt an “MLLM as a judge” methodology to evaluate explanations generated by other MLLMs. However, how well those MLLMs perform when judging explanations for AI-generated image detection generated by themselves or other MLLMs has not been well studied. We therefore propose \textbf{XAIGID-RewardBench}, the first benchmark designed to evaluate the ability of current MLLMs to judge the quality of explanations about whether an image is real or AI-generated. The benchmark consists of approximately 3,000 annotated triplets sourced from various image generation models and MLLMs as policy models (detectors) to assess the capabilities of current MLLMs as reward models (judges). Our results show that the current best reward model scored 88.76% on this benchmark (while human inter-annotator agreement reaches 98.30%), demonstrating that a visible gap remains between the reasoning abilities of today’s MLLMs and human-level performance. In addition, we provide an analysis of common pitfalls that these models frequently encounter. Code and benchmark are available at https://github.com/RewardBench/XAIGID-RewardBench.

Method: Train large language models using GRPO with a novel reward that validates structural integrity and output accuracy to construct digital twin representations of visual inputs.

Result: DT-R1 achieves consistent improvements over state-of-the-art task-specific models across six visual reasoning benchmarks covering two modalities and four task types.

Conclusion: DT-R1 demonstrates that visual reasoning can emerge from reinforcement learning with digital twin representations, opening a new research direction.

Abstract: Visual reasoning may require models to interpret images and videos and respond to implicit text queries across diverse output formats, from pixel-level segmentation masks to natural language descriptions. Existing approaches rely on supervised fine-tuning with task-specific architectures. For example, reasoning segmentation, grounding, summarization, and visual question answering each demand distinct model designs and training, preventing unified solutions and limiting cross-task and cross-modality generalization. Hence, we propose DT-R1, a reinforcement learning framework that trains large language models to construct digital twin representations of complex multi-modal visual inputs and then reason over these high-level representations as a unified approach to visual reasoning. Specifically, we train DT-R1 using GRPO with a novel reward that validates both structural integrity and output accuracy. Evaluations in six visual reasoning benchmarks, covering two modalities and four task types, demonstrate that DT-R1 consistently achieves improvements over state-of-the-art task-specific models. DT-R1 opens a new direction where visual reasoning emerges from reinforcement learning with digital twin representations.

Method: Proposes FastReasonSeg using digital twin representations to decouple perception from reasoning. Uses two-stage distillation: supervised fine-tuning on teacher-generated reasoning chains followed by reinforcement fine-tuning with joint rewards for segmentation accuracy and reasoning quality.

Result: Achieves state-of-the-art performance on two video (JiTBench, RVTBench) and two image benchmarks (ReasonSeg, LLM-Seg40K). The distilled 0.6B variant outperforms models with 20x more parameters while achieving 7.79 FPS throughput with only 2.1GB memory consumption.

Conclusion: FastReasonSeg enables efficient deployment in resource-constrained environments for real-time reasoning segmentation, making it suitable for embodied agents operating autonomously in real-world environments.

Abstract: Reasoning segmentation enables open-set object segmentation via implicit text queries, therefore serving as a foundation for embodied agents that should operate autonomously in real-world environments. However, existing methods for reasoning segmentation require multimodal large language models with billions of parameters that exceed the computational capabilities of edge devices that typically deploy the embodied AI systems. Distillation offers a pathway to compress these models while preserving their capabilities. Yet, existing distillation approaches fail to transfer the multi-step reasoning capabilities that reasoning segmentation demands, as they focus on matching output predictions and intermediate features rather than preserving reasoning chains. The emerging paradigm of reasoning over digital twin representations presents an opportunity for more effective distillation by re-framing the problem. Consequently, we propose FastReasonSeg, which employs digital twin representations that decouple perception from reasoning to enable more effective distillation. Our distillation scheme first relies on supervised fine-tuning on teacher-generated reasoning chains. Then it is followed by reinforcement fine-tuning with joint rewards evaluating both segmentation accuracy and reasoning quality alignment. Experiments on two video (JiTBench, RVTBench) and two image benchmarks (ReasonSeg, LLM-Seg40K) demonstrate that our FastReasonSeg achieves state-of-the-art reasoning segmentation performance. Moreover, the distilled 0.6B variant outperforms models with 20 times more parameters while achieving 7.79 FPS throughput with only 2.1GB memory consumption. This efficiency enables deployment in resource-constrained environments to enable real-time reasoning segmentation.

Method: Self-supervised fusion loss to infer scene changes from multiple cues, PnP-based fast pose estimation against reference scene, and fast change-guided update strategy for 3D Gaussian Splatting scene representation.

Result: Outperforms both online and offline baselines on complex real-world datasets, achieving new state-of-the-art performance while operating at over 10 FPS.

Conclusion: The approach successfully addresses the challenges of online scene change detection by combining efficient pose estimation, self-supervised learning, and optimized scene representation updates.

Abstract: Online Scene Change Detection (SCD) is an extremely challenging problem that requires an agent to detect relevant changes on the fly while observing the scene from unconstrained viewpoints. Existing online SCD methods are significantly less accurate than offline approaches. We present the first online SCD approach that is pose-agnostic, label-free, and ensures multi-view consistency, while operating at over 10 FPS and achieving new state-of-the-art performance, surpassing even the best offline approaches. Our method introduces a new self-supervised fusion loss to infer scene changes from multiple cues and observations, PnP-based fast pose estimation against the reference scene, and a fast change-guided update strategy for the 3D Gaussian Splatting scene representation. Extensive experiments on complex real-world datasets demonstrate that our approach outperforms both online and offline baselines.

Method: Two-stage framework: 1) compositional alignment between sub-queries and digital twin representations for candidate identification, 2) LLM-based reasoning with just-in-time refinement using specialist models to address information gaps.

Result: Achieves 81.2% R@1 on ReasonT2VBench-135 (outperforming strongest baseline by >50 percentage points), 81.7% R@1 on ReasonT2VBench-1000, and state-of-the-art results on MSR-VTT, MSVD, and VATEX benchmarks.

Conclusion: The proposed reasoning text-to-video retrieval paradigm successfully handles implicit queries through structured scene representations and LLM reasoning, significantly outperforming existing methods.

Abstract: The goal of text-to-video retrieval is to search large databases for relevant videos based on text queries. Existing methods have progressed to handling explicit queries where the visual content of interest is described explicitly; however, they fail with implicit queries where identifying videos relevant to the query requires reasoning. We introduce reasoning text-to-video retrieval, a paradigm that extends traditional retrieval to process implicit queries through reasoning while providing object-level grounding masks that identify which entities satisfy the query conditions. Instead of relying on vision-language models directly, we propose representing video content as digital twins, i.e., structured scene representations that decompose salient objects through specialist vision models. This approach is beneficial because it enables large language models to reason directly over long-horizon video content without visual token compression. Specifically, our two-stage framework first performs compositional alignment between decomposed sub-queries and digital twin representations for candidate identification, then applies large language model-based reasoning with just-in-time refinement that invokes additional specialist models to address information gaps. We construct a benchmark of 447 manually created implicit queries with 135 videos (ReasonT2VBench-135) and another more challenging version of 1000 videos (ReasonT2VBench-1000). Our method achieves 81.2% R@1 on ReasonT2VBench-135, outperforming the strongest baseline by greater than 50 percentage points, and maintains 81.7% R@1 on the extended configuration while establishing state-of-the-art results in three conventional benchmarks (MSR-VTT, MSVD, and VATEX).

Method: Proposed AGGRNet framework that extracts both informative and non-informative features to better understand fine-grained visual patterns in medical images.

Result: Achieves state-of-the-art performance on various medical imaging datasets, with up to 5% improvement over SOTA models on the Kvasir dataset.

Conclusion: AGGRNet effectively addresses challenges in complex medical image analysis by capturing fine-grained visual patterns through dual feature extraction, leading to improved classification accuracy.

Abstract: Medical image analysis for complex tasks such as severity grading and disease subtype classification poses significant challenges due to intricate and similar visual patterns among classes, scarcity of labeled data, and variability in expert interpretations. Despite the usefulness of existing attention-based models in capturing complex visual patterns for medical image classification, underlying architectures often face challenges in effectively distinguishing subtle classes since they struggle to capture inter-class similarity and intra-class variability, resulting in incorrect diagnosis. To address this, we propose AGGRNet framework to extract informative and non-informative features to effectively understand fine-grained visual patterns and improve classification for complex medical image analysis tasks. Experimental results show that our model achieves state-of-the-art performance on various medical imaging datasets, with the best improvement up to 5% over SOTA models on the Kvasir dataset.

Method: Combines pretrained ResNet50 encoder with Quantum CNN, using image preprocessing (denoising, CLAHE), data augmentation, and angle encoding to transform features into qubits. Evaluated on 8-qubit and 12-qubit configurations.

Result: Both configurations achieved 0.99 test accuracy with stable convergence. 12-qubit model showed improved recall and precision - perfect recall for cysts and 0.9956 F1-score for tumors. Very few misclassifications across all classes.

Conclusion: Integration of classical preprocessing and deep feature extraction with quantum circuits enhances medical diagnostic performance for kidney condition classification.

Abstract: The objective of this study is to diagnose and differentiate kidney stones, cysts, and tumors using Computed Tomography (CT) images of the kidney. This study leverages a hybrid quantum-classical framework in this regard. We combine a pretrained ResNet50 encoder, with a Quantum Convolutional Neural Network (QCNN) to explore quantum-assisted diagnosis. We pre-process the kidney images using denoising and contrast limited adaptive histogram equalization to enhance feature extraction. We address class imbalance through data augmentation and weighted sampling. Latent features extracted by the encoder are transformed into qubits via angle encoding and processed by a QCNN. The model is evaluated on both 8-qubit and 12-qubit configurations. Both architectures achieved rapid convergence with stable learning curves and high consistency between training and validation performance. The models reached a test accuracy of 0.99, with the 12-qubit configuration providing improvements in overall recall and precision, particularly for Cyst and Tumor detection, where it achieved perfect recall for Cysts and a tumor F1-score of 0.9956. Confusion matrix analysis further confirmed reliable classification behavior across all classes, with very few misclassifications. Results demonstrate that integrating classical pre-processing and deep feature extraction with quantum circuits enhances medical diagnostic performance.

Method: Uncertainty-Guided Inference-Time Selection framework using regularized global density model for aleatoric uncertainty, and three orthogonal components (local support deficiency, manifold spectral collapse, cross-layer feature inconsistency) for epistemic uncertainty without sampling, ensembling, or additional forward passes.

Result: 60% compute reduction on MOT17 with negligible accuracy loss; 13.6 percentage point improvement in computational savings over total-uncertainty baseline; significantly tighter prediction intervals at matched coverage through conformal calibration.

Conclusion: The proposed orthogonal uncertainty decomposition enables practical self-regulating visual inference through uncertainty-guided adaptive model selection, providing substantial computational savings while maintaining performance.

Abstract: Most estimators collapse all uncertainty modes into a single confidence score, preventing reliable reasoning about when to allocate more compute or adjust inference. We introduce Uncertainty-Guided Inference-Time Selection, a lightweight inference time framework that disentangles aleatoric (data-driven) and epistemic (model-driven) uncertainty directly in deep feature space. Aleatoric uncertainty is estimated using a regularized global density model, while epistemic uncertainty is formed from three complementary components that capture local support deficiency, manifold spectral collapse, and cross-layer feature inconsistency. These components are empirically orthogonal and require no sampling, no ensembling, and no additional forward passes. We integrate the decomposed uncertainty into a distribution free conformal calibration procedure that yields significantly tighter prediction intervals at matched coverage. Using these components for uncertainty guided adaptive model selection reduces compute by approximately 60 percent on MOT17 with negligible accuracy loss, enabling practical self regulating visual inference. Additionally, our ablation results show that the proposed orthogonal uncertainty decomposition consistently yields higher computational savings across all MOT17 sequences, improving margins by 13.6 percentage points over the total-uncertainty baseline.

Method: Combines low-rank linear projection with multi-scale nonlinear transformation that jointly modulates spatial and channel attention, fused through pointwise multiplication and residual connection.

Result: Consistently improves transfer performance on classification, detection, and segmentation tasks with <5% backbone parameters, enabling stable optimization and fast convergence.

Conclusion: MSLoRA provides a simple and universal approach for efficient adaptation of frozen vision backbones by reweighting rather than re-tuning features.

Abstract: We introduce MSLoRA, a backbone-agnostic, parameter-efficient adapter that reweights feature responses rather than re-tuning the underlying backbone. Existing low-rank adaptation methods are mostly confined to vision transformers (ViTs) and struggle to generalize across architectures. MSLoRA unifies adaptation for both convolutional neural networks (CNNs) and ViTs by combining a low-rank linear projection with a multi-scale nonlinear transformation that jointly modulates spatial and channel attention. The two components are fused through pointwise multiplication and a residual connection, yielding a lightweight module that shifts feature attention while keeping pretrained weights frozen. Extensive experiments demonstrate that MSLoRA consistently improves transfer performance on classification, detection, and segmentation tasks with roughly less than 5% of backbone parameters. The design further enables stable optimization, fast convergence, and strong cross-architecture generalization. By reweighting rather than re-tuning, MSLoRA provides a simple and universal approach for efficient adaptation of frozen vision backbones.

Method: Integrates open-world perception using frozen vision-language models for detection/segmentation, Q-Former bottleneck for feature aggregation, and vision-action contrastive learning to align vision-language and action embeddings.

Result: Demonstrates strong generalization and exploratory performance in unstructured, unseen environments on real-world robotic platforms, even with limited training data.

Conclusion: The proposed VLA-R framework effectively handles open-world autonomous driving by combining vision-language perception with vision-action retrieval, enabling transferable driving behaviors in unfamiliar environments.

Abstract: Exploring open-world situations in an end-to-end manner is a promising yet challenging task due to the need for strong generalization capabilities. In particular, end-to-end autonomous driving in unstructured outdoor environments often encounters conditions that were unfamiliar during training. In this work, we present Vision-Language Action Retrieval (VLA-R), an open-world end-to-end autonomous driving (OW-E2EAD) framework that integrates open-world perception with a novel vision-action retrieval paradigm. We leverage a frozen vision-language model for open-world detection and segmentation to obtain multi-scale, prompt-guided, and interpretable perception features without domain-specific tuning. A Q-Former bottleneck aggregates fine-grained visual representations with language-aligned visual features, bridging perception and action domains. To learn transferable driving behaviors, we introduce a vision-action contrastive learning scheme that aligns vision-language and action embeddings for effective open-world reasoning and action retrieval. Our experiments on a real-world robotic platform demonstrate strong generalization and exploratory performance in unstructured, unseen environments, even with limited data. Demo videos are provided in the supplementary material.

Method: Introduces Self-supervised Prompt Enhancement Module (SPEM) to derive defect-aware prompts from unlabeled target data, and Domain-Aware Prompt Alignment (DAPA) to align prompt-conditioned source and target representations.

Result: Outperforms supervised, self-supervised, and adaptation baselines on four benchmarks, achieving robust zero-shot transfer, improved domain resilience, and high data efficiency in few-shot adaptation.

Conclusion: Self-supervised prompting is a practical direction for building scalable and adaptive visual inspection systems.

Abstract: The deployment of automated pavement defect detection is often hindered by poor cross-domain generalization. Supervised detectors achieve strong in-domain accuracy but require costly re-annotation for new environments, while standard self-supervised methods capture generic features and remain vulnerable to domain shift. We propose \ours, a self-supervised framework that \emph{visually probes} target domains without labels. \ours introduces a Self-supervised Prompt Enhancement Module (SPEM), which derives defect-aware prompts from unlabeled target data to guide a frozen ViT backbone, and a Domain-Aware Prompt Alignment (DAPA) objective, which aligns prompt-conditioned source and target representations. Experiments on four challenging benchmarks show that \ours consistently outperforms strong supervised, self-supervised, and adaptation baselines, achieving robust zero-shot transfer, improved resilience to domain variations, and high data efficiency in few-shot adaptation. These results highlight self-supervised prompting as a practical direction for building scalable and adaptive visual inspection systems. Source code is publicly available: https://github.com/xixiaouab/PROBE/tree/main

Method: Proposes a rotation-only optimization framework based on reprojection error that expresses translation in terms of rotation, condensing imaging geometry representation onto rotation manifold for both two-view and multi-view scenarios.

Result: Demonstrates superior accuracy and robustness over current state-of-the-art rotation estimation methods, with performance comparable to multiple bundle adjustment iterations.

Conclusion: This work contributes to more accurate, efficient and reliable 3D visual computing by leveraging rotation-only optimization in Structure from Motion.

Abstract: Structure from Motion (SfM) is a critical task in computer vision, aiming to recover the 3D scene structure and camera motion from a sequence of 2D images. The recent pose-only imaging geometry decouples 3D coordinates from camera poses and demonstrates significantly better SfM performance through pose adjustment. Continuing the pose-only perspective, this paper explores the critical relationship between the scene structures, rotation and translation. Notably, the translation can be expressed in terms of rotation, allowing us to condense the imaging geometry representation onto the rotation manifold. A rotation-only optimization framework based on reprojection error is proposed for both two-view and multi-view scenarios. The experiment results demonstrate superior accuracy and robustness performance over the current state-of-the-art rotation estimation methods, even comparable to multiple bundle adjustment iteration results. Hopefully, this work contributes to even more accurate, efficient and reliable 3D visual computing.

Method: Proposes DHGM, a Diffusion-based High-frequency Guided Model that integrates pre-trained diffusion priors with high-pass filters to simultaneously remove rain artifacts and enhance structural details in a unified framework.

Result: Extensive experiments demonstrate that DHGM achieves superior performance over existing methods with lower computational costs, effectively generating clean and high-resolution images.

Conclusion: DHGM successfully addresses the conflict between weather removal and super-resolution by using diffusion priors with high-frequency guidance, providing an effective solution for generating high-quality images suitable for tasks like small object detection.

Abstract: Clean images are crucial for visual tasks such as small object detection, especially at high resolutions. However, real-world images are often degraded by adverse weather, and weather restoration methods may sacrifice high-frequency details critical for analyzing small objects. A natural solution is to apply super-resolution (SR) after weather removal to recover both clarity and fine structures. However, simply cascading restoration and SR struggle to bridge their inherent conflict: removal aims to remove high-frequency weather-induced noise, while SR aims to hallucinate high-frequency textures from existing details, leading to inconsistent restoration contents. In this paper, we take deraining as a case study and propose DHGM, a Diffusion-based High-frequency Guided Model for generating clean and high-resolution images. DHGM integrates pre-trained diffusion priors with high-pass filters to simultaneously remove rain artifacts and enhance structural details. Extensive experiments demonstrate that DHGM achieves superior performance over existing methods, with lower costs.

Method: Compression-expansion strategy: compress multi-layer ResNet stages to 1-2 MeanFlow modules, then selectively expand first three stages to baseline configuration while keeping last stage in MeanFlow form, followed by fine-tuning.

Result: Reduced parameters by 46.28% and 45.59% compared to ResNet-50 on CIFAR-10 and CIFAR-100 respectively, while improving accuracy by 0.23% and 0.17%.

Conclusion: Generative flow-fields can effectively characterize ResNet’s feature transformations, providing new insights into the relationship between generative modeling and discriminative learning.

Abstract: ResNet has achieved tremendous success in computer vision through its residual connection mechanism. ResNet can be viewed as a discretized form of ordinary differential equations (ODEs). From this perspective, the multiple residual blocks within a single ResNet stage essentially perform multi-step discrete iterations of the feature transformation for that stage. The recently proposed flow matching model, MeanFlow, enables one-step generative modeling by learning the mean velocity field to transform distributions. Inspired by this, we propose MeanFlow-Incubated ResNet (MFI-ResNet), which employs a compression-expansion strategy to jointly improve parameter efficiency and discriminative performance. In the compression phase, we simplify the multi-layer structure within each ResNet stage to one or two MeanFlow modules to construct a lightweight meta model. In the expansion phase, we apply a selective incubation strategy to the first three stages, expanding them to match the residual block configuration of the baseline ResNet model, while keeping the last stage in MeanFlow form, and fine-tune the incubated model. Experimental results show that on CIFAR-10 and CIFAR-100 datasets, MFI-ResNet achieves remarkable parameter efficiency, reducing parameters by 46.28% and 45.59% compared to ResNet-50, while still improving accuracy by 0.23% and 0.17%, respectively. This demonstrates that generative flow-fields can effectively characterize the feature transformation process in ResNet, providing a new perspective for understanding the relationship between generative modeling and discriminative learning.

Method: Proposes RedVTP that estimates visual token importance using attention from masked response tokens and prunes less important tokens after the first inference step, leveraging consistent importance scores across steps.

Result: Improves token generation throughput by up to 186% for LLaDA-V and 28.05% for LaViDa, reduces inference latency by up to 64.97% and 21.87% respectively, without compromising accuracy.

Conclusion: RedVTP effectively addresses the computational inefficiency of DVLMs through response-driven visual token pruning, achieving substantial speed improvements while maintaining or even improving model accuracy.

Abstract: Vision-Language Models (VLMs) have achieved remarkable progress in multimodal reasoning and generation, yet their high computational demands remain a major challenge. Diffusion Vision-Language Models (DVLMs) are particularly attractive because they enable parallel token decoding, but the large number of visual tokens still significantly hinders their inference efficiency. While visual token pruning has been extensively studied for autoregressive VLMs (AVLMs), it remains largely unexplored for DVLMs. In this work, we propose RedVTP, a response-driven visual token pruning strategy that leverages the inference dynamics of DVLMs. Our method estimates visual token importance using attention from the masked response tokens. Based on the observation that these importance scores remain consistent across steps, RedVTP prunes the less important visual tokens from the masked tokens after the first inference step, thereby maximizing inference efficiency. Experiments show that RedVTP improves token generation throughput of LLaDA-V and LaViDa by up to 186% and 28.05%, respectively, and reduces inference latency by up to 64.97% and 21.87%, without compromising-and in some cases improving-accuracy.

Method: Proposes UP-Fusion with three modules: Semantic-Aware Channel Pruning Module (SCPM) for filtering redundant features using pre-trained knowledge, Geometric Affine Modulation Module (GAM) for maintaining modal discriminability, and Text-Guided Channel Perturbation Module (TCPM) for reshaping channel distribution during decoding.

Result: Extensive experiments show the proposed algorithm outperforms existing methods on both multi-modality image fusion and downstream tasks.

Conclusion: UP-Fusion effectively addresses gradient conflicts and generalization limitations in multi-modality image fusion through channel perturbation and pre-trained knowledge integration.

Abstract: Multi-modality image fusion enhances scene perception by combining complementary information. Unified models aim to share parameters across modalities for multi-modality image fusion, but large modality differences often cause gradient conflicts, limiting performance. Some methods introduce modality-specific encoders to enhance feature perception and improve fusion quality. However, this strategy reduces generalisation across different fusion tasks. To overcome this limitation, we propose a unified multi-modality image fusion framework based on channel perturbation and pre-trained knowledge integration (UP-Fusion). To suppress redundant modal information and emphasize key features, we propose the Semantic-Aware Channel Pruning Module (SCPM), which leverages the semantic perception capability of a pre-trained model to filter and enhance multi-modality feature channels. Furthermore, we proposed the Geometric Affine Modulation Module (GAM), which uses original modal features to apply affine transformations on initial fusion features to maintain the feature encoder modal discriminability. Finally, we apply a Text-Guided Channel Perturbation Module (TCPM) during decoding to reshape the channel distribution, reducing the dependence on modality-specific channels. Extensive experiments demonstrate that the proposed algorithm outperforms existing methods on both multi-modality image fusion and downstream tasks.

Method: Uses deep convolutional neural networks (DCNNs) with OpenCV to analyze real-time facial images, detecting facial landmarks including eye openings and yawn-like mouth movements.

Result: Achieved 99.6% accuracy on NTHU-DDD dataset and 97% accuracy on Yawn-Eye-Dataset for drowsiness detection classification.

Conclusion: The proposed system offers a non-invasive, inexpensive, and cost-effective solution for real-time drowsiness detection that can be embedded in smart car technology to enhance road safety.

Abstract: A long road trip is fun for drivers. However, a long drive for days can be tedious for a driver to accommodate stringent deadlines to reach distant destinations. Such a scenario forces drivers to drive extra miles, utilizing extra hours daily without sufficient rest and breaks. Once a driver undergoes such a scenario, it occasionally triggers drowsiness during driving. Drowsiness in driving can be life-threatening to any individual and can affect other drivers’ safety; therefore, a real-time detection system is needed. To identify fatigued facial characteristics in drivers and trigger the alarm immediately, this research develops a real-time driver drowsiness detection system utilizing deep convolutional neural networks (DCNNs) and OpenCV.Our proposed and implemented model takes real- time facial images of a driver using a live camera and utilizes a Python-based library named OpenCV to examine the facial images for facial landmarks like sufficient eye openings and yawn-like mouth movements. The DCNNs framework then gathers the data and utilizes a per-trained model to detect the drowsiness of a driver using facial landmarks. If the driver is identified as drowsy, the system issues a continuous alert in real time, embedded in the Smart Car technology.By potentially saving innocent lives on the roadways, the proposed technique offers a non-invasive, inexpensive, and cost-effective way to identify drowsiness. Our proposed and implemented DCNNs embedded drowsiness detection model successfully react with NTHU-DDD dataset and Yawn-Eye-Dataset with drowsiness detection classification accuracy of 99.6% and 97% respectively.

Method: Test-time training approach that updates only continuous soft prompts, identifies question-relevant regions through visual chain-of-thought, and encourages answer consistency across original image and localized crop. Label-free and plug-and-play with frozen backbones.

Result: Improves medical VQA performance significantly - adding CoTBox-TTT to LLaVA increases closed-ended accuracy by 12.3% on pathVQA.

Conclusion: The approach is practical for real deployments in medical VQA, providing reliable adaptation at inference time without requiring retraining or additional labels.

Abstract: Medical visual question answering could support clinical decision making, yet current systems often fail under domain shift and produce answers that are weakly grounded in image evidence. This reliability gap arises when models attend to spurious regions and when retraining or additional labels are impractical at deployment time. We address this setting with CoTBox-TTT, an evidence-first test-time training approach that adapts a vision-language model at inference while keeping all backbones frozen. The method updates only a small set of continuous soft prompts. It identifies question-relevant regions through a visual chain-of-thought signal and encourages answer consistency across the original image and a localized crop. The procedure is label free, and plug and play with diverse backbones. Experiments on medical VQA show that the approach is practical for real deployments. For instance, adding CoTBox-TTT to LLaVA increases closed-ended accuracy by 12.3% on pathVQA.

Method: Proposes three components: (1) Modality-driven Mixture-of-Experts for adaptive processing based on modality composition, (2) Dual-level Alignment to leverage semantic alignment within products, and (3) MLLM-based Image-text Co-augmentation with Dynamic Sample Filtering for data quality improvement.

Result: Achieves state-of-the-art zero-shot performance on MBE2.0 benchmark and multiple public datasets. Attention-based heatmap visualization shows improved multimodal alignment.

Conclusion: MOON2.0 effectively addresses key challenges in e-commerce multimodal representation learning and demonstrates superior performance through balanced modality processing and enhanced alignment strategies.

Abstract: The rapid growth of e-commerce calls for multimodal models that comprehend rich visual and textual product information. Although recent multimodal large language models (MLLMs) for product understanding exhibit strong capability in representation learning for e-commerce, they still face three challenges: (i) the modality imbalance induced by modality mixed training; (ii) underutilization of the intrinsic alignment relationships among visual and textual information within a product; and (iii) limited handling of noise in e-commerce multimodal data. To address these, we propose MOON2.0, a dynamic modality-balanced multimodal representation learning framework for e-commerce product understanding. MOON2.0 comprises: (1) a Modality-driven Mixture-of-Experts (MoE) module that adaptively processes input samples by their modality composition, enabling Multimodal Joint Learning to mitigate the modality imbalance; (2) a Dual-level Alignment method to better leverage semantic alignment properties inside individual products; and (3) an MLLM-based Image-text Co-augmentation strategy that integrates textual enrichment with visual expansion, coupled with Dynamic Sample Filtering to improve training data quality. We further introduce MBE2.0, a co-augmented multimodal representation benchmark for e-commerce representation learning and evaluation. Experiments show that MOON2.0 delivers state-of-the-art zero-shot performance on MBE2.0 and multiple public datasets. Furthermore, attention-based heatmap visualization provides qualitative evidence of improved multimodal alignment of MOON2.0.

Method: Proposes MaskAnyNet that combines masking with a relearning mechanism, adding an additional branch to any model to jointly learn from recomposed masked regions, treating masked content as auxiliary knowledge.

Result: Experiments on CNN and Transformer backbones show consistent performance gains across multiple benchmarks, with analysis confirming improved semantic diversity through masked content reuse.

Conclusion: Masked regions should be treated as valuable auxiliary knowledge rather than ignored, and the proposed approach effectively leverages semantic diversity from masked content to enrich features and preserve fine-grained details.

Abstract: In supervised learning, traditional image masking faces two key issues: (i) discarded pixels are underutilized, leading to a loss of valuable contextual information; (ii) masking may remove small or critical features, especially in fine-grained tasks. In contrast, masked image modeling (MIM) has demonstrated that masked regions can be reconstructed from partial input, revealing that even incomplete data can exhibit strong contextual consistency with the original image. This highlights the potential of masked regions as sources of semantic diversity. Motivated by this, we revisit the image masking approach, proposing to treat masked content as auxiliary knowledge rather than ignored. Based on this, we propose MaskAnyNet, which combines masking with a relearning mechanism to exploit both visible and masked information. It can be easily extended to any model with an additional branch to jointly learn from the recomposed masked region. This approach leverages the semantic diversity of the masked regions to enrich features and preserve fine-grained details. Experiments on CNN and Transformer backbones show consistent gains across multiple benchmarks. Further analysis confirms that the proposed method improves semantic diversity through the reuse of masked content.

Method: Proposes Current-Centric Contextual 3D Fusion (C3DFusion) module that generates hidden region-aware 3D feature geometry by explicitly aligning 3D-lifted point features from current and historical frames, using historical context blurring and current-centric feature densification.

Result: Significantly outperforms state-of-the-art methods on SemanticKITTI and SSCBench-KITTI-360 datasets, and shows robust generalization with notable performance gains when applied to other baseline models.

Conclusion: C3DFusion effectively addresses the limitation of reconstructing out-of-frame areas in 3D semantic scene completion through enhanced temporal fusion techniques.

Abstract: Recent camera-based 3D semantic scene completion (SSC) methods have increasingly explored leveraging temporal cues to enrich the features of the current frame. However, while these approaches primarily focus on enhancing in-frame regions, they often struggle to reconstruct critical out-of-frame areas near the sides of the ego-vehicle, although previous frames commonly contain valuable contextual information about these unseen regions. To address this limitation, we propose the Current-Centric Contextual 3D Fusion (C3DFusion) module, which generates hidden region-aware 3D feature geometry by explicitly aligning 3D-lifted point features from both current and historical frames. C3DFusion performs enhanced temporal fusion through two complementary techniques-historical context blurring and current-centric feature densification-which suppress noise from inaccurately warped historical point features by attenuating their scale, and enhance current point features by increasing their volumetric contribution. Simply integrated into standard SSC architectures, C3DFusion demonstrates strong effectiveness, significantly outperforming state-of-the-art methods on the SemanticKITTI and SSCBench-KITTI-360 datasets. Furthermore, it exhibits robust generalization, achieving notable performance gains when applied to other baseline models.

Method: Learns a direct mapping from images to visible scene structure using a novel visible structure retrieval network that outputs the subset of 3D points visible in a query image.

Result: Achieves localization accuracy comparable to state-of-the-art methods while requiring lower computational and storage footprint.

Conclusion: The proposed neural network paradigm enables tractable structure-based relocalization without the limitations of traditional retrieval-based approaches.

Abstract: Accurate camera pose estimation from an image observation in a previously mapped environment is commonly done through structure-based methods: by finding correspondences between 2D keypoints on the image and 3D structure points in the map. In order to make this correspondence search tractable in large scenes, existing pipelines either rely on search heuristics, or perform image retrieval to reduce the search space by comparing the current image to a database of past observations. However, these approaches result in elaborate pipelines or storage requirements that grow with the number of past observations. In this work, we propose a new paradigm for making structure-based relocalisation tractable. Instead of relying on image retrieval or search heuristics, we learn a direct mapping from image observations to the visible scene structure in a compact neural network. Given a query image, a forward pass through our novel visible structure retrieval network allows obtaining the subset of 3D structure points in the map that the image views, thus reducing the search space of 2D-3D correspondences. We show that our proposed method enables performing localisation with an accuracy comparable to the state of the art, while requiring lower computational and storage footprint.

Method: Teacher-student knowledge distillation with frozen DINOv3 teacher trained on clean images providing stable representations, distilled to student trained on blurred images to maintain consistent detection under motion degradation.

Result: Achieves state-of-the-art performance under both motion-blurred and clean conditions, demonstrating improved generalization and real-world applicability.

Conclusion: The proposed framework effectively addresses motion blur challenges in AI-generated image detection through knowledge distillation, enhancing real-world deployment capabilities.

Abstract: With growing concerns over image authenticity and digital safety, the field of AI-generated image (AIGI) detection has progressed rapidly. Yet, most AIGI detectors still struggle under real-world degradations, particularly motion blur, which frequently occurs in handheld photography, fast motion, and compressed video. Such blur distorts fine textures and suppresses high-frequency artifacts, causing severe performance drops in real-world settings. We address this limitation with a blur-robust AIGI detection framework based on teacher-student knowledge distillation. A high-capacity teacher (DINOv3), trained on clean (i.e., sharp) images, provides stable and semantically rich representations that serve as a reference for learning. By freezing the teacher to maintain its generalization ability, we distill its feature and logit responses from sharp images to a student trained on blurred counterparts, enabling the student to produce consistent representations under motion degradation. Extensive experiments benchmarks show that our method achieves state-of-the-art performance under both motion-blurred and clean conditions, demonstrating improved generalization and real-world applicability. Source codes will be released at: https://github.com/JiaLiangShen/Dino-Detect-for-blur-robust-AIGC-Detection.

Method: Uses mixture-of-experts (MoE) system for different degradation scenarios, pre-trained VLM for semantic priors, degradation-aware channel attention module (DCAM) with prototype decomposition, and expert routing guided by semantic priors.

Result: Extensive experiments show superior performance over state-of-the-art methods in complex degradation scenarios.

Conclusion: MdaIF framework effectively addresses multi-degradation image fusion by leveraging LLM-driven MoE and semantic priors, demonstrating robust performance across various weather conditions.

Abstract: Infrared and visible image fusion aims to integrate complementary multi-modal information into a single fused result. However, existing methods 1) fail to account for the degradation visible images under adverse weather conditions, thereby compromising fusion performance; and 2) rely on fixed network architectures, limiting their adaptability to diverse degradation scenarios. To address these issues, we propose a one-stop degradation-aware image fusion framework for multi-degradation scenarios driven by a large language model (MdaIF). Given the distinct scattering characteristics of different degradation scenarios (e.g., haze, rain, and snow) in atmospheric transmission, a mixture-of-experts (MoE) system is introduced to tackle image fusion across multiple degradation scenarios. To adaptively extract diverse weather-aware degradation knowledge and scene feature representations, collectively referred to as the semantic prior, we employ a pre-trained vision-language model (VLM) in our framework. Guided by the semantic prior, we propose degradation-aware channel attention module (DCAM), which employ degradation prototype decomposition to facilitate multi-modal feature interaction in channel domain. In addition, to achieve effective expert routing, the semantic prior and channel-domain modulated features are utilized to guide the MoE, enabling robust image fusion in complex degradation scenarios. Extensive experiments validate the effectiveness of our MdaIF, demonstrating superior performance over SOTA methods.

Method: Uses two-stage training with knowledge distillation and fine-tuning, plus a Distillation Recovery Module for feature alignment and a Top-Down-attention-based Deformable Aggregator for adaptive region selection.

Result: Achieves competitive performance with state-of-the-art methods while reducing parameters by ~64.2% and FLOPs by ~62.6% compared to CricaVPR.

Conclusion: The proposed framework successfully balances performance and efficiency, making VPR more deployable on resource-constrained devices without sacrificing accuracy.

Abstract: Visual Place Recognition (VPR) aims to determine the geographic location of a query image by retrieving its most visually similar counterpart from a geo-tagged reference database. Recently, the emergence of the powerful visual foundation model, DINOv2, trained in a self-supervised manner on massive datasets, has significantly improved VPR performance. This improvement stems from DINOv2’s exceptional feature generalization capabilities but is often accompanied by increased model complexity and computational overhead that impede deployment on resource-constrained devices. To address this challenge, we propose $D^{2}$-VPR, a $D$istillation- and $D$eformable-based framework that retains the strong feature extraction capabilities of visual foundation models while significantly reducing model parameters and achieving a more favorable performance-efficiency trade-off. Specifically, first, we employ a two-stage training strategy that integrates knowledge distillation and fine-tuning. Additionally, we introduce a Distillation Recovery Module (DRM) to better align the feature spaces between the teacher and student models, thereby minimizing knowledge transfer losses to the greatest extent possible. Second, we design a Top-Down-attention-based Deformable Aggregator (TDDA) that leverages global semantic features to dynamically and adaptively adjust the Regions of Interest (ROI) used for aggregation, thereby improving adaptability to irregular structures. Extensive experiments demonstrate that our method achieves competitive performance compared to state-of-the-art approaches. Meanwhile, it reduces the parameter count by approximately 64.2% and FLOPs by about 62.6% (compared to CricaVPR).Code is available at https://github.com/tony19980810/D2VPR.

Method: Uses a learnable policy network to select keyframes from candidate frames, assesses causal necessity via counterfactual interventions, and guides selection through reinforcement learning with a CIB-aligned reward.

Result: Consistently outperforms state-of-the-art methods on NExT-QA, EgoSchema, and Video-MME datasets under limited-frame settings.

Conclusion: ReaSon effectively selects causally necessary and predictively sufficient keyframes, demonstrating strong generalization ability across multiple video understanding benchmarks.

Abstract: Keyframe selection has become essential for video understanding with vision-language models (VLMs) due to limited input tokens and the temporal sparsity of relevant information across video frames. Video understanding often relies on effective keyframes that are not only informative but also causally decisive. To this end, we propose Reinforced Causal Search with Information Bottleneck (ReaSon), a framework that formulates keyframe selection as an optimization problem with the help of a novel Causal Information Bottleneck (CIB), which explicitly defines keyframes as those satisfying both predictive sufficiency and causal necessity. Specifically, ReaSon employs a learnable policy network to select keyframes from a visually relevant pool of candidate frames to capture predictive sufficiency, and then assesses causal necessity via counterfactual interventions. Finally, a composite reward aligned with the CIB principle is designed to guide the selection policy through reinforcement learning. Extensive experiments on NExT-QA, EgoSchema, and Video-MME demonstrate that ReaSon consistently outperforms existing state-of-the-art methods under limited-frame settings, validating its effectiveness and generalization ability.

Method: Hierarchical guidance: early stages use strong text and contour guidance for scene/style/structure; final stages activate fine-grained classifier guidance with dynamic modulation based on prediction confidence to balance global structure with detail refinement.

Result: Experiments on multiple FGVC datasets demonstrate HiGFA’s effectiveness in generating diverse yet faithful synthetic images for fine-grained classification tasks.

Conclusion: HiGFA’s hierarchical, confidence-driven orchestration of guidance signals enables successful fine-grained data augmentation by intelligently balancing global structure formation with precise detail refinement throughout the diffusion sampling process.

Abstract: Generative diffusion models show promise for data augmentation. However, applying them to fine-grained tasks presents a significant challenge: ensuring synthetic images accurately capture the subtle, category-defining features critical for high fidelity. Standard approaches, such as text-based Classifier-Free Guidance (CFG), often lack the required specificity, potentially generating misleading examples that degrade fine-grained classifier performance. To address this, we propose Hierarchically Guided Fine-grained Augmentation (HiGFA). HiGFA leverages the temporal dynamics of the diffusion sampling process. It employs strong text and transformed contour guidance with fixed strengths in the early-to-mid sampling stages to establish overall scene, style, and structure. In the final sampling stages, HiGFA activates a specialized fine-grained classifier guidance and dynamically modulates the strength of all guidance signals based on prediction confidence. This hierarchical, confidence-driven orchestration enables HiGFA to generate diverse yet faithful synthetic images by intelligently balancing global structure formation with precise detail refinement. Experiments on several FGVC datasets demonstrate the effectiveness of HiGFA.

Method: Created EmoVerse dataset with 219k images using a multi-stage annotation pipeline that decomposes emotions into Background-Attribute-Subject triplets, grounds elements to visual regions, and provides dual annotations in both categorical and dimensional emotion spaces.

Result: The dataset enables word-level and subject-level emotional reasoning with high annotation reliability achieved through minimal human effort. An interpretable model was also introduced that maps visual cues to dimensional emotion representations with detailed attribution explanations.

Conclusion: EmoVerse provides a comprehensive foundation for advancing explainable high-level emotion understanding in visual content through its dataset, annotation pipeline, and interpretable model.

Abstract: Visual Emotion Analysis (VEA) aims to bridge the affective gap between visual content and human emotional responses. Despite its promise, progress in this field remains limited by the lack of open-source and interpretable datasets. Most existing studies assign a single discrete emotion label to an entire image, offering limited insight into how visual elements contribute to emotion. In this work, we introduce EmoVerse, a large-scale open-source dataset that enables interpretable visual emotion analysis through multi-layered, knowledge-graph-inspired annotations. By decomposing emotions into Background-Attribute-Subject (B-A-S) triplets and grounding each element to visual regions, EmoVerse provides word-level and subject-level emotional reasoning. With over 219k images, the dataset further includes dual annotations in Categorical Emotion States (CES) and Dimensional Emotion Space (DES), facilitating unified discrete and continuous emotion representation. A novel multi-stage pipeline ensures high annotation reliability with minimal human effort. Finally, we introduce an interpretable model that maps visual cues into DES representations and provides detailed attribution explanations. Together, the dataset, pipeline, and model form a comprehensive foundation for advancing explainable high-level emotion understanding.

Method: Proposes SEMC with two key modules: 1) Semantic-Structure Fusion Module (SSFM) to exploit multi-scale structural information and align shallow/deep features, 2) Mixture-of-Experts Contrastive Recognition Module (MCRM) for hierarchical contrastive learning using MoE mechanism. Also curated a large-scale liver ultrasound dataset with six standard planes.

Result: Extensive experiments on in-house dataset and two public datasets demonstrate that SEMC outperforms recent state-of-the-art methods across various metrics.

Conclusion: SEMC effectively addresses the limitations of existing methods by combining structure-aware feature fusion with expert-guided contrastive learning, achieving superior performance in ultrasound standard plane recognition.

Abstract: Ultrasound standard plane recognition is essential for clinical tasks such as disease screening, organ evaluation, and biometric measurement. However, existing methods fail to effectively exploit shallow structural information and struggle to capture fine-grained semantic differences through contrastive samples generated by image augmentations, ultimately resulting in suboptimal recognition of both structural and discriminative details in ultrasound standard planes. To address these issues, we propose SEMC, a novel Structure-Enhanced Mixture-of-Experts Contrastive learning framework that combines structure-aware feature fusion with expert-guided contrastive learning. Specifically, we first introduce a novel Semantic-Structure Fusion Module (SSFM) to exploit multi-scale structural information and enhance the model’s ability to perceive fine-grained structural details by effectively aligning shallow and deep features. Then, a novel Mixture-of-Experts Contrastive Recognition Module (MCRM) is designed to perform hierarchical contrastive learning and classification across multi-level features using a mixture-of-experts (MoE) mechanism, further improving class separability and recognition performance. More importantly, we also curate a large-scale and meticulously annotated liver ultrasound dataset containing six standard planes. Extensive experimental results on our in-house dataset and two public datasets demonstrate that SEMC outperforms recent state-of-the-art methods across various metrics.

Method: Train a visual state space model to recover subtle thermal signals from blurred SA data, using a latent diffusion model in vector quantized framework to generate realistic surface temperature simulations from real wildfire recordings, with temperature augmentation and procedural thermal forest simulation.

Result: On simulated data, reduced RMSE by 2-2.5x compared to conventional thermal and uncorrected SA imaging. In field experiments for high-temperature hotspots, achieved 12.8x RMSE gain over conventional thermal and 2.6x gain over uncorrected SA. Successfully reconstructed complete morphology of fire and human signatures.

Conclusion: The method enables accurate temperature reconstruction through occluding vegetation, outperforming conventional approaches significantly, and demonstrates generalization to other thermal signals like human signatures for search and rescue applications.

Abstract: We introduce a novel method for reconstructing surface temperatures through occluding forest vegetation by combining signal processing and machine learning. Our goal is to enable fully automated aerial wildfire monitoring using autonomous drones, allowing for the early detection of ground fires before smoke or flames are visible. While synthetic aperture (SA) sensing mitigates occlusion from the canopy and sunlight, it introduces thermal blur that obscures the actual surface temperatures. To address this, we train a visual state space model to recover the subtle thermal signals of partially occluded soil and fire hotspots from this blurred data. A key challenge was the scarcity of real-world training data. We overcome this by integrating a latent diffusion model into a vector quantized to generated a large volume of realistic surface temperature simulations from real wildfire recordings, which we further expanded through temperature augmentation and procedural thermal forest simulation. On simulated data across varied ambient and surface temperatures, forest densities, and sunlight conditions, our method reduced the RMSE by a factor of 2 to 2.5 compared to conventional thermal and uncorrected SA imaging. In field experiments focused on high-temperature hotspots, the improvement was even more significant, with a 12.8-fold RMSE gain over conventional thermal and a 2.6-fold gain over uncorrected SA images. We also demonstrate our model’s generalization to other thermal signals, such as human signatures for search and rescue. Since simple thresholding is frequently inadequate for detecting subtle thermal signals, the morphological characteristics are equally essential for accurate classification. Our experiments demonstrated another clear advantage: we reconstructed the complete morphology of fire and human signatures, whereas conventional imaging is defeated by partial occlusion.

Method: Two-stage, semantics-aware typographical attack that generates deceptive text extensions outside visual content.

Result: Significantly reduces geolocation prediction accuracy of five state-of-the-art commercial LVLMs across three datasets.

Conclusion: Provides practical and visually-preserving protection strategy against emerging geo-privacy threats from LVLMs.

Abstract: Large Visual Language Models (LVLMs) now pose a serious yet overlooked privacy threat, as they can infer a social media user’s geolocation directly from shared images, leading to unintended privacy leakage. While adversarial image perturbations provide a potential direction for geo-privacy protection, they require relatively strong distortions to be effective against LVLMs, which noticeably degrade visual quality and diminish an image’s value for sharing. To overcome this limitation, we identify typographical attacks as a promising direction for protecting geo-privacy by adding text extension outside the visual content. We further investigate which textual semantics are effective in disrupting geolocation inference and design a two-stage, semantics-aware typographical attack that generates deceptive text to protect user privacy. Extensive experiments across three datasets demonstrate that our approach significantly reduces geolocation prediction accuracy of five state-of-the-art commercial LVLMs, establishing a practical and visually-preserving protection strategy against emerging geo-privacy threats.

Method: Formulates video generation as next-frame-rate prediction, using bidirectional attention within frame-rate levels and autoregression across frame rates.

Result: Establishes new state-of-the-art in long video generation with excellent visual and temporal quality.

Conclusion: TempoMaster’s hierarchical frame-rate approach effectively balances temporal coherence and generation efficiency for long videos.

Abstract: We present TempoMaster, a novel framework that formulates long video generation as next-frame-rate prediction. Specifically, we first generate a low-frame-rate clip that serves as a coarse blueprint of the entire video sequence, and then progressively increase the frame rate to refine visual details and motion continuity. During generation, TempoMaster employs bidirectional attention within each frame-rate level while performing autoregression across frame rates, thus achieving long-range temporal coherence while enabling efficient and parallel synthesis. Extensive experiments demonstrate that TempoMaster establishes a new state-of-the-art in long video generation, excelling in both visual and temporal quality.

Method: Proposes rank-aware agglomeration with RATS strategy for sample-wise teacher selection based on global-to-local patch rankings. Uses vision-language alignment with discrete semantic anchors from structured text prompts to guide class-specific density map regression.

Result: Outperforms state-of-the-art methods across 12 IHC biomarkers and 5 tissue types, showing high agreement with pathologists’ assessments. Also effective on H&E-stained data, demonstrating scalability.

Conclusion: CountIHC successfully addresses multi-class cell counting challenges in IHC images through selective knowledge distillation and vision-language alignment, achieving superior performance and clinical relevance.

Abstract: Accurate cell counting in immunohistochemistry (IHC) images is critical for quantifying protein expression and aiding cancer diagnosis. However, the task remains challenging due to the chromogen overlap, variable biomarker staining, and diverse cellular morphologies. Regression-based counting methods offer advantages over detection-based ones in handling overlapped cells, yet rarely support end-to-end multi-class counting. Moreover, the potential of foundation models remains largely underexplored in this paradigm. To address these limitations, we propose a rank-aware agglomeration framework that selectively distills knowledge from multiple strong foundation models, leveraging their complementary representations to handle IHC heterogeneity and obtain a compact yet effective student model, CountIHC. Unlike prior task-agnostic agglomeration strategies that either treat all teachers equally or rely on feature similarity, we design a Rank-Aware Teacher Selecting (RATS) strategy that models global-to-local patch rankings to assess each teacher’s inherent counting capacity and enable sample-wise teacher selection. For multi-class cell counting, we introduce a fine-tuning stage that reformulates the task as vision-language alignment. Discrete semantic anchors derived from structured text prompts encode both category and quantity information, guiding the regression of class-specific density maps and improving counting for overlapping cells. Extensive experiments demonstrate that CountIHC surpasses state-of-the-art methods across 12 IHC biomarkers and 5 tissue types, while exhibiting high agreement with pathologists’ assessments. Its effectiveness on H&E-stained data further confirms the scalability of the proposed method.

Method: Divides topology prediction into three phases: Hierarchical Prior Extractor (HPE) for global spatial and local sequential priors, Region-Focused Decoder (RFD) for fine-grained query construction, and Robust Boundary-Point Topology Reasoning (RBTR) with denoising strategy.

Result: Achieves state-of-the-art performance on OpenLane-V2 benchmark with OLS of 48.0% on subsetA and 45.4% on subsetB.

Conclusion: Integrating spatial and sequential priors into fine-grained queries with boundary-point topology denoising enables precise modeling of complex lane structures and trustworthy topology predictions.

Abstract: Precise modeling of lane topology is essential for autonomous driving, as it directly impacts navigation and control decisions.Existing methods typically represent each lane with a single query and infer topological connectivity based on the similarity between lane queries.However, this kind of design struggles to accurately model complex lane structures, leading to unreliable topology prediction.In this view, we propose a Fine-Grained lane topology reasoning framework (TopoFG).It divides the procedure from bird’s-eye-view (BEV) features to topology prediction via fine-grained queries into three phases, i.e., Hierarchical Prior Extractor (HPE), Region-Focused Decoder (RFD), and Robust Boundary-Point Topology Reasoning (RBTR).Specifically, HPE extracts global spatial priors from the BEV mask and local sequential priors from in-lane keypoint sequences to guide subsequent fine-grained query modeling.RFD constructs fine-grained queries by integrating the spatial and sequential priors. It then samples reference points in RoI regions of the mask and applies cross-attention with BEV features to refine the query representations of each lane.RBTR models lane connectivity based on boundary-point query features and further employs a topological denoising strategy to reduce matching ambiguity.By integrating spatial and sequential priors into fine-grained queries and applying a denoising strategy to boundary-point topology reasoning, our method precisely models complex lane structures and delivers trustworthy topology predictions.Extensive experiments on the OpenLane-V2 benchmark demonstrate that TopoFG achieves new state-of-the-art performance, with an OLS of 48.0% on subsetA and 45.4% on subsetB.

Method: Three core components: (1) image encoder generating latent priors, (2) spatial-aware seglat encoder mapping masks to discrete latent tokens using location-sensitive color mapping, (3) decoder reconstructing masks from latents. Multi-stage training: first learning seglat representations, then refining latent transformations, finally aligning image-encoder-derived latents with seglat distributions.

Result: Seg-VAR outperforms previous discriminative and generative methods on various segmentation tasks and validation benchmarks.

Conclusion: By framing segmentation as a sequential hierarchical prediction task, Seg-VAR opens new avenues for integrating autoregressive reasoning into spatial-aware vision systems.

Abstract: While visual autoregressive modeling (VAR) strategies have shed light on image generation with the autoregressive models, their potential for segmentation, a task that requires precise low-level spatial perception, remains unexplored. Inspired by the multi-scale modeling of classic Mask2Former-based models, we propose Seg-VAR, a novel framework that rethinks segmentation as a conditional autoregressive mask generation problem. This is achieved by replacing the discriminative learning with the latent learning process. Specifically, our method incorporates three core components: (1) an image encoder generating latent priors from input images, (2) a spatial-aware seglat (a latent expression of segmentation mask) encoder that maps segmentation masks into discrete latent tokens using a location-sensitive color mapping to distinguish instances, and (3) a decoder reconstructing masks from these latents. A multi-stage training strategy is introduced: first learning seglat representations via image-seglat joint training, then refining latent transformations, and finally aligning image-encoder-derived latents with seglat distributions. Experiments show Seg-VAR outperforms previous discriminative and generative methods on various segmentation tasks and validation benchmarks. By framing segmentation as a sequential hierarchical prediction task, Seg-VAR opens new avenues for integrating autoregressive reasoning into spatial-aware vision systems. Code will be available at https://github.com/rkzheng99/Seg-VAR.

Method: Proposes a teacher-student framework where a CNN teacher refines a ViT student model, integrated with Low-Rank Adaptation (LoRA) for efficient fine-tuning.

Result: Superior detection performance and computational efficiency compared to six state-of-the-art S-MAD techniques on a comprehensive morphing dataset.

Conclusion: The proposed approach effectively addresses morphing attack detection with high accuracy and reduced computational costs.

Abstract: Face Recognition Systems (FRS) are critical for security but remain vulnerable to morphing attacks, where synthetic images blend biometric features from multiple individuals. We propose a novel Single-Image Morphing Attack Detection (S-MAD) approach using a teacher-student framework, where a CNN-based teacher model refines a ViT-based student model. To improve efficiency, we integrate Low-Rank Adaptation (LoRA) for fine-tuning, reducing computational costs while maintaining high detection accuracy. Extensive experiments are conducted on a morphing dataset built from three publicly available face datasets, incorporating ten different morphing generation algorithms to assess robustness. The proposed method is benchmarked against six state-of-the-art S-MAD techniques, demonstrating superior detection performance and computational efficiency.

Method: Introduced SoccerNet-GAR dataset with synchronized broadcast video and player tracking data from 64 World Cup 2022 matches. Developed unified evaluation protocol and two unimodal approaches: competitive video-based classifiers and novel role-aware graph neural networks for tracking-based GAR.

Result: Tracking model achieved 67.2% balanced accuracy vs 58.1% for best video baseline, while training 4.25x faster with 438x fewer parameters (197K vs 86.3M).

Conclusion: Tracking-based GAR with role-aware modeling significantly outperforms video-based approaches in both accuracy and efficiency, highlighting the importance of modality choice and tactical structure encoding for group activity recognition.

Abstract: Group Activity Recognition (GAR) is well studied on the video modality for surveillance and indoor team sports (e.g., volleyball, basketball). Yet, other modalities such as agent positions and trajectories over time, i.e. tracking, remain comparatively under-explored despite being compact, agent-centric signals that explicitly encode spatial interactions. Understanding whether pixel (video) or position (tracking) modalities leads to better group activity recognition is therefore important to drive further research on the topic. However, no standardized benchmark currently exists that aligns broadcast video and tracking data for the same group activities, leading to a lack of apples-to-apples comparison between these modalities for GAR. In this work, we introduce SoccerNet-GAR, a multimodal dataset built from the $64$ matches of the football World Cup 2022. Specifically, the broadcast videos and player tracking modalities for $94{,}285$ group activities are synchronized and annotated with $10$ categories. Furthermore, we define a unified evaluation protocol to benchmark two strong unimodal approaches: (i) a competitive video-based classifiers and (ii) a tracking-based classifiers leveraging graph neural networks. In particular, our novel role-aware graph architecture for tracking-based GAR directly encodes tactical structure through positional edges and temporal attention. Our tracking model achieves $67.2%$ balanced accuracy compared to $58.1%$ for the best video baseline, while training $4.25 \times$ faster with $438 \times$ fewer parameters ($197K$ \vs $86.3M$). This study provides new insights into the relative strengths of pixels and positions for group activity recognition. Overall, it highlights the importance of modality choice and role-aware modeling for GAR.

Method: Uses Hierarchical Ladder Network to extract OOD features from Transformer class tokens, Attention Affine Network to refine self-attention for domain drift, and weighted entropy to suppress low-confidence samples.

Result: Significantly improves performance on benchmark classification datasets compared to existing methods.

Conclusion: The proposed hierarchical approach effectively addresses OOD detection and domain adaptation challenges in test-time adaptation, enhancing model robustness and performance.

Abstract: Test-time adaptation (TTA) refers to adjusting the model during the testing phase to cope with changes in sample distribution and enhance the model’s adaptability to new environments. In real-world scenarios, models often encounter samples from unseen (out-of-distribution, OOD) categories. Misclassifying these as known (in-distribution, ID) classes not only degrades predictive accuracy but can also impair the adaptation process, leading to further errors on subsequent ID samples. Many existing TTA methods suffer substantial performance drops under such conditions. To address this challenge, we propose a Hierarchical Ladder Network that extracts OOD features from class tokens aggregated across all Transformer layers. OOD detection performance is enhanced by combining the original model prediction with the output of the Hierarchical Ladder Network (HLN) via weighted probability fusion. To improve robustness under domain shift, we further introduce an Attention Affine Network (AAN) that adaptively refines the self-attention mechanism conditioned on the token information to better adapt to domain drift, thereby improving the classification performance of the model on datasets with domain shift. Additionally, a weighted entropy mechanism is employed to dynamically suppress the influence of low-confidence samples during adaptation. Experimental results on benchmark datasets show that our method significantly improves the performance on the most widely used classification datasets.

Method: Pipeline with onboarding stage generating object representations from CAD or NeRF reconstruction, CNOS detector for object localization, and OPFormer transformer architecture that encodes multiple template views with 3D geometric priors using NOCS for robust 2D-3D correspondences.

Result: Demonstrates strong balance between accuracy and efficiency on BOP benchmarks, showing practical applicability in both model-based and model-free scenarios.

Conclusion: The integrated system provides a versatile solution for 6D pose estimation that works effectively with both traditional CAD models and neural reconstructions, achieving competitive performance on challenging benchmarks.

Abstract: We introduce a unified, end-to-end framework that seamlessly integrates object detection and pose estimation with a versatile onboarding process. Our pipeline begins with an onboarding stage that generates object representations from either traditional 3D CAD models or, in their absence, by rapidly reconstructing a high-fidelity neural representation (NeRF) from multi-view images. Given a test image, our system first employs the CNOS detector to localize target objects. For each detection, our novel pose estimation module, OPFormer, infers the precise 6D pose. The core of OPFormer is a transformer-based architecture that leverages a foundation model for robust feature extraction. It uniquely learns a comprehensive object representation by jointly encoding multiple template views and enriches these features with explicit 3D geometric priors using Normalized Object Coordinate Space (NOCS). A decoder then establishes robust 2D-3D correspondences to determine the final pose. Evaluated on the challenging BOP benchmarks, our integrated system demonstrates a strong balance between accuracy and efficiency, showcasing its practical applicability in both model-based and model-free scenarios.

Method: Uses unified tokenization for semantic masks and text inputs, stacked multivariate transformer blocks for synchronous condition processing, and a novel decoupled attention mechanism that separates mask tokens from temporal embeddings into dynamic and static pathways.

Result: Significantly outperforms competing methods in facial fidelity and conditional consistency while reducing computational overhead introduced by mask condition by over 94%.

Conclusion: MDiTFace effectively addresses cross-modal interaction challenges in multimodal facial generation through unified tokenization and decoupled attention, achieving superior performance with significantly reduced computational costs.

Abstract: While significant progress has been achieved in multimodal facial generation using semantic masks and textual descriptions, conventional feature fusion approaches often fail to enable effective cross-modal interactions, thereby leading to suboptimal generation outcomes. To address this challenge, we introduce MDiTFace–a customized diffusion transformer framework that employs a unified tokenization strategy to process semantic mask and text inputs, eliminating discrepancies between heterogeneous modality representations. The framework facilitates comprehensive multimodal feature interaction through stacked, newly designed multivariate transformer blocks that process all conditions synchronously. Additionally, we design a novel decoupled attention mechanism by dissociating implicit dependencies between mask tokens and temporal embeddings. This mechanism segregates internal computations into dynamic and static pathways, enabling caching and reuse of features computed in static pathways after initial calculation, thereby reducing additional computational overhead introduced by mask condition by over 94% while maintaining performance. Extensive experiments demonstrate that MDiTFace significantly outperforms other competing methods in terms of both facial fidelity and conditional consistency.

Method: Proposes spectral self-regularization to suppress redundant high-frequency noise in latent spaces while preserving reconstruction quality. Also introduces spectral alignment strategy to optimize Denoising-VAE-based generative models. Uses ViT-based autoencoder architecture without VFM dependency.

Result: Enables diffusion models to converge 2× faster than SD-VAE. Achieves state-of-the-art reconstruction quality (rFID = 0.28, PSNR = 27.26) and competitive generation performance (gFID = 1.82) on ImageNet 256×256 benchmark.

Conclusion: The analysis reveals that redundant high-frequency components in high-dimensional latent spaces hinder diffusion model convergence. Denoising-VAE effectively addresses this by producing cleaner latents, improving both reconstruction and generation while accelerating training.

Abstract: Variational autoencoders (VAEs) typically encode images into a compact latent space, reducing computational cost but introducing an optimization dilemma: a higher-dimensional latent space improves reconstruction fidelity but often hampers generative performance. Recent methods attempt to address this dilemma by regularizing high-dimensional latent spaces using external vision foundation models (VFMs). However, it remains unclear how high-dimensional VAE latents affect the optimization of generative models. To our knowledge, our analysis is the first to reveal that redundant high-frequency components in high-dimensional latent spaces hinder the training convergence of diffusion models and, consequently, degrade generation quality. To alleviate this problem, we propose a spectral self-regularization strategy to suppress redundant high-frequency noise while simultaneously preserving reconstruction quality. The resulting Denoising-VAE, a ViT-based autoencoder that does not rely on VFMs, produces cleaner, lower-noise latents, leading to improved generative quality and faster optimization convergence. We further introduce a spectral alignment strategy to facilitate the optimization of Denoising-VAE-based generative models. Our complete method enables diffusion models to converge approximately 2$\times$ faster than with SD-VAE, while achieving state-of-the-art reconstruction quality (rFID = 0.28, PSNR = 27.26) and competitive generation performance (gFID = 1.82) on the ImageNet 256$\times$256 benchmark.

Method: CILMP extracts disease-specific representations from LLMs, intervenes within a low-rank linear subspace, and uses them to create disease-specific prompts with a conditional mechanism for instance-adaptive prompt generation.

Result: Extensive experiments across diverse medical image datasets show that CILMP consistently outperforms state-of-the-art prompt tuning methods.

Conclusion: CILMP effectively bridges LLMs and VLMs to transfer medical knowledge into VLM prompts, demonstrating superior performance in medical image classification tasks.

Abstract: Vision-language foundation models (VLMs) have shown great potential in feature transfer and generalization across a wide spectrum of medical-related downstream tasks. However, fine-tuning these models is resource-intensive due to their large number of parameters. Prompt tuning has emerged as a viable solution to mitigate memory usage and reduce training time while maintaining competitive performance. Nevertheless, the challenge is that existing prompt tuning methods cannot precisely distinguish different kinds of medical concepts, which miss essentially specific disease-related features across various medical imaging modalities in medical image classification tasks. We find that Large Language Models (LLMs), trained on extensive text corpora, are particularly adept at providing this specialized medical knowledge. Motivated by this, we propose incorporating LLMs into the prompt tuning process. Specifically, we introduce the CILMP, Conditional Intervention of Large Language Models for Prompt Tuning, a method that bridges LLMs and VLMs to facilitate the transfer of medical knowledge into VLM prompts. CILMP extracts disease-specific representations from LLMs, intervenes within a low-rank linear subspace, and utilizes them to create disease-specific prompts. Additionally, a conditional mechanism is incorporated to condition the intervention process on each individual medical image, generating instance-adaptive prompts and thus enhancing adaptability. Extensive experiments across diverse medical image datasets demonstrate that CILMP consistently outperforms state-of-the-art prompt tuning methods, demonstrating its effectiveness. Code is available at https://github.com/usr922/cilmp.

Method: Uses learnable scale parameterization, heterogeneous precision design (front-end floating-point fake quantization with FP16/FP32; back-end full precision), and GPU-native kernel fusion for fake quantization with custom CUDA kernels.

Result: On TartanAir: 52.1% FPS increase, 29.1% latency reduction, 64.9% GPU memory reduction. On EuRoC: 30.1% FPS increase, 23.1% latency reduction, 37.7% GPU memory reduction, while maintaining comparable trajectory accuracy to original model.

Conclusion: DPVO-QAT++ effectively bridges the gap between high-precision deep VO and practical deployment efficiency, offering a viable engineering paradigm for real-world embedded platforms.

Abstract: Deep learning-based Visual SLAM (vSLAM) systems exhibit exceptional geometric reasoning capabilities, yet their prohibitive computational overhead severely restricts deployment on resource-constrained autonomous platforms. This paper presents a hierarchical quantization optimization framework, DPVO-QAT++ (DPVO-QAT++: Heterogeneous QAT and CUDA Kernel Fusion for High-Performance Deep Patch Visual Odometry). Through the synergistic integration of learnable scale parameterization, a heterogeneous precision design for the Visual Odometry (VO) front-end and back-end (front-end floating-point fake quantization with FP16/FP32; back-end full precision), and GPU-native kernel fusion for fake quantization (custom CUDA kernels), our framework significantly reduces memory footprint and increases processing speed while preserving the trajectory accuracy of the original model. On the TartanAir dataset, our framework achieves an average FPS increase of 52.1%, a 29.1% reduction in median latency, and a 64.9% reduction in peak GPU memory reservation, while maintaining trajectory accuracy (ATE) comparable to the original DPVO model across 32 validation sequences. On the EuRoC dataset, it realizes an average FPS increase of 30.1%, a 23.1% reduction in median latency, and a 37.7% reduction in peak GPU memory reservation, maintaining comparable trajectory accuracy (ATE) across 11 validation sequences. Experimental results demonstrate that DPVO-QAT++ effectively bridges the gap between high-precision deep VO and the efficiency requirements for practical deployment, offering a viable engineering paradigm for the application of this technology on real-world embedded platforms. Keywords: Visual Odometry, Heterogeneous Precision Architecture, Quantization-Aware Training, CUDA Kernel Fusion, Scale-Only Training, Deep Patch Visual Odometry, GPU-Native Kernel Fusion.

Method: Collects 16,750 real tampering instances, analyzes editing traces, and models tampering parameters hierarchically using Fourier series-inspired basis functions to synthesize realistic training data.

Result: Models trained with FSTS data achieve significantly improved generalization on real-world datasets across four evaluation protocols.

Conclusion: FSTS provides an interpretable framework for generating diverse and realistic tampered text images that better reflect real-world forgery traces, improving model generalization.

Abstract: Existing Text Image Forgery Localization (T-IFL) methods often suffer from poor generalization due to the limited scale of real-world datasets and the distribution gap caused by synthetic data that fails to capture the complexity of real-world tampering. To tackle this issue, we propose Fourier Series-based Tampering Synthesis (FSTS), a structured and interpretable framework for synthesizing tampered text images. FSTS first collects 16,750 real-world tampering instances from five representative tampering types, using a structured pipeline that records human-performed editing traces via multi-format logs (e.g., video, PSD, and editing logs). By analyzing these collected parameters and identifying recurring behavioral patterns at both individual and population levels, we formulate a hierarchical modeling framework. Specifically, each individual tampering parameter is represented as a compact combination of basis operation-parameter configurations, while the population-level distribution is constructed by aggregating these behaviors. Since this formulation draws inspiration from the Fourier series, it enables an interpretable approximation using basis functions and their learned weights. By sampling from this modeled distribution, FSTS synthesizes diverse and realistic training data that better reflect real-world forgery traces. Extensive experiments across four evaluation protocols demonstrate that models trained with FSTS data achieve significantly improved generalization on real-world datasets. Dataset is available at \href{https://github.com/ZeqinYu/FSTS}{Project Page}.

Method: Asynchronous execution pipeline coordinating multi-modal components, retrieval-augmented methods with history augmentation and intent-based routing, wake word detection, and emotionally expressive prosody.

Result: Integrated system enabling responsive and believable digital humans with minimal latency, supporting natural conversational flow and context-aware response generation.

Conclusion: The system successfully combines visual realism with real-time performance, making high-fidelity digital humans suitable for immersive interactive applications.

Abstract: High-fidelity digital humans are increasingly used in interactive applications, yet achieving both visual realism and real-time responsiveness remains a major challenge. We present a high-fidelity, real-time conversational digital human system that seamlessly combines a visually realistic 3D avatar, persona-driven expressive speech synthesis, and knowledge-grounded dialogue generation. To support natural and timely interaction, we introduce an asynchronous execution pipeline that coordinates multi-modal components with minimal latency. The system supports advanced features such as wake word detection, emotionally expressive prosody, and highly accurate, context-aware response generation. It leverages novel retrieval-augmented methods, including history augmentation to maintain conversational flow and intent-based routing for efficient knowledge access. Together, these components form an integrated system that enables responsive and believable digital humans, suitable for immersive applications in communication, education, and entertainment.

Method: Uses non-causal Mamba block-based model with pairwise input image fusion in selective state space for dense perception tasks.

Result: Reduces inference times while maintaining high accuracy and low GPU usage for optical flow and disparity map generation.

Conclusion: The proposed model is suitable for unified real-time and accurate 3D dense perception estimation tasks, validated in real-life scenarios.

Abstract: In this work, we propose an accurate and real-time optical flow and disparity estimation model by fusing pairwise input images in the proposed non-causal selective state space for dense perception tasks. We propose a non-causal Mamba block-based model that is fast and efficient and aptly manages the constraints present in a real-time applications. Our proposed model reduces inference times while maintaining high accuracy and low GPU usage for optical flow and disparity map generation. The results and analysis, and validation in real-life scenario justify that our proposed model can be used for unified real-time and accurate 3D dense perception estimation tasks. The code, along with the models, can be found at https://github.com/vimstereo/DensePerceptNCSSD

Method: Leverages features from Zero123 foundation model with lightweight tuning, introducing two metrics: reference-based D_PRISM and reference-free MMD_PRISM that use these enhanced features to evaluate synthesis quality.

Result: Both metrics reliably identify incorrect generations and rank models in agreement with human preference studies. MMD_PRISM produces clear and stable rankings across three benchmarks (Toys4K, GSO, OmniObject3D) with lower scores indicating stronger models.

Conclusion: The framework provides a principled and practical approach to assessing NVS quality, addressing a fundamental gap in evaluation and enabling more reliable progress in novel view synthesis.

Abstract: The goal of Novel View Synthesis (NVS) is to generate realistic images of a given content from unseen viewpoints. But how can we trust that a generated image truly reflects the intended transformation? Evaluating its reliability remains a major challenge. While recent generative models, particularly diffusion-based approaches, have significantly improved NVS quality, existing evaluation metrics struggle to assess whether a generated image is both realistic and faithful to the source view and intended viewpoint transformation. Standard metrics, such as pixel-wise similarity and distribution-based measures, often mis-rank incorrect results as they fail to capture the nuanced relationship between the source image, viewpoint change, and generated output. We propose a task-aware evaluation framework that leverages features from a strong NVS foundation model, Zero123, combined with a lightweight tuning step to enhance discrimination. Using these features, we introduce two complementary evaluation metrics: a reference-based score, $D_{\text{PRISM}}$, and a reference-free score, $\text{MMD}{\text{PRISM}}$. Both reliably identify incorrect generations and rank models in agreement with human preference studies, addressing a fundamental gap in NVS evaluation. Our framework provides a principled and practical approach to assessing synthesis quality, paving the way for more reliable progress in novel view synthesis. To further support this goal, we apply our reference-free metric to six NVS methods across three benchmarks: Toys4K, Google Scanned Objects (GSO), and OmniObject3D, where $\text{MMD}{\text{PRISM}}$ produces a clear and stable ranking, with lower scores consistently indicating stronger models.

Method: Proposed Embodied Memory Visual Reasoning (EMVR) that formulates inspection as sequential navigation over an image-based scene graph, treating images as nodes and using Markov decision process for traversal and reasoning.

Result: Evaluations show substantial performance gaps in state-of-the-art models; EMVR demonstrates strong performance over baselines on the BridgeEQA benchmark with 2,200 question-answer pairs across 200 bridge scenes.

Conclusion: BridgeEQA provides a faithful benchmark for open-vocabulary embodied question answering, and EMVR effectively addresses the challenges through sequential navigation and reasoning over visual evidence.

Abstract: Deploying embodied agents that can answer questions about their surroundings in realistic real-world settings remains difficult, partly due to the scarcity of benchmarks that faithfully capture practical operating conditions. We propose infrastructure inspection as a compelling domain for open-vocabulary Embodied Question Answering (EQA): it naturally demands multi-scale reasoning, long-range spatial understanding, and complex semantic relationships, while offering unique evaluation advantages via standardized National Bridge Inventory (NBI) condition ratings (0-9), professional inspection reports, and egocentric imagery. We introduce BridgeEQA, a benchmark of 2,200 open-vocabulary question-answer pairs (in the style of OpenEQA) grounded in professional inspection reports across 200 real-world bridge scenes with 47.93 images on average per scene. Questions require synthesizing visual evidence across multiple images and aligning responses with NBI condition ratings. We further propose a new EQA metric Image Citation Relevance to evaluate the ability of a model to cite relevant images. Evaluations of state-of-the-art vision-language models reveal substantial performance gaps under episodic memory EQA settings. To address this, we propose Embodied Memory Visual Reasoning (EMVR), which formulates inspection as sequential navigation over an image-based scene graph: images are nodes, and an agent takes actions to traverse views, compare evidence, and reason within a Markov decision process. EMVR shows strong performance over the baselines. We publicly release both the dataset and code.

Method: Two-stage Reason-and-Reject process: 1) LLM-guided anatomical reasoning planner localizes organ anchors and generates multi-scale ROIs, 2) Statistical rejection filter applies two-sample testing to candidates from frozen foundation model, retaining only those significantly different from normal tissue.

Result: Substantially improves Dice, specificity, and sensitivity on multi-center and multi-modal tumor segmentation benchmarks over strong baselines and original foundation models.

Conclusion: R²Seg provides effective OOD tumor segmentation without parameter updates, avoiding catastrophic forgetting and maintaining compatibility with zero-update test-time augmentation.

Abstract: Foundation models for medical image segmentation struggle under out-of-distribution (OOD) shifts, often producing fragmented false positives on OOD tumors. We introduce R$^{2}$Seg, a training-free framework for robust OOD tumor segmentation that operates via a two-stage Reason-and-Reject process. First, the Reason step employs an LLM-guided anatomical reasoning planner to localize organ anchors and generate multi-scale ROIs. Second, the Reject step applies two-sample statistical testing to candidates generated by a frozen foundation model (BiomedParse) within these ROIs. This statistical rejection filter retains only candidates significantly different from normal tissue, effectively suppressing false positives. Our framework requires no parameter updates, making it compatible with zero-update test-time augmentation and avoiding catastrophic forgetting. On multi-center and multi-modal tumor segmentation benchmarks, R$^{2}$Seg substantially improves Dice, specificity, and sensitivity over strong baselines and the original foundation models. Code are available at https://github.com/Eurekashen/R2Seg.

Method: Combines controlled visual perturbations, semantic clustering (entailment- and embedding-based), and robust uncertainty metrics in a reproducible pipeline applicable across multimodal architectures.

Result: Hallucination detectability varies by architecture and prompt design - highest for unified-fusion models with dense visual tokenization (Qwen2.5-VL) and lowest for restricted tokenization models (Med-Gemma). VASE metric consistently provides the most robust hallucination signal when paired with embedding clustering.

Conclusion: HEDGE provides a principled, compute-aware foundation for evaluating multimodal reliability by framing hallucination detection as a geometric robustness problem shaped by sampling scale, prompt structure, model architecture, and clustering strategy.

Abstract: Vision-language models (VLMs) enable open-ended visual question answering but remain prone to hallucinations. We present HEDGE, a unified framework for hallucination detection that combines controlled visual perturbations, semantic clustering, and robust uncertainty metrics. HEDGE integrates sampling, distortion synthesis, clustering (entailment- and embedding-based), and metric computation into a reproducible pipeline applicable across multimodal architectures. Evaluations on VQA-RAD and KvasirVQA-x1 with three representative VLMs (LLaVA-Med, Med-Gemma, Qwen2.5-VL) reveal clear architecture- and prompt-dependent trends. Hallucination detectability is highest for unified-fusion models with dense visual tokenization (Qwen2.5-VL) and lowest for architectures with restricted tokenization (Med-Gemma). Embedding-based clustering often yields stronger separation when applied directly to the generated answers, whereas NLI-based clustering remains advantageous for LLaVA-Med and for longer, sentence-level responses. Across configurations, the VASE metric consistently provides the most robust hallucination signal, especially when paired with embedding clustering and a moderate sampling budget (n ~ 10-15). Prompt design also matters: concise, label-style outputs offer clearer semantic structure than syntactically constrained one-sentence responses. By framing hallucination detection as a geometric robustness problem shaped jointly by sampling scale, prompt structure, model architecture, and clustering strategy, HEDGE provides a principled, compute-aware foundation for evaluating multimodal reliability. The hedge-bench PyPI library enables reproducible and extensible benchmarking, with full code and experimental resources available at https://github.com/Simula/HEDGE .

Method: Two complementary formulations: Jacobian-based method for any SSM architecture measuring influence through state propagation chain, and Gramian-based approach for diagonal SSMs with closed-form analytical solutions for superior speed. Both operate in single forward pass with linear complexity.

Result: SSMs naturally implement hierarchical feature refinement from diffuse low-level textures in early layers to focused, clinically meaningful patterns in deeper layers. Revealed domain-specific controllability signatures, progressive spatial selectivity, and influence of scanning strategies on attention patterns.

Conclusion: Controllability analysis establishes a unified interpretability paradigm for SSMs across domains, with applications spanning computer vision, NLP, and cross-domain tasks.

Abstract: State Space Models (SSMs), particularly the Mamba architecture, have recently emerged as powerful alternatives to Transformers for sequence modeling, offering linear computational complexity while achieving competitive performance. Yet, despite their effectiveness, understanding how these Vision SSMs process spatial information remains challenging due to the lack of transparent, attention-like mechanisms. To address this gap, we introduce a controllability-based interpretability framework that quantifies how different parts of the input sequence (tokens or patches) influence the internal state dynamics of SSMs. We propose two complementary formulations: a Jacobian-based method applicable to any SSM architecture that measures influence through the full chain of state propagation, and a Gramian-based approach for diagonal SSMs that achieves superior speed through closed-form analytical solutions. Both methods operate in a single forward pass with linear complexity, requiring no architectural modifications or hyperparameter tuning. We validate our framework through experiments on three diverse medical imaging modalities, demonstrating that SSMs naturally implement hierarchical feature refinement from diffuse low-level textures in early layers to focused, clinically meaningful patterns in deeper layers. Our analysis reveals domain-specific controllability signatures aligned with diagnostic criteria, progressive spatial selectivity across the network hierarchy, and the substantial influence of scanning strategies on attention patterns. Beyond medical imaging, we articulate applications spanning computer vision, natural language processing, and cross-domain tasks. Our framework establishes controllability analysis as a unified, foundational interpretability paradigm for SSMs across all domains. Code and analysis tools will be made available upon publication

Method: CountOCC reconstructs occluded object features by integrating spatial context from visible fragments with semantic priors from text and visual embeddings, generating class-discriminative features across pyramid levels. It also uses a visual equivalence objective to enforce attention consistency between occluded and unoccluded views.

Result: CountOCC achieves 26.72% and 20.80% MAE reduction on FSC 147 validation and test sets under occlusion, 49.89% MAE reduction on CARPK, and 28.79% MAE reduction on CAPTUREReal, setting new SOTA across diverse visual domains.

Conclusion: CountOCC effectively addresses occlusion challenges in object counting through hierarchical multimodal feature reconstruction and attention consistency enforcement, demonstrating robust amodal counting performance across various real-world scenarios.

Abstract: Object counting has achieved remarkable success on visible instances, yet state-of-the-art (SOTA) methods fail under occlusion, a pervasive challenge in real world deployment. This failure stems from a fundamental architectural limitation where backbone networks encode occluding surfaces rather than target objects, thereby corrupting the feature representations required for accurate enumeration. To address this, we present CountOCC, an amodal counting framework that explicitly reconstructs occluded object features through hierarchical multimodal guidance. Rather than accepting degraded encodings, we synthesize complete representations by integrating spatial context from visible fragments with semantic priors from text and visual embeddings, generating class-discriminative features at occluded locations across multiple pyramid levels. We further introduce a visual equivalence objective that enforces consistency in attention space, ensuring that both occluded and unoccluded views of the same scene produce spatially aligned gradient-based attention maps. Together, these complementary mechanisms preserve discriminative properties essential for accurate counting under occlusion. For rigorous evaluation, we establish occlusion-augmented versions of FSC 147 and CARPK spanning both structured and unstructured scenes. CountOCC achieves SOTA performance on FSC 147 with 26.72% and 20.80% MAE reduction over prior baselines under occlusion in validation and test, respectively. CountOCC also demonstrates exceptional generalization by setting new SOTA results on CARPK with 49.89% MAE reduction and on CAPTUREReal with 28.79% MAE reduction, validating robust amodal counting across diverse visual domains. Code will be released soon.

Method: Dual-pathway architecture with separate modules for spatial attention prediction and caption generation, maintaining semantic consistency through cross-modal alignment.

Result: Achieves competitive attention prediction performance, generates coherent context-aware explanations, and demonstrates robust zero-shot generalization across multiple driving benchmarks.

Conclusion: Effective attention-conditioned generation is achievable with limited supervision, opening possibilities for practical deployment of explainable driver attention systems.

Abstract: Understanding where drivers look and why they shift their attention is essential for autonomous systems that read human intent and justify their actions. Most existing models rely on large-scale gaze datasets to learn these patterns; however, such datasets are labor-intensive to collect and time-consuming to curate. We present FSDAM (Few-Shot Driver Attention Modeling), a framework that achieves joint attention prediction and caption generation with approximately 100 annotated examples, two orders of magnitude fewer than existing approaches. Our approach introduces a dual-pathway architecture where separate modules handle spatial prediction and caption generation while maintaining semantic consistency through cross-modal alignment. Despite minimal supervision, FSDAM achieves competitive performance on attention prediction, generates coherent, and context-aware explanations. The model demonstrates robust zero-shot generalization across multiple driving benchmarks. This work shows that effective attention-conditioned generation is achievable with limited supervision, opening new possibilities for practical deployment of explainable driver attention systems in data-constrained scenarios.

Method: Proposed TrAP (Trigger-Aware Prompt tuning) - a multi-modal backdoor injection strategy that jointly optimizes prompt parameters in both image and text modalities along with visual triggers, using curriculum-based training to progressively shrink trigger size.

Result: Achieves high attack success rates for both object misclassification and disappearance attacks across multiple datasets, while improving clean image performance on downstream datasets compared to zero-shot setting.

Conclusion: Demonstrates significant security vulnerability in open-vocabulary object detectors through prompt tuning backdoor attacks, highlighting the need for robust security measures in multi-modal vision-language systems.

Abstract: Open-vocabulary object detectors (OVODs) unify vision and language to detect arbitrary object categories based on text prompts, enabling strong zero-shot generalization to novel concepts. As these models gain traction in high-stakes applications such as robotics, autonomous driving, and surveillance, understanding their security risks becomes crucial. In this work, we conduct the first study of backdoor attacks on OVODs and reveal a new attack surface introduced by prompt tuning. We propose TrAP (Trigger-Aware Prompt tuning), a multi-modal backdoor injection strategy that jointly optimizes prompt parameters in both image and text modalities along with visual triggers. TrAP enables the attacker to implant malicious behavior using lightweight, learnable prompt tokens without retraining the base model weights, thus preserving generalization while embedding a hidden backdoor. We adopt a curriculum-based training strategy that progressively shrinks the trigger size, enabling effective backdoor activation using small trigger patches at inference. Experiments across multiple datasets show that TrAP achieves high attack success rates for both object misclassification and object disappearance attacks, while also improving clean image performance on downstream datasets compared to the zero-shot setting.

Method: Propose a KL attention loss that directly supervises visual token attention by aligning attention distributions to ground truth maps from task geometry or grounding annotations, combined with standard next-token prediction loss.

Result: Significant improvements across geometric tasks, pointing, and referring expression comprehension on both synthetic and real-world data. Also introduced a new line tracing dataset where even commercial VLMs perform poorly.

Conclusion: Direct supervision of visual token attention through KL attention loss effectively grounds answer language tokens in images and improves VLM performance on visual reasoning tasks.

Abstract: Vision Language Models (VLMs) mix visual tokens and text tokens. A puzzling issue is the fact that visual tokens most related to the query receive little to no attention in the final layers of the LLM module of VLMs from the answer tokens, where all tokens are treated equally, in particular, visual and language tokens in the LLM attention layers. This fact may result in wrong answers to visual questions, as our experimental results confirm. It appears that the standard next-token prediction (NTP) loss provides an insufficient signal for directing attention to visual tokens. We hypothesize that a more direct supervision of the attention of visual tokens to corresponding language tokens in the LLM module of VLMs will lead to improved performance on visual tasks. To demonstrate that this is indeed the case, we propose a novel loss function that directly supervises the attention of visual tokens. It directly grounds the answer language tokens in images by directing their attention to the relevant visual tokens. This is achieved by aligning the attention distribution of visual tokens to ground truth attention maps with KL divergence. The ground truth attention maps are obtained from task geometry in synthetic cases or from standard grounding annotations (e.g., bounding boxes or point annotations) in real images, and are used inside the LLM for attention supervision without requiring new labels. The obtained KL attention loss (KLAL) when combined with NTP encourages VLMs to attend to relevant visual tokens while generating answer tokens. This results in notable improvements across geometric tasks, pointing, and referring expression comprehension on both synthetic and real-world data, as demonstrated by our experiments. We also introduce a new dataset to evaluate the line tracing abilities of VLMs. Surprisingly, even commercial VLMs do not perform well on this task.

Method: Proposed multi-target regression using Kernel Point Convolutions (KPConv) with cost-sensitive learning (density-based relevance), weighted MSE, Focal Regression, and regularization. Performed sensitivity analysis on voxel sizes (0.25-2 meters).

Result: Larger voxel sizes (2m) result in lower errors due to reduced variability, while smaller voxel sizes (0.25-0.5m) show higher errors, especially in canopy areas. Bark and leaf targets had significantly higher errors at fine resolutions.

Conclusion: Voxel size choice is application-dependent. The work fills gaps in deep imbalance learning for multi-target regression and simulated datasets for 3D LiDAR forest point clouds.

Abstract: Voxelization is an effective approach to reduce the computational cost of processing Light Detection and Ranging (LiDAR) data, yet it results in a loss of fine-scale structural information. This study explores whether low-level voxel content information, specifically target occupancy percentage within a voxel, can be inferred from high-level voxelized LiDAR point cloud data collected from Digital Imaging and remote Sensing Image Generation (DIRSIG) software. In our study, the targets include bark, leaf, soil, and miscellaneous materials. We propose a multi-target regression approach in the context of imbalanced learning using Kernel Point Convolutions (KPConv). Our research leverages cost-sensitive learning to address class imbalance called density-based relevance (DBR). We employ weighted Mean Saquared Erorr (MSE), Focal Regression (FocalR), and regularization to improve the optimization of KPConv. This study performs a sensitivity analysis on the voxel size (0.25 - 2 meters) to evaluate the effect of various grid representations in capturing the nuances of the forest. This sensitivity analysis reveals that larger voxel sizes (e.g., 2 meters) result in lower errors due to reduced variability, while smaller voxel sizes (e.g., 0.25 or 0.5 meter) exhibit higher errors, particularly within the canopy, where variability is greatest. For bark and leaf targets, error values at smaller voxel size datasets (0.25 and 0.5 meter) were significantly higher than those in larger voxel size datasets (2 meters), highlighting the difficulty in accurately estimating within-canopy voxel content at fine resolutions. This suggests that the choice of voxel size is application-dependent. Our work fills the gap in deep imbalance learning models for multi-target regression and simulated datasets for 3D LiDAR point clouds of forests.

Method: Reframe interpolation as an optimal transport problem to compute geodesics between embeddings, treating them as point clouds in Wasserstein space rather than using standard matrix interpolation.

Result: Optimal transport-based interpolation produces smoother and more coherent intermediate images compared to standard interpolation methods in Stable Diffusion.

Conclusion: Viewing embeddings as point clouds rather than matrices better captures the geometry of embedding space and enables more natural interpolations through optimal transport.

Abstract: It can be shown that Stable Diffusion has a permutation-invariance property with respect to the rows of Contrastive Language-Image Pretraining (CLIP) embedding matrices. This inspired the novel observation that these embeddings can naturally be interpreted as point clouds in a Wasserstein space rather than as matrices in a Euclidean space. This perspective opens up new possibilities for understanding the geometry of embedding space. For example, when interpolating between embeddings of two distinct prompts, we propose reframing the interpolation problem as an optimal transport problem. By solving this optimal transport problem, we compute a shortest path (or geodesic) between embeddings that captures a more natural and geometrically smooth transition through the embedding space. This results in smoother and more coherent intermediate (interpolated) images when rendered by the Stable Diffusion generative model. We conduct experiments to investigate this effect, comparing the quality of interpolated images produced using optimal transport to those generated by other standard interpolation methods. The novel optimal transport–based approach presented indeed gives smoother image interpolations, suggesting that viewing the embeddings as point clouds (rather than as matrices) better reflects and leverages the geometry of the embedding space.

Method: Proposes SAGE loss function that applies salient-preserving and saliency-degrading signal augmentations to input, captures changes in embeddings and model logits, and uses contrastive triplet loss to guide model toward salient features and away from non-salient features. Includes sanity check on logit distributions.

Result: Demonstrates boost in classification performance across both open- and closed-set scenarios against state-of-the-art saliency-based methods, showing effectiveness across various backbones and wide generalization across tasks.

Conclusion: Moving saliency guidance from image space to latent space embeddings enables more effective integration of human perceptual priors into neural network training, improving generalization and performance across diverse scenarios.

Abstract: Integrating human perceptual priors into the training of neural networks has been shown to raise model generalization, serve as an effective regularizer, and align models with human expertise for applications in high-risk domains. Existing approaches to integrate saliency into model training often rely on internal model mechanisms, which recent research suggests may be unreliable. Our insight is that many challenges associated with saliency-guided training stem from the placement of the guidance approaches solely within the image space. Instead, we move away from the image space, use the model’s latent space embeddings to steer human guidance during training, and we propose SAGE (Saliency-Guided Contrastive Embeddings): a loss function that integrates human saliency into network training using contrastive embeddings. We apply salient-preserving and saliency-degrading signal augmentations to the input and capture the changes in embeddings and model logits. We guide the model towards salient features and away from non-salient features using a contrastive triplet loss. Additionally, we perform a sanity check on the logit distributions to ensure that the model outputs match the saliency-based augmentations. We demonstrate a boost in classification performance across both open- and closed-set scenarios against SOTA saliency-based methods, showing SAGE’s effectiveness across various backbones, and include experiments to suggest its wide generalization across tasks.

Method: Created RoCoISLR dataset with 9,000+ video samples across 6,000 standardized glosses, then evaluated 7 video recognition models (I3D, SlowFast, Swin Transformer, TimeSformer, Uniformer, VideoMAE, PoseConv3D) under consistent experimental setups.

Result: Transformer-based architectures outperformed convolutional baselines, with Swin Transformer achieving 34.1% Top-1 accuracy. Results highlight challenges of long-tail class distributions in low-resource sign languages.

Conclusion: RoCoISLR provides the foundational dataset for systematic Romanian Isolated Sign Language Recognition research, enabling future work in this under-resourced domain.

Abstract: Automatic sign language recognition plays a crucial role in bridging the communication gap between deaf communities and hearing individuals; however, most available datasets focus on American Sign Language. For Romanian Isolated Sign Language Recognition (RoISLR), no large-scale, standardized dataset exists, which limits research progress. In this work, we introduce a new corpus for RoISLR, named RoCoISLR, comprising over 9,000 video samples that span nearly 6,000 standardized glosses from multiple sources. We establish benchmark results by evaluating seven state-of-the-art video recognition models-I3D, SlowFast, Swin Transformer, TimeSformer, Uniformer, VideoMAE, and PoseConv3D-under consistent experimental setups, and compare their performance with that of the widely used WLASL2000 corpus. According to the results, transformer-based architectures outperform convolutional baselines; Swin Transformer achieved a Top-1 accuracy of 34.1%. Our benchmarks highlight the challenges associated with long-tail class distributions in low-resource sign languages, and RoCoISLR provides the initial foundation for systematic RoISLR research.

Method: Train a compact encoder to predict Monge-Kantorovich transport map using classical optimal transport theory.

Result: Achieves best aggregated score on real composite AR images compared to state-of-the-art methods.

Conclusion: Proposed MKL-Harmonizer enables real-time color harmonization for AR applications with released dataset and toolkit.

Abstract: Color harmonization adjusts the colors of an inserted object so that it perceptually matches the surrounding image, resulting in a seamless composite. The harmonization problem naturally arises in augmented reality (AR), yet harmonization algorithms are not currently integrated into AR pipelines because real-time solutions are scarce. In this work, we address color harmonization for AR by proposing a lightweight approach that supports on-device inference. For this, we leverage classical optimal transport theory by training a compact encoder to predict the Monge-Kantorovich transport map. We benchmark our MKL-Harmonizer algorithm against state-of-the-art methods and demonstrate that for real composite AR images our method achieves the best aggregated score. We release our dedicated AR dataset of composite images with pixel-accurate masks and data-gathering toolkit to support further data acquisition by researchers.

Method: Augments nnUNet with a channel for voxel-wise uncertainty and combines normal and cancer datasets in a unified model to segment both tumors and healthy brain structures.

Result: Achieved 0.750 correlation and 0.047 RMSD for uncertainty estimation without compromising tumor accuracy (DSC 0.86), plus 0.81 DSC for brain structures in unified model.

Conclusion: First model providing tumor segmentation in natural surroundings with overlaid uncertainty maps, offering key insights for surgical decision-making and error correction.

Abstract: Accurate segmentation of brain tumors is vital for diagnosis, surgical planning, and treatment monitoring. Deep learning has advanced on benchmarks, but two issues limit clinical use: no uncertainty estimates for errors and no segmentation of healthy brain structures around tumors for surgery. Current methods fail to unify tumor localization with anatomical context and lack confidence scores. This study presents an uncertainty-aware framework augmenting nnUNet with a channel for voxel-wise uncertainty. Trained on BraTS2023, it yields a correlation of 0.750 and RMSD of 0.047 for uncertainty without hurting tumor accuracy. It predicts uncertainty in one pass, with no extra networks or inferences, aiding clinical decisions. For whole-brain context, a unified model combines normal and cancer datasets, achieving a DSC of 0.81 for brain structures and 0.86 for tumor, with robust key-region performance. Combining both innovations gives the first model outputting tumor in natural surroundings plus an overlaid uncertainty map. Visual checks of outputs show uncertainty offers key insights to evaluate predictions and fix errors, helping informed surgical decisions from AI.

Method: Uses a novel video transformer architecture with features from a robust vision foundation model, plus a data-efficient pretrain-and-attribute strategy requiring only 0.5% source-labeled data per class, and introduces Temporal Attention Signatures for interpretability.

Result: SAGA achieves state-of-the-art attribution performance matching fully supervised methods while using minimal labeled data, and provides multi-granular attribution across authenticity, generation task, model version, development team, and precise generator.

Conclusion: SAGA sets a new benchmark for synthetic video provenance, offering crucial interpretable insights for forensic and regulatory applications through comprehensive source attribution.

Abstract: The proliferation of generative AI has led to hyper-realistic synthetic videos, escalating misuse risks and outstripping binary real/fake detectors. We introduce SAGA (Source Attribution of Generative AI videos), the first comprehensive framework to address the urgent need for AI-generated video source attribution at a large scale. Unlike traditional detection, SAGA identifies the specific generative model used. It uniquely provides multi-granular attribution across five levels: authenticity, generation task (e.g., T2V/I2V), model version, development team, and the precise generator, offering far richer forensic insights. Our novel video transformer architecture, leveraging features from a robust vision foundation model, effectively captures spatio-temporal artifacts. Critically, we introduce a data-efficient pretrain-and-attribute strategy, enabling SAGA to achieve state-of-the-art attribution using only 0.5% of source-labeled data per class, matching fully supervised performance. Furthermore, we propose Temporal Attention Signatures (T-Sigs), a novel interpretability method that visualizes learned temporal differences, offering the first explanation for why different video generators are distinguishable. Extensive experiments on public datasets, including cross-domain scenarios, demonstrate that SAGA sets a new benchmark for synthetic video provenance, providing crucial, interpretable insights for forensic and regulatory applications.

Method: Proposed Visual Chain-of-Thought (vCoT) - an explicit reasoning process that generates transitional event descriptions between consecutive frames, and systematically compared image-only LVLMs with video-finetuned counterparts.

Result: vCoT significantly improves image-only models on long-form video QA but yields only marginal gains for video-finetuned models, suggesting video models already capture frame transitions implicitly. Video models also transfer temporal reasoning to static settings.

Conclusion: Video finetuning enables implicit temporal reasoning that benefits both video understanding and static relational visual reasoning tasks.

Abstract: Multimodal large language models (LLMs) have made rapid progress in visual understanding, yet their extension from images to videos often reduces to a naive concatenation of frame tokens. In this work, we investigate what video finetuning brings to multimodal LLMs. We propose Visual Chain-of-Thought (vCoT), an explicit reasoning process that generates transitional event descriptions between consecutive frames. Using vCoT, we systematically compare image-only LVLMs with their video-finetuned counterparts, both with and without access to these transitional cues. Our experiments show that vCoT significantly improves the performance of image-only models on long-form video question answering, while yielding only marginal gains for video-finetuned models. This suggests that the latter already capture frame-to-frame transitions implicitly. Moreover, we find that video models transfer this temporal reasoning ability to purely static settings, outperforming image models’ baselines on relational visual reasoning tasks.

Method: Uses cross-modal adapter with cross-modal attention to exploit multi-modal correlations, and View-aware Consistency module with human-detection masks and confidence-weighted Jensen-Shannon divergence to handle view misalignment.

Result: Outperforms competitive distillation methods on MultiSensor-Home dataset across multiple backbones and environments, achieving significant gains and even surpassing the teacher model under limited conditions.

Conclusion: ViCoKD effectively addresses challenges in partially overlapping multi-view action recognition by distilling knowledge from multi-modal teachers to modality-limited students while handling view misalignment.

Abstract: The widespread use of multi-sensor systems has increased research in multi-view action recognition. While existing approaches in multi-view setups with fully overlapping sensors benefit from consistent view coverage, partially overlapping settings where actions are visible in only a subset of views remain underexplored. This challenge becomes more severe in real-world scenarios, as many systems provide only limited input modalities and rely on sequence-level annotations instead of dense frame-level labels. In this study, we propose View-aware Cross-modal Knowledge Distillation (ViCoKD), a framework that distills knowledge from a fully supervised multi-modal teacher to a modality- and annotation-limited student. ViCoKD employs a cross-modal adapter with cross-modal attention, allowing the student to exploit multi-modal correlations while operating with incomplete modalities. Moreover, we propose a View-aware Consistency module to address view misalignment, where the same action may appear differently or only partially across viewpoints. It enforces prediction alignment when the action is co-visible across views, guided by human-detection masks and confidence-weighted Jensen-Shannon divergence between their predicted class distributions. Experiments on the real-world MultiSensor-Home dataset show that ViCoKD consistently outperforms competitive distillation methods across multiple backbones and environments, delivering significant gains and surpassing the teacher model under limited conditions.

Method: Proposes EgoLoc with hand-dynamics-guided sampling to generate visual prompts, uses vision-language models to identify contact attributes and localize timestamps, and employs closed-loop feedback for refinement.

Result: Achieves plausible temporal interaction localization on public datasets and novel benchmarks, eliminating need for object masks and verb-noun taxonomies while enabling zero-shot implementation.

Conclusion: EgoLoc effectively facilitates downstream applications in egocentric vision and robotic manipulation, demonstrating generalizable zero-shot performance for temporal interaction localization.

Abstract: Analyzing hand-object interaction in egocentric vision facilitates VR/AR applications and human-robot policy transfer. Existing research has mostly focused on modeling the behavior paradigm of interactive actions (i.e., “how to interact”). However, the more challenging and fine-grained problem of capturing the critical moments of contact and separation between the hand and the target object (i.e., “when to interact”) is still underexplored, which is crucial for immersive interactive experiences in mixed reality and robotic motion planning. Therefore, we formulate this problem as temporal interaction localization (TIL). Some recent works extract semantic masks as TIL references, but suffer from inaccurate object grounding and cluttered scenarios. Although current temporal action localization (TAL) methods perform well in detecting verb-noun action segments, they rely on category annotations during training and exhibit limited precision in localizing hand-object contact/separation moments. To address these issues, we propose a novel zero-shot approach dubbed EgoLoc to localize hand-object contact and separation timestamps in egocentric videos. EgoLoc introduces hand-dynamics-guided sampling to generate high-quality visual prompts. It exploits the vision-language model to identify contact/separation attributes, localize specific timestamps, and provide closed-loop feedback for further refinement. EgoLoc eliminates the need for object masks and verb-noun taxonomies, leading to generalizable zero-shot implementation. Comprehensive experiments on the public dataset and our novel benchmarks demonstrate that EgoLoc achieves plausible TIL for egocentric videos. It is also validated to effectively facilitate multiple downstream applications in egocentric vision and robotic manipulation tasks. Code and relevant data will be released at https://github.com/IRMVLab/EgoLoc.

Method: Multi-modal, multi-task learning framework that predicts creativity scores, categorizes content types, and extracts stylistic features using conditional learning mechanism.

Result: Achieves state-of-the-art performance compared to existing regression approaches and provides interpretable visualizations aligned with human judgments.

Conclusion: The framework enables automatic, interpretable creativity assessment from drawings by jointly modeling content and style dimensions, offering a scalable alternative to subjective expert evaluation.

Abstract: Assessing human creativity through visual outputs, such as drawings, plays a critical role in fields including psychology, education, and cognitive science. However, current assessment practices still rely heavily on expert-based subjective scoring, which is both labor-intensive and inherently subjective. In this paper, we propose a data-driven framework for automatic and interpretable creativity assessment from drawings. Motivated by the cognitive understanding that creativity can emerge from both what is drawn (content) and how it is drawn (style), we reinterpret the creativity score as a function of these two complementary dimensions.Specifically, we first augment an existing creativity labeled dataset with additional annotations targeting content categories. Based on the enriched dataset, we further propose a multi-modal, multi-task learning framework that simultaneously predicts creativity scores, categorizes content types, and extracts stylistic features. In particular, we introduce a conditional learning mechanism that enables the model to adapt its visual feature extraction by dynamically tuning it to creativity-relevant signals conditioned on the drawing’s stylistic and semantic cues.Experimental results demonstrate that our model achieves state-of-the-art performance compared to existing regression-based approaches and offers interpretable visualizations that align well with human judgments. The code and annotations will be made publicly available at https://github.com/WonderOfU9/CSCA_PRCV_2025

Method: Decomposes FFNs into lightweight expert sub-networks with learnable router for token-specific expert selection, plus gated token selector for high-update-potential tokens while reconstructing unselected tokens. Uses two-stage knowledge distillation supervised by original VAR model.

Result: Achieves up to 21.2% FLOPs reduction on ImageNet 256×256 benchmark with minimal performance degradation.

Conclusion: ActVAR enables efficient VAR model inference through dynamic activation framework that preserves model capacity while significantly reducing computational costs.

Abstract: Visual Autoregressive (VAR) models enable efficient image generation via next-scale prediction but face escalating computational costs as sequence length grows. Existing static pruning methods degrade performance by permanently removing weights or tokens, disrupting pretrained dependencies. To address this, we propose ActVAR, a dynamic activation framework that introduces dual sparsity across model weights and token sequences to enhance efficiency without sacrificing capacity. ActVAR decomposes feedforward networks (FFNs) into lightweight expert sub-networks and employs a learnable router to dynamically select token-specific expert subsets based on content. Simultaneously, a gated token selector identifies high-update-potential tokens for computation while reconstructing unselected tokens to preserve global context and sequence alignment. Training employs a two-stage knowledge distillation strategy, where the original VAR model supervises the learning of routing and gating policies to align with pretrained knowledge. Experiments on the ImageNet $256\times 256$ benchmark demonstrate that ActVAR achieves up to $21.2%$ FLOPs reduction with minimal performance degradation.

Method: Proposes Native High dynamic range 3D Gaussian Splatting (NH-3DGS) with a novel luminance-chromaticity decomposition of color representation that preserves full dynamic range throughout the reconstruction pipeline, enabling direct optimization from native HDR camera data.

Result: NH-3DGS significantly outperforms existing methods in reconstruction quality and dynamic range preservation on both synthetic and real multi-view HDR datasets.

Conclusion: The method enables professional-grade 3D reconstruction directly from native HDR captures, advancing the applicability of 3D scene reconstruction to professional digital media workflows.

Abstract: High Dynamic Range (HDR) imaging is essential for professional digital media creation, e.g., filmmaking, virtual production, and photorealistic rendering. However, 3D scene reconstruction has primarily focused on Low Dynamic Range (LDR) data, limiting its applicability to professional workflows. Existing approaches that reconstruct HDR scenes from LDR observations rely on multi-exposure fusion or inverse tone-mapping, which increase capture complexity and depend on synthetic supervision. With the recent emergence of cameras that directly capture native HDR data in a single exposure, we present the first method for 3D scene reconstruction that directly models native HDR observations. We propose {\bf Native High dynamic range 3D Gaussian Splatting (NH-3DGS)}, which preserves the full dynamic range throughout the reconstruction pipeline. Our key technical contribution is a novel luminance-chromaticity decomposition of the color representation that enables direct optimization from native HDR camera data. We demonstrate on both synthetic and real multi-view HDR datasets that NH-3DGS significantly outperforms existing methods in reconstruction quality and dynamic range preservation, enabling professional-grade 3D reconstruction directly from native HDR captures. Code and datasets will be made available.

Method: Frequency-Decomposition Preprocessing (FDP) framework that uses frequency-domain reconstruction for simultaneous pathology suppression and anatomical preservation. It integrates with existing anomaly simulation techniques.

Result: FDP consistently improves anomaly detection performance across diverse architectures, achieving 17.63% increase in DICE score with LDM while maintaining robust improvements across multiple baselines.

Conclusion: FDP effectively enhances unsupervised anomaly detection in brain MRI by leveraging frequency-domain properties, providing a systematic approach that maintains diagnostic fidelity and integrates with existing methods.

Abstract: Due to the diversity of brain anatomy and the scarcity of annotated data, supervised anomaly detection for brain MRI remains challenging, driving the development of unsupervised anomaly detection (UAD) approaches. Current UAD methods typically utilize artificially generated noise perturbations on healthy MRIs to train generative models for normal anatomy reconstruction, enabling anomaly detection via residual mapping. However, such simulated anomalies lack the biophysical fidelity and morphological complexity characteristic of true clinical lesions. To advance UAD in brain MRI, we conduct the first systematic frequency-domain analysis of pathological signatures, revealing two key properties: (1) anomalies exhibit unique frequency patterns distinguishable from normal anatomy, and (2) low-frequency signals maintain consistent representations across healthy scans. These insights motivate our Frequency-Decomposition Preprocessing (FDP) framework, the first UAD method to leverage frequency-domain reconstruction for simultaneous pathology suppression and anatomical preservation. FDP can integrate seamlessly with existing anomaly simulation techniques, consistently enhancing detection performance across diverse architectures while maintaining diagnostic fidelity. Experimental results demonstrate that FDP consistently improves anomaly detection performance when integrated with existing methods. Notably, FDP achieves a 17.63% increase in DICE score with LDM while maintaining robust improvements across multiple baselines. The code is available at https://github.com/ls1rius/MRI_FDP.

Method: Proposes an end-to-end MLLM framework with active iterative reasoning using specialized frame-extraction tool, data distillation pipeline synthesizing CoT trajectories from 10 data sources, and two-stage training (SFT + RL with gated tool-use reward).

Result: Achieves state-of-the-art performance on 6.7k question benchmark, significantly outperforming both proprietary and open-source baseline models.

Conclusion: Establishes a new foundation for domain-specific video reasoning to address the complexities of diverse sports through learned reasoning processes.

Abstract: Sports video understanding presents unique challenges, requiring models to perceive high-speed dynamics, comprehend complex rules, and reason over long temporal contexts. While Multimodal Large Language Models (MLLMs) have shown promise in genral domains, the current state of research in sports remains narrowly focused: existing approaches are either single-sport centric, limited to specific tasks, or rely on training-free paradigms that lack robust, learned reasoning process. To address this gap, we introduce DeepSport, the first end-to-end trained MLLM framework designed for multi-task, multi-sport video understanding. DeepSport shifts the paradigm from passive frame processing to active, iterative reasoning, empowering the model to ``think with videos’’ by dynamically interrogating content via a specialized frame-extraction tool. To enable this, we propose a data distillation pipeline that synthesizes high-quality Chain-of-Thought (CoT) trajectories from 10 diverse data source, creating a unified resource of 78k training data. We then employ a two-stage training strategy, Supervised Fine-Tuning (SFT) followed by Reinforcement Learning (RL) with a novel gated tool-use reward, to optimize the model’s reasoning process. Extensive experiments on the testing benchmark of 6.7k questions demonstrate that DeepSport achieves state-of-the-art performance, significantly outperforming baselines of both proprietary model and open-source models. Our work establishes a new foundation for domain-specific video reasoning to address the complexities of diverse sports.

Method: Proposes CASL framework based on reconstruction paradigm using U-Net architecture with multi-scale curvature prompts to guide decoder in predicting spatial coordinates. Uses only curvature as anomaly score and achieves detection through straightforward anomaly classification fine-tuning.

Result: Outperforms classical self-supervised and dedicated anomaly detection models. Achieves leading detection performance and the learned representations generalize well to standard 3D understanding tasks like point cloud classification.

Conclusion: Curvature plays a critical role in 3D anomaly detection. CASL demonstrates that effective anomaly detection can be achieved without dedicated mechanisms, and the framework provides generalizable representations for broader 3D understanding tasks.

Abstract: Deep learning-based 3D anomaly detection methods have demonstrated significant potential in industrial manufacturing. However, many approaches are specifically designed for anomaly detection tasks, which limits their generalizability to other 3D understanding tasks. In contrast, self-supervised point cloud models aim for general-purpose representation learning, yet our investigation reveals that these classical models are suboptimal at anomaly detection under the unified fine-tuning paradigm. This motivates us to develop a more generalizable 3D model that can effectively detect anomalies without relying on task-specific designs. Interestingly, we find that using only the curvature of each point as its anomaly score already outperforms several classical self-supervised and dedicated anomaly detection models, highlighting the critical role of curvature in 3D anomaly detection. In this paper, we propose a Curvature-Augmented Self-supervised Learning (CASL) framework based on a reconstruction paradigm. Built upon the classical U-Net architecture, our approach introduces multi-scale curvature prompts to guide the decoder in predicting the spatial coordinates of each point. Without relying on any dedicated anomaly detection mechanisms, it achieves leading detection performance through straightforward anomaly classification fine-tuning. Moreover, the learned representations generalize well to standard 3D understanding tasks such as point cloud classification. The code is available at https://github.com/zyh16143998882/CASL.

Method: Develops Multimodal Noise Generator (MuNG) that analyzes cross-modal relationships in image-text pairs to generate task-adaptive beneficial noise, which is injected into frozen MLLMs to suppress irrelevant semantic components.

Result: Experiments on QwenVL and LLaVA show the method surpasses full-parameter fine-tuning and other existing approaches while requiring only 1-2% additional parameters.

Conclusion: The proposed noise injection strategy effectively improves cross-modal representation alignment and enhances downstream task performance with minimal parameter overhead.

Abstract: Multimodal Large Language Models (MLLMs) have played an increasingly important role in multimodal intelligence. However, the existing fine-tuning methods often ignore cross-modal heterogeneity, limiting their full potential. In this work, we propose a novel fine-tuning strategy by injecting beneficial random noise, which outperforms previous methods and even surpasses full fine-tuning, with minimal additional parameters. The proposed Multimodal Noise Generator (MuNG) enables efficient modality fine-tuning by injecting customized noise into the frozen MLLMs. Specifically, we reformulate the reasoning process of MLLMs from a variational inference perspective, upon which we design a multimodal noise generator that dynamically analyzes cross-modal relationships in image-text pairs to generate task-adaptive beneficial noise. Injecting this type of noise into the MLLMs effectively suppresses irrelevant semantic components, leading to significantly improved cross-modal representation alignment and enhanced performance on downstream tasks. Experiments on two mainstream MLLMs, QwenVL and LLaVA, demonstrate that our method surpasses full-parameter fine-tuning and other existing fine-tuning approaches, while requiring adjustments to only about $1\sim2%$ additional parameters. The relevant code is uploaded in the supplementary.

Method: Formulates 3D-3D correspondences as discrete tokens using coordinate map tokenization, modality-decoupled encoding for RGB and coordinate cues, and autoregressive transformer decoder conditioned on query features and partial token sequences.

Result: Significantly outperforms existing methods on multiple benchmarks and demonstrates strong robustness to symmetry, occlusion, and other real-world challenges.

Conclusion: CoordAR provides an effective autoregressive approach for one-reference 6D pose estimation that overcomes key limitations of previous methods through probabilistic modeling and discrete tokenization.

Abstract: Object 6D pose estimation, a crucial task for robotics and augmented reality applications, becomes particularly challenging when dealing with novel objects whose 3D models are not readily available. To reduce dependency on 3D models, recent studies have explored one-reference-based pose estimation, which requires only a single reference view instead of a complete 3D model. However, existing methods that rely on real-valued coordinate regression suffer from limited global consistency due to the local nature of convolutional architectures and face challenges in symmetric or occluded scenarios owing to a lack of uncertainty modeling. We present CoordAR, a novel autoregressive framework for one-reference 6D pose estimation of unseen objects. CoordAR formulates 3D-3D correspondences between the reference and query views as a map of discrete tokens, which is obtained in an autoregressive and probabilistic manner. To enable accurate correspondence regression, CoordAR introduces 1) a novel coordinate map tokenization that enables probabilistic prediction over discretized 3D space; 2) a modality-decoupled encoding strategy that separately encodes RGB appearance and coordinate cues; and 3) an autoregressive transformer decoder conditioned on both position-aligned query features and the partially generated token sequence. With these novel mechanisms, CoordAR significantly outperforms existing methods on multiple benchmarks and demonstrates strong robustness to symmetry, occlusion, and other challenges in real-world tests.

Method: Introduces a decoupled cross-attention mechanism to disentangle camera motion from photographic inputs, and develops a comprehensive data generation strategy using simulated effects and real-world collection to build a large-scale training dataset.

Result: The model generates high-fidelity videos with precisely controlled, user-specified photographic camera effects, as demonstrated through extensive experiments.

Conclusion: CineCtrl successfully enables fine-grained, independent control over professional camera parameters in video generation without compromising scene consistency.

Abstract: Cinematic storytelling is profoundly shaped by the artful manipulation of photographic elements such as depth of field and exposure. These effects are crucial in conveying mood and creating aesthetic appeal. However, controlling these effects in generative video models remains highly challenging, as most existing methods are restricted to camera motion control. In this paper, we propose CineCtrl, the first video cinematic editing framework that provides fine control over professional camera parameters (e.g., bokeh, shutter speed). We introduce a decoupled cross-attention mechanism to disentangle camera motion from photographic inputs, allowing fine-grained, independent control without compromising scene consistency. To overcome the shortage of training data, we develop a comprehensive data generation strategy that leverages simulated photographic effects with a dedicated real-world collection pipeline, enabling the construction of a large-scale dataset for robust model training. Extensive experiments demonstrate that our model generates high-fidelity videos with precisely controlled, user-specified photographic camera effects.

Method: Uses a unified framework with controllable mask mechanism, incorporates vehicle-side and roadside multi-view data, employs two-stage training (conceptual learning then fine-tuning), and introduces mask-region-weighted loss for small traffic elements.

Result: Achieves leading performance in text-based image generation and editing within traffic scenes, with enhanced geometric diversity and generation fidelity.

Conclusion: The proposed framework effectively addresses key challenges in traffic scene generation and editing, providing a robust solution for intelligent transportation applications.

Abstract: With the rapid advancement of intelligent transportation systems, text-driven image generation and editing techniques have demonstrated significant potential in providing rich, controllable visual scene data for applications such as traffic monitoring and autonomous driving. However, several challenges remain, including insufficient semantic richness of generated traffic elements, limited camera viewpoints, low visual fidelity of synthesized images, and poor alignment between textual descriptions and generated content. To address these issues, we propose a unified text-driven framework for both image generation and editing, leveraging a controllable mask mechanism to seamlessly integrate the two tasks. Furthermore, we incorporate both vehicle-side and roadside multi-view data to enhance the geometric diversity of traffic scenes. Our training strategy follows a two-stage paradigm: first, we perform conceptual learning using large-scale coarse-grained text-image data; then, we fine-tune with fine-grained descriptive data to enhance text-image alignment and detail quality. Additionally, we introduce a mask-region-weighted loss that dynamically emphasizes small yet critical regions during training, thereby substantially enhancing the generation fidelity of small-scale traffic elements. Extensive experiments demonstrate that our method achieves leading performance in text-based image generation and editing within traffic scenes.

Method: Two-stage approach: (1) Fine-tune pose-aware diffusion model with ControlNet for pose estimation and Condition Prior Preservation Loss (CPPL) for few-shot training; (2) Distill 3D avatar using NeRF representation with canonical SMPL-X space sampling and Multi-Resolution 3D-SDS.

Result: Achieves 48× speed-up (5 minutes personalization), outperforms state-of-the-art in reconstruction fidelity, detail preservation, and robustness to occlusions/truncations. Preserves high-frequency textures and handles occlusions correctly.

Conclusion: PFAvatar advances practical 3D avatar generation from real-world OOTD albums and supports downstream applications like virtual try-on, animation, and human video reenactment.

Abstract: We propose PFAvatar (Pose-Fusion Avatar), a new method that reconstructs high-quality 3D avatars from ``Outfit of the Day’’ (OOTD) photos, which exhibit diverse poses, occlusions, and complex backgrounds. Our method consists of two stages: (1) fine-tuning a pose-aware diffusion model from few-shot OOTD examples and (2) distilling a 3D avatar represented by a neural radiance field (NeRF). In the first stage, unlike previous methods that segment images into assets (e.g., garments, accessories) for 3D assembly, which is prone to inconsistency, we avoid decomposition and directly model the full-body appearance. By integrating a pre-trained ControlNet for pose estimation and a novel Condition Prior Preservation Loss (CPPL), our method enables end-to-end learning of fine details while mitigating language drift in few-shot training. Our method completes personalization in just 5 minutes, achieving a 48$\times$ speed-up compared to previous approaches. In the second stage, we introduce a NeRF-based avatar representation optimized by canonical SMPL-X space sampling and Multi-Resolution 3D-SDS. Compared to mesh-based representations that suffer from resolution-dependent discretization and erroneous occluded geometry, our continuous radiance field can preserve high-frequency textures (e.g., hair) and handle occlusions correctly through transmittance. Experiments demonstrate that PFAvatar outperforms state-of-the-art methods in terms of reconstruction fidelity, detail preservation, and robustness to occlusions/truncations, advancing practical 3D avatar generation from real-world OOTD albums. In addition, the reconstructed 3D avatar supports downstream applications such as virtual try-on, animation, and human video reenactment, further demonstrating the versatility and practical value of our approach.

Method: Uses Grounded SAM with text prompts to localize objects, introduces Anomaly-Map-Guided Attention with Region Guidance Factor to distinguish regions, and employs prototype learning for clustering unseen anomaly classes.

Result: Outperforms state-of-the-art approaches on MVTec AD, MTD, and Real-IAD datasets.

Conclusion: The framework successfully discovers unseen anomaly classes while enabling multi-type classification and achieves task-level unification for outlier detection.

Abstract: Existing industrial anomaly detection methods mainly determine whether an anomaly is present. However, real-world applications also require discovering and classifying multiple anomaly types. Since industrial anomalies are semantically subtle and current methods do not sufficiently exploit image priors, direct clustering approaches often perform poorly. To address these challenges, we propose ProtoAnomalyNCD, a prototype-learning-based framework for discovering unseen anomaly classes of multiple types that can be integrated with various anomaly detection methods. First, to suppress background clutter, we leverage Grounded SAM with text prompts to localize object regions as priors for the anomaly classification network. Next, because anomalies usually appear as subtle and fine-grained patterns on the product, we introduce an Anomaly-Map-Guided Attention block. Within this block, we design a Region Guidance Factor that helps the attention module distinguish among background, object regions, and anomalous regions. By using both localized product regions and anomaly maps as priors, the module enhances anomalous features while suppressing background noise and preserving normal features for contrastive learning. Finally, under a unified prototype-learning framework, ProtoAnomalyNCD discovers and clusters unseen anomaly classes while simultaneously enabling multi-type anomaly classification. We further extend our method to detect unseen outliers, achieving task-level unification. Our method outperforms state-of-the-art approaches on the MVTec AD, MTD, and Real-IAD datasets.

Method: Uses semi-supervised learning with teacher-student framework where teacher generates pseudo HDR ground truths, then applies uncertainty-based masking at pixel and patch levels to discard unreliable parts of pseudo ground truths before student learns from trusted areas.

Result: Outperforms previous annotation-efficient algorithms and achieves comparable performance with state-of-the-art fully-supervised methods using only 6.7% HDR ground truths.

Conclusion: The proposed uncertainty-based masking process effectively addresses confirmation bias in semi-supervised HDR reconstruction, enabling high performance with significantly reduced annotation requirements.

Abstract: Reconstructing high dynamic range (HDR) images from low dynamic range (LDR) bursts plays an essential role in the computational photography. Impressive progress has been achieved by learning-based algorithms which require LDR-HDR image pairs. However, these pairs are hard to obtain, which motivates researchers to delve into the problem of annotation-efficient HDR image reconstructing: how to achieve comparable performance with limited HDR ground truths (GTs). This work attempts to address this problem from the view of semi-supervised learning where a teacher model generates pseudo HDR GTs for the LDR samples without GTs and a student model learns from pseudo GTs. Nevertheless, the confirmation bias, i.e., the student may learn from the artifacts in pseudo HDR GTs, presents an impediment. To remove this impediment, an uncertainty-based masking process is proposed to discard unreliable parts of pseudo GTs at both pixel and patch levels, then the trusted areas can be learned from by the student. With this novel masking process, our semi-supervised HDR reconstructing method not only outperforms previous annotation-efficient algorithms, but also achieves comparable performance with up-to-date fully-supervised methods by using only 6.7% HDR GTs.

Method: Proposes Recurrent Autoregressive Diffusion (RAD) framework with LSTM integration in diffusion transformers, enabling frame-wise autoregression with consistent memory update across training and inference.

Result: Achieves superior performance on Memory Maze and Minecraft datasets, demonstrating LSTM’s efficiency in sequence modeling for long video generation.

Conclusion: RAD framework effectively enhances historical information retention within fixed memory budget, outperforming existing approaches in long video generation tasks.

Abstract: Recent advancements in video generation have demonstrated the potential of using video diffusion models as world models, with autoregressive generation of infinitely long videos through masked conditioning. However, such models, usually with local full attention, lack effective memory compression and retrieval for long-term generation beyond the window size, leading to issues of forgetting and spatiotemporal inconsistencies. To enhance the retention of historical information within a fixed memory budget, we introduce a recurrent neural network (RNN) into the diffusion transformer framework. Specifically, a diffusion model incorporating LSTM with attention achieves comparable performance to state-of-the-art RNN blocks, such as TTT and Mamba2. Moreover, existing diffusion-RNN approaches often suffer from performance degradation due to training-inference gap or the lack of overlap across windows. To address these limitations, we propose a novel Recurrent Autoregressive Diffusion (RAD) framework, which executes frame-wise autoregression for memory update and retrieval, consistently across training and inference time. Experiments on Memory Maze and Minecraft datasets demonstrate the superiority of RAD for long video generation, highlighting the efficiency of LSTM in sequence modeling.

Method: Deep learning architecture using the Hyper-Kvasir dataset, incorporating clinically relevant metrics and a novel thermal-aware procedure for model robustness and efficiency.

Result: Achieves 88.3% mAP for polyp detection, 69% Dice coefficient for segmentation, and real-time inference speeds exceeding 35 FPS on GPU hardware.

Conclusion: The integrated AI solution is designed for seamless deployment in endoscopy workflows, promising to advance diagnostic accuracy and clinical decision-making in gastrointestinal healthcare.

Abstract: Precise and real-time detection of gastrointestinal polyps during endoscopic procedures is crucial for early diagnosis and prevention of colorectal cancer. This work presents EndoSight AI, a deep learning architecture developed and evaluated independently to enable accurate polyp localization and detailed boundary delineation. Leveraging the publicly available Hyper-Kvasir dataset, the system achieves a mean Average Precision (mAP) of 88.3% for polyp detection and a Dice coefficient of up to 69% for segmentation, alongside real-time inference speeds exceeding 35 frames per second on GPU hardware. The training incorporates clinically relevant performance metrics and a novel thermal-aware procedure to ensure model robustness and efficiency. This integrated AI solution is designed for seamless deployment in endoscopy workflows, promising to advance diagnostic accuracy and clinical decision-making in gastrointestinal healthcare.

Method: Proposes a T2I-based attack pipeline with CLIP-based loss and masked prompts for improved focus and controllability, plus two novel style customization methods to guide visual appearance and enhance out-of-domain generalization and stealthiness.

Result: Achieves 83.3% average physical-world attack success rate under varied real-world conditions (different distances, angles, light conditions, and sign categories), demonstrating high effectiveness and transferability.

Conclusion: DiffSign overcomes limitations of prior approaches by generating physically robust, highly effective, transferable, practical, and stealthy appearance attacks against TSR systems.

Abstract: Traffic Sign Recognition (TSR) systems play a critical role in Autonomous Driving (AD) systems, enabling real-time detection of road signs, such as STOP and speed limit signs. While these systems are increasingly integrated into commercial vehicles, recent research has exposed their vulnerability to physical-world adversarial appearance attacks. In such attacks, carefully crafted visual patterns are misinterpreted by TSR models as legitimate traffic signs, while remaining inconspicuous or benign to human observers. However, existing adversarial appearance attacks suffer from notable limitations. Pixel-level perturbation-based methods often lack stealthiness and tend to overfit to specific surrogate models, resulting in poor transferability to real-world TSR systems. On the other hand, text-to-image (T2I) diffusion model-based approaches demonstrate limited effectiveness and poor generalization to out-of-distribution sign types. In this paper, we present DiffSign, a novel T2I-based appearance attack framework designed to generate physically robust, highly effective, transferable, practical, and stealthy appearance attacks against TSR systems. To overcome the limitations of prior approaches, we propose a carefully designed attack pipeline that integrates CLIP-based loss and masked prompts to improve attack focus and controllability. We also propose two novel style customization methods to guide visual appearance and improve out-of-domain traffic sign attack generalization and attack stealthiness. We conduct extensive evaluations of DiffSign under varied real-world conditions, including different distances, angles, light conditions, and sign categories. Our method achieves an average physical-world attack success rate of 83.3%, leveraging DiffSign’s high effectiveness in attack transferability.

Method: CalibrateMix uses training dynamics to identify easy and hard samples, then performs targeted mixup between these sample types to improve calibration.

Result: Experimental results show CalibrateMix achieves lower expected calibration error (ECE) and superior accuracy across multiple benchmark image datasets.

Conclusion: Targeted mixup based on sample difficulty effectively improves SSL model calibration while maintaining or enhancing classification performance.

Abstract: Semi-supervised learning (SSL) has demonstrated high performance in image classification tasks by effectively utilizing both labeled and unlabeled data. However, existing SSL methods often suffer from poor calibration, with models yielding overconfident predictions that misrepresent actual prediction likelihoods. Recently, neural networks trained with {\tt mixup} that linearly interpolates random examples from the training set have shown better calibration in supervised settings. However, calibration of neural models remains under-explored in semi-supervised settings. Although effective in supervised model calibration, random mixup of pseudolabels in SSL presents challenges due to the overconfidence and unreliability of pseudolabels. In this work, we introduce CalibrateMix, a targeted mixup-based approach that aims to improve the calibration of SSL models while maintaining or even improving their classification accuracy. Our method leverages training dynamics of labeled and unlabeled samples to identify easy-to-learn'' and hard-to-learn’’ samples, which in turn are utilized in a targeted mixup of easy and hard samples. Experimental results across several benchmark image datasets show that our method achieves lower expected calibration error (ECE) and superior accuracy compared to existing SSL approaches.

Method: GrOCE models concepts as a dynamic semantic graph with three components: Dynamic Topological Graph Construction, Adaptive Cluster Identification with multi-hop traversal and similarity-decay scoring, and Selective Edge Severing for targeted removal.

Result: Extensive experiments show GrOCE achieves state-of-the-art performance on Concept Similarity (CS) and Fréchet Inception Distance (FID) metrics.

Conclusion: GrOCE offers efficient, accurate, and stable concept erasure without retraining through principled graph-based reasoning over semantic dependencies.

Abstract: Concept erasure aims to remove harmful, inappropriate, or copyrighted content from text-to-image diffusion models while preserving non-target semantics. However, existing methods either rely on costly fine-tuning or apply coarse semantic separation, often degrading unrelated concepts and lacking adaptability to evolving concept sets. To alleviate this issue, we propose Graph-Guided Online Concept Erasure (GrOCE), a training-free framework that performs precise and adaptive concept removal through graph-based semantic reasoning. GrOCE models concepts and their interrelations as a dynamic semantic graph, enabling principled reasoning over dependencies and fine-grained isolation of undesired content. It comprises three components: (1) Dynamic Topological Graph Construction for incremental graph building, (2) Adaptive Cluster Identification for multi-hop traversal with similarity-decay scoring, and (3) Selective Edge Severing for targeted edge removal while preserving global semantics. Extensive experiments demonstrate that GrOCE achieves state-of-the-art performance on Concept Similarity (CS) and Fréchet Inception Distance (FID) metrics, offering efficient, accurate, and stable concept erasure without retraining.

Method: HiFusion integrates two components: Hierarchical Intra-Spot Modeling that extracts fine-grained morphological representations through multi-resolution sub-patch decomposition with feature alignment, and Context-aware Cross-scale Fusion that selectively incorporates relevant regional context using cross-attention.

Result: Extensive experiments on two benchmark ST datasets show HiFusion achieves state-of-the-art performance in both 2D slide-wise cross-validation and more challenging 3D sample-specific scenarios.

Conclusion: HiFusion demonstrates potential as a robust, accurate, and scalable solution for spatial transcriptomics inference from routine histopathology, overcoming limitations of existing approaches.

Abstract: Spatial transcriptomics (ST) bridges gene expression and tissue morphology but faces clinical adoption barriers due to technical complexity and prohibitive costs. While computational methods predict gene expression from H&E-stained whole-slide images (WSIs), existing approaches often fail to capture the intricate biological heterogeneity within spots and are susceptible to morphological noise when integrating contextual information from surrounding tissue. To overcome these limitations, we propose HiFusion, a novel deep learning framework that integrates two complementary components. First, we introduce the Hierarchical Intra-Spot Modeling module that extracts fine-grained morphological representations through multi-resolution sub-patch decomposition, guided by a feature alignment loss to ensure semantic consistency across scales. Concurrently, we present the Context-aware Cross-scale Fusion module, which employs cross-attention to selectively incorporate biologically relevant regional context, thereby enhancing representational capacity. This architecture enables comprehensive modeling of both cellular-level features and tissue microenvironmental cues, which are essential for accurate gene expression prediction. Extensive experiments on two benchmark ST datasets demonstrate that HiFusion achieves state-of-the-art performance across both 2D slide-wise cross-validation and more challenging 3D sample-specific scenarios. These results underscore HiFusion’s potential as a robust, accurate, and scalable solution for ST inference from routine histopathology.

Method: Uses five morphological metrics (fractal dimension, texture entropy, gradient variance, edge density, contour complexity) to characterize local visual morphology and guide spatially adaptive bit allocation, combined with curriculum-based quantization-aware training.

Result: Achieves 85.6% mAP@0.5 with average 4.2 bits and 7.6x compression ratio, outperforming uniform 4-bit quantization by 3.5 percentage points with only 1.8ms additional runtime overhead. Consistent gains validated on COCO and Pascal VOC datasets.

Conclusion: Morphology-driven spatial quantization enhances efficiency and robustness for computationally constrained, safety-critical visual recognition tasks, demonstrating strong correlation between morphological complexity and quantization sensitivity.

Abstract: Most neural network quantization methods apply uniform bit precision across spatial regions, ignoring the heterogeneous structural and textural complexity of visual data. This paper introduces MCAQ-YOLO, a morphological complexity-aware quantization framework for object detection. The framework employs five morphological metrics - fractal dimension, texture entropy, gradient variance, edge density, and contour complexity - to characterize local visual morphology and guide spatially adaptive bit allocation. By correlating these metrics with quantization sensitivity, MCAQ-YOLO dynamically adjusts bit precision according to spatial complexity. In addition, a curriculum-based quantization-aware training scheme progressively increases quantization difficulty to stabilize optimization and accelerate convergence. Experimental results demonstrate a strong correlation between morphological complexity and quantization sensitivity and show that MCAQ-YOLO achieves superior detection accuracy and convergence efficiency compared with uniform quantization. On a safety equipment dataset, MCAQ-YOLO attains 85.6 percent mAP@0.5 with an average of 4.2 bits and a 7.6x compression ratio, yielding 3.5 percentage points higher mAP than uniform 4-bit quantization while introducing only 1.8 ms of additional runtime overhead per image. Cross-dataset validation on COCO and Pascal VOC further confirms consistent performance gains, indicating that morphology-driven spatial quantization can enhance efficiency and robustness for computationally constrained, safety-critical visual recognition tasks.

Method: Propose UNSEEN framework that scores samples from generalization perspective using models not exposed to them during training, with multi-step incremental selection through models trained on varying coresets for dynamic optimization.

Result: Significantly outperforms SOTA methods on CIFAR-10, CIFAR-100, and ImageNet-1K, achieving lossless performance while reducing training data by 30% on ImageNet-1K.

Conclusion: Generalization-based scoring and multi-step incremental selection effectively address limitations of fitting-centric approaches, enabling more effective dataset pruning with comparable performance to full datasets.

Abstract: The growing scale of datasets in deep learning has introduced significant computational challenges. Dataset pruning addresses this challenge by constructing a compact but informative coreset from the full dataset with comparable performance. Previous approaches typically establish scoring metrics based on specific criteria to identify representative samples. However, these methods predominantly rely on sample scores obtained from the model’s performance during the training (i.e., fitting) phase. As scoring models achieve near-optimal performance on training data, such fitting-centric approaches induce a dense distribution of sample scores within a narrow numerical range. This concentration reduces the distinction between samples and hinders effective selection. To address this challenge, we conduct dataset pruning from the perspective of generalization, i.e., scoring samples based on models not exposed to them during training. We propose a plug-and-play framework, UNSEEN, which can be integrated into existing dataset pruning methods. Additionally, conventional score-based methods are single-step and rely on models trained solely on the complete dataset, providing limited perspective on the importance of samples. To address this limitation, we scale UNSEEN to multi-step scenarios and propose an incremental selection technique through scoring models trained on varying coresets, and optimize the quality of the coreset dynamically. Extensive experiments demonstrate that our method significantly outperforms existing state-of-the-art (SOTA) methods on CIFAR-10, CIFAR-100, and ImageNet-1K. Notably, on ImageNet-1K, UNSEEN achieves lossless performance while reducing training data by 30%.

Method: Uses Arti4URDF pipeline with 3D point clouds, LLM prior knowledge, and URDF-oriented prompts to identify articulable objects and reconstruct executable URDF models.

Result: Outperforms existing approaches across 3D simulated objects, full scenes, and real-world scans, achieving state-of-the-art performance while preserving geometry and interactivity.

Conclusion: Provides practical path for building interactive, robot-ready simulation environments directly from existing 3D assets.

Abstract: Building interactive simulators and scalable robot-learning environments requires a large number of articulated assets. However, most existing 3D assets in simulation are rigid, and manually converting them into articulated objects is extremely labor- and cost-intensive. This raises a natural question: can we automatically identify articulable objects in a scene and convert them into articulated assets directly? In this paper, we present ArtiWorld, a scene-aware pipeline that localizes candidate articulable objects from textual scene descriptions and reconstructs executable URDF models that preserve the original geometry. At the core of this pipeline is Arti4URDF, which leverages 3D point cloud, prior knowledge of a large language model (LLM), and a URDF-oriented prompt design to rapidly convert rigid objects into interactive URDF-based articulated objects while maintaining their 3D shape. We evaluate ArtiWorld at three levels: 3D simulated objects, full 3D simulated scenes, and real-world scan scenes. Across all three settings, our method consistently outperforms existing approaches and achieves state-of-the-art performance, while preserving object geometry and correctly capturing object interactivity to produce usable URDF-based articulated models. This provides a practical path toward building interactive, robot-ready simulation environments directly from existing 3D assets. Code and data will be released.

Method: SAGE explores prompt templates and selects those that create the largest semantic separation between classes, guided by theoretical analysis of multimodal spurious bias in zero-shot classification.

Result: Extensive experiments on 4 benchmark datasets and 5 backbone models show SAGE consistently improves zero-shot performance and generalization, outperforming previous zero-shot approaches without external knowledge.

Conclusion: SAGE effectively mitigates multimodal spurious bias in CLIP models through guided prompt selection, enhancing robustness while maintaining zero-shot capabilities without requiring training or fine-tuning.

Abstract: Large vision-language models, such as CLIP, have shown strong zero-shot classification performance by aligning images and text in a shared embedding space. However, CLIP models often develop multimodal spurious biases, which is the undesirable tendency to rely on spurious features. For example, CLIP may infer object types in images based on frequently co-occurring backgrounds rather than the object’s core features. This bias significantly impairs the robustness of pre-trained CLIP models on out-of-distribution data, where such cross-modal associations no longer hold. Existing methods for mitigating multimodal spurious bias typically require fine-tuning on downstream data or prior knowledge of the bias, which undermines the out-of-the-box usability of CLIP. In this paper, we first theoretically analyze the impact of multimodal spurious bias in zero-shot classification. Based on this insight, we propose Spuriousness-Aware Guided Exploration (SAGE), a simple and effective method that mitigates spurious bias through guided prompt selection. SAGE requires no training, fine-tuning, or external annotations. It explores a space of prompt templates and selects the prompts that induce the largest semantic separation between classes, thereby improving worst-group robustness. Extensive experiments on four real-world benchmark datasets and five popular backbone models demonstrate that SAGE consistently improves zero-shot performance and generalization, outperforming previous zero-shot approaches without any external knowledge or model updates.

Method: CCI clusters CLIP’s patch embeddings into semantic groups, masks them, and evaluates prediction changes. Combined with GroundedSAM, it categorizes predictions as foreground- or background-driven.

Result: CCI achieves more than twofold improvement on deletion-AUC for MS COCO retrieval and sets new SOTA on faithfulness benchmarks. COVAR benchmark reveals that background correlations aren’t the only source of errors - viewpoint, scale, and fine-grained confusions also contribute.

Conclusion: CCI provides superior interpretability for VLMs, and the COVAR benchmark enables comprehensive evaluation of model robustness, charting a path toward more reliable vision-language models.

Abstract: Contrastive vision-language models (VLMs) such as CLIP achieve strong zero-shot recognition yet remain vulnerable to spurious correlations, particularly background over-reliance. We introduce Cluster-based Concept Importance (CCI), a novel interpretability method that uses CLIP’s own patch embeddings to group spatial patches into semantically coherent clusters, mask them, and evaluate relative changes in model predictions. CCI sets a new state of the art on faithfulness benchmarks, surpassing prior methods by large margins; for example, it yields more than a twofold improvement on the deletion-AUC metric for MS COCO retrieval. We further propose that CCI, when combined with GroundedSAM, automatically categorizes predictions as foreground- or background-driven, providing a crucial diagnostic ability. Existing benchmarks such as CounterAnimals, however, rely solely on accuracy and implicitly attribute all performance degradation to background correlations. Our analysis shows this assumption to be incomplete, since many errors arise from viewpoint variation, scale shifts, and fine-grained object confusions. To disentangle these effects, we introduce COVAR, a benchmark that systematically varies object foregrounds and backgrounds. Leveraging CCI with COVAR, we present a comprehensive evaluation of eighteen CLIP variants, offering methodological advances and empirical evidence that chart a path toward more robust VLMs.

Method: Uses RAE latent space with pre-trained vision encoder, implements Consistency Mid-Training for initialization, two-stage training with distillation from pre-trained flow matching teacher and optional bootstrapping with one-point velocity estimator.

Result: Achieves 1-step FID of 2.03 (vs vanilla MF’s 3.43), reduces sampling GFLOPS by 38% and total training cost by 83% on ImageNet 256. Scales to ImageNet 512 with 1-step FID of 3.23.

Conclusion: Proposed method removes need for guidance, simplifies training configurations, and significantly reduces computation while achieving superior performance.

Abstract: MeanFlow (MF) is a diffusion-motivated generative model that enables efficient few-step generation by learning long jumps directly from noise to data. In practice, it is often used as a latent MF by leveraging the pre-trained Stable Diffusion variational autoencoder (SD-VAE) for high-dimensional data modeling. However, MF training remains computationally demanding and is often unstable. During inference, the SD-VAE decoder dominates the generation cost, and MF depends on complex guidance hyperparameters for class-conditional generation. In this work, we develop an efficient training and sampling scheme for MF in the latent space of a Representation Autoencoder (RAE), where a pre-trained vision encoder (e.g., DINO) provides semantically rich latents paired with a lightweight decoder. We observe that naive MF training in the RAE latent space suffers from severe gradient explosion. To stabilize and accelerate training, we adopt Consistency Mid-Training for trajectory-aware initialization and use a two-stage scheme: distillation from a pre-trained flow matching teacher to speed convergence and reduce variance, followed by an optional bootstrapping stage with a one-point velocity estimator to further reduce deviation from the oracle mean flow. This design removes the need for guidance, simplifies training configurations, and reduces computation in both training and sampling. Empirically, our method achieves a 1-step FID of 2.03, outperforming vanilla MF’s 3.43, while reducing sampling GFLOPS by 38% and total training cost by 83% on ImageNet 256. We further scale our approach to ImageNet 512, achieving a competitive 1-step FID of 3.23 with the lowest GFLOPS among all baselines. Code is available at https://github.com/sony/mf-rae.

Method: Proposes WSAE-Net with two key innovations: weighted semantic maps to reduce non-semantic feature computations, and auto-adaptive candidate editing sequences to determine optimal processing order while maintaining semantic relevance.

Result: The methodology demonstrates superior performance in comprehensive experiments, enabling more efficient generation of counterfactuals while preserving semantic relationships.

Conclusion: WSAE-Net contributes to clearer and deeper understanding of visual counterfactual explanations by addressing semantic relevance and computational efficiency challenges in traditional approaches.

Abstract: In the domain of non-generative visual counterfactual explanations (CE), traditional techniques frequently involve the substitution of sections within a query image with corresponding sections from distractor images. Such methods have historically overlooked the semantic relevance of the replacement regions to the target object, thereby impairing the model’s interpretability and hindering the editing workflow. Addressing these challenges, the present study introduces an innovative methodology named as Weighted Semantic Map with Auto-adaptive Candidate Editing Network (WSAE-Net). Characterized by two significant advancements: the determination of an weighted semantic map and the auto-adaptive candidate editing sequence. First, the generation of the weighted semantic map is designed to maximize the reduction of non-semantic feature units that need to be computed, thereby optimizing computational efficiency. Second, the auto-adaptive candidate editing sequences are designed to determine the optimal computational order among the feature units to be processed, thereby ensuring the efficient generation of counterfactuals while maintaining the semantic relevance of the replacement feature units to the target object. Through comprehensive experimentation, our methodology demonstrates superior performance, contributing to a more lucid and in-depth understanding of visual counterfactual explanations.

Method: Proposes SpectralAdapt framework with two key components: Spectral Density Masking (SDM) that adaptively masks RGB channels based on spectral complexity, and Spectral Endmember Representation Alignment (SERA) that uses physically interpretable endmembers as domain-invariant anchors with momentum updates.

Result: Experiments show consistent improvements in spectral fidelity, cross-domain generalization, and training stability on benchmark datasets.

Conclusion: SpectralAdapt effectively mitigates domain shift, spectral degradation, and data scarcity in HSI reconstruction, demonstrating SSDA as an efficient solution for hyperspectral imaging in healthcare.

Abstract: Hyperspectral imaging (HSI) holds great potential for healthcare due to its rich spectral information. However, acquiring HSI data remains costly and technically demanding. Hyperspectral image reconstruction offers a practical solution by recovering HSI data from accessible modalities, such as RGB. While general domain datasets are abundant, the scarcity of human HSI data limits progress in medical applications. To tackle this, we propose SpectralAdapt, a semi-supervised domain adaptation (SSDA) framework that bridges the domain gap between general and human-centered HSI datasets. To fully exploit limited labels and abundant unlabeled data, we enhance spectral reasoning by introducing Spectral Density Masking (SDM), which adaptively masks RGB channels based on their spectral complexity, encouraging recovery of informative regions from complementary cues during consistency training. Furthermore, we introduce Spectral Endmember Representation Alignment (SERA), which derives physically interpretable endmembers from valuable labeled pixels and employs them as domain-invariant anchors to guide unlabeled predictions, with momentum updates ensuring adaptability and stability. These components are seamlessly integrated into SpectralAdapt, a spectral prior-guided framework that effectively mitigates domain shift, spectral degradation, and data scarcity in HSI reconstruction. Experiments on benchmark datasets demonstrate consistent improvements in spectral fidelity, cross-domain generalization, and training stability, highlighting the promise of SSDA as an efficient solution for hyperspectral imaging in healthcare.

Method: Uses parameter maps with attribute values in semantic regions as input to construct explicit parameter-to-image mapping, with semantic replacement and parameter perturbation mechanisms for better boundary perception, and VLM-driven agents with feedback-driven rethinking and scene-aware memory.

Result: Extensive experiments demonstrate each component’s effectiveness and superior performance in personalized image retouching.

Conclusion: PerTouch provides an effective unified framework for personalized image retouching that better aligns with user intent and captures long-term preferences.

Abstract: Image retouching aims to enhance visual quality while aligning with users’ personalized aesthetic preferences. To address the challenge of balancing controllability and subjectivity, we propose a unified diffusion-based image retouching framework called PerTouch. Our method supports semantic-level image retouching while maintaining global aesthetics. Using parameter maps containing attribute values in specific semantic regions as input, PerTouch constructs an explicit parameter-to-image mapping for fine-grained image retouching. To improve semantic boundary perception, we introduce semantic replacement and parameter perturbation mechanisms in the training process. To connect natural language instructions with visual control, we develop a VLM-driven agent that can handle both strong and weak user instructions. Equipped with mechanisms of feedback-driven rethinking and scene-aware memory, PerTouch better aligns with user intent and captures long-term preferences. Extensive experiments demonstrate each component’s effectiveness and the superior performance of PerTouch in personalized image retouching. Code is available at: https://github.com/Auroral703/PerTouch.

Method: Uses channel-wise alignment between volumetric prompts and text embeddings, preserves full 3D context, employs lightweight 3D convolutional module for voxel-space refinement, supports two prompting modes (text-only and hybrid), and implements dynamic resampling for data augmentation.

Result: Outperforms SAT with DSC of 75.44 vs 69.83, NSD of 77.34 vs 71.06, F1 of 38.24 vs 24.88, and DSC TP of 65.46 vs 46.97 on five-modality average, while reducing inference time by over 90% for 24-class segmentation.

Conclusion: Medal S successfully harmonizes spatial precision with semantic textual guidance, demonstrating superior efficiency and accuracy in multi-class medical segmentation tasks compared to sequential prompt-based approaches.

Abstract: We introduce Medal S, a medical segmentation foundation model that supports native-resolution spatial and textual prompts within an end-to-end trainable framework. Unlike text-only methods lacking spatial awareness, Medal S achieves channel-wise alignment between volumetric prompts and text embeddings, mitigating inaccuracies from resolution mismatches. By preserving full 3D context, it efficiently processes multiple native-resolution masks in parallel, enhancing multi-class segmentation performance. A lightweight 3D convolutional module enables precise voxel-space refinement guided by both prompt types, supporting up to 243 classes across CT, MRI, PET, ultrasound, and microscopy modalities in the BiomedSegFM dataset. Medal S offers two prompting modes: a text-only mode, where model predictions serve as spatial prompts for self-refinement without human input, and a hybrid mode, incorporating manual annotations for enhanced flexibility. For 24-class segmentation, parallel spatial prompting reduces inference time by more than 90% compared to sequential prompting. We propose dynamic resampling to address target-patch ratio imbalance, extending SAT and nnU-Net for data augmentation. Furthermore, we develop optimized text preprocessing, a two-stage inference strategy, and post-processing techniques to improve memory efficiency, precision, and inference speed. On the five-modality average on the validation set, Medal S outperforms SAT with a DSC of 75.44 (vs. 69.83), NSD of 77.34 (vs. 71.06), F1 of 38.24 (vs. 24.88), and DSC TP of 65.46 (vs. 46.97). Medal S achieves excellent performance by harmonizing spatial precision with semantic textual guidance, demonstrating superior efficiency and accuracy in multi-class medical segmentation tasks compared to sequential prompt-based approaches. Medal S will be publicly available at https://github.com/yinghemedical/Medal-S.

Method: Uses scale-wise autoregressive model with three techniques: Identity Prompt Replacement to mitigate context bias, and unified attention guidance with Adaptive Style Injection and Synchronized Guidance Adaptation for global consistency.

Result: Achieves high identity and style consistency across diverse prompts with 1.72 seconds per image inference speed (6x faster than existing methods).

Conclusion: The framework is effective and practical for real-world visual storytelling, operating entirely at test time without fine-tuning.

Abstract: We present Infinite-Story, a training-free framework for consistent text-to-image (T2I) generation tailored for multi-prompt storytelling scenarios. Built upon a scale-wise autoregressive model, our method addresses two key challenges in consistent T2I generation: identity inconsistency and style inconsistency. To overcome these issues, we introduce three complementary techniques: Identity Prompt Replacement, which mitigates context bias in text encoders to align identity attributes across prompts; and a unified attention guidance mechanism comprising Adaptive Style Injection and Synchronized Guidance Adaptation, which jointly enforce global style and identity appearance consistency while preserving prompt fidelity. Unlike prior diffusion-based approaches that require fine-tuning or suffer from slow inference, Infinite-Story operates entirely at test time, delivering high identity and style consistency across diverse prompts. Extensive experiments demonstrate that our method achieves state-of-the-art generation performance, while offering over 6X faster inference (1.72 seconds per image) than the existing fastest consistent T2I models, highlighting its effectiveness and practicality for real-world visual storytelling.

Method: DTGS performs joint optimization across enhancement, geometry, and thermal supervision through a cyclic enhancement-reconstruction mechanism, using thermal guidance to stabilize color restoration and geometry learning, and embedding Retinex-based decomposition within the 3DGS loop.

Result: Extensive experiments on the new RGBT-LOW dataset show DTGS significantly outperforms existing low-light enhancement and 3D reconstruction baselines, achieving superior radiometric consistency, geometric fidelity, and color stability under extreme illumination.

Conclusion: DTGS provides a unified solution for illumination-invariant reconstruction under extreme low-light conditions by tightly coupling enhancement with 3D reconstruction through thermal guidance and Retinex decomposition.

Abstract: Under extremely low-light conditions, novel view synthesis (NVS) faces severe degradation in terms of geometry, color consistency, and radiometric stability. Standard 3D Gaussian Splatting (3DGS) pipelines fail when applied directly to underexposed inputs, as independent enhancement across views causes illumination inconsistencies and geometric distortion. To address this, we present DTGS, a unified framework that tightly couples Retinex-inspired illumination decomposition with thermal-guided 3D Gaussian Splatting for illumination-invariant reconstruction. Unlike prior approaches that treat enhancement as a pre-processing step, DTGS performs joint optimization across enhancement, geometry, and thermal supervision through a cyclic enhancement-reconstruction mechanism. A thermal supervisory branch stabilizes both color restoration and geometry learning by dynamically balancing enhancement, structural, and thermal losses. Moreover, a Retinex-based decomposition module embedded within the 3DGS loop provides physically interpretable reflectance-illumination separation, ensuring consistent color and texture across viewpoints. To evaluate our method, we construct RGBT-LOW, a new multi-view low-light thermal dataset capturing severe illumination degradation. Extensive experiments show that DTGS significantly outperforms existing low-light enhancement and 3D reconstruction baselines, achieving superior radiometric consistency, geometric fidelity, and color stability under extreme illumination.

Method: Proposes BP-FPN with Gradient-Isolated Low-Level Shortcut (GILS) to incorporate fine-grained target details without shortcut learning, and Directional Gradient Regularization (DGR) to enforce hierarchical feature consistency during backpropagation.

Result: Extensive experiments on multiple public datasets show that BP-FPN consistently establishes new state-of-the-art performance.

Conclusion: BP-FPN is the first FPN designed for infrared small target detection entirely from the backpropagation perspective, offering theoretically grounded design with negligible computational overhead that can be seamlessly integrated into existing frameworks.

Abstract: Moving infrared small target detection is a key component of infrared search and tracking systems, yet it remains extremely challenging due to low signal-to-clutter ratios, severe target-background imbalance, and weak discriminative features. Existing deep learning methods primarily focus on spatio-temporal feature aggregation, but their gains are limited, revealing that the fundamental bottleneck lies in ambiguous per-frame feature representations rather than spatio-temporal modeling itself. Motivated by this insight, we propose BP-FPN, a backpropagation-driven feature pyramid architecture that fundamentally rethinks feature learning for small target. BP-FPN introduces Gradient-Isolated Low-Level Shortcut (GILS) to efficiently incorporate fine-grained target details without inducing shortcut learning, and Directional Gradient Regularization (DGR) to enforce hierarchical feature consistency during backpropagation. The design is theoretically grounded, introduces negligible computational overhead, and can be seamlessly integrated into existing frameworks. Extensive experiments on multiple public datasets show that BP-FPN consistently establishes new state-of-the-art performance. To the best of our knowledge, it is the first FPN designed for this task entirely from the backpropagation perspective.

Method: Uses a Light-Geometry Dual-Branch Encoder to extract multi-illumination cues and geometric priors from frozen 3D reconstruction models, and introduces PS-Perp dataset with perspective projection to address orthographic projection limitations.

Result: State-of-the-art performance across multiple datasets, especially in complex in-the-wild scenes, both quantitatively and qualitatively.

Conclusion: Integrating geometric priors from 3D foundation models significantly improves universal photometric stereo performance in challenging real-world conditions.

Abstract: Universal Photometric Stereo is a promising approach for recovering surface normals without strict lighting assumptions. However, it struggles when multi-illumination cues are unreliable, such as under biased lighting or in shadows or self-occluded regions of complex in-the-wild scenes. We propose GeoUniPS, a universal photometric stereo network that integrates synthetic supervision with high-level geometric priors from large-scale 3D reconstruction models pretrained on massive in-the-wild data. Our key insight is that these 3D reconstruction models serve as visual-geometry foundation models, inherently encoding rich geometric knowledge of real scenes. To leverage this, we design a Light-Geometry Dual-Branch Encoder that extracts both multi-illumination cues and geometric priors from the frozen 3D reconstruction model. We also address the limitations of the conventional orthographic projection assumption by introducing the PS-Perp dataset with realistic perspective projection to enable learning of spatially varying view directions. Extensive experiments demonstrate that GeoUniPS delivers state-of-the-arts performance across multiple datasets, both quantitatively and qualitatively, especially in the complex in-the-wild scenes.

Method: Proposed Reference-Frame × Granularity (RFxG) taxonomy framework and four novel faithfulness metrics to systematically evaluate explanation quality across both dimensions, applied to ten saliency methods, four model architectures, and three datasets.

Result: Demonstrated critical limitations in existing evaluation metrics that prioritize pointwise faithfulness while neglecting contrastive reasoning and semantic granularity. Comprehensive evaluation framework provides tools for user-intent-driven assessment.

Conclusion: The work provides conceptual foundation and practical tools to develop visual explanations that are faithful to model behavior and meaningfully aligned with human understanding and inquiry complexity.

Abstract: Saliency maps are widely used for visual explanations in deep learning, but a fundamental lack of consensus persists regarding their intended purpose and alignment with diverse user queries. This ambiguity hinders the effective evaluation and practical utility of explanation methods.We address this gap by introducing the Reference-Frame $\times$ Granularity (RFxG) taxonomy, a principled conceptual framework that organizes saliency explanations along two essential axes:Reference-Frame: Distinguishing between pointwise (“Why this prediction?”) and contrastive (“Why this and not an alternative?”) explanations.Granularity: Ranging from fine-grained class-level (e.g., “Why Husky?”) to coarse-grained group-level (e.g., “Why Dog?”) interpretations.Using the RFxG lens, we demonstrate critical limitations in existing evaluation metrics, which overwhelmingly prioritize pointwise faithfulness while neglecting contrastive reasoning and semantic granularity. To systematically assess explanation quality across both RFxG dimensions, we propose four novel faithfulness metrics. Our comprehensive evaluation framework applies these metrics to ten state-of-the-art saliency methods, four model architectures, and three datasets.By advocating a shift toward user-intent-driven evaluation, our work provides both the conceptual foundation and the practical tools necessary to develop visual explanations that are not only faithful to the underlying model behavior but are also meaningfully aligned with the complexity of human understanding and inquiry.

Method: Proposed REVISOR framework with Dual Attribution Decoupled Reward (DADR) mechanism integrated into GRPO training strategy, enabling MLLMs to collaboratively construct introspective reflection processes across textual and visual modalities.

Result: Significantly enhances long-form video understanding capability of MLLMs without requiring supplementary supervised fine-tuning or external models, achieving impressive results on four benchmarks including VideoMME, LongVideoBench, MLVU, and LVBench.

Conclusion: REVISOR framework successfully addresses the limitations of text-only reflection in long-form video understanding by enabling cross-modal collaborative reflection, demonstrating substantial improvements in multimodal reasoning capabilities.

Abstract: Self-reflection mechanisms that rely on purely text-based rethinking processes perform well in most multimodal tasks. However, when directly applied to long-form video understanding scenarios, they exhibit clear limitations. The fundamental reasons for this lie in two points: (1)long-form video understanding involves richer and more dynamic visual input, meaning rethinking only the text information is insufficient and necessitates a further rethinking process specifically targeting visual information; (2) purely text-based reflection mechanisms lack cross-modal interaction capabilities, preventing them from fully integrating visual information during reflection. Motivated by these insights, we propose REVISOR (REflective VIsual Segment Oriented Reasoning), a novel framework for tool-augmented multimodal reflection. REVISOR enables MLLMs to collaboratively construct introspective reflection processes across textual and visual modalities, significantly enhancing their reasoning capability for long-form video understanding. To ensure that REVISOR can learn to accurately review video segments highly relevant to the question during reinforcement learning, we designed the Dual Attribution Decoupled Reward (DADR) mechanism. Integrated into the GRPO training strategy, this mechanism enforces causal alignment between the model’s reasoning and the selected video evidence. Notably, the REVISOR framework significantly enhances long-form video understanding capability of MLLMs without requiring supplementary supervised fine-tuning or external models, achieving impressive results on four benchmarks including VideoMME, LongVideoBench, MLVU, and LVBench.

Method: 1) Use MobileSAM for instance mask extraction from input images; 2) 3D Semantic Group Attention module with linear attention for object-centric feature aggregation; 3) Global Similarity-Guided Attention module to handle segmentation errors; 4) Instance-aware Local Diffusion module for feature refinement in BEV space.

Result: Achieves state-of-the-art performance on SemanticKITTI (17.40 mIoU) and SSCBench-KITTI360 (20.28 mIoU) benchmarks.

Conclusion: Object-centric decomposition enables more accurate semantic occupancy prediction in complex 3D scenes, demonstrating superior performance over ego-centric approaches.

Abstract: Vision-based 3D Semantic Scene Completion (SSC) has received growing attention due to its potential in autonomous driving. While most existing approaches follow an ego-centric paradigm by aggregating and diffusing features over the entire scene, they often overlook fine-grained object-level details, leading to semantic and geometric ambiguities, especially in complex environments. To address this limitation, we propose Ocean, an object-centric prediction framework that decomposes the scene into individual object instances to enable more accurate semantic occupancy prediction. Specifically, we first employ a lightweight segmentation model, MobileSAM, to extract instance masks from the input image. Then, we introduce a 3D Semantic Group Attention module that leverages linear attention to aggregate object-centric features in 3D space. To handle segmentation errors and missing instances, we further design a Global Similarity-Guided Attention module that leverages segmentation features for global interaction. Finally, we propose an Instance-aware Local Diffusion module that improves instance features through a generative process and subsequently refines the scene representation in the BEV space. Extensive experiments on the SemanticKITTI and SSCBench-KITTI360 benchmarks demonstrate that Ocean achieves state-of-the-art performance, with mIoU scores of 17.40 and 20.28, respectively.

Method: Proposes two attack modes: SILA (static constant lighting intensity) and DILA (dynamic lighting changes at critical moments), both manipulating global illumination in a black-box manner.

Result: ILA significantly increases failure rates and reduces trajectory efficiency across two state-of-the-art VLN models and three navigation tasks.

Conclusion: VLN agents have previously unrecognized vulnerabilities to realistic indoor lighting variations, highlighting the need for more robust navigation systems.

Abstract: Vision-and-Language Navigation (VLN) agents have made remarkable progress, but their robustness remains insufficiently studied. Existing adversarial evaluations often rely on perturbations that manifest as unusual textures rarely encountered in everyday indoor environments. Errors under such contrived conditions have limited practical relevance, as real-world agents are unlikely to encounter such artificial patterns. In this work, we focus on indoor lighting, an intrinsic yet largely overlooked scene attribute that strongly influences navigation. We propose Indoor Lighting-based Adversarial Attack (ILA), a black-box framework that manipulates global illumination to disrupt VLN agents. Motivated by typical household lighting usage, we design two attack modes: Static Indoor Lighting-based Attack (SILA), where the lighting intensity remains constant throughout an episode, and Dynamic Indoor Lighting-based Attack (DILA), where lights are switched on or off at critical moments to induce abrupt illumination changes. We evaluate ILA on two state-of-the-art VLN models across three navigation tasks. Results show that ILA significantly increases failure rates while reducing trajectory efficiency, revealing previously unrecognized vulnerabilities of VLN agents to realistic indoor lighting variations.

Method: Introduces Unified Interactive Volume (UIV) - a volumetric representation that encodes heterogeneous interactive entities into a shared spatial field, enabling consistent relational reasoning and joint-wise probabilistic prediction.

Result: Achieves competitive performance across three interaction tasks and demonstrates strong generalization to novel entity combinations.

Conclusion: Unified modeling of compound interactions offers a promising direction for scalable motion synthesis in complex environments.

Abstract: We present Uni-Inter, a unified framework for human motion generation that supports a wide range of interaction scenarios: including human-human, human-object, and human-scene-within a single, task-agnostic architecture. In contrast to existing methods that rely on task-specific designs and exhibit limited generalization, Uni-Inter introduces the Unified Interactive Volume (UIV), a volumetric representation that encodes heterogeneous interactive entities into a shared spatial field. This enables consistent relational reasoning and compound interaction modeling. Motion generation is formulated as joint-wise probabilistic prediction over the UIV, allowing the model to capture fine-grained spatial dependencies and produce coherent, context-aware behaviors. Experiments across three representative interaction tasks demonstrate that Uni-Inter achieves competitive performance and generalizes well to novel combinations of entities. These results suggest that unified modeling of compound interactions offers a promising direction for scalable motion synthesis in complex environments.

Method: Freeze both pretrained image encoder and multilingual text encoder, train only a compact 1.7M-parameter projection module using contrastive loss over English representations as semantic anchors. No image-text pairs or text-text pairs required.

Result: Significant gains in retrieval performance for five underrepresented languages (Czech, Finnish, Croatian, Hungarian, Romanian) on Crossmodal-3600 benchmark. Robust multilingual alignment achieved even with limited supervision.

Conclusion: The pivot-based, parameter-efficient alignment strategy effectively enables inclusive multimodal learning for underrepresented languages, demonstrating that minimal training with English anchors can bridge the multilingual gap in vision-language models.

Abstract: Contrastive Language-Image Pre-training (CLIP) has demonstrated strong generalization across a wide range of visual tasks by leveraging large-scale English-image pairs. However, its extension to low-resource languages remains limited due to the scarcity of high-quality multilingual image-text data. Existing multilingual vision-language models exhibit consistently low retrieval performance in underrepresented languages including Czech, Finnish, Croatian, Hungarian, and Romanian on the Crossmodal-3600 (XM3600) benchmark. To address this, we propose a lightweight and data-efficient framework for multilingual vision-language alignment. Our approach requires no image-text pairs or text-text pairs and freezes both the pretrained image encoder and multilingual text encoder during training. Only a compact 1.7M-parameter projection module is trained, using a contrastive loss over English representations as semantic anchors. This minimal training setup enables robust multilingual alignment even for languages with limited supervision. Extensive evaluation across multiple multilingual retrieval benchmarks confirms the effectiveness of our method, showing significant gains in five underrepresented languages where existing models typically underperform. These findings highlight the effectiveness of our pivot-based, parameter-efficient alignment strategy for inclusive multimodal learning.

Method: Proposes MGCA-Net with four components: localizer for action proposals, action presence predictor, conventional classifier for base actions at snippet granularity, and coarse-to-fine classifier for novel actions at video and proposal granularities.

Result: Achieves state-of-the-art performance on THUMOS'14 and ActivityNet-1.3 benchmarks, and also achieves state-of-the-art results under Zero-Shot Temporal Action Localization setting.

Conclusion: Multi-grained category awareness effectively enhances localization performance for both base and novel action categories in open-vocabulary temporal action localization.

Abstract: Open-Vocabulary Temporal Action Localization (OV-TAL) aims to recognize and localize instances of any desired action categories in videos without explicitly curating training data for all categories. Existing methods mostly recognize action categories at a single granularity, which degrades the recognition accuracy of both base and novel action categories. To address these issues, we propose a Multi-Grained Category-Aware Network (MGCA-Net) comprising a localizer, an action presence predictor, a conventional classifier, and a coarse-to-fine classifier. Specifically, the localizer localizes category-agnostic action proposals. For these action proposals, the action presence predictor estimates the probability that they belong to an action instance. At the same time, the conventional classifier predicts the probability of each action proposal over base action categories at the snippet granularity. Novel action categories are recognized by the coarse-to-fine classifier, which first identifies action presence at the video granularity. Finally, it assigns each action proposal to one category from the coarse categories at the proposal granularity. Through coarse-to-fine category awareness for novel actions and the conventional classifier’s awareness of base actions, multi-grained category awareness is achieved, effectively enhancing localization performance. Comprehensive evaluations on the THUMOS'14 and ActivityNet-1.3 benchmarks demonstrate that our method achieves state-of-the-art performance. Furthermore, our MGCA-Net achieves state-of-the-art results under the Zero-Shot Temporal Action Localization setting.

Method: Evaluated synthetic data generated with GANs for model training, applied CNNs for road distress segmentation, and examined transformer-based MaskFormer model.

Result: GAN-generated data improves model performance. MaskFormer outperforms CNN model in mAP50 and IoU metrics.

Conclusion: Computer vision methods using synthetic data and transformer-based models show promise for efficient road distress monitoring and infrastructure management.

Abstract: The American Society of Civil Engineers has graded Americas infrastructure condition as a C, with the road system receiving a dismal D. Roads are vital to regional economic viability, yet their management, maintenance, and repair processes remain inefficient, relying on outdated manual or laser-based inspection methods that are both costly and time-consuming. With the increasing availability of real-time visual data from autonomous vehicles, there is an opportunity to apply computer vision (CV) methods for advanced road monitoring, providing insights to guide infrastructure rehabilitation efforts. This project explores the use of state-of-the-art CV techniques for road distress segmentation. It begins by evaluating synthetic data generated with Generative Adversarial Networks (GANs) to assess its usefulness for model training. The study then applies Convolutional Neural Networks (CNNs) for road distress segmentation and subsequently examines the transformer-based model MaskFormer. Results show that GAN-generated data improves model performance and that MaskFormer outperforms the CNN model in two metrics: mAP50 and IoU.

Method: Proposes DiffPixelFormer with Intra-Inter Modal Interaction Block (IIMIB) that uses self-attention for intra-modal dependencies and Differential-Shared Inter-Modal (DSIM) module to disentangle modality-specific and shared cues. Includes dynamic fusion strategy to balance modality contributions based on scene characteristics.

Result: Achieves mIoU scores of 54.28% on SUN RGB-D and 59.95% on NYUDv2, outperforming DFormer-L by 1.78% and 2.75% respectively.

Conclusion: DiffPixelFormer effectively addresses limitations in existing RGB-D fusion methods by enabling fine-grained pixel-level cross-modal alignment and dynamic modality balancing, demonstrating superior performance for indoor semantic segmentation tasks.

Abstract: Indoor semantic segmentation is fundamental to computer vision and robotics, supporting applications such as autonomous navigation, augmented reality, and smart environments. Although RGB-D fusion leverages complementary appearance and geometric cues, existing methods often depend on computationally intensive cross-attention mechanisms and insufficiently model intra- and inter-modal feature relationships, resulting in imprecise feature alignment and limited discriminative representation. To address these challenges, we propose DiffPixelFormer, a differential pixel-aware Transformer for RGB-D indoor scene segmentation that simultaneously enhances intra-modal representations and models inter-modal interactions. At its core, the Intra-Inter Modal Interaction Block (IIMIB) captures intra-modal long-range dependencies via self-attention and models inter-modal interactions with the Differential-Shared Inter-Modal (DSIM) module to disentangle modality-specific and shared cues, enabling fine-grained, pixel-level cross-modal alignment. Furthermore, a dynamic fusion strategy balances modality contributions and fully exploits RGB-D information according to scene characteristics. Extensive experiments on the SUN RGB-D and NYUDv2 benchmarks demonstrate that DiffPixelFormer-L achieves mIoU scores of 54.28% and 59.95%, outperforming DFormer-L by 1.78% and 2.75%, respectively. Code is available at https://github.com/gongyan1/DiffPixelFormer.

Method: 1) Pretext-GRPO: Self-supervised reinforcement learning algorithm that rewards correct solving of pretext tasks on transformed visual inputs. 2) ViSS-R1: Framework integrating pretext-task-based self-supervised learning into MLLM’s R1 post-training, forcing models to reason about transformed visual inputs by processing both pretext questions and user queries.

Result: Comprehensive evaluations on six video reasoning benchmarks demonstrate the effectiveness and superiority of Pretext-GRPO and ViSS-R1 for complex video reasoning.

Conclusion: The proposed frameworks successfully address visual underutilization in video reasoning by compelling models to non-trivially process visual information through pretext tasks, leading to more robust video understanding.

Abstract: Complex video reasoning remains a significant challenge for Multimodal Large Language Models (MLLMs), as current R1-based methodologies often prioritize text-centric reasoning derived from text-based and image-based developments. In video tasks, such strategies frequently underutilize rich visual information, leading to potential shortcut learning and increased susceptibility to hallucination. To foster a more robust, visual-centric video understanding, we start by introducing a novel self-supervised reinforcement learning GRPO algorithm (Pretext-GRPO) within the standard R1 pipeline, in which positive rewards are assigned for correctly solving pretext tasks on transformed visual inputs, which makes the model to non-trivially process the visual information. Building on the effectiveness of Pretext-GRPO, we further propose the ViSS-R1 framework, which streamlines and integrates pretext-task-based self-supervised learning directly into the MLLM’s R1 post-training paradigm. Instead of relying solely on sparse visual cues, our framework compels models to reason about transformed visual input by simultaneously processing both pretext questions (concerning transformations) and true user queries. This necessitates identifying the applied transformation and reconstructing the original video to formulate accurate final answers. Comprehensive evaluations on six widely-used video reasoning and understanding benchmarks demonstrate the effectiveness and superiority of our Pretext-GRPO and ViSS-R1 for complex video reasoning. Our codes and models will be publicly available.

Method: Proposes SOMA with two key components: Feature Gradient Enhancer (FGE) that embeds multi-scale, multi-directional gradient filters into feature space using attention and reconstruction, and Global-Local Affine-Flow Matcher (GLAM) that combines affine transformation and flow-based refinement in a coarse-to-fine architecture.

Result: Significantly improves registration precision, increasing CMR@1px by 12.29% on SEN1-2 dataset and 18.50% on GFGE_SO dataset. Shows strong robustness and generalization across diverse scenes and resolutions.

Conclusion: SOMA effectively bridges the gap between traditional gradient-based approaches and deep learning for SAR-Optical registration, demonstrating that integrating structural gradient priors into deep features substantially improves performance.

Abstract: Achieving pixel-level registration between SAR and optical images remains a challenging task due to their fundamentally different imaging mechanisms and visual characteristics. Although deep learning has achieved great success in many cross-modal tasks, its performance on SAR-Optical registration tasks is still unsatisfactory. Gradient-based information has traditionally played a crucial role in handcrafted descriptors by highlighting structural differences. However, such gradient cues have not been effectively leveraged in deep learning frameworks for SAR-Optical image matching. To address this gap, we propose SOMA, a dense registration framework that integrates structural gradient priors into deep features and refines alignment through a hybrid matching strategy. Specifically, we introduce the Feature Gradient Enhancer (FGE), which embeds multi-scale, multi-directional gradient filters into the feature space using attention and reconstruction mechanisms to boost feature distinctiveness. Furthermore, we propose the Global-Local Affine-Flow Matcher (GLAM), which combines affine transformation and flow-based refinement within a coarse-to-fine architecture to ensure both structural consistency and local accuracy. Experimental results demonstrate that SOMA significantly improves registration precision, increasing the CMR@1px by 12.29% on the SEN1-2 dataset and 18.50% on the GFGE_SO dataset. In addition, SOMA exhibits strong robustness and generalizes well across diverse scenes and resolutions.

Method: Projects 3D lanes to front-view space as parametric curves, generates curve-point query embeddings for 3D predictions, models each lane segment as a 3D Gaussian parameterized by local structure and uncertainty, and uses a novel 3D Gaussian matching loss.

Result: Outperforms previous state-of-the-art methods across all benchmarks on ONCE-3DLanes and OpenLane datasets under stricter evaluation criteria, with proposed comprehensive metrics for global and local error quantification.

Conclusion: MonoUnc effectively addresses aleatoric uncertainty in monocular 3D lane detection through local structural modeling and uncertainty estimation, demonstrating superior performance and robustness in real-world scenarios.

Abstract: Monocular 3D lane detection is challenged by aleatoric uncertainty arising from inherent observation noise. Existing methods rely on simplified geometric assumptions, such as independent point predictions or global planar modeling, failing to capture structural variations and aleatoric uncertainty in real-world scenarios. In this paper, we propose MonoUnc, a bird’s-eye view (BEV)-free 3D lane detector that explicitly models aleatoric uncertainty informed by local lane structures. Specifically, 3D lanes are projected onto the front-view (FV) space and approximated by parametric curves. Guided by curve predictions, curve-point query embeddings are dynamically generated for lane point predictions in 3D space. Each segment formed by two adjacent points is modeled as a 3D Gaussian, parameterized by the local structure and uncertainty estimations. Accordingly, a novel 3D Gaussian matching loss is designed to constrain these parameters jointly. Experiments on the ONCE-3DLanes and OpenLane datasets demonstrate that MonoUnc outperforms previous state-of-the-art (SoTA) methods across all benchmarks under stricter evaluation criteria. Additionally, we propose two comprehensive evaluation metrics for ONCE-3DLanes, calculating the average and maximum bidirectional Chamfer distances to quantify global and local errors. Codes are released at https://github.com/lrx02/MonoUnc.

Method: Proposes a framework that extracts features from SAM2, uses Feature-Guided Attention module to guide a lightweight Fine-Grained Encoder with semantic cues, and employs a dual-affinity decoder generating coarse and refined affinity maps.

Result: Achieves performance comparable to SOTA with frozen SAM2 weights, and significantly outperforms existing SOTA methods after fine-tuning on EM data.

Conclusion: Transferring representations pre-trained on natural images combined with domain-adaptive guidance effectively addresses specific challenges in neuron segmentation.

Abstract: Accurate segmentation of neural structures in Electron Microscopy (EM) images is paramount for neuroscience. However, this task is challenged by intricate morphologies, low signal-to-noise ratios, and scarce annotations, limiting the accuracy and generalization of existing methods. To address these challenges, we seek to leverage the priors learned by visual foundation models on a vast amount of natural images to better tackle this task. Specifically, we propose a novel framework that can effectively transfer knowledge from Segment Anything 2 (SAM2), which is pre-trained on natural images, to the EM domain. We first use SAM2 to extract powerful, general-purpose features. To bridge the domain gap, we introduce a Feature-Guided Attention module that leverages semantic cues from SAM2 to guide a lightweight encoder, the Fine-Grained Encoder (FGE), in focusing on these challenging regions. Finally, a dual-affinity decoder generates both coarse and refined affinity maps. Experimental results demonstrate that our method achieves performance comparable to state-of-the-art (SOTA) approaches with the SAM2 weights frozen. Upon further fine-tuning on EM data, our method significantly outperforms existing SOTA methods. This study validates that transferring representations pre-trained on natural images, when combined with targeted domain-adaptive guidance, can effectively address the specific challenges in neuron segmentation.

Method: Developed RobustGait framework with four-dimensional evaluation: perturbation types (digital, environmental, temporal, occlusion), silhouette extraction methods (segmentation/parsing networks), model architectural capacities, and deployment scenarios. Evaluated 15 corruption types at 5 severity levels across multiple datasets including CASIA-B, CCPG, SUSTech1K, and MEVID.

Result: Key findings: 1) RGB-level noise better reflects real-world degradation and shows distortion propagation through silhouette extraction; 2) Gait accuracy is highly sensitive to silhouette extractor biases; 3) Robustness depends on both perturbation type and architectural design; 4) Noise-aware training and knowledge distillation improve performance.

Conclusion: RobustGait provides comprehensive robustness evaluation revealing critical vulnerabilities in gait recognition systems and demonstrates that robustness-enhancing strategies like noise-aware training and knowledge distillation can improve performance toward deployment-ready systems.

Abstract: Appearance-based gait recognition have achieved strong performance on controlled datasets, yet systematic evaluation of its robustness to real-world corruptions and silhouette variability remains lacking. We present RobustGait, a framework for fine-grained robustness evaluation of appearance-based gait recognition systems. RobustGait evaluation spans four dimensions: the type of perturbation (digital, environmental, temporal, occlusion), the silhouette extraction method (segmentation and parsing networks), the architectural capacities of gait recognition models, and various deployment scenarios. The benchmark introduces 15 corruption types at 5 severity levels across CASIA-B, CCPG, and SUSTech1K, with in-the-wild validation on MEVID, and evaluates six state-of-the-art gait systems. We came across several exciting insights. First, applying noise at the RGB level better reflects real-world degradation, and reveal how distortions propagate through silhouette extraction to the downstream gait recognition systems. Second, gait accuracy is highly sensitive to silhouette extractor biases, revealing an overlooked source of benchmark bias. Third, robustness is dependent on both the type of perturbation and the architectural design. Finally, we explore robustness-enhancing strategies, showing that noise-aware training and knowledge distillation improve performance and move toward deployment-ready systems.

Method: Proposes a dual-branch architecture: one branch performs scene-driven reasoning without ego status in BEV encoder, another conducts ego-driven planning. Uses scene-aware fusion module to integrate decisions, with path attention mechanism and auxiliary tasks (BEV unidirectional distillation, autoregressive online mapping).

Result: Achieves state-of-the-art open-loop planning performance on nuScenes dataset, significantly mitigates over-reliance on ego status, and exhibits impressive generalization across diverse scenarios.

Conclusion: AdaptiveAD effectively addresses the ego status over-reliance problem through architectural decoupling and adaptive fusion, demonstrating superior generalization and robust planning capabilities.

Abstract: Modular design of planning-oriented autonomous driving has markedly advanced end-to-end systems. However, existing architectures remain constrained by an over-reliance on ego status, hindering generalization and robust scene understanding. We identify the root cause as an inherent design within these architectures that allows ego status to be easily leveraged as a shortcut. Specifically, the premature fusion of ego status in the upstream BEV encoder allows an information flow from this strong prior to dominate the downstream planning module. To address this challenge, we propose AdaptiveAD, an architectural-level solution based on a multi-context fusion strategy. Its core is a dual-branch structure that explicitly decouples scene perception and ego status. One branch performs scene-driven reasoning based on multi-task learning, but with ego status deliberately omitted from the BEV encoder, while the other conducts ego-driven reasoning based solely on the planning task. A scene-aware fusion module then adaptively integrates the complementary decisions from the two branches to form the final planning trajectory. To ensure this decoupling does not compromise multi-task learning, we introduce a path attention mechanism for ego-BEV interaction and add two targeted auxiliary tasks: BEV unidirectional distillation and autoregressive online mapping. Extensive evaluations on the nuScenes dataset demonstrate that AdaptiveAD achieves state-of-the-art open-loop planning performance. Crucially, it significantly mitigates the over-reliance on ego status and exhibits impressive generalization capabilities across diverse scenarios.

Method: Uses class-aware prompts to define new tasks, fine-tunes with MLP-free backbone for few epochs, applies orthogonal continual merging strategy, and introduces Task-to-Class Prompt-aligned (TCP) inference for class-incremental learning where task identity is unknown.

Result: Outperforms both rehearsal-based continual learning and vision-language zero-shot baselines on experiments with six TCGA datasets.

Conclusion: MergeSlide provides an effective framework for lifelong learning on WSIs by treating it as a model merging problem, demonstrating superior performance while mitigating catastrophic forgetting through orthogonal merging and prompt-aligned inference.

Abstract: Lifelong learning on Whole Slide Images (WSIs) aims to train or fine-tune a unified model sequentially on cancer-related tasks, reducing the resources and effort required for data transfer and processing, especially given the gigabyte-scale size of WSIs. In this paper, we introduce MergeSlide, a simple yet effective framework that treats lifelong learning as a model merging problem by leveraging a vision-language pathology foundation model. When a new task arrives, it is: 1) defined with class-aware prompts, 2) fine-tuned for a few epochs using an MLP-free backbone, and 3) merged into a unified model using an orthogonal continual merging strategy that preserves performance and mitigates catastrophic forgetting. For inference under the class-incremental learning (CLASS-IL) setting, where task identity is unknown, we introduce Task-to-Class Prompt-aligned (TCP) inference. Specifically, TCP first identifies the most relevant task using task-level prompts and then applies the corresponding class-aware prompts to generate predictions. To evaluate MergeSlide, we conduct experiments on a stream of six TCGA datasets. The results show that MergeSlide outperforms both rehearsal-based continual learning and vision-language zero-shot baselines. Code and data are available at https://github.com/caodoanh2001/MergeSlide.

Method: Proposes a framework with hierarchical cross-modal interaction and dual-stream feature refinement to enhance joint embeddings with class-level and instance-specific cues from both text and images.

Result: CapeNext consistently outperforms state-of-the-art CAPE methods by a large margin on MP-100 dataset, regardless of network backbone.

Conclusion: The proposed hierarchical cross-modal interaction and dual-stream refinement effectively address the limitations of static joint embeddings in CAPE, leading to significant performance improvements.

Abstract: Recent research in Category-Agnostic Pose Estimation (CAPE) has adopted fixed textual keypoint description as semantic prior for two-stage pose matching frameworks. While this paradigm enhances robustness and flexibility by disentangling the dependency of support images, our critical analysis reveals two inherent limitations of static joint embedding: (1) polysemy-induced cross-category ambiguity during the matching process(e.g., the concept “leg” exhibiting divergent visual manifestations across humans and furniture), and (2) insufficient discriminability for fine-grained intra-category variations (e.g., posture and fur discrepancies between a sleeping white cat and a standing black cat). To overcome these challenges, we propose a new framework that innovatively integrates hierarchical cross-modal interaction with dual-stream feature refinement, enhancing the joint embedding with both class-level and instance-specific cues from textual description and specific images. Experiments on the MP-100 dataset demonstrate that, regardless of the network backbone, CapeNext consistently outperforms state-of-the-art CAPE methods by a large margin.

Method: Created GeoX-Bench with 10,859 panoramic-satellite image pairs from 128 cities in 49 countries and 755,976 QA pairs, then evaluated 25 state-of-the-art LMMs and explored instruction-tuning capabilities.

Result: Current LMMs achieve impressive performance in geo-localization tasks but decline significantly on more complex pose estimation tasks. Instruction-tuning on GeoX-Bench data significantly improves cross-view geo-sense abilities.

Conclusion: GeoX-Bench reveals critical gaps in LMMs’ pose estimation capabilities and demonstrates that instruction-tuning can substantially enhance cross-view geo-localization performance, highlighting areas for future improvement.

Abstract: Large multimodal models (LMMs) have demonstrated remarkable capabilities across a wide range of tasks, however their knowledge and abilities in the cross-view geo-localization and pose estimation domains remain unexplored, despite potential benefits for navigation, autonomous driving, outdoor robotics, \textit{etc}. To bridge this gap, we introduce \textbf{GeoX-Bench}, a comprehensive \underline{Bench}mark designed to explore and evaluate the capabilities of LMMs in \underline{cross}-view \underline{Geo}-localization and pose estimation. Specifically, GeoX-Bench contains 10,859 panoramic-satellite image pairs spanning 128 cities in 49 countries, along with corresponding 755,976 question-answering (QA) pairs. Among these, 42,900 QA pairs are designated for benchmarking, while the remaining are intended to enhance the capabilities of LMMs. Based on GeoX-Bench, we evaluate the capabilities of 25 state-of-the-art LMMs on cross-view geo-localization and pose estimation tasks, and further explore the empowered capabilities of instruction-tuning. Our benchmark demonstrate that while current LMMs achieve impressive performance in geo-localization tasks, their effectiveness declines significantly on the more complex pose estimation tasks, highlighting a critical area for future improvement, and instruction-tuning LMMs on the training data of GeoX-Bench can significantly improve the cross-view geo-sense abilities. The GeoX-Bench is available at \textcolor{magenta}{https://github.com/IntMeGroup/GeoX-Bench}.

Method: Proposes PlugTrack framework that uses multi-perceptive motion analysis to generate adaptive blending factors for fusing Kalman filter and data-driven motion predictors, leveraging their complementary strengths.

Result: Achieves significant performance gains on MOT17/MOT20 datasets and state-of-the-art performance on DanceTrack dataset, with Kalman filter outperforming data-driven predictors in up to 34% of cases even in non-linear motion scenarios.

Conclusion: PlugTrack successfully bridges classical and modern motion prediction paradigms through adaptive fusion, demonstrating that combining both approaches is more effective than using either alone in multi-object tracking.

Abstract: Multi-object tracking (MOT) predominantly follows the tracking-by-detection paradigm, where Kalman filters serve as the standard motion predictor due to computational efficiency but inherently fail on non-linear motion patterns. Conversely, recent data-driven motion predictors capture complex non-linear dynamics but suffer from limited domain generalization and computational overhead. Through extensive analysis, we reveal that even in datasets dominated by non-linear motion, Kalman filter outperforms data-driven predictors in up to 34% of cases, demonstrating that real-world tracking scenarios inherently involve both linear and non-linear patterns. To leverage this complementarity, we propose PlugTrack, a novel framework that adaptively fuses Kalman filter and data-driven motion predictors through multi-perceptive motion understanding. Our approach employs multi-perceptive motion analysis to generate adaptive blending factors. PlugTrack achieves significant performance gains on MOT17/MOT20 and state-of-the-art on DanceTrack without modifying existing motion predictors. To the best of our knowledge, PlugTrack is the first framework to bridge classical and modern motion prediction paradigms through adaptive fusion in MOT.

Method: Uses shared anatomical prior from representative patients as initialization, then personalizes with Structure-Preserving Personalized Generation (SPG) module. Constructs task-specific training pairs and aligns gradients between distilled data and raw patient data while preserving privacy.

Result: Develops a method that enables effective low-level dataset distillation for medical image enhancement, maintaining pixel-level fidelity while compressing dataset size and protecting patient privacy.

Conclusion: The proposed approach successfully addresses the challenges of low-level dataset distillation in medical imaging by leveraging anatomical priors and gradient alignment, enabling practical deployment with reduced costs and preserved privacy.

Abstract: Medical image enhancement is clinically valuable, but existing methods require large-scale datasets to learn complex pixel-level mappings. However, the substantial training and storage costs associated with these datasets hinder their practical deployment. While dataset distillation (DD) can alleviate these burdens, existing methods mainly target high-level tasks, where multiple samples share the same label. This many-to-one mapping allows distilled data to capture shared semantics and achieve information compression. In contrast, low-level tasks involve a many-to-many mapping that requires pixel-level fidelity, making low-level DD an underdetermined problem, as a small distilled dataset cannot fully constrain the dense pixel-level mappings. To address this, we propose the first low-level DD method for medical image enhancement. We first leverage anatomical similarities across patients to construct the shared anatomical prior based on a representative patient, which serves as the initialization for the distilled data of different patients. This prior is then personalized for each patient using a Structure-Preserving Personalized Generation (SPG) module, which integrates patient-specific anatomical information into the distilled dataset while preserving pixel-level fidelity. For different low-level tasks, the distilled data is used to construct task-specific high- and low-quality training pairs. Patient-specific knowledge is injected into the distilled data by aligning the gradients computed from networks trained on the distilled pairs with those from the corresponding patient’s raw data. Notably, downstream users cannot access raw patient data. Instead, only a distilled dataset containing abstract training information is shared, which excludes patient-specific details and thus preserves privacy.

Method: Propose Distillation-guided Gradient Surgery Network (DGS-Net) with gradient-space decomposition that separates harmful and beneficial descent directions, projecting task gradients onto orthogonal complement of harmful directions and aligning with beneficial ones distilled from frozen CLIP encoder.

Result: Outperforms state-of-the-art approaches by average margin of 6.6 across 50 generative models, achieving superior detection performance and generalization across diverse generation techniques.

Conclusion: DGS-Net effectively addresses catastrophic forgetting in fine-tuning pre-trained models for AI-generated image detection, enabling unified optimization of prior preservation and irrelevant suppression for improved cross-domain generalization.

Abstract: The rapid progress of generative models such as GANs and diffusion models has led to the widespread proliferation of AI-generated images, raising concerns about misinformation, privacy violations, and trust erosion in digital media. Although large-scale multimodal models like CLIP offer strong transferable representations for detecting synthetic content, fine-tuning them often induces catastrophic forgetting, which degrades pre-trained priors and limits cross-domain generalization. To address this issue, we propose the Distillation-guided Gradient Surgery Network (DGS-Net), a novel framework that preserves transferable pre-trained priors while suppressing task-irrelevant components. Specifically, we introduce a gradient-space decomposition that separates harmful and beneficial descent directions during optimization. By projecting task gradients onto the orthogonal complement of harmful directions and aligning with beneficial ones distilled from a frozen CLIP encoder, DGS-Net achieves unified optimization of prior preservation and irrelevant suppression. Extensive experiments on 50 generative models demonstrate that our method outperforms state-of-the-art approaches by an average margin of 6.6, achieving superior detection performance and generalization across diverse generation techniques.

Method: Uses Mask R-CNN for actor localization with bounding boxes and masks, multiple backbones (Inception V3, MobileNet, VGG16) for feature extraction, RoIAlign for spatial alignment, mask fusion for refined features, Actor Relation Graphs with similarity measures (NCC, SAD, dot product), and Graph Convolutional Networks for relationship reasoning.

Result: Experiments on Collective Activity dataset show improved recognition performance in both crowded and non-crowded scenarios through mask-based feature refinement, robust similarity search, and graph neural network reasoning.

Conclusion: Integration of segmentation, feature extraction, and relational graph reasoning shows strong potential for complex video understanding tasks.

Abstract: Group activity detection in multi-person scenes is challenging due to complex human interactions, occlusions, and variations in appearance over time. This work presents a computer vision based framework for group activity recognition and action spotting using a combination of deep learning models and graph based relational reasoning. The system first applies Mask R-CNN to obtain accurate actor localization through bounding boxes and instance masks. Multiple backbone networks, including Inception V3, MobileNet, and VGG16, are used to extract feature maps, and RoIAlign is applied to preserve spatial alignment when generating actor specific features. The mask information is then fused with the feature maps to obtain refined masked feature representations for each actor. To model interactions between individuals, we construct Actor Relation Graphs that encode appearance similarity and positional relations using methods such as normalized cross correlation, sum of absolute differences, and dot product. Graph Convolutional Networks operate on these graphs to reason about relationships and predict both individual actions and group level activities. Experiments on the Collective Activity dataset demonstrate that the combination of mask based feature refinement, robust similarity search, and graph neural network reasoning leads to improved recognition performance across both crowded and non crowded scenarios. This approach highlights the potential of integrating segmentation, feature extraction, and relational graph reasoning for complex video understanding tasks.

Method: Uses implicit neural representation to model haze degradation as a continuous function, combining channel-independent and channel-dependent mechanisms inspired by Kolmogorov-Arnold theorem, with dense residual enhancement module.

Result: Achieves competitive dehazing performance on various public and real-world datasets with good visual perception in complex scenes.

Conclusion: The proposed unsupervised method effectively models haze degradation through implicit neural representation, eliminating redundant information and dependence on explicit feature extraction while maintaining high-quality image restoration.

Abstract: Image dehazing is an important task in the field of computer vision, aiming at restoring clear and detail-rich visual content from haze-affected images. However, when dealing with complex scenes, existing methods often struggle to strike a balance between fine-grained feature representation of inhomogeneous haze distribution and global consistency modeling. Furthermore, to better learn the common degenerate representation of haze in spatial variations, we propose an unsupervised dehaze method for implicit neural degradation representation. Firstly, inspired by the Kolmogorov-Arnold representation theorem, we propose a mechanism combining the channel-independent and channel-dependent mechanisms, which efficiently enhances the ability to learn from nonlinear dependencies. which in turn achieves good visual perception in complex scenes. Moreover, we design an implicit neural representation to model haze degradation as a continuous function to eliminate redundant information and the dependence on explicit feature extraction and physical models. To further learn the implicit representation of the haze features, we also designed a dense residual enhancement module from it to eliminate redundant information. This achieves high-quality image restoration. Experimental results show that our method achieves competitive dehaze performance on various public and real-world datasets. This project code will be available at https://github.com/Fan-pixel/NeDR-Dehaze.

Method: Multi-Prior Hierarchical Mamba network that combines macro-semantic textual priors (CLIP) and micro-structural visual priors (DINOv2) with progressive Priors Fusion Injection and Fourier-enhanced dual-path hierarchical Mamba modules.

Result: Achieves 0.57 dB PSNR gain on Rain200H dataset and demonstrates superior generalization on real-world rainy scenarios.

Conclusion: MPHM network effectively addresses semantic and structural detail preservation in image deraining through synergistic integration of heterogeneous priors and hierarchical feature modeling.

Abstract: Rain significantly degrades the performance of computer vision systems, particularly in applications like autonomous driving and video surveillance. While existing deraining methods have made considerable progress, they often struggle with fidelity of semantic and spatial details. To address these limitations, we propose the Multi-Prior Hierarchical Mamba (MPHM) network for image deraining. This novel architecture synergistically integrates macro-semantic textual priors (CLIP) for task-level semantic guidance and micro-structural visual priors (DINOv2) for scene-aware structural information. To alleviate potential conflicts between heterogeneous priors, we devise a progressive Priors Fusion Injection (PFI) that strategically injects complementary cues at different decoder levels. Meanwhile, we equip the backbone network with an elaborate Hierarchical Mamba Module (HMM) to facilitate robust feature representation, featuring a Fourier-enhanced dual-path design that concurrently addresses global context modeling and local detail recovery. Comprehensive experiments demonstrate MPHM’s state-of-the-art performance, achieving a 0.57 dB PSNR gain on the Rain200H dataset while delivering superior generalization on real-world rainy scenarios.

Method: Uses Point Coordinate Mapping (PCM) to map points into rotationally invariant space, then employs lightweight Convolutional Transform Feature Network (CTF-Net) with transfer learning pre-training using 3D data augmentation.

Result: Achieves 17.7% average P-AUROC improvement on Anomaly-ShapeNet and 1.6% improvement on Real3D-AD dataset, with strong generalization when combined with traditional methods.

Conclusion: RIF framework demonstrates robust performance and strong generalization ability for 3D anomaly detection, showing great potential for industrial applications.

Abstract: 3D anomaly detection (AD) is a crucial task in computer vision, aiming to identify anomalous points or regions from point cloud data. However, existing methods may encounter challenges when handling point clouds with changes in orientation and position because the resulting features may vary significantly. To address this problem, we propose a novel Rotationally Invariant Features (RIF) framework for 3D AD. Firstly, to remove the adverse effect of variations on point cloud data, we develop a Point Coordinate Mapping (PCM) technique, which maps each point into a rotationally invariant space to maintain consistency of representation. Then, to learn robust and discriminative features, we design a lightweight Convolutional Transform Feature Network (CTF-Net) to extract rotationally invariant features for the memory bank. To improve the ability of the feature extractor, we introduce the idea of transfer learning to pre-train the feature extractor with 3D data augmentation. Experimental results show that the proposed method achieves the advanced performance on the Anomaly-ShapeNet dataset, with an average P-AUROC improvement of 17.7%, and also gains the best performance on the Real3D-AD dataset, with an average P-AUROC improvement of 1.6%. The strong generalization ability of RIF has been verified by combining it with traditional feature extraction methods on anomaly detection tasks, demonstrating great potential for industrial applications.

Method: Multi-task framework using ResNet-18 backbone that combines manual labels for overall quality with pseudo-labels generated by a Teacher model trained on small dataset, then fine-tunes pre-trained model.

Result: Outperforms single-task baselines (F1: 0.875 vs. 0.863 on EyeQ, 0.778 vs. 0.763 on DeepDRiD), matches/surpasses existing methods, achieves performance comparable to Teacher model for detail prediction, and performs similarly to experts on newly annotated subset.

Conclusion: Semi-supervised approach improves overall quality assessment and provides interpretable feedback on capture conditions (illumination, clarity, contrast) at no extra labeling cost, offering clinically actionable outputs to guide image recapture.

Abstract: Retinal image quality assessment (RIQA) supports computer-aided diagnosis of eye diseases. However, most tools classify only overall image quality, without indicating acquisition defects to guide recapture. This gap is mainly due to the high cost of detailed annotations. In this paper, we aim to mitigate this limitation by introducing a hybrid semi-supervised learning approach that combines manual labels for overall quality with pseudo-labels of quality details within a multi-task framework. Our objective is to obtain more interpretable RIQA models without requiring extensive manual labeling. Pseudo-labels are generated by a Teacher model trained on a small dataset and then used to fine-tune a pre-trained model in a multi-task setting. Using a ResNet-18 backbone, we show that these weak annotations improve quality assessment over single-task baselines (F1: 0.875 vs. 0.863 on EyeQ, and 0.778 vs. 0.763 on DeepDRiD), matching or surpassing existing methods. The multi-task model achieved performance statistically comparable to the Teacher for most detail prediction tasks (p > 0.05). In a newly annotated EyeQ subset released with this paper, our model performed similarly to experts, suggesting that pseudo-label noise aligns with expert variability. Our main finding is that the proposed semi-supervised approach not only improves overall quality assessment but also provides interpretable feedback on capture conditions (illumination, clarity, contrast). This enhances interpretability at no extra manual labeling cost and offers clinically actionable outputs to guide image recapture.

Method: Proposes hierarchical warping and occlusion-aware noise suppression to enhance conditioning image quality, and introduces global structure guidance using dense fused point clouds to provide consistent geometric context to the diffusion process.

Result: Extensive experiments on multiple datasets show that CloseUpShot outperforms existing approaches, particularly in close-up novel view synthesis, validating the effectiveness of the proposed design.

Conclusion: The method successfully addresses the limitations of sparse input views in close-up scenarios through improved conditioning techniques and global geometric constraints, demonstrating superior performance in novel view synthesis.

Abstract: Reconstructing 3D scenes and synthesizing novel views from sparse input views is a highly challenging task. Recent advances in video diffusion models have demonstrated strong temporal reasoning capabilities, making them a promising tool for enhancing reconstruction quality under sparse-view settings. However, existing approaches are primarily designed for modest viewpoint variations, which struggle in capturing fine-grained details in close-up scenarios since input information is severely limited. In this paper, we present a diffusion-based framework, called CloseUpShot, for close-up novel view synthesis from sparse inputs via point-conditioned video diffusion. Specifically, we observe that pixel-warping conditioning suffers from severe sparsity and background leakage in close-up settings. To address this, we propose hierarchical warping and occlusion-aware noise suppression, enhancing the quality and completeness of the conditioning images for the video diffusion model. Furthermore, we introduce global structure guidance, which leverages a dense fused point cloud to provide consistent geometric context to the diffusion process, to compensate for the lack of globally consistent 3D constraints in sparse conditioning inputs. Extensive experiments on multiple datasets demonstrate that our method outperforms existing approaches, especially in close-up novel view synthesis, clearly validating the effectiveness of our design.

Method: Created VIR-Bench with 200 travel videos and framed itinerary reconstruction as a challenging task to evaluate MLLMs’ geospatial-temporal intelligence.

Result: State-of-the-art MLLMs, including proprietary ones, struggle to achieve high scores, showing difficulty in handling videos spanning extended spatial and temporal scales. A prototype travel-planning agent developed using insights from VIR-Bench showed markedly improved itinerary recommendations.

Conclusion: VIR-Bench effectively benchmarks MLLMs’ geospatial-temporal capabilities and translates into concrete performance gains in practical applications like travel planning.

Abstract: Recent advances in multimodal large language models (MLLMs) have significantly enhanced video understanding capabilities, opening new possibilities for practical applications. Yet current video benchmarks focus largely on indoor scenes or short-range outdoor activities, leaving the challenges associated with long-distance travel largely unexplored. Mastering extended geospatial-temporal trajectories is critical for next-generation MLLMs, underpinning real-world tasks such as embodied-AI planning and navigation. To bridge this gap, we present VIR-Bench, a novel benchmark consisting of 200 travel videos that frames itinerary reconstruction as a challenging task designed to evaluate and push forward MLLMs’ geospatial-temporal intelligence. Experimental results reveal that state-of-the-art MLLMs, including proprietary ones, struggle to achieve high scores, underscoring the difficulty of handling videos that span extended spatial and temporal scales. Moreover, we conduct an in-depth case study in which we develop a prototype travel-planning agent that leverages the insights gained from VIR-Bench. The agent’s markedly improved itinerary recommendations verify that our evaluation protocol not only benchmarks models effectively but also translates into concrete performance gains in user-facing applications.

Method: RePo maps GPS trajectories to grid sequences for region-wise representation (structural and semantic features) and uses three expert networks for point-wise representation (local, correlation, continuous patterns). A router network fuses point-wise features, which are combined with region-wise features via cross-attention to produce final embeddings. Training uses contrastive loss with hard negative samples.

Result: RePo achieves an average accuracy improvement of 22.2% over state-of-the-art baselines across all evaluation metrics.

Conclusion: The joint encoding of region-wise and point-wise features effectively captures comprehensive trajectory information, significantly improving trajectory similarity computation performance.

Abstract: Recent learning-based methods have reduced the computational complexity of traditional trajectory similarity computation, but state-of-the-art (SOTA) methods still fail to leverage the comprehensive spectrum of trajectory information for similarity modeling. To tackle this problem, we propose \textbf{RePo}, a novel method that jointly encodes \textbf{Re}gion-wise and \textbf{Po}int-wise features to capture both spatial context and fine-grained moving patterns. For region-wise representation, the GPS trajectories are first mapped to grid sequences, and spatial context are captured by structural features and semantic context enriched by visual features. For point-wise representation, three lightweight expert networks extract local, correlation, and continuous movement patterns from dense GPS sequences. Then, a router network adaptively fuses the learned point-wise features, which are subsequently combined with region-wise features using cross-attention to produce the final trajectory embedding. To train RePo, we adopt a contrastive loss with hard negative samples to provide similarity ranking supervision. Experiment results show that RePo achieves an average accuracy improvement of 22.2% over SOTA baselines across all evaluation metrics.

Method: VEIL uses modular prompts with three components: neutral scene anchors, latent auditory triggers (innocuous audio descriptions), and stylistic modulators (cinematic directives). Attack generation is formulated as constrained optimization solved via guided search.

Result: Extensive experiments on 7 T2V models show VEIL achieves 23% improvement in average attack success rate for commercial models compared to prior methods.

Conclusion: Benign-looking prompts with rich implicit cues can effectively jailbreak T2V models by exploiting their cross-modal associative patterns, revealing critical safety vulnerabilities.

Abstract: Jailbreak attacks can circumvent model safety guardrails and reveal critical blind spots. Prior attacks on text-to-video (T2V) models typically add adversarial perturbations to obviously unsafe prompts, which are often easy to detect and defend. In contrast, we show that benign-looking prompts containing rich, implicit cues can induce T2V models to generate semantically unsafe videos that both violate policy and preserve the original (blocked) intent. To realize this, we propose VEIL, a jailbreak framework that leverages T2V models’ cross-modal associative patterns via a modular prompt design. Specifically, our prompts combine three components: neutral scene anchors, which provide the surface-level scene description extracted from the blocked intent to maintain plausibility; latent auditory triggers, textual descriptions of innocuous-sounding audio events (e.g., creaking, muffled noises) that exploit learned audio-visual co-occurrence priors to bias the model toward particular unsafe visual concepts; and stylistic modulators, cinematic directives (e.g., camera framing, atmosphere) that amplify and stabilize the latent trigger’s effect. We formalize attack generation as a constrained optimization over the above modular prompt space and solve it with a guided search procedure that balances stealth and effectiveness. Extensive experiments over 7 T2V models demonstrate the efficacy of our attack, achieving a 23 percent improvement in average attack success rate in commercial models.

Method: Extends DDCM by replacing random noise in backward process with noise from specific sets according to predefined rules, making it compatible with multiple diffusion model types.

Result: Successfully generalized DDCM to various diffusion models and achieved improved performance on CIFAR-10 and LSUN Bedroom datasets.

Conclusion: gDDCM provides a generalized framework that enables image compression across multiple diffusion model variants with enhanced performance.

Abstract: Recently, the Denoising Diffusion Codebook Models (DDCM) was proposed. DDCM leverages the Denoising Diffusion Probabilistic Model (DDPM) and replaces the random noise in the backward process with noise sampled from specific sets according to a predefined rule, thereby enabling image compression. However, DDCM cannot be applied to methods other than DDPM. In this paper, we propose the generalized Denoising Diffusion Compression Model (gDDCM), which extends DDCM to mainstream diffusion models and their variants, including DDPM, Score-Based Models, Consistency Models, and Rectified Flow. We evaluate our method on CIFAR-10 and LSUN Bedroom datasets. Experimental results demonstrate that our approach successfully generalizes DDCM to the aforementioned models and achieves improved performance.

Method: Developed MedGEN-Bench with 6,422 expert-validated image-text pairs across 6 imaging modalities, 16 clinical tasks, and 28 subtasks in three formats: Visual Question Answering, Image Editing, and Contextual Multimodal Generation.

Result: Systematically evaluated 10 compositional frameworks, 3 unified models, and 5 VLMs using a novel three-tier assessment framework integrating pixel-level metrics, semantic text analysis, and expert-guided clinical relevance scoring.

Conclusion: MedGEN-Bench advances medical AI research by enabling sophisticated cross-modal reasoning and open-ended generative outputs, moving beyond multiple-choice formats to better align with clinical workflow needs.

Abstract: As Vision-Language Models (VLMs) increasingly gain traction in medical applications, clinicians are progressively expecting AI systems not only to generate textual diagnoses but also to produce corresponding medical images that integrate seamlessly into authentic clinical workflows. Despite the growing interest, existing medical visual benchmarks present notable limitations. They often rely on ambiguous queries that lack sufficient relevance to image content, oversimplify complex diagnostic reasoning into closed-ended shortcuts, and adopt a text-centric evaluation paradigm that overlooks the importance of image generation capabilities. To address these challenges, we introduce \textsc{MedGEN-Bench}, a comprehensive multimodal benchmark designed to advance medical AI research. MedGEN-Bench comprises 6,422 expert-validated image-text pairs spanning six imaging modalities, 16 clinical tasks, and 28 subtasks. It is structured into three distinct formats: Visual Question Answering, Image Editing, and Contextual Multimodal Generation. What sets MedGEN-Bench apart is its focus on contextually intertwined instructions that necessitate sophisticated cross-modal reasoning and open-ended generative outputs, moving beyond the constraints of multiple-choice formats. To evaluate the performance of existing systems, we employ a novel three-tier assessment framework that integrates pixel-level metrics, semantic text analysis, and expert-guided clinical relevance scoring. Using this framework, we systematically assess 10 compositional frameworks, 3 unified models, and 5 VLMs.

Method: Created DTPQA benchmark with synthetic (simulator-based) and real-world components, each sample containing image, question, ground truth answer, and object distance annotation.

Result: Provides a specialized dataset and tools to evaluate VLM perception performance degradation as object distance increases in traffic scenarios.

Conclusion: DTPQA enables isolated evaluation of VLM perception capabilities in safety-critical driving contexts, addressing the need for robust long-range scene understanding.

Abstract: The remarkable progress of Vision-Language Models (VLMs) on a variety of tasks has raised interest in their application to automated driving. However, for these models to be trusted in such a safety-critical domain, they must first possess robust perception capabilities, i.e., they must be capable of understanding a traffic scene, which can often be highly complex, with many things happening simultaneously. Moreover, since critical objects and agents in traffic scenes are often at long distances, we require systems with not only strong perception capabilities at close distances (up to 20 meters), but also at long (30+ meters) range. Therefore, it is important to evaluate the perception capabilities of these models in isolation from other skills like reasoning or advanced world knowledge. Distance-Annotated Traffic Perception Question Answering (DTPQA) is a Visual Question Answering (VQA) benchmark designed specifically for this purpose: it can be used to evaluate the perception systems of VLMs in traffic scenarios using trivial yet crucial questions relevant to driving decisions. It consists of two parts: a synthetic benchmark (DTP-Synthetic) created using a simulator, and a real-world benchmark (DTP-Real) built on top of existing images of real traffic scenes. Additionally, DTPQA includes distance annotations, i.e., how far the object in question is from the camera. More specifically, each DTPQA sample consists of (at least): (a) an image, (b) a question, (c) the ground truth answer, and (d) the distance of the object in question, enabling analysis of how VLM performance degrades with increasing object distance. In this article, we provide the dataset itself along with the Python scripts used to create it, which can be used to generate additional data of the same kind.

Method: Proposed WinMamba backbone with stacked WinMamba blocks featuring: 1) window-scale-adaptive module for multi-scale voxel feature compensation, 2) learnable positional encoding, and 3) window-shift strategy for rich contextual cues in linear state space.

Result: Extensive experiments on KITTI and Waymo datasets show WinMamba significantly outperforms baseline methods. Ablation studies confirm the effectiveness of the window-scale-adaptive and window-shift modules.

Conclusion: WinMamba successfully addresses the efficiency-accuracy trade-off in 3D object detection by preserving spatial information while maintaining computational efficiency through Mamba-based architecture with adaptive window strategies.

Abstract: 3D object detection is critical for autonomous driving, yet it remains fundamentally challenging to simultaneously maximize computational efficiency and capture long-range spatial dependencies. We observed that Mamba-based models, with their linear state-space design, capture long-range dependencies at lower cost, offering a promising balance between efficiency and accuracy. However, existing methods rely on axis-aligned scanning within a fixed window, inevitably discarding spatial information. To address this problem, we propose WinMamba, a novel Mamba-based 3D feature-encoding backbone composed of stacked WinMamba blocks. To enhance the backbone with robust multi-scale representation, the WinMamba block incorporates a window-scale-adaptive module that compensates voxel features across varying resolutions during sampling. Meanwhile, to obtain rich contextual cues within the linear state space, we equip the WinMamba layer with a learnable positional encoding and a window-shift strategy. Extensive experiments on the KITTI and Waymo datasets demonstrate that WinMamba significantly outperforms the baseline. Ablation studies further validate the individual contributions of the WSF and AWF modules in improving detection accuracy. The code will be made publicly available.

Method: Uses SCB Synthesis dataset with SCB Groups for triple feature disentanglement, employs inter-group contrastive regularization and intra-sample multi-feature orthogonality, and performs feature remapping to prevent shortcuts and feature leakage.

Result: Achieves state-of-the-art image fidelity (SSIM 44.54) and text accuracy (93.58%) on mainstream STE benchmarks, trained on 125,000 SCB Groups.

Conclusion: TripleFDS enables more flexible editing including style replacement and background transfer while maintaining superior performance compared to previous methods.

Abstract: Scene Text Editing (STE) aims to naturally modify text in images while preserving visual consistency, the decisive factors of which can be divided into three parts, i.e., text style, text content, and background. Previous methods have struggled with incomplete disentanglement of editable attributes, typically addressing only one aspect - such as editing text content - thus limiting controllability and visual consistency. To overcome these limitations, we propose TripleFDS, a novel framework for STE with disentangled modular attributes, and an accompanying dataset called SCB Synthesis. SCB Synthesis provides robust training data for triple feature disentanglement by utilizing the “SCB Group”, a novel construct that combines three attributes per image to generate diverse, disentangled training groups. Leveraging this construct as a basic training unit, TripleFDS first disentangles triple features, ensuring semantic accuracy through inter-group contrastive regularization and reducing redundancy through intra-sample multi-feature orthogonality. In the synthesis phase, TripleFDS performs feature remapping to prevent “shortcut” phenomena during reconstruction and mitigate potential feature leakage. Trained on 125,000 SCB Groups, TripleFDS achieves state-of-the-art image fidelity (SSIM of 44.54) and text accuracy (ACC of 93.58%) on the mainstream STE benchmarks. Besides superior performance, the more flexible editing of TripleFDS supports new operations such as style replacement and background transfer. Code: https://github.com/yusenbao01/TripleFDS

Method: Two-stage framework: (1) Contrastive learning to align skeleton and visual features at sequence level, (2) Dynamic Prototype Fusion Updater (PFU) to refine multimodal identity prototypes, and Skeleton Guided Temporal Modeling (SGTM) module to distill temporal cues from skeletons into visual features.

Result: Achieves new state-of-the-art results on standard video ReID benchmarks (MARS, LS-VID, iLIDS-VID) and exhibits strong generalization to skeleton-only ReID tasks (BIWI, IAS), significantly outperforming previous methods.

Conclusion: CSIP-ReID pioneers an annotation-free and motion-aware pretraining paradigm for ReID, opening a new frontier in multimodal representation learning by using skeletons as a spatiotemporally informative modality aligned with video frames.

Abstract: Multimodal pretraining has revolutionized visual understanding, but its impact on video-based person re-identification (ReID) remains underexplored. Existing approaches often rely on video-text pairs, yet suffer from two fundamental limitations: (1) lack of genuine multimodal pretraining, and (2) text poorly captures fine-grained temporal motion-an essential cue for distinguishing identities in video. In this work, we take a bold departure from text-based paradigms by introducing the first skeleton-driven pretraining framework for ReID. To achieve this, we propose Contrastive Skeleton-Image Pretraining for ReID (CSIP-ReID), a novel two-stage method that leverages skeleton sequences as a spatiotemporally informative modality aligned with video frames. In the first stage, we employ contrastive learning to align skeleton and visual features at sequence level. In the second stage, we introduce a dynamic Prototype Fusion Updater (PFU) to refine multimodal identity prototypes, fusing motion and appearance cues. Moreover, we propose a Skeleton Guided Temporal Modeling (SGTM) module that distills temporal cues from skeleton data and integrates them into visual features. Extensive experiments demonstrate that CSIP-ReID achieves new state-of-the-art results on standard video ReID benchmarks (MARS, LS-VID, iLIDS-VID). Moreover, it exhibits strong generalization to skeleton-only ReID tasks (BIWI, IAS), significantly outperforming previous methods. CSIP-ReID pioneers an annotation-free and motion-aware pretraining paradigm for ReID, opening a new frontier in multimodal representation learning.

Method: Proposes Foresee pipeline with type-prior-driven strategy and Flexible Feature Detector (FFD) module specifically designed for copy-move manipulations, enabling training-free inference while effectively utilizing vanilla MLLMs.

Result: Extensive experiments show Foresee achieves superior localization accuracy and richer textual explanations compared to existing methods, with strong generalization across copy-move, splicing, removal, local enhancement, deepfake, and AIGC-based editing.

Conclusion: Foresee successfully unleashes vanilla MLLMs’ potential for image forensics without additional training, offering an efficient and interpretable solution that outperforms existing IFDL methods across diverse tampering types.

Abstract: With the rapid advancement of artificial intelligence-generated content (AIGC) technologies, including multimodal large language models (MLLMs) and diffusion models, image generation and manipulation have become remarkably effortless. Existing image forgery detection and localization (IFDL) methods often struggle to generalize across diverse datasets and offer limited interpretability. Nowadays, MLLMs demonstrate strong generalization potential across diverse vision-language tasks, and some studies introduce this capability to IFDL via large-scale training. However, such approaches cost considerable computational resources, while failing to reveal the inherent generalization potential of vanilla MLLMs to address this problem. Inspired by this observation, we propose Foresee, a training-free MLLM-based pipeline tailored for image forgery analysis. It eliminates the need for additional training and enables a lightweight inference process, while surpassing existing MLLM-based methods in both tamper localization accuracy and the richness of textual explanations. Foresee employs a type-prior-driven strategy and utilizes a Flexible Feature Detector (FFD) module to specifically handle copy-move manipulations, thereby effectively unleashing the potential of vanilla MLLMs in the forensic domain. Extensive experiments demonstrate that our approach simultaneously achieves superior localization accuracy and provides more comprehensive textual explanations. Moreover, Foresee exhibits stronger generalization capability, outperforming existing IFDL methods across various tampering types, including copy-move, splicing, removal, local enhancement, deepfake, and AIGC-based editing. The code will be released in the final version.

Method: Extracts topological fingerprints directly from RGB histopathological images using cubical persistence, encoding loop evolution as compact feature vectors, then performs similarity retrieval by computing distances between these topological descriptors.

Result: Outperforms state-of-the-art supervised and unsupervised methods on BreaKHis dataset, processing entire dataset in under 20 minutes on standard CPU.

Conclusion: THIR provides a fast, scalable, training-free solution for clinical image retrieval that doesn’t require annotated datasets or GPU resources.

Abstract: According to the World Health Organization, breast cancer claimed the lives of approximately 685,000 women in 2020. Early diagnosis and accurate clinical decision making are critical in reducing this global burden. In this study, we propose THIR, a novel Content-Based Medical Image Retrieval (CBMIR) framework that leverages topological data analysis specifically, Betti numbers derived from persistent homology to characterize and retrieve histopathological images based on their intrinsic structural patterns. Unlike conventional deep learning approaches that rely on extensive training, annotated datasets, and powerful GPU resources, THIR operates entirely without supervision. It extracts topological fingerprints directly from RGB histopathological images using cubical persistence, encoding the evolution of loops as compact, interpretable feature vectors. The similarity retrieval is then performed by computing the distances between these topological descriptors, efficiently returning the top-K most relevant matches. Extensive experiments on the BreaKHis dataset demonstrate that THIR outperforms state of the art supervised and unsupervised methods. It processes the entire dataset in under 20 minutes on a standard CPU, offering a fast, scalable, and training free solution for clinical image retrieval.

Method: Proposes HDW-SR with wavelet decomposition, performing diffusion only on residual maps, using wavelet-based downsampling for multi-scale frequency decomposition, sparse cross-attention between frequency subbands, and a Dynamic Thresholding Block for high-frequency refinement.

Result: HDW-SR achieves competitive super-resolution performance on both synthetic and real-world datasets, particularly excelling in recovering fine-grained image details compared to existing methods.

Conclusion: The wavelet-based diffusion approach with explicit high-frequency guidance effectively addresses the blurring issue in diffusion-based super-resolution, enabling superior restoration of fine image details.

Abstract: Diffusion-based methods have shown great promise in single image super-resolution (SISR); however, existing approaches often produce blurred fine details due to insufficient guidance in the high-frequency domain. To address this issue, we propose a High-Frequency Guided Diffusion Network based on Wavelet Decomposition (HDW-SR), which replaces the conventional U-Net backbone in diffusion frameworks. Specifically, we perform diffusion only on the residual map, allowing the network to focus more effectively on high-frequency information restoration. We then introduce wavelet-based downsampling in place of standard CNN downsampling to achieve multi-scale frequency decomposition, enabling sparse cross-attention between the high-frequency subbands of the pre-super-resolved image and the low-frequency subbands of the diffused image for explicit high-frequency guidance. Moreover, a Dynamic Thresholding Block (DTB) is designed to refine high-frequency selection during the sparse attention process. During upsampling, the invertibility of the wavelet transform ensures low-loss feature reconstruction. Experiments on both synthetic and real-world datasets demonstrate that HDW-SR achieves competitive super-resolution performance, excelling particularly in recovering fine-grained image details. The code will be available after acceptance.

Method: Frames tractography as a generative task using both diffusion-based and flow matching paradigms to learn direct mapping from dMRI to complete streamlines.

Result: GenTract achieves precision 2.1x higher than the next-best method (TractOracle), with even greater advantages in challenging low-resolution and noisy settings where it outperforms competitors by an order of magnitude.

Conclusion: GenTract represents a promising solution for global tractography, producing high-precision tractograms on research-grade data while maintaining reliability on imperfect, lower-resolution data.

Abstract: Tractography is the process of inferring the trajectories of white-matter pathways in the brain from diffusion magnetic resonance imaging (dMRI). Local tractography methods, which construct streamlines by following local fiber orientation estimates stepwise through an image, are prone to error accumulation and high false positive rates, particularly on noisy or low-resolution data. In contrast, global methods, which attempt to optimize a collection of streamlines to maximize compatibility with underlying fiber orientation estimates, are computationally expensive. To address these challenges, we introduce GenTract, the first generative model for global tractography. We frame tractography as a generative task, learning a direct mapping from dMRI to complete, anatomically plausible streamlines. We compare both diffusion-based and flow matching paradigms and evaluate GenTract’s performance against state-of-the-art baselines. Notably, GenTract achieves precision 2.1x higher than the next-best method, TractOracle. This advantage becomes even more pronounced in challenging low-resolution and noisy settings, where it outperforms the closest competitor by an order of magnitude. By producing tractograms with high precision on research-grade data while also maintaining reliability on imperfect, lower-resolution data, GenTract represents a promising solution for global tractography.

Method: Uses Generative Latent Prediction (GLP) architecture with autoregressive latent dynamics based on LLM for text-based reasoning, combined with video diffusion decoder for visual reconstruction.

Result: Achieves strong performance in action-conditioned simulation, long-horizon forecasting, and simulative reasoning across diverse domains compared to other models.

Conclusion: PAN represents progress toward general world models that enable predictive simulation of future states for reasoning and acting.

Abstract: A world model enables an intelligent agent to imagine, predict, and reason about how the world evolves in response to its actions, and accordingly to plan and strategize. While recent video generation models produce realistic visual sequences, they typically operate in the prompt-to-full-video manner without causal control, interactivity, or long-horizon consistency required for purposeful reasoning. Existing world modeling efforts, on the other hand, often focus on restricted domains (e.g., physical, game, or 3D-scene dynamics) with limited depth and controllability, and struggle to generalize across diverse environments and interaction formats. In this work, we introduce PAN, a general, interactable, and long-horizon world model that predicts future world states through high-quality video simulation conditioned on history and natural language actions. PAN employs the Generative Latent Prediction (GLP) architecture that combines an autoregressive latent dynamics backbone based on a large language model (LLM), which grounds simulation in extensive text-based knowledge and enables conditioning on language-specified actions, with a video diffusion decoder that reconstructs perceptually detailed and temporally coherent visual observations, to achieve a unification between latent space reasoning (imagination) and realizable world dynamics (reality). Trained on large-scale video-action pairs spanning diverse domains, PAN supports open-domain, action-conditioned simulation with coherent, long-term dynamics. Extensive experiments show that PAN achieves strong performance in action-conditioned world simulation, long-horizon forecasting, and simulative reasoning compared to other video generators and world models, taking a step towards general world models that enable predictive simulation of future world states for reasoning and acting.

Method: ViXML efficiently integrates foundation vision models by pooling a single embedding per image and combines decoder-only models with vision capabilities, limiting computational growth while unlocking multi-modal capabilities.

Result: ViXML with small encoders outperforms text-only decoders in most cases, showing substantial improvements of up to +8.21% in P@1 on the largest dataset, surpassing previous state-of-the-art methods.

Conclusion: Both decoder-only models and visual information play critical roles in XMC, and their combination through ViXML delivers superior performance while maintaining computational efficiency, demonstrating that visual information can compensate for billions of parameters in text-only models.

Abstract: Foundation models have revolutionized artificial intelligence across numerous domains, yet their transformative potential remains largely untapped in Extreme Multi-label Classification (XMC). Queries in XMC are associated with relevant labels from extremely large label spaces, where it is critical to strike a balance between efficiency and performance. Therefore, many recent approaches efficiently pose XMC as a maximum inner product search between embeddings learned from small encoder-only transformer architectures. In this paper, we address two important aspects in XMC: how to effectively harness larger decoder-only models, and how to exploit visual information while maintaining computational efficiency. We demonstrate that both play a critical role in XMC separately and can be combined for improved performance. We show that a few billion-size decoder can deliver substantial improvements while keeping computational overhead manageable. Furthermore, our Vision-enhanced eXtreme Multi-label Learning framework (ViXML) efficiently integrates foundation vision models by pooling a single embedding per image. This limits computational growth while unlocking multi-modal capabilities. Remarkably, ViXML with small encoders outperforms text-only decoder in most cases, showing that an image is worth billions of parameters. Finally, we present an extension of existing text-only datasets to exploit visual metadata and make them available for future benchmarking. Comprehensive experiments across four public text-only datasets and their corresponding image enhanced versions validate our proposals’ effectiveness, surpassing previous state-of-the-art by up to +8.21% in P@1 on the largest dataset. ViXML’s code is available at https://github.com/DiegoOrtego/vixml.

Method: Uses Vision-Language Models (VLMs) to detect and extract attributes from individual image and text elements, organizing them into coherent SVG format with iterative refinement during inference.

Result: Achieves reconstruction LPIPS of 0.069 and is preferred by human evaluators in 82.9% of cases compared to strongest zero-shot VLM baseline.

Conclusion: SliDer effectively restores editability to slide documents by converting raster images to structured SVG representations while preserving semantic distinctions between elements.

Abstract: Multimedia documents such as slide presentations and posters are designed to be interactive and easy to modify. Yet, they are often distributed in a static raster format, which limits editing and customization. Restoring their editability requires converting these raster images back into structured vector formats. However, existing geometric raster-vectorization methods, which rely on low-level primitives like curves and polygons, fall short at this task. Specifically, when applied to complex documents like slides, they fail to preserve the high-level structure, resulting in a flat collection of shapes where the semantic distinction between image and text elements is lost. To overcome this limitation, we address the problem of semantic document derendering by introducing SliDer, a novel framework that uses Vision-Language Models (VLMs) to derender slide images as compact and editable Scalable Vector Graphic (SVG) representations. SliDer detects and extracts attributes from individual image and text elements in a raster input and organizes them into a coherent SVG format. Crucially, the model iteratively refines its predictions during inference in a process analogous to human design, generating SVG code that more faithfully reconstructs the original raster upon rendering. Furthermore, we introduce Slide2SVG, a novel dataset comprising raster-SVG pairs of slide documents curated from real-world scientific presentations, to facilitate future research in this domain. Our results demonstrate that SliDer achieves a reconstruction LPIPS of 0.069 and is favored by human evaluators in 82.9% of cases compared to the strongest zero-shot VLM baseline.

Method: OCR introduces structured perturbations to 3D object geometry during training, degrades object-specific visual cues, projects altered geometry to 2D space, and uses a rollout-based training pipeline with vanilla and region-noisy videos.

Result: Achieves 47.5% accuracy on VSI-Bench with a 3B-parameter model, outperforming several 7B baselines and showing superiority over prior rollout strategies.

Conclusion: OCR effectively addresses query-locked reasoning limitations and enables more robust spatial reasoning in video understanding tasks.

Abstract: Recent advances in Multi-modal Large Language Models (MLLMs) have showcased remarkable capabilities in vision-language understanding. However, enabling robust video spatial reasoning-the ability to comprehend object locations, orientations, and inter-object relationships in dynamic 3D scenes-remains a key unsolved challenge. Existing approaches primarily rely on spatially grounded supervised fine-tuning or reinforcement learning, yet we observe that such models often exhibit query-locked reasoning, focusing narrowly on objects explicitly mentioned in the prompt while ignoring critical contextual cues. To address this limitation, we propose Object-Centric 3D Rollout (OCR), a novel strategy that introduces structured perturbations to the 3D geometry of selected objects during training. By degrading object-specific visual cues and projecting the altered geometry into 2D space, OCR compels the model to reason holistically across the entire scene. We further design a rollout-based training pipeline that jointly leverages vanilla and region-noisy videos to optimize spatial reasoning trajectories. Experiments demonstrate state-of-the-art performance: our 3B-parameter model achieves 47.5% accuracy on VSI-Bench, outperforming several 7B baselines. Ablations confirm OCR’s superiority over prior rollout strategies (e.g., T-GRPO, NoisyRollout).

Method: Differentiable framework that optimizes single- and dual-color Bezier strokes through parallel differentiable paint renderer, with style generation for geometry-conditioned textures and differentiable smudge operator for natural blending. Uses coarse-to-fine optimization strategy for joint optimization of stroke geometry, color, and texture.

Result: Produces realistic and expressive stroke reconstructions, smooth tonal transitions, and richly stylized appearances across oil, watercolor, ink, and digital paintings.

Conclusion: Offers a unified model for expressive digital painting creation that faithfully reproduces the human painting-smudging loop with explicit stroke structure and realistic shading.

Abstract: Painting embodies a unique form of visual storytelling, where the creation process is as significant as the final artwork. Although recent advances in generative models have enabled visually compelling painting synthesis, most existing methods focus solely on final image generation or patch-based process simulation, lacking explicit stroke structure and failing to produce smooth, realistic shading. In this work, we present a differentiable stroke reconstruction framework that unifies painting, stylized texturing, and smudging to faithfully reproduce the human painting-smudging loop. Given an input image, our framework first optimizes single- and dual-color Bezier strokes through a parallel differentiable paint renderer, followed by a style generation module that synthesizes geometry-conditioned textures across diverse painting styles. We further introduce a differentiable smudge operator to enable natural color blending and shading. Coupled with a coarse-to-fine optimization strategy, our method jointly optimizes stroke geometry, color, and texture under geometric and semantic guidance. Extensive experiments on oil, watercolor, ink, and digital paintings demonstrate that our approach produces realistic and expressive stroke reconstructions, smooth tonal transitions, and richly stylized appearances, offering a unified model for expressive digital painting creation. See our project page for more demos: https://yingjiang96.github.io/DiffPaintWebsite/.

Method: Proposes a Difficulty-Aware Label-Guided Denoising framework that applies adaptive perturbations (stronger for easier instances, weaker for harder cases) and reconstructs ground-truth labels to provide explicit geometric supervision through joint optimization of label reconstruction and 3D detection.

Result: Extensive experiments on KITTI benchmark demonstrate state-of-the-art performance across all difficulty levels.

Conclusion: MonoDLGD effectively addresses depth ambiguity in monocular 3D detection through geometry-aware representation learning and adaptive difficulty handling, achieving superior performance on challenging benchmarks.

Abstract: Monocular 3D object detection is a cost-effective solution for applications like autonomous driving and robotics, but remains fundamentally ill-posed due to inherently ambiguous depth cues. Recent DETR-based methods attempt to mitigate this through global attention and auxiliary depth prediction, yet they still struggle with inaccurate depth estimates. Moreover, these methods often overlook instance-level detection difficulty, such as occlusion, distance, and truncation, leading to suboptimal detection performance. We propose MonoDLGD, a novel Difficulty-Aware Label-Guided Denoising framework that adaptively perturbs and reconstructs ground-truth labels based on detection uncertainty. Specifically, MonoDLGD applies stronger perturbations to easier instances and weaker ones into harder cases, and then reconstructs them to effectively provide explicit geometric supervision. By jointly optimizing label reconstruction and 3D object detection, MonoDLGD encourages geometry-aware representation learning and improves robustness to varying levels of object complexity. Extensive experiments on the KITTI benchmark demonstrate that MonoDLGD achieves state-of-the-art performance across all difficulty levels.

Method: Proposed a self-supervised pipeline to extract ultrasound images from photographs of the monitor display.

Result: In proof-of-concept study, rectified images retained sufficient visual fidelity to classify cardiac views with balanced accuracy of 0.79 compared to native DICOMs.

Conclusion: The method successfully bypasses the DICOM bottleneck and enables rapid testing and prototyping of ultrasound algorithms using extracted images from monitor photos.

Abstract: Ultrasound (US) machines display images on a built-in monitor, but routine transfer to hospital systems relies on DICOM. We propose a self-supervised pipeline to extract the US image from a photograph of the monitor. This removes the DICOM bottleneck and enables rapid testing and prototyping of new algorithms. In a proof-of-concept study, the rectified images retained enough visual fidelity to classify cardiac views with a balanced accuracy of 0.79 with respect to the native DICOMs.

Method: Two core modules: Motion-aware Temporal Attention and Recalibration (MoTAR) estimates motion salience and adjusts temporal focus using shift-based attention and Transformer modeling. Category-Oriented Refinement (CORE) injects soft anomaly category priors by aligning segment-level features with learnable category prototypes via cross-attention.

Result: Extensive experiments on WVAD benchmark validate the effectiveness of RefineVAD and demonstrate the importance of integrating semantic context to guide feature refinement toward anomaly-relevant patterns.

Conclusion: The framework successfully models both “how” motion evolves and “what” semantic category it resembles, providing a more nuanced approach to weakly-supervised video anomaly detection by leveraging temporal dynamics and semantic structure.

Abstract: Weakly-Supervised Video Anomaly Detection aims to identify anomalous events using only video-level labels, balancing annotation efficiency with practical applicability. However, existing methods often oversimplify the anomaly space by treating all abnormal events as a single category, overlooking the diverse semantic and temporal characteristics intrinsic to real-world anomalies. Inspired by how humans perceive anomalies, by jointly interpreting temporal motion patterns and semantic structures underlying different anomaly types, we propose RefineVAD, a novel framework that mimics this dual-process reasoning. Our framework integrates two core modules. The first, Motion-aware Temporal Attention and Recalibration (MoTAR), estimates motion salience and dynamically adjusts temporal focus via shift-based attention and global Transformer-based modeling. The second, Category-Oriented Refinement (CORE), injects soft anomaly category priors into the representation space by aligning segment-level features with learnable category prototypes through cross-attention. By jointly leveraging temporal dynamics and semantic structure, explicitly models both “how” motion evolves and “what” semantic category it resembles. Extensive experiments on WVAD benchmark validate the effectiveness of RefineVAD and highlight the importance of integrating semantic context to guide feature refinement toward anomaly-relevant patterns.

Method: Uses an image segmentation oracle to supervise poisoned CLIP outputs, develops algorithms to identify triggers and victim samples, and creates a compact fine-tuning dataset for model rectification.

Result: Extensive experiments show the strategy effectively defends against backdoor attacks in CLIP-based models on visual recognition benchmarks.

Conclusion: The proposed approach successfully enhances CLIP model robustness against backdoor attacks by efficiently identifying triggers and affected components, enabling targeted rectification.

Abstract: The advent of multimodal deep learning models, such as CLIP, has unlocked new frontiers in a wide range of applications, from image-text understanding to classification tasks. However, these models are not safe for adversarial attacks, particularly backdoor attacks, which can subtly manipulate model behavior. Moreover, existing defense methods typically involve training from scratch or fine-tuning using a large dataset without pinpointing the specific labels that are affected. In this study, we introduce an innovative strategy to enhance the robustness of multimodal contrastive learning models against such attacks. In particular, given a poisoned CLIP model, our approach can identify the backdoor trigger and pinpoint the victim samples and labels in an efficient manner. To that end, an image segmentation ``oracle’’ is introduced as the supervisor for the output of the poisoned CLIP. We develop two algorithms to rectify the poisoned model: (1) differentiating between CLIP and Oracle’s knowledge to identify potential triggers; (2) pinpointing affected labels and victim samples, and curating a compact fine-tuning dataset. With this knowledge, we are allowed to rectify the poisoned CLIP model to negate backdoor effects. Extensive experiments on visual recognition benchmarks demonstrate our strategy is effective in CLIP-based backdoor defense.

Method: Proposes PAVE-Net with spatial encoder for intra-frame relations and spatiotemporal pose decoder for global dependencies. Uses pose-aware attention mechanism for selective feature aggregation across frames and explicit modeling of spatiotemporal dependencies among keypoints.

Result: Achieves 6.0 mAP improvement on PoseTrack2017 compared to prior image-based end-to-end methods, and delivers competitive accuracy with state-of-the-art two-stage video approaches while offering significant efficiency gains.

Conclusion: PAVE-Net successfully demonstrates the viability of end-to-end multi-person video pose estimation, outperforming existing methods in both accuracy and efficiency through its novel pose-aware attention and spatiotemporal modeling approach.

Abstract: Existing multi-person video pose estimation methods typically adopt a two-stage pipeline: detecting individuals in each frame, followed by temporal modeling for single-person pose estimation. This design relies on heuristic operations such as detection, RoI cropping, and non-maximum suppression (NMS), limiting both accuracy and efficiency. In this paper, we present a fully end-to-end framework for multi-person 2D pose estimation in videos, effectively eliminating heuristic operations. A key challenge is to associate individuals across frames under complex and overlapping temporal trajectories. To address this, we introduce a novel Pose-Aware Video transformEr Network (PAVE-Net), which features a spatial encoder to model intra-frame relations and a spatiotemporal pose decoder to capture global dependencies across frames. To achieve accurate temporal association, we propose a pose-aware attention mechanism that enables each pose query to selectively aggregate features corresponding to the same individual across consecutive frames.Additionally, we explicitly model spatiotemporal dependencies among pose keypoints to improve accuracy. Notably, our approach is the first end-to-end method for multi-frame 2D human pose estimation.Extensive experiments show that PAVE-Net substantially outperforms prior image-based end-to-end methods, achieving a \textbf{6.0} mAP improvement on PoseTrack2017, and delivers accuracy competitive with state-of-the-art two-stage video-based approaches, while offering significant gains in efficiency.Project page: https://github.com/zgspose/PAVENet

Method: Proposed Hierarchical Prompt Learning with Task-Routed Transformer using dual classification tokens, hierarchical prompt generation with identity-level and instance-level pseudo-text tokens, and Cross-Modal Prompt Regularization for semantic alignment.

Result: Achieved state-of-the-art performance on multiple ReID benchmarks for both I2I and T2I tasks, validating the effectiveness of the unified framework.

Conclusion: The HPL framework successfully unifies I2I and T2I person re-identification through hierarchical prompt learning, demonstrating that joint optimization with task-aware prompts and cross-modal regularization leads to superior performance on both tasks.

Abstract: Person re-identification (ReID) aims to retrieve target pedestrian images given either visual queries (image-to-image, I2I) or textual descriptions (text-to-image, T2I). Although both tasks share a common retrieval objective, they pose distinct challenges: I2I emphasizes discriminative identity learning, while T2I requires accurate cross-modal semantic alignment. Existing methods often treat these tasks separately, which may lead to representation entanglement and suboptimal performance. To address this, we propose a unified framework named Hierarchical Prompt Learning (HPL), which leverages task-aware prompt modeling to jointly optimize both tasks. Specifically, we first introduce a Task-Routed Transformer, which incorporates dual classification tokens into a shared visual encoder to route features for I2I and T2I branches respectively. On top of this, we develop a hierarchical prompt generation scheme that integrates identity-level learnable tokens with instance-level pseudo-text tokens. These pseudo-tokens are derived from image or text features via modality-specific inversion networks, injecting fine-grained, instance-specific semantics into the prompts. Furthermore, we propose a Cross-Modal Prompt Regularization strategy to enforce semantic alignment in the prompt token space, ensuring that pseudo-prompts preserve source-modality characteristics while enhancing cross-modal transferability. Extensive experiments on multiple ReID benchmarks validate the effectiveness of our method, achieving state-of-the-art performance on both I2I and T2I tasks.

Method: Proposes dynamic attention policy (DAP) with Hierarchical Attention Fusion module for token-to-point attentions, optimized via Monte Carlo tree search with hybrid reward signals. Also introduces Efficient Retrieval Strategy (ERS) for hierarchical searching in large-scale embedding spaces.

Result: Extensive experiments demonstrate superior performance on diverse benchmarks, outperforming traditional methods in both accuracy and efficiency.

Conclusion: 3DAlign-DAER effectively addresses fine-grained text-3D alignment challenges and scales well to large databases, with plans to release codes, models, and the constructed Align3D-2M dataset.

Abstract: Despite recent advancements in 3D-text cross-modal alignment, existing state-of-the-art methods still struggle to align fine-grained textual semantics with detailed geometric structures, and their alignment performance degrades significantly when scaling to large-scale 3D databases. To overcome this limitation, we introduce 3DAlign-DAER, a unified framework designed to align text and 3D geometry via the proposed dynamic attention policy and the efficient retrieval strategy, capturing subtle correspondences for diverse cross-modal retrieval and classification tasks. Specifically, during the training, our proposed dynamic attention policy (DAP) employs the Hierarchical Attention Fusion (HAF) module to represent the alignment as learnable fine-grained token-to-point attentions. To optimize these attentions across different tasks and geometric hierarchies, our DAP further exploits the Monte Carlo tree search to dynamically calibrate HAF attention weights via a hybrid reward signal and further enhances the alignment between textual descriptions and local 3D geometry. During the inference, our 3DAlign-DAER introduces an Efficient Retrieval Strategy (ERS) to leverage efficient hierarchical searching in the large-scale embedding spaces, outperforming traditional methods (e.g., KNN) in accuracy and efficiency. Furthermore, to facilitate text-3D alignment research and train our 3DAlign-DAER, we construct Align3D-2M, a large-scale dataset featuring 2M text-3D pairs, to provide sufficient fine-grained cross-modal annotations. Extensive and comprehensive experiments demonstrate the superior performance of our 3DAlign-DAER on diverse benchmarks. We will release our codes, models, and datasets.

Method: Proposes VVS framework with three modules: verification-free token selector with dynamical truncation, token-level feature caching and reuse, and fine-grained skipped step scheduling. Leverages visual token interchangeability to skip verification steps.

Result: Reduces number of target model forward passes by 2.8x compared to vanilla AR decoding while maintaining competitive generation quality.

Conclusion: VVS offers superior speed-quality trade-off over conventional SD frameworks and has strong potential to reshape the speculative decoding paradigm for visual AR generation.

Abstract: Visual autoregressive (AR) generation models have demonstrated strong potential for image generation, yet their next-token-prediction paradigm introduces considerable inference latency. Although speculative decoding (SD) has been proven effective for accelerating visual AR models, its “draft one step, then verify one step” paradigm prevents a direct reduction of the forward passes, thus restricting acceleration potential. Motivated by the visual token interchangeability, we for the first time to explore verification skipping in the SD process of visual AR model generation to explicitly cut the number of target model forward passes, thereby reducing inference latency. Based on an analysis of the drafting stage’s characteristics, we observe that verification redundancy and stale feature reusability are key factors to retain generation quality and speedup for verification-free steps. Inspired by these two observations, we propose a novel SD framework VVS to accelerate visual AR generation via partial verification skipping, which integrates three complementary modules: (1) a verification-free token selector with dynamical truncation, (2) token-level feature caching and reuse, and (3) fine-grained skipped step scheduling. Consequently, VVS reduces the number of target model forward passes by a factor of $2.8\times$ relative to vanilla AR decoding while maintaining competitive generation quality, offering a superior speed-quality trade-off over conventional SD frameworks and revealing strong potential to reshape the SD paradigm.

Method: Proposes Hybrid-domain Adaptative Representation Learning (HARL) that: 1) Disentangles gaze-relevant representation by aligning features from high-quality near-eye images in unsupervised domain adaptation, 2) Uses sparse graph fusion module to explore geometric constraints between gaze direction and head-pose.

Result: Achieves state-of-the-art accuracy of 5.02° on EyeDiap, 3.36° on MPIIFaceGaze, and 9.26° on Gaze360 datasets, with competitive cross-dataset performance.

Conclusion: HARL framework effectively learns robust gaze representation by combining hybrid-domain adaptation with geometric constraints, significantly improving cross-domain gaze estimation performance without additional computational costs.

Abstract: Appearance-based gaze estimation, aiming to predict accurate 3D gaze direction from a single facial image, has made promising progress in recent years. However, most methods suffer significant performance degradation in cross-domain evaluation due to interference from gaze-irrelevant factors, such as expressions, wearables, and image quality. To alleviate this problem, we present a novel Hybrid-domain Adaptative Representation Learning (shorted by HARL) framework that exploits multi-source hybrid datasets to learn robust gaze representation. More specifically, we propose to disentangle gaze-relevant representation from low-quality facial images by aligning features extracted from high-quality near-eye images in an unsupervised domain-adaptation manner, which hardly requires any computational or inference costs. Additionally, we analyze the effect of head-pose and design a simple yet efficient sparse graph fusion module to explore the geometric constraint between gaze direction and head-pose, leading to a dense and robust gaze representation. Extensive experiments on EyeDiap, MPIIFaceGaze, and Gaze360 datasets demonstrate that our approach achieves state-of-the-art accuracy of $\textbf{5.02}^{\circ}$ and $\textbf{3.36}^{\circ}$, and $\textbf{9.26}^{\circ}$ respectively, and present competitive performances through cross-dataset evaluation. The code is available at https://github.com/da60266/HARL.

Method: 3D conditional diffusion framework combining K-space degradation for physics-consistent simulation, v-prediction with classifier-free guidance, and SNR-weighted 3D perceptual loss using volumetric attention-UNet architecture.

Result: Achieved 15.3% PSNR improvement over state-of-the-art methods, with 85% of outputs rated as good quality by physicians with clear pathology present.

Conclusion: MRIQT enables high-fidelity enhancement of portable ultra-low-field MRI for reliable neonatal brain assessment through diffusion-based image quality transfer.

Abstract: Portable ultra-low-field MRI (uLF-MRI, 0.064 T) offers accessible neuroimaging for neonatal care but suffers from low signal-to-noise ratio and poor diagnostic quality compared to high-field (HF) MRI. We propose MRIQT, a 3D conditional diffusion framework for image quality transfer (IQT) from uLF to HF MRI. MRIQT combines realistic K-space degradation for physics-consistent uLF simulation, v-prediction with classifier-free guidance for stable image-to-image generation, and an SNR-weighted 3D perceptual loss for anatomical fidelity. The model denoises from a noised uLF input conditioned on the same scan, leveraging volumetric attention-UNet architecture for structure-preserving translation. Trained on a neonatal cohort with diverse pathologies, MRIQT surpasses recent GAN and CNN baselines in PSNR 15.3% with 1.78% over the state of the art, while physicians rated 85% of its outputs as good quality with clear pathology present. MRIQT enables high-fidelity, diffusion-based enhancement of portable ultra-low-field (uLF) MRI for deliable neonatal brain assessment.

Method: 1) Develop tailor-designed thinking modes for misinformation detection; 2) Task-specific instruction tuning to inject thinking modes into MLLMs; 3) Reinforcement learning with mixed advantage function to enhance reasoning; 4) Construct MMR dataset with 8K+ image-text pairs with reasoning processes.

Result: Achieves state-of-the-art performance on both in-domain and out-of-domain benchmark datasets while maintaining flexible inference and token usage.

Conclusion: MMD-Thinker effectively addresses reasoning limitations in multimodal misinformation detection through adaptive multi-dimensional thinking and specialized training approaches.

Abstract: Multimodal misinformation floods on various social media, and continues to evolve in the era of AI-generated content (AIGC). The emerged misinformation with low creation cost and high deception poses significant threats to society. While recent studies leverage general-purpose multimodal large language models (MLLMs) to achieve remarkable results in detection, they encounter two critical limitations: (1) Insufficient reasoning, where general-purpose MLLMs often follow the uniform reasoning paradigm but generate inaccurate explanations and judgments, due to the lack of the task-specific knowledge of multimodal misinformation detection. (2) Reasoning biases, where a single thinking mode make detectors a suboptimal path for judgment, struggling to keep pace with the fast-growing and intricate multimodal misinformation. In this paper, we propose MMD-Thinker, a two-stage framework for multimodal misinformation detection through adaptive multi-dimensional thinking. First, we develop tailor-designed thinking mode for multimodal misinformation detection. Second, we adopt task-specific instruction tuning to inject the tailored thinking mode into general-purpose MLLMs. Third, we further leverage reinforcement learning strategy with a mixed advantage function, which incentivizes the reasoning capabilities in trajectories. Furthermore, we construct the multimodal misinformation reasoning (MMR) dataset, encompasses more than 8K image-text pairs with both reasoning processes and classification labels, to make progress in the relam of multimodal misinformation detection. Experimental results demonstrate that our proposed MMD-Thinker achieves state-of-the-art performance on both in-domain and out-of-domain benchmark datasets, while maintaining flexible inference and token usage. Code will be publicly available at Github.

Method: Proposes RFMNet with multi-stage feature fusion, overlapped windows cross-attention for local information matching, and Referring Feature Aggregation module for progressive decoding.

Result: Achieves state-of-the-art performance on Ref-COD benchmark through extensive experiments.

Conclusion: The proposed multi-context fusion approach with local attention mechanisms effectively improves camouflaged object detection performance.

Abstract: Referring camouflaged object detection (Ref-COD) aims to identify hidden objects by incorporating reference information such as images and text descriptions. Previous research has transformed reference images with salient objects into one-dimensional prompts, yielding significant results. We explore ways to enhance performance through multi-context fusion of rich salient image features and camouflaged object features. Therefore, we propose RFMNet, which utilizes features from multiple encoding stages of the reference salient images and performs interactive fusion with the camouflage features at the corresponding encoding stages. Given that the features in salient object images contain abundant object-related detail information, performing feature fusion within local areas is more beneficial for detecting camouflaged objects. Therefore, we propose an Overlapped Windows Cross-attention mechanism to enable the model to focus more attention on the local information matching based on reference features. Besides, we propose the Referring Feature Aggregation (RFA) module to decode and segment the camouflaged objects progressively. Extensive experiments on the Ref-COD benchmark demonstrate that our method achieves state-of-the-art performance.

Method: Proposed two distinct ways to integrate angular margin into α-divergence: Q-Margin (margin in reference measure) and A3M (margin in logits). Addressed A3M training instability with prototype re-initialization strategy.

Result: Achieved significant performance gains on IJB-B and IJB-C face verification benchmarks and strong performance on VoxCeleb speaker verification. Models significantly outperform baselines at low false acceptance rates.

Conclusion: The proposed margin-based α-divergence losses provide crucial capability for practical high-security applications where minimizing false authentications is paramount, demonstrating superior performance especially at low FARs.

Abstract: Performance in face and speaker verification is largely driven by margin based softmax losses like CosFace and ArcFace. Recently introduced $α$-divergence loss functions offer a compelling alternative, particularly for their ability to induce sparse solutions (when $α>1$). However, integrating an angular margin-crucial for verification tasks-is not straightforward. We find this integration can be achieved in at least two distinct ways: via the reference measure (prior probabilities) or via the logits (unnormalized log-likelihoods). In this paper, we explore both pathways, deriving two novel margin-based $α$-divergence losses: Q-Margin (margin in the reference measure) and A3M (margin in the logits). We identify and address a critical training instability in A3M-caused by the interplay of penalized logits and sparsity-with a simple yet effective prototype re-initialization strategy. Our methods achieve significant performance gains on the challenging IJB-B and IJB-C face verification benchmarks. We demonstrate similarly strong performance in speaker verification on VoxCeleb. Crucially, our models significantly outperform strong baselines at low false acceptance rates (FAR). This capability is crucial for practical high-security applications, such as banking authentication, when minimizing false authentications is paramount.

Method: Comprehensive review of current techniques, datasets, and metrics; novel experiments evaluating representative VLM-based methods; technical analysis and gap identification.

Result: Established a new taxonomy for egocentric procedural assistance, identified limitations of existing VLM-based approaches, and created an active repository for ongoing research collection.

Conclusion: The work highlights current challenges and suggests future research directions for developing effective egocentric procedural AI assistants, with continuous updates through a public repository.

Abstract: Driven by recent advances in vision language models (VLMs) and egocentric perception research, we introduce the concept of an egocentric procedural AI assistant (EgoProceAssist) tailored to step-by-step support daily procedural tasks in a first-person view. In this work, we start by identifying three core tasks: egocentric procedural error detection, egocentric procedural learning, and egocentric procedural question answering. These tasks define the essential functions of EgoProceAssist within a new taxonomy. Specifically, our work encompasses a comprehensive review of current techniques, relevant datasets, and evaluation metrics across these three core areas. To clarify the gap between the proposed EgoProceAssist and existing VLM-based AI assistants, we introduce novel experiments and provide a comprehensive evaluation of representative VLM-based methods. Based on these findings and our technical analysis, we discuss the challenges ahead and suggest future research directions. Furthermore, an exhaustive list of this study is publicly available in an active repository that continuously collects the latest work: https://github.com/z1oong/Building-Egocentric-Procedural-AI-Assistant

Method: Proposes SymGS framework with learnable mirrors to eliminate local and global reflective redundancies. Works as plug-and-play enhancement to existing compression methods like HAC.

Result: Achieves 1.66× compression over HAC across benchmarks (up to 3× on large scenes) and 108× overall compression of 3DGS scenes while maintaining rendering quality.

Conclusion: Symmetry-aware compression through learnable mirrors effectively reduces memory footprint of 3D Gaussian Splatting beyond existing methods.

Abstract: 3D Gaussian Splatting has emerged as a transformative technique in novel view synthesis, primarily due to its high rendering speed and photorealistic fidelity. However, its memory footprint scales rapidly with scene complexity, often reaching several gigabytes. Existing methods address this issue by introducing compression strategies that exploit primitive-level redundancy through similarity detection and quantization. We aim to surpass the compression limits of such methods by incorporating symmetry-aware techniques, specifically targeting mirror symmetries to eliminate redundant primitives. We propose a novel compression framework, \textbf{\textit{SymGS}}, introducing learnable mirrors into the scene, thereby eliminating local and global reflective redundancies for compression. Our framework functions as a plug-and-play enhancement to state-of-the-art compression methods, (e.g. HAC) to achieve further compression. Compared to HAC, we achieve $1.66 \times$ compression across benchmark datasets (upto $3\times$ on large-scale scenes). On an average, SymGS enables $\bf{108\times}$ compression of a 3DGS scene, while preserving rendering quality. The project page and supplementary can be found at \textbf{\color{cyan}{symgs.github.io}}

Method: Created SpatialSky-Bench with 13 subcategories across Environmental Perception and Scene Understanding. Developed SpatialSky-Dataset with 1M samples and trained Sky-VLM for UAV spatial reasoning.

Result: Mainstream VLMs show unsatisfactory performance in complex UAV navigation. Sky-VLM achieves state-of-the-art performance across all benchmark tasks.

Conclusion: Sky-VLM paves the way for developing VLMs suitable for UAV scenarios, addressing spatial intelligence gaps in existing models.

Abstract: Vision-Language Models (VLMs), leveraging their powerful visual perception and reasoning capabilities, have been widely applied in Unmanned Aerial Vehicle (UAV) tasks. However, the spatial intelligence capabilities of existing VLMs in UAV scenarios remain largely unexplored, raising concerns about their effectiveness in navigating and interpreting dynamic environments. To bridge this gap, we introduce SpatialSky-Bench, a comprehensive benchmark specifically designed to evaluate the spatial intelligence capabilities of VLMs in UAV navigation. Our benchmark comprises two categories-Environmental Perception and Scene Understanding-divided into 13 subcategories, including bounding boxes, color, distance, height, and landing safety analysis, among others. Extensive evaluations of various mainstream open-source and closed-source VLMs reveal unsatisfactory performance in complex UAV navigation scenarios, highlighting significant gaps in their spatial capabilities. To address this challenge, we developed the SpatialSky-Dataset, a comprehensive dataset containing 1M samples with diverse annotations across various scenarios. Leveraging this dataset, we introduce Sky-VLM, a specialized VLM designed for UAV spatial reasoning across multiple granularities and contexts. Extensive experimental results demonstrate that Sky-VLM achieves state-of-the-art performance across all benchmark tasks, paving the way for the development of VLMs suitable for UAV scenarios. The source code is available at https://github.com/linglingxiansen/SpatialSKy.

Method: Proposes a dual-backbone framework that combines convolutional neural network (CNN) and transformer representations through top-k pooling mechanism.

Result: Achieves 90.7% area under the curve (AUC) on the UCF-Crime dataset for anomaly detection.

Conclusion: The dual-backbone approach with top-k pooling effectively detects diverse anomalies in surveillance videos using only video-level supervision.

Abstract: We address the challenge of detecting rare and diverse anomalies in surveillance videos using only video-level supervision. Our dual-backbone framework combines convolutional and transformer representations through top-k pooling, achieving 90.7% area under the curve (AUC) on the UCF-Crime dataset.

Method: Train initial 3D Gaussian Splatting field, use normal-gradient-guided Gaussian optimization to select primitives aligned with building structures, apply multi-view edge-consistency pruning, and perform multi-view depth-constrained Delaunay triangulation.

Result: SF-Recon achieves substantially fewer faces and vertices while maintaining computational efficiency, directly reconstructing lightweight building models from multi-view imagery.

Conclusion: The method successfully reconstructs lightweight building surfaces without post-hoc simplification, demonstrating improved efficiency and structural fidelity compared to conventional pipelines.

Abstract: Lightweight building surface models are crucial for digital city, navigation, and fast geospatial analytics, yet conventional multi-view geometry pipelines remain cumbersome and quality-sensitive due to their reliance on dense reconstruction, meshing, and subsequent simplification. This work presents SF-Recon, a method that directly reconstructs lightweight building surfaces from multi-view images without post-hoc mesh simplification. We first train an initial 3D Gaussian Splatting (3DGS) field to obtain a view-consistent representation. Building structure is then distilled by a normal-gradient-guided Gaussian optimization that selects primitives aligned with roof and wall boundaries, followed by multi-view edge-consistency pruning to enhance structural sharpness and suppress non-structural artifacts without external supervision. Finally, a multi-view depth-constrained Delaunay triangulation converts the structured Gaussian field into a lightweight, structurally faithful building mesh. Based on a proposed SF dataset, the experimental results demonstrate that our SF-Recon can directly reconstruct lightweight building models from multi-view imagery, achieving substantially fewer faces and vertices while maintaining computational efficiency. Website:https://lzh282140127-cell.github.io/SF-Recon-project/

Method: Introduces Depth-conditioned Translation Optimization (DTO) for joint refinement of camera-space translations using anthropometric priors and monocular depth cues, plus Metric-Aware HMR network with relative metric loss.

Result: Created DTO-Humans dataset with 0.56M high-quality, scene-consistent multi-person images (avg 4.8 persons/image), achieving state-of-the-art performance on relative depth reasoning and mesh recovery.

Conclusion: The proposed approach successfully addresses scene-level consistency in multi-person mesh recovery and enables direct metric-scale estimation through joint optimization and metric-aware learning.

Abstract: Multi-person human mesh recovery from a single image is a challenging task, hindered by the scarcity of in-the-wild training data. Prevailing in-the-wild human mesh pseudo-ground-truth (pGT) generation pipelines are single-person-centric, where each human is processed individually without joint optimization. This oversight leads to a lack of scene-level consistency, producing individuals with conflicting depths and scales within the same image. To address this, we introduce Depth-conditioned Translation Optimization (DTO), a novel optimization-based method that jointly refines the camera-space translations of all individuals in a crowd. By leveraging anthropometric priors on human height and depth cues from a monocular depth estimator, DTO solves for a scene-consistent placement of all subjects within a principled Maximum a posteriori (MAP) framework. Applying DTO to the 4D-Humans dataset, we construct DTO-Humans, a new large-scale pGT dataset of 0.56M high-quality, scene-consistent multi-person images, featuring dense crowds with an average of 4.8 persons per image. Furthermore, we propose Metric-Aware HMR, an end-to-end network that directly estimates human mesh and camera parameters in metric scale. This is enabled by a camera branch and a novel relative metric loss that enforces plausible relative scales. Extensive experiments demonstrate that our method achieves state-of-the-art performance on relative depth reasoning and human mesh recovery. Code and data will be released publicly.

Method: Extends divide-and-conquer strategy with mask-granularity pair discovery and granularity control embedding for continuous scale control.

Result: Achieves significant improvements: NoC90 (5.69→4.75), 1-IoU (58.0→73.1), AR1000 (49.6→68.3) across 11 benchmarks with only 6K unlabeled images.

Conclusion: Small unlabeled datasets with granularity-aware self-supervised learning can unlock vision foundation model potential.

Abstract: The Segment Anything Model (SAM) family has become a widely adopted vision foundation model, but its ability to control segmentation granularity remains limited. Users often need to refine results manually - by adding more prompts or selecting from pre-generated masks - to achieve the desired level of detail. This process can be ambiguous, as the same prompt may correspond to several plausible masks, and collecting dense annotations across all granularities is prohibitively expensive, making supervised solutions infeasible. To address this limitation, we introduce UnSAMv2, which enables segment anything at any granularity without human annotations. UnSAMv2 extends the divide-and-conquer strategy of UnSAM by discovering abundant mask-granularity pairs and introducing a novel granularity control embedding that enables precise, continuous control over segmentation scale. Remarkably, with only $6$K unlabeled images and $0.02%$ additional parameters, UnSAMv2 substantially enhances SAM-2, achieving segment anything at any granularity across interactive, whole-image, and video segmentation tasks. Evaluated on over $11$ benchmarks, UnSAMv2 improves $\text{NoC}{90}$ (5.69 $\rightarrow$ 4.75), 1-IoU (58.0 $\rightarrow$ 73.1), and $\text{AR}{1000}$ (49.6 $\rightarrow$ 68.3), showing that small amounts of unlabeled data with a granularity-aware self-supervised learning method can unlock the potential of vision foundation models.

Method: Three key approaches: 1) Progressive question conditioning - injects questions into Vision Transformer layers with increasing frequency; 2) Token pruning - discards background tokens to reduce redundancy; 3) Token focusing - training strategy to concentrate essential information in retained tokens.

Result: TabFlash achieves state-of-the-art performance, outperforming both open-source and proprietary MLLMs, while requiring 27% less FLOPs and 30% less memory usage compared to the second-best MLLM.

Conclusion: The proposed TabFlash model effectively addresses table understanding challenges by generating informative and compact visual features through progressive conditioning, pruning, and focusing strategies, achieving superior efficiency and performance.

Abstract: Table images present unique challenges for effective and efficient understanding due to the need for question-specific focus and the presence of redundant background regions. Existing Multimodal Large Language Model (MLLM) approaches often overlook these characteristics, resulting in uninformative and redundant visual representations. To address these issues, we aim to generate visual features that are both informative and compact to improve table understanding. We first propose progressive question conditioning, which injects the question into Vision Transformer layers with gradually increasing frequency, considering each layer’s capacity to handle additional information, to generate question-aware visual features. To reduce redundancy, we introduce a pruning strategy that discards background tokens, thereby improving efficiency. To mitigate information loss from pruning, we further propose token focusing, a training strategy that encourages the model to concentrate essential information in the retained tokens. By combining these approaches, we present TabFlash, an efficient and effective MLLM for table understanding. TabFlash achieves state-of-the-art performance, outperforming both open-source and proprietary MLLMs, while requiring 27% less FLOPs and 30% less memory usage compared to the second-best MLLM.

Method: Users provide cropped glyph patches of desired typography as input. The framework enables simultaneous editing of multiple text regions with distinct fonts, preserving non-edited regions, without requiring font labels or fine-tuning during inference.

Result: Achieves state-of-the-art performance on multiple datasets including handwritten text benchmarks, with superior text fidelity and visual realism, offering unprecedented control over font families and stylistic nuances.

Conclusion: Bridges the gap between general-purpose image editing and professional-grade typographic design by providing precise font-controllable text editing capabilities.

Abstract: Artistic design such as poster design often demands rapid yet precise modification of textual content while preserving visual harmony and typographic intent, especially across diverse font styles. Although modern image editing models have grown increasingly powerful, they still fall short in fine-grained, font-aware text manipulation, limiting their utility in professional design workflows such as poster editing. To address this issue, we present SkyReels-Text, a novel font-controllable framework for precise poster text editing. Our method enables simultaneous editing of multiple text regions, each rendered in distinct typographic styles, while preserving the visual appearance of non-edited regions. Notably, our model requires neither font labels nor fine-tuning during inference: users can simply provide cropped glyph patches corresponding to their desired typography, even if the font is not included in any standard library. Extensive experiments on multiple datasets, including handwrittent text benchmarks, SkyReels-Text achieves state-of-the-art performance in both text fidelity and visual realism, offering unprecedented control over font families, and stylistic nuances. This work bridges the gap between general-purpose image editing and professional-grade typographic design.

Method: Propose CorrectAD system with PM-Agent for data requirements, DriveSora for generating spatiotemporally consistent videos aligned with 3D layouts, and automated annotation.

Result: Corrects 62.5% and 49.8% of failure cases on nuScenes and in-house datasets, reducing collision rates by 39% and 27% respectively across multiple planners.

Conclusion: The proposed end-to-end model-agnostic pipeline effectively self-corrects autonomous driving failures using generative world models and structured 3D layouts.

Abstract: End-to-end planning methods are the de facto standard of the current autonomous driving system, while the robustness of the data-driven approaches suffers due to the notorious long-tail problem (i.e., rare but safety-critical failure cases). In this work, we explore whether recent diffusion-based video generation methods (a.k.a. world models), paired with structured 3D layouts, can enable a fully automated pipeline to self-correct such failure cases. We first introduce an agent to simulate the role of product manager, dubbed PM-Agent, which formulates data requirements to collect data similar to the failure cases. Then, we use a generative model that can simulate both data collection and annotation. However, existing generative models struggle to generate high-fidelity data conditioned on 3D layouts. To address this, we propose DriveSora, which can generate spatiotemporally consistent videos aligned with the 3D annotations requested by PM-Agent. We integrate these components into our self-correcting agentic system, CorrectAD. Importantly, our pipeline is an end-to-end model-agnostic and can be applied to improve any end-to-end planner. Evaluated on both nuScenes and a more challenging in-house dataset across multiple end-to-end planners, CorrectAD corrects 62.5% and 49.8% of failure cases, reducing collision rates by 39% and 27%, respectively.

Method: Proposes DriveLiDAR4D pipeline with multimodal conditions and a novel sequential noise prediction model called LiDAR4DNet, enabling end-to-end sequential generation of LiDAR scenes with full scene manipulation capability.

Result: Achieved FRD score of 743.13 and FVD score of 16.96 on nuScenes dataset, outperforming SOTA method UniScene by 37.2% in FRD and 24.1% in FVD respectively.

Conclusion: DriveLiDAR4D is the first work to address sequential generation of LiDAR scenes with full scene manipulation capability, demonstrating significant improvements over existing methods for autonomous driving applications.

Abstract: The generation of realistic LiDAR point clouds plays a crucial role in the development and evaluation of autonomous driving systems. Although recent methods for 3D LiDAR point cloud generation have shown significant improvements, they still face notable limitations, including the lack of sequential generation capabilities and the inability to produce accurately positioned foreground objects and realistic backgrounds. These shortcomings hinder their practical applicability. In this paper, we introduce DriveLiDAR4D, a novel LiDAR generation pipeline consisting of multimodal conditions and a novel sequential noise prediction model LiDAR4DNet, capable of producing temporally consistent LiDAR scenes with highly controllable foreground objects and realistic backgrounds. To the best of our knowledge, this is the first work to address the sequential generation of LiDAR scenes with full scene manipulation capability in an end-to-end manner. We evaluated DriveLiDAR4D on the nuScenes and KITTI datasets, where we achieved an FRD score of 743.13 and an FVD score of 16.96 on the nuScenes dataset, surpassing the current state-of-the-art (SOTA) method, UniScene, with an performance boost of 37.2% in FRD and 24.1% in FVD, respectively.

Method: Mixture-of-Experts framework with adaptive routing mechanism that dynamically selects among multiple YOLOv9-T experts for different input features.

Result: Achieved higher mean Average Precision (mAP) and Average Recall (AR) compared to a single YOLOv9-T model.

Conclusion: The Mixture-of-Experts approach with adaptive routing effectively improves object detection performance by leveraging specialized feature processing from multiple experts.

Abstract: This paper presents a novel Mixture-of-Experts framework for object detection, incorporating adaptive routing among multiple YOLOv9-T experts to enable dynamic feature specialization and achieve higher mean Average Precision (mAP) and Average Recall (AR) compared to a single YOLOv9-T model.

Method: Introduce the ‘What Color Is It’ dataset using a simple method to trigger single-modality visual hallucination in MLMs, and investigate underlying causes.

Result: Validation of the hypothesis that MLMs are prone to visual perception interference, with the dataset successfully triggering hallucinations.

Conclusion: Proposed potential solutions to enhance the robustness of MLMs against visual modality hallucinations.

Abstract: With the rapid advancement of Large Models, numerous text-and-vision-fused Multimodal Large Models (MLMs) have emerged. However, these MLMs remain susceptible to informational interference in visual perception, particularly in color perception, which introduces an additional risk of hallucination. To validate this hypothesis, we introduce the “What Color Is It” dataset, a novel benchmark constructed using a simple method to trigger single-modality visual hallucination in MLMs. Based on this dataset, we further investigate the underlying causes of hallucination in the visual modality of MLMs and propose potential solutions to enhance their robustness.

Method: DelAnyFlow combines the DelAny instance segmentation model (based on YOLOv11 backbone) trained on the FBIS 22M dataset with structured post-processing, merging, and vectorization sequence. FBIS 22M contains 672,909 multi-resolution image patches and 22.9 million validated field instances.

Result: DelAny model achieves over 100% higher mAP and 400x faster inference than SAM2. DelAnyFlow generated a complete field boundary layer for Ukraine (603,000km²) in under 6 hours, delineating 3.75M fields at 5m and 5.15M at 2.5m resolution - significantly more than existing operational products.

Conclusion: DelAnyFlow provides a scalable, cost-effective methodology for field delineation in regions lacking digital cadastral data, with strong zero-shot generalization capabilities and support for national-scale applications.

Abstract: Accurate delineation of agricultural field boundaries from satellite imagery is essential for land management and crop monitoring, yet existing methods often produce incomplete boundaries, merge adjacent fields, and struggle to scale. We present the Delineate Anything Flow (DelAnyFlow) methodology, a resolution-agnostic approach for large-scale field boundary mapping. DelAnyFlow combines the DelAny instance segmentation model, based on a YOLOv11 backbone and trained on the large-scale Field Boundary Instance Segmentation-22M (FBIS 22M) dataset, with a structured post-processing, merging, and vectorization sequence to generate topologically consistent vector boundaries. FBIS 22M, the largest dataset of its kind, contains 672,909 multi-resolution image patches (0.25-10m) and 22.9million validated field instances. The DelAny model delivers state-of-the-art accuracy with over 100% higher mAP and 400x faster inference than SAM2. DelAny demonstrates strong zero-shot generalization and supports national-scale applications: using Sentinel 2 data for 2024, DelAnyFlow generated a complete field boundary layer for Ukraine (603,000km2) in under six hours on a single workstation. DelAnyFlow outputs significantly improve boundary completeness relative to operational products from Sinergise Solutions and NASA Harvest, particularly in smallholder and fragmented systems (0.25-1ha). For Ukraine, DelAnyFlow delineated 3.75M fields at 5m and 5.15M at 2.5m, compared to 2.66M detected by Sinergise Solutions and 1.69M by NASA Harvest. This work delivers a scalable, cost-effective methodology for field delineation in regions lacking digital cadastral data. A project landing page with links to model weights, code, national-scale vector outputs, and dataset is available at https://lavreniuk.github.io/Delineate-Anything/.

Method: VOPE uses recheck-based questions to evaluate how LVLMs interpret the presence of imagined objects in their own responses, measuring consistency between model interpretation and actual object presence in images.

Result: Most LVLMs hallucinate heavily during voluntary imagination and perform poorly on presence evaluation for imagined objects; existing hallucination mitigation methods show limited effectiveness.

Conclusion: Voluntary imagination tasks present significant hallucination challenges that current methods cannot adequately address, highlighting an important research direction for future work.

Abstract: Most research on hallucinations in Large Vision-Language Models (LVLMs) focuses on factual description tasks that prohibit any output absent from the image. However, little attention has been paid to hallucinations in voluntary imagination tasks, e.g., story writing, where the models are expected to generate novel content beyond the given image. In these tasks, it is inappropriate to simply regard such imagined novel content as hallucinations. To address this limitation, we introduce Voluntary-imagined Object Presence Evaluation (VOPE)-a novel method to assess LVLMs’ hallucinations in voluntary imagination tasks via presence evaluation. Specifically, VOPE poses recheck-based questions to evaluate how an LVLM interprets the presence of the imagined objects in its own response. The consistency between the model’s interpretation and the object’s presence in the image is then used to determine whether the model hallucinates when generating the response. We apply VOPE to several mainstream LVLMs and hallucination mitigation methods, revealing two key findings: (1) most LVLMs hallucinate heavily during voluntary imagination, and their performance in presence evaluation is notably poor on imagined objects; (2) existing hallucination mitigation methods show limited effect in voluntary imagination tasks, making this an important direction for future research.

Method: Represents 3D shapes as probability distributions via continuous invertible flows from a fixed anchor distribution, then composes inverse and forward flows to map between source and target shapes using task-tailored embeddings.

Result: Achieves high coverage and accuracy in shape matching benchmarks, and also shows promising results in UV mapping and point cloud registration tasks.

Conclusion: The flow-matching framework provides an effective, modality-agnostic approach for 3D shape mapping that works well across diverse representations and challenging scenarios.

Abstract: We introduce a novel neural representation for maps between 3D shapes based on flow-matching models, which is computationally efficient and supports cross-representation shape matching without large-scale training or data-driven procedures. 3D shapes are represented as the probability distribution induced by a continuous and invertible flow mapping from a fixed anchor distribution. Given a source and a target shape, the composition of the inverse flow (source to anchor) with the forward flow (anchor to target), we continuously map points between the two surfaces. By encoding the shapes with a pointwise task-tailored embedding, this construction provides an invertible and modality-agnostic representation of maps between shapes across point clouds, meshes, signed distance fields (SDFs), and volumetric data. The resulting representation consistently achieves high coverage and accuracy across diverse benchmarks and challenging settings in shape matching. Beyond shape matching, our framework shows promising results in other tasks, including UV mapping and registration of raw point cloud scans of human bodies.

Method: Built on Dynamic Temporal-Selective Mixture of Experts with a routing mechanism that uses both high-level text semantics and low-level motion context to dispatch temporal motion features to specialized experts, allowing dynamic capacity determination and focus on critical temporal features.

Result: Achieves state-of-the-art performance, reducing FID scores by 9% on InterHuman dataset and 22% on InterX dataset for individual-specific high-fidelity 3D human interaction generation.

Conclusion: InterMoE effectively addresses the challenges of preserving individual characteristics and semantic fidelity in human interaction generation through its specialized expert routing mechanism.

Abstract: Generating high-quality human interactions holds significant value for applications like virtual reality and robotics. However, existing methods often fail to preserve unique individual characteristics or fully adhere to textual descriptions. To address these challenges, we introduce InterMoE, a novel framework built on a Dynamic Temporal-Selective Mixture of Experts. The core of InterMoE is a routing mechanism that synergistically uses both high-level text semantics and low-level motion context to dispatch temporal motion features to specialized experts. This allows experts to dynamically determine the selection capacity and focus on critical temporal features, thereby preserving specific individual characteristic identities while ensuring high semantic fidelity. Extensive experiments show that InterMoE achieves state-of-the-art performance in individual-specific high-fidelity 3D human interaction generation, reducing FID scores by 9% on the InterHuman dataset and 22% on InterX.

Method: Language-Guided Invariance Probing (LGIP) uses 40k MS COCO images with human captions to automatically generate paraphrases and rule-based semantic flips (object category, color, count changes), measuring invariance error, semantic sensitivity gap, and positive-rate statistics.

Result: EVA02-CLIP and large OpenCLIP variants show favorable invariance-sensitivity balance, while SigLIP/SigLIP2 exhibit high invariance error and often prefer flipped captions over human descriptions, especially for object and color edits.

Conclusion: LGIP provides crucial diagnostic for VLM linguistic robustness, revealing failures invisible to standard retrieval metrics and highlighting the need for systematic evaluation of linguistic sensitivity.

Abstract: Recent vision-language models (VLMs) such as CLIP, OpenCLIP, EVA02-CLIP and SigLIP achieve strong zero-shot performance, but it is unclear how reliably they respond to controlled linguistic perturbations. We introduce Language-Guided Invariance Probing (LGIP), a benchmark that measures (i) invariance to meaning-preserving paraphrases and (ii) sensitivity to meaning-changing semantic flips in image-text matching. Using 40k MS COCO images with five human captions each, we automatically generate paraphrases and rule-based flips that alter object category, color or count, and summarize model behavior with an invariance error, a semantic sensitivity gap and a positive-rate statistic. Across nine VLMs, EVA02-CLIP and large OpenCLIP variants lie on a favorable invariance-sensitivity frontier, combining low paraphrase-induced variance with consistently higher scores for original captions than for their flipped counterparts. In contrast, SigLIP and SigLIP2 show much larger invariance error and often prefer flipped captions to the human descriptions, especially for object and color edits. These failures are largely invisible to standard retrieval metrics, indicating that LGIP provides a model-agnostic diagnostic for the linguistic robustness of VLMs beyond conventional accuracy scores.

Method: Semantic segmentation of multi-temporal remote sensing imagery to map UV boundaries, then classifying post-demolition land use into six categories across four representative Chinese cities.

Result: UV redevelopment processes were frequently prolonged; transitions primarily in peripheral areas; three spatiotemporal transformation pathways identified: synchronized, delayed, and gradual optimization.

Conclusion: UV redevelopment is fragmented, complex and nonlinear, requiring tiered and context-sensitive planning strategies for more inclusive, efficient, and sustainable urban renewal.

Abstract: Urban villages (UVs), informal settlements embedded within China’s urban fabric, have undergone widespread demolition and redevelopment in recent decades. However, there remains a lack of systematic evaluation of whether the demolished land has been effectively reused, raising concerns about the efficacy and sustainability of current redevelopment practices. To address the gap, this study proposes a deep learning-based framework to monitor the spatiotemporal changes of UVs in China. Specifically, semantic segmentation of multi-temporal remote sensing imagery is first used to map evolving UV boundaries, and then post-demolition land use is classified into six categories based on the “remained-demolished-redeveloped” phase: incomplete demolition, vacant land, construction sites, buildings, green spaces, and others. Four representative cities from China’s four economic regions were selected as the study areas, i.e., Guangzhou (East), Zhengzhou (Central), Xi’an (West), and Harbin (Northeast). The results indicate: 1) UV redevelopment processes were frequently prolonged; 2) redevelopment transitions primarily occurred in peripheral areas, whereas urban cores remained relatively stable; and 3) three spatiotemporal transformation pathways, i.e., synchronized redevelopment, delayed redevelopment, and gradual optimization, were revealed. This study highlights the fragmented, complex and nonlinear nature of UV redevelopment, underscoring the need for tiered and context-sensitive planning strategies. By linking spatial dynamics with the context of redevelopment policies, the findings offer valuable empirical insights that support more inclusive, efficient, and sustainable urban renewal, while also contributing to a broader global understanding of informal settlement transformations.

Method: Developed an asymptotically minimax approach to multi-target conformal prediction that ensures joint marginal coverage while providing tight prediction intervals.

Result: Numerical demonstrations using synthetic and MRI data show benefits over existing multi-target conformal prediction methods.

Conclusion: The proposed minimax approach effectively addresses multi-target uncertainty quantification in imaging inverse problems and can be applied to various applications including multi-metric image quality assessment and multi-task uncertainty quantification.

Abstract: In ill-posed imaging inverse problems, uncertainty quantification remains a fundamental challenge, especially in safety-critical applications. Recently, conformal prediction has been used to quantify the uncertainty that the inverse problem contributes to downstream tasks like image classification, image quality assessment, fat mass quantification, etc. While existing works handle only a scalar estimation target, practical applications often involve multiple targets. In response, we propose an asymptotically minimax approach to multi-target conformal prediction that provides tight prediction intervals while ensuring joint marginal coverage. We then outline how our minimax approach can be applied to multi-metric blind image quality assessment, multi-task uncertainty quantification, and multi-round measurement acquisition. Finally, we numerically demonstrate the benefits of our minimax method, relative to existing multi-target conformal prediction methods, using both synthetic and magnetic resonance imaging (MRI) data.

Method: Proposed Chromatic Perturbation Module that systematically crafts adversarial examples by altering color contrast between foreground and background to disrupt explanation fidelity, accumulating perturbations across federated learning training rounds.

Result: Attack reduces peak activation overlap in Grad-CAM explanations by up to 35% while preserving classification accuracy above 96% across multiple datasets, showing standard training pipelines fail to detect explanation degradation.

Conclusion: Interpretability can be compromised independently of accuracy, challenging the assumption that correct predictions imply faithful explanations, especially in federated learning where subtle color perturbations are hard to detect.

Abstract: As machine learning models are increasingly deployed in safety-critical domains, visual explanation techniques have become essential tools for supporting transparency. In this work, we reveal a new class of attacks that compromise model interpretability without affecting accuracy. Specifically, we show that small color perturbations applied by adversarial clients in a federated learning setting can shift a model’s saliency maps away from semantically meaningful regions while keeping the prediction unchanged. The proposed saliency-aware attack framework, called Chromatic Perturbation Module, systematically crafts adversarial examples by altering the color contrast between foreground and background in a way that disrupts explanation fidelity. These perturbations accumulate across training rounds, poisoning the global model’s internal feature attributions in a stealthy and persistent manner. Our findings challenge a common assumption in model auditing that correct predictions imply faithful explanations and demonstrate that interpretability itself can be an attack surface. We evaluate this vulnerability across multiple datasets and show that standard training pipelines are insufficient to detect or mitigate explanation degradation, especially in the federated learning setting, where subtle color perturbations are harder to discern. Our attack reduces peak activation overlap in Grad-CAM explanations by up to 35% while preserving classification accuracy above 96% on all evaluated datasets.

Method: BootOOD synthesizes pseudo-OOD features through transformations of ID representations and uses a lightweight auxiliary head for radius-based classification on feature norms, leveraging Neural Collapse properties.

Result: BootOOD outperforms prior post-hoc methods, surpasses training-based methods without outlier exposure, and is competitive with state-of-the-art outlier-exposure approaches while maintaining or improving ID accuracy on CIFAR-10, CIFAR-100, and ImageNet-200.

Conclusion: BootOOD provides an effective self-supervised framework for OOD detection that handles semantically challenging cases by focusing on feature norm differences rather than orthogonal subspaces.

Abstract: Out-of-distribution (OOD) detection is critical for deploying image classifiers in safety-sensitive environments, yet existing detectors often struggle when OOD samples are semantically similar to the in-distribution (ID) classes. We present BootOOD, a fully self-supervised OOD detection framework that bootstraps exclusively from ID data and is explicitly designed to handle semantically challenging OOD samples. BootOOD synthesizes pseudo-OOD features through simple transformations of ID representations and leverages Neural Collapse (NC), where ID features cluster tightly around class means with consistent feature norms. Unlike prior approaches that aim to constrain OOD features into subspaces orthogonal to the collapsed ID means, BootOOD introduces a lightweight auxiliary head that performs radius-based classification on feature norms. This design decouples OOD detection from the primary classifier and imposes a relaxed requirement: OOD samples are learned to have smaller feature norms than ID features, which is easier to satisfy when ID and OOD are semantically close. Experiments on CIFAR-10, CIFAR-100, and ImageNet-200 show that BootOOD outperforms prior post-hoc methods, surpasses training-based methods without outlier exposure, and is competitive with state-of-the-art outlier-exposure approaches while maintaining or improving ID accuracy.

Method: TSE-Net integrates teacher, student, and exam networks. The teacher generates pseudo-labels through joint regression and classification with hierarchical bi-cut strategy for long-tailed height distribution. The student learns from pseudo-labels, while the exam stabilizes performance as a temporal ensemble.

Result: The proposed pipeline was evaluated on three datasets spanning different resolutions and imaging modalities, demonstrating improved performance through semi-supervised learning.

Conclusion: TSE-Net effectively addresses data scarcity in monocular height estimation by leveraging unlabeled data through a semi-supervised framework, improving model generalization and performance while reducing dependency on expensive labeled data.

Abstract: Monocular height estimation plays a critical role in 3D perception for remote sensing, offering a cost-effective alternative to multi-view or LiDAR-based methods. While deep learning has significantly advanced the capabilities of monocular height estimation, these methods remain fundamentally limited by the availability of labeled data, which are expensive and labor-intensive to obtain at scale. The scarcity of high-quality annotations hinders the generalization and performance of existing models. To overcome this limitation, we propose leveraging large volumes of unlabeled data through a semi-supervised learning framework, enabling the model to extract informative cues from unlabeled samples and improve its predictive performance. In this work, we introduce TSE-Net, a self-training pipeline for semi-supervised monocular height estimation. The pipeline integrates teacher, student, and exam networks. The student network is trained on unlabeled data using pseudo-labels generated by the teacher network, while the exam network functions as a temporal ensemble of the student network to stabilize performance. The teacher network is formulated as a joint regression and classification model: the regression branch predicts height values that serve as pseudo-labels, and the classification branch predicts height value classes along with class probabilities, which are used to filter pseudo-labels. Height value classes are defined using a hierarchical bi-cut strategy to address the inherent long-tailed distribution of heights, and the predicted class probabilities are calibrated with a Plackett-Luce model to reflect the expected accuracy of pseudo-labels. We evaluate the proposed pipeline on three datasets spanning different resolutions and imaging modalities. Codes are available at https://github.com/zhu-xlab/tse-net.

Method: Two-stage optimization: 1) Exploration phase with Adaptive Weighted Stochastic Gradient Langevin Dynamics for global search, 2) Exploitation phase with Local Quasi-Newton Direction-guided Adam optimizer using curvature information.

Result: Extensive experiments show Opt3DGS achieves state-of-the-art rendering quality on diverse benchmark datasets without modifying 3DGS’s underlying representation.

Conclusion: Opt3DGS provides a robust framework that significantly improves 3DGS optimization through enhanced global search and precise convergence, achieving superior rendering performance.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a leading framework for novel view synthesis, yet its core optimization challenges remain underexplored. We identify two key issues in 3DGS optimization: entrapment in suboptimal local optima and insufficient convergence quality. To address these, we propose Opt3DGS, a robust framework that enhances 3DGS through a two-stage optimization process of adaptive exploration and curvature-guided exploitation. In the exploration phase, an Adaptive Weighted Stochastic Gradient Langevin Dynamics (SGLD) method enhances global search to escape local optima. In the exploitation phase, a Local Quasi-Newton Direction-guided Adam optimizer leverages curvature information for precise and efficient convergence. Extensive experiments on diverse benchmark datasets demonstrate that Opt3DGS achieves state-of-the-art rendering quality by refining the 3DGS optimization process without modifying its underlying representation.

Method: NuClass uses two components: Path local (224×224 pixel nuclear morphology) and Path global (1024×1024 pixel neighborhood context), with a learnable gating module and uncertainty-guided objective that directs global path to prioritize uncertain regions. Uses spatial transcriptomics to create marker-guided dataset.

Result: Achieves up to 96% F1 for best-performing class on three fully held-out cohorts, outperforming strong baselines. Provides calibrated confidence estimates and Grad-CAM visualizations.

Conclusion: Multi-scale, uncertainty-aware fusion can bridge the gap between slide-level pathological foundation models and reliable, cell-level phenotype prediction.

Abstract: Identifying cell types and subtypes from routine histopathology images is essential for improving the computational understanding of human disease. Existing tile-based models can capture detailed nuclear morphology but often fail to incorporate the broader tissue context that influences a cell’s function and identity. In addition, available human annotations are typically coarse-grained and unevenly distributed across studies, making fine-grained subtype-level supervision difficult to obtain. To address these limitations, we introduce NuClass, a pathologist workflow inspired framework for cell-wise multi-scale integration of nuclear morphology and microenvironmental context. NuClass includes two main components: Path local, which focuses on nuclear morphology from 224-by-224 pixel crops, and Path global, which models the surrounding 1024-by-1024 pixel neighborhood. A learnable gating module adaptively balances local detail and contextual cues. To encourage complementary learning, we incorporate an uncertainty-guided objective that directs the global path to prioritize regions where the local path is uncertain. We also provide calibrated confidence estimates and Grad-CAM visualizations to enhance interpretability. To overcome the lack of high-quality annotations, we construct a marker-guided dataset from Xenium spatial transcriptomics assays, yielding single-cell resolution labels for more than two million cells across eight organs and 16 classes. Evaluated on three fully held-out cohorts, NuClass achieves up to 96 percent F1 for its best-performing class, outperforming strong baselines. Our results show that multi-scale, uncertainty-aware fusion can bridge the gap between slide-level pathological foundation models and reliable, cell-level phenotype prediction.

Method: ICLR framework with Dual-stream Interaction Enhancement Module (DIEM) for complementary information extraction from fusion and enhancement dimensions, and Covariance Correction Loss (CCL) using luminance residual statistics to penalize chrominance errors and balance gradient conflicts via chrominance covariance constraints.

Result: Experimental results on multiple datasets demonstrate that the proposed ICLR framework outperforms state-of-the-art methods in low-light image enhancement.

Conclusion: The ICLR framework effectively addresses chrominance-luminance interaction challenges in HVI color space for LLIE, achieving superior performance through improved complementary feature extraction and gradient conflict resolution.

Abstract: Low-Light Image Enhancement (LLIE) task aims at improving contrast while restoring details and textures for images captured in low-light conditions. HVI color space has made significant progress in this task by enabling precise decoupling of chrominance and luminance. However, for the interaction of chrominance and luminance branches, substantial distributional differences between the two branches prevalent in natural images limit complementary feature extraction, and luminance errors are propagated to chrominance channels through the nonlinear parameter. Furthermore, for interaction between different chrominance branches, images with large homogeneous-color regions usually exhibit weak correlation between chrominance branches due to concentrated distributions. Traditional pixel-wise losses exploit strong inter-branch correlations for co-optimization, causing gradient conflicts in weakly correlated regions. Therefore, we propose an Inter-Chrominance and Luminance Interaction (ICLR) framework including a Dual-stream Interaction Enhancement Module (DIEM) and a Covariance Correction Loss (CCL). The DIEM improves the extraction of complementary information from two dimensions, fusion and enhancement, respectively. The CCL utilizes luminance residual statistics to penalize chrominance errors and balances gradient conflicts by constraining chrominance branches covariance. Experimental results on multiple datasets show that the proposed ICLR framework outperforms state-of-the-art methods.

Method: Convolutional registration neural networks are used to learn a function that outputs templates conditioned on subject-specific attributes (age, sex), leveraging segmentations when available to produce anatomical segmentation maps for the templates.

Result: The method learns high-quality templates that are representative of populations, and annotated conditional templates enable better registration than unlabeled unconditional templates, outperforming other template construction methods.

Conclusion: The proposed framework efficiently creates population-specific templates that improve registration performance and are more representative than existing templates.

Abstract: Deformable templates, or atlases, are images that represent a prototypical anatomy for a population, and are often enhanced with probabilistic anatomical label maps. They are commonly used in medical image analysis for population studies and computational anatomy tasks such as registration and segmentation. Because developing a template is a computationally expensive process, relatively few templates are available. As a result, analysis is often conducted with sub-optimal templates that are not truly representative of the study population, especially when there are large variations within this population. We propose a machine learning framework that uses convolutional registration neural networks to efficiently learn a function that outputs templates conditioned on subject-specific attributes, such as age and sex. We also leverage segmentations, when available, to produce anatomical segmentation maps for the resulting templates. The learned network can also be used to register subject images to the templates. We demonstrate our method on a compilation of 3D brain MRI datasets, and show that it can learn high-quality templates that are representative of populations. We find that annotated conditional templates enable better registration than their unlabeled unconditional counterparts, and outperform other templates construction methods.

Method: TAND couples a ConvNeXt-based encoder-decoder with a frozen Virchow-2 tissue segmentation branch, using semantic tissue probabilities to selectively modulate the classification stream through novel multi-scale Spatial Feature-wise Linear Modulation (Spatial-FiLM).

Result: On the PUMA benchmark, TAND achieves state-of-the-art performance, surpassing both tissue-agnostic baselines and mask-supervised methods, with remarkable improvements in tissue-dependent cell types such as epithelium, endothelium, and stroma.

Conclusion: This is the first method to condition per-cell classification on learned tissue masks, offering a practical pathway to reduce annotation burden in computational pathology.

Abstract: Accurate nuclei detection and classification are fundamental to computational pathology, yet existing approaches are hindered by reliance on detailed expert annotations and insufficient use of tissue context. We present Tissue-Aware Nuclei Detection (TAND), a novel framework achieving joint nuclei detection and classification using point-level supervision enhanced by tissue mask conditioning. TAND couples a ConvNeXt-based encoder-decoder with a frozen Virchow-2 tissue segmentation branch, where semantic tissue probabilities selectively modulate the classification stream through a novel multi-scale Spatial Feature-wise Linear Modulation (Spatial-FiLM). On the PUMA benchmark, TAND achieves state-of-the-art performance, surpassing both tissue-agnostic baselines and mask-supervised methods. Notably, our approach demonstrates remarkable improvements in tissue-dependent cell types such as epithelium, endothelium, and stroma. To the best of our knowledge, this is the first method to condition per-cell classification on learned tissue masks, offering a practical pathway to reduce annotation burden.

Method: Uses standard webcam with MediaPipe Face Mesh for facial landmark detection and Eye Aspect Ratio (EAR) method to monitor eye closure duration and blink rate for drowsiness detection.

Result: System achieves high accuracy and quick response times in experimental tests, providing a low-cost, high-performance driver monitoring solution.

Conclusion: The system can be effectively integrated into Advanced Driving Assistance Systems (ADAS) to enhance road safety by preventing fatigue-related accidents.

Abstract: One of the major causes of road accidents is driver fatigue that causes thousands of fatalities and injuries every year. This study shows development of a Driver Drowsiness Detection System meant to improve the safety of the road by alerting drivers who are showing signs of being drowsy. The system is based on a standard webcam that tracks the facial features of the driver with the main emphasis on the examination of eye movements that can be conducted with the help of the Eye Aspect Ratio (EAR) method. The Face Mesh by MediaPipe is a lightweight framework that can identify facial landmarks with high accuracy and efficiency, which is considered to be important in real time use. The system detects the moments of long eye shutdowns or a very low rate of blinking which are manifestations of drowsiness and alerts the driver through sound to get her attention back. This system achieves a high-performance and low-cost driver monitoring solution with the help of the computational power of OpenCV to process the image and the MediaPipe to identify faces. Test data experimental analyses indicate that the system is very accurate and responds quicker; this confirms that it can be a component of the current Advanced Driving Assistance System (ADAS).

Method: Pairs Dynamic Token Dropping (prunes tokens via cosine similarity to previous frame) with compressive long-term memory (summarizes keys for retrieval, offloads full KV pairs, rehydrates for generation). Uses consensus-based retrieval of Top-K relevant blocks.

Result: Outperforms current baselines on offline and streaming VQA benchmarks while processing up to 87% less tokens. Enables efficient and context-aware long-form video understanding.

Conclusion: CacheFlow provides a practical solution for long-form video understanding that is drop-in, architecture-agnostic, and requires no fine-tuning, making VLMs both efficient and context-aware.

Abstract: Long-form video question answering (VQA) overwhelms current vision-language models (VLMs) because attention and key-value (KV) caches grow with runtime, forcing either expensive inference or near-sighted sliding windows. We introduce CacheFlow, a training-free pipeline that pairs Dynamic Token Dropping (DTD) with a compressive long-term memory. DTD prunes per-patch tokens online via cosine similarity to the previous frame, and surviving tokens are packed into fixed-size blocks. This online, per-frame processing makes our approach fundamentally suited for live streaming VQA. As blocks are processed, each one’s keys are summarized by a tiny recurrent encoder to form a retrieval index, while the block’s full KV pairs are offloaded and later rehydrated for generation, preserving answer fidelity. At inference, a consensus-based retrieval mechanism retrieves only the Top-K most relevant blocks and attends over both the retrieved and local context for precise, long-range reasoning. CacheFlow is drop-in, architecture-agnostic, and requires no fine-tuning. Experiments on both offline and streaming VQA benchmarks demonstrate that CacheFlow outperforms current strong baselines, while processing up to 87% less tokens. Our dual approach enables VLMs to be both efficient and context-aware, paving the way for practical long-form video understanding.

Method: Uses a dual-encoder architecture pre-trained to disentangle structure from semantics, instruction-tuned on a large-scale part-centric dataset to autoregressively generate structured token sequences encoding part-level information.

Result: Achieves state-of-the-art performance in grounded Q&A, compositional generation, and localized editing through a single unified interface, producing high-quality structured plans.

Conclusion: The approach successfully enables versatile 3D task execution by treating them as programs in a structured grammar, with the structured output driving downstream geometry-aware modules for part-based operations.

Abstract: We introduce Part-X-MLLM, a native 3D multimodal large language model that unifies diverse 3D tasks by formulating them as programs in a structured, executable grammar. Given an RGB point cloud and a natural language prompt, our model autoregressively generates a single, coherent token sequence encoding part-level bounding boxes, semantic descriptions, and edit commands. This structured output serves as a versatile interface to drive downstream geometry-aware modules for part-based generation and editing. By decoupling the symbolic planning from the geometric synthesis, our approach allows any compatible geometry engine to be controlled through a single, language-native frontend. We pre-train a dual-encoder architecture to disentangle structure from semantics and instruction-tune the model on a large-scale, part-centric dataset. Experiments demonstrate that our model excels at producing high-quality, structured plans, enabling state-of-the-art performance in grounded Q&A, compositional generation, and localized editing through one unified interface. Project page: https://chunshi.wang/Part-X-MLLM/

Method: Proposes a VLM-based physical 3D generative model with a new 3D representation that tokenizes geometry efficiently (193x reduction in tokens). Creates PhysX-Mobility dataset with 2K+ real-world objects and rich physical annotations, expanding object categories by 2x.

Result: Strong generative performance and robust generalization demonstrated on PhysX-Mobility and in-the-wild images. Simulation experiments in MuJoCo-style environment validate that generated assets can be directly used for contact-rich robotic policy learning.

Conclusion: PhysX-Anything substantially empowers embodied AI and physics-based simulation applications by providing sim-ready 3D assets with explicit physical properties, bridging the gap between visual representation and physical interaction.

Abstract: 3D modeling is shifting from static visual representations toward physical, articulated assets that can be directly used in simulation and interaction. However, most existing 3D generation methods overlook key physical and articulation properties, thereby limiting their utility in embodied AI. To bridge this gap, we introduce PhysX-Anything, the first simulation-ready physical 3D generative framework that, given a single in-the-wild image, produces high-quality sim-ready 3D assets with explicit geometry, articulation, and physical attributes. Specifically, we propose the first VLM-based physical 3D generative model, along with a new 3D representation that efficiently tokenizes geometry. It reduces the number of tokens by 193x, enabling explicit geometry learning within standard VLM token budgets without introducing any special tokens during fine-tuning and significantly improving generative quality. In addition, to overcome the limited diversity of existing physical 3D datasets, we construct a new dataset, PhysX-Mobility, which expands the object categories in prior physical 3D datasets by over 2x and includes more than 2K common real-world objects with rich physical annotations. Extensive experiments on PhysX-Mobility and in-the-wild images demonstrate that PhysX-Anything delivers strong generative performance and robust generalization. Furthermore, simulation-based experiments in a MuJoCo-style environment validate that our sim-ready assets can be directly used for contact-rich robotic policy learning. We believe PhysX-Anything can substantially empower a broad range of downstream applications, especially in embodied AI and physics-based simulation.

Method: Integrates RL into distillation process using DMD loss as regularization, with dynamic distribution guidance and dynamic renoise sampling strategies.

Result: Achieves leading visual quality and prompt coherence among few-step methods, even exceeding multi-step teacher performance.

Conclusion: DMDR successfully unlocks the capacity of few-step generators through simultaneous distillation and RL, demonstrating superior performance over existing methods.

Abstract: Distribution Matching Distillation (DMD) distills a pre-trained multi-step diffusion model to a few-step one to improve inference efficiency. However, the performance of the latter is often capped by the former. To circumvent this dilemma, we propose DMDR, a novel framework that combines Reinforcement Learning (RL) techniques into the distillation process. We show that for the RL of the few-step generator, the DMD loss itself is a more effective regularization compared to the traditional ones. In turn, RL can help to guide the mode coverage process in DMD more effectively. These allow us to unlock the capacity of the few-step generator by conducting distillation and RL simultaneously. Meanwhile, we design the dynamic distribution guidance and dynamic renoise sampling training strategies to improve the initial distillation process. The experiments demonstrate that DMDR can achieve leading visual quality, prompt coherence among few-step methods, and even exhibit performance that exceeds the multi-step teacher.

Method: Uses novel self-supervised learning formulation, masking strategy, and loss specifically designed for Earth observation data.

Result: Achieves best performance on 15/24 tasks with embeddings and 19/29 tasks with full fine-tuning, outperforming 12 other foundation models.

Conclusion: OlmoEarth provides an effective foundation model for Earth observation and is deployed as an end-to-end platform for non-profits and NGOs, with open-source code and pre-trained weights available.

Abstract: Earth observation data presents a unique challenge: it is spatial like images, sequential like video or text, and highly multimodal. We present OlmoEarth: a multimodal, spatio-temporal foundation model that employs a novel self-supervised learning formulation, masking strategy, and loss all designed for the Earth observation domain. OlmoEarth achieves state-of-the-art performance compared to 12 other foundation models across a variety of research benchmarks and real-world tasks from external partners. When evaluating embeddings OlmoEarth achieves the best performance on 15 out of 24 tasks, and with full fine-tuning it is the best on 19 of 29 tasks. We deploy OlmoEarth as the backbone of an end-to-end platform for data collection, labeling, training, and inference of Earth observation models. The OlmoEarth Platform puts frontier foundation models and powerful data management tools into the hands of non-profits and NGOs working to solve the world’s biggest problems. OlmoEarth source code, training data, and pre-trained weights are available at $\href{https://github.com/allenai/olmoearth_pretrain}{\text{https://github.com/allenai/olmoearth_pretrain}}$.

Method: Uses LVLM to parse text prompts into lighting priors, fuses with geometry/semantic constraints to compute illumination maps, generates initial latent codes for multi-view diffusion model, and fine-tunes 3DGS scene with relit appearance.

Result: Demonstrates consistent improvements over state-of-the-art baselines in multi-view consistency, imaging quality, aesthetic score, and semantic similarity across indoor and outdoor scenes.

Conclusion: GS-Light provides an effective training-free solution for high-fidelity text-guided 3D scene relighting with accurate lighting direction control and improved user expectation alignment.

Abstract: We introduce GS-Light, an efficient, textual position-aware pipeline for text-guided relighting of 3D scenes represented via Gaussian Splatting (3DGS). GS-Light implements a training-free extension of a single-input diffusion model to handle multi-view inputs. Given a user prompt that may specify lighting direction, color, intensity, or reference objects, we employ a large vision-language model (LVLM) to parse the prompt into lighting priors. Using off-the-shelf estimators for geometry and semantics (depth, surface normals, and semantic segmentation), we fuse these lighting priors with view-geometry constraints to compute illumination maps and generate initial latent codes for each view. These meticulously derived init latents guide the diffusion model to generate relighting outputs that more accurately reflect user expectations, especially in terms of lighting direction. By feeding multi-view rendered images, along with the init latents, into our multi-view relighting model, we produce high-fidelity, artistically relit images. Finally, we fine-tune the 3DGS scene with the relit appearance to obtain a fully relit 3D scene. We evaluate GS-Light on both indoor and outdoor scenes, comparing it to state-of-the-art baselines including per-view relighting, video relighting, and scene editing methods. Using quantitative metrics (multi-view consistency, imaging quality, aesthetic score, semantic similarity, etc.) and qualitative assessment (user studies), GS-Light demonstrates consistent improvements over baselines. Code and assets will be made available upon publication.

Method: Proposed TiViBench with 4 reasoning dimensions (structural, spatial, symbolic, action) across 24 tasks and 3 difficulty levels. Also introduced VideoTPO, a test-time strategy using LLM self-analysis on generated candidates to identify strengths/weaknesses for preference optimization.

Result: Commercial models (Sora 2, Veo 3.1) show stronger reasoning potential, while open-source models have untapped potential limited by training scale and data diversity. VideoTPO significantly enhances reasoning performance without additional training, data, or reward models.

Conclusion: TiViBench and VideoTPO provide foundations for evaluating and advancing reasoning in video generation models, setting the stage for future research in this emerging field of reasoning-capable video generation.

Abstract: The rapid evolution of video generative models has shifted their focus from producing visually plausible outputs to tackling tasks requiring physical plausibility and logical consistency. However, despite recent breakthroughs such as Veo 3’s chain-of-frames reasoning, it remains unclear whether these models can exhibit reasoning capabilities similar to large language models (LLMs). Existing benchmarks predominantly evaluate visual fidelity and temporal coherence, failing to capture higher-order reasoning abilities. To bridge this gap, we propose TiViBench, a hierarchical benchmark specifically designed to evaluate the reasoning capabilities of image-to-video (I2V) generation models. TiViBench systematically assesses reasoning across four dimensions: i) Structural Reasoning & Search, ii) Spatial & Visual Pattern Reasoning, iii) Symbolic & Logical Reasoning, and iv) Action Planning & Task Execution, spanning 24 diverse task scenarios across 3 difficulty levels. Through extensive evaluations, we show that commercial models (e.g., Sora 2, Veo 3.1) demonstrate stronger reasoning potential, while open-source models reveal untapped potential that remains hindered by limited training scale and data diversity. To further unlock this potential, we introduce VideoTPO, a simple yet effective test-time strategy inspired by preference optimization. By performing LLM self-analysis on generated candidates to identify strengths and weaknesses, VideoTPO significantly enhances reasoning performance without requiring additional training, data, or reward models. Together, TiViBench and VideoTPO pave the way for evaluating and advancing reasoning in video generation models, setting a foundation for future research in this emerging field.

Method: FFSE models editing as sequences of learned 3D transformations, using a 3DObjectEditor dataset constructed from simulated editing sequences across diverse objects and scenes for training under multi-round dynamic conditions.

Result: Extensive experiments show FFSE significantly outperforms existing methods in both single-round and multi-round 3D-aware editing scenarios, maintaining realistic background effects and global scene consistency.

Conclusion: FFSE provides an effective framework for 3D-aware object manipulation that enables arbitrary transformations while preserving physical realism and scene consistency across multiple editing rounds.

Abstract: Recent advances in text-to-image (T2I) diffusion models have significantly improved semantic image editing, yet most methods fall short in performing 3D-aware object manipulation. In this work, we present FFSE, a 3D-aware autoregressive framework designed to enable intuitive, physically-consistent object editing directly on real-world images. Unlike previous approaches that either operate in image space or require slow and error-prone 3D reconstruction, FFSE models editing as a sequence of learned 3D transformations, allowing users to perform arbitrary manipulations, such as translation, scaling, and rotation, while preserving realistic background effects (e.g., shadows, reflections) and maintaining global scene consistency across multiple editing rounds. To support learning of multi-round 3D-aware object manipulation, we introduce 3DObjectEditor, a hybrid dataset constructed from simulated editing sequences across diverse objects and scenes, enabling effective training under multi-round and dynamic conditions. Extensive experiments show that the proposed FFSE significantly outperforms existing methods in both single-round and multi-round 3D-aware editing scenarios.

Method: Proposes transition mimicking data augmentation (TMA) for cross-shot generalization with single-shot data, and SAAS model that detects and comprehends shot transitions effectively.

Result: SAAS achieves state-of-the-art performance on YouMVOS and Cut-VOS benchmarks by effectively handling complex transitions across shots.

Conclusion: The work enables effective multi-shot VOS through novel data augmentation and model design, supported by new benchmark datasets for future research.

Abstract: This work focuses on multi-shot semi-supervised video object segmentation (MVOS), which aims at segmenting the target object indicated by an initial mask throughout a video with multiple shots. The existing VOS methods mainly focus on single-shot videos and struggle with shot discontinuities, thereby limiting their real-world applicability. We propose a transition mimicking data augmentation strategy (TMA) which enables cross-shot generalization with single-shot data to alleviate the severe annotated multi-shot data sparsity, and the Segment Anything Across Shots (SAAS) model, which can detect and comprehend shot transitions effectively. To support evaluation and future study in MVOS, we introduce Cut-VOS, a new MVOS benchmark with dense mask annotations, diverse object categories, and high-frequency transitions. Extensive experiments on YouMVOS and Cut-VOS demonstrate that the proposed SAAS achieves state-of-the-art performance by effectively mimicking, understanding, and segmenting across complex transitions. The code and datasets are released at https://henghuiding.com/SAAS/.

Method: Use simple large-patch Transformers on pixels that directly predict clean images, without tokenizers, pre-training, or extra losses. Operates with large patch sizes (16, 32) on ImageNet at 256×256 and 512×512 resolutions.

Result: Competitive results on ImageNet at 256×256 and 512×512 resolutions, showing that predicting clean data allows apparently under-capacity networks to work effectively in high-dimensional spaces.

Conclusion: Directly predicting clean data according to the manifold assumption enables effective generative modeling with simple Transformers, providing a self-contained paradigm for Transformer-based diffusion on raw natural data.

Abstract: Today’s denoising diffusion models do not “denoise” in the classical sense, i.e., they do not directly predict clean images. Rather, the neural networks predict noise or a noised quantity. In this paper, we suggest that predicting clean data and predicting noised quantities are fundamentally different. According to the manifold assumption, natural data should lie on a low-dimensional manifold, whereas noised quantities do not. With this assumption, we advocate for models that directly predict clean data, which allows apparently under-capacity networks to operate effectively in very high-dimensional spaces. We show that simple, large-patch Transformers on pixels can be strong generative models: using no tokenizer, no pre-training, and no extra loss. Our approach is conceptually nothing more than “$\textbf{Just image Transformers}$”, or $\textbf{JiT}$, as we call it. We report competitive results using JiT with large patch sizes of 16 and 32 on ImageNet at resolutions of 256 and 512, where predicting high-dimensional noised quantities can fail catastrophically. With our networks mapping back to the basics of the manifold, our research goes back to basics and pursues a self-contained paradigm for Transformer-based diffusion on raw natural data.

Method: Hybrid Multi-Task Learning (HMTL) that augments supervised learning with self-supervised learning objectives (puzzling and inpainting with perceptual loss) during training while keeping inference model unchanged.

Result: Achieved state-of-the-art accuracy on AffectNet in eight-emotion setting without additional pretraining data, with larger gains in low-data regimes. Also improved performance on other fine-grained facial analysis tasks like head pose estimation and gender recognition.

Conclusion: Aligned SSL auxiliaries are an effective and simple way to strengthen supervised fine-grained facial representation without adding extra computation cost during inference time.

Abstract: Facial emotion recognition (FER) is a fine-grained problem where the value of transfer learning is often assumed. We first quantify this assumption and show that, on AffectNet, training from random initialization with sufficiently strong augmentation consistently matches or surpasses fine-tuning from ImageNet. Motivated by this result, we propose Hybrid Multi-Task Learning (HMTL) for FER in the wild. HMTL augments supervised learning (SL) with self-supervised learning (SSL) objectives during training, while keeping the inference-time model unchanged. We instantiate HMTL with two tailored pretext tasks, puzzling and inpainting with a perceptual loss, that encourage part-aware and expression-relevant features. On AffectNet, both HMTL variants achieve state-of-the-art accuracy in the eight-emotion setting without any additional pretraining data, and they provide larger gains under low-data regimes. Compared with conventional SSL pretraining, HMTL yields stronger downstream performance. Beyond FER, the same strategy improves fine-grained facial analysis tasks, including head pose estimation and gender recognition. These results suggest that aligned SSL auxiliaries are an effective and simple way to strengthen supervised fine-grained facial representation without adding extra computation cost during inference time.

Method: Uses hypergraphs for multimodal data representation, Hypergraph Information Bottleneck (HIB) to filter irrelevant information, consistency regularization between HIB and classifier, and cross-modal contrastive loss for unlabeled data utilization.

Result: Extensive experiments on ADNI dataset show HIBMatch surpasses existing state-of-the-art methods in Alzheimer’s disease prognosis.

Conclusion: HIBMatch effectively addresses limitations of current methods by focusing on relevant future-progression information and leveraging unlabeled data through semi-supervised learning.

Abstract: Alzheimer’s disease progression prediction is critical for patients with early Mild Cognitive Impairment (MCI) to enable timely intervention and improve their quality of life. While existing progression prediction techniques demonstrate potential with multimodal data, they are highly limited by their reliance on labelled data and fail to account for a key element of future progression prediction: not all features extracted at the current moment may be relevant for predicting progression several years later. To address these limitations in the literature, we design a novel semi-supervised multimodal learning hypergraph architecture, termed HIBMatch, by harnessing hypergraph knowledge based on information bottleneck and consistency regularisation strategies. Firstly, our framework utilises hypergraphs to represent multimodal data, encompassing both imaging and non-imaging modalities. Secondly, to harmonise relevant information from the currently captured data for future MCI conversion prediction, we propose a Hypergraph Information Bottleneck (HIB) that discriminates against irrelevant information, thereby focusing exclusively on harmonising relevant information for future MCI conversion prediction. Thirdly, our method enforces consistency regularisation between the HIB and a discriminative classifier to enhance the robustness and generalisation capabilities of HIBMatch under both topological and feature perturbations. Finally, to fully exploit the unlabeled data, HIBMatch incorporates a cross-modal contrastive loss for data efficiency. Extensive experiments on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) dataset demonstrate that our proposed HIBMatch framework surpasses existing state-of-the-art methods in Alzheimer’s disease prognosis.

Method: Utilizes a diffusion autoencoder to generate semantically meaningful perturbations for creating adversarial examples against facial recognition systems.

Result: Achieves 24.5% and 25.1% absolute improvements in attack success rates on CelebA-HQ and FFHQ datasets while producing more natural-looking encrypted images than state-of-the-art methods.

Conclusion: Diffusion models can effectively generate high-quality adversarial examples for facial recognition protection, balancing both visual quality and attack performance.

Abstract: The increasingly pervasive facial recognition (FR) systems raise serious concerns about personal privacy, especially for billions of users who have publicly shared their photos on social media. Several attempts have been made to protect individuals from being identified by unauthorized FR systems utilizing adversarial attacks to generate encrypted face images. However, existing methods suffer from poor visual quality or low attack success rates, which limit their utility. Recently, diffusion models have achieved tremendous success in image generation. In this work, we ask: can diffusion models be used to generate adversarial examples to improve both visual quality and attack performance? We propose DiffProtect, which utilizes a diffusion autoencoder to generate semantically meaningful perturbations on FR systems. Extensive experiments demonstrate that DiffProtect produces more natural-looking encrypted images than state-of-the-art methods while achieving significantly higher attack success rates, e.g., 24.5% and 25.1% absolute improvements on the CelebA-HQ and FFHQ datasets.

Method: Created MedIMeta - a standardized meta-dataset containing 19 medical imaging datasets spanning 10 domains and 54 tasks, formatted for direct use in PyTorch and other ML frameworks.

Result: Technical validation showed MedIMeta’s utility through fully supervised and cross-domain few-shot learning baselines, demonstrating its effectiveness as a standardized resource.

Conclusion: MedIMeta provides a comprehensive, standardized solution to overcome data scarcity and preprocessing challenges in medical imaging, enabling more efficient machine learning research and applications.

Abstract: While the field of medical image analysis has undergone a transformative shift with the integration of machine learning techniques, the main challenge of these techniques is often the scarcity of large, diverse, and well-annotated datasets. Medical images vary in format, size, and other parameters and therefore require extensive preprocessing and standardization, for usage in machine learning. Addressing these challenges, we introduce the Medical Imaging Meta-Dataset (MedIMeta), a novel multi-domain, multi-task meta-dataset. MedIMeta contains 19 medical imaging datasets spanning 10 different domains and encompassing 54 distinct medical tasks, all of which are standardized to the same format and readily usable in PyTorch or other ML frameworks. We perform a technical validation of MedIMeta, demonstrating its utility through fully supervised and cross-domain few-shot learning baselines.

Method: Proposes refining CNN-generated lane masks using diffusion models to enhance mask quality before applying segmentation-to-graph algorithms.

Result: Outperforms existing CNN-only and diffusion-only methods, with gains of 1.5% in GEO F1 and 3.5% in TOPO F1 over best CNN method, and 28%/34% improvements over prior diffusion approach.

Conclusion: The diffusion-based refinement approach significantly enhances lane graph quality, particularly improving connectivity metrics, with ablation studies validating the effectiveness of individual components.

Abstract: The lane graph is critical for applications such as autonomous driving and lane-level route planning. While previous research has focused on extracting lane-level graphs from aerial imagery using convolutional neural networks (CNNs) followed by post-processing segmentation-to-graph algorithms, these methods often face challenges in producing sharp and complete segmentation masks. Challenges such as occlusions, variations in lighting, and changes in road texture can lead to incomplete and inaccurate lane masks, resulting in poor-quality lane graphs. To address these challenges, we propose a novel approach that refines the lane masks, output by a CNN, using diffusion models. Experimental results on a publicly available dataset demonstrate that our method outperforms existing methods based solely on CNNs or diffusion models, particularly in terms of graph connectivity. Our lane mask refinement approach enhances the quality of the extracted lane graph, yielding gains of approximately 1.5% in GEO F1 and 3.5% in TOPO F1 scores over the best-performing CNN-based method, and improvements of 28% and 34%, respectively, compared to a prior diffusion-based approach. Both GEO F1 and TOPO F1 scores are critical metrics for evaluating lane graph quality. Additionally, ablation studies are conducted to evaluate the individual components of our approach, providing insights into their respective contributions and effectiveness.

Method: Leverages pretrained NVS models for weak guidance, integrates this into 3D-free view synthesis approach, and enriches CLIP vision-language space with 3D camera angle information.

Result: Outperforms existing models in qualitative and quantitative evaluations, achieving high-fidelity, consistent novel view synthesis at desired camera angles across diverse scenes while maintaining image clarity.

Conclusion: The method successfully combines 3D-free and 3D-based approaches to enable camera-controlled novel view synthesis from single images without extensive 3D training data, demonstrating superior performance across various scenes.

Abstract: Recent 3D novel view synthesis (NVS) methods often require extensive 3D data for training, and also typically lack generalization beyond the training distribution. Moreover, they tend to be object centric and struggle with complex and intricate scenes. Conversely, 3D-free methods can generate text-controlled views of complex, in-the-wild scenes using a pretrained stable diffusion model without the need for a large amount of 3D-based training data, but lack camera control. In this paper, we introduce a method capable of generating camera-controlled viewpoints from a single input image, by combining the benefits of 3D-free and 3D-based approaches. Our method excels in handling complex and diverse scenes without extensive training or additional 3D and multiview data. It leverages widely available pretrained NVS models for weak guidance, integrating this knowledge into a 3D-free view synthesis style approach, along with enriching the CLIP vision-language space with 3D camera angle information, to achieve the desired results. Experimental results demonstrate that our method outperforms existing models in both qualitative and quantitative evaluations, achieving high-fidelity, consistent novel view synthesis at desired camera angles across a wide variety of scenes while maintaining accurate, natural detail representation and image clarity across various viewpoints. We also support our method with a comprehensive analysis of 2D image generation models and the 3D space, providing a solid foundation and rationale for our solution.

Method: Developed BadVim framework that uses low-rank perturbations on state-wise transitions during training to create backdoors, requiring only minimal data poisoning (0.3% of training data).

Result: Achieved over 97% attack success rate across three datasets, bypassing state-of-the-art defenses. VSSMs showed comparable backdoor robustness to Vision Transformers and superior robustness to CNNs.

Conclusion: The state-space representation that enhances VSSM capability also contributes to vulnerability against backdoor attacks, highlighting the trade-off between performance and robustness in model design.

Abstract: Visual State Space Models (VSSM) have shown remarkable performance in various computer vision tasks. However, backdoor attacks pose significant security challenges, causing compromised models to predict target labels when specific triggers are present while maintaining normal behavior on benign samples. In this paper, we investigate the robustness of VSSMs against backdoor attacks. Specifically, we delicately design a novel framework for VSSMs, dubbed BadVim, which utilizes low-rank perturbations on state-wise to uncover their impact on state transitions during training. By poisoning only $0.3%$ of the training data, our attacks cause any trigger-embedded input to be misclassified to the targeted class with a high attack success rate (over 97%) at inference time. Our findings suggest that the state-space representation property of VSSMs, which enhances model capability, may also contribute to its vulnerability to backdoor attacks. Our attack exhibits effectiveness across three datasets, even bypassing state-of-the-art defenses against such attacks. Extensive experiments show that the backdoor robustness of VSSMs is comparable to that of Transformers (ViTs) and superior to that of Convolutional Neural Networks (CNNs). We believe our findings will prompt the community to reconsider the trade-offs between performance and robustness in model design.

Method: Used RISE method as baseline to evaluate combinations of mask sampling, segmentation techniques, smoothing, attribution calculation, and per-segment vs per-pixel attribution using a proxy metric.

Result: Attribution calculation has little impact on results, while segmentation and per-pixel attribution (rarely examined parameters) have significant impact.

Conclusion: Future work on perturbation-based explanation methods should focus more on segmentation and per-pixel attribution rather than attribution calculation.

Abstract: Perturbation-based post-hoc image explanation methods are commonly used to explain image prediction models. These methods perturb parts of the input to measure how those parts affect the output. Since the methods only require the input and output, they can be applied to any model, making them a popular choice to explain black-box models. While many different methods exist and have been compared with one another, it remains poorly understood which parameters of the different methods are responsible for their varying performance. This work uses the Randomized Input Sampling for Explanations (RISE) method as a baseline to evaluate many combinations of mask sampling, segmentation techniques, smoothing, attribution calculation, and per-segment or per-pixel attribution, using a proxy metric. The results show that attribution calculation, which is frequently the focus of other works, has little impact on the results. Conversely, segmentation and per-pixel attribution, rarely examined parameters, have a significant impact. The implementation of and data gathered in this work are available online: https://github.com/guspih/post-hoc-image-perturbation and https://bit.ly/smooth-mask-perturbation.

Method: Uses Low-Rank Adaptation (LoRA) to introduce trainable low-rank parameters into frozen LDMs, preserving original weights. Includes dynamic loss weight scheduler to balance generative quality and watermark fidelity.

Result: Method ensures fast and accurate watermark embedding with high-quality generated images, maintaining robustness comparable or superior to state-of-the-art approaches. Generalizes well across datasets and base LDMs.

Conclusion: The proposed EW-LoRA method provides an efficient and effective watermarking solution for large diffusion models, balancing computational efficiency with robust watermark protection.

Abstract: The rapid proliferation of Deep Neural Networks (DNNs) is driving a surge in model watermarking technologies, as the trained models themselves constitute valuable intellectual property. Existing watermarking approaches primarily focus on modifying model parameters or altering sampling behaviors. However, with the emergence of increasingly large models, improving the efficiency of watermark embedding becomes essential to manage increasing computational demands. Prioritizing efficiency not only optimizes resource utilization, making the watermarking process more applicable for large models, but also mitigates potential degradation of model performance. In this paper, we propose an efficient watermarking method for Latent Diffusion Models (LDMs) based on Low-Rank Adaptation (LoRA). The core idea is to introduce trainable low-rank parameters into the frozen LDM to embed watermark, thereby preserving the integrity of the original model weights. Furthermore, a dynamic loss weight scheduler is designed to adaptively balance the objectives of generative quality and watermark fidelity, enabling the model to achieve effective watermark embedding with minimal impact on quality of the generated images. Experimental results show that the proposed method ensures fast and accurate watermark embedding and a high quality of the generated images, at the same time maintaining a level of robustness aligned - in some cases superior - with state-of-the-art approaches. Moreover, the method generalizes well across different datasets and base LDMs. Codes are available at: https://github.com/MrDongdongLin/EW-LoRA.

Method: Systematic analysis of network components (topology, convolutions, activation functions), proposal of two optimized convolutional operations, and development of LT-DARTS - a NAS method with novel search space and strategy for LT data.

Result: The approach consistently outperforms existing architectures across multiple LT datasets, achieving parameter-efficient state-of-the-art results when integrated with current LT methods.

Conclusion: Neural architecture design is crucial for Long-Tailed recognition, and the proposed LT-DARTS method effectively addresses LT challenges through optimized operations and NAS-based exploration.

Abstract: Long-Tailed (LT) recognition has been widely studied to tackle the challenge of imbalanced data distributions in real-world applications. However, the design of neural architectures for LT settings has received limited attention, despite evidence showing that architecture choices can substantially affect performance. This paper aims to bridge the gap between LT challenges and neural network design by providing an in-depth analysis of how various architectures influence LT performance. Specifically, we systematically examine the effects of key network components on LT handling, such as topology, convolutions, and activation functions. Based on these observations, we propose two convolutional operations optimized for improved performance. Recognizing that operation interactions are also crucial to network effectiveness, we apply Neural Architecture Search (NAS) to facilitate efficient exploration. We propose LT-DARTS, a NAS method with a novel search space and search strategy specifically designed for LT data. Experimental results demonstrate that our approach consistently outperforms existing architectures across multiple LT datasets, achieving parameter-efficient, state-of-the-art results when integrated with current LT methods.

Method: Proposes using semantic segmentation models (UNet, UNet++, DeepLabV3+, SwinUNet) on 2D representations of multi-channel seismic data with minimal preprocessing, enabling end-to-end simultaneous detection and classification of five seismic event classes.

Result: Evaluated on ~25,000 events from four Chilean volcanoes, UNet performed best with mean F1 score of 0.91 and IoU of 0.88, showing superior noise robustness and generalization to unseen volcano datasets.

Conclusion: The semantic segmentation approach provides an effective, data-driven solution for automated seismic event recognition that integrates multi-station data with minimal preprocessing, achieving high performance and demonstrating strong generalization capabilities across different volcanic environments.

Abstract: In volcano monitoring, effective recognition of seismic events is essential for understanding volcanic activity and raising timely warning alerts. Traditional methods rely on manual analysis, which can be subjective and labor-intensive. Furthermore, current automatic approaches often tackle detection and classification separately, mostly rely on single station information and generally require tailored preprocessing and representations to perform predictions. These limitations often hinder their application to real-time monitoring and utilization across different volcano conditions. This study introduces a novel approach that utilizes Semantic Segmentation models to automate seismic event recognition by applying a straight forward transformation of multi-channel 1D signals into 2D representations, enabling their use as images. Our framework employs a data-driven, end-to-end design that integrates multi-station seismic data with minimal preprocessing, performing both detection and classification simultaneously for five seismic event classes. We evaluated four state-of-the-art segmentation models (UNet, UNet++, DeepLabV3+ and SwinUNet) on approximately 25.000 seismic events recorded at four different Chilean volcanoes: Nevados del Chillán Volcanic Complex, Laguna del Maule, Villarrica and Puyehue-Cordón Caulle. Among these models, the UNet architecture was identified as the most effective model, achieving mean F1 and Intersection over Union (IoU) scores of up to 0.91 and 0.88, respectively, and demonstrating superior noise robustness and model flexibility to unseen volcano datasets.

Method: Proposes MonoASRH with two key components: Efficient Hybrid Feature Aggregation Module (EH-FAM) using multi-head attention for global semantic features and lightweight convolution for cross-scale aggregation, and Adaptive Scale-Aware 3D Regression Head (ASRH) that fuses scale features with semantic features and learns dynamic receptive field offsets.

Result: Extensive experiments on KITTI and Waymo datasets demonstrate that MonoASRH achieves state-of-the-art performance in monocular 3D object detection.

Conclusion: The proposed framework effectively addresses scale variation issues and improves detection performance by combining global semantic awareness with adaptive scale-aware regression.

Abstract: Monocular 3D object detection has attracted great attention due to simplicity and low cost. Existing methods typically follow conventional 2D detection paradigms, first locating object centers and then predicting 3D attributes via neighboring features. However, these methods predominantly rely on progressive cross-scale feature aggregation and focus solely on local information, which may result in a lack of global awareness and the omission of small-scale objects. In addition, due to large variation in object scales across different scenes and depths, inaccurate receptive fields often lead to background noise and degraded feature representation. To address these issues, we introduces MonoASRH, a novel monocular 3D detection framework composed of Efficient Hybrid Feature Aggregation Module (EH-FAM) and Adaptive Scale-Aware 3D Regression Head (ASRH). Specifically, EH-FAM employs multi-head attention with a global receptive field to extract semantic features for small-scale objects and leverages lightweight convolutional modules to efficiently aggregate visual features across different scales. The ASRH encodes 2D bounding box dimensions and then fuses scale features with the semantic features aggregated by EH-FAM through a scale-semantic feature fusion module. The scale-semantic feature fusion module guides ASRH in learning dynamic receptive field offsets, incorporating scale priors into 3D position prediction for better scale-awareness. Extensive experiments on the KITTI and Waymo datasets demonstrate that MonoASRH achieves state-of-the-art performance.

Method: Introduces Cosine-similarity Preserving Compression (CosPress) - a feature distillation technique that learns a mapping to compress teacher’s latent space into student’s smaller space while preserving cosine similarities between image embeddings.

Result: Produces more accurate models with better performance on generalizability, robustness and OOD detection benchmarks across various datasets including ImageNet, and enables training highly performant lightweight models on small datasets.

Conclusion: CosPress provides a competitive pathway for compressing foundation models while faithfully reproducing teacher properties, outperforming existing distillation methods in critical areas like robustness and OOD detection.

Abstract: Knowledge distillation methods compress models by training a student network using the classification outputs of a high quality teacher model, but can fail to effectively transfer the properties of computer vision foundation models from the teacher to the student. While it has been recently shown that feature distillation$\unicode{x2013}$where a teacher model’s output features are replicated instead$\unicode{x2013}$can reproduce performance for foundation models across numerous downstream tasks, they fall short in matching critical properties such as robustness and out-of-distribution (OOD) detection performance. This paper overcomes this shortcoming by introducing Cosine-similarity Preserving Compression (CosPress), a feature distillation technique that learns a mapping to compress the latent space of the teacher model into the smaller latent space of the student, by preserving the cosine similarities between image embeddings. This enables direct optimisation of the student network and produces a more faithful reproduction of the teacher’s properties. It is shown that distillation with CosPress on a variety of datasets, including ImageNet, produces more accurate models with greater performance on generalisability, robustness and OOD detection benchmarks, and that this technique provides a competitive pathway for training highly performant lightweight models on small datasets. Code is available at github.com/emannix/cospress.

Method: Filter-correlate-compress framework with FiCoCo-V (training-free vision encoder optimization using redundancy-based token discard and correlation-based information recycling) and FiCoCo-L (task-aware textual priors in LLM decoder).

Result: Achieves up to 14.7x FLOPs reduction with 93.6% performance retention, consistently outperforms state-of-the-art training-free approaches across various model architectures, sizes, and tasks.

Conclusion: FiCoCo series effectively accelerates MLLMs while maintaining performance, demonstrating effectiveness and generalizability without requiring retraining.

Abstract: The quadratic complexity of Multimodal Large Language Models (MLLMs) with respect to context length poses significant computational and memory challenges, hindering their real-world deployment. In the paper, we devise a ‘‘filter-correlate-compress’’ framework to accelerate the MLLM by systematically optimizing multimodal context length during prefilling. The framework first implements FiCoCo-V, a training-free method operating within the vision encoder. It employs a redundancy-based token discard mechanism that uses a novel integrated metric to accurately filter out redundant visual tokens. To mitigate information loss, the framework introduces a correlation-based information recycling mechanism that allows preserved tokens to selectively recycle information from correlated discarded tokens with a self-preserving compression, thereby preventing the dilution of their own core content. The framework’s FiCoCo-L variant further leverages task-aware textual priors to perform token reduction directly within the LLM decoder. Extensive experiments demonstrate that the FiCoCo series effectively accelerates a range of MLLMs, achieves up to 14.7x FLOPs reduction with 93.6% performance retention. Our methods consistently outperform state-of-the-art training-free approaches, showcasing effectiveness and generalizability across model architectures, sizes, and tasks without requiring retraining. Code: https://github.com/kawhiiiileo/FiCoCo

Method: Proposes a network that divides different sized windows for local areas with different densities to compute attention, treats local areas as tokens for global attention, and introduces category-response loss to balance different categories and object sizes.

Result: Achieves competitive results in semantic segmentation and part segmentation tasks on multiple public datasets, and demonstrates strong segmentation capability for small objects and small sample categories in complex real-world scenes.

Conclusion: The proposed density-aware local attention fusion with global attention effectively handles small objects and small sample categories in point cloud segmentation, providing improved performance in real-world applications.

Abstract: 3D point cloud segmentation has a wide range of applications in areas such as autonomous driving, augmented reality, virtual reality and digital twins. The point cloud data collected in real scenes often contain small objects and categories with small sample sizes, which are difficult to handle by existing networks. In this regard, we propose a point cloud segmentation network that fuses local attention based on density perception with global attention. The core idea is to increase the effective receptive field of each point while reducing the loss of information about small objects in dense areas. Specifically, we divide different sized windows for local areas with different densities to compute attention within the window. Furthermore, we consider each local area as an independent token for the global attention of the entire input. A category-response loss is also proposed to balance the processing of different categories and sizes of objects. In particular, we set up an additional fully connected layer in the middle of the network for prediction of the presence of object categories, and construct a binary cross-entropy loss to respond to the presence of categories in the scene. In experiments, our method achieves competitive results in semantic segmentation and part segmentation tasks on several publicly available datasets. Experiments on point cloud data obtained from complex real-world scenes filled with tiny objects also validate the strong segmentation capability of our method for small objects as well as small sample categories.

Method: Integrates frequency-spatial fusion image encoder, cross-modal alignment training with hard/semi-hard negative mining, K-Means clustering with HNSW indexing for fast retrieval, multimodal chain-of-thought prompting, and continual learning mechanism.

Result: Achieves highest score of 74.46 on CODA-LM benchmark and shows consistent performance gains when integrated with end-to-end frameworks like DriveLM, significantly improving corner case comprehension across multiple downstream tasks.

Conclusion: Demonstrates effectiveness of retrieval-augmented strategies and cross-modal alignment for safer and more interpretable autonomous driving.

Abstract: Understanding and addressing corner cases is essential for ensuring the safety and reliability of autonomous driving systems. Vision-language models (VLMs) play a crucial role in enhancing scenario comprehension, yet they face significant challenges, such as hallucination and insufficient real-world grounding, which compromise their performance in critical driving scenarios. In this work, RAC3, a novel framework designed to enhance the performance of VLMs in corner case comprehension, is proposed. RAC3 integrates a frequency-spatial fusion (FSF) image encoder, a cross-modal alignment training method for embedding models with hard and semi-hard negative mining, and a fast querying and retrieval pipeline based on K-Means clustering and hierarchical navigable small world (HNSW) indexing. A multimodal chain-of-thought (CoT) prompting strategy to guide analogical reasoning and reduce hallucinations during inference is introduced. Moreover, an update mechanism is integrated into RAC3 to ensure continual learning within the framework. Extensive experiments on the CODA and nuScenes datasets demonstrate that RAC3 significantly improves corner case comprehension across multiple downstream tasks. Compared to prior state-of-the-art methods, RAC3 achieves the highest final score of 74.46 on the CODA-LM benchmark and shows consistent performance gains when integrated with end-to-end frameworks like DriveLM. These results demonstrate the effectiveness of retrieval-augmented strategies and cross-modal alignment for safer and more interpretable autonomous driving.

Method: Processes multi-camera 360-degree imagery to generate BEV features, refined through transformer decoder with Bezier Deformable Attention using Bezier control points, plus auxiliary instance mask loss and one-to-many set prediction loss.

Result: Outperforms existing methods on OpenLane-V2 dataset for centerline detection and topology reasoning, and achieves best results on OpenLane-V1 for 3D lane detection. Multimodal inputs further enhance performance.

Conclusion: TopoBDA effectively improves road topology comprehension through Bezier Deformable Attention and auxiliary components, with potential for further enhancement through multimodal data integration.

Abstract: Understanding road topology is crucial for autonomous driving. This paper introduces TopoBDA (Topology with Bezier Deformable Attention), a novel approach that enhances road topology comprehension by leveraging Bezier Deformable Attention (BDA). TopoBDA processes multi-camera 360-degree imagery to generate Bird’s Eye View (BEV) features, which are refined through a transformer decoder employing BDA. BDA utilizes Bezier control points to drive the deformable attention mechanism, improving the detection and representation of elongated and thin polyline structures, such as lane centerlines. Additionally, TopoBDA integrates two auxiliary components: an instance mask formulation loss and a one-to-many set prediction loss strategy, to further refine centerline detection and enhance road topology understanding. Experimental evaluations on the OpenLane-V2 dataset demonstrate that TopoBDA outperforms existing methods, achieving state-of-the-art results in centerline detection and topology reasoning. TopoBDA also achieves the best results on the OpenLane-V1 dataset in 3D lane detection. Further experiments on integrating multi-modal data – such as LiDAR, radar, and SDMap – show that multimodal inputs can further enhance performance in road topology understanding.

Method: I2V-MLLM attack uses an image-based MLLM as surrogate to craft adversarial videos. It integrates multimodal interactions and spatiotemporal information to disrupt video representations, and employs perturbation propagation to handle different frame sampling strategies.

Result: The method achieves strong transferability across different V-MLLMs on multiple video-text tasks. Black-box attacks using BLIP-2 as surrogate achieve average attack success rates of 57.98% on MSVD-QA and 58.26% on MSRVTT-QA for Zero-Shot VideoQA tasks.

Conclusion: I2V-MLLM effectively addresses limitations of existing adversarial attacks for V-MLLMs in black-box scenarios, demonstrating competitive performance compared to white-box attacks and highlighting vulnerabilities in video-text multimodal systems.

Abstract: Video-based multimodal large language models (V-MLLMs) have shown vulnerability to adversarial examples in video-text multimodal tasks. However, the transferability of adversarial videos to unseen models - a common and practical real-world scenario - remains unexplored. In this paper, we pioneer an investigation into the transferability of adversarial video samples across V-MLLMs. We find that existing adversarial attack methods face significant limitations when applied in black-box settings for V-MLLMs, which we attribute to the following shortcomings: (1) lacking generalization in perturbing video features, (2) focusing only on sparse key-frames, and (3) failing to integrate multimodal information. To address these limitations and deepen the understanding of V-MLLM vulnerabilities in black-box scenarios, we introduce the Image-to-Video MLLM (I2V-MLLM) attack. In I2V-MLLM, we utilize an image-based multimodal large language model (I-MLLM) as a surrogate model to craft adversarial video samples. Multimodal interactions and spatiotemporal information are integrated to disrupt video representations within the latent space, improving adversarial transferability. Additionally, a perturbation propagation technique is introduced to handle different unknown frame sampling strategies. Experimental results demonstrate that our method can generate adversarial examples that exhibit strong transferability across different V-MLLMs on multiple video-text multimodal tasks. Compared to white-box attacks on these models, our black-box attacks (using BLIP-2 as a surrogate model) achieve competitive performance, with average attack success rate (AASR) of 57.98% on MSVD-QA and 58.26% on MSRVTT-QA for Zero-Shot VideoQA tasks, respectively.

Method: Learn explicit Gaussian representation using atmospheric scattering model, establish transmission function directly on Gaussian primitives via depth-to-transmission mapping, jointly learn atmospheric light and scattering coefficients during training, and remove scattering effects at inference.

Result: Achieves state-of-the-art performance on both real-world and synthetic foggy datasets, with improved computational efficiency compared to NeRF-based approaches.

Conclusion: Explicit Gaussian representation with physical forward rendering effectively handles fog removal from multi-view images while recovering fine details and maintaining computational efficiency.

Abstract: Current novel view synthesis methods are typically designed for high-quality and clean input images. However, in foggy scenes, scattering and attenuation can significantly degrade the quality of rendering. Although NeRF-based dehazing approaches have been developed, their reliance on deep fully connected neural networks and per-ray sampling strategies leads to high computational costs. Furthermore, NeRF’s implicit representation limits its ability to recover fine-grained details from hazy scenes. To overcome these limitations, we propose learning an explicit Gaussian representation to explain the formation mechanism of foggy images through a physically forward rendering process. Our method, DehazeGS, reconstructs and renders fog-free scenes using only multi-view foggy images as input. Specifically, based on the atmospheric scattering model, we simulate the formation of fog by establishing the transmission function directly onto Gaussian primitives via depth-to-transmission mapping. During training, we jointly learn the atmospheric light and scattering coefficients while optimizing the Gaussian representation of foggy scenes. At inference time, we remove the effects of scattering and attenuation in Gaussian distributions and directly render the scene to obtain dehazed views. Experiments on both real-world and synthetic foggy datasets demonstrate that DehazeGS achieves state-of-the-art performance. visualizations are available at https://dehazegs.github.io/

Method: GUSLO uses two coordinated innovations: (1) single-shot calibration via 2D triangulation-based interpolation to convert sparse matches into dense correspondence fields, and (2) artifact-aware photometric adaptation via explicit transfer functions to balance generalization and color fidelity.

Result: Experiments across binary, speckle, and color-coded settings show GUSLO consistently improves accuracy and cross-encoding robustness over conventional methods in challenging industrial and cultural scenarios.

Conclusion: GUSLO provides a general and unified solution for structured light 3D reconstruction that overcomes the limitations of existing methods by eliminating scene-specific calibration requirements and working effectively across different SL patterns.

Abstract: Structured light (SL) 3D reconstruction captures the precise surface shape of objects, providing high-accuracy 3D data essential for industrial inspection and cultural heritage digitization. However, existing methods suffer from two key limitations: reliance on scene-specific calibration with manual parameter tuning, and optimization frameworks tailored to specific SL patterns, limiting their generalizability across varied scenarios. We propose General and Unified Structured Light Optimization (GUSLO), a novel framework addressing these issues through two coordinated innovations: (1) single-shot calibration via 2D triangulation-based interpolation that converts sparse matches into dense correspondence fields, and (2) artifact-aware photometric adaptation via explicit transfer functions, balancing generalization and color fidelity. We conduct diverse experiments covering binary, speckle, and color-coded settings. Results show that GUSLO consistently improves accuracy and cross-encoding robustness over conventional methods in challenging industrial and cultural scenarios.

Method: Created SVBench with 49,979 QA pairs from 1,353 streaming videos using a semi-automated annotation pipeline that generates QA chains representing consecutive multi-turn dialogues over video segments and constructs temporal linkages between successive chains.

Result: GPT-4o outperforms others, but most open-source LVLMs struggle with long-context streaming video understanding. The authors’ StreamingChat model significantly outperforms open-source LVLMs on SVBench while maintaining comparable performance on other vision-language benchmarks.

Conclusion: SVBench advances streaming video understanding research by providing comprehensive analysis of current LVLMs’ capabilities, highlighting the need for improved temporal reasoning in long-context video streams.

Abstract: Despite the significant advancements of Large Vision-Language Models (LVLMs) on established benchmarks, there remains a notable gap in suitable evaluation regarding their applicability in the emerging domain of long-context streaming video understanding. Current benchmarks for video understanding typically emphasize isolated single-instance text inputs and fail to evaluate the capacity to sustain temporal reasoning throughout the entire duration of video streams. To address these limitations, we introduce SVBench, a pioneering benchmark with temporal multi-turn question-answering chains specifically designed to thoroughly assess the capabilities of streaming video understanding of current LVLMs. We design a semi-automated annotation pipeline to obtain 49,979 Question-Answer (QA) pairs of 1,353 streaming videos, which includes generating QA chains that represent a series of consecutive multi-turn dialogues over video segments and constructing temporal linkages between successive QA chains. Our experimental results, obtained from 14 models in dialogue and streaming evaluations, reveal that while the closed-source GPT-4o outperforms others, most open-source LVLMs struggle with long-context streaming video understanding. We also construct a StreamingChat model, which significantly outperforms open-source LVLMs on our SVBench and achieves comparable performance on diverse vision-language benchmarks. We expect SVBench to advance the research of streaming video understanding by providing a comprehensive and in-depth analysis of current LVLMs. Our benchmark and model can be accessed at https://github.com/sotayang/SVBench.

Method: Post-hoc normalizing flow-based approach that seamlessly integrates with existing pre-trained models without altering their weights, evaluated on MedOOD and MedMNIST datasets.

Result: Achieved 84.61% AUROC on MedOOD (outperforming ViM and MDS) and 93.8% AUROC on MedMNIST (surpassing ViM and ReAct), demonstrating strong OOD detection performance.

Conclusion: The method provides a practical and effective safeguard for clinical imaging workflows due to its strong performance and post-hoc integration capability without model modifications.

Abstract: In AI-driven medical imaging, the failure to detect out-of-distribution (OOD) data poses a severe risk to clinical reliability, potentially leading to critical diagnostic errors. Current OOD detection methods often demand impractical retraining or modifications to pre-trained models, hindering their adoption in regulated clinical environments. To address this challenge, we propose a post-hoc normalizing flow-based approach that seamlessly integrates with existing pre-trained models without altering their weights. We evaluate the approach on our in-house-curated MedOOD dataset, designed to capture clinically relevant distribution shifts, and on the MedMNIST benchmark. The proposed method achieves an AUROC of 84.61% on MedOOD, outperforming ViM (80.65%) and MDS (80.87%), and reaches 93.8% AUROC on MedMNIST, surpassing ViM (88.08%) and ReAct (87.05%). This combination of strong performance and post-hoc integration capability makes our approach a practical and effective safeguard for clinical imaging workflows. The model and code to build OOD datasets are publicly accessible at https://github.com/dlotfi/MedOODFlow.

Method: Proposes RTGen with a succinct encoder-decoder architecture featuring a Region-Language Decoder that organizes textual side as a Directed Acyclic Graph for non-autoregressive category naming.

Result: RTGen-R34 achieves 131.3 FPS on T4 GPUs (270x faster than GenerateU) and learns to generate category names directly from detection labels without external supervision.

Conclusion: RTGen enables efficient and flexible open-ended detection with real-time performance while eliminating dependency on external models like CLIP or pretrained language models.

Abstract: Although open-vocabulary object detectors can generalize to unseen categories, they still rely on predefined textual prompts or classifier heads during inference. Recent generative object detectors address this limitation by coupling an autoregressive language model with a detector backbone, enabling direct category name generation for each detected object. However, this straightforward design introduces structural redundancy and substantial latency. In this paper, we propose a Real-Time Generative Detection Transformer (RTGen), a real-time generative object detector with a succinct encoder-decoder architecture. Specifically, we introduce a novel Region-Language Decoder (RL-Decoder) that jointly decodes visual and textual representations within a unified framework. The textual side is organized as a Directed Acyclic Graph (DAG), enabling non-autoregressive category naming. Benefiting from these designs, RTGen-R34 achieves 131.3 FPS on T4 GPUs, over 270x faster than GenerateU. Moreover, our models learn to generate category names directly from detection labels, without relying on external supervision such as CLIP or pretrained language models, achieving efficient and flexible open-ended detection.

Method: Created Open-Insect dataset for fine-grained unknown species detection across geographic regions, benchmarked 38 OSR algorithms in three categories: post-hoc, training-time regularization, and training with auxiliary data.

Result: Simple post-hoc approaches remain strong baselines, auxiliary data can improve species discovery in data-limited regions.

Conclusion: Provides insights for developing computer vision methods for biodiversity monitoring and species discovery, addressing the open-set recognition challenge in highly diverse taxa.

Abstract: Global biodiversity is declining at an unprecedented rate, yet little information is known about most species and how their populations are changing. Indeed, some 90% of Earth’s species are estimated to be completely unknown. Machine learning has recently emerged as a promising tool to facilitate long-term, large-scale biodiversity monitoring, including algorithms for fine-grained classification of species from images. However, such algorithms typically are not designed to detect examples from categories unseen during training – the problem of open-set recognition (OSR) – limiting their applicability for highly diverse, poorly studied taxa such as insects. To address this gap, we introduce Open-Insect, a large-scale, fine-grained dataset to evaluate unknown species detection across different geographic regions with varying difficulty. We benchmark 38 OSR algorithms across three categories: post-hoc, training-time regularization, and training with auxiliary data, finding that simple post-hoc approaches remain a strong baseline. We also demonstrate how to leverage auxiliary data to improve species discovery in regions with limited data. Our results provide insights to guide the development of computer vision methods for biodiversity monitoring and species discovery.

Method: Three novel designs: 1) bidirectional temporal correlation transforming dense motion cues, 2) adaptive temporal sampling for consistency, 3) spatially adaptive temporal motion aggregation to efficiently aggregate consistent motion features.

Result: Ranked 1st on DSEC-Flow benchmark, outperforming state-of-the-art methods by large margin with sharp edges and high-quality details. Accurately predicts future optical flow using only past events.

Conclusion: BAT framework effectively addresses spatial sparsity in event data and enables superior optical flow estimation with future prediction capabilities, significantly advancing event-based vision methods.

Abstract: Event cameras deliver visual information characterized by a high dynamic range and high temporal resolution, offering significant advantages in estimating optical flow for complex lighting conditions and fast-moving objects. Current advanced optical flow methods for event cameras largely adopt established image-based frameworks. However, the spatial sparsity of event data limits their performance. In this paper, we present BAT, an innovative framework that estimates event-based optical flow using bidirectional adaptive temporal correlation. BAT includes three novel designs: 1) a bidirectional temporal correlation that transforms bidirectional temporally dense motion cues into spatially dense ones, enabling accurate and spatially dense optical flow estimation; 2) an adaptive temporal sampling strategy for maintaining temporal consistency in correlation; 3) spatially adaptive temporal motion aggregation to efficiently and adaptively aggregate consistent target motion features into adjacent motion features while suppressing inconsistent ones. Our results rank $1^{st}$ on the DSEC-Flow benchmark, outperforming existing state-of-the-art methods by a large margin while also exhibiting sharp edges and high-quality details. Notably, our BAT can accurately predict future optical flow using only past events, significantly outperforming E-RAFT’s warm-start approach. Code: \textcolor{magenta}{https://github.com/gangweiX/BAT}.

Method: Proposed S4M with structured 4-point prompts (extreme points or major/minor axis endpoints), role-specific embeddings, and Canvas pretext task for geometry-aware reasoning.

Result: +3.42 mIoU improvement over SAM baseline across 8 ultrasound/surgical endoscopy datasets; major/minor prompts enable faster clinician annotation.

Conclusion: S4M increases performance, reduces annotation effort, and aligns with clinical practice, enabling more scalable medical imaging dataset development.

Abstract: Purpose: The Segment Anything Model (SAM) promises to ease the annotation bottleneck in medical segmentation, but overlapping anatomy and blurred boundaries make its point prompts ambiguous, leading to cycles of manual refinement to achieve precise masks. Better prompting strategies are needed. Methods: We propose a structured prompting strategy using 4 points as a compact instance-level shape description. We study two 4-point variants: extreme points and the proposed major/minor axis endpoints, inspired by ultrasound measurement practice. SAM cannot fully exploit such structured prompts because it treats all points identically and lacks geometry-aware reasoning. To address this, we introduce S4M (4-points to Segment Anything), which augments SAM to interpret 4 points as relational cues rather than isolated clicks. S4M expands the prompt space with role-specific embeddings and adds an auxiliary “Canvas” pretext task that sketches coarse masks directly from prompts, fostering geometry-aware reasoning. Results: Across eight datasets in ultrasound and surgical endoscopy, S4M improves segmentation by +3.42 mIoU over a strong SAM baseline at equal prompt budget. An annotation study with three clinicians further shows that major/minor prompts enable faster annotation. Conclusion: S4M increases performance, reduces annotation effort, and aligns prompting with clinical practice, enabling more scalable dataset development in medical imaging.

Method: Proposes Perceptual-Consistency Clipping using attention focus overlap for aggressive outlier suppression, and Prompt-Aware Reconstruction incorporating image-prompt interactions via cross-attention in mask decoder. Also includes layer-skipping strategy for efficient image token processing in encoder.

Result: Extensive experiments show consistent advantages across various SAM sizes and tasks. When quantizing SAM-B to 4-bit, achieves 11.7% higher mAP than baseline in instance segmentation task.

Conclusion: SAQ-SAM effectively boosts PTQ for SAM through semantic alignment perspective, addressing both outlier suppression and semantic preservation, making SAM more suitable for edge deployment while maintaining performance.

Abstract: Segment Anything Model (SAM) exhibits remarkable zero-shot segmentation capability; however, its prohibitive computational costs make edge deployment challenging. Although post-training quantization (PTQ) offers a promising compression solution, existing methods yield unsatisfactory results when applied to SAM, owing to its specialized model components and promptable workflow: (i) The mask decoder’s attention exhibits extreme activation outliers, and we find that aggressive clipping (even 100x), without smoothing or isolation, is effective in suppressing outliers while maintaining performance. Unfortunately, traditional distribution-based metrics (e.g., MSE) fail to provide such large-scale clipping. (ii) Existing quantization reconstruction methods neglect semantic interactivity of SAM, leading to misalignment between image feature and prompt intention. To address the above issues, we propose SAQ-SAM in this paper, which boosts PTQ for SAM from the perspective of semantic alignment. Specifically, we propose Perceptual-Consistency Clipping, which exploits attention focus overlap to promote aggressive clipping while preserving semantic capabilities. Furthermore, we propose Prompt-Aware Reconstruction, which incorporates image-prompt interactions by leveraging cross-attention in mask decoder, thus facilitating alignment in both distribution and semantic. Moreover, to ensure the interaction efficiency, we design a layer-skipping strategy for image tokens in encoder. Extensive experiments are conducted on various SAM sizes and tasks, including instance segmentation, oriented object detection, and semantic segmentation, and the results show that our method consistently exhibits advantages. For example, when quantizing SAM-B to 4-bit, SAQ-SAM achieves 11.7% higher mAP than the baseline in instance segmentation task.

Method: Proposes TEMPLE framework with automated pipeline for constructing temporality-intensive preference pairs (selecting rich videos, designing perturbations, evaluating responses) and Progressive Pre-SFT Alignment with curriculum learning and preference optimization before instruction tuning.

Result: Consistently improves Video LLM performance across multiple benchmarks using relatively small self-generated DPO data.

Conclusion: TEMPLE serves as a scalable and efficient complement to SFT-based methods, enabling development of more reliable Video LLMs with enhanced temporal reasoning capabilities.

Abstract: Video Large Language Models (Video LLMs) have achieved significant success by adopting the paradigm of large-scale pre-training followed by supervised fine-tuning (SFT). However, existing approaches struggle with temporal reasoning due to weak temporal correspondence in the data and over-reliance on the next-token prediction paradigm}, which collectively result in the absence temporal supervision. To address these limitations, we propose TEMPLE (TEMporal Preference LEarning), a systematic framework that enhances temporal reasoning capabilities through Direct Preference Optimization (DPO). To address temporal information scarcity in data, we introduce an automated pipeline for systematically constructing temporality-intensive preference pairs comprising three steps: selecting temporally rich videos, designing video-specific perturbation strategies, and evaluating model responses on clean and perturbed inputs. Complementing this data pipeline, we provide additional supervision signals via preference learning and propose a novel Progressive Pre-SFT Alignment strategy featuring two key innovations: a curriculum learning strategy which progressively increases perturbation difficulty to maximize data efficiency; and applying preference optimization before instruction tuning to incentivize fundamental temporal alignment. Extensive experiments demonstrate that our approach consistently improves Video LLM performance across multiple benchmarks with a relatively small set of self-generated DPO data. Our findings highlight TEMPLE as a scalable and efficient complement to SFT-based methods, paving the way for developing reliable Video LLMs.

Method: Proposes SimROD with Global Gamma Enhancement (GGE) module using learnable global gamma transformation with only four parameters, and leverages the green channel’s richer signal to enhance local details based on human eye sensitivity and Bayer filter design.

Result: Extensive experiments on multiple RAW object detection datasets show SimROD outperforms state-of-the-art methods like RAW-Adapter and DIAP while maintaining efficiency.

Conclusion: The work highlights the potential of RAW data for real-world object detection, providing an effective and efficient solution for RAW-based detection tasks.

Abstract: Most visual models are designed for sRGB images, yet RAW data offers significant advantages for object detection by preserving sensor information before ISP processing. This enables improved detection accuracy and more efficient hardware designs by bypassing the ISP. However, RAW object detection is challenging due to limited training data, unbalanced pixel distributions, and sensor noise. To address this, we propose SimROD, a lightweight and effective approach for RAW object detection. We introduce a Global Gamma Enhancement (GGE) module, which applies a learnable global gamma transformation with only four parameters, improving feature representation while keeping the model efficient. Additionally, we leverage the green channel’s richer signal to enhance local details, aligning with the human eye’s sensitivity and Bayer filter design. Extensive experiments on multiple RAW object detection datasets and detectors demonstrate that SimROD outperforms state-of-the-art methods like RAW-Adapter and DIAP while maintaining efficiency. Our work highlights the potential of RAW data for real-world object detection. Code is available at https://ocean146.github.io/SimROD2025/.

Method: Uses patch attention algorithm for rendering quality refinement, Gaussian constraints strategy to minimize redundancy, and subdivision reconstruction strategy that divides large scenes into manageable blocks for individual training.

Result: Significantly reduces unnecessary Gaussians and enhances rendering quality, surpassing State-of-The-Art methods. Successfully manages and renders large urban scenes while maintaining high visual fidelity.

Conclusion: GaussianFocus effectively addresses key limitations of 3D Gaussian Splatting, enabling high-quality reconstruction and rendering of both small and large-scale scenes with improved efficiency and reduced redundancy.

Abstract: Recent developments in 3D reconstruction and neural rendering have significantly propelled the capabilities of photo-realistic 3D scene rendering across various academic and industrial fields. The 3D Gaussian Splatting technique, alongside its derivatives, integrates the advantages of primitive-based and volumetric representations to deliver top-tier rendering quality and efficiency. Despite these advancements, the method tends to generate excessive redundant noisy Gaussians overfitted to every training view, which degrades the rendering quality. Additionally, while 3D Gaussian Splatting excels in small-scale and object-centric scenes, its application to larger scenes is hindered by constraints such as limited video memory, excessive optimization duration, and variable appearance across views. To address these challenges, we introduce GaussianFocus, an innovative approach that incorporates a patch attention algorithm to refine rendering quality and implements a Gaussian constraints strategy to minimize redundancy. Moreover, we propose a subdivision reconstruction strategy for large-scale scenes, dividing them into smaller, manageable blocks for individual training. Our results indicate that GaussianFocus significantly reduces unnecessary Gaussians and enhances rendering quality, surpassing existing State-of-The-Art (SoTA) methods. Furthermore, we demonstrate the capability of our approach to effectively manage and render large scenes, such as urban environments, whilst maintaining high fidelity in the visual output.

Method: Use a learned initialization function to provide coarse but effective starting points for optimization-based pose estimators in 2D/3D pelvis registration.

Result: Experimental validation shows the method consistently achieves robust and accurate registration, even in challenging cases with extreme pose variation, while improving computational efficiency.

Conclusion: A learned initialization function significantly enhances the reliability and performance of 2D/3D pelvis registration for clinical applications.

Abstract: This paper presents an approach for improving 2D/3D pelvis registration in optimization-based pose estimators using a learned initialization function. Current methods often fail to converge to the optimal solution when initialized naively. We find that even a coarse initializer greatly improves pose estimator accuracy, and improves overall computational efficiency. This approach proves to be effective also in challenging cases under more extreme pose variation. Experimental validation demonstrates that our method consistently achieves robust and accurate registration, enhancing the reliability of 2D/3D registration for clinical applications.

Method: Introduces decamouflaged tokens for feature adjustment, implicit/explicit object-aware fusion modules for utilizing fine-grained features, and object prototype generation to memorize object details from previous frames.

Result: Achieves substantial improvements over SAM2: 12.2 mDice gains with click prompt on MoCA-Mask and 19.6 mDice gains with mask prompt on SUN-SEG-Hard using Hiera-T backbone, while adding negligible parameters.

Conclusion: CamSAM2 effectively enhances SAM2’s VCOS capability through lightweight architectural additions, demonstrating significant performance improvements on camouflaged object segmentation tasks.

Abstract: Video camouflaged object segmentation (VCOS), aiming at segmenting camouflaged objects that seamlessly blend into their environment, is a fundamental vision task with various real-world applications. With the release of SAM2, video segmentation has witnessed significant progress. However, SAM2’s capability of segmenting camouflaged videos is suboptimal, especially when given simple prompts such as point and box. To address the problem, we propose Camouflaged SAM2 (CamSAM2), which enhances SAM2’s ability to handle camouflaged scenes without modifying SAM2’s parameters. Specifically, we introduce a decamouflaged token to provide the flexibility of feature adjustment for VCOS. To make full use of fine-grained and high-resolution features from the current frame and previous frames, we propose implicit object-aware fusion (IOF) and explicit object-aware fusion (EOF) modules, respectively. Object prototype generation (OPG) is introduced to abstract and memorize object prototypes with informative details using high-quality features from previous frames. Extensive experiments are conducted to validate the effectiveness of our approach. While CamSAM2 only adds negligible learnable parameters to SAM2, it substantially outperforms SAM2 on three VCOS datasets, especially achieving 12.2 mDice gains with click prompt on MoCA-Mask and 19.6 mDice gains with mask prompt on SUN-SEG-Hard, with Hiera-T as the backbone. The code is available at https://github.com/zhoustan/CamSAM2.

Method: Proposes NIRNet with a Noise-based Imprint Simulator to capture intrinsic patterns from different models, and a pipeline using noise patterns alongside visual features for detection.

Result: Achieves state-of-the-art performance across seven diverse benchmarks, including five public datasets and two new generalization tests.

Conclusion: The noise-based imprint approach significantly enhances generalization and robustness in AI-generated image detection, demonstrating superior effectiveness compared to existing methods.

Abstract: With the rapid advancement of vision generation models, the potential security risks stemming from synthetic visual content have garnered increasing attention, posing significant challenges for AI-generated image detection. Existing methods suffer from inadequate generalization capabilities, resulting in unsatisfactory performance on emerging generative models. To address this issue, this paper presents NIRNet (Noise-based Imprint Revealing Network), a novel framework that leverages noise-based imprint for the detection task. Specifically, we propose a novel Noise-based Imprint Simulator to capture intrinsic patterns imprinted in images generated by different models. By aggregating imprint from various generative models, imprint of future models can be extrapolated to expand training data, thereby enhancing generalization and robustness. Furthermore, we design a new pipeline that pioneers the use of noise patterns, derived from a Noise-based Imprint Extractor, alongside other visual features for AI-generated image detection, significantly improving detection performance. Our approach achieves state-of-the-art performance across seven diverse benchmarks, including five public datasets and two newly proposed generalization tests, demonstrating its superior generalization and effectiveness. Paper Submission: pdf

Method: Decouples kernel adaptation into input-dependent dynamic routing and pre-trained static modulation using cross-layer weight sharing and adapter-based modulation, where dense convolutional layers are replaced by derived ‘child’ layers generated from shared ‘parent’ kernels.

Result: Achieves state-of-the-art accuracy-efficiency balance on image classification and dense prediction tasks, preserving native computational efficiency of standard convolutions while enhancing representation power through input-adaptive kernel adjustments.

Conclusion: KernelDNA provides an effective solution for dynamic convolution that maintains parameter efficiency and hardware-friendly inference while enabling dynamic kernel specialization through innovative weight-sharing mechanisms.

Abstract: Dynamic convolution enhances model capacity by adaptively combining multiple kernels, yet faces critical trade-offs: prior works either (1) incur significant parameter overhead by scaling kernel numbers linearly, (2) compromise inference speed through complex kernel interactions, or (3) struggle to jointly optimize dynamic attention and static kernels. We observe that pre-trained Convolutional Neural Networks (CNNs) exhibit inter-layer redundancy akin to that in Large Language Models (LLMs). Specifically, dense convolutional layers can be efficiently replaced by derived “child” layers generated from a shared “parent” convolutional kernel through an adapter. To address these limitations and implement the weight-sharing mechanism, we propose a lightweight convolution kernel plug-in, named KernelDNA. It decouples kernel adaptation into input-dependent dynamic routing and pre-trained static modulation, ensuring both parameter efficiency and hardware-friendly inference. Unlike existing dynamic convolutions that expand parameters via multi-kernel ensembles, our method leverages cross-layer weight sharing and adapter-based modulation, enabling dynamic kernel specialization without altering the standard convolution structure. This design preserves the native computational efficiency of standard convolutions while enhancing representation power through input-adaptive kernel adjustments. Experiments on image classification and dense prediction tasks demonstrate that KernelDNA achieves a state-of-the-art accuracy-efficiency balance among dynamic convolution variants.

Method: Proposes Quadtree Diffusion Model (QDM) - a mask-guided, two-stream architecture that uses quadtree structure derived from low-quality input to identify key regions needing refinement and applies minimal computation elsewhere.

Result: QDM outperforms or is comparable to state-of-the-art SR methods on standard benchmarks while significantly reducing computational costs, particularly effective in medical imaging with large homogeneous regions.

Conclusion: QDM provides an efficient and adaptive approach to super-resolution that balances quality and computational efficiency, making it suitable for resource-limited environments.

Abstract: Deep learning-based super-resolution (SR) methods often perform pixel-wise computations uniformly across entire images, even in homogeneous regions where high-resolution refinement is redundant. We propose the Quadtree Diffusion Model (QDM), a region-adaptive diffusion framework that leverages a quadtree structure to selectively enhance detail-rich regions while reducing computations in homogeneous areas. By guiding the diffusion with a quadtree derived from the low-quality input, QDM identifies key regions-represented by leaf nodes-where fine detail is essential and applies minimal refinement elsewhere. This mask-guided, two-stream architecture adaptively balances quality and efficiency, producing high-fidelity outputs with low computational redundancy. Experiments demonstrate QDM’s effectiveness in high-resolution SR tasks across diverse image types, particularly in medical imaging (e.g., CT scans), where large homogeneous regions are prevalent. Furthermore, QDM outperforms or is comparable to state-of-the-art SR methods on standard benchmarks while significantly reducing computational costs, highlighting its efficiency and suitability for resource-limited environments. Our code is available at https://github.com/linYDTHU/QDM.

Method: Built on a novel Sparse Focal Modulation (SFM) module that integrates short- and long-range contexts with linear complexity using hierarchical sparse convolution design.

Result: Achieves state-of-the-art performance on autonomous driving datasets with improved efficiency, making it suitable for large-scale LiDAR scenes.

Conclusion: SFMNet successfully combines the efficiency of sparse convolutions with the ability to model long-range dependencies, providing an effective solution for 3D object detection in large-scale scenes.

Abstract: We propose SFMNet, a novel 3D sparse detector that combines the efficiency of sparse convolutions with the ability to model long-range dependencies. While traditional sparse convolution techniques efficiently capture local structures, they struggle with modeling long-range relationships. However, capturing long-range dependencies is fundamental for 3D object detection. In contrast, transformers are designed to capture these long-range dependencies through attention mechanisms. But, they come with high computational costs, due to their quadratic query-key-value interactions. Furthermore, directly applying attention to non-empty voxels is inefficient due to the sparse nature of 3D scenes. Our SFMNet is built on a novel Sparse Focal Modulation (SFM) module, which integrates short- and long-range contexts with linear complexity by leveraging a new hierarchical sparse convolution design. This approach enables SFMNet to achieve high detection performance with improved efficiency, making it well-suited for large-scale LiDAR scenes. We show that our detector achieves state-of-the-art performance on autonomous driving datasets.

Method: Uses only semantic labels to predict instances without training or annotations, running in real-time on single-threaded CPU with no learning or parameter tuning required.

Result: Outperforms most state-of-the-art supervised methods on SemanticKITTI and nuScenes, ranks first on SemanticKITTI panoptic leaderboard when combined with state-of-the-art semantic segmentation.

Conclusion: Competitive panoptic segmentation can be achieved without instance annotations, offering a fully explainable, real-time solution that eliminates the need for costly manual labeling.

Abstract: Panoptic segmentation of LiDAR point clouds is fundamental to outdoor scene understanding, with autonomous driving being a primary application. While state-of-the-art approaches typically rely on end-to-end deep learning architectures and extensive manual annotations of instances, the significant cost and time investment required for labeling large-scale point cloud datasets remains a major bottleneck in this field. In this work, we demonstrate that competitive panoptic segmentation can be achieved using only semantic labels, with instances predicted without any training or annotations. Our method outperforms {most} state-of-the-art supervised methods on standard benchmarks including SemanticKITTI and nuScenes, and outperforms every publicly available method on SemanticKITTI as a drop-in instance head replacement, while running in real-time on a single-threaded CPU and requiring no instance labels. It is fully explainable, and requires no learning or parameter tuning. Alpine combined with state-of-the-art semantic segmentation ranks first on the official panoptic segmentation leaderboard of SemanticKITTI. Code is available at https://github.com/valeoai/Alpine/

Method: Creates Diverse Reflection Removal (DRR) dataset via random rotation of reflective mediums, and develops diffusion-based framework with one-step diffusion and three-stage progressive training including reflection-invariant finetuning.

Result: Achieves state-of-the-art performance on common benchmarks and challenging in-the-wild images, demonstrating superior generalization across diverse real-world scenes.

Conclusion: The proposed comprehensive solution with novel dataset and training strategy enables robust reflection removal with excellent generalization capabilities.

Abstract: Reflection removal of a single image remains a highly challenging task due to the complex entanglement between target scenes and unwanted reflections. Despite significant progress, existing methods are hindered by the scarcity of high-quality, diverse data and insufficient restoration priors, resulting in limited generalization across various real-world scenarios. In this paper, we propose Dereflection Any Image, a comprehensive solution with an efficient data preparation pipeline and a generalizable model for robust reflection removal. First, we introduce a dataset named Diverse Reflection Removal (DRR) created by randomly rotating reflective mediums in target scenes, enabling variation of reflection angles and intensities, and setting a new benchmark in scale, quality, and diversity. Second, we propose a diffusion-based framework with one-step diffusion for deterministic outputs and fast inference. To ensure stable learning, we design a three-stage progressive training strategy, including reflection-invariant finetuning to encourage consistent outputs across varying reflection patterns that characterize our dataset. Extensive experiments show that our method achieves SOTA performance on both common benchmarks and challenging in-the-wild images, showing superior generalization across diverse real-world scenes.

Method: Proposed Multi-Level Recurrent Attention Model (MRAM) that explicitly models neural hierarchy by decoupling glimpse location generation and task execution in two recurrent layers.

Result: MRAM achieves more human-like attention dynamics and consistently outperforms CNN, RAM and DRAM baselines on standard image classification benchmarks.

Conclusion: The hierarchical modeling approach in MRAM successfully produces balanced fixation-saccadic behavior while improving performance over existing attention models.

Abstract: Inspired by foveal vision, hard attention models promise interpretability and parameter economy. However, existing models like the Recurrent Model of Visual Attention (RAM) and Deep Recurrent Attention Model (DRAM) failed to model the hierarchy of human vision system, that compromise on the visual exploration dynamics. As a result, they tend to produce attention that are either overly fixational or excessively saccadic, diverging from human eye movement behavior. In this paper, we propose a Multi-Level Recurrent Attention Model (MRAM), a novel hard attention framework that explicitly models the neural hierarchy of human visual processing. By decoupling the function of glimpse location generation and task execution in two recurrent layers, MRAM emergent a balanced behavior between fixation and saccadic movement. Our results show that MRAM not only achieves more human-like attention dynamics, but also consistently outperforms CNN, RAM and DRAM baselines on standard image classification benchmarks.

Method: Examined 15 participants (3 healthy, 12 breast cancer patients) using 7T MRI with sodium imaging reconstructed using weighted TV, directional TV, anatomically guided TV, and adaptive combine methods.

Result: All methods preserved sodium signal and tissue structures. Anatomically guided TV had highest Dice score (75%). TSC values varied by method (61-88 mmol/L). Strong correlations found between different reconstruction methods.

Conclusion: Tumor appearance and TSC estimates depend on reconstruction method type and parameters, likely due to differences in reducing partial volume effects.

Abstract: Introduction: In sodium (23Na) magnetic resonance imaging (MRI), partial volume effects (PVE) are one of the most common causes of errors in the in vivo quantification of tissue sodium concentration (TSC). Advanced image reconstruction algorithms, such as compressed sensing (CS), have the potential to reduce PVE. Therefore, we investigated the feasibility of using CS-based methods to improve image quality and TSC quantification accuracy in patients with breast cancer. Subjects and methods: In this study, three healthy participants and 12 female participants with breast cancer were examined on a 7T MRI scanner. 23Na-MRI images were reconstructed using weighted total variation (wTV), directional total variation (dTV), anatomically guided total variation (AG-TV) and adaptive combine (ADC) methods. The consistency of tumor volume delineations based on sodium data was assessed using the Dice score, and TSC quantification was performed for various image reconstruction methods. Pearsons correlation coefficients were calculated to assess the relationships between wTV, dTV, AG-TV, and ADC values. Results: All methods provided breast MRI images with well-preserved sodium signal and tissue structures. The mean Dice scores for wTV, dTV, and AG-TV were 65%, 72%, and 75%, respectively. Average TSC values in breast tumors were 61.0, 72.0, 73.0, and 88.0 mmol/L for wTV, dTV, AG-TV, and ADC, respectively. A strong negative correlation was observed between wTV and dTV (r = -0.78, 95% CI [-0.94, -0.31], p = 0.0076) and a strong positive correlation between dTV and AG-TV (r = 0.71, 95% CI [0.16, 0.92], p = 0.0207) was found. Conclusion: The results of this study showed that differences in tumor appearance and TSC estimations may depend on the type of image reconstruction and the parameters used. This is most likely due to differences in their ability to reduce PVE.

Method: Uses a model-agnostic meta-architecture combining VLM and LLM with exploration-based search guided by calibrated confidence scores.

Result: Outperforms all open-source 7B VLMs and comparable agents on FALCON-Bench, and surpasses GPT-4o on single-detail tasks in MLVU while reducing inference cost by ~10x.

Conclusion: FALCONEye effectively addresses long-video QA challenges with a cost-efficient, training-free approach that generalizes well across different benchmarks.

Abstract: Finding information in hour-long videos is a challenging task even for top-performing Vision Language Models (VLMs), as encoding visual content quickly exceeds available context windows. To tackle this challenge, we present FALCONEye, a novel video agent based on a training-free, model-agnostic meta-architecture composed of a VLM and a Large Language Model (LLM). FALCONEye answers open-ended questions using an exploration-based search algorithm guided by calibrated confidence from the VLM’s answers. We also introduce the FALCON-Bench benchmark, extending Question Answering problem to Video Answer Search-requiring models to return both the answer and its supporting temporal window for open-ended questions in hour-long videos. With just a 7B VLM and a lightweight LLM, FALCONEye outscores all open-source 7B VLMs and comparable agents in FALCON-Bench. It further demonstrates its generalization capability in MLVU benchmark with shorter videos and different tasks, surpassing GPT-4o on single-detail tasks while slashing inference cost by roughly an order of magnitude.

Method: vGamba integrates SSMs with attention mechanisms through a Gamba bottleneck block containing Gamba Cell (Mamba adaptation for 2D), Multi-Head Self-Attention, and a Gated Fusion Module for effective feature representation.

Result: Extensive experiments on classification, detection, and segmentation tasks show vGamba achieves superior trade-off between accuracy and computational efficiency, outperforming several existing models.

Conclusion: The hybrid approach of combining SSMs with attention mechanisms enables efficient long-range dependency modeling in vision tasks while maintaining accuracy, demonstrating vGamba’s effectiveness across multiple computer vision applications.

Abstract: Capturing long-range dependencies efficiently is essential for visual recognition tasks, yet existing methods face limitations. Convolutional neural networks (CNNs) struggle with restricted receptive fields, while Vision Transformers (ViTs) achieve global context and long-range modeling at a high computational cost. State-space models (SSMs) offer an alternative, but their application in vision remains underexplored. This work introduces vGamba, a hybrid vision backbone that integrates SSMs with attention mechanisms to enhance efficiency and expressiveness. At its core, the Gamba bottleneck block that includes, Gamba Cell, an adaptation of Mamba for 2D spatial structures, alongside a Multi-Head Self-Attention (MHSA) mechanism and a Gated Fusion Module for effective feature representation. The interplay of these components ensures that vGamba leverages the low computational demands of SSMs while maintaining the accuracy of attention mechanisms for modeling long-range dependencies in vision tasks. Additionally, the Fusion module enables seamless interaction between these components. Extensive experiments on classification, detection, and segmentation tasks demonstrate that vGamba achieves a superior trade-off between accuracy and computational efficiency, outperforming several existing models.

Method: Two-stream architecture with Cross-Attention in Space and Time (CAST) using RGB input, extended to Cross-Attention in Visual and Audio (CAVA) by adding audio expert. Uses Bottleneck Cross-Attention (B-CA) for information exchange between experts.

Result: Consistent balanced performance on EPIC-KITCHENS-100, Something-Something-V2, Kinetics-400; favorable performance on audio-visual benchmarks UCF-101, VGG-Sound, KineticsSound, EPIC-SOUNDS.

Conclusion: CA^2ST effectively combines spatial, temporal, and audio experts through cross-attention to achieve balanced and holistic video understanding.

Abstract: We propose Cross-Attention in Audio, Space, and Time (CA^2ST), a transformer-based method for holistic video recognition. Recognizing actions in videos requires both spatial and temporal understanding, yet most existing models lack a balanced spatio-temporal understanding of videos. To address this, we propose a novel two-stream architecture, called Cross-Attention in Space and Time (CAST), using only RGB input. In each layer of CAST, Bottleneck Cross-Attention (B-CA) enables spatial and temporal experts to exchange information and make synergistic predictions. For holistic video understanding, we extend CAST by integrating an audio expert, forming Cross-Attention in Visual and Audio (CAVA). We validate the CAST on benchmarks with different characteristics, EPIC-KITCHENS-100, Something-Something-V2, and Kinetics-400, consistently showing balanced performance. We also validate the CAVA on audio-visual action recognition benchmarks, including UCF-101, VGG-Sound, KineticsSound, and EPIC-SOUNDS. With a favorable performance of CAVA across these datasets, we demonstrate the effective information exchange among multiple experts within the B-CA module. In summary, CA^2ST combines CAST and CAVA by employing spatial, temporal, and audio experts through cross-attention, achieving balanced and holistic video understanding.

Method: Bottom-up pipeline using foundation models: object pair detection, VLM description, LLM predicate generation (fine- and coarse-grained), and VQA validation. Modular, dataset-independent design.

Result: Achieves performance on par with state-of-the-art weakly-supervised models on SGG benchmarks and matches supervised methods in tasks like Sentence-to-Graph Retrieval. Enriches existing datasets with diverse, unbiased graphs.

Conclusion: PRISM-0 demonstrates that zero-shot open-vocabulary SGG using foundation models can overcome training bias and generate diverse predicates, performing competitively without requiring training data.

Abstract: In Scene Graph Generation (SGG), structured representations are extracted from visual inputs as object nodes and connecting predicates, enabling image-based reasoning for diverse downstream tasks. While fully supervised SGG has improved steadily, it suffers from training bias due to limited curated data and long-tail predicate distributions, leading to poor predicate diversity and degraded downstream performance. We present PRISM-0, a zero-shot open-vocabulary SGG framework that leverages foundation models in a bottom-up pipeline to capture a broad spectrum of predicates. Detected object pairs are filtered, described via a Vision-Language Model (VLM), and processed by a Large Language Model (LLM) to generate fine- and coarse-grained predicates, which are then validated by a Visual Question Answering (VQA) model. PRISM-0 modular, dataset-independent design enriches existing SGG datasets such as Visual Genome and produces diverse, unbiased graphs. While operating entirely in a zero-shot setting, PRISM-0 achieves performance on par with state-of-the-art weakly-supervised models on SGG benchmarks and even state-of-the-art supervised methods in tasks such as Sentence-to-Graph Retrieval.

Method: Annotation-free fine-tuning that distills three types of geometric cues from 3D foundation models: sparse correspondences, relative depth relations, and dense cost volumes, while maintaining compatibility with natural image-text inputs.

Result: Consistently outperforms prior approaches on 3D vision-language reasoning and 3D perception benchmarks, achieving improved 3D spatial reasoning with significantly lower computational cost.

Conclusion: Demonstrates a scalable and efficient path to bridge 2D-trained VLMs with 3D understanding, enabling wider use in spatially grounded multimodal tasks.

Abstract: Vision-Language Models (VLMs) have shown remarkable performance on diverse visual and linguistic tasks, yet they remain fundamentally limited in their understanding of 3D spatial structures. We propose Geometric Distillation, a lightweight, annotation-free fine-tuning framework that injects human-inspired geometric cues into pretrained VLMs without modifying their architecture. By distilling (1) sparse correspondences, (2) relative depth relations, and (3) dense cost volumes from off-the-shelf 3D foundation models (e.g., MASt3R, VGGT), our method shapes representations to be geometry-aware while remaining compatible with natural image-text inputs. Through extensive evaluations on 3D vision-language reasoning and 3D perception benchmarks, our method consistently outperforms prior approaches, achieving improved 3D spatial reasoning with significantly lower computational cost. Our work demonstrates a scalable and efficient path to bridge 2D-trained VLMs with 3D understanding, opening up wider use in spatially grounded multimodal tasks.

Method: Integrates pre-trained detector, pre-trained segmentor with tracking logic into a zero-shot MOT system that requires no fine-tuning, using segmentation as the primary tracking mechanism.

Result: Achieves state-of-the-art results on DanceTrack (+2.1 HOTA, +4.5 IDF1), UAVDT, and BDD100K benchmarks, demonstrating effectiveness in handling complex tracking scenarios.

Conclusion: SAM2MOT significantly reduces dependence on labeled data and paves the way for transitioning MOT research from task-specific solutions to general-purpose systems.

Abstract: Inspired by Segment Anything 2, which generalizes segmentation from images to videos, we propose SAM2MOT–a novel segmentation-driven paradigm for multi-object tracking that breaks away from the conventional detection-association framework. In contrast to previous approaches that treat segmentation as auxiliary information, SAM2MOT places it at the heart of the tracking process, systematically tackling challenges like false positives and occlusions. Its effectiveness has been thoroughly validated on major MOT benchmarks. Furthermore, SAM2MOT integrates pre-trained detector, pre-trained segmentor with tracking logic into a zero-shot MOT system that requires no fine-tuning. This significantly reduces dependence on labeled data and paves the way for transitioning MOT research from task-specific solutions to general-purpose systems. Experiments on DanceTrack, UAVDT, and BDD100K show state-of-the-art results. Notably, SAM2MOT outperforms existing methods on DanceTrack by +2.1 HOTA and +4.5 IDF1, highlighting its effectiveness in MOT. Code is available at https://github.com/TripleJoy/SAM2MOT.

Method: The authors identify two forms of redundancy (data-level and model-level) and design techniques including attention sparsity manipulation, attention head permutation, clean token regularization, ghost MoE diversification, and test-time adversarial training.

Result: Extensive experiments on ImageNet-1k show the proposed methods significantly outperform existing baselines in both transferability and generality across diverse model architectures.

Conclusion: Computational redundancy in Vision Transformers can be effectively harnessed to create more transferable and general adversarial examples, demonstrating superior performance compared to existing attack methods.

Abstract: Vision Transformers (ViTs) have demonstrated impressive performance across a range of applications, including many safety-critical tasks. However, their unique architectural properties raise new challenges and opportunities in adversarial robustness. In particular, we observe that adversarial examples crafted on ViTs exhibit higher transferability compared to those crafted on CNNs, suggesting that ViTs contain structural characteristics favorable for transferable attacks. In this work, we investigate the role of computational redundancy in ViTs and its impact on adversarial transferability. Unlike prior studies that aim to reduce computation for efficiency, we propose to exploit this redundancy to improve the quality and transferability of adversarial examples. Through a detailed analysis, we identify two forms of redundancy, including the data-level and model-level, that can be harnessed to amplify attack effectiveness. Building on this insight, we design a suite of techniques, including attention sparsity manipulation, attention head permutation, clean token regularization, ghost MoE diversification, and test-time adversarial training. Extensive experiments on the ImageNet-1k dataset validate the effectiveness of our approach, showing that our methods significantly outperform existing baselines in both transferability and generality across diverse model architectures.

Method: Treats all video visual modalities in color space to learn joint distributions, employs adaptive control strategy that dynamically adjusts each modality’s role (generation or conditioning) during diffusion process.

Result: Achieves state-of-the-art performance in video generation tasks and competitive results in video understanding, demonstrating flexibility for downstream applications like video-to-video translation and scene reconstruction.

Conclusion: OmniVDiff provides a scalable and flexible framework that effectively handles multiple video modalities in a unified manner, showing strong performance in both generation and understanding tasks.

Abstract: In this paper, we propose a novel framework for controllable video diffusion, OmniVDiff , aiming to synthesize and comprehend multiple video visual content in a single diffusion model. To achieve this, OmniVDiff treats all video visual modalities in the color space to learn a joint distribution, while employing an adaptive control strategy that dynamically adjusts the role of each visual modality during the diffusion process, either as a generation modality or a conditioning modality. Our framework supports three key capabilities: (1) Text-conditioned video generation, where all modalities are jointly synthesized from a textual prompt; (2) Video understanding, where structural modalities are predicted from rgb inputs in a coherent manner; and (3) X-conditioned video generation, where video synthesis is guided by finegrained inputs such as depth, canny and segmentation. Extensive experiments demonstrate that OmniVDiff achieves state-of-the-art performance in video generation tasks and competitive results in video understanding. Its flexibility and scalability make it well-suited for downstream applications such as video-to-video translation, modality adaptation for visual tasks, and scene reconstruction.

Method: Proposes PCMNet that learns human-comprehensible prototypes from meaningful image regions without additional supervision, clusters prototypes into concept groups, and extracts concept activation vectors for structured explanations.

Result: Outperforms state-of-the-art methods in interpretability, stability, and robustness across multiple image classification benchmarks. Enhances robustness to occlusion and challenging conditions.

Conclusion: Contributes to AI alignment by enhancing transparency, controllability, and trustworthiness in AI systems through structured concept-level explanations.

Abstract: As AI systems grow more capable, it becomes increasingly important that their decisions remain understandable and aligned with human expectations. A key challenge is the limited interpretability of deep models. Post-hoc methods like GradCAM offer heatmaps but provide limited conceptual insight, while prototype-based approaches offer example-based explanations but often rely on rigid region selection and lack semantic consistency. To address these limitations, we propose PCMNet, a part-prototypical concept mining network that learns human-comprehensible prototypes from meaningful image regions without additional supervision. By clustering these prototypes into concept groups and extracting concept activation vectors, PCMNet provides structured, concept-level explanations and enhances robustness to occlusion and challenging conditions, which are both critical for building reliable and aligned AI systems. Experiments across multiple image classification benchmarks show that PCMNet outperforms state-of-the-art methods in interpretability, stability, and robustness. This work contributes to AI alignment by enhancing transparency, controllability, and trustworthiness in AI systems. Our code is available at: https://github.com/alehdaghi/PCMNet.

Method: Developed LibraAlign-100K dataset with dual safety-quality annotations; proposed Synergistic Preference Optimization (T2I-SPO) algorithm extending DPO with composite reward function; introduced Unified Alignment Score metric.

Result: Achieves state-of-the-art safety alignment against NSFW concepts while better maintaining generation quality and general model capabilities compared to existing methods.

Conclusion: The unified framework successfully mitigates the safety-quality trade-off through synergistic alignment across data, optimization methods, and evaluation protocols.

Abstract: Content safety is a fundamental challenge for text-to-image (T2I) models, yet prevailing methods enforce a debilitating trade-off between safety and generation quality. We argue that mitigating this trade-off hinges on addressing systemic challenges in current T2I safety alignment across data, methods, and evaluation protocols. To this end, we introduce a unified framework for synergistic safety alignment. First, to overcome the flawed data paradigm that provides biased optimization signals, we develop LibraAlign-100K, the first large-scale dataset with dual annotations for safety and quality. Second, to address the myopic optimization of existing methods focus solely on safety reward, we propose Synergistic Preference Optimization (T2I-SPO), a novel alignment algorithm that extends the DPO paradigm with a composite reward function that integrates generation safety and quality to holistically model user preferences. Finally, to overcome the limitations of quality-agnostic and binary evaluation in current protocols, we introduce the Unified Alignment Score, a holistic, fine-grained metric that fairly quantifies the balance between safety and generative capability. Extensive experiments demonstrate that T2I-SPO achieves state-of-the-art safety alignment against a wide range of NSFW concepts, while better maintaining the model’s generation quality and general capability

Method: Proposed Attention Ablation Technique (AAT) that systematically identifies harmful attention heads and suppresses them by directly manipulating their attention weights using two complementary strategies for different application scenarios.

Result: AAT consistently improves downstream performance across diverse domains, boosting recall by up to 11.1% on cross-modal retrieval benchmarks with minimal overhead.

Conclusion: AAT effectively refines large-scale vision-language models with virtually no extra inference cost while yielding semantically meaningful patterns that align with existing interpretability findings.

Abstract: This paper investigates the role of attention heads in CLIP’s image encoder. Building on interpretability studies, we conduct an exhaustive analysis and find that certain heads, distributed across layers, are detrimental to the resulting representations. To mitigate their impact, we propose a simple yet effective Attention Ablation Technique (AAT) that suppresses selected heads by directly manipulating their attention weights. By incorporating two complementary strategies tailored to different application scenarios, AAT enables the systematic identification and ablation of harmful heads with minimal overhead. Experiments show that AAT consistently improves downstream performance across diverse domains, boosting recall by up to 11.1% on cross-modal retrieval benchmarks. These results highlight that AAT can effectively refine large-scale VLMs with virtually no extra inference cost, while yielding semantically meaningful patterns that align with existing interpretability findings.

Method: Lifts 2D video features into 3D world space using depth and camera motion information to create camera-stabilized spatio-temporal feature clouds. Uses iterative multi-frame motion estimation and a novel 3D Neighborhood-to-Neighborhood (N2N) attention mechanism for spatially coherent feature neighborhoods.

Result: Significantly outperforms existing 3D point tracking methods and surpasses state-of-the-art 2D pixel trackers in accuracy when reliable depth is available. Shows substantial gains in tracking robustness by compensating for camera motion.

Conclusion: TAPIP3D’s 3D-centric formulation with camera motion compensation and 3D attention mechanism provides strong and consistent performance across multiple 3D point tracking benchmarks, demonstrating the advantages of spatially grounded 3D representations over traditional 2D approaches.

Abstract: We introduce TAPIP3D, a novel approach for long-term 3D point tracking in monocular RGB and RGB-D videos. TAPIP3D represents videos as camera-stabilized spatio-temporal feature clouds, leveraging depth and camera motion information to lift 2D video features into a 3D world space where camera movement is effectively canceled out. Within this stabilized 3D representation, TAPIP3D iteratively refines multi-frame motion estimates, enabling robust point tracking over long time horizons. To handle the irregular structure of 3D point distributions, we propose a 3D Neighborhood-to-Neighborhood (N2N) attention mechanism - a 3D-aware contextualization strategy that builds informative, spatially coherent feature neighborhoods to support precise trajectory estimation. Our 3D-centric formulation significantly improves performance over existing 3D point tracking methods and even surpasses state-of-the-art 2D pixel trackers in accuracy when reliable depth is available. The model supports inference in both camera-centric (unstabilized) and world-centric (stabilized) coordinates, with experiments showing that compensating for camera motion leads to substantial gains in tracking robustness. By replacing the conventional 2D square correlation windows used in prior 2D and 3D trackers with a spatially grounded 3D attention mechanism, TAPIP3D achieves strong and consistent results across multiple 3D point tracking benchmarks. Project Page: https://tapip3d.github.io

Method: Proposes VistaDepth with two key innovations: 1) Latent Frequency Modulation (LFM) module that uses a lightweight network to predict dynamic spectral filters for refining latent features, and 2) BiasMap mechanism that adaptively reweights diffusion loss across timesteps to mitigate data bias.

Result: VistaDepth achieves state-of-the-art performance for diffusion-based MDE, particularly excelling in reconstructing detailed and accurate depth in far-range regions.

Conclusion: The proposed VistaDepth framework effectively addresses far-range depth estimation challenges through frequency-aware processing and adaptive loss reweighting, demonstrating superior performance in detailed depth reconstruction.

Abstract: Monocular depth estimation (MDE) aims to infer per-pixel depth from a single RGB image. While diffusion models have advanced MDE with impressive generalization, they often exhibit limitations in accurately reconstructing far-range regions. This difficulty arises from two key challenges. First, the implicit multi-scale processing in standard spatial-domain models can be insufficient for preserving the fine-grained, high-frequency details crucial for distant structures. Second, the intrinsic long-tail distribution of depth data imposes a strong training bias towards more prevalent near-range regions. To address these, we propose VistaDepth, a novel diffusion framework designed for balanced and accurate depth perception. We introduce two key innovations. First, the Latent Frequency Modulation (LFM) module enhances the model’s ability to represent high-frequency details. It operates by having a lightweight network predict a dynamic, content-aware spectral filter to refine latent features, thereby improving the reconstruction of distant structures. Second, our BiasMap mechanism introduces an adaptive reweighting of the diffusion loss strategically scaled across diffusion timesteps. It further aligns the supervision with the progressive denoising process, establishing a more consistent learning signal. As a result, it mitigates data bias without sacrificing training stability. Experiments show that VistaDepth achieves state-of-the-art performance for diffusion-based MDE, particularly excelling in reconstructing detailed and accurate depth in far-range regions.

Method: Proposes a flexible approach that regularizes a subset of first-layer convolutional filters by making them pairwise-orthogonal, reducing feature redundancy without excessively constraining the network or increasing computational load.

Result: Evaluated on three open-set visual tasks (chest X-ray anomaly detection, synthetic face detection, iris presentation attack detection) and showed increased generalization capabilities compared to state-of-the-art kernel orthogonalization methods.

Conclusion: The proposed lightweight orthogonal regularization method effectively improves CNN generalization in open-set tasks while avoiding excessive constraints and computational overhead of existing approaches.

Abstract: Despite several algorithmic advances in the training of convolutional neural networks (CNNs) over the years, their generalization capabilities are still subpar across several pertinent domains, particularly within open-set tasks often found in biometric and medical contexts. On the contrary, humans have an uncanny ability to generalize to unknown visual stimuli. The efficient coding hypothesis posits that early visual structures (retina, Lateral Geniculate Nucleus, and primary visual cortex) transform inputs to reduce redundancy and maximize information efficiency. This mechanism of redundancy minimization in early vision was the inspiration for CNN regularization techniques that force convolutional kernels to be orthogonal. However, the existing works rely upon matrix projections, architectural modifications, or specific weight initializations, which frequently overtly constrain the network’s learning process and excessively increase the computational load during loss function calculation. In this paper, we introduce a flexible and lightweight approach that regularizes a subset of first-layer convolutional filters by making them pairwise-orthogonal, which reduces the redundancy of the extracted features but at the same time prevents putting excessive constraints on the network. We evaluate the proposed method on three open-set visual tasks (anomaly detection in chest X-ray images, synthetic face detection, and iris presentation attack detection) and observe an increase in the generalization capabilities of models trained with the proposed regularizer compared to state-of-the-art kernel orthogonalization approaches. We offer source codes along with the paper.

Method: Uses geometrically-interpretable sub-networks for cutting, deforming, unwrapping, and wrapping to create a bi-directional cycle mapping framework. Also includes multi-chart parameterization with adaptively-learned chart assignment.

Result: Extensive experiments show universality, superiority, and inspiring potential of the neural surface parameterization paradigm.

Conclusion: FlexPara provides a flexible neural approach for both global and multi-chart surface parameterizations that adapts to different surface structures without manual intervention.

Abstract: Surface parameterization is a fundamental geometry processing task, laying the foundations for the visual presentation of 3D assets and numerous downstream shape analysis scenarios. Conventional parameterization approaches demand high-quality mesh triangulation and are restricted to certain simple topologies unless additional surface cutting and decomposition are provided. In practice, the optimal configurations (e.g., type of parameterization domains, distribution of cutting seams, number of mapping charts) may vary drastically with different surface structures and task characteristics, thus requiring more flexible and controllable processing pipelines. To this end, this paper introduces FlexPara, an unsupervised neural optimization framework to achieve both global and multi-chart surface parameterizations by establishing point-wise mappings between 3D surface points and adaptively-deformed 2D UV coordinates. We ingeniously design and combine a series of geometrically-interpretable sub-networks, with specific functionalities of cutting, deforming, unwrapping, and wrapping, to construct a bi-directional cycle mapping framework for global parameterization without the need for manually specified cutting seams. Furthermore, we construct a multi-chart parameterization framework with adaptively-learned chart assignment. Extensive experiments demonstrate the universality, superiority, and inspiring potential of our neural surface parameterization paradigm. The code will be publicly available at https://github.com/AidenZhao/FlexPara

Method: Used YOLOv11 with multi-scale processing (whole image + segmented parts), copy-paste data augmentation for diverse drone/bird examples, and frame-to-frame consistency post-processing.

Result: Achieved first place in the 8th WOSDETC Drone-vs-Bird Detection Grand Challenge at IJCNN 2025.

Conclusion: The proposed approach effectively detects drones in complex environments by combining multi-scale processing, data augmentation, and temporal consistency techniques.

Abstract: Detecting small drones, often indistinguishable from birds, is crucial for modern surveillance. This work introduces a drone detection methodology built upon the medium-sized YOLOv11 object detection model. To enhance its performance on small targets, we implemented a multi-scale approach in which the input image is processed both as a whole and in segmented parts, with subsequent prediction aggregation. We also utilized a copy-paste data augmentation technique to enrich the training dataset with diverse drone and bird examples. Finally, we implemented a post-processing technique that leverages frame-to-frame consistency to mitigate missed detections. The proposed approach attained first place in the 8th WOSDETC Drone-vs-Bird Detection Grand Challenge, held at the 2025 International Joint Conference on Neural Networks (IJCNN), showcasing its capability to detect drones in complex environments effectively.

Method: Three core modules: 1) ICE captures inter-sample correlation using semantically similar unpaired text-video combinations; 2) IRM mitigates intra-sample redundancy by distinguishing relevant from redundant moments; 3) TCP enhances moment-level semantics by predicting temporal order of shuffled frames.

Result: Extensive experiments demonstrate state-of-the-art performance in partially relevant video retrieval tasks.

Conclusion: The proposed framework effectively addresses semantic asymmetry in PRVR by systematically exploiting inter-sample correlation and intra-sample redundancy, achieving superior retrieval performance.

Abstract: Partially Relevant Video Retrieval (PRVR) aims to retrieve the target video that is partially relevant to the text query. The primary challenge in PRVR arises from the semantic asymmetry between textual and visual modalities, as videos often contain substantial content irrelevant to the query. Existing methods coarsely align paired videos and text queries to construct the semantic space, neglecting the critical cross-modal dual nature inherent in this task: inter-sample correlation and intra-sample redundancy. To this end, we propose a novel PRVR framework to systematically exploit these two characteristics. Our framework consists of three core modules. First, the Inter Correlation Enhancement (ICE) module captures inter-sample correlation by identifying semantically similar yet unpaired text queries and video moments, combining them to form pseudo-positive pairs for more robust semantic space construction. Second, the Intra Redundancy Mining (IRM) module mitigates intra-sample redundancy by mining redundant moment features and distinguishing them from query-relevant moments, encouraging the model to learn more discriminative representations. Finally, to reinforce these modules, we introduce the Temporal Coherence Prediction (TCP) module, which enhances discrimination of fine-grained moment-level semantics by training the model to predict the original temporal order of randomly shuffled video frames and moments. Extensive experiments demonstrate the superiority of our method, achieving state-of-the-art results.

Method: Proposes TransPrune which progressively prunes tokens using Token Transition Variation (measuring changes in token representation magnitude/direction) and Instruction-Guided Attention (measuring instruction attention to image tokens).

Result: Achieves comparable multimodal performance to original LVLMs across eight benchmarks while reducing inference TFLOPs by more than half. TTV alone performs comparably to attention-based methods.

Conclusion: Token transition variation provides an effective alternative to attention-based importance criteria, enabling efficient token pruning without training while maintaining model performance.

Abstract: Large Vision-Language Models (LVLMs) have advanced multimodal learning but face high computational costs due to the large number of visual tokens, motivating token pruning to improve inference efficiency. The key challenge lies in identifying which tokens are truly important. Most existing approaches rely on attention-based criteria to estimate token importance. However, they inherently suffer from certain limitations, such as positional bias. In this work, we explore a new perspective on token importance based on token transitions in LVLMs. We observe that the transition of token representations provides a meaningful signal of semantic information. Based on this insight, we propose TransPrune, a training-free and efficient token pruning method. Specifically, TransPrune progressively prunes tokens by assessing their importance through a combination of Token Transition Variation (TTV)-which measures changes in both the magnitude and direction of token representations-and Instruction-Guided Attention (IGA), which measures how strongly the instruction attends to image tokens via attention. Extensive experiments demonstrate that TransPrune achieves comparable multimodal performance to original LVLMs, such as LLaVA-v1.5 and LLaVA-Next, across eight benchmarks, while reducing inference TFLOPs by more than half. Moreover, TTV alone can serve as an effective criterion without relying on attention, achieving performance comparable to attention-based methods. The code will be made publicly available upon acceptance of the paper at https://github.com/liaolea/TransPrune.

Method: Propose Image Dehazing Diffusion Models (IDDM) that incorporates atmospheric scattering model into noise diffusion, using gradual haze formation to help denoising Unet learn clear image distribution from hazy inputs. Includes specialized training strategy with simultaneous haze and noise introduction in forward process.

Result: IDDM shows domain generalization ability and effectively restores real-world hazy images despite being trained only on synthetic datasets, outperforming state-of-the-art approaches in quantitative and qualitative comparisons.

Conclusion: The proposed physics-guided diffusion model successfully bridges the synthetic-to-real domain gap for image dehazing, demonstrating robust performance on real-world images through incorporation of atmospheric scattering principles.

Abstract: Due to the domain gap between real-world and synthetic hazy images, current data-driven dehazing algorithms trained on synthetic datasets perform well on synthetic data but struggle to generalize to real-world scenarios. To address this challenge, we propose \textbf{I}mage \textbf{D}ehazing \textbf{D}iffusion \textbf{M}odels (IDDM), a novel diffusion process that incorporates the atmospheric scattering model into noise diffusion. IDDM aims to use the gradual haze formation process to help the denoising Unet robustly learn the distribution of clear images from the conditional input hazy images. We design a specialized training strategy centered around IDDM. Diffusion models are leveraged to bridge the domain gap from synthetic to real-world, while the atmospheric scattering model provides physical guidance for haze formation. During the forward process, IDDM simultaneously introduces haze and noise into clear images, and then robustly separates them during the sampling process. By training with physics-guided information, IDDM shows the ability of domain generalization, and effectively restores the real-world hazy images despite being trained on synthetic datasets. Extensive experiments demonstrate the effectiveness of our method through both quantitative and qualitative comparisons with state-of-the-art approaches.

Method: Integrates CLIP, SigLIP, DINOv2 and ConvNeXt vision backbones with sparse MoECs module for dynamic expert selection, plus Hierarchical Group Attention with intra/inter-group operations and adaptive gating.

Result: Outperforms state-of-the-art models, showing 3.25% improvement in POPE for hallucination mitigation and 153-point increase on MME for visual instruction following.

Conclusion: MoCHA effectively addresses VLLM limitations through multi-backbone integration and specialized attention mechanisms, with ablation studies confirming the robustness of MoECs and HGA.

Abstract: Vision large language models (VLLMs) are focusing primarily on handling complex and fine-grained visual information by incorporating advanced vision encoders and scaling up visual models. However, these approaches face high training and inference costs, as well as challenges in extracting visual details, effectively bridging across modalities. In this work, we propose a novel visual framework, MoCHA, to address these issues. Our framework integrates four vision backbones (i.e., CLIP, SigLIP, DINOv2 and ConvNeXt) to extract complementary visual features and is equipped with a sparse Mixture of Experts Connectors (MoECs) module to dynamically select experts tailored to different visual dimensions. To mitigate redundant or insufficient use of the visual information encoded by the MoECs module, we further design a Hierarchical Group Attention (HGA) with intra- and inter-group operations and an adaptive gating strategy for encoded visual features. We train MoCHA on two mainstream LLMs (e.g., Phi2-2.7B and Vicuna-7B) and evaluate their performance across various benchmarks. Notably, MoCHA outperforms state-of-the-art open-weight models on various tasks. For example, compared to CuMo (Mistral-7B), our MoCHA (Phi2-2.7B) presents outstanding abilities to mitigate hallucination by showing improvements of 3.25% in POPE and to follow visual instructions by raising 153 points on MME. Finally, ablation studies further confirm the effectiveness and robustness of the proposed MoECs and HGA in improving the overall performance of MoCHA.

Method: Treats sketch reconstruction as set prediction with improved transformer decoder using explicit position queries to directly detect and predict sketch curves from point clouds.

Result: Significantly outperforms implicit methods in both primitive accuracy and overall geometric fidelity.

Conclusion: Direct prediction paradigm is superior to implicit methods for CAD model recovery from point clouds.

Abstract: Recovering CAD models from point clouds requires reconstructing their topology and sketch-based extrusion primitives. A dominant paradigm for representing sketches involves implicit neural representations such as Signed Distance Fields (SDFs). However, this indirect approach inherently struggles with precision, leading to unintended curved edges and models that are difficult to edit. In this paper, we propose Point2Primitive, a framework that learns to directly predict the explicit, parametric primitives of CAD models. Our method treats sketch reconstruction as a set prediction problem, employing a improved transformer-based decoder with explicit position queries to directly detect and predict the fundamental sketch curves (i.e., type and parameter) from the point cloud. Instead of approximating a continuous field, we formulate curve parameters as explicit position queries, which are optimized autoregressively to achieve high accuracy. The overall topology is rebuilt via extrusion segmentation. Extensive experiments demonstrate that this direct prediction paradigm significantly outperforms implicit methods in both primitive accuracy and overall geometric fidelity.

Method: A unified two-stage framework: 1) extracts semantically rich features from hand-object interactions and enhances action representations using verb-noun co-occurrence matrix; 2) uses reinforcement learning-based module simulating cognitive reasoning through visual perception -> intention inference -> action anticipation.

Result: Achieves state-of-the-art performance on Ego4D, EPIC-Kitchens-55, and EGTEA Gaze+ benchmarks, demonstrating effectiveness and strong generalization capability.

Conclusion: INSIGHT successfully addresses key limitations in egocentric action anticipation by integrating hand-object interaction features, semantic dependencies, and explicit cognitive reasoning, leading to improved performance and generalization.

Abstract: Long-term action anticipation from egocentric video is critical for applications such as human-computer interaction and assistive technologies, where anticipating user intent enables proactive and context-aware AI assistance. However, existing approaches suffer from three key limitations: 1) underutilization of fine-grained visual cues from hand-object interactions, 2) neglect of semantic dependencies between verbs and nouns, and 3) lack of explicit cognitive reasoning, limiting generalization and long-term forecasting ability. To overcome these challenges, we propose INSIGHT, a unified two-stage framework for egocentric action anticipation. In the first stage, INSIGHT focuses on extracting semantically rich features from hand-object interaction regions and enhances action representations using a verb-noun co-occurrence matrix. In the second stage, it introduces a reinforcement learning-based module that simulates explicit cognitive reasoning through a structured process: visual perception (think) -> intention inference (reason) -> action anticipation (answer). Extensive experiments on Ego4D, EPIC-Kitchens-55, and EGTEA Gaze+ benchmarks show that INSIGHT achieves state-of-the-art performance, demonstrating its effectiveness and strong generalization capability.

Method: Combined bottom-up survey of 31 existing datasets with top-down strategy using cognitive science classification systems to design novel tasks about view-taking and dynamic scenes, creating a multi-choice visual question-answering benchmark.

Result: Leading vision-language models significantly underperform human experts, particularly in spatial orientation. A positive correlation was found between spatial reasoning proficiency and performance on embodied AI tasks.

Conclusion: The SITE benchmark effectively reveals gaps in current AI spatial intelligence capabilities and demonstrates the importance of spatial reasoning for embodied AI applications, highlighting areas for future model improvement.

Abstract: Spatial intelligence (SI) represents a cognitive ability encompassing the visualization, manipulation, and reasoning about spatial relationships, underpinning disciplines from neuroscience to robotics. We introduce SITE, a benchmark dataset towards SI Thorough Evaluation in a standardized format of multi-choice visual question-answering, designed to assess large vision-language models’ spatial intelligence across diverse visual modalities (single-image, multi-image, and video) and SI factors (figural to environmental scales, spatial visualization and orientation, intrinsic and extrinsic, static and dynamic). Our approach to curating the benchmark combines a bottom-up survey about 31 existing datasets and a top-down strategy drawing upon three classification systems in cognitive science, which prompt us to design two novel types of tasks about view-taking and dynamic scenes. Extensive experiments reveal that leading models fall behind human experts especially in spatial orientation, a fundamental SI factor. Moreover, we demonstrate a positive correlation between a model’s spatial reasoning proficiency and its performance on an embodied AI task.

Method: Created Landsat30-AU dataset with two components: Landsat30-AU-Cap (196,262 image-caption pairs) and Landsat30-AU-VQA (17,725 human-verified VQA samples) across eight remote sensing domains, using a bootstrapped pipeline with generic VLMs and iterative refinement.

Result: Off-the-shelf VLMs struggle with satellite imagery (EarthDial: 0.07 SPIDEr captioning, 0.48 VQA accuracy). Fine-tuning Qwen2.5-VL-7B on Landsat30-AU improves captioning from 0.11 to 0.31 SPIDEr and VQA accuracy from 0.74 to 0.87.

Conclusion: Landsat30-AU addresses critical gaps in satellite vision-language datasets and enables significant performance improvements through domain-specific fine-tuning, advancing accessible Earth observation capabilities.

Abstract: Vision language models (VLMs) that enable natural language interaction with satellite imagery can democratize Earth observation by accelerating expert workflows, making data accessible to non-specialists, and enabling planet-scale automation. However, existing datasets focus mainly on short-term, high-resolution imagery from a limited number of satellites, overlooking low-resolution, multi-satellite, long-term archives, such as Landsat, that are essential for affordable and bias-robust global monitoring. We address this gap with Landsat30-AU, a large-scale vision-language dataset built from 30-meter resolution imagery collected by four Landsat satellites (5, 7, 8, and 9) over Australia, spanning more than 36 years. The dataset includes two components: Landsat30-AU-Cap, containing $196,262$ image-caption pairs, and Landsat30-AU-VQA, comprising 17,725 human-verified visual question answering (VQA) samples across eight remote sensing domains. Both datasets are curated through a bootstrapped pipeline that leverages generic VLMs with iterative refinement and human verification to ensure quality. Our evaluation of eight VLMs on our benchmark reveals that off-the-shelf models struggle to understand satellite imagery. The open-source remote-sensing VLM EarthDial achieves only 0.07 SPIDEr in captioning and a VQA accuracy of 0.48, highlighting the limitations of current approaches. Encouragingly, lightweight fine-tuning of Qwen2.5-VL-7B on Landsat30-AU improves captioning performance from 0.11 to 0.31 SPIDEr and boosts VQA accuracy from 0.74 to 0.87. Code and data are available at https://github.com/papersubmit1/landsat30-au.

Method: Uses spoof-aware vision perception with original images and auxiliary information, plus prompt-guided vision token masking to improve generalization. Pre-trained on FaceShield-pre10K and fine-tuned on FaceShield-sft45K datasets.

Result: Significantly outperforms previous deep learning models and general MLLMs on four FAS tasks: coarse-grained classification, fine-grained classification, reasoning, and attack localization across three benchmark datasets.

Conclusion: FaceShield provides a comprehensive MLLM solution for face anti-spoofing with strong performance and interpretability, addressing the gap in specialized FAS models.

Abstract: Face anti-spoofing (FAS) is crucial for protecting facial recognition systems from presentation attacks. Previous methods approached this task as a classification problem, lacking interpretability and reasoning behind the predicted results. Recently, multimodal large language models (MLLMs) have shown strong capabilities in perception, reasoning, and decision-making in visual tasks. However, there is currently no universal and comprehensive MLLM and dataset specifically designed for FAS task. To address this gap, we propose FaceShield, a MLLM for FAS, along with the corresponding pre-training and supervised fine-tuning (SFT) datasets, FaceShield-pre10K and FaceShield-sft45K. FaceShield is capable of determining the authenticity of faces, identifying types of spoofing attacks, providing reasoning for its judgments, and detecting attack areas. Specifically, we employ spoof-aware vision perception (SAVP) that incorporates both the original image and auxiliary information based on prior knowledge. We then use an prompt-guided vision token masking (PVTM) strategy to random mask vision tokens, thereby improving the model’s generalization ability. We conducted extensive experiments on three benchmark datasets, demonstrating that FaceShield significantly outperforms previous deep learning models and general MLLMs on four FAS tasks, i.e., coarse-grained classification, fine-grained classification, reasoning, and attack localization. Our instruction datasets, protocols, and codes will be released at https://github.com/Why0912/FaceShield.

Method: Three synergistic techniques: Hierarchical Position Pruning (removes less essential later blocks), Positional Weight Preservation (protects early semantic blocks), and Sensitivity-Guided Distillation (adjusts knowledge transfer based on block sensitivity).

Result: Achieves 77.5-80.4% memory reduction (15.8GB to 3.2GB), 27.9-38.0% latency reduction, with minimal quality drop (2.6% GenEval, 7% HPSv2). User study shows perceptual quality comparable to original model.

Conclusion: HierarchicalPrune successfully compresses billion-scale diffusion models for on-device inference while preserving output quality, significantly outperforming prior compression methods.

Abstract: State-of-the-art text-to-image diffusion models (DMs) achieve remarkable quality, yet their massive parameter scale (8-11B) poses significant challenges for inferences on resource-constrained devices. In this paper, we present HierarchicalPrune, a novel compression framework grounded in a key observation: DM blocks exhibit distinct functional hierarchies, where early blocks establish semantic structures while later blocks handle texture refinements. HierarchicalPrune synergistically combines three techniques: (1) Hierarchical Position Pruning, which identifies and removes less essential later blocks based on position hierarchy; (2) Positional Weight Preservation, which systematically protects early model portions that are essential for semantic structural integrity; and (3) Sensitivity-Guided Distillation, which adjusts knowledge-transfer intensity based on our discovery of block-wise sensitivity variations. As a result, our framework brings billion-scale diffusion models into a range more suitable for on-device inference, while preserving the quality of the output images. Specifically, combined with INT4 weight quantisation, HierarchicalPrune achieves 77.5-80.4% memory footprint reduction (e.g., from 15.8 GB to 3.2 GB) and 27.9-38.0% latency reduction, measured on server and consumer grade GPUs, with the minimum drop of 2.6% in GenEval score and 7% in HPSv2 score compared to the original model. Finally, our comprehensive user study with 85 participants demonstrates that HierarchicalPrune maintains perceptual quality comparable to the original model while significantly outperforming prior works.

Method: Self-NPO uses truncated diffusion fine-tuning, a data-free approach that directly learns from the model itself to perform negative preference optimization, eliminating the need for manual data labeling or reward model training.

Result: The method is highly efficient (less than 1% training cost of Diffusion-NPO) and achieves comparable performance. It integrates seamlessly with SD1.5, SDXL, CogVideoX, and preference-optimized models, enhancing both generation quality and human preference alignment.

Conclusion: Self-NPO provides an efficient, data-free alternative to existing negative preference optimization methods, making preference alignment more practical and accessible across various diffusion models without requiring costly annotation procedures.

Abstract: Diffusion models have demonstrated remarkable success in various visual generation tasks, including image, video, and 3D content generation. Preference optimization (PO) is a prominent and growing area of research that aims to align these models with human preferences. While existing PO methods primarily concentrate on producing favorable outputs, they often overlook the significance of classifier-free guidance (CFG) in mitigating undesirable results. Diffusion-NPO addresses this gap by introducing negative preference optimization (NPO), training models to generate outputs opposite to human preferences and thereby steering them away from unfavorable outcomes through CFG. However, prior NPO approaches rely on costly and fragile procedures for obtaining explicit preference annotations (e.g., manual pairwise labeling or reward model training), limiting their practicality in domains where such data are scarce or difficult to acquire. In this work, we propose Self-NPO, specifically truncated diffusion fine-tuning, a data-free approach of negative preference optimization by directly learning from the model itself, eliminating the need for manual data labeling or reward model training. This data-free approach is highly efficient (less than 1% training cost of Diffusion-NPO) and achieves comparable performance to Diffusion-NPO in a data-free manner. We demonstrate that Self-NPO integrates seamlessly into widely used diffusion models, including SD1.5, SDXL, and CogVideoX, as well as models already optimized for human preferences, consistently enhancing both their generation quality and alignment with human preferences. Code is available at https://github.com/G-U-N/Diffusion-NPO.

Method: Models cross-view feature space as a composite manifold of content and viewpoint factors. Uses intra-view independence constraint (minimizing mutual information) and inter-view reconstruction constraint (cross-combining factors from paired images) for effective disentanglement.

Result: Extensive experiments on University-1652 and SUES-200 show strong robustness and generalization across scenarios, viewpoints and altitudes. Further evaluations on CVUSA and CVACT confirm consistent improvements.

Conclusion: CVD effectively addresses viewpoint discrepancies in DVGL through explicit factor disentanglement, integrates seamlessly as plug-and-play module, reduces inference latency, and demonstrates superior performance across multiple benchmarks.

Abstract: Drone-view geo-localization (DVGL) aims to match images of the same geographic location captured from drone and satellite perspectives. Despite recent advances, DVGL remains challenging due to significant appearance changes and spatial distortions caused by viewpoint variations. Existing methods typically assume that drone and satellite images can be directly aligned in a shared feature space via contrastive learning. Nonetheless, this assumption overlooks the inherent conflicts induced by viewpoint discrepancies, resulting in extracted features containing inconsistent information that hinders precise localization. In this study, we take a manifold learning perspective and model $\textit{the feature space of cross-view images as a composite manifold jointly governed by content and viewpoint}$. Building upon this insight, we propose $\textbf{CVD}$, a new DVGL framework that explicitly disentangles $\textit{content}$ and $\textit{viewpoint}$ factors. To promote effective disentanglement, we introduce two constraints: $\textit{(i)}$ an intra-view independence constraint that encourages statistical independence between the two factors by minimizing their mutual information; and $\textit{(ii)}$ an inter-view reconstruction constraint that reconstructs each view by cross-combining $\textit{content}$ and $\textit{viewpoint}$ from paired images, ensuring factor-specific semantics are preserved. As a plug-and-play module, CVD integrates seamlessly into existing DVGL pipelines and reduces inference latency. Extensive experiments on University-1652 and SUES-200 show that CVD exhibits strong robustness and generalization across various scenarios, viewpoints and altitudes, with further evaluations on CVUSA and CVACT confirming consistent improvements.

Method: Context Transfer paradigm that uses delayed LVLM outputs as historical context for SVLM inference, with edgeVLM implementation featuring context replacement and visual focus modules to refine text input and enhance visual grounding.

Result: Extensive experiments on three real-time vision-language reasoning tasks across four datasets demonstrate the framework’s effectiveness.

Conclusion: The new paradigm provides groundwork for more effective and latency-aware collaboration strategies in future VLM systems.

Abstract: Vision-Language Models (VLMs) are increasingly deployed in real-time applications such as autonomous driving and human-computer interaction, which demand fast and reliable responses based on accurate perception. To meet these requirements, existing systems commonly employ cloud-edge collaborative architectures, such as partitioned Large Vision-Language Models (LVLMs) or task offloading strategies between Large and Small Vision-Language Models (SVLMs). However, these methods fail to accommodate cloud latency fluctuations and overlook the full potential of delayed but accurate LVLM responses. In this work, we propose a novel cloud-edge collaborative paradigm for VLMs, termed Context Transfer, which treats the delayed outputs of LVLMs as historical context to provide real-time guidance for SVLMs inference. Based on this paradigm, we design edgeVLM, which incorporates both context replacement and visual focus modules to refine historical textual input and enhance visual grounding consistency. Extensive experiments on three real-time vision-lanuage reasoning tasks across four datasets demonstrate the effectiveness of the proposed framework. The new paradigm lays the groundwork for more effective and latency-aware collaboration strategies in future VLM systems.

Method: SEA dynamically selects diverse models from accessible pre-trained models across iterations, decoupling within-iteration and cross-iteration model diversity. It maintains a fixed number of within-iteration models for efficiency while increasing cross-iteration diversity for transferability.

Result: Experiments on ImageNet show SEA achieves 8.5% higher transferability than existing attacks under the same efficiency when selecting 4 from 20 models. The method also generalizes well to commercial vision APIs and large vision-language models.

Conclusion: SEA enables adaptive balancing of transferability and efficiency according to specific resource requirements, opening new possibilities for efficient ensemble attacks.

Abstract: In surrogate ensemble attacks, using more surrogate models yields higher transferability but lower resource efficiency. This practical trade-off between transferability and efficiency has largely limited existing attacks despite many pre-trained models are easily accessible online. In this paper, we argue that such a trade-off is caused by an unnecessary common assumption, i.e., all models should be \textit{identical} across iterations. By lifting this assumption, we can use as many surrogates as we want to unleash transferability without sacrificing efficiency. Concretely, we propose Selective Ensemble Attack (SEA), which dynamically selects diverse models (from easily accessible pre-trained models) across iterations based on our new interpretation of decoupling within-iteration and cross-iteration model diversity. In this way, the number of within-iteration models is fixed for maintaining efficiency, while only cross-iteration model diversity is increased for higher transferability. Experiments on ImageNet demonstrate the superiority of SEA in various scenarios. For example, when dynamically selecting 4 from 20 accessible models, SEA yields 8.5% higher transferability than existing attacks under the same efficiency. The superiority of SEA also generalizes to real-world systems, such as commercial vision APIs and large vision-language models. Overall, SEA opens up the possibility of adaptively balancing transferability and efficiency according to specific resource requirements.

Method: Proposed Panoramic AutoRegressive model (PAR) using masked autoregressive modeling, circular padding for spatial coherence, and consistency alignment strategy for improved quality.

Result: Extensive experiments show competitive performance in text-to-image generation and panorama outpainting, with promising scalability and generalization capabilities.

Conclusion: PAR provides a unified solution that overcomes fundamental limitations of diffusion models for panoramic generation while integrating text and image conditioning in a cohesive architecture.

Abstract: Recent progress in panoramic image generation has underscored two critical limitations in existing approaches. First, most methods are built upon diffusion models, which are inherently ill-suited for equirectangular projection (ERP) panoramas due to the violation of the identically and independently distributed (i.i.d.) Gaussian noise assumption caused by their spherical mapping. Second, these methods often treat text-conditioned generation (text-to-panorama) and image-conditioned generation (panorama outpainting) as separate tasks, relying on distinct architectures and task-specific data. In this work, we propose a unified framework, Panoramic AutoRegressive model (PAR), which leverages masked autoregressive modeling to address these challenges. PAR avoids the i.i.d. assumption constraint and integrates text and image conditioning into a cohesive architecture, enabling seamless generation across tasks. To address the inherent discontinuity in existing generative models, we introduce circular padding to enhance spatial coherence and propose a consistency alignment strategy to improve generation quality. Extensive experiments demonstrate competitive performance in text-to-image generation and panorama outpainting tasks while showcasing promising scalability and generalization capabilities.

Method: KSDiff constructs convolutional kernels through integration of a low-rank core tensor generator and unified factor generator, orchestrated by structure-aware multi-head attention, with a two-stage training strategy for pansharpening.

Result: KSDiff achieves superior performance compared to recent methods and demonstrates over 500x faster inference than diffusion-based pansharpening baselines.

Conclusion: The proposed KSDiff framework effectively enhances pansharpening quality while significantly accelerating inference, with ablation studies and evaluations confirming its effectiveness.

Abstract: Pansharpening seeks to fuse high-resolution panchromatic (PAN) and low-resolution multispectral (LRMS) images into a single image with both fine spatial and rich spectral detail. Despite progress in deep learning-based approaches, existing methods often fail to capture global priors inherent in remote sensing data distributions. Diffusion-based models have recently emerged as promising solutions due to their powerful distribution mapping capabilities, however, they suffer from heavy inference latency. We introduce KSDiff, a fast kernel-space diffusion framework that generates convolutional kernels enriched with global context to enhance pansharpening quality and accelerate inference. Specifically, KSDiff constructs these kernels through the integration of a low-rank core tensor generator and a unified factor generator, orchestrated by a structure-aware multi-head attention mechanism. We further introduce a two-stage training strategy tailored for pansharpening, facilitating integration into existing pansharpening architectures. Experiments show that KSDiff achieves superior performance compared to recent promising methods, and with over $500 \times$ faster inference than diffusion-based pansharpening baselines. Ablation studies, visualizations and further evaluations substantiate the effectiveness of our approach. Code will be released upon possible acceptance.

Method: Uses Feature Injection Module (FIM) and class-agnostic objectives to extract transferable features from a single target image, enabling attacks on arbitrary unseen classes across different datasets.

Result: Outperforms existing methods on ImageNet and seven additional datasets for unseen target classes in black-box and cross-domain scenarios.

Conclusion: CD-MTA provides a practical solution for cross-domain targeted attacks without data leakage risks, using only visual target representations.

Abstract: Multi-targeted adversarial attacks aim to mislead classifiers toward specific target classes using a single perturbation generator with a conditional input specifying the desired target class. Existing methods face two key limitations: (1) a single generator supports only a limited number of predefined target classes, and (2) it requires access to the victim model’s training data to learn target class semantics. This dependency raises data leakage concerns in practical black-box scenarios where the training data is typically private. To address these limitations, we propose a novel Cross-Domain Multi-Targeted Attack (CD-MTA) that can generate perturbations toward arbitrary target classes, even those that do not exist in the attacker’s training data. CD-MTA is trained on a single public dataset but can perform targeted attacks on black-box models trained on different datasets with disjoint and unknown class sets. Our method requires only a single example image that visually represents the desired target class, without relying its label, class distribution or pretrained embeddings. We achieve this through a Feature Injection Module (FIM) and class-agnostic objectives which guide the generator to extract transferable, fine-grained features from the target image without inferring class semantics. Experiments on ImageNet and seven additional datasets show that CD-MTA outperforms existing multi-targeted attack methods on unseen target classes in black-box and cross-domain scenarios. The code is available at https://github.com/tgoncalv/CD-MTA.

Method: Self-distillation approach where teacher (frozen pre-trained ViT) and student (same ViT + register tokens) are initialized from same model. Teacher generates denoised embeddings via test-time augmentation, which optimize small subset of student weights.

Result: Effectively reduces artifact tokens and improves segmentation and depth prediction performance under zero-shot and linear probing settings.

Conclusion: PH-Reg provides an efficient way to integrate registers into existing ViTs without full retraining, addressing artifact token issues and enhancing model performance on localization tasks.

Abstract: Vision Transformers (ViTs) have emerged as the dominant architecture for visual processing tasks, demonstrating excellent scalability with increased training data and model size. However, recent work has identified the emergence of artifact tokens in ViTs that are incongruous with local semantics. These anomalous tokens degrade ViT performance in tasks that require fine-grained localization or structural coherence. An effective mitigation of this issue is the addition of register tokens to ViTs, which implicitly “absorb” the artifact term during training.Given the availability of existing large-scale pre-trained ViTs, in this paper we seek add register tokens to existing models without needing to re-train from scratch, which is infeasible considering their size. Specifically, we propose Post Hoc Registers (PH-Reg), an efficient self-distillation method that integrates registers into an existing ViT without requiring additional labeled data and full retraining. PH-Reg initializes both teacher and student networks from the same pre-trained ViT. The teacher remains frozen and unmodified, while the student is augmented with randomly initialized register tokens. By applying test-time augmentation to the teacher’s inputs, we generate denoised dense embeddings free of artifacts, which are then used to optimize only a small subset of unlocked student weights. We show that our approach can effectively reduce the number of artifact tokens, improving the segmentation and depth prediction of the student ViT under zero-shot and linear probing.

Method: Proposes Realism Controlled One-step Diffusion (RCOD) with latent domain grouping strategy, degradation-aware sampling, and visual prompt injection module to replace text prompts with degradation-aware visual tokens.

Result: Achieves superior fidelity and perceptual quality while maintaining computational efficiency, outperforming state-of-the-art OSD methods in both quantitative metrics and visual quality.

Conclusion: RCOD provides flexible realism control capabilities during inference while maintaining the efficiency of one-step diffusion methods for real-world image super-resolution.

Abstract: Pre-trained diffusion models have shown great potential in real-world image super-resolution (Real-ISR) tasks by enabling high-resolution reconstructions. While one-step diffusion (OSD) methods significantly improve efficiency compared to traditional multi-step approaches, they still have limitations in balancing fidelity and realism across diverse scenarios. Since the OSDs for SR are usually trained or distilled by a single timestep, they lack flexible control mechanisms to adaptively prioritize these competing objectives, which are inherently manageable in multi-step methods through adjusting sampling steps. To address this challenge, we propose a Realism Controlled One-step Diffusion (RCOD) framework for Real-ISR. RCOD provides a latent domain grouping strategy that enables explicit control over fidelity-realism trade-offs during the noise prediction phase with minimal training paradigm modifications and original training data. A degradation-aware sampling strategy is also introduced to align distillation regularization with the grouping strategy and enhance the controlling of trade-offs. Moreover, a visual prompt injection module is used to replace conventional text prompts with degradation-aware visual tokens, enhancing both restoration accuracy and semantic consistency. Our method achieves superior fidelity and perceptual quality while maintaining computational efficiency. Extensive experiments demonstrate that RCOD outperforms state-of-the-art OSD methods in both quantitative metrics and visual qualities, with flexible realism control capabilities in the inference stage.

Method: Proposes SANSA framework that makes SAM2’s latent semantic structure explicit through minimal task-specific modifications, repurposing it for few-shot segmentation while supporting various prompts (points, boxes, scribbles).

Result: Achieves state-of-the-art performance on few-shot segmentation benchmarks for generalization assessment, outperforms generalist methods in in-context setting, remains significantly faster and more compact than prior approaches.

Conclusion: SAM2 already encodes rich semantic structure that can be effectively leveraged for few-shot segmentation with minimal modifications, demonstrating strong performance while maintaining efficiency and flexibility.

Abstract: Few-shot segmentation aims to segment unseen object categories from just a handful of annotated examples. This requires mechanisms that can both identify semantically related objects across images and accurately produce segmentation masks. We note that Segment Anything 2 (SAM2), with its prompt-and-propagate mechanism, offers both strong segmentation capabilities and a built-in feature matching process. However, we show that its representations are entangled with task-specific cues optimized for object tracking, which impairs its use for tasks requiring higher level semantic understanding. Our key insight is that, despite its class-agnostic pretraining, SAM2 already encodes rich semantic structure in its features. We propose SANSA (Semantically AligNed Segment Anything 2), a framework that makes this latent structure explicit, and repurposes SAM2 for few-shot segmentation through minimal task-specific modifications. SANSA achieves state-of-the-art performance on few-shot segmentation benchmarks specifically designed to assess generalization, outperforms generalist methods in the popular in-context setting, supports various prompts flexible interaction via points, boxes, or scribbles, and remains significantly faster and more compact than prior approaches. Code is available at https://github.com/ClaudiaCuttano/SANSA.

Method: Architecture-agnostic framework using conformal inference enhanced by a novel clipping block technique to provide provable guarantees while mitigating excessive conservatism.

Result: Experiments on large-scale segmentation models across multiple datasets (CamVid, OCTA-500, Lung Segmentation, Cityscapes) show reliable safety guarantees with substantially reduced conservatism compared to state-of-the-art approaches.

Conclusion: The proposed framework delivers scalable, reliable probabilistic verification for semantic segmentation networks while addressing the conservatism limitations of prior methods, with code available for reproducibility.

Abstract: Semantic segmentation networks (SSNs) are central to safety-critical applications such as medical imaging and autonomous driving, where robustness under uncertainty is essential. However, existing probabilistic verification methods often fail to scale with the complexity and dimensionality of modern segmentation tasks, producing guarantees that are overly conservative and of limited practical value. We propose a probabilistic verification framework that is architecture-agnostic and scalable to high-dimensional input-output spaces. Our approach employs conformal inference (CI), enhanced by a novel technique that we call the \textbf{clipping block}, to provide provable guarantees while mitigating the excessive conservatism of prior methods. Experiments on large-scale segmentation models across CamVid, OCTA-500, Lung Segmentation, and Cityscapes demonstrate that our framework delivers reliable safety guarantees while substantially reducing conservatism compared to state-of-the-art approaches on segmentation tasks. We also provide a public GitHub repository (https://github.com/Navidhashemicodes/SSN_Reach_CLP_Surrogate) for this approach, to support reproducibility.

Method: Represent LiDAR configurations in 6D design space, learn task-specific implicit densities via flow-based generative modeling, and synthesize systems using parametric distributions fitted via expectation-maximization

Result: Validated on diverse 3D vision tasks including face scanning, robotic tracking, and object detection

Conclusion: Enables automated, constraint-aware LiDAR system design for real-world applications

Abstract: Imaging system design is a complex, time-consuming, and largely manual process; LiDAR design, ubiquitous in mobile devices, autonomous vehicles, and aerial imaging platforms, adds further complexity through unique spatial and temporal sampling requirements. In this work, we propose a framework for automated, task-driven LiDAR system design under arbitrary constraints. To achieve this, we represent LiDAR configurations in a continuous six-dimensional design space and learn task-specific implicit densities in this space via flow-based generative modeling. We then synthesize new LiDAR systems by modeling sensors as parametric distributions in 6D space and fitting these distributions to our learned implicit density using expectation-maximization, enabling efficient, constraint-aware LiDAR system design. We validate our method on diverse tasks in 3D vision, enabling automated LiDAR system design across real-world-inspired applications in face scanning, robotic tracking, and object detection.

Method: ZPressor partitions views into anchor and support sets, using cross attention to compress information from support views into anchor views, forming a compact latent state Z that retains essential scene information while discarding redundancy.

Result: Enables scaling to over 100 input views at 480P resolution on 80GB GPU, consistently improves performance under moderate input views and enhances robustness under dense view settings on DL3DV-10K and RealEstate10K benchmarks.

Conclusion: ZPressor provides an effective solution for scaling feed-forward 3DGS models to handle large numbers of input views while maintaining performance and efficiency.

Abstract: Feed-forward 3D Gaussian Splatting (3DGS) models have recently emerged as a promising solution for novel view synthesis, enabling one-pass inference without the need for per-scene 3DGS optimization. However, their scalability is fundamentally constrained by the limited capacity of their models, leading to degraded performance or excessive memory consumption as the number of input views increases. In this work, we analyze feed-forward 3DGS frameworks through the lens of the Information Bottleneck principle and introduce ZPressor, a lightweight architecture-agnostic module that enables efficient compression of multi-view inputs into a compact latent state $Z$ that retains essential scene information while discarding redundancy. Concretely, ZPressor enables existing feed-forward 3DGS models to scale to over 100 input views at 480P resolution on an 80GB GPU, by partitioning the views into anchor and support sets and using cross attention to compress the information from the support views into anchor views, forming the compressed latent state $Z$. We show that integrating ZPressor into several state-of-the-art feed-forward 3DGS models consistently improves performance under moderate input views and enhances robustness under dense view settings on two large-scale benchmarks DL3DV-10K and RealEstate10K. The video results, code and trained models are available on our project page: https://lhmd.top/zpressor.

Method: Proposed RSKT-Seg framework with three key components: 1) Multi-Directional Cost Map Aggregation (RS-CMA) for rotation-invariant visual cues, 2) Efficient Cost Map Fusion (RS-Fusion) transformer for spatial-semantic modeling, and 3) Remote Sensing Knowledge Transfer (RS-Transfer) for domain adaptation.

Result: Extensive experiments show RSKT-Seg outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation.

Conclusion: RSKT-Seg effectively bridges the domain gap in open-vocabulary remote sensing segmentation and establishes a new state-of-the-art performance with improved efficiency.

Abstract: Open-Vocabulary Remote Sensing Image Segmentation (OVRSIS), an emerging task that adapts Open-Vocabulary Segmentation (OVS) to the remote sensing (RS) domain, remains underexplored due to the absence of a unified evaluation benchmark and the domain gap between natural and RS images. To bridge these gaps, we first establish a standardized OVRSIS benchmark (\textbf{OVRSISBench}) based on widely-used RS segmentation datasets, enabling consistent evaluation across methods. Using this benchmark, we comprehensively evaluate several representative OVS/OVRSIS models and reveal their limitations when directly applied to remote sensing scenarios. Building on these insights, we propose \textbf{RSKT-Seg}, a novel open-vocabulary segmentation framework tailored for remote sensing. RSKT-Seg integrates three key components: (1) a Multi-Directional Cost Map Aggregation (RS-CMA) module that captures rotation-invariant visual cues by computing vision-language cosine similarities across multiple directions; (2) an Efficient Cost Map Fusion (RS-Fusion) transformer, which jointly models spatial and semantic dependencies with a lightweight dimensionality reduction strategy; and (3) a Remote Sensing Knowledge Transfer (RS-Transfer) module that injects pre-trained knowledge and facilitates domain adaptation via enhanced upsampling. Extensive experiments on the benchmark show that RSKT-Seg consistently outperforms strong OVS baselines by +3.8 mIoU and +5.9 mACC, while achieving 2x faster inference through efficient aggregation. Our code is \href{https://github.com/LiBingyu01/RSKT-Seg}{\textcolor{blue}{here}}.

Method: Partially fine-tunes the latent decoder with Perturbation-Aware Suppression (PAS) to freeze sensitive layers, and adds a Temporal Alignment module for frame coherence. Enables adaptive watermark integration during generation.

Result: Achieves best trade-off among watermark extraction accuracy, video quality, and latency. Shows strong robustness against spatial/temporal tampering, and stability across different video lengths and resolutions.

Conclusion: VidSig provides a practical solution for AIGC video protection with imperceptible watermarking, minimal computational overhead, and reliable tracing capabilities suitable for real-world applications.

Abstract: The rapid development of Artificial Intelligence Generated Content (AIGC) has led to significant progress in video generation, but also raises serious concerns about intellectual property protection and reliable content tracing. Watermarking is a widely adopted solution to this issue, yet existing methods for video generation mainly follow a post-generation paradigm, which often fails to effectively balance the trade-off between video quality and watermark extraction. Meanwhile, current in-generation methods that embed the watermark into the initial Gaussian noise usually incur substantial additional computation. To address these issues, we propose \textbf{Video Signature} (\textsc{VidSig}), an implicit watermarking method for video diffusion models that enables imperceptible and adaptive watermark integration during video generation with almost no extra latency. Specifically, we partially fine-tune the latent decoder, where \textbf{Perturbation-Aware Suppression} (PAS) pre-identifies and freezes perceptually sensitive layers to preserve visual quality. Beyond spatial fidelity, we further enhance temporal consistency by introducing a lightweight \textbf{Temporal Alignment} module that guides the decoder to generate coherent frame sequences during fine-tuning. Experimental results show that \textsc{VidSig} achieves the best trade-off among watermark extraction accuracy, video quality, and watermark latency. It also demonstrates strong robustness against both spatial and temporal tamper, and remains stable across different video lengths and resolutions, highlighting its practicality in real-world scenarios.

Method: Parameter-efficient denoising-diffusion architecture trained directly in pixel-space on synthetic object-motion videos with shape and material supervision, generating multiple disentangled attributes simultaneously.

Result: For static observations, produces diverse multimodal predictions capturing ambiguities; with object motion, distributions converge to more accurate explanations; achieves high-quality estimates for real-world objects.

Conclusion: Moving beyond single-view to continuous motion observations with generative perception effectively captures visual ambiguities and improves visual reasoning in physically-embodied systems.

Abstract: Perceiving the shape and material of an object from a single image is inherently ambiguous, especially when lighting is unknown and unconstrained. Despite this, humans can often disentangle shape and material, and when they are uncertain, they often move their head slightly or rotate the object to help resolve the ambiguities. Inspired by this behavior, we introduce a novel conditional denoising-diffusion model that generates samples of shape-and-material maps from a short video of an object undergoing differential motions. Our parameter-efficient architecture allows training directly in pixel-space, and it generates many disentangled attributes of an object simultaneously. Trained on a modest number of synthetic object-motion videos with supervision on shape and material, the model exhibits compelling emergent behavior: For static observations, it produces diverse, multimodal predictions of plausible shape-and-material maps that capture the inherent ambiguities; and when objects move, the distributions converge to more accurate explanations. The model also produces high-quality shape-and-material estimates for less ambiguous, real-world objects. By moving beyond single-view to continuous motion observations, and by using generative perception to capture visual ambiguities, our work suggests ways to improve visual reasoning in physically-embodied systems.

Method: Introduces an ensemble-based pipeline compatible with any monocular height estimation network, featuring architecture and loss functions designed for weak supervision using balanced soft losses and ordinal constraints to leverage information from noisy labels.

Result: Experiments on DFC23 (0.5-1 m) and GBH (3 m) datasets show the method achieves more consistent cross-domain performance, reducing average RMSE by up to 22.94% on DFC23 and 18.62% on GBH compared to baselines.

Conclusion: The proposed approach effectively addresses the challenge of limited high-quality annotations by leveraging imperfect out-of-domain labels, enabling improved cross-domain generalization for monocular height estimation tasks.

Abstract: Monocular height estimation provides an efficient and cost-effective solution for three-dimensional perception in remote sensing. However, training deep neural networks for this task demands abundant annotated data, while high-quality labels are scarce and typically available only in developed regions, which limits model generalization and constrains their applicability at large scales. This work addresses the problem by leveraging imperfect labels from out-of-domain regions to train pixel-wise height estimation networks, which may be incomplete, inexact, or inaccurate compared to high-quality annotations. We introduce an ensemble-based pipeline compatible with any monocular height estimation network, featuring architecture and loss functions specifically designed to leverage information in noisy labels through weak supervision, utilizing balanced soft losses and ordinal constraints. Experiments on two datasets – DFC23 (0.5–1 m) and GBH (3 m) – show that our method achieves more consistent cross-domain performance, reducing average RMSE by up to 22.94% on DFC23 and 18.62% on GBH compared with baselines. Ablation studies confirm the contribution of each design component.

Method: Proposes Semantic Reward Defense (SRD) using deep Q-network policy to apply discrete perturbations to sensitive image regions, guided by semantic fidelity scores that assess caption consistency and fluency.

Result: Effectively mitigates TrojVLM and Shadowcast backdoor attacks, reducing ASR to 3.6% and 5.6% respectively with less than 15% average CIDEr drop on clean inputs.

Conclusion: SRD provides a trigger-agnostic, interpretable defense paradigm that successfully protects VLMs from backdoor attacks without requiring prior knowledge of attack patterns.

Abstract: Visual language models (VLMs) have made significant progress in image captioning tasks, yet recent studies have found they are vulnerable to backdoor attacks. Attackers can inject undetectable perturbations into the data during inference, triggering abnormal behavior and generating malicious captions. These attacks are particularly challenging to detect and defend against due to the stealthiness and cross-modal propagation of the trigger signals. In this paper, we identify two key vulnerabilities by analyzing existing attack patterns: (1) the model exhibits abnormal attention concentration on certain regions of the input image, and (2) backdoor attacks often induce semantic drift and sentence incoherence. Based on these insights, we propose Semantic Reward Defense (SRD), a reinforcement learning framework that mitigates backdoor behavior without requiring any prior knowledge of trigger patterns. SRD learns to apply discrete perturbations to sensitive contextual regions of image inputs via a deep Q-network policy, aiming to confuse attention and disrupt the activation of malicious paths. To guide policy optimization, we design a reward signal named semantic fidelity score, which jointly assesses the semantic consistency and linguistic fluency of the generated captions, encouraging the agent to achieve a robust yet faithful output. SRD offers a trigger-agnostic, policy-interpretable defense paradigm that effectively mitigates local (TrojVLM) and global (Shadowcast) backdoor attacks, reducing ASR to 3.6% and 5.6% respectively, with less than 15% average CIDEr drop on the clean inputs. Our codes can be found at https://github.com/Ciconey/SRD.git.

Method: Comparative analysis between supervised lightweight CNN and zero-shot BiomedCLIP VLM on pneumonia detection (PneumoniaMNIST) and tuberculosis detection (Shenzhen TB dataset), with decision threshold calibration applied to the VLM.

Result: After calibration, BiomedCLIP achieved F1-score of 0.8841 for pneumonia (surpassing CNN’s 0.8803) and 0.7684 for tuberculosis (close to CNN’s 0.7834), significantly improving from uncalibrated performance of 0.4812.

Conclusion: Proper calibration is essential for unlocking the full diagnostic potential of zero-shot VLMs, enabling them to compete with or outperform task-specific supervised models in medical imaging.

Abstract: The accurate interpretation of chest radiographs using automated methods is a critical task in medical imaging. This paper presents a comparative analysis between a supervised lightweight Convolutional Neural Network (CNN) and a state-of-the-art, zero-shot medical Vision-Language Model (VLM), BiomedCLIP, across two distinct diagnostic tasks: pneumonia detection on the PneumoniaMNIST benchmark and tuberculosis detection on the Shenzhen TB dataset. Our experiments show that supervised CNNs serve as highly competitive baselines in both cases. While the default zero-shot performance of the VLM is lower, we demonstrate that its potential can be unlocked via a simple yet crucial remedy: decision threshold calibration. By optimizing the classification threshold on a validation set, the performance of BiomedCLIP is significantly boosted across both datasets. For pneumonia detection, calibration enables the zero-shot VLM to achieve a superior F1-score of 0.8841, surpassing the supervised CNN’s 0.8803. For tuberculosis detection, calibration dramatically improves the F1-score from 0.4812 to 0.7684, bringing it close to the supervised baseline’s 0.7834. This work highlights a key insight: proper calibration is essential for leveraging the full diagnostic power of zero-shot VLMs, enabling them to match or even outperform efficient, task-specific supervised models.

Method: APVR uses two complementary components: Pivot Frame Retrieval with query expansion and iterative spatio-semantic confidence scoring to identify relevant frames, and Pivot Token Retrieval that performs query-aware attention-driven token selection within up to 1024 pivot frames.

Result: APVR achieves significant performance improvements of up to 9.5%, 4.6% and 9.7% on LongVideoBench, VideoMME and MLVU benchmarks respectively, and achieves state-of-the-art results for both training-free and training-based approaches.

Conclusion: APVR effectively addresses the memory wall limitation in long video understanding through hierarchical visual information retrieval, enabling processing of hour-long videos while maintaining semantic fidelity without requiring additional training.

Abstract: Current multimodal large language models (MLLMs) struggle with hour-level video understanding, facing significant challenges not only in modeling the substantial information volume of long videos but also in overcoming the memory wall and resource constraints during both training and inference. Although recent training-free approaches have alleviated resource demands by compressing visual features, their reliance on incomplete visual information limits the performance potential. To address these limitations, we propose Adaptive Pivot Visual information Retrieval (APVR), a training-free framework that hierarchically retrieves and retains sufficient and important visual information. It breakthroughs the memory wall limitation via two complementary components: Pivot Frame Retrieval employs query expansion and iterative spatio-semantic confidence scoring to identify relevant video frames, and Pivot Token Retrieval performs query-aware attention-driven token selection within up to 1024 pivot frames. This dual granularity approach enables the processing of hour-long videos while maintaining semantic fidelity. Experimental validations on three different baseline MLLMs demonstrate significant performance improvements up to 9.5%, 4.6% and 9.7% on LongVideoBench, VideoMME and MLVU, respectively. APVR achieves state-of-the-art results for both training-free and training-based approaches.

Method: Integrates hierarchical clustering with contrastive learning, extending contrastive learning-based GCD with a novel semi-supervised hierarchical clustering loss to capture hierarchical tumor taxonomy structures.

Result: Achieves +28% improvement in accuracy over state-of-the-art GCD methods for patch-level classification on OpenSRH dataset, particularly in identifying unseen tumor categories, and demonstrates generalizability on slide-level classification across imaging modalities.

Conclusion: HGCD-BT effectively bridges the gap between supervised and unsupervised learning for brain tumor classification, enabling discovery of both known and unknown tumor types while capturing hierarchical relationships in tumor taxonomies.

Abstract: Accurate brain tumor classification is critical for intra-operative decision making in neuro-oncological surgery. However, existing approaches are restricted to a fixed set of predefined classes and are therefore unable to capture patterns of tumor types not available during training. Unsupervised learning can extract general-purpose features, but it lacks the ability to incorporate prior knowledge from labelled data, and semi-supervised methods often assume that all potential classes are represented in the labelled data. Generalized Category Discovery (GCD) aims to bridge this gap by categorizing both known and unknown classes within unlabelled data. To reflect the hierarchical structure of brain tumor taxonomies, in this work, we introduce Hierarchical Generalized Category Discovery for Brain Tumor Classification (HGCD-BT), a novel approach that integrates hierarchical clustering with contrastive learning. Our method extends contrastive learning based GCD by incorporating a novel semi-supervised hierarchical clustering loss. We evaluate HGCD-BT on OpenSRH, a dataset of stimulated Raman histology brain tumor images, achieving a +28% improvement in accuracy over state-of-the-art GCD methods for patch-level classification, particularly in identifying previously unseen tumor categories. Furthermore, we demonstrate the generalizability of HGCD-BT on slide-level classification of hematoxylin and eosin stained whole-slide images from the Digital Brain Tumor Atlas, confirming its utility across imaging modalities.

Method: Proposes MR-COSMO with two key components: (1) direct cross-modal alignment module for explicit 3D point cloud-text/2D image alignment, and (2) visual-text memory module with specialized feature banks for dynamic knowledge recall via attention mechanisms.

Result: Achieves state-of-the-art performance across 3D instruction, reference, and semantic segmentation benchmarks.

Conclusion: The proposed direct alignment and memory recall approach effectively bridges the 3D-text gap and enables precise fusion of geometric and semantic features for improved query-driven 3D segmentation.

Abstract: The rapid advancement of vision-language models (VLMs) in 3D domains has accelerated research in text-query-guided point cloud processing, though existing methods underperform in point-level segmentation due to inadequate 3D-text alignment that limits local feature-text context linking. To address this limitation, we propose MR-COSMO, a Visual-Text Memory Recall and Direct CrOSs-MOdal Alignment Method for Query-Driven 3D Segmentation, establishing explicit alignment between 3D point clouds and text/2D image data through a dedicated direct cross-modal alignment module while implementing a visual-text memory module with specialized feature banks. This direct alignment mechanism enables precise fusion of geometric and semantic features, while the memory module employs specialized banks storing text features, visual features, and their correspondence mappings to dynamically enhance scene-specific representations via attention-based knowledge recall. Comprehensive experiments across 3D instruction, reference, and semantic segmentation benchmarks confirm state-of-the-art performance.

Method: Systematic exploration of architectural adaptations to create a novel YOLOv8 architecture optimized for FPGA-based processing on Kria KV260 MPSoC hardware.

Result: Model analyzes ~700 megapixel SAR images in <1 minute using <10W power, with detection and classification performance only 2-3% lower than GPU-based models while being 50-2500x more computationally efficient.

Conclusion: This work enables on-satellite ML for time-critical SAR analysis and contributes to more autonomous, scalable satellite systems.

Abstract: Rapid analysis of satellite imagery within minutes-to-hours of acquisition is increasingly vital for many remote sensing applications, and is an essential component for developing next-generation autonomous and distributed satellite systems. On-satellite machine learning (ML) has the potential for such rapid analysis, by overcoming latency associated with intermittent satellite connectivity to ground stations or relay satellites, but state-of-the-art models are often too large or power-hungry for on-board deployment. Vessel detection using Synthetic Aperture Radar (SAR) is a critical time-sensitive application in maritime security that exemplifies this challenge. SAR vessel detection has previously been demonstrated only by ML models that either are too large for satellite deployment, have not been developed for sufficiently low-power hardware, or have only been tested on small SAR datasets that do not sufficiently represent the difficulty of the real-world task. Here we systematically explore a suite of architectural adaptations to develop a novel YOLOv8 architecture optimized for this task and FPGA-based processing. We deploy our model on a Kria KV260 MPSoC, and show it can analyze a ~700 megapixel SAR image in less than a minute, within common satellite power constraints (<10W). Our model has detection and classification performance only ~2% and 3% lower than values from state-of-the-art GPU-based models on the largest and most diverse open SAR vessel dataset, xView3-SAR, despite being ~50 and ~2500 times more computationally efficient. This work represents a key contribution towards on-satellite ML for time-critical SAR analysis, and more autonomous, scalable satellites.

Method: Introduces CAPTCHA-X benchmark with 7 CAPTCHA categories and reasoning annotations, plus a VLM-based framework that incorporates step-by-step reasoning before generating final coordinates.

Result: The proposed method achieves 83.9% average solving accuracy across five high-difficulty CAPTCHA types, significantly surpassing the baseline of 21.9% from commercial models.

Conclusion: Step-by-step reasoning is crucial for solving spatial reasoning tasks like CAPTCHAs, revealing limitations in current models and highlighting the importance of reasoning for advancing visual-spatial challenges.

Abstract: CAPTCHA, originally designed to distinguish humans from robots, has evolved into a real-world benchmark for assessing the spatial reasoning capabilities of vision-language models. In this work, we first show that step-by-step reasoning is crucial for vision-language models (VLMs) to solve CAPTCHAs, which represent high-difficulty spatial reasoning tasks, and that current commercial vision-language models still struggle with such reasoning. In particular, we observe that most commercial VLMs (e.g., Gemini, Claude, GPT, etc.) fail to effectively solve CAPTCHAs and thus achieve low accuracy (around 21.9 percent). However, our findings indicate that requiring the model to perform step-by-step reasoning before generating the final coordinates can significantly enhance its solving accuracy, underscoring the severity of the gap. To systematically study this issue, we introduce CAPTCHA-X, the first real-world CAPTCHA benchmark with reasoning, covering seven categories of CAPTCHAs (such as Gobang, hCaptcha, etc.) with step-by-step action solutions and grounding annotations. We further define five reasoning-oriented metrics that enable a comprehensive evaluation of models reasoning capabilities. To validate the effectiveness of reasoning, we also propose a general agentic VLM-based framework that incorporates the models inherent reasoning abilities. Our method achieves state-of-the-art performance across five high-difficulty CAPTCHA types, with an average solving accuracy of 83.9 percent, substantially surpassing existing baselines. These results reveal the limitations of current models and highlight the importance of reasoning in advancing visual-spatial challenges in the future.

Method: Transformer-based model trained on 35 1.5T and 108 3T T1w MRI paired with 7T T1 maps from MS patients, using mixed field strength training strategy.

Result: Achieved PSNR 26.0 dB, SSIM 0.861, NMSE 0.019 for 1.5T; 64% NMSE reduction vs ResShift, 41% vs ResViT with only 10.5M parameters. Mixed training outperformed single-field strategies.

Conclusion: Novel method successfully predicts 7T MP2RAGE maps from standard clinical scanners, making 7T benefits more accessible to routine clinical practice.

Abstract: Purpose: Ultra-high-field 7T MRI offers improved resolution and contrast over standard clinical field strengths (1.5T, 3T). However, 7T scanners are costly, scarce, and introduce additional challenges such as susceptibility artifacts. We propose an efficient transformer-based model (7T-Restormer) to synthesize 7T-quality T1-maps from routine 1.5T or 3T T1-weighted (T1W) images. Methods: Our model was validated on 35 1.5T and 108 3T T1w MRI paired with corresponding 7T T1 maps of patients with confirmed MS. A total of 141 patient cases (32,128 slices) were randomly divided into 105 (25; 80) training cases (19,204 slices), 19 (5; 14) validation cases (3,476 slices), and 17 (5; 14) test cases (3,145 slices) where (X; Y) denotes the patients with 1.5T and 3T T1W scans, respectively. The synthetic 7T T1 maps were compared against the ResViT and ResShift models. Results: The 7T-Restormer model achieved a PSNR of 26.0 +/- 4.6 dB, SSIM of 0.861 +/- 0.072, and NMSE of 0.019 +/- 0.011 for 1.5T inputs, and 25.9 +/- 4.9 dB, and 0.866 +/- 0.077 for 3T inputs, respectively. Using 10.5 M parameters, our model reduced NMSE by 64 % relative to 56.7M parameter ResShift (0.019 vs 0.052, p = <.001 and by 41 % relative to 70.4M parameter ResViT (0.019 vs 0.032, p = <.001) at 1.5T, with similar advantages at 3T (0.021 vs 0.060 and 0.033; p < .001). Training with a mixed 1.5 T + 3 T corpus was superior to single-field strategies. Restricting the model to 1.5T increased the 1.5T NMSE from 0.019 to 0.021 (p = 1.1E-3) while training solely on 3T resulted in lower performance on input 1.5T T1W MRI. Conclusion: We propose a novel method for predicting quantitative 7T MP2RAGE maps from 1.5T and 3T T1W scans with higher quality than existing state-of-the-art methods. Our approach makes the benefits of 7T MRI more accessible to standard clinical workflows.

Method: Uses dual-path structured reasoning traces (symbolic relation paths over text/vision with natural-language explanations) to provide stronger inductive bias, builds trace-enriched dataset via structure-aware self-distillation with a single MLLM.

Result: Achieves up to 11.3% higher answer accuracy on OK-VQA over strongest baseline, with improved transparency of intermediate reasoning.

Conclusion: Structured reasoning traces provide effective modality-aware scaffolds for IK-KVQA, enabling better accuracy and interpretability without external knowledge resources.

Abstract: Knowledge-based Visual Question Answering (KVQA) requires models to ground entities in images and reason over factual knowledge. Recent work has introduced its implicit-knowledge variant, IK-KVQA, where a multimodal large language model (MLLM) is the sole knowledge source and answers are produced without external retrieval. Existing IK-KVQA approaches, however, are typically trained with answer-only supervision: reasoning remains implicit, justifications are often weak or inconsistent, and generalization after standard supervised fine-tuning (SFT) can be brittle. We propose MODELNAME, a framework that equips IK-KVQA with dual-path structured reasoning traces (symbolic relation paths over text and vision together with path-grounded natural-language explanations) to provide a stronger inductive bias than generic answer-only supervision. These traces act as modality-aware scaffolds that guide the model toward relevant entities and attributes, offering more structure than generic chain-of-thought supervision while not constraining reasoning to any single fixed path. Using a single open-source MLLM, MODELNAME constructs and selects traces to build an offline trace-enriched dataset and then performs structure-aware self-distillation; no external retrievers, verifiers, or curated knowledge bases are used, and inference is a single autoregressive pass. Across benchmarks, MODELNAME consistently improves both answer accuracy and the transparency of intermediate reasoning, achieving up to 11.3% higher answer accuracy on OK-VQA over the strongest baseline.

Method: ThinkingViT uses progressive thinking stages that start with a small subset of attention heads, then iteratively activates larger subsets if confidence thresholds aren’t met. It employs Token Recycling to fuse previous embeddings with current inputs for better subsequent rounds.

Result: ThinkingViT outperforms nested baselines by up to 2.0 p.p. in accuracy at same throughput and up to 2.9 p.p. at equal GMACs on ImageNet-1K. It also transfers effectively to other architectures like Swin and serves as plug-in upgrade for downstream tasks.

Conclusion: ThinkingViT provides an efficient backbone-preserving design for adaptive computation in Vision Transformers, enabling scalable deployment across heterogeneous hardware while maintaining performance across various tasks and architectures.

Abstract: ViTs deliver SOTA performance, yet their fixed computational budget prevents scalable deployment across heterogeneous hardware. Recent Matryoshka-style Transformer architectures mitigate this by embedding nested subnetworks within a single model to enable scalable inference. However, these models allocate the same amount of compute to all inputs, regardless of their complexity, which leads to inefficiencies. To address this, we introduce ThinkingViT, a nested ViT architecture that employs progressive thinking stages to dynamically adjust inference computation based on input difficulty. ThinkingViT first activates a small subset of the most important attention heads to produce an initial prediction. If the prediction confidence exceeds a predefined threshold, inference terminates early. Otherwise, within the same backbone, it activates a larger subset of attention heads and conducts a new forward pass. This process continues iteratively until the model reaches the predefined confidence level or exhausts its maximum capacity. To boost the performance of subsequent rounds, we introduce a Token Recycling approach that fuses the input embeddings with the embeddings from the previous stage. Experiments show that ThinkingViT surpasses nested baselines by up to 2.0 percentage points (p.p.) in accuracy at the same throughput and by up to 2.9 p.p. at equal GMACs on ImageNet-1K. We show that the backbone-preserving design of ThinkingViT allows it to serve as a plug-in upgrade for ViTs in downstream tasks such as semantic segmentation. We also demonstrate that ThinkingViT transfers effectively to other architectures such as Swin. The source code is available at https://github.com/ds-kiel/ThinkingViT.

Method: SCALAR uses a pretrained image encoder to extract semantic control signal encodings, projects them into scale-specific representations, and injects them into corresponding layers of the VAR backbone through a Scale-wise Conditional Decoding mechanism. SCALAR-Uni extends this to align multiple control modalities in a shared latent space.

Result: Extensive experiments show that SCALAR achieves superior generation quality and control precision across various tasks compared to existing methods.

Conclusion: SCALAR provides persistent and structurally aligned guidance throughout the generation process, enabling efficient and precise controllable image synthesis with VAR models, with SCALAR-Uni supporting flexible multi-conditional guidance in a single model.

Abstract: Controllable image synthesis, which enables fine-grained control over generated outputs, has emerged as a key focus in visual generative modeling. However, controllable generation remains challenging for Visual Autoregressive (VAR) models due to their hierarchical, next-scale prediction style. Existing VAR-based methods often suffer from inefficient control encoding and disruptive injection mechanisms that compromise both fidelity and efficiency. In this work, we present SCALAR, a controllable generation method based on VAR, incorporating a novel Scale-wise Conditional Decoding mechanism. SCALAR leverages a pretrained image encoder to extract semantic control signal encodings, which are projected into scale-specific representations and injected into the corresponding layers of the VAR backbone. This design provides persistent and structurally aligned guidance throughout the generation process. Building on SCALAR, we develop SCALAR-Uni, a unified extension that aligns multiple control modalities into a shared latent space, supporting flexible multi-conditional guidance in a single model. Extensive experiments show that SCALAR achieves superior generation quality and control precision across various tasks. The code is released at https://github.com/AMAP-ML/SCALAR.

Method: Uses diffusion models and large language models to optimize text prompts that synthesize class-conditional images strongly activating target classifiers, then generates natural language explanations from these synthetic samples to describe decision patterns and biases.

Result: Outperforms existing approaches in global model explanation and class-level bias reporting on datasets including ImageNet, Waterbirds, CelebA, and FairFaces. Produces accurate, interpretable outputs validated through quantitative/qualitative evaluations and user study.

Conclusion: DEXTER provides an effective data-free framework for generating natural language explanations of visual classifiers’ decision processes, enabling better model transparency and bias understanding without requiring original training data.

Abstract: Understanding and explaining the behavior of machine learning models is essential for building transparent and trustworthy AI systems. We introduce DEXTER, a data-free framework that employs diffusion models and large language models to generate global, textual explanations of visual classifiers. DEXTER operates by optimizing text prompts to synthesize class-conditional images that strongly activate a target classifier. These synthetic samples are then used to elicit detailed natural language reports that describe class-specific decision patterns and biases. Unlike prior work, DEXTER enables natural language explanation about a classifier’s decision process without access to training data or ground-truth labels. We demonstrate DEXTER’s flexibility across three tasks-activation maximization, slice discovery and debiasing, and bias explanation-each illustrating its ability to uncover the internal mechanisms of visual classifiers. Quantitative and qualitative evaluations, including a user study, show that DEXTER produces accurate, interpretable outputs. Experiments on ImageNet, Waterbirds, CelebA, and FairFaces confirm that DEXTER outperforms existing approaches in global model explanation and class-level bias reporting. Code is available at https://github.com/perceivelab/dexter.

Method: Uses a single lightweight core block that can be re-applied multiple times with a score-based selector determining whether further passes benefit each feature map region, implementing module reuse and mild regularization on selection scores.

Result: Attains competitive accuracy on standard benchmarks with reduced parameters, lower floating-point operations, and faster inference compared to similarly sized baselines, while transferring well to multiple datasets without task-specific customization.

Conclusion: The multi-pass strategy with content-conditioned processing provides an effective approach for computation-efficient image categorization while maintaining competitive performance and architectural minimalism.

Abstract: We present a compact encoder for image categorization that emphasizes computation economy through content-conditioned multi-pass processing. The model employs a single lightweight core block that can be re-applied a small number of times, while a simple score-based selector decides whether further passes are beneficial for each region unit in the feature map. This design provides input-conditioned depth without introducing heavy auxiliary modules or specialized pretraining. On standard benchmarks, the approach attains competitive accuracy with reduced parameters, lower floating-point operations, and faster inference compared to similarly sized baselines. The method keeps the architecture minimal, implements module reuse to control footprint, and preserves stable training via mild regularization on selection scores. We discuss implementation choices for efficient masking, pass control, and representation caching, and show that the multi-pass strategy transfers well to several datasets without requiring task-specific customization.

Method: Proposes Range-Adaptive Perception module with learnable queries modulated by ego vehicle states, State-Conditioned Forecasting module using regression instead of classification, and Temporal-Aware Self-Scheduling training strategy.

Result: Achieves state-of-the-art performance in perception, forecasting, and planning tasks, with advantages in flexibility, adaptability, and efficiency demonstrated through extensive experiments and visualizations.

Conclusion: SparseWorld successfully addresses limitations of static occupancy models by introducing dynamic query-based approach that better aligns with continuous 4D environments, enabling more effective world modeling for autonomous systems.

Abstract: Semantic occupancy has emerged as a powerful representation in world models for its ability to capture rich spatial semantics. However, most existing occupancy world models rely on static and fixed embeddings or grids, which inherently limit the flexibility of perception. Moreover, their ``in-place classification" over grids exhibits a potential misalignment with the dynamic and continuous nature of real scenarios. In this paper, we propose SparseWorld, a novel 4D occupancy world model that is flexible, adaptive, and efficient, powered by sparse and dynamic queries. We propose a Range-Adaptive Perception module, in which learnable queries are modulated by the ego vehicle states and enriched with temporal-spatial associations to enable extended-range perception. To effectively capture the dynamics of the scene, we design a State-Conditioned Forecasting module, which replaces classification-based forecasting with regression-guided formulation, precisely aligning the dynamic queries with the continuity of the 4D environment. In addition, We specifically devise a Temporal-Aware Self-Scheduling training strategy to enable smooth and efficient training. Extensive experiments demonstrate that SparseWorld achieves state-of-the-art performance across perception, forecasting, and planning tasks. Comprehensive visualizations and ablation studies further validate the advantages of SparseWorld in terms of flexibility, adaptability, and efficiency.

Method: Proposes a pure 2D framework with height-score projection to encode vertical structure, and direction-aware convolution to extract geometric features along vertical and horizontal orientations.

Result: Achieves 39.3% mIoU on Occ3D-nuScenes with 27.7 FPS inference speed, and 14.8 FPS on edge devices, demonstrating real-time capability.

Conclusion: The method effectively balances accuracy and efficiency, making it suitable for real-time deployment in resource-constrained autonomous driving systems.

Abstract: Efficient and high-accuracy 3D occupancy prediction is crucial for ensuring the performance of autonomous driving (AD) systems. However, many existing methods involve trade-offs between accuracy and efficiency. Some achieve high precision but with slow inference speed, while others adopt purely bird’s-eye-view (BEV)-based 2D representations to accelerate processing, inevitably sacrificing vertical cues and compromising geometric integrity. To overcome these limitations, we propose a pure 2D framework that achieves efficient 3D occupancy prediction while preserving geometric integrity. Unlike conventional Lift-Splat-Shoot (LSS) methods that rely solely on depth scores to lift 2D features into 3D space, our approach additionally introduces a height-score projection to encode vertical geometric structure. We further employ direction-aware convolution to extract geometric features along both vertical and horizontal orientations, effectively balancing accuracy and computational efficiency. On the Occ3D-nuScenes, the proposed method achieves an mIoU of 39.3% and an inference speed of 27.7 FPS, effectively balancing accuracy and efficiency. In simulations on edge devices, the inference speed reaches 14.8 FPS, further demonstrating the method’s applicability for real-time deployment in resource-constrained environments.

Method: Developed AniMer+ with family-aware Vision Transformer using Mixture-of-Experts design that partitions layers into taxa-specific and shared components. Created diffusion-based synthetic datasets (CtrlAni3D for quadrupeds, CtrlAVES3D for birds) to overcome 3D data shortage.

Result: Superior performance over existing approaches across benchmarks including challenging Animal Kingdom dataset, trained on 41.3k mammalian and 12.4k avian images (real + synthetic). Ablation studies confirm effectiveness of architecture and synthetic data.

Conclusion: AniMer+ successfully enables unified reconstruction of mammals and birds through innovative network design and synthetic data generation, demonstrating strong performance on diverse benchmarks and real-world applications.

Abstract: In the era of foundation models, achieving a unified understanding of different dynamic objects through a single network has the potential to empower stronger spatial intelligence. Moreover, accurate estimation of animal pose and shape across diverse species is essential for quantitative analysis in biological research. However, this topic remains underexplored due to the limited network capacity of previous methods and the scarcity of comprehensive multi-species datasets. To address these limitations, we introduce AniMer+, an extended version of our scalable AniMer framework. In this paper, we focus on a unified approach for reconstructing mammals (mammalia) and birds (aves). A key innovation of AniMer+ is its high-capacity, family-aware Vision Transformer (ViT) incorporating a Mixture-of-Experts (MoE) design. Its architecture partitions network layers into taxa-specific components (for mammalia and aves) and taxa-shared components, enabling efficient learning of both distinct and common anatomical features within a single model. To overcome the critical shortage of 3D training data, especially for birds, we introduce a diffusion-based conditional image generation pipeline. This pipeline produces two large-scale synthetic datasets: CtrlAni3D for quadrupeds and CtrlAVES3D for birds. To note, CtrlAVES3D is the first large-scale, 3D-annotated dataset for birds, which is crucial for resolving single-view depth ambiguities. Trained on an aggregated collection of 41.3k mammalian and 12.4k avian images (combining real and synthetic data), our method demonstrates superior performance over existing approaches across a wide range of benchmarks, including the challenging out-of-domain Animal Kingdom dataset. Ablation studies confirm the effectiveness of both our novel network architecture and the generated synthetic datasets in enhancing real-world application performance.

Method: Proposed NS-Net framework with NULL-Space projection to decouple semantic information from CLIP’s visual features, contrastive learning to capture intrinsic distributional differences, and Patch Selection strategy to preserve fine-grained artifacts by mitigating semantic bias from global image structures.

Result: Extensive experiments on 40 diverse generative models show NS-Net outperforms state-of-the-art methods with 7.4% improvement in detection accuracy, demonstrating strong generalization across both GAN- and diffusion-based image generation techniques.

Conclusion: NS-Net effectively addresses the generalization problem in AI-generated image detection by removing semantic bias from CLIP features and capturing intrinsic distributional differences, achieving superior performance across diverse generative models.

Abstract: The rapid progress of generative models, such as GANs and diffusion models, has facilitated the creation of highly realistic images, raising growing concerns over their misuse in security-sensitive domains. While existing detectors perform well under known generative settings, they often fail to generalize to unknown generative models, especially when semantic content between real and fake images is closely aligned. In this paper, we revisit the use of CLIP features for AI-generated image detection and uncover a critical limitation: the high-level semantic information embedded in CLIP’s visual features hinders effective discrimination. To address this, we propose NS-Net, a novel detection framework that leverages NULL-Space projection to decouple semantic information from CLIP’s visual features, followed by contrastive learning to capture intrinsic distributional differences between real and generated images. Furthermore, we design a Patch Selection strategy to preserve fine-grained artifacts by mitigating semantic bias caused by global image structures. Extensive experiments on an open-world benchmark comprising images generated by 40 diverse generative models show that NS-Net outperforms existing state-of-the-art methods, achieving a 7.4% improvement in detection accuracy, thereby demonstrating strong generalization across both GAN- and diffusion-based image generation techniques.

Method: Uses instruction-editing datasets as hard negative image-text pairs with symmetric contrastive loss, and incorporates long descriptive captions with rotary positional encodings to capture rich semantic context.

Result: Achieves substantial gains on MMVP benchmark and fine-grained visual recognition tasks without compromising zero-shot performance, and reduces visual hallucinations in Multimodal Large Language Models.

Conclusion: Synergizing targeted instruction-based contrastive learning with comprehensive descriptive information significantly elevates fine-grained understanding in Vision-Language Models.

Abstract: Despite the success of Vision-Language Models (VLMs) like CLIP in aligning vision and language, their proficiency in detailed, fine-grained visual comprehension remains a key challenge. We present CLIP-IN, a novel framework that bolsters CLIP’s fine-grained perception through two core innovations. Firstly, we leverage instruction-editing datasets, originally designed for image manipulation, as a unique source of hard negative image-text pairs. Coupled with a symmetric hard negative contrastive loss, this enables the model to effectively distinguish subtle visual-semantic differences. Secondly, CLIP-IN incorporates long descriptive captions, utilizing rotary positional encodings to capture rich semantic context often missed by standard CLIP. Our experiments demonstrate that CLIP-IN achieves substantial gains on the MMVP benchmark and various fine-grained visual recognition tasks, without compromising robust zero-shot performance on broader classification and retrieval tasks. Critically, integrating CLIP-IN’s visual representations into Multimodal Large Language Models significantly reduces visual hallucinations and enhances reasoning abilities. This work underscores the considerable potential of synergizing targeted, instruction-based contrastive learning with comprehensive descriptive information to elevate the fine-grained understanding of VLMs.

Method: Proposes MonoDream framework with Unified Navigation Representation (UNR) and Latent Panoramic Dreaming (LPD) tasks that predict latent features of panoramic RGB-D observations from monocular input.

Result: Consistently improves monocular navigation performance across multiple VLN benchmarks and significantly narrows the gap with panoramic-based agents.

Conclusion: MonoDream demonstrates that lightweight VLA frameworks can effectively enable monocular agents to achieve competitive navigation performance without requiring panoramic RGB-D sensors.

Abstract: Vision-Language Navigation (VLN) tasks often leverage panoramic RGB and depth inputs to provide rich spatial cues for action planning, but these sensors can be costly or less accessible in real-world deployments. Recent approaches based on Vision-Language Action (VLA) models achieve strong results with monocular input, yet they still lag behind methods using panoramic RGB-D information. We present MonoDream, a lightweight VLA framework that enables monocular agents to learn a Unified Navigation Representation (UNR). This shared feature representation jointly aligns navigation-relevant visual semantics (e.g., global layout, depth, and future cues) and language-grounded action intent, enabling more reliable action prediction. MonoDream further introduces Latent Panoramic Dreaming (LPD) tasks to supervise the UNR, which train the model to predict latent features of panoramic RGB and depth observations at both current and future steps based on only monocular input. Experiments on multiple VLN benchmarks show that MonoDream consistently improves monocular navigation performance and significantly narrows the gap with panoramic-based agents.

Method: Part-based decomposition separates avatar into body, face, hands, and clothing components, with dedicated cloth simulation module using temporal motion cues and geometric constraints for garment deformation.

Result: MonoCloth improves both visual reconstruction quality and animation realism compared to existing methods, and supports additional tasks like clothing transfer.

Conclusion: The part-based design enables versatile and practical avatar reconstruction from monocular videos with enhanced realism and additional functionality.

Abstract: Reconstructing realistic 3D human avatars from monocular videos is a challenging task due to the limited geometric information and complex non-rigid motion involved. We present MonoCloth, a new method for reconstructing and animating clothed human avatars from monocular videos. To overcome the limitations of monocular input, we introduce a part-based decomposition strategy that separates the avatar into body, face, hands, and clothing. This design reflects the varying levels of reconstruction difficulty and deformation complexity across these components. Specifically, we focus on detailed geometry recovery for the face and hands. For clothing, we propose a dedicated cloth simulation module that captures garment deformation using temporal motion cues and geometric constraints. Experimental results demonstrate that MonoCloth improves both visual reconstruction quality and animation realism compared to existing methods. Furthermore, thanks to its part-based design, MonoCloth also supports additional tasks such as clothing transfer, underscoring its versatility and practical utility.

Method: MMEdge decomposes inference into fine-grained sensing/encoding units for incremental computation, uses temporal aggregation to capture dynamics, incorporates adaptive configuration optimizer for resource constraints, and employs cross-modal speculative skipping for early decision-making.

Result: Evaluation on multimodal datasets and UAV testbed shows MMEdge significantly reduces end-to-end latency while maintaining high task accuracy across various system and data dynamics.

Conclusion: MMEdge provides an effective pipelined approach for real-time multimodal inference on edge devices, achieving both low latency and high accuracy through its incremental processing and adaptive optimization techniques.

Abstract: Real-time multimodal inference on resource-constrained edge devices is essential for applications such as autonomous driving, human-computer interaction, and mobile health. However, prior work often overlooks the tight coupling between sensing dynamics and model execution, as well as the complex inter-modality dependencies. In this paper, we propose MMEdge, an new on-device multi-modal inference framework based on pipelined sensing and encoding. Instead of waiting for complete sensor inputs, MMEdge decomposes the entire inference process into a sequence of fine-grained sensing and encoding units, allowing computation to proceed incrementally as data arrive. MMEdge also introduces a lightweight but effective temporal aggregation module that captures rich temporal dynamics across different pipelined units to maintain accuracy performance. Such pipelined design also opens up opportunities for fine-grained cross-modal optimization and early decision-making during inference. To further enhance system performance under resource variability and input data complexity, MMEdge incorporates an adaptive multimodal configuration optimizer that dynamically selects optimal sensing and model configurations for each modality under latency constraints, and a cross-modal speculative skipping mechanism that bypasses future units of slower modalities when early predictions reach sufficient confidence. We evaluate MMEdge using two public multimodal datasets and deploy it on a real-world unmanned aerial vehicle (UAV)-based multimodal testbed. The results show that MMEdge significantly reduces end-to-end latency while maintaining high task accuracy across various system and data dynamics.

Method: Two-stage architecture: 1) Conditional graph VAE learns canonical T-pose priors, autoencoder encodes motion into shared latent space with morphological loss; 2) Masked motion modeling generates motion embeddings conditioned on text descriptions with morphological consistency module.

Result: Outperforms state-of-the-art methods on both seen and unseen species, as demonstrated through extensive experiments on the UniMo4D dataset.

Conclusion: X-MoGen successfully addresses cross-species motion generation challenges and provides a unified framework that generalizes well across different species.

Abstract: Text-driven motion generation has attracted increasing attention due to its broad applications in virtual reality, animation, and robotics. While existing methods typically model human and animal motion separately, a joint cross-species approach offers key advantages, such as a unified representation and improved generalization. However, morphological differences across species remain a key challenge, often compromising motion plausibility. To address this, we propose X-MoGen, the first unified framework for cross-species text-driven motion generation covering both humans and animals. X-MoGen adopts a two-stage architecture. First, a conditional graph variational autoencoder learns canonical T-pose priors, while an autoencoder encodes motion into a shared latent space regularized by morphological loss. In the second stage, we perform masked motion modeling to generate motion embeddings conditioned on textual descriptions. During training, a morphological consistency module is employed to promote skeletal plausibility across species. To support unified modeling, we construct UniMo4D, a large-scale dataset of 115 species and 119k motion sequences, which integrates human and animal motions under a shared skeletal topology for joint training. Extensive experiments on UniMo4D demonstrate that X-MoGen outperforms state-of-the-art methods on both seen and unseen species.

Method: Integrates rectified flow for accelerated generation and introduces a novel region-specific contrastive loss to enhance sensitivity to regions of interest, improving condition fidelity.

Result: Achieves state-of-the-art image quality with 33x acceleration for latent diffusion models, and synthetic images can be effectively used for data augmentation in downstream segmentation tasks.

Conclusion: MAISI-v2 provides a fast, high-quality medical image synthesis solution with improved condition consistency, and the released code and resources will facilitate further development in the community.

Abstract: Medical image synthesis is an important topic for both clinical and research applications. Recently, diffusion models have become a leading approach in this area. Despite their strengths, many existing methods struggle with (1) limited generalizability that only work for specific body regions or voxel spacings, (2) slow inference, which is a common issue for diffusion models, and (3) weak alignment with input conditions, which is a critical issue for medical imaging. MAISI, a previously proposed framework, addresses generalizability issues but still suffers from slow inference and limited condition consistency. In this work, we present MAISI-v2, the first accelerated 3D medical image synthesis framework that integrates rectified flow to enable fast and high quality generation. To further enhance condition fidelity, we introduce a novel region-specific contrastive loss to enhance the sensitivity to region of interest. Our experiments show that MAISI-v2 can achieve SOTA image quality with $33 \times$ acceleration for latent diffusion model. We also conducted a downstream segmentation experiment to show that the synthetic images can be used for data augmentation. We release our code, training details, model weights, and a GUI demo to facilitate reproducibility and promote further development within the community.

Method: Evaluated UNet-based architectures with various backbones (ResNet, Transformer, Mamba) pretrained, then incorporated attention mechanisms including CBAM, and used Grad-CAM for interpretability.

Result: ResNet-based models consistently outperformed alternatives. ResNetUNet3+ with CBAM achieved best metrics: Dice 0.755, IoU 0.662, HD95 77.911, accuracy 0.925, specificity 0.926.

Conclusion: Classical ResNet architecture combined with modern attention modules remains highly competitive for medical image segmentation, offering promising direction for liver tumor detection in clinical practice.

Abstract: Segmentation of liver structures in multi-phase contrast-enhanced computed tomography (CECT) plays a crucial role in computer-aided diagnosis and treatment planning for liver diseases, including tumor detection. In this study, we investigate the performance of UNet-based architectures for liver tumor segmentation, starting from the original UNet and extending to UNet3+ with various backbone networks. We evaluate ResNet, Transformer-based, and State-space (Mamba) backbones, all initialized with pretrained weights. Surprisingly, despite the advances in modern architecture, ResNet-based models consistently outperform Transformer- and Mamba-based alternatives across multiple evaluation metrics. To further improve segmentation quality, we introduce attention mechanisms into the backbone and observe that incorporating the Convolutional Block Attention Module (CBAM) yields the best performance. ResNetUNet3+ with CBAM module not only produced the best overlap metrics with a Dice score of 0.755 and IoU of 0.662, but also achieved the most precise boundary delineation, evidenced by the lowest HD95 distance of 77.911. The model’s superiority was further cemented by its leading overall accuracy of 0.925 and specificity of 0.926, showcasing its robust capability in accurately identifying both lesion and healthy tissue. To further enhance interpretability, Grad-CAM visualizations were employed to highlight the region’s most influential predictions, providing insights into its decision-making process. These findings demonstrate that classical ResNet architecture, when combined with modern attention modules, remain highly competitive for medical image segmentation tasks, offering a promising direction for liver tumor detection in clinical practice.

Method: Proposes Multi-Category Contrastive Learning (MCCL) for intra- and inter-category alignment and separation, combined with Feature Synergy Structure (FSS) for discriminative feature reconstruction through prior fusion and semantic-activation-map enhancement.

Result: Extensive experiments show SynSeg outperforms state-of-the-art methods with 6.9-26.2% higher accuracy, improving semantic localization and discrimination abilities under weak supervision while enabling real-time inference.

Conclusion: SynSeg provides an effective lightweight end-to-end solution for open-vocabulary semantic segmentation that avoids foreground bias and achieves superior performance without relying on large-scale pretrained models.

Abstract: Semantic segmentation in open-vocabulary scenarios presents significant challenges due to the wide range and granularity of semantic categories. Existing weakly-supervised methods often rely on category-specific supervision and ill-suited feature construction methods for contrastive learning, leading to semantic misalignment and poor performance. In this work, we propose a novel weakly-supervised approach, SynSeg, to address the challenges. SynSeg performs Multi-Category Contrastive Learning (MCCL) as a stronger training signal with a new feature reconstruction framework named Feature Synergy Structure (FSS). Specifically, MCCL strategy robustly combines both intra- and inter-category alignment and separation in order to make the model learn the knowledge of correlations from different categories within the same image. Moreover, FSS reconstructs discriminative features for contrastive learning through prior fusion and semantic-activation-map enhancement, effectively avoiding the foreground bias introduced by the visual encoder. Furthermore, SynSeg is a lightweight end-to-end solution without using any mid-term output from large-scale pretrained models and capable for real-time inference. In general, SynSeg effectively improves the abilities in semantic localization and discrimination under weak supervision in an efficient manner. Extensive experiments on benchmarks demonstrate that our method outperforms state-of-the-art (SOTA) performance. Particularly, SynSeg achieves higher accuracy than SOTA baselines with a ratio from 6.9% up to 26.2%.

Method: Formulates cache scheduling as a directed graph with error-weighted edges and introduces Lexicographic Minimax Path Optimization to explicitly bound worst-case path error, improving global content and style consistency.

Result: Achieves 2.9x speedup on Latte model and LPIPS score of 0.05 on Open-Sora, outperforming prior caching techniques with minimal perceptual quality degradation.

Conclusion: LeMiCa provides a robust and generalizable paradigm for accelerating diffusion-based video generation while maintaining quality, serving as a foundation for future efficient video synthesis research.

Abstract: We present LeMiCa, a training-free and efficient acceleration framework for diffusion-based video generation. While existing caching strategies primarily focus on reducing local heuristic errors, they often overlook the accumulation of global errors, leading to noticeable content degradation between accelerated and original videos. To address this issue, we formulate cache scheduling as a directed graph with error-weighted edges and introduce a Lexicographic Minimax Path Optimization strategy that explicitly bounds the worst-case path error. This approach substantially improves the consistency of global content and style across generated frames. Extensive experiments on multiple text-to-video benchmarks demonstrate that LeMiCa delivers dual improvements in both inference speed and generation quality. Notably, our method achieves a 2.9x speedup on the Latte model and reaches an LPIPS score of 0.05 on Open-Sora, outperforming prior caching techniques. Importantly, these gains come with minimal perceptual quality degradation, making LeMiCa a robust and generalizable paradigm for accelerating diffusion-based video generation. We believe this approach can serve as a strong foundation for future research on efficient and reliable video synthesis. Our code is available at :https://github.com/UnicomAI/LeMiCa

Method: Parameter-efficient adaptation of pre-trained vision encoders by fine-tuning only Layer Normalization parameters, enforcing hyperspherical feature manifold using L2 normalization and metric learning.

Result: Achieves state-of-the-art performance across 14 benchmark datasets (2019-2025), outperforming more complex approaches in average cross-dataset AUROC. Key findings: training on paired real-fake data is essential, and detection difficulty hasn’t strictly increased over time.

Conclusion: State-of-the-art generalization in deepfake detection is achievable through minimal, targeted changes to pre-trained foundational models, providing a computationally efficient and reproducible solution.

Abstract: The generalization of deepfake detectors to unseen manipulation techniques remains a challenge for practical deployment. Although many approaches adapt foundation models by introducing significant architectural complexity, this work demonstrates that robust generalization is achievable through a parameter-efficient adaptation of one of the foundational pre-trained vision encoders. The proposed method, GenD, fine-tunes only the Layer Normalization parameters (0.03% of the total) and enhances generalization by enforcing a hyperspherical feature manifold using L2 normalization and metric learning on it. We conducted an extensive evaluation on 14 benchmark datasets spanning from 2019 to 2025. The proposed method achieves state-of-the-art performance, outperforming more complex, recent approaches in average cross-dataset AUROC. Our analysis yields two primary findings for the field: 1) training on paired real-fake data from the same source video is essential for mitigating shortcut learning and improving generalization, and 2) detection difficulty on academic datasets has not strictly increased over time, with models trained on older, diverse datasets showing strong generalization capabilities. This work delivers a computationally efficient and reproducible method, proving that state-of-the-art generalization is attainable by making targeted, minimal changes to a pre-trained foundational image encoder model. The code is at: https://github.com/yermandy/GenD

Method: Uses dual-branch parallel tracker with affine compensation: detection/ReID branch with RCDN for object detection and Li-TAE for appearance features, plus camera motion estimation branch that decouples platform and target motion using projective transformation and Kalman filter compensation.

Result: Achieves state-of-the-art performance on Singapore Maritime Dataset with fastest runtime among ReID-based MOT frameworks, maintaining high identity consistency and robustness to jitter/occlusion.

Conclusion: DMSORT effectively addresses maritime MOT challenges through its dual-branch architecture and motion compensation, providing efficient and robust tracking suitable for marine applications.

Abstract: Accurate perception of the marine environment through robust multi-object tracking (MOT) is essential for ensuring safe vessel navigation and effective maritime surveillance. However, the complicated maritime environment often causes camera motion and subsequent visual degradation, posing significant challenges to MOT. To address this challenge, we propose an efficient Dual-branch Maritime SORT (DMSORT) method for maritime MOT. The core of the framework is a parallel tracker with affine compensation, which incorporates an object detection and re-identification (ReID) branch, along with a dedicated branch for dynamic camera motion estimation. Specifically, a Reversible Columnar Detection Network (RCDN) is integrated into the detection module to leverage multi-level visual features for robust object detection. Furthermore, a lightweight Transformer-based appearance extractor (Li-TAE) is designed to capture global contextual information and generate robust appearance features. Another branch decouples platform-induced and target-intrinsic motion by constructing a projective transformation, applying platform-motion compensation within the Kalman filter, and thereby stabilizing true object trajectories. Finally, a clustering-optimized feature fusion module effectively combines motion and appearance cues to ensure identity consistency under noise, occlusion, and drift. Extensive evaluations on the Singapore Maritime Dataset demonstrate that DMSORT achieves state-of-the-art performance. Notably, DMSORT attains the fastest runtime among existing ReID-based MOT frameworks while maintaining high identity consistency and robustness to jitter and occlusion. Code is available at: https://github.com/BiscuitsLzy/DMSORT-An-efficient-parallel-maritime-multi-object-tracking-architecture-.

Method: Proposed end-to-end spatio-temporal reasoning framework with instance-aware feature encoding, time/camera encoding, and spatially adaptive down-sampling to compress 4D scenes into LLM-manageable token sequences.

Result: Experiments show the method consistently outperforms baselines on the EgoDynamic4D benchmark, validating effectiveness of multimodal temporal modeling.

Conclusion: The work addresses the gap in dynamic 4D scene understanding through a comprehensive benchmark and effective reasoning framework, advancing egocentric scene analysis capabilities.

Abstract: Understanding dynamic 4D scenes from an egocentric perspective-modeling changes in 3D spatial structure over time-is crucial for human-machine interaction, autonomous navigation, and embodied intelligence. While existing egocentric datasets contain dynamic scenes, they lack unified 4D annotations and task-driven evaluation protocols for fine-grained spatio-temporal reasoning, especially on motion of objects and human, together with their interactions. To address this gap, we introduce EgoDynamic4D, a novel QA benchmark on highly dynamic scenes, comprising RGB-D video, camera poses, globally unique instance masks, and 4D bounding boxes. We construct 927K QA pairs accompanied by explicit Chain-of-Thought (CoT), enabling verifiable, step-by-step spatio-temporal reasoning. We design 12 dynamic QA tasks covering agent motion, human-object interaction, trajectory prediction, relation understanding, and temporal-causal reasoning, with fine-grained, multidimensional metrics. To tackle these tasks, we propose an end-to-end spatio-temporal reasoning framework that unifies dynamic and static scene information, using instance-aware feature encoding, time and camera encoding, and spatially adaptive down-sampling to compress large 4D scenes into token sequences manageable by LLMs. Experiments on EgoDynamic4D show that our method consistently outperforms baselines, validating the effectiveness of multimodal temporal modeling for egocentric dynamic scene understanding.

Method: The framework operates in the latent domain of diffusion models, re-simulating partial forward diffusion to expose generator-specific cues. It uses a temporal attention encoder for multi-step latent feature aggregation and a feature-weighted prototype head for transparent attribution with closed-set classification and open-set evaluation.

Result: Proto-LeakNet achieves 98.13% Macro AUC, learns robust latent geometry that withstands post-processing, outperforms state-of-the-art methods, and shows strong separability between real images, known generators, and unseen generators.

Conclusion: The proposed framework effectively addresses source attribution challenges for diffusion models by leveraging signal-leak analysis and interpretable prototype-based learning, providing robust performance without requiring retraining for unseen generators.

Abstract: The growing sophistication of synthetic image and deepfake generation models has turned source attribution and authenticity verification into a critical challenge for modern computer vision systems. Recent studies suggest that diffusion pipelines unintentionally imprint persistent statistical traces, known as signal-leaks, within their outputs, particularly in latent representations. Building on this observation, we propose Proto-LeakNet, a signal-leak-aware and interpretable attribution framework that integrates closed-set classification with a density-based open-set evaluation on the learned embeddings, enabling analysis of unseen generators without retraining. Acting in the latent domain of diffusion models, our method re-simulates partial forward diffusion to expose residual generator-specific cues. A temporal attention encoder aggregates multi-step latent features, while a feature-weighted prototype head structures the embedding space and enables transparent attribution. Trained solely on closed data and achieving a Macro AUC of 98.13%, Proto-LeakNet learns a latent geometry that remains robust under post-processing, surpassing state-of-the-art methods, and achieves strong separability both between real images and known generators, and between known and unseen ones. The codebase will be available after acceptance.

Method: The challenge provides paired 3D ceT1 MRI, T2 MRI, and post-resection 3D iUS volumes from 114 cases (99 training, 5 validation, 10 test) without annotations for training, with validation and test evaluated on manually annotated landmarks.

Result: Performance is evaluated using target registration error (TRE), robustness to worst-case landmark misalignment (TRE30), and runtime metrics.

Conclusion: The ReMIND2Reg benchmark aims to accelerate development of robust, generalizable multimodal registration algorithms for clinically deployable image-guided neurosurgery.

Abstract: Accurate intraoperative image guidance is critical for achieving maximal safe resection in brain tumor surgery, yet neuronavigation systems based on preoperative MRI lose accuracy during the procedure due to brain shift. Aligning post-resection intraoperative ultrasound (iUS) with preoperative MRI can restore spatial accuracy by estimating brain shift deformations, but it remains a challenging problem given the large anatomical and topological changes and substantial modality intensity gap. The ReMIND2Reg 2025 Challenge provides the largest public benchmark for this task, built upon the ReMIND dataset. It offers 99 training cases, 5 validation cases, and 10 private test cases comprising paired 3D ceT1 MRI, T2 MRI, and post-resection 3D iUS volumes. Data are provided without annotations for training, while validation and test performance are evaluated on manually annotated anatomical landmarks. Metrics include target registration error (TRE), robustness to worst-case landmark misalignment (TRE30), and runtime. By establishing a standardized evaluation framework for this clinically critical and technically complex problem, ReMIND2Reg aims to accelerate the development of robust, generalizable, and clinically deployable multimodal registration algorithms for image-guided neurosurgery.

Method: ViMoNet uses joint training with both detailed motion-text data and comprehensive video-text data, and introduces the VIMOS dataset with films, motion sequences, instructions, and subtitles.

Result: ViMoNet outperforms existing methods in caption generation, motion understanding, and behavior interpretation, as shown through tests on the ViMoNet-Bench benchmark.

Conclusion: Combining video and motion data through joint training enables better understanding of human behavior, and ViMoNet provides an effective framework for this purpose.

Abstract: This study investigates how large language models (LLMs) can be used to understand human behavior using motion and video data. We think that mixing both types is essential to completely capture the nuanced movements and meanings of human actions, in contrast to recent models that simply concentrate on motion data or films. To address this, we provide ViMoNet, a straightforward yet effective framework for comprehending, characterizing, and deducing human action. ViMoNet employs a joint training strategy that leverages the advantages of two data types: detailed motion-text data, which is more exact, and generic video-text data, which is more comprehensive but less detailed. This aids in the model’s acquisition of rich data regarding time and space in human behavior. Additionally, we provide a brand new dataset named VIMOS that contains a variety of films, motion sequences, instructions, and subtitles. We developed ViMoNet-Bench, a standardized benchmark with carefully labeled samples, to evaluate how well models understand human behavior. Our tests show that ViMoNet outperforms existing methods in caption generation, motion understanding, and behavior interpretation.

Method: Introduces LLMC+ benchmark with over 20 algorithms across five VLM families, supporting systematic study of token-level and model-level compression through a plug-and-play toolkit.

Result: Reveals that spatial and temporal redundancies require distinct strategies, token reduction methods degrade in multi-turn dialogue and detail-sensitive tasks, and combining token and model compression achieves extreme compression with minimal performance loss.

Conclusion: LLMC+ facilitates fair evaluation and inspires future research in efficient VLM compression, providing a comprehensive framework for studying compression techniques.

Abstract: Large Vision-Language Models (VLMs) exhibit impressive multi-modal capabilities but suffer from prohibitive computational and memory demands, due to their long visual token sequences and massive parameter sizes. To address these issues, recent works have proposed training-free compression methods. However, existing efforts often suffer from three major limitations: (1) Current approaches do not decompose techniques into comparable modules, hindering fair evaluation across spatial and temporal redundancy. (2) Evaluation confined to simple single-turn tasks, failing to reflect performance in realistic scenarios. (3) Isolated use of individual compression techniques, without exploring their joint potential. To overcome these gaps, we introduce LLMC+, a comprehensive VLM compression benchmark with a versatile, plug-and-play toolkit. LLMC+ supports over 20 algorithms across five representative VLM families and enables systematic study of token-level and model-level compression. Our benchmark reveals that: (1) Spatial and temporal redundancies demand distinct technical strategies. (2) Token reduction methods degrade significantly in multi-turn dialogue and detail-sensitive tasks. (3) Combining token and model compression achieves extreme compression with minimal performance loss. We believe LLMC+ will facilitate fair evaluation and inspire future research in efficient VLM. Our code is available at https://github.com/ModelTC/LightCompress.

Method: Proposes Bolt-SAM for bolt attribute segmentation with CLAHE-FFT Adapter and Multipart-Aware Mask Decoder, then uses MOD-LaMa for defect attribute editing, and ERA strategy to place edited defects back into original scenes.

Result: Generated bolt defect images significantly outperform state-of-the-art image editing models and effectively improve bolt defect detection performance in extensive experiments.

Conclusion: SBDE method proves effective for dataset augmentation and has strong application potential for improving bolt defect detection in transmission line safety systems.

Abstract: Bolt defect detection is critical to ensure the safety of transmission lines. However, the scarcity of defect images and imbalanced data distributions significantly limit detection performance. To address this problem, we propose a segmentationdriven bolt defect editing method (SBDE) to augment the dataset. First, a bolt attribute segmentation model (Bolt-SAM) is proposed, which enhances the segmentation of complex bolt attributes through the CLAHE-FFT Adapter (CFA) and Multipart- Aware Mask Decoder (MAMD), generating high-quality masks for subsequent editing tasks. Second, a mask optimization module (MOD) is designed and integrated with the image inpainting model (LaMa) to construct the bolt defect attribute editing model (MOD-LaMa), which converts normal bolts into defective ones through attribute editing. Finally, an editing recovery augmentation (ERA) strategy is proposed to recover and put the edited defect bolts back into the original inspection scenes and expand the defect detection dataset. We constructed multiple bolt datasets and conducted extensive experiments. Experimental results demonstrate that the bolt defect images generated by SBDE significantly outperform state-of-the-art image editing models, and effectively improve the performance of bolt defect detection, which fully verifies the effectiveness and application potential of the proposed method. The code of the project is available at https://github.com/Jay-xyj/SBDE.

Method: Introduces HumanSense benchmark and develops a multi-stage, modality-progressive reinforcement learning approach (HumanSense-Omni-Reasoning) to enhance reasoning abilities. Also designs prompts to improve non-reasoning models training-free.

Result: Leading MLLMs show considerable room for improvement, especially for interaction tasks. Supplementing visual with audio/text improves performance, and omni-modal models have advantages. The proposed approach substantially enhances higher-level understanding and interactive tasks.

Conclusion: Reasoning ability is key to providing appropriate feedback, and consistent thought patterns in reasoning processes can be leveraged through prompt design to enhance model performance without additional training.

Abstract: While Multimodal Large Language Models (MLLMs) show immense promise for achieving truly human-like interactions, progress is hindered by the lack of fine-grained evaluation frameworks for human-centered scenarios, encompassing both the understanding of complex human intentions and the provision of empathetic, context-aware responses. Here we introduce HumanSense, a comprehensive benchmark designed to evaluate the human-centered perception and interaction capabilities of MLLMs, with a particular focus on deep understanding of extended multimodal contexts and the formulation of rational feedback. Our evaluation reveals that leading MLLMs still have considerable room for improvement, particularly for advanced interaction-oriented tasks. Supplementing visual input with audio and text information yields substantial improvements, and Omni-modal models show advantages on these tasks.Furthermore, grounded in the observation that appropriate feedback stems from a contextual analysis of the interlocutor’s needs and emotions, we posit that reasoning ability serves as the key to unlocking it. We devise a multi-stage, modality-progressive reinforcement learning approach, resulting in HumanSense-Omni-Reasoning, which substantially enhances performance on higher-level understanding and interactive tasks. Additionally, we observe that successful reasoning processes appear to exhibit consistent thought patterns. By designing corresponding prompts, we also enhance the performance of non-reasoning models in a training-free manner.Project page: \textcolor{brightpink}{https://digital-avatar.github.io/ai/HumanSense/}

Method: Unifies StyleGAN3-based identity-preserving generation with diffusion-based attribute control to manipulate pose (30°), illumination (4 directions), and expression (5 levels). Synthesizes 10,000 demographically balanced faces across five cohorts.

Result: AdaFace reduces inter-group TPR disparity by 60% (2.5% vs 6.3%), with illumination accounting for 42% of residual bias. Strong synthetic-to-real transfer (r=0.85) confirmed on RFW, BUPT, and CASIA WebFace datasets.

Conclusion: GANDiff FR establishes a reproducible standard for fairness auditing with 3x more attribute-conditioned variants than pure GANs, despite 20% computational overhead, supporting transparent and scalable bias evaluation aligned with EU AI Act regulations.

Abstract: We introduce GANDiff FR, the first synthetic framework that precisely controls demographic and environmental factors to measure, explain, and reduce bias with reproducible rigor. GANDiff FR unifies StyleGAN3-based identity-preserving generation with diffusion-based attribute control, enabling fine-grained manipulation of pose around 30 degrees, illumination (four directions), and expression (five levels) under ceteris paribus conditions. We synthesize 10,000 demographically balanced faces across five cohorts validated for realism via automated detection (98.2%) and human review (89%) to isolate and quantify bias drivers. Benchmarking ArcFace, CosFace, and AdaFace under matched operating points shows AdaFace reduces inter-group TPR disparity by 60% (2.5% vs. 6.3%), with illumination accounting for 42% of residual bias. Cross-dataset evaluation on RFW, BUPT, and CASIA WebFace confirms strong synthetic-to-real transfer (r 0.85). Despite around 20% computational overhead relative to pure GANs, GANDiff FR yields three times more attribute-conditioned variants, establishing a reproducible, regulation-aligned (EU AI Act) standard for fairness auditing. Code and data are released to support transparent, scalable bias evaluation.

Method: Conducted analysis experiments to trace temporal information integration, revealing a causal pathway where temporal cues are synthesized through inter-frame attention and aggregated in the final frame. Proposed staged cross-modal attention and temporal exit mechanisms for efficiency.

Result: Found that removing/modifying positional encodings causes minimal degradation, while reversing frame sequence with original PEs causes substantial performance drop. Temporal reasoning emerges from inter-visual token interactions under causal attention constraints.

Conclusion: Temporal understanding in VideoLMs emerges from implicit temporal structure in causal attention mechanisms rather than explicit positional encodings, enabling more efficient model designs through the proposed strategies.

Abstract: Video language models (VideoLMs) have made significant progress in multimodal understanding. However, temporal understanding, which involves identifying event order, duration, and relationships across time, still remains a core challenge. Prior works emphasize positional encodings (PEs) as a key mechanism for encoding temporal structure. Surprisingly, we find that removing or modifying PEs in video inputs yields minimal degradation in the performance of temporal understanding. In contrast, reversing the frame sequence while preserving the original PEs causes a substantial drop. To explain this behavior, we conduct substantial analysis experiments to trace how temporal information is integrated within the model. We uncover a causal information pathway: temporal cues are progressively synthesized through inter-frame attention, aggregated in the final frame, and subsequently integrated into the query tokens. This emergent mechanism shows that temporal reasoning emerges from inter-visual token interactions under the constraints of causal attention, which implicitly encodes temporal structure. Based on these insights, we propose two efficiency-oriented strategies: staged cross-modal attention and a temporal exit mechanism for early token truncation. Experiments on two benchmarks validate the effectiveness of both approaches. To the best of our knowledge, this is the first systematic study of video temporal understanding in VideoLMs, offering insights for future model improvement. Our code is available at https://github.com/ANDgate99/Causality-Matters .

Method: Proposed S5 framework with: 1) Data selection strategy combining entropy-based filtering and diversity expansion to create RS4P-1M dataset; 2) Pre-training RS foundation models of varying sizes on this dataset; 3) Mixture-of-Experts-based multi-dataset fine-tuning for efficient adaptation to multiple benchmarks.

Result: The framework significantly boosts performance on land cover segmentation and object detection tasks. RSFMs achieve state-of-the-art performance across all benchmarks, demonstrating the viability of scaling semi-supervised learning for remote sensing applications.

Conclusion: S5 successfully unlocks the potential of vast unlabeled Earth observation data through scalable semi-supervised learning, providing a practical solution for remote sensing analysis with improved generalization and versatility across diverse benchmarks.

Abstract: Semi-supervised semantic segmentation (S4) has advanced remote sensing (RS) analysis by leveraging unlabeled data through pseudo-labeling and consistency learning. However, existing S4 studies often rely on small-scale datasets and models, limiting their practical applicability. To address this, we propose S5, the first scalable framework for semi-supervised semantic segmentation in RS, which unlocks the potential of vast unlabeled Earth observation data typically underutilized due to costly pixel-level annotations. Built upon existing large-scale RS datasets, S5 introduces a data selection strategy that integrates entropy-based filtering and diversity expansion, resulting in the RS4P-1M dataset. Using this dataset, we systematically scale up S4 methods by pre-training RS foundation models (RSFMs) of varying sizes on this extensive corpus, significantly boosting their performance on land cover segmentation and object detection tasks. Furthermore, during fine-tuning, we incorporate a Mixture-of-Experts (MoE)-based multi-dataset fine-tuning approach, which enables efficient adaptation to multiple RS benchmarks with fewer parameters. This approach improves the generalization and versatility of RSFMs across diverse RS benchmarks. The resulting RSFMs achieve state-of-the-art performance across all benchmarks, underscoring the viability of scaling semi-supervised learning for RS applications. All datasets, code, and models will be released at https://github.com/MiliLab/S5

Method: Used Nile Red-stained sebocyte images annotated into 14 classes by droplet count, benchmarked baseline MLP vs attention-based MIL with ResNet-50 features and instance weighting.

Result: Baseline MLP achieved more stable performance (mean MAE = 5.6) vs attention-based MIL (mean MAE = 10.7), though MIL occasionally superior in specific folds.

Conclusion: Simple bag-level aggregation provides robust baseline for slide-level droplet counting, while attention-based MIL requires task-aligned pooling and regularization to reach full potential.

Abstract: Sebocytes are lipid-secreting cells whose differentiation is marked by the accumulation of intracellular lipid droplets, making their quantification a key readout in sebocyte biology. Manual counting is labor-intensive and subjective, motivating automated solutions. Here, we introduce a simple attention-based multiple instance learning (MIL) framework for sebocyte image analysis. Nile Red-stained sebocyte images were annotated into 14 classes according to droplet counts, expanded via data augmentation to about 50,000 cells. Two models were benchmarked: a baseline multi-layer perceptron (MLP) trained on aggregated patch-level counts, and an attention-based MIL model leveraging ResNet-50 features with instance weighting. Experiments using five-fold cross-validation showed that the baseline MLP achieved more stable performance (mean MAE = 5.6) compared with the attention-based MIL, which was less consistent (mean MAE = 10.7) but occasionally superior in specific folds. These findings indicate that simple bag-level aggregation provides a robust baseline for slide-level droplet counting, while attention-based MIL requires task-aligned pooling and regularization to fully realize its potential in sebocyte image analysis.

Method: Lightweight smartphone application with real-time capture, annotation, classification, on-device quality checks, and local model adaptation using diverse clinical data.

Result: Models trained on public datasets failed to generalize to DermAI samples, but fine-tuning with local data improved performance significantly.

Conclusion: Standardized, diverse data collection aligned with healthcare needs is crucial for effective machine learning development in dermatology.

Abstract: AI-based dermatology adoption remains limited by biased datasets, variable image quality, and limited validation. We introduce DermAI, a lightweight, smartphone-based application that enables real-time capture, annotation, and classification of skin lesions during routine consultations. Unlike prior dermoscopy-focused tools, DermAI performs on-device quality checks, and local model adaptation. The DermAI clinical dataset, encompasses a wide range of skin tones, ethinicity and source devices. In preliminary experiments, models trained on public datasets failed to generalize to our samples, while fine-tuning with local data improved performance. These results highlight the importance of standardized, diverse data collection aligned with healthcare needs and oriented to machine learning development.

Method: The framework identifies nearest inter-class neighbors for each adversarial sample and removes their projections in feature space to enforce stronger feature separability, with theoretical analysis showing reduced Lipschitz constant and Rademacher complexity.

Result: Extensive experiments on CIFAR-10, CIFAR-100, and SVHN show competitive performance with leading adversarial training techniques, achieving improvements in both robust and clean accuracy.

Conclusion: Explicitly addressing inter-class feature proximity is crucial for enhancing adversarial robustness in deep neural networks.

Abstract: Deep neural networks have exhibited impressive performance in image classification tasks but remain vulnerable to adversarial examples. Standard adversarial training enhances robustness but typically fails to explicitly address inter-class feature overlap, a significant contributor to adversarial susceptibility. In this work, we introduce a novel adversarial training framework that actively mitigates inter-class proximity by projecting out inter-class dependencies from adversarial and clean samples in the feature space. Specifically, our approach first identifies the nearest inter-class neighbors for each adversarial sample and subsequently removes projections onto these neighbors to enforce stronger feature separability. Theoretically, we demonstrate that our proposed logits correction reduces the Lipschitz constant of neural networks, thereby lowering the Rademacher complexity, which directly contributes to improved generalization and robustness. Extensive experiments across standard benchmarks including CIFAR-10, CIFAR-100, and SVHN show that our method demonstrates strong performance that is competitive with leading adversarial training techniques, highlighting significant achievements in both robust and clean accuracy. Our findings reveal the importance of addressing inter-class feature proximity explicitly to bolster adversarial robustness in DNNs.

Method: Two-stage pipeline: first stage uses large multimodal model for joint layout and reading order prediction; second stage performs localized recognition of text, formulas, and tables. Includes visual consistency-based reinforcement learning for tables and specialized modules for image-decoupled table parsing and type-guided table merging.

Result: Achieves state-of-the-art performance on OmniDocBench v1.5, outperforming PPOCR-VL and MinerU 2.5, with exceptional robustness in visually complex document scenarios.

Conclusion: MonkeyOCR v1.5 provides an effective unified framework for complex document parsing, demonstrating superior performance and robustness through its innovative two-stage approach and specialized table handling modules.

Abstract: Document parsing is a core task in document intelligence, supporting applications such as information extraction, retrieval-augmented generation, and automated document analysis. However, real-world documents often feature complex layouts with multi-level tables, embedded images or formulas, and cross-page structures, which remain challenging for existing OCR systems. We introduce MonkeyOCR v1.5, a unified vision-language framework that enhances both layout understanding and content recognition through a two-stage pipeline. The first stage employs a large multimodal model to jointly predict layout and reading order, leveraging visual information to ensure sequential consistency. The second stage performs localized recognition of text, formulas, and tables within detected regions, maintaining high visual fidelity while reducing error propagation. To address complex table structures, we propose a visual consistency-based reinforcement learning scheme that evaluates recognition quality via render-and-compare alignment, improving structural accuracy without manual annotations. Additionally, two specialized modules, Image-Decoupled Table Parsing and Type-Guided Table Merging, are introduced to enable reliable parsing of tables containing embedded images and reconstruction of tables crossing pages or columns. Comprehensive experiments on OmniDocBench v1.5 demonstrate that MonkeyOCR v1.5 achieves state-of-the-art performance, outperforming PPOCR-VL and MinerU 2.5 while showing exceptional robustness in visually complex document scenarios. A trial link can be found at https://github.com/Yuliang-Liu/MonkeyOCR .

Method: Proposes Layer Image Attention Entropy (LIAE) to flag anomalous layers and Image Attention Focus (IAF) to score attention heads, then uses Dynamic Layer-wise Entropy and Attention Fusion (D-LEAF) to dynamically correct errors during inference.

Result: 53% relative improvement on captioning benchmarks, and approximately 4% improvement in both accuracy and F1-score on VQA tasks, substantially suppressing hallucinations while preserving efficiency.

Conclusion: D-LEAF effectively localizes and corrects hallucinations in MLLMs through attention-based diagnostics, with theoretical justification connecting it to DPO, achieving state-of-the-art performance with negligible computational overhead.

Abstract: Multimodal Large Language Models (MLLMs) achieve strong performance on tasks like image captioning and visual question answering, but remain prone to hallucinations, where generated text conflicts with the visual input. Prior work links this partly to insufficient visual attention, but existing attention-based detectors and mitigation typically apply uniform adjustments across layers and heads, obscuring where errors originate. In this paper, we first show these methods fail to accurately localize problematic layers. Then, we introduce two diagnostics: Layer Image Attention Entropy (LIAE) which flags anomalous layers, and Image Attention Focus (IAF) which scores attention heads within those layers. Analysis shows that LIAE pinpoints faulty layers and IAF reliably ranks heads that warrant correction. Guided by these signals, we propose Dynamic Layer-wise Entropy and Attention Fusion (D-LEAF), a task-agnostic, attention-guided method that dynamically localizes and corrects errors during inference with negligible overhead. Furthermore, by establishing a connection between D-LEAF and DPO, we provide theoretical justification for the effectiveness of D-LEAF. Results show our D-LEAF delivers a 53% relative improvement on standard captioning benchmarks, and on VQA both accuracy and F1-score improve by approximately 4%, substantially suppressing hallucinations while preserving efficiency.

Method: Train a discrete style codebook from image collections to extract style embeddings, then train an autoregressive style generator on these embeddings to model their distribution and synthesize novel style embeddings. Use these embeddings to condition a text-to-image diffusion model.

Result: CoTyle effectively turns numerical codes into style controllers, demonstrating that a style can be represented by one code. The method offers unparalleled simplicity and diversity in style generation.

Conclusion: A style is worth one numerical code. CoTyle unlocks a vast space of reproducible styles from minimal input, providing a novel approach to code-to-style image generation.

Abstract: Innovative visual stylization is a cornerstone of artistic creation, yet generating novel and consistent visual styles remains a significant challenge. Existing generative approaches typically rely on lengthy textual prompts, reference images, or parameter-efficient fine-tuning to guide style-aware image generation, but often struggle with style consistency, limited creativity, and complex style representations. In this paper, we affirm that a style is worth one numerical code by introducing the novel task, code-to-style image generation, which produces images with novel, consistent visual styles conditioned solely on a numerical style code. To date, this field has only been primarily explored by the industry (e.g., Midjourney), with no open-source research from the academic community. To fill this gap, we propose CoTyle, the first open-source method for this task. Specifically, we first train a discrete style codebook from a collection of images to extract style embeddings. These embeddings serve as conditions for a text-to-image diffusion model (T2I-DM) to generate stylistic images. Subsequently, we train an autoregressive style generator on the discrete style embeddings to model their distribution, allowing the synthesis of novel style embeddings. During inference, a numerical style code is mapped to a unique style embedding by the style generator, and this embedding guides the T2I-DM to generate images in the corresponding style. Unlike existing methods, our method offers unparalleled simplicity and diversity, unlocking a vast space of reproducible styles from minimal input. Extensive experiments validate that CoTyle effectively turns a numerical code into a style controller, demonstrating a style is worth one code.

Method: Proposes InsFusion which extracts proposals from both raw and fused features, uses these proposals to query raw features, and incorporates attention mechanisms on raw features to mitigate accumulated errors.

Result: Experiments on nuScenes dataset show InsFusion is compatible with various baseline methods and achieves new state-of-the-art performance for 3D object detection.

Conclusion: InsFusion effectively reduces accumulated errors in multi-sensor 3D object detection and delivers superior performance, making it a promising approach for autonomous driving applications.

Abstract: Three-dimensional Object Detection from multi-view cameras and LiDAR is a crucial component for autonomous driving and smart transportation. However, in the process of basic feature extraction, perspective transformation, and feature fusion, noise and error will gradually accumulate. To address this issue, we propose InsFusion, which can extract proposals from both raw and fused features and utilizes these proposals to query the raw features, thereby mitigating the impact of accumulated errors. Additionally, by incorporating attention mechanisms applied to the raw features, it thereby mitigates the impact of accumulated errors. Experiments on the nuScenes dataset demonstrate that InsFusion is compatible with various advanced baseline methods and delivers new state-of-the-art performance for 3D object detection.

Method: Used state-of-the-art architectures (DenseNet121, SwinV2-B, MedMamba) trained on chest X-rays from MIMIC-CXR-JPG and CheXpert datasets, with patch-based occlusion analysis to localize signals.

Result: Models achieved AUC around 0.67-0.68 predicting health insurance type, with signal persisting after controlling for age, race, and sex, and remaining detectable when trained on single racial groups. Signal was diffuse in upper and mid-thoracic regions.

Conclusion: Medical AI models learn socioeconomic segregation from clinical data, requiring reframing of fairness approaches to interrogate and disentangle embedded social fingerprints rather than just balancing datasets.

Abstract: Artificial intelligence is revealing what medicine never intended to encode. Deep vision models, trained on chest X-rays, can now detect not only disease but also invisible traces of social inequality. In this study, we show that state-of-the-art architectures (DenseNet121, SwinV2-B, MedMamba) can predict a patient’s health insurance type, a strong proxy for socioeconomic status, from normal chest X-rays with significant accuracy (AUC around 0.67 on MIMIC-CXR-JPG, 0.68 on CheXpert). The signal persists even when age, race, and sex are controlled for, and remains detectable when the model is trained exclusively on a single racial group. Patch-based occlusion reveals that the signal is diffuse rather than localized, embedded in the upper and mid-thoracic regions. This suggests that deep networks may be internalizing subtle traces of clinical environments, equipment differences, or care pathways; learning socioeconomic segregation itself. These findings challenge the assumption that medical images are neutral biological data. By uncovering how models perceive and exploit these hidden social signatures, this work reframes fairness in medical AI: the goal is no longer only to balance datasets or adjust thresholds, but to interrogate and disentangle the social fingerprints embedded in clinical data itself.

Method: MATR uses a Motion-Aware Transformer that explicitly predicts object movements across frames to update track queries in advance, reducing query collisions and enabling more consistent training.

Result: MATR achieves state-of-the-art results: 71.3 HOTA on DanceTrack (9+ point improvement over MOTR), 72.2 HOTA on SportsMOT, and 54.7 mTETA/41.6 mHOTA on BDD100k without external datasets.

Conclusion: Explicitly modeling motion within end-to-end Transformers provides a simple yet highly effective approach to advancing multi-object tracking performance.

Abstract: Multi-object tracking (MOT) in videos remains challenging due to complex object motions and crowded scenes. Recent DETR-based frameworks offer end-to-end solutions but typically process detection and tracking queries jointly within a single Transformer Decoder layer, leading to conflicts and degraded association accuracy. We introduce the Motion-Aware Transformer (MATR), which explicitly predicts object movements across frames to update track queries in advance. By reducing query collisions, MATR enables more consistent training and improves both detection and association. Extensive experiments on DanceTrack, SportsMOT, and BDD100k show that MATR delivers significant gains across standard metrics. On DanceTrack, MATR improves HOTA by more than 9 points over MOTR without additional data and reaches a new state-of-the-art score of 71.3 with supplementary data. MATR also achieves state-of-the-art results on SportsMOT (72.2 HOTA) and BDD100k (54.7 mTETA, 41.6 mHOTA) without relying on external datasets. These results demonstrate that explicitly modeling motion within end-to-end Transformers offers a simple yet highly effective approach to advancing multi-object tracking.

Method: Combines hybrid attention (mixing softmax and linear tokens) with lightweight distillation and fine-tuning, plus a cost-aware block-rate strategy to balance expressiveness and efficiency across layers.

Result: Applied to Wan2.1 1.3B VDM, achieves competitive results on VBench, VBench2.0 and human preference studies, with notable improvements in on-mobile latency, memory usage, and FLOPs for longer videos.

Conclusion: Attention Surgery provides an efficient framework for enabling linear/hybrid attention in pretrained VDMs, eliminating training from scratch while maintaining quality and improving scaling behavior.

Abstract: Transformer-based video diffusion models (VDMs) deliver state-of-the-art video generation quality but are constrained by the quadratic cost of self-attention, making long sequences and high resolutions computationally expensive. While linear attention offers sub-quadratic complexity, previous approaches have failed to match the expressiveness of softmax attention unless retrained at significant computational cost. We introduce Attention Surgery, an efficient framework that enables linear or hybrid attention in pretrained VDMs, eliminating the need for training from scratch. Inspired by recent advances in language models, our method combines a novel hybrid attention mechanism-mixing softmax and linear tokens-with a lightweight distillation and fine-tuning pipeline requiring only a few GPU-days. Additionally, we incorporate a cost-aware block-rate strategy to balance expressiveness and efficiency across layers. Applied to Wan2.1 1.3B, a state-of-the-art efficient transformer VDM and evaluated on VBench, VBench2.0 and a human preference study, Attention Surgery achieves competitive results. Furthermore, measurements of on-mobile latency, memory usage, and FLOPs demonstrate notable improvements in scaling behavior for longer videos. Project page is available at: https://qualcomm-ai-research.github.io/attention-surgery.

Method: Introduced custom subtests for Physion/Physion++ and integrated existing categories for CLEVRER to isolate perception (object, color, occlusion) and physics understanding (motion prediction, spatial reasoning) capabilities.

Result: Counterintuitive weak correlation between subtest mastery and benchmark accuracy - models often answer correctly without proper grounding in perception or physics.

Conclusion: Current VLMs sometimes achieve benchmark scores for wrong reasons, highlighting need for diagnostics that expose hidden failure modes beyond aggregate metrics.

Abstract: While recent Vision-Language Models (VLMs) have achieved impressive progress, it remains difficult to determine why they succeed or fail on complex reasoning tasks. Traditional benchmarks evaluate what models can answer correctly, not why they succeed or fail. In this work, we perform a failure-mode analysis of six frontier VLMs on three physics-based benchmarks - Physion, Physion++, and CLEVRER - by introducing custom subtests (for Physion and Physion++) and an integration of existing benchmark categories (for CLEVRER) to factor benchmark performance into distinct, testable capabilities. These subtests isolate perception (object, color, and occlusion recognition) and physics understanding (motion prediction and spatial reasoning), enabling us to test whether models attend to the correct entities and dynamics underlying their answers. Counterintuitively, subtest mastery correlates only weakly with benchmark accuracy: models often answer correctly without grounding in perception or physics. This suggests that current VLMs sometimes achieve benchmark scores for the wrong reasons, underscoring the need for diagnostics that expose hidden failure modes beyond aggregate metrics.

Method: Uses IP-Adapter’s automatically generated masks in a second pass to mask image tokens, restricting them to the subject area so text prompts can attend to the background.

Result: Produces images that accurately depict the subject while definitively matching the prompt, with high prompt and source image alignment validated through user study.

Conclusion: The proposed masking approach effectively balances subject preservation and prompt adherence in personalized image generation.

Abstract: Personalizing diffusion models allows users to generate new images that incorporate a given subject, allowing more control than a text prompt. These models often suffer somewhat when they end up just recreating the subject image and ignoring the text prompt. We observe that one popular method for personalization, IP-Adapter, automatically generates masks that segment the subject from the background during inference. We propose to use this automatically generated mask on a second pass to mask the image tokens, thus restricting them to the subject, not the background, allowing the text prompt to attend to the rest of the image. For text prompts describing locations and places, this produces images that accurately depict the subject while definitively matching the prompt. We compare our method to a few other test time personalization methods, and find our method displays high prompt and source image alignment. We also perform a user study to validate whether end users would appreciate our method. Code available at https://github.com/jamesBaker361/monkey

Method: Constructs lightweight variants by adapting ViTPose models and proposes using only first 2-3 stages of hierarchical VFMs (Swin Transformer, GroupMixFormer, VMamba) as encoders, based on observation that intermediate stages produce feature maps with comparable resolutions.

Result: Comprehensive evaluation of 27 hierarchical-VFM-based models shows truncated models achieve performance on par with full-stage models while exhibiting better accuracy-computation trade-offs than existing lightweight alternatives.

Conclusion: Truncated hierarchical vision foundation models provide an effective approach for efficient human mesh recovery and pose estimation, offering superior computational efficiency without sacrificing performance.

Abstract: In this work, we aim to develop simple and efficient models for human mesh recovery (HMR) and its predecessor task, human pose estimation (HPE). State-of-the-art HMR methods, such as HMR2.0 and its successors, rely on large, non-hierarchical vision transformers as encoders, which are inherited from the corresponding HPE models like ViTPose. To establish baselines across varying computational budgets, we first construct three lightweight HMR2.0 variants by adapting the corresponding ViTPose models. In addition, we propose leveraging the early stages of hierarchical vision foundation models (VFMs), including Swin Transformer, GroupMixFormer, and VMamba, as encoders. This design is motivated by the observation that intermediate stages of hierarchical VFMs produce feature maps with resolutions comparable to or higher than those of non-hierarchical counterparts. We conduct a comprehensive evaluation of 27 hierarchical-VFM-based HMR and HPE models, demonstrating that using only the first two or three stages achieves performance on par with full-stage models. Moreover, we show that the resulting truncated models exhibit better trade-offs between accuracy and computational efficiency compared to existing lightweight alternatives. The source code is available at https://github.com/nttcom/TruncHierVFM.

Method: Uses 3D Gaussian Splatting representations to render photorealistic views for MLLM comprehension, with a Global-to-Local Spatial Grounding strategy: multiple global views for coarse localization followed by close-up views for fine-grained segmentation.

Result: Achieves remarkable performance in interpreting both explicit and implicit instructions across LERF, 3D-OVS, and REALM3D benchmarks, and supports various 3D interaction tasks including object removal, replacement, and style transfer.

Conclusion: REALM effectively bridges the gap between complex instructions and 3D object grounding without extensive 3D-specific training, demonstrating practical utility and versatility in 3D interaction tasks.

Abstract: Bridging the gap between complex human instructions and precise 3D object grounding remains a significant challenge in vision and robotics. Existing 3D segmentation methods often struggle to interpret ambiguous, reasoning-based instructions, while 2D vision-language models that excel at such reasoning lack intrinsic 3D spatial understanding. In this paper, we introduce REALM, an innovative MLLM-agent framework that enables open-world reasoning-based segmentation without requiring extensive 3D-specific post-training. We perform segmentation directly on 3D Gaussian Splatting representations, capitalizing on their ability to render photorealistic novel views that are highly suitable for MLLM comprehension. As directly feeding one or more rendered views to the MLLM can lead to high sensitivity to viewpoint selection, we propose a novel Global-to-Local Spatial Grounding strategy. Specifically, multiple global views are first fed into the MLLM agent in parallel for coarse-level localization, aggregating responses to robustly identify the target object. Then, several close-up novel views of the object are synthesized to perform fine-grained local segmentation, yielding accurate and consistent 3D masks. Extensive experiments show that REALM achieves remarkable performance in interpreting both explicit and implicit instructions across LERF, 3D-OVS, and our newly introduced REALM3D benchmarks. Furthermore, our agent framework seamlessly supports a range of 3D interaction tasks, including object removal, replacement, and style transfer, demonstrating its practical utility and versatility. Project page: https://ChangyueShi.github.io/REALM.

Method: Optimizes LED spectral profiles to be minimally visible to humans but highly detectable by cameras, using spectral modulation instead of intensity modulation, and works with standard frame rates (30-60 fps).

Result: Successfully embeds 128 bits of data within 10-second video clips, providing sufficient capacity for essential metadata while maintaining visual imperceptibility.

Conclusion: The approach enables practical invisible watermarking for consumer applications, supporting privacy protection and content verification without requiring high-speed cameras or visible light modifications.

Abstract: This paper introduces a method for using LED-based environmental lighting to produce visually imperceptible watermarks for consumer cameras. Our approach optimizes an LED light source’s spectral profile to be minimally visible to the human eye while remaining highly detectable by typical consumer cameras. The method jointly considers the human visual system’s sensitivity to visible spectra, modern consumer camera sensors’ spectral sensitivity, and narrowband LEDs’ ability to generate broadband spectra perceived as “white light” (specifically, D65 illumination). To ensure imperceptibility, we employ spectral modulation rather than intensity modulation. Unlike conventional visible light communication, our approach enables watermark extraction at standard low frame rates (30-60 fps). While the information transfer rate is modest-embedding 128 bits within a 10-second video clip-this capacity is sufficient for essential metadata supporting privacy protection and content verification.

Method: Uses panoramic Plucker ray-map representation for precise trajectory control, generates multiple modalities (RGB, semantics, depth, 3D occupancy) with flexible forcing strategy for long-horizon generation, and leverages 3D occupancy for rule-based dense rewards.

Result: Achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability while providing reliable closed-loop evaluation through occupancy-grounded rewards.

Conclusion: OmniNWM successfully addresses the three core dimensions of autonomous driving world models in a unified framework, demonstrating superior performance across multiple metrics and enabling effective closed-loop evaluation.

Abstract: Autonomous driving world models are expected to work effectively across three core dimensions: state, action, and reward. Existing models, however, are typically restricted to limited state modalities, short video sequences, imprecise action control, and a lack of reward awareness. In this paper, we introduce OmniNWM, an omniscient panoramic navigation world model that addresses all three dimensions within a unified framework. For state, OmniNWM jointly generates panoramic videos of RGB, semantics, metric depth, and 3D occupancy. A flexible forcing strategy enables high-quality long-horizon auto-regressive generation. For action, we introduce a normalized panoramic Plucker ray-map representation that encodes input trajectories into pixel-level signals, enabling highly precise and generalizable control over panoramic video generation. Regarding reward, we move beyond learning reward functions with external image-based models: instead, we leverage the generated 3D occupancy to directly define rule-based dense rewards for driving compliance and safety. Extensive experiments demonstrate that OmniNWM achieves state-of-the-art performance in video generation, control accuracy, and long-horizon stability, while providing a reliable closed-loop evaluation framework through occupancy-grounded rewards. Project page is available at https://arlo0o.github.io/OmniNWM/.

Method: Conducted quantitative analysis of existing metrics’ sensitivity to ground truth perturbations, introduced a new metric based on relative surface normals, developed new depth visualization tools, and created a principled method for composite metrics.

Result: Analysis revealed existing metrics are severely under-sensitive to curvature perturbations (making smooth surfaces bumpy), and the proposed relative surface normals metric shows better human alignment.

Conclusion: The paper provides improved evaluation methodology for monocular depth estimation through new metrics, visualization tools, and composite metric construction that better align with human perception.

Abstract: Monocular depth estimation is an important task with rapid progress, but how to evaluate it is not fully resolved, as evidenced by a lack of standardization in existing literature and a large selection of evaluation metrics whose trade-offs and behaviors are not fully understood. This paper contributes a novel, quantitative analysis of existing metrics in terms of their sensitivity to various types of perturbations of ground truth, emphasizing comparison to human judgment. Our analysis reveals that existing metrics are severely under-sensitive to curvature perturbation such as making smooth surfaces bumpy. To remedy this, we introduce a new metric based on relative surface normals, along with new depth visualization tools and a principled method to create composite metrics with better human alignment. Code and data are available at: https://github.com/princeton-vl/evalmde.

Method: Strategic fusion of public generation and understanding models by retaining original blocks while interleaving multimodal self-attention blocks throughout networks, enabling double fusion mechanism.

Result: Achieved strong results with only ~35B tokens: 0.91 on GenEval, 82.16 on DPG-Bench, 6.06 on GEditBench, and 3.77 on ImgEdit-Bench across text-to-image generation and image editing tasks.

Conclusion: Proposed fusion approach enables efficient multimodal modeling with competitive performance while preserving original model strengths, with full code and model release to support future research.

Abstract: Unified multimodal models have recently shown remarkable gains in both capability and versatility, yet most leading systems are still trained from scratch and require substantial computational resources. In this paper, we show that competitive performance can be obtained far more efficiently by strategically fusing publicly available models specialized for either generation or understanding. Our key design is to retain the original blocks while additionally interleaving multimodal self-attention blocks throughout the networks. This double fusion mechanism (1) effectively enables rich multi-modal fusion while largely preserving the original strengths of the base models, and (2) catalyzes synergistic fusion of high-level semantic representations from the understanding encoder with low-level spatial signals from the generation encoder. By training with only ~ 35B tokens, this approach achieves strong results across multiple benchmarks: 0.91 on GenEval for compositional text-to-image generation, 82.16 on DPG-Bench for complex text-to-image generation, 6.06 on GEditBench, and 3.77 on ImgEdit-Bench for image editing. By fully releasing the entire suite of code, model weights, and datasets, we hope to support future research on unified multimodal modeling.

Method: Uses ’learn from synthesis’ strategy: trains diffusion model to synthesize sketches from 2D poses, creates SKEP-120K synthetic dataset, combines 2D pose detectors with diffusion priors and feed-forward network, incorporates heuristic loss functions for geometric coherence.

Result: Model substantially surpasses previous methods in both estimation accuracy and speed for sketch-to-pose tasks, as shown by qualitative, quantitative, and subjective evaluations.

Conclusion: The proposed framework effectively addresses the challenges of sketch-based 3D pose estimation through synthetic data generation and data-driven approach, enabling accurate and efficient pose estimation from diverse sketch styles.

Abstract: 3D human pose estimation from sketches has broad applications in computer animation and film production. Unlike traditional human pose estimation, this task presents unique challenges due to the abstract and disproportionate nature of sketches. Previous sketch-to-pose methods, constrained by the lack of large-scale sketch-3D pose annotations, primarily relied on optimization with heuristic rules-an approach that is both time-consuming and limited in generalizability. To address these challenges, we propose a novel approach leveraging a “learn from synthesis” strategy. First, a diffusion model is trained to synthesize sketch images from 2D poses projected from 3D human poses, mimicking disproportionate human structures in sketches. This process enables the creation of a synthetic dataset, SKEP-120K, consisting of 120k accurate sketch-3D pose annotation pairs across various sketch styles. Building on this synthetic dataset, we introduce an end-to-end data-driven framework for estimating human poses and shapes from diverse sketch styles. Our framework combines existing 2D pose detectors and generative diffusion priors for sketch feature extraction with a feed-forward neural network for efficient 2D pose estimation. Multiple heuristic loss functions are incorporated to guarantee geometric coherence between the derived 3D poses and the detected 2D poses while preserving accurate self-contacts. Qualitative, quantitative, and subjective evaluations collectively show that our model substantially surpasses previous ones in both estimation accuracy and speed for sketch-to-pose tasks.

Method: Uses flux-based generation with unpaired visual references, directly generating try-on results from source image and target garment without structural guidance or auxiliary components.

Result: Achieves competitive or superior performance compared to state-of-the-art methods on public benchmarks while maintaining simple and efficient person-to-person design.

Conclusion: RefTON provides an effective virtual try-on solution that enhances garment realism through reference guidance while simplifying the overall framework architecture.

Abstract: We introduce RefTON, a flux-based person-to-person virtual try-on framework that enhances garment realism through unpaired visual references. Unlike conventional approaches that rely on complex auxiliary inputs such as body parsing and warped mask or require finely designed extract branches to process various input conditions, RefTON streamlines the process by directly generating try-on results from a source image and a target garment, without the need for structural guidance or auxiliary components to handle diverse inputs. Moreover, inspired by human clothing selection behavior, RefTON leverages additional reference images (the target garment worn on different individuals) to provide powerful guidance for refining texture alignment and maintaining the garment details. To enable this capability, we built a dataset containing unpaired reference images for training. Extensive experiments on public benchmarks demonstrate that RefTON achieves competitive or superior performance compared to state-of-the-art methods, while maintaining a simple and efficient person-to-person design.

Method: Integrates CLIP model’s text-image alignment with guided loss to incorporate natural language semantics. Combines concentrated perturbation strategy from SSAE with dissimilar text embeddings from GAMA for multi-object scene manipulation.

Result: The method achieves competitive or superior performance compared to existing techniques across various black-box victim models while preserving greater visual fidelity.

Conclusion: The proposed approach successfully generates effective adversarial examples that deceive classification models while maintaining high visual similarity to original inputs, demonstrating the power of integrating CLIP’s cross-modal capabilities with perturbation strategies.

Abstract: The rapid growth of deep learning has brought about powerful models that can handle various tasks, like identifying images and understanding language. However, adversarial attacks, an unnoticed alteration, can deceive models, leading to inaccurate predictions. In this paper, a generative adversarial attack method is proposed that uses the CLIP model to create highly effective and visually imperceptible adversarial perturbations. The CLIP model’s ability to align text and image representation helps incorporate natural language semantics with a guided loss to generate effective adversarial examples that look identical to the original inputs. This integration allows extensive scene manipulation, creating perturbations in multi-object environments specifically designed to deceive multilabel classifiers. Our approach integrates the concentrated perturbation strategy from Saliency-based Auto-Encoder (SSAE) with the dissimilar text embeddings similar to Generative Adversarial Multi-Object Scene Attacks (GAMA), resulting in perturbations that both deceive classification models and maintain high structural similarity to the original images. The model was tested on various tasks across diverse black-box victim models. The experimental results show that our method performs competitively, achieving comparable or superior results to existing techniques, while preserving greater visual fidelity.

Method: Developed LLM-driven multimodal fusion model to synthesize PET images from blood biomarkers and T1-weighted MRI scans from 566 participants, with automated diagnostic pipeline.

Result: Synthetic PET closely matches real PET (SSIM=0.920, r=0.955), achieves 80% diagnostic accuracy, and outperforms MRI-only (AUC=0.68) and biomarker-only (AUC=0.73) models with AUC=0.78.

Conclusion: Language-enhanced generative modeling enables realistic PET synthesis from accessible data, improving Alzheimer’s diagnostic workflow and spatial pattern assessment.

Abstract: Background: Alzheimer’s disease (AD) diagnosis heavily relies on amyloid-beta positron emission tomography (Abeta-PET), which is limited by high cost and limited accessibility. This study explores whether Abeta-PET spatial patterns can be predicted from blood-based biomarkers (BBMs) and MRI scans. Methods: We collected Abeta-PET images, T1-weighted MRI scans, and BBMs from 566 participants. A language-enhanced generative model, driven by a large language model (LLM) and multimodal information fusion, was developed to synthesize PET images. Synthesized images were evaluated for image quality, diagnostic consistency, and clinical applicability within a fully automated diagnostic pipeline. Findings: The synthetic PET images closely resemble real PET scans in both structural details (SSIM = 0.920 +/- 0.003) and regional patterns (Pearson’s r = 0.955 +/- 0.007). Diagnostic outcomes using synthetic PET show high agreement with real PET-based diagnoses (accuracy = 0.80). Using synthetic PET, we developed a fully automatic AD diagnostic pipeline integrating PET synthesis and classification. The synthetic PET-based model (AUC = 0.78) outperforms T1-based (AUC = 0.68) and BBM-based (AUC = 0.73) models, while combining synthetic PET and BBMs further improved performance (AUC = 0.79). Ablation analysis supports the advantages of LLM integration and prompt engineering. Interpretation: Our language-enhanced generative model synthesizes realistic PET images, enhancing the utility of MRI and BBMs for Abeta spatial pattern assessment and improving the diagnostic workflow for Alzheimer’s disease.

Method: Developed UniPruneBench with standardized protocols across multiple dimensions, incorporating both task accuracy and system-level metrics (runtime, prefilling latency) to evaluate various pruning algorithms on different LMM families.

Result: Key findings: random pruning is surprisingly strong, no single method consistently outperforms others, pruning sensitivity varies by task (OCR most vulnerable), and pruning ratio is the dominant factor in performance degradation.

Conclusion: UniPruneBench provides a reliable foundation for future research on efficient multimodal modeling by offering standardized evaluation and revealing important insights about token pruning behavior.

Abstract: Large multimodal models (LMMs) often suffer from severe inference inefficiency due to the large number of visual tokens introduced by image encoders. While recent token compression methods, such as pruning and merging, have shown promise in reducing redundancy, their evaluation remains fragmented and inconsistent. In this work, we present UniPruneBench, a unified and extensible benchmark for visual token pruning in multimodal LLMs. UniPruneBench provides standardized protocols across six ability dimensions and ten datasets, covering ten representative compression algorithms and three families of LMMs (LLaVA-v1.5, Intern-VL3, and Qwen2.5-VL). Beyond task accuracy, it incorporates system-level metrics such as runtime and prefilling latency to provide a holistic view. Our experiments uncover several key findings: (1) random pruning is a surprisingly strong baseline, (2) no single method consistently outperforms others across scenarios, (3) pruning sensitivity varies significantly across tasks, with OCR being most vulnerable, and (4) pruning ratio is the dominant factor governing performance degradation. We believe UniPruneBench will serve as a reliable foundation for future research on efficient multimodal modeling.

Method: Pre-training–fine-tuning pipeline that learns from multiple satellites with global coverage to translate 2D satellite imagery into 3D cloud maps of relevant cloud properties.

Result: First global instantaneous 3D cloud maps created, accurately reconstructing 3D structure of intense storms, extending available observations and providing estimates when observations are missing.

Conclusion: This framework is crucial for advancing understanding of TC intensification and improving forecasts by providing comprehensive 3D cloud structure data.

Abstract: Accurate forecasting of tropical cyclones (TCs) remains challenging due to limited satellite observations probing TC structure and difficulties in resolving cloud properties involved in TC intensification. Recent research has demonstrated the capabilities of machine learning methods for 3D cloud reconstruction from satellite observations. However, existing approaches have been restricted to regions where TCs are uncommon, and are poorly validated for intense storms. We introduce a new framework, based on a pre-training–fine-tuning pipeline, that learns from multiple satellites with global coverage to translate 2D satellite imagery into 3D cloud maps of relevant cloud properties. We apply our model to a custom-built TC dataset to evaluate performance in the most challenging and relevant conditions. We show that we can - for the first time - create global instantaneous 3D cloud maps and accurately reconstruct the 3D structure of intense storms. Our model not only extends available satellite observations but also provides estimates when observations are missing entirely. This is crucial for advancing our understanding of TC intensification and improving forecasts.

Method: Develops Histogram of Oriented Hessian (HOH) for second-order geometric features, fuses with first-order gradients to create H2H descriptor, and incorporates into extended SR_NMR model with global low-rank constraint on residual matrix.

Result: Significantly outperforms state-of-the-art regression-based classifiers in both recognition accuracy and computational efficiency on benchmark datasets.

Conclusion: The H2H-GLRSR model achieves superior discrimination and robustness by effectively capturing structural features and exploiting cross-sample correlations in structured noise.

Abstract: Low-rank sparse regression models have been widely adopted in face recognition due to their robustness against occlusion and illumination variations. However, existing methods often suffer from insufficient feature representation and limited modeling of structured corruption across samples. To address these issues, this paper proposes a Hybrid second-order gradient Histogram based Global Low-Rank Sparse Regression (H2H-GLRSR) model. First, we propose the Histogram of Oriented Hessian (HOH) to capture second-order geometric characteristics such as curvature and ridge patterns. By fusing HOH and first-order gradient histograms, we construct a unified local descriptor, termed the Hybrid second-order gradient Histogram (H2H), which enhances structural discriminability under challenging conditions. Subsequently, the H2H features are incorporated into an extended version of the Sparse Regularized Nuclear Norm based Matrix Regression (SR_NMR) model, where a global low-rank constraint is imposed on the residual matrix to exploit cross-sample correlations in structured noise. The resulting H2H-GLRSR model achieves superior discrimination and robustness. Experimental results on benchmark datasets demonstrate that the proposed method significantly outperforms state-of-the-art regression-based classifiers in both recognition accuracy and computational efficiency.

Method: Proposes CIF method with three modules: 1) Semantic-aware hypergraph construction for intra-class structural information, 2) Training-free hypergraph message passing to update test features, 3) Hyperedge-guided memory search using structural information to reduce false positives.

Result: Experimental results on MVTec 3D-AD and Eyecandies datasets show CIF outperforms state-of-the-art methods in few-shot settings.

Conclusion: CIF effectively addresses few-shot multimodal industrial anomaly detection by leveraging structural commonality through hypergraphs and memory banks, achieving superior performance compared to existing methods.

Abstract: Few-shot multimodal industrial anomaly detection is a critical yet underexplored task, offering the ability to quickly adapt to complex industrial scenarios. In few-shot settings, insufficient training samples often fail to cover the diverse patterns present in test samples. This challenge can be mitigated by extracting structural commonality from a small number of training samples. In this paper, we propose a novel few-shot unsupervised multimodal industrial anomaly detection method based on structural commonality, CIF (Commonality In Few). To extract intra-class structural information, we employ hypergraphs, which are capable of modeling higher-order correlations, to capture the structural commonality within training samples, and use a memory bank to store this intra-class structural prior. Firstly, we design a semantic-aware hypergraph construction module tailored for single-semantic industrial images, from which we extract common structures to guide the construction of the memory bank. Secondly, we use a training-free hypergraph message passing module to update the visual features of test samples, reducing the distribution gap between test features and features in the memory bank. We further propose a hyperedge-guided memory search module, which utilizes structural information to assist the memory search process and reduce the false positive rate. Experimental results on the MVTec 3D-AD dataset and the Eyecandies dataset show that our method outperforms the state-of-the-art (SOTA) methods in few-shot settings. Code is available at https://github.com/Sunny5250/CIF.

Method: Fine-tune a large language model to translate text into JSON representations of NURBS surface parameters, using a hybrid representation combining untrimmed NURBS with analytic primitives to handle complex surfaces and reduce token complexity.

Result: NURBGen demonstrates strong performance on diverse prompts, surpassing prior methods in geometric fidelity and dimensional accuracy as confirmed by expert evaluations.

Conclusion: The framework successfully generates high-fidelity 3D CAD models directly from text using NURBS, with the code and partABC dataset to be released publicly.

Abstract: Generating editable 3D CAD models from natural language remains challenging, as existing text-to-CAD systems either produce meshes or rely on scarce design-history data. We present NURBGen, the first framework to generate high-fidelity 3D CAD models directly from text using Non-Uniform Rational B-Splines (NURBS). To achieve this, we fine-tune a large language model (LLM) to translate free-form texts into JSON representations containing NURBS surface parameters (\textit{i.e}, control points, knot vectors, degrees, and rational weights) which can be directly converted into BRep format using Python. We further propose a hybrid representation that combines untrimmed NURBS with analytic primitives to handle trimmed surfaces and degenerate regions more robustly, while reducing token complexity. Additionally, we introduce partABC, a curated subset of the ABC dataset consisting of individual CAD components, annotated with detailed captions using an automated annotation pipeline. NURBGen demonstrates strong performance on diverse prompts, surpassing prior methods in geometric fidelity and dimensional accuracy, as confirmed by expert evaluations. Code and dataset will be released publicly.

Method: Four innovations: Quantile-Gated Patch Selection for relevant regions, Graph-Guided Clustering for patch grouping, Hierarchical Context Attention for local/global modeling, and Expert-Driven Mixture of Log-Logistics for survival distributions.

Result: State-of-the-art performance on TCGA cohorts with time-dependent concordance indices: 0.644 on LUAD, 0.751 on KIRC, and 0.752 on BRCA, outperforming histology-only and multimodal baselines.

Conclusion: The framework provides improved calibration and interpretability, advancing the use of whole-slide images for personalized cancer prognosis.

Abstract: Accurate survival prediction from histopathology whole-slide images (WSIs) remains challenging due to their gigapixel resolution, strong spatial heterogeneity, and complex survival distributions. We introduce a comprehensive computational pathology framework that addresses these limitations through four complementary innovations: (1) Quantile-Gated Patch Selection for dynamically identifying prognostically relevant regions, (2) Graph-Guided Clustering to group patches by spatial and morphological similarity, (3) Hierarchical Context Attention to model both local tissue interactions and global slide-level context, and (4) an Expert-Driven Mixture of Log-Logistics module that flexibly models complex survival distributions. Across large TCGA cohorts, our method achieves state-of-the-art performance, yielding time-dependent concordance indices of 0.644 on LUAD, 0.751 on KIRC, and 0.752 on BRCA, consistently outperforming both histology-only and multimodal baselines. The framework further provides improved calibration and interpretability, advancing the use of WSIs for personalized cancer prognosis.

Method: Proposed SFFR method with two core modules: FCEKAN for selective frequency component exchange between RGB and IR images, and MSGKAN using multi-scale Gaussian basis functions to capture spatial feature variations at different UAV flight altitudes.

Result: Extensive experiments on SeaDroneSee, DroneVehicle and DVTOD datasets demonstrate superior performance and significant advantages in UAV multispectral object perception tasks.

Conclusion: The FCEKAN and MSGKAN modules are complementary, effectively capturing frequency and spatial semantic features respectively for better feature fusion in multispectral object detection.

Abstract: Recent multispectral object detection methods have primarily focused on spatial-domain feature fusion based on CNNs or Transformers, while the potential of frequency-domain feature remains underexplored. In this work, we propose a novel Spatial and Frequency Feature Reconstruction method (SFFR) method, which leverages the spatial-frequency feature representation mechanisms of the Kolmogorov-Arnold Network (KAN) to reconstruct complementary representations in both spatial and frequency domains prior to feature fusion. The core components of SFFR are the proposed Frequency Component Exchange KAN (FCEKAN) module and Multi-Scale Gaussian KAN (MSGKAN) module. The FCEKAN introduces an innovative selective frequency component exchange strategy that effectively enhances the complementarity and consistency of cross-modal features based on the frequency feature of RGB and IR images. The MSGKAN module demonstrates excellent nonlinear feature modeling capability in the spatial domain. By leveraging multi-scale Gaussian basis functions, it effectively captures the feature variations caused by scale changes at different UAV flight altitudes, significantly enhancing the model’s adaptability and robustness to scale variations. It is experimentally validated that our proposed FCEKAN and MSGKAN modules are complementary and can effectively capture the frequency and spatial semantic features respectively for better feature fusion. Extensive experiments on the SeaDroneSee, DroneVehicle and DVTOD datasets demonstrate the superior performance and significant advantages of the proposed method in UAV multispectral object perception task. Code will be available at https://github.com/qchenyu1027/SFFR.

Method: Created a large-scale benchmark with hierarchical QA pairs progressing from simple infraction identification to complex penalty prediction, incorporating both image and video modalities with manual bounding box annotations for visual grounding.

Result: State-of-the-art baseline models perform poorly on challenging tasks, and while training on SportR data improves scores, they remain relatively low, highlighting significant capability gaps in current multimodal models.

Conclusion: SportR presents a challenging benchmark that exposes limitations in current multimodal reasoning capabilities and provides a critical resource to drive future research in sports intelligence and multimodal reasoning.

Abstract: Deeply understanding sports requires an intricate blend of fine-grained visual perception and rule-based reasoning - a challenge that pushes the limits of current multimodal models. To succeed, models must master three critical capabilities: perceiving nuanced visual details, applying abstract sport rule knowledge, and grounding that knowledge in specific visual evidence. Current sports benchmarks either cover single sports or lack the detailed reasoning chains and precise visual grounding needed to robustly evaluate these core capabilities in a multi-sport context. To address this gap, we introduce SportR, the first multi-sports large-scale benchmark designed to train and evaluate MLLMs on the fundamental reasoning required for sports intelligence. Our benchmark provides a dataset of 5,017 images and 2,101 videos. To enable granular evaluation, we structure our benchmark around a progressive hierarchy of question-answer (QA) pairs designed to probe reasoning at increasing depths - from simple infraction identification to complex penalty prediction. For the most advanced tasks requiring multi-step reasoning, such as determining penalties or explaining tactics, we provide 7,118 high-quality, human-authored Chain of Thought (CoT) annotations. In addition, our benchmark incorporates both image and video modalities and provides manual bounding box annotations to test visual grounding in the image part directly. Extensive experiments demonstrate the profound difficulty of our benchmark. State-of-the-art baseline models perform poorly on our most challenging tasks. While training on our data via Supervised Fine-Tuning and Reinforcement Learning improves these scores, they remain relatively low, highlighting a significant gap in current model capabilities. SportR presents a new challenge for the community, providing a critical resource to drive future research in multimodal sports reasoning.

Method: Developed ‘distillation dynamics’ analytical framework combining frequency spectrum analysis, information entropy metrics, and activation magnitude tracking to study ViT behavior during distillation.

Result: Found ViTs exhibit U-shaped information processing pattern and identified representational paradigm mismatch as root cause - teacher models use distributed high-dimensional encoding that students can’t replicate due to limited channel capacity.

Conclusion: Successful ViT distillation requires moving beyond naive feature mimicry to methods respecting representational constraints, providing theoretical guidance for effective ViT compression strategies.

Abstract: While feature-based knowledge distillation has proven highly effective for compressing CNNs, these techniques unexpectedly fail when applied to Vision Transformers (ViTs), often performing worse than simple logit-based distillation. We provide the first comprehensive analysis of this phenomenon through a novel analytical framework termed as “distillation dynamics”, combining frequency spectrum analysis, information entropy metrics, and activation magnitude tracking. Our investigation reveals that ViTs exhibit a distinctive U-shaped information processing pattern: initial compression followed by expansion. We identify the root cause of negative transfer in feature distillation: a fundamental representational paradigm mismatch between teacher and student models. Through frequency-domain analysis, we show that teacher models employ distributed, high-dimensional encoding strategies in later layers that smaller student models cannot replicate due to limited channel capacity. This mismatch causes late-layer feature alignment to actively harm student performance. Our findings reveal that successful knowledge transfer in ViTs requires moving beyond naive feature mimicry to methods that respect these fundamental representational constraints, providing essential theoretical guidance for designing effective ViTs compression strategies. All source code and experimental logs are provided at https://github.com/thy960112/Distillation-Dynamics.

Method: Uses ray-based sampling from multiple viewpoints to generate candidate points, then refines positions and predicts opacity scores via transformer decoder. Includes lightweight classification head for semantic guidance using geometric features.

Result: Outperforms state-of-the-art methods on PCN and MVP benchmarks in accuracy and structural consistency, while remaining robust to varying input densities and noise levels.

Conclusion: DANCE provides effective point cloud completion that preserves observed geometry and works well in real-world conditions with variable sparsity, without requiring external image supervision.

Abstract: Point cloud completion aims to recover missing geometric structures from incomplete 3D scans, which often suffer from occlusions or limited sensor viewpoints. Existing methods typically assume fixed input/output densities or rely on image-based representations, making them less suitable for real-world scenarios with variable sparsity and limited supervision. In this paper, we introduce Density-agnostic and Class-aware Network (DANCE), a novel framework that completes only the missing regions while preserving the observed geometry. DANCE generates candidate points via ray-based sampling from multiple viewpoints. A transformer decoder then refines their positions and predicts opacity scores, which determine the validity of each point for inclusion in the final surface. To incorporate semantic guidance, a lightweight classification head is trained directly on geometric features, enabling category-consistent completion without external image supervision. Extensive experiments on the PCN and MVP benchmarks show that DANCE outperforms state-of-the-art methods in accuracy and structural consistency, while remaining robust to varying input densities and noise levels.

Method: Introduced SynWeather dataset covering four regions (Continental US, Europe, East Asia, Tropical Cyclone regions) with high-resolution observations of key weather variables. Developed SynWeatherDiff, a probabilistic weather synthesis model based on Diffusion Transformer framework to address over-smoothing.

Result: Experiments on the SynWeather dataset demonstrated the effectiveness of SynWeatherDiff compared to both task-specific and general models.

Conclusion: SynWeather enables unified multi-region and multi-variable weather observation data synthesis, and SynWeatherDiff provides an effective probabilistic solution that overcomes limitations of current deterministic approaches.

Abstract: With the advancement of meteorological instruments, abundant data has become available. Current approaches are typically focus on single-variable, single-region tasks and primarily rely on deterministic modeling. This limits unified synthesis across variables and regions, overlooks cross-variable complementarity and often leads to over-smoothed results. To address above challenges, we introduce SynWeather, the first dataset designed for Unified Multi-region and Multi-variable Weather Observation Data Synthesis. SynWeather covers four representative regions: the Continental United States, Europe, East Asia, and Tropical Cyclone regions, as well as provides high-resolution observations of key weather variables, including Composite Radar Reflectivity, Hourly Precipitation, Visible Light, and Microwave Brightness Temperature. In addition, we introduce SynWeatherDiff, a general and probabilistic weather synthesis model built upon the Diffusion Transformer framework to address the over-smoothed problem. Experiments on the SynWeather dataset demonstrate the effectiveness of our network compared with both task-specific and general models.

Method: Preserves original model margins while learning improved model. Uses explicit margin-calibration term on logits and integrates double-source focal distillation loss with previous model and new independently trained model to learn appropriate decision margins from both old and new data.

Result: Extensive experiments on image classification benchmarks show the approach consistently reduces negative flip rate while maintaining high overall accuracy.

Conclusion: The proposed method effectively mitigates negative flips in model updates by balancing margin preservation and new class learning through margin-calibration and dual-source distillation.

Abstract: Minimizing inconsistencies across successive versions of an AI system is as crucial as reducing the overall error. In image classification, such inconsistencies manifest as negative flips, where an updated model misclassifies test samples that were previously classified correctly. This issue becomes increasingly pronounced as the number of training classes grows over time, since adding new categories reduces the margin of each class and may introduce conflicting patterns that undermine their learning process, thereby degrading performance on the original subset. To mitigate negative flips, we propose a novel approach that preserves the margins of the original model while learning an improved one. Our method encourages a larger relative margin between the previously learned and newly introduced classes by introducing an explicit margin-calibration term on the logits. However, overly constraining the logit margin for the new classes can significantly degrade their accuracy compared to a new independently trained model. To address this, we integrate a double-source focal distillation loss with the previous model and a new independently trained model, learning an appropriate decision margin from both old and new data, even under a logit margin calibration. Extensive experiments on image classification benchmarks demonstrate that our approach consistently reduces the negative flip rate with high overall accuracy.

Method: Proposes WDT-MD with three innovations: noise-encoded image conditioning to avoid identity mapping, pseudo-normal pattern synthesis via inpainting for pixel-level supervision, and wavelet diffusion Transformer combining global modeling with multi-scale analysis.

Result: Outperforms state-of-the-art methods on IDRiD and e-ophtha MA datasets in both pixel-level and image-level microaneurysm detection.

Conclusion: The framework shows significant promise for improving early diabetic retinopathy screening by addressing fundamental limitations of existing diffusion-based approaches.

Abstract: Microaneurysms (MAs), the earliest pathognomonic signs of Diabetic Retinopathy (DR), present as sub-60 $μm$ lesions in fundus images with highly variable photometric and morphological characteristics, rendering manual screening not only labor-intensive but inherently error-prone. While diffusion-based anomaly detection has emerged as a promising approach for automated MA screening, its clinical application is hindered by three fundamental limitations. First, these models often fall prey to “identity mapping”, where they inadvertently replicate the input image. Second, they struggle to distinguish MAs from other anomalies, leading to high false positives. Third, their suboptimal reconstruction of normal features hampers overall performance. To address these challenges, we propose a Wavelet Diffusion Transformer framework for MA Detection (WDT-MD), which features three key innovations: a noise-encoded image conditioning mechanism to avoid “identity mapping” by perturbing image conditions during training; pseudo-normal pattern synthesis via inpainting to introduce pixel-level supervision, enabling discrimination between MAs and other anomalies; and a wavelet diffusion Transformer architecture that combines the global modeling capability of diffusion Transformers with multi-scale wavelet analysis to enhance reconstruction of normal retinal features. Comprehensive experiments on the IDRiD and e-ophtha MA datasets demonstrate that WDT-MD outperforms state-of-the-art methods in both pixel-level and image-level MA detection. This advancement holds significant promise for improving early DR screening.

Method: Proposes TDCNet with temporal difference convolution re-parameterization module containing three parallel TDC blocks that fuse temporal difference and 3D convolution. Also includes TDC-guided spatio-temporal attention mechanism for cross-attention between TDC-based and 3D backbone features.

Result: Extensive experiments on IRSTD-UAV and public infrared datasets show state-of-the-art detection performance in moving target detection.

Conclusion: TDCNet effectively captures spatio-temporal features for accurate moving infrared small target detection by combining temporal difference and 3D convolution approaches.

Abstract: Moving infrared small target detection (IRSTD) plays a critical role in practical applications, such as surveillance of unmanned aerial vehicles (UAVs) and UAV-based search system. Moving IRSTD still remains highly challenging due to weak target features and complex background interference. Accurate spatio-temporal feature modeling is crucial for moving target detection, typically achieved through either temporal differences or spatio-temporal (3D) convolutions. Temporal difference can explicitly leverage motion cues but exhibits limited capability in extracting spatial features, whereas 3D convolution effectively represents spatio-temporal features yet lacks explicit awareness of motion dynamics along the temporal dimension. In this paper, we propose a novel moving IRSTD network (TDCNet), which effectively extracts and enhances spatio-temporal features for accurate target detection. Specifically, we introduce a novel temporal difference convolution (TDC) re-parameterization module that comprises three parallel TDC blocks designed to capture contextual dependencies across different temporal ranges. Each TDC block fuses temporal difference and 3D convolution into a unified spatio-temporal convolution representation. This re-parameterized module can effectively capture multi-scale motion contextual features while suppressing pseudo-motion clutter in complex backgrounds, significantly improving detection performance. Moreover, we propose a TDC-guided spatio-temporal attention mechanism that performs cross-attention between the spatio-temporal features from the TDC-based backbone and a parallel 3D backbone. This mechanism models their global semantic dependencies to refine the current frame’s features. Extensive experiments on IRSTD-UAV and public infrared datasets demonstrate that our TDCNet achieves state-of-the-art detection performance in moving target detection.

Method: Introduces MMaDA-Parallel framework with parallel multimodal diffusion for continuous bidirectional text-image interaction, trained with supervised finetuning and optimized via Parallel Reinforcement Learning (ParaRL) that applies semantic rewards along the trajectory.

Result: Achieves 6.9% improvement in Output Alignment on ParaBench benchmark compared to state-of-the-art model Bagel, significantly improving cross-modal alignment and semantic consistency.

Conclusion: Establishes a more robust paradigm for thinking-aware image synthesis by addressing error propagation through parallel multimodal interaction and trajectory-based reinforcement learning.

Abstract: While thinking-aware generation aims to improve performance on complex tasks, we identify a critical failure mode where existing sequential, autoregressive approaches can paradoxically degrade performance due to error propagation. To systematically analyze this issue, we propose ParaBench, a new benchmark designed to evaluate both text and image output modalities. Our analysis using ParaBench reveals that this performance degradation is strongly correlated with poor alignment between the generated reasoning and the final image. To resolve this, we propose a parallel multimodal diffusion framework, MMaDA-Parallel, that enables continuous, bidirectional interaction between text and images throughout the entire denoising trajectory. MMaDA-Parallel is trained with supervised finetuning and then further optimized by Parallel Reinforcement Learning (ParaRL), a novel strategy that applies semantic rewards along the trajectory to enforce cross-modal consistency. Experiments validate that our model significantly improves cross-modal alignment and semantic consistency, achieving a 6.9% improvement in Output Alignment on ParaBench compared to the state-of-the-art model, Bagel, establishing a more robust paradigm for thinking-aware image synthesis. Our code is open-sourced at https://github.com/tyfeld/MMaDA-Parallel

Method: Created PriVi dataset with 424 hours of curated primate videos, pretrained V-JEPA model on this data, and evaluated using lightweight frozen classifier across four benchmark datasets.

Result: Outperformed prior work including fully finetuned baselines across all four datasets (ChimpACT, BaboonLand, PanAf500, ChimpBehave), with better scaling using fewer labels.

Conclusion: Primate-centric pretraining substantially improves data efficiency and generalization, making it promising for low-label applications in primate behavior research.

Abstract: Non-human primates are our closest living relatives, and analyzing their behavior is central to research in cognition, evolution, and conservation. Computer vision could greatly aid this research, but existing methods often rely on human-centric pretrained models and focus on single datasets, which limits generalization. We address this limitation by shifting from a model-centric to a data-centric approach and introduce PriVi, a large-scale primate-centric video pretraining dataset. PriVi contains 424 hours of curated video, combining 174 hours from behavioral research across 11 settings with 250 hours of diverse web-sourced footage, assembled through a scalable data curation pipeline. We pretrain V-JEPA, a large-scale video model, on PriVi to learn primate-specific representations and evaluate it using a lightweight frozen classifier. Across four benchmark datasets, ChimpACT, BaboonLand, PanAf500, and ChimpBehave, our approach consistently outperforms prior work, including fully finetuned baselines, and scales favorably with fewer labels. These results demonstrate that primate-centric pretraining substantially improves data efficiency and generalization, making it a promising approach for low-label applications. Code, models, and the majority of the dataset will be made available.

Method: Proposes a dual-mechanism collaborative optimization framework that integrates structural fairness decoupling (decoupling demographic-sensitive channels at model architecture level) and global distribution alignment (reducing distance between overall sample distribution and demographic group distributions at feature level).

Result: Experimental results show the framework improves both inter-group and intra-group fairness while maintaining overall detection accuracy across domains, outperforming other methods.

Conclusion: The proposed framework successfully addresses the fairness-accuracy trade-off in deepfake detection, enabling more equitable deployment of detection models in digital identity security applications.

Abstract: Fairness is a core element in the trustworthy deployment of deepfake detection models, especially in the field of digital identity security. Biases in detection models toward different demographic groups, such as gender and race, may lead to systemic misjudgments, exacerbating the digital divide and social inequities. However, current fairness-enhanced detectors often improve fairness at the cost of detection accuracy. To address this challenge, we propose a dual-mechanism collaborative optimization framework. Our proposed method innovatively integrates structural fairness decoupling and global distribution alignment: decoupling channels sensitive to demographic groups at the model architectural level, and subsequently reducing the distance between the overall sample distribution and the distributions corresponding to each demographic group at the feature level. Experimental results demonstrate that, compared with other methods, our framework improves both inter-group and intra-group fairness while maintaining overall detection accuracy across domains.

Method: Transformer-VAE architecture incorporating PROSAIL radiative transfer model as a differentiable physical decoder, trained exclusively on simulated data to ensure physically plausible leaf and canopy property inference.

Result: Achieved comparable accuracy to state-of-the-art methods using real imagery on field datasets (FRM4Veg and BelSAR) for retrieving LAI and canopy chlorophyll content, without requiring real satellite data or calibration.

Conclusion: The approach demonstrates that integrating physical models with deep networks enables effective RTM inversion using only simulated data, offering a self-supervised solution for large-scale vegetation monitoring.

Abstract: Accurate retrieval of vegetation biophysical variables from satellite imagery is crucial for ecosystem monitoring and agricultural management. In this work, we propose a physics-informed Transformer-VAE architecture to invert the PROSAIL radiative transfer model for simultaneous estimation of key canopy parameters from Sentinel-2 data. Unlike previous hybrid approaches that require real satellite images for self-supevised training. Our model is trained exclusively on simulated data, yet achieves performance on par with state-of-the-art methods that utilize real imagery. The Transformer-VAE incorporates the PROSAIL model as a differentiable physical decoder, ensuring that inferred latent variables correspond to physically plausible leaf and canopy properties. We demonstrate retrieval of leaf area index (LAI) and canopy chlorophyll content (CCC) on real-world field datasets (FRM4Veg and BelSAR) with accuracy comparable to models trained with real Sentinel-2 data. Our method requires no in-situ labels or calibration on real images, offering a cost-effective and self-supervised solution for global vegetation monitoring. The proposed approach illustrates how integrating physical models with advanced deep networks can improve the inversion of RTMs, opening new prospects for large-scale, physically-constrained remote sensing of vegetation traits.

Method: Hybrid State-Space Vision-Language Model using Liquid State-Space Dynamics instead of attention, with Token-Grid Correlation Module for visual grounding via FiLM conditioning.

Result: Achieves accurate fine-grained understanding with significantly improved efficiency across multiple benchmarks while maintaining linear-time inference.

Conclusion: Viper-F1 provides an efficient alternative to attention-based MLLMs, enabling deployment in resource-constrained scenarios without sacrificing fine-grained visual reasoning capabilities.

Abstract: Recent advances in multimodal large language models (MLLMs) have enabled impressive progress in vision-language understanding, yet their high computational cost limits deployment in resource-constrained scenarios such as robotic manipulation, personal assistants, and smart cameras. Most existing methods rely on Transformer-based cross-attention, whose quadratic complexity hinders efficiency. Moreover, small vision-language models often struggle to precisely capture fine-grained, task-relevant visual regions, leading to degraded performance on fine-grained reasoning tasks that limit their effectiveness in the real world. To address these issues, we introduce Viper-F1, a Hybrid State-Space Vision-Language Model that replaces attention with efficient Liquid State-Space Dynamics. To further enhance visual grounding, we propose a Token-Grid Correlation Module, which computes lightweight correlations between text tokens and image patches and modulates the state-space dynamics via FiLM conditioning. This enables the model to selectively emphasize visual regions relevant to the textual prompt while maintaining linear-time inference. Experimental results across multiple benchmarks demonstrate that Viper-F1 achieves accurate, fine-grained understanding with significantly improved efficiency.

Method: Arcee creates a differentiable boundary map that passes each block’s terminal state-space representation as the initial condition to the next block, enabling gradient flow across blocks while maintaining compatibility with existing vision-mamba variants.

Result: On CelebA-HQ 256x256 unconditional generation with Flow Matching, Arcee reduces FID from 82.81 to 15.33 (5.4x improvement) on a Zigzag Mamba baseline.

Conclusion: Arcee provides a parameter-free, efficient method to improve Mamba models for vision by preserving cross-block memory through recurrent state chains, significantly enhancing generation quality.

Abstract: State-space models (SSMs), Mamba in particular, are increasingly adopted for long-context sequence modeling, providing linear-time aggregation via an input-dependent, causal selective-scan operation. Along this line, recent “Mamba-for-vision” variants largely explore multiple scan orders to relax strict causality for non-sequential signals (e.g., images). Rather than preserving cross-block memory, the conventional formulation of the selective-scan operation in Mamba reinitializes each block’s state-space dynamics from zero, discarding the terminal state-space representation (SSR) from the previous block. Arcee, a cross-block recurrent state chain, reuses each block’s terminal state-space representation as the initial condition for the next block. Handoff across blocks is constructed as a differentiable boundary map whose Jacobian enables end-to-end gradient flow across terminal boundaries. Key to practicality, Arcee is compatible with all prior “vision-mamba” variants, parameter-free, and incurs constant, negligible cost. As a modeling perspective, we view terminal SSR as a mild directional prior induced by a causal pass over the input, rather than an estimator of the non-sequential signal itself. To quantify the impact, for unconditional generation on CelebA-HQ (256$\times$256) with Flow Matching, Arcee reduces FID$\downarrow$ from $82.81$ to $15.33$ ($5.4\times$ lower) on a single scan-order Zigzag Mamba baseline. Efficient CUDA kernels and training code will be released to support rigorous and reproducible research.

Method: Uses a CNN with residual layers for piece recognition from smartphone images, involving Hough Line Transform for edge detection, projective transform for board alignment, segmentation into 64 squares, and classification into 13 piece classes.

Result: The system successfully processes chessboard images and generates FEN notation that can be fed into chess engines to determine optimal moves.

Conclusion: CVChess provides an effective solution for bringing digital chess assistance to physical games, using computer vision and deep learning to bridge the analog-digital divide in chess.

Abstract: Chess has experienced a large increase in viewership since the pandemic, driven largely by the accessibility of online learning platforms. However, no equivalent assistance exists for physical chess games, creating a divide between analog and digital chess experiences. This paper presents CVChess, a deep learning framework for converting chessboard images to Forsyth-Edwards Notation (FEN), which is later input into online chess engines to provide you with the best next move. Our approach employs a convolutional neural network (CNN) with residual layers to perform piece recognition from smartphone camera images. The system processes RGB images of a physical chess board through a multistep process: image preprocessing using the Hough Line Transform for edge detection, projective transform to achieve a top-down board alignment, segmentation into 64 individual squares, and piece classification into 13 classes (6 unique white pieces, 6 unique black pieces and an empty square) using the residual CNN. Residual connections help retain low-level visual features while enabling deeper feature extraction, improving accuracy and stability during training. We train and evaluate our model using the Chess Recognition Dataset (ChessReD), containing 10,800 annotated smartphone images captured under diverse lighting conditions and angles. The resulting classifications are encoded as an FEN string, which can be fed into a chess engine to generate the most optimal move

Method: Created synthetic negative news headlines using tailored LLM prompts, validated through expert review, and analyzed in embedding space. Compared with real headlines using multiple metrics including perplexity, readability, POS profiling, BERTScore, and semantic similarity.

Result: Synthetic headlines closely match real headlines across most metrics, with the only significant difference being in proper noun usage in POS profiling.

Conclusion: LLM-generated synthetic datasets are a viable alternative to real-world data for sentiment analysis tasks, addressing privacy and data acquisition issues while maintaining quality.

Abstract: This research examines the potential of datasets generated by Large Language Models (LLMs) to support Natural Language Processing (NLP) tasks, aiming to overcome challenges related to data acquisition and privacy concerns associated with real-world data. Focusing on negative valence text, a critical component of sentiment analysis, we explore the use of LLM-generated synthetic news headlines as an alternative to real-world data. A specialized corpus of negative news headlines was created using tailored prompts to capture diverse negative sentiments across various societal domains. The synthetic headlines were validated by expert review and further analyzed in embedding space to assess their alignment with real-world negative news in terms of content, tone, length, and style. Key metrics such as correlation with real headlines, perplexity, coherence, and realism were evaluated. The synthetic dataset was benchmarked against two sets of real news headlines using evaluations including the Comparative Perplexity Test, Comparative Readability Test, Comparative POS Profiling, BERTScore, and Comparative Semantic Similarity. Results show the generated headlines match real headlines with the only marked divergence being in the proper noun score of the POS profile test.

Method: Created CLINB benchmark using real users’ questions and expert-curated rubrics, implementing model-based evaluation of frontier models on multimodal question answering.

Result: Frontier models show PhD-level knowledge synthesis and outperform expert-assisted hybrid answers, but suffer from poor grounding with substantial hallucination in references and images.

Conclusion: Bridging the gap between knowledge synthesis and verifiable attribution is crucial for AI deployment in science, requiring reliable benchmarks like CLINB for trustworthy AI systems.

Abstract: Evaluating how Large Language Models (LLMs) handle complex, specialized knowledge remains a critical challenge. We address this through the lens of climate change by introducing CLINB, a benchmark that assesses models on open-ended, grounded, multimodal question answering tasks with clear requirements for knowledge quality and evidential support. CLINB relies on a dataset of real users’ questions and evaluation rubrics curated by leading climate scientists. We implement and validate a model-based evaluation process and evaluate several frontier models. Our findings reveal a critical dichotomy. Frontier models demonstrate remarkable knowledge synthesis capabilities, often exhibiting PhD-level understanding and presentation quality. They outperform “hybrid” answers curated by domain experts assisted by weaker models. However, this performance is countered by failures in grounding. The quality of evidence varies, with substantial hallucination rates for references and images. We argue that bridging this gap between knowledge synthesis and verifiable attribution is essential for the deployment of AI in scientific workflows and that reliable, interpretable benchmarks like CLINB are needed to progress towards building trustworthy AI systems.

Method: Leverage large language models (LLMs) to simulate realistic bullying interactions with conversational structure, context-aware annotations, and fine-grained labeling across various CB categories.

Result: The dataset was evaluated across five dimensions including conversational structure, lexical patterns, sentiment/toxicity, role dynamics, harm intensity, and CB-type distribution, and showed utility as both standalone training data and augmentation source for CB classification.

Conclusion: SynBullying provides an effective synthetic dataset for cyberbullying research that captures realistic conversational dynamics and enables comprehensive analysis of bullying behaviors.

Abstract: We introduce SynBullying, a synthetic multi-LLM conversational dataset for studying and detecting cyberbullying (CB). SynBullying provides a scalable and ethically safe alternative to human data collection by leveraging large language models (LLMs) to simulate realistic bullying interactions. The dataset offers (i) conversational structure, capturing multi-turn exchanges rather than isolated posts; (ii) context-aware annotations, where harmfulness is assessed within the conversational flow considering context, intent, and discourse dynamics; and (iii) fine-grained labeling, covering various CB categories for detailed linguistic and behavioral analysis. We evaluate SynBullying across five dimensions, including conversational structure, lexical patterns, sentiment/toxicity, role dynamics, harm intensity, and CB-type distribution. We further examine its utility by testing its performance as standalone training data and as an augmentation source for CB classification.

Method: CausalGuard works through two complementary paths: tracing causal relationships between what the model knows and what it generates, and checking logical consistency using automated reasoning. Unlike previous methods that only check outputs after generation, it understands the causal chain leading to false statements and intervenes early.

Result: Testing across twelve benchmarks showed CausalGuard correctly identifies hallucinations 89.3% of the time with only 8.3% missed hallucinations. It reduces false claims by nearly 80% while keeping responses natural and helpful, performing especially well on complex reasoning tasks requiring multiple logic steps.

Conclusion: CausalGuard effectively addresses the hallucination problem in LLMs by combining causal reasoning with symbolic logic, providing transparent reasoning processes that make it suitable for sensitive applications like medical diagnosis or financial analysis where understanding decision rationale is crucial.

Abstract: While large language models have transformed how we interact with AI systems, they have a critical weakness: they confidently state false information that sounds entirely plausible. This “hallucination” problem has become a major barrier to using these models where accuracy matters most. Existing solutions either require retraining the entire model, add significant computational costs, or miss the root causes of why these hallucinations occur in the first place. We present CausalGuard, a new approach that combines causal reasoning with symbolic logic to catch and prevent hallucinations as they happen. Unlike previous methods that only check outputs after generation, our system understands the causal chain that leads to false statements and intervenes early in the process. CausalGuard works through two complementary paths: one that traces causal relationships between what the model knows and what it generates, and another that checks logical consistency using automated reasoning. Testing across twelve different benchmarks, we found that CausalGuard correctly identifies hallucinations 89.3% of the time while missing only 8.3% of actual hallucinations. More importantly, it reduces false claims by nearly 80% while keeping responses natural and helpful. The system performs especially well on complex reasoning tasks where multiple steps of logic are required. Because CausalGuard shows its reasoning process, it works well in sensitive areas like medical diagnosis or financial analysis where understanding why a decision was made matters as much as the decision itself.

Method: Define Skill-Luck Index S(G) in [-1, 1] by decomposing game outcomes into skill leverage K and luck leverage L, and introduce volatility Sigma to quantify outcome uncertainty over successive turns.

Result: Applied to 30 games revealing a continuum from pure chance (coin toss, S = -1) to pure skill (chess, S = +1), with poker showing moderate skill dominance (S = 0.33). Backgammon shows balanced skill and chance (S = 0).

Conclusion: The framework enables principled comparisons of player influence, game balance, and predictive stability, with broad applications in game design, AI evaluation, and risk assessment for general stochastic decision systems.

Abstract: We introduce a quantitative framework for separating skill and chance in games by modeling them as complementary sources of control over stochastic decision trees. We define the Skill-Luck Index S(G) in [-1, 1] by decomposing game outcomes into skill leverage K and luck leverage L. Applying this to 30 games reveals a continuum from pure chance (coin toss, S = -1) through mixed domains such as backgammon (S = 0, Sigma = 1.20) to pure skill (chess, S = +1, Sigma = 0). Poker exhibits moderate skill dominance (S = 0.33) with K = 0.40 +/- 0.03 and Sigma = 0.80. We further introduce volatility Sigma to quantify outcome uncertainty over successive turns. The framework extends to general stochastic decision systems, enabling principled comparisons of player influence, game balance, and predictive stability, with applications to game design, AI evaluation, and risk assessment.

Method: VALOR integrates layered prompt analysis with human-aligned value reasoning: multi-level NSFW detection, cultural value alignment, and intention disambiguation. When unsafe content is detected, prompts are selectively rewritten by an LLM under dynamic instructions, with optional stylistic regeneration if needed.

Result: Experiments show VALOR reduces unsafe outputs by up to 100.00% while preserving prompt usefulness and creativity across adversarial, ambiguous, and value-sensitive prompts.

Conclusion: VALOR provides a scalable and effective approach for deploying safe, aligned, and helpful image generation systems in open-world settings.

Abstract: Generative vision-language models like Stable Diffusion demonstrate remarkable capabilities in creative media synthesis, but they also pose substantial risks of producing unsafe, offensive, or culturally inappropriate content when prompted adversarially. Current defenses struggle to align outputs with human values without sacrificing generation quality or incurring high costs. To address these challenges, we introduce VALOR (Value-Aligned LLM-Overseen Rewriter), a modular, zero-shot agentic framework for safer and more helpful text-to-image generation. VALOR integrates layered prompt analysis with human-aligned value reasoning: a multi-level NSFW detector filters lexical and semantic risks; a cultural value alignment module identifies violations of social norms, legality, and representational ethics; and an intention disambiguator detects subtle or indirect unsafe implications. When unsafe content is detected, prompts are selectively rewritten by a large language model under dynamic, role-specific instructions designed to preserve user intent while enforcing alignment. If the generated image still fails a safety check, VALOR optionally performs a stylistic regeneration to steer the output toward a safer visual domain without altering core semantics. Experiments across adversarial, ambiguous, and value-sensitive prompts show that VALOR significantly reduces unsafe outputs by up to 100.00% while preserving prompt usefulness and creativity. These results highlight VALOR as a scalable and effective approach for deploying safe, aligned, and helpful image generation systems in open-world settings.

Method: AI-Mandel formulates ideas from literature and uses domain-specific AI tools to convert them into concrete experiment designs that can be directly implemented in laboratories.

Result: The system generates scientifically interesting ideas including new quantum teleportation variations, quantum network primitives in indefinite causal orders, and new geometric phase concepts. Two ideas have already led to independent scientific follow-up papers.

Conclusion: AI-Mandel demonstrates a prototypical AI physicist capable of generating actionable ideas, accelerating science while revealing concrete challenges toward achieving human-level artificial scientists.

Abstract: Artificial intelligence (AI) is used in numerous fields of science, yet the initial research questions and targets are still almost always provided by human researchers. AI-generated creative ideas in science are rare and often vague, so that it remains a human task to execute them. Automating idea generation and implementation in one coherent system would significantly shift the role of humans in the scientific process. Here we present AI-Mandel, an LLM agent that can generate and implement ideas in quantum physics. AI-Mandel formulates ideas from the literature and uses a domain-specific AI tool to turn them into concrete experiment designs that can readily be implemented in laboratories. The generated ideas by AI-Mandel are often scientifically interesting - for two of them we have already written independent scientific follow-up papers. The ideas include new variations of quantum teleportation, primitives of quantum networks in indefinite causal orders, and new concepts of geometric phases based on closed loops of quantum information transfer. AI-Mandel is a prototypical demonstration of an AI physicist that can generate and implement concrete, actionable ideas. Building such a system is not only useful to accelerate science, but it also reveals concrete open challenges on the path to human-level artificial scientists.

Method: An LLM agent learns a resilient policy through outcome-driven Reinforcement Learning (GRPO) without supervised fine-tuning, iteratively constructing SPARQL queries and systematically recovering from execution errors using real-time feedback.

Result: Achieved 49.7% accuracy post-entity-linking on LC-QuAD 2.0, representing a 17.5 percentage point improvement over the strongest iterative zero-shot baseline. Performance is enhanced by an explicit deliberative reasoning step.

Conclusion: The framework provides a generalizable blueprint for teaching agents to master formal, symbolic tools through interaction, bridging the gap between probabilistic LLMs and the structured world of Knowledge Graphs.

Abstract: Generating complex, logically-sound SPARQL queries for multi-hop questions remains a critical bottleneck for Knowledge Graph Question Answering, as the brittle nature of one-shot generation by Large Language Models (LLMs) hinders reliable interaction with structured data. Current methods lack the adaptive policies needed to dynamically debug queries based on real-time execution feedback. This paper introduces a novel agentic framework where an LLM learns a resilient policy for the sequential process of iterative SPARQL construction. We show that a compact 3B-parameter model, trained exclusively via outcome-driven Reinforcement Learning (GRPO) without supervised fine-tuning, can learn effective policies for this task, discovering how to systematically recover from execution errors and refine its queries toward a correct answer. On a curated, executable single-answer subset of LC-QuAD 2.0, our agent achieves 49.7% accuracy post-entity-linking, a significant 17.5 percentage point improvement over the strongest iterative zero-shot baseline. Further analysis reveals that while the agent’s capability is driven by RL, its performance is enhanced by an explicit deliberative reasoning step that acts as a cognitive scaffold to improve policy precision. This work presents a generalizable blueprint for teaching agents to master formal, symbolic tools through interaction, bridging the gap between probabilistic LLMs and the structured world of Knowledge Graphs.

Method: Conceptual and formal analysis of three assumptions underlying intelligence-focused evaluation: generality, stability, and realism, examining which withstands scrutiny.

Result: Only generality proves conceptually and empirically sound; intelligence is not what enables generality, but rather generality should be understood as a multitask learning problem directly linking evaluation to performance breadth and reliability.

Conclusion: Evaluation should be grounded in generality rather than abstract intelligence, reframing AI progress assessment and establishing generality as a more stable foundation for evaluating capabilities across diverse and evolving tasks.

Abstract: Benchmarks such as ARC, Raven-inspired tests, and the Blackbird Task are widely used to evaluate the intelligence of large language models (LLMs). Yet, the concept of intelligence remains elusive- lacking a stable definition and failing to predict performance on practical tasks such as question answering, summarization, or coding. Optimizing for such benchmarks risks misaligning evaluation with real-world utility. Our perspective is that evaluation should be grounded in generality rather than abstract notions of intelligence. We identify three assumptions that often underpin intelligence-focused evaluation: generality, stability, and realism. Through conceptual and formal analysis, we show that only generality withstands conceptual and empirical scrutiny. Intelligence is not what enables generality; generality is best understood as a multitask learning problem that directly links evaluation to measurable performance breadth and reliability. This perspective reframes how progress in AI should be assessed and proposes generality as a more stable foundation for evaluating capability across diverse and evolving tasks.

Method: Proposed a novel evaluation protocol to distinguish genuine semantic-level logical understanding from superficial pattern recognition, memorization, and dataset contamination. Critically examined existing datasets and evaluation approaches.

Result: State-of-the-art dialogue-oriented LLMs demonstrate strong NL-FOL translation skills and genuine grasp of sentence-level logic, while embedding-centric models perform markedly worse.

Conclusion: Proper evaluation reveals that modern LLMs have strong capabilities in NL-FOL translation, with dialogue-oriented models showing particularly good performance in understanding sentence-level logic semantics.

Abstract: Due to its expressiveness and unambiguous nature, First-Order Logic (FOL) is a powerful formalism for representing concepts expressed in natural language (NL). This is useful, e.g., for specifying and verifying desired system properties. While translating FOL into human-readable English is relatively straightforward, the inverse problem, converting NL to FOL (NL-FOL translation), has remained a longstanding challenge, for both humans and machines. Although the emergence of Large Language Models (LLMs) promised a breakthrough, recent literature provides contrasting results on their ability to perform NL-FOL translation. In this work, we provide a threefold contribution. First, we critically examine existing datasets and protocols for evaluating NL-FOL translation performance, revealing key limitations that may cause a misrepresentation of LLMs’ actual capabilities. Second, to overcome these shortcomings, we propose a novel evaluation protocol explicitly designed to distinguish genuine semantic-level logical understanding from superficial pattern recognition, memorization, and dataset contamination. Third, using this new approach, we show that state-of-the-art, dialogue-oriented LLMs demonstrate strong NL-FOL translation skills and a genuine grasp of sentence-level logic, whereas embedding-centric models perform markedly worse.

Method: Developed TopoPerception benchmark leveraging topological properties that depend on global image structure and are invariant to local features, enabling shortcut-free assessment of global perception across various granularities.

Result: All evaluated state-of-the-art models performed no better than random chance at the coarsest perceptual granularity. More powerful models with stronger reasoning capabilities exhibited lower accuracy, suggesting scaling up models may exacerbate the deficit.

Conclusion: Current LVLMs have a profound inability to perceive global visual features. Merely scaling up models is insufficient and may worsen performance. New training paradigms or architectures are needed, and TopoPerception provides direction for improving global visual perception.

Abstract: Large Vision-Language Models (LVLMs) typically align visual features from an encoder with a pre-trained Large Language Model (LLM). However, this makes the visual perception module a bottleneck, which constrains the overall capabilities of LVLMs. Conventional evaluation benchmarks, while rich in visual semantics, often contain unavoidable local shortcuts that can lead to an overestimation of models’ perceptual abilities. Here, we introduce TopoPerception, a benchmark that leverages topological properties to rigorously evaluate the global visual perception capabilities of LVLMs across various granularities. Since topology depends on the global structure of an image and is invariant to local features, TopoPerception enables a shortcut-free assessment of global perception, fundamentally distinguishing it from semantically rich tasks. We evaluate state-of-the-art models on TopoPerception and find that even at the coarsest perceptual granularity, all models perform no better than random chance, indicating a profound inability to perceive global visual features. Notably, a consistent trend emerge within model families: more powerful models with stronger reasoning capabilities exhibit lower accuracy. This suggests that merely scaling up models is insufficient to address this deficit and may even exacerbate it. Progress may require new training paradigms or architectures. TopoPerception not only exposes a critical bottleneck in current LVLMs but also offers a lens and direction for improving their global visual perception. The data and code are publicly available at: https://github.com/Wenhao-Zhou/TopoPerception.

Method: Leverages transformer-based spatial and temporal modeling with frame-wise classification to detect consecutive short (~2 seconds) gestures in robot-assisted radical prostatectomy videos.

Result: Achieved AUC of 0.80 frame-level and 0.81 video-level for gesture detection. F2O-derived features predicted postoperative outcomes with 0.79 accuracy (vs 0.75 human annotations), with strong correlation (r=0.96) and captured key patterns linked to erectile function recovery.

Conclusion: F2O enables automatic interpretable assessment and establishes a foundation for data-driven surgical feedback and prospective clinical decision support.

Abstract: Fine-grained analysis of intraoperative behavior and its impact on patient outcomes remain a longstanding challenge. We present Frame-to-Outcome (F2O), an end-to-end system that translates tissue dissection videos into gesture sequences and uncovers patterns associated with postoperative outcomes. Leveraging transformer-based spatial and temporal modeling and frame-wise classification, F2O robustly detects consecutive short (2 seconds) gestures in the nerve-sparing step of robot-assisted radical prostatectomy (AUC: 0.80 frame-level; 0.81 video-level). F2O-derived features (gesture frequency, duration, and transitions) predicted postoperative outcomes with accuracy comparable to human annotations (0.79 vs. 0.75; overlapping 95% CI). Across 25 shared features, effect size directions were concordant with small differences ( 0.07), and strong correlation (r = 0.96, p < 1e-14). F2O also captured key patterns linked to erectile function recovery, including prolonged tissue peeling and reduced energy use. By enabling automatic interpretable assessment, F2O establishes a foundation for data-driven surgical feedback and prospective clinical decision support.

Method: Penalizes marginal information contributed by data to be unlearned, providing explicit upper bounds on residual influence and provable undetectability of unlearned data.

Result: Outperforms state-of-the-art unlearning methods with reliable forgetting and better preserved general model performance across diverse benchmarks.

Conclusion: Represents an important step toward making AI systems more controllable and compliant with privacy/copyright regulations without compromising effectiveness.

Abstract: As AI models are trained on ever-expanding datasets, the ability to remove the influence of specific data from trained models has become essential for privacy protection and regulatory compliance. Unlearning addresses this challenge by selectively removing parametric knowledge from the trained models without retraining from scratch, which is critical for resource-intensive models such as Large Language Models (LLMs). Existing unlearning methods often degrade model performance by removing more information than necessary when attempting to ‘‘forget’’ specific data. We introduce Forgetting-MarI, an LLM unlearning framework that provably removes only the additional (marginal) information contributed by the data to be unlearned, while preserving the information supported by the data to be retained. By penalizing marginal information, our method yields an explicit upper bound on the unlearn dataset’s residual influence in the trained models, providing provable undetectability. Extensive experiments confirm that our approach outperforms current state-of-the-art unlearning methods, delivering reliable forgetting and better preserved general model performance across diverse benchmarks. This advancement represents an important step toward making AI systems more controllable and compliant with privacy and copyright regulations without compromising their effectiveness.

Method: Used four LLM models with four reasoning architectures (single-shot, embedding-controlled repetition, self-reflection, multi-agent) on RAVEN-FAIR dataset. Visual responses generated via three-stage process (JSON extraction, LLM reasoning, Tool Function) and evaluated using SSIM and LPIPS metrics. Analyzed Chain-of-Thought scores and error types.

Result: GPT-4.1-Mini consistently achieved highest overall accuracy across all architectures. Multi-agent architecture occasionally altered semantic/numeric balance but effects weren’t uniformly beneficial. Each model showed distinct sensitivity to architectural design. Response coverage variations complicated cross-architecture comparisons.

Conclusion: Reasoning effectiveness remains model-specific, with GPT-4.1-Mini demonstrating strongest capabilities. Multi-run evaluations are essential as single-run assessments are fragile and unreliable for drawing conclusions about LLM performance.

Abstract: This study aims to systematically evaluate the performance of large language models (LLMs) in abstract visual reasoning problems. We examined four LLM models (GPT-4.1-Mini, Claude-3.5-Haiku, Gemini-1.5-Flash, Llama-3.3-70b) utilizing four different reasoning architectures (single-shot, embedding-controlled repetition, self-reflection, and multi-agent) on the RAVEN-FAIR dataset. Visual responses generated through a three-stage process (JSON extraction, LLM reasoning, and Tool Function) were evaluated using SSIM and LPIPS metrics; Chain-of-Thought scores and error types (semantic hallucination, numeric misperception) were analyzed. Results demonstrate that GPT-4.1-Mini consistently achieved the highest overall accuracy across all architectures, indicating a strong reasoning capability. While the multi-agent architecture occasionally altered semantic and numeric balance across models, these effects were not uniformly beneficial. Instead, each model exhibited distinct sensitivity patterns to architectural design, underscoring that reasoning effectiveness remains model-specific. Variations in response coverage further emerged as a confounding factor that complicates direct cross-architecture comparison. To estimate the upper-bound performance of each configuration, we report the best of five independent runs, representing a best-case scenario rather than an averaged outcome. This multi-run strategy aligns with recent recommendations, which emphasize that single-run evaluations are fragile and may lead to unreliable conclusions.

Method: The chapter presents perspectives and discusses open research problems through analysis of various aspects including cascading failures, dynamic environments, task execution consistency, emergent behaviors, and reliability mechanisms.

Result: Identification of multiple research challenges and opportunities in building reliable agentic AI systems, with specific focus on testing and evaluation methods for reliability assessment.

Conclusion: There are significant research gaps and opportunities in developing reliable agentic AI systems, particularly in areas of failure mitigation, dynamic adaptation, and comprehensive testing frameworks.

Abstract: This chapter presents perspectives for challenges and future development in building reliable AI systems, particularly, agentic AI systems. Several open research problems related to mitigating the risks of cascading failures are discussed. The chapter also sheds lights on research challenges and opportunities in aspects including dynamic environments, inconsistent task execution, unpredictable emergent behaviors, as well as resource-intensive reliability mechanisms. In addition, several research directions along the line of testing and evaluating reliability of agentic AI systems are also discussed.

Method: Developed the RWTA motif as the basic building block, creating an event-based framework that inherits discrete computation capabilities from winner-take-all state machines and continuous tuning capabilities from excitable biophysical circuits.

Result: The architecture demonstrates versatility, robustness, and modularity through its application in designing the nervous system of a snake robot, successfully handling both rhythmic generation and decision-making tasks.

Conclusion: The RWTA-based neuromorphic architecture provides a scalable solution that unifies discrete and continuous control paradigms, offering a promising approach for neuromorphic system design with applications in robotics and beyond.

Abstract: This paper introduces the ``rebound Winner-Take-All (RWTA)" motif as the basic element of a scalable neuromorphic control architecture. From the cellular level to the system level, the resulting architecture combines the reliability of discrete computation and the tunability of continuous regulation: it inherits the discrete computation capabilities of winner-take-all state machines and the continuous tuning capabilities of excitable biophysical circuits. The proposed event-based framework addresses continuous rhythmic generation and discrete decision-making in a unified physical modeling language. We illustrate the versatility, robustness, and modularity of the architecture through the nervous system design of a snake robot.

Method: Multi-agent framework with retrieval agent, decision agent, and chat agent that uses 100 AMA flowcharts to guide LLMs through structured, auditable patient decision support.

Result: 95.29% top-3 accuracy in flowchart retrieval (N=2,000) and 99.10% accuracy in flowchart navigation across varied conversational styles (N=37,200).

Conclusion: Combining free-text interaction with standardized clinical protocols enables transparent, accurate, and generalizable AI-assisted self-triage for informed patient decision-making.

Abstract: Online health resources and large language models (LLMs) are increasingly used as a first point of contact for medical decision-making, yet their reliability in healthcare remains limited by low accuracy, lack of transparency, and susceptibility to unverified information. We introduce a proof-of-concept conversational self-triage system that guides LLMs with 100 clinically validated flowcharts from the American Medical Association, providing a structured and auditable framework for patient decision support. The system leverages a multi-agent framework consisting of a retrieval agent, a decision agent, and a chat agent to identify the most relevant flowchart, interpret patient responses, and deliver personalized, patient-friendly recommendations, respectively. Performance was evaluated at scale using synthetic datasets of simulated conversations. The system achieved 95.29% top-3 accuracy in flowchart retrieval (N=2,000) and 99.10% accuracy in flowchart navigation across varied conversational styles and conditions (N=37,200). By combining the flexibility of free-text interaction with the rigor of standardized clinical protocols, this approach demonstrates the feasibility of transparent, accurate, and generalizable AI-assisted self-triage, with potential to support informed patient decision-making while improving healthcare resource utilization.

Method: CFA-SMOTE combines instance-based counterfactual methods from Explainable AI with SMOTE class-imbalance technique to create synthetic data-points representing climate outlier events.

Result: Comparative experiments show CFA-SMOTE improves predictive performance for grass growth prediction during the 2018 European drought crisis, outperforming benchmark methods under different class-imbalance ratios.

Conclusion: Treating climate change prediction as a class-imbalance problem and using counterfactual-based data augmentation can significantly improve AI’s ability to handle climate-disrupted data and extreme weather events.

Abstract: In recent years, humanity has begun to experience the catastrophic effects of climate change as economic sectors (such as agriculture) struggle with unpredictable and extreme weather events. Artificial Intelligence (AI) should help us handle these climate challenges but its most promising solutions are not good at dealing with climate-disrupted data; specifically, machine learning methods that work from historical data-distributions, are not good at handling out-of-distribution, outlier events. In this paper, we propose a novel data augmentation method, that treats the predictive problems around climate change as being, in part, due to class-imbalance issues; that is, prediction from historical datasets is difficult because, by definition, they lack sufficient minority-class instances of “climate outlier events”. This novel data augmentation method – called Counterfactual-Based SMOTE (CFA-SMOTE) – combines an instance-based counterfactual method from Explainable AI (XAI) with the well-known class-imbalance method, SMOTE. CFA-SMOTE creates synthetic data-points representing outlier, climate-events that augment the dataset to improve predictive performance. We report comparative experiments using this CFA-SMOTE method, comparing it to benchmark counterfactual and class-imbalance methods under different conditions (i.e., class-imbalance ratios). The focal climate-change domain used relies on predicting grass growth on Irish dairy farms, during Europe-wide drought and forage crisis of 2018.

Method: Learn latent strategy space with variational autoencoder from trajectory data, cluster strategies, train cooperator conditioned on clusters, and use fixed-share regret minimization for online adaptation to novel partners.

Result: Achieves state-of-the-art performance with novel human and agent teammates in Overcooked domain, validated through experiments and user study.

Conclusion: The proposed framework effectively enables real-time adaptation to diverse partner strategies in complex collaborative environments.

Abstract: Adaptation is the cornerstone of effective collaboration among heterogeneous team members. In human-agent teams, artificial agents need to adapt to their human partners in real time, as individuals often have unique preferences and policies that may change dynamically throughout interactions. This becomes particularly challenging in tasks with time pressure and complex strategic spaces, where identifying partner behaviors and selecting suitable responses is difficult. In this work, we introduce a strategy-conditioned cooperator framework that learns to represent, categorize, and adapt to a broad range of potential partner strategies in real-time. Our approach encodes strategies with a variational autoencoder to learn a latent strategy space from agent trajectory data, identifies distinct strategy types through clustering, and trains a cooperator agent conditioned on these clusters by generating partners of each strategy type. For online adaptation to novel partners, we leverage a fixed-share regret minimization algorithm that dynamically infers and adjusts the partner’s strategy estimation during interaction. We evaluate our method in a modified version of the Overcooked domain, a complex collaborative cooking environment that requires effective coordination among two players with a diverse potential strategy space. Through these experiments and an online user study, we demonstrate that our proposed agent achieves state of the art performance compared to existing baselines when paired with novel human, and agent teammates.

Method: Combined GPT-4o and GPT-5 with Prolog symbolic logic - used LLMs to translate tax code sections into Prolog rules, then employed Prolog for deterministic reasoning and inconsistency detection.

Result: LLM-only approaches achieved only 33% accuracy, while the hybrid Prolog model produced deterministic, reproducible results and successfully identified inconsistency zones with validation confirming accuracy and internal consistency.

Conclusion: LLM-assisted formalization anchored in symbolic logic enables transparent and reliable statutory inconsistency detection, demonstrating the value of hybrid neuro-symbolic approaches for complex legal reasoning.

Abstract: This study introduces a hybrid neuro-symbolic framework that achieves deterministic detection of statutory inconsistency in complex law. We use the U.S. Internal Revenue Code (IRC) as a case study because its complexity makes it a fertile domain for identifying conflicts. Our research offers a solution for detecting inconsistent provisions by combining Large Language Models (LLMs) with symbolic logic. LLM-based methods can support compliance, fairness, and statutory drafting, yet tax-specific applications remain sparse. A key challenge is that such models struggle with hierarchical processing and deep structured reasoning, especially over long text. This research addresses these gaps through experiments using GPT-4o, GPT-5, and Prolog. GPT-4o was first used to translate Section 121 into Prolog rules and refine them in SWISH. These rules were then incorporated into prompts to test whether Prolog-augmented prompting improved GPT-4o’s inconsistency detection. GPT-4o, whether prompted with natural language alone or with Prolog augmentation, detected the inconsistency in only one of three strategies (33 percent accuracy), but its reasoning quality differed: natural-language prompting achieved 100 percent rule coverage, while Prolog-augmented prompting achieved 66 percent, indicating more incomplete statutory analysis. In contrast to probabilistic prompting, the hybrid Prolog model produced deterministic and reproducible results. Guided by GPT-5 for refinement, the model formalized the IRC section’s competing interpretations and successfully detected an inconsistency zone. Validation tests confirm that the Prolog implementation is accurate, internally consistent, deterministic, and capable of autonomously identifying inconsistencies. These findings show that LLM-assisted formalization, anchored in symbolic logic, enables transparent and reliable statutory inconsistency detection.

Method: DDR framework that generates candidate library dependencies from natural language, verifies them via efficient suffix array checks, constructs large retrieval dataset (500k+ samples), and fine-tunes high-precision DDR model.

Result: DDR model significantly outperforms SOTA methods in retrieval precision and recall. Autoformalizer with DDR shows consistent advantages in single-attempt accuracy and multi-attempt stability over traditional RAG methods.