Method: Evaluated six open-source LLMs compared to human experts on inductive coding tasks. Experts rated perceived difficulty of quotes they coded. Compared both human and LLM generated labels to golden standard test set, and examined systematic deviations.

Result: Found a dichotomy: human coders perform well with complex sentences but struggle with simpler ones, while LLMs show opposite trend. Human annotations sometimes differ from golden standard but are rated more favorably by other humans. Some LLMs demonstrate closer alignment with true labels but receive lower evaluations from experts.

Conclusion: LLMs and humans have complementary strengths in inductive coding - humans excel with complexity while LLMs handle simplicity better. Human evaluation bias exists where human-generated labels are preferred even when less accurate, suggesting need for careful evaluation metrics in automated qualitative analysis.

Abstract: This paper investigates the automation of qualitative data analysis, focusing on inductive coding using large language models (LLMs). Unlike traditional approaches that rely on deductive methods with predefined labels, this research investigates the inductive process where labels emerge from the data. The study evaluates the performance of six open-source LLMs compared to human experts. As part of the evaluation, experts rated the perceived difficulty of the quotes they coded. The results reveal a peculiar dichotomy: human coders consistently perform well when labeling complex sentences but struggle with simpler ones, while LLMs exhibit the opposite trend. Additionally, the study explores systematic deviations in both human and LLM generated labels by comparing them to the golden standard from the test set. While human annotations may sometimes differ from the golden standard, they are often rated more favorably by other humans. In contrast, some LLMs demonstrate closer alignment with the true labels but receive lower evaluations from experts.

Method: Simulated 7500 multi-agent, multi-round discussions in inductive coding tasks, generating 125k+ utterances. Introduced process-level metrics (code stability, semantic self-consistency, lexical confidence, sentiment, convergence) and analyzed evolving geometry of output embeddings to track coordination dynamics.

Result: LLM groups converge lexically and semantically, develop asymmetric influence patterns, exhibit negotiation-like behaviors without explicit prompting. Intrinsic dimensionality of output embeddings declines over rounds, suggesting semantic compression. Black-box interaction analysis reveals emergent coordination strategies.

Conclusion: Black-box interaction analysis can surface emergent coordination strategies in LLM agents, offering a scalable complement to internal probe-based interpretability methods. LLMs demonstrate sophisticated coordination capabilities even without explicit role assignments.

Abstract: Large language models (LLMs) are increasingly deployed in collaborative settings, yet little is known about how they coordinate when treated as black-box agents. We simulate 7500 multi-agent, multi-round discussions in an inductive coding task, generating over 125000 utterances that capture both final annotations and their interactional histories. We introduce process-level metrics: code stability, semantic self-consistency, and lexical confidence alongside sentiment and convergence measures, to track coordination dynamics. To probe deeper alignment signals, we analyze the evolving geometry of output embeddings, showing that intrinsic dimensionality declines over rounds, suggesting semantic compression. The results reveal that LLM groups converge lexically and semantically, develop asymmetric influence patterns, and exhibit negotiation-like behaviors despite the absence of explicit role prompting. This work demonstrates how black-box interaction analysis can surface emergent coordination strategies, offering a scalable complement to internal probe-based interpretability methods.

Method: Adapt Graph Matching Networks (GMN) to operate on dependency parse trees, creating Tree Matching Networks (TMN). Compare TMN to BERT on SNLI entailment and SemEval similarity tasks. Propose multi-headed attention aggregation to address scalability limitations.

Result: TMN achieves significantly better results than BERT on SNLI task with reduced memory footprint and less training time. Both models struggle on SemEval task. Explicit structural representations outperform sequence-based models at comparable scales.

Conclusion: Dependency parse tree-based models (TMN) are more efficient and effective than BERT for NLI tasks, but current aggregation methods limit scalability. Multi-headed attention aggregation is proposed to address this limitation.

Abstract: In creating sentence embeddings for Natural Language Inference (NLI) tasks, using transformer-based models like BERT leads to high accuracy, but require hundreds of millions of parameters. These models take in sentences as a sequence of tokens, and learn to encode the meaning of the sequence into embeddings such that those embeddings can be used reliably for NLI tasks. Essentially, every word is considered against every other word in the sequence, and the transformer model is able to determine the relationships between them, entirely from scratch. However, a model that accepts explicit linguistic structures like dependency parse trees may be able to leverage prior encoded information about these relationships, without having to learn them from scratch, thus improving learning efficiency. To investigate this, we adapt Graph Matching Networks (GMN) to operate on dependency parse trees, creating Tree Matching Networks (TMN). We compare TMN to a BERT based model on the SNLI entailment task and on the SemEval similarity task. TMN is able to achieve significantly better results with a significantly reduced memory footprint and much less training time than the BERT based model on the SNLI task, while both models struggled to preform well on the SemEval. Explicit structural representations significantly outperform sequence-based models at comparable scales, but current aggregation methods limit scalability. We propose multi-headed attention aggregation to address this limitation.

Method: Agentic LLM framework integrating natural-language task interpretation, code generation, and data-driven reasoning applied to Universal Dependencies corpora for multilingual grammatical tasks inspired by WALS.

Result: Evaluated on 13 word-order features across 170+ languages using three dimensions: dominant-order accuracy, order-coverage completeness, and distributional fidelity, demonstrating feasibility of LLM reasoning with structured linguistic data.

Conclusion: Combining LLM reasoning with structured linguistic data offers a first step toward interpretable, scalable automation of corpus-based grammatical inquiry.

Abstract: Empirical grammar research has become increasingly data-driven, but the systematic analysis of annotated corpora still requires substantial methodological and technical effort. We explore how agentic large language models (LLMs) can streamline this process by reasoning over annotated corpora and producing interpretable, data-grounded answers to linguistic questions. We introduce an agentic framework for corpus-grounded grammatical analysis that integrates concepts such as natural-language task interpretation, code generation, and data-driven reasoning. As a proof of concept, we apply it to Universal Dependencies (UD) corpora, testing it on multilingual grammatical tasks inspired by the World Atlas of Language Structures (WALS). The evaluation spans 13 word-order features and over 170 languages, assessing system performance across three complementary dimensions - dominant-order accuracy, order-coverage completeness, and distributional fidelity - which reflect how well the system generalizes, identifies, and quantifies word-order variations. The results demonstrate the feasibility of combining LLM reasoning with structured linguistic data, offering a first step toward interpretable, scalable automation of corpus-based grammatical inquiry.

Method: Instruction-tune a 12B GEMMA 3 model with 4-bit precision and PEFT using Alpaca-style formatting. Use deterministic, constraint-aware decoding with lightweight normalizer for minimal edits. Employ classifier-informed prompts based on error taxonomy and distributions.

Result: Achieved 92.41 GLEU on Malayalam (6th overall) and 81.44 GLEU on Hindi (3rd overall) under official untuned evaluation, showing competitive performance.

Conclusion: Classifier-informed prompts, adapter-based instruction tuning, and deterministic decoding provide reproducible, computationally efficient alternative to augmentation pipelines for Indic GEC, motivating future work on stronger constraints and human evaluation.

Abstract: Grammatical error correction for Indic languages faces limited supervision, diverse scripts, and rich morphology. We propose an augmentation-free setup that uses instruction-tuned large language models and conservative decoding. A 12B GEMMA 3 model is instruction-tuned in bnb 4-bit precision with parameter-efficient fine-tuning (PEFT) and Alpaca-style formatting. Decoding follows a deterministic, constraint-aware procedure with a lightweight normaliser that encourages minimal, meaning-preserving edits. We operationalise inference, subsequent to instruction fine-tuning (IFT), via a fixed, language-specific prompt directly synthesised from a deterministic error classifier’s taxonomy, label distributions, and precedence ordering computed on the training corpus. Under the official untuned GLEU evaluation, the system scores 92.41 on Malayalam, sixth overall, and 81.44 on Hindi, third overall. These results indicate that classifier-informed prompt design, adapter-based instruction tuning, and deterministic decoding provide a reproducible and a computationally efficient alternative to augmentation-centred pipelines for Indic GEC. The approach also motivates future work on stronger morphosyntactic constraints and human-centred evaluation of conservative edits.

Method: Introduces MAC-SLU dataset with authentic multi-intent automotive cabin data, then benchmarks leading open-source LLMs and LALMs using in-context learning, supervised fine-tuning, and both end-to-end and pipeline paradigms.

Result: LLMs and LALMs can complete SLU tasks via in-context learning but performance lags significantly behind supervised fine-tuning. End-to-end LALMs show comparable performance to pipeline approaches while avoiding speech recognition error propagation.

Conclusion: MAC-SLU provides a challenging benchmark for SLU evaluation, demonstrating current limitations of LLMs/LALMs via in-context learning while showing promise of end-to-end approaches that avoid error propagation.

Abstract: Spoken Language Understanding (SLU), which aims to extract user semantics to execute downstream tasks, is a crucial component of task-oriented dialog systems. Existing SLU datasets generally lack sufficient diversity and complexity, and there is an absence of a unified benchmark for the latest Large Language Models (LLMs) and Large Audio Language Models (LALMs). This work introduces MAC-SLU, a novel Multi-Intent Automotive Cabin Spoken Language Understanding Dataset, which increases the difficulty of the SLU task by incorporating authentic and complex multi-intent data. Based on MAC-SLU, we conducted a comprehensive benchmark of leading open-source LLMs and LALMs, covering methods like in-context learning, supervised fine-tuning (SFT), and end-to-end (E2E) and pipeline paradigms. Our experiments show that while LLMs and LALMs have the potential to complete SLU tasks through in-context learning, their performance still lags significantly behind SFT. Meanwhile, E2E LALMs demonstrate performance comparable to pipeline approaches and effectively avoid error propagation from speech recognition. Code\footnote{https://github.com/Gatsby-web/MAC_SLU} and datasets\footnote{huggingface.co/datasets/Gatsby1984/MAC_SLU} are released publicly.

Method: Proposes an end-to-end fusion approach connecting hidden states from multiple layers of a pretrained multimodal foundation model (Omni 2.5-7B) to a translation LLM (SeedX PPO-7B). Uses a novel fusion strategy that enables joint end-to-end training for multimodal translation tasks.

Result: OmniFusion effectively leverages both audio and visual inputs, achieves 1-second latency reduction in simultaneous speech translation compared to cascaded pipelines, and improves overall translation quality across speech-to-text, speech-and-image-to-text, and text-and-image-to-text translation tasks.

Conclusion: The proposed fusion approach successfully combines the strengths of multimodal foundation models and specialized translation LLMs, creating an effective end-to-end multimodal translation system that reduces latency and improves translation quality while enabling multimodal context exploitation.

Abstract: There has been significant progress in open-source text-only translation large language models (LLMs) with better language coverage and quality. However, these models can be only used in cascaded pipelines for speech translation (ST), performing automatic speech recognition first followed by translation. This introduces additional latency, which is particularly critical in simultaneous ST (SimulST), and prevents the model from exploiting multimodal context, such as images, which can aid disambiguation. Pretrained multimodal foundation models (MMFMs) already possess strong perception and reasoning capabilities across multiple modalities, but generally lack the multilingual coverage and specialized translation performance of dedicated translation LLMs. To build an effective multimodal translation system, we propose an end-to-end approach that fuses MMFMs with translation LLMs. We introduce a novel fusion strategy that connects hidden states from multiple layers of a pretrained MMFM to a translation LLM, enabling joint end-to-end training. The resulting model, OmniFusion, built on Omni 2.5-7B as the MMFM and SeedX PPO-7B as the translation LLM, can perform speech-to-text, speech-and-image-to-text, and text-and-image-to-text translation. Experiments demonstrate that OmniFusion effectively leverages both audio and visual inputs, achieves a 1-second latency reduction in SimulST compared to cascaded pipelines and also improves the overall translation quality\footnote{Code is available at https://github.com/saikoneru/OmniFusion}.

Method: Adapted compact Conformer architecture to raw 306-channel MEG signals with lightweight convolutional projection layer and task-specific heads. Used MEG-oriented SpecAugment for Speech Detection, inverse-square-root class weighting and dynamic grouping loader for Phoneme Classification, plus instance-level normalization to handle distribution shifts.

Result: Achieved 88.9% on Speech Detection and 65.8% on Phoneme Classification on the competition leaderboard, surpassing baselines and ranking within top-10 in both tasks using official Standard track splits and F1-macro for model selection.

Conclusion: Conformer-based decoders with task-specific adaptations effectively handle MEG signal processing for speech-related tasks, with instance normalization proving critical for robustness against distribution shifts. The approach demonstrates competitive performance in the LibriBrain competition.

Abstract: We present Conformer-based decoders for the LibriBrain 2025 PNPL competition, targeting two foundational MEG tasks: Speech Detection and Phoneme Classification. Our approach adapts a compact Conformer to raw 306-channel MEG signals, with a lightweight convolutional projection layer and task-specific heads. For Speech Detection, a MEG-oriented SpecAugment provided a first exploration of MEG-specific augmentation. For Phoneme Classification, we used inverse-square-root class weighting and a dynamic grouping loader to handle 100-sample averaged examples. In addition, a simple instance-level normalization proved critical to mitigate distribution shifts on the holdout split. Using the official Standard track splits and F1-macro for model selection, our best systems achieved 88.9% (Speech) and 65.8% (Phoneme) on the leaderboard, surpassing the competition baselines and ranking within the top-10 in both tasks. For further implementation details, the technical documentation, source code, and checkpoints are available at https://github.com/neural2speech/libribrain-experiments.

Method: Collected 247 unique phrases for four mood states across 11 Indic languages. Tested seven conditions comparing k-means clustering on direct embeddings of native/Romanised scripts vs. embeddings of phrases translated to English/Chinese. Used multilingual and Indic-specific models, measuring performance with composite scores based on multiple metrics.

Result: Direct embedding of Indic languages failed completely (Composite Score = 0.002). Translation-based approaches showed significant improvement, with best performance from human-translated English further translated to Chinese and embedded with Chinese model (Composite = 0.67). Specialized Indic models performed poorly.

Conclusion: Current models cannot meaningfully represent mood states directly from Indic languages, creating a barrier for psychiatric applications in India. While translation bridges this gap, reliance on proprietary models is unsustainable. Models must be built to understand diverse local languages for effective global mental health applications.

Abstract: Background: Large Language Models show promise in psychiatry but are English-centric. Their ability to understand mood states in other languages is unclear, as different languages have their own idioms of distress. Aim: To quantify the ability of language models to faithfully represent phrases (idioms of distress) of four distinct mood states (depression, euthymia, euphoric mania, dysphoric mania) expressed in Indian languages. Methods: We collected 247 unique phrases for the four mood states across 11 Indic languages. We tested seven experimental conditions, comparing k-means clustering performance on: (a) direct embeddings of native and Romanised scripts (using multilingual and Indic-specific models) and (b) embeddings of phrases translated to English and Chinese. Performance was measured using a composite score based on Adjusted Rand Index, Normalised Mutual Information, Homogeneity and Completeness. Results: Direct embedding of Indic languages failed to cluster mood states (Composite Score = 0.002). All translation-based approaches showed significant improvement. High performance was achieved using Gemini-translated English (Composite=0.60) and human-translated English (Composite=0.61) embedded with gemini-001. Surprisingly, human-translated English, further translated into Chinese and embedded with a Chinese model, performed best (Composite = 0.67). Specialised Indic models (IndicBERT and Sarvam-M) performed poorly. Conclusion: Current models cannot meaningfully represent mood states directly from Indic languages, posing a fundamental barrier to their psychiatric application for diagnostic or therapeutic purposes in India. While high-quality translation bridges this gap, reliance on proprietary models or complex translation pipelines is unsustainable. Models must first be built to understand diverse local languages to be effective in global mental health.

Method: Created EduEval benchmark with three key contributions: (1) EduAbility Taxonomy unifying Bloom’s Taxonomy and Webb’s Depth of Knowledge across six cognitive dimensions; (2) Integration of real exam questions, classroom conversations, student essays, and expert prompts; (3) 24 task types with 11,000+ questions spanning primary to high school levels. Evaluated 14 LLMs under zero-shot and few-shot settings.

Result: Models perform well on factual tasks but struggle with classroom dialogue classification and show inconsistent creative content generation. Open-source models sometimes outperform proprietary systems on complex educational reasoning. Few-shot prompting effectiveness varies across cognitive dimensions.

Conclusion: Different educational objectives require tailored approaches, and EduEval provides targeted benchmarking metrics for developing LLMs optimized for diverse Chinese educational tasks.

Abstract: Large language models (LLMs) demonstrate significant potential for educational applications. However, their unscrutinized deployment poses risks to educational standards, underscoring the need for rigorous evaluation. We introduce EduEval, a comprehensive hierarchical benchmark for evaluating LLMs in Chinese K-12 education. This benchmark makes three key contributions: (1) Cognitive Framework: We propose the EduAbility Taxonomy, which unifies Bloom’s Taxonomy and Webb’s Depth of Knowledge to organize tasks across six cognitive dimensions including Memorization, Understanding, Application, Reasoning, Creativity, and Ethics. (2) Authenticity: Our benchmark integrates real exam questions, classroom conversation, student essays, and expert-designed prompts to reflect genuine educational challenges; (3) Scale: EduEval comprises 24 distinct task types with over 11,000 questions spanning primary to high school levels. We evaluate 14 leading LLMs under both zero-shot and few-shot settings, revealing that while models perform well on factual tasks, they struggle with classroom dialogue classification and exhibit inconsistent results in creative content generation. Interestingly, several open source models outperform proprietary systems on complex educational reasoning. Few-shot prompting shows varying effectiveness across cognitive dimensions, suggesting that different educational objectives require tailored approaches. These findings provide targeted benchmarking metrics for developing LLMs specifically optimized for diverse Chinese educational tasks.

Method: Benchmarked 47 CBQA models from Hugging Face on eight different datasets, analyzed performance factors (context length, model size, answer length, context complexity), and used Genetic algorithm to improve accuracy by integrating responses from other models.

Result: ahotrod/electra_large_discriminator_squad2_512 performed best overall (43% accuracy across all datasets), with top performance on bioasq10b-factoid (65.92%), biomedical_cpgQA (96.45%), QuAC (11.13%), and Question Answer Dataset (41.6%). Bert-large-uncased-whole-word-masking-finetuned-squad achieved 82% on IELTS dataset.

Conclusion: Best-performing models were trained on SQuAD datasets, performance depends on context length, model size, answer length, and context complexity, and model integration via Genetic algorithm can improve overall accuracy.

Abstract: Context-based question answering (CBQA) models provide more accurate and relevant answers by considering the contextual information. They effectively extract specific information given a context, making them functional in various applications involving user support, information retrieval, and educational platforms. In this manuscript, we benchmarked the performance of 47 CBQA models from Hugging Face on eight different datasets. This study aims to identify the best-performing model across diverse datasets without additional fine-tuning. It is valuable for practical applications where the need to retrain models for specific datasets is minimized, streamlining the implementation of these models in various contexts. The best-performing models were trained on the SQuAD v2 or SQuAD v1 datasets. The best-performing model was ahotrod/electra_large_discriminator_squad2_512, which yielded 43% accuracy across all datasets. We observed that the computation time of all models depends on the context length and the model size. The model’s performance usually decreases with an increase in the answer length. Moreover, the model’s performance depends on the context complexity. We also used the Genetic algorithm to improve the overall accuracy by integrating responses from other models. ahotrod/electra_large_discriminator_squad2_512 generated the best results for bioasq10b-factoid (65.92%), biomedical_cpgQA (96.45%), QuAC (11.13%), and Question Answer Dataset (41.6%). Bert-large-uncased-whole-word-masking-finetuned-squad achieved an accuracy of 82% on the IELTS dataset.

Method: Three-step SQL-based approach: (1) generate 3NF schema from Wikipedia infoboxes, (2) generate SQL queries, (3) execute queries. Focuses on schema design principles including normalization preserving context, semantic naming reducing ambiguity, and consistent temporal anchoring.

Result: Best configuration (Gemini 2.5 Flash schema + Gemini-2.0-Flash queries) achieves 80.39 EM, a 16.8% improvement over baseline (68.89 EM). Key finding: schema design quality has greater impact on QA precision than model capacity.

Conclusion: For temporal reasoning over semi-structured tables, high-quality schema design (with proper normalization, semantic naming, and temporal anchoring) is more critical than model scaling, establishing evidence-based principles for effective QA systems.

Abstract: Temporal reasoning over evolving semi-structured tables poses a challenge to current QA systems. We propose a SQL-based approach that involves (1) generating a 3NF schema from Wikipedia infoboxes, (2) generating SQL queries, and (3) query execution. Our central finding challenges model scaling assumptions: the quality of schema design has a greater impact on QA precision than model capacity. We establish three evidence-based principles: normalization that preserves context, semantic naming that reduces ambiguity, and consistent temporal anchoring. Our best configuration (Gemini 2.5 Flash schema + Gemini-2.0-Flash queries) achieves 80.39 EM, a 16.8% improvement over the baseline (68.89 EM).

Method: Proposes Assertion-Conditioned Compliance (A-CC) evaluation paradigm with holistic metrics to assess model behavior when confronted with two types of misleading assertions: User-Sourced Assertions (USAs) that test sycophancy toward misinformed user beliefs, and Function-Sourced Assertions (FSAs) that test compliance with contradictory system policies.

Result: Models show high vulnerability to both USA sycophancy (complying with plausible but misinformed user beliefs) and FSA policy conflicts (complying with contradictory system policies), revealing A-CC as a critical latent vulnerability in deployed agents.

Conclusion: A-CC identifies significant robustness gaps in multi-turn function-calling LLMs, highlighting the need for better evaluation of conversation-level vulnerabilities in safety-critical applications where models interact with real-world systems.

Abstract: Multi-turn tool-calling LLMs (models capable of invoking external APIs or tools across several user turns) have emerged as a key feature in modern AI assistants, enabling extended dialogues from benign tasks to critical business, medical, and financial operations. Yet implementing multi-turn pipelines remains difficult for many safety-critical industries due to ongoing concerns regarding model resilience. While standardized benchmarks such as the Berkeley Function-Calling Leaderboard (BFCL) have underpinned confidence concerning advanced function-calling models (like Salesforce’s xLAM V2), there is still a lack of visibility into multi-turn conversation-level robustness, especially given their exposure to real-world systems. In this paper, we introduce Assertion-Conditioned Compliance (A-CC), a novel evaluation paradigm for multi-turn function-calling dialogues. A-CC provides holistic metrics that evaluate a model’s behavior when confronted with misleading assertions originating from two distinct vectors: (1) user-sourced assertions (USAs), which measure sycophancy toward plausible but misinformed user beliefs, and (2) function-sourced assertions (FSAs), which measure compliance with plausible but contradictory system policies (e.g., stale hints from unmaintained tools). Our results show that models are highly vulnerable to both USA sycophancy and FSA policy conflicts, confirming A-CC as a critical, latent vulnerability in deployed agents.

Method: Two key strategies: 1) Lightweight modular ASR fine-tuning of whisper-small using language-specific experts, and 2) Knowledge distillation from whisper-large-v2 to boost performance while retaining robustness.

Result: More effective than standard fine-tuning or LoRA adapters, improves performance for targeted languages on both in- and out-of-domain test sets with negligible parameter overhead at inference.

Conclusion: DistilWhisper successfully bridges the ASR performance gap for under-represented languages while preserving the multitask and multilingual advantages of the original Whisper architecture.

Abstract: Whisper is a multitask and multilingual speech model covering 99 languages. It yields commendable automatic speech recognition (ASR) results in a subset of its covered languages, but the model still underperforms on a non-negligible number of under-represented languages, a problem exacerbated in smaller model versions. In this work, we propose DistilWhisper, an approach able to bridge the performance gap in ASR for these languages while retaining the advantages of multitask and multilingual capabilities. Our approach involves two key strategies: lightweight modular ASR fine-tuning of whisper-small using language-specific experts, and knowledge distillation from whisper-large-v2. This dual approach allows us to effectively boost ASR performance while keeping the robustness inherited from the multitask and multilingual pre-training. Results demonstrate that our approach is more effective than standard fine-tuning or LoRA adapters, boosting performance in the targeted languages for both in- and out-of-domain test sets, while introducing only a negligible parameter overhead at inference.

Method: Multi-Task Correction MoE with experts specialized for speech-to-text, language-to-text, and vision-to-text datasets, using token routing to mapped experts.

Result: Achieved 5.0% average relative WER reduction on Open ASR Leaderboard, outperformed GPT-3.5 and Claude-Opus by 15.5-27.6% in zero-shot evaluation, and performed competitively on grammar and post-OCR correction.

Conclusion: MoEs are effective for multi-domain error correction, enabling single-model expertise across domains without parameter explosion, establishing new SOTA performance.

Abstract: Construction of a general-purpose post-recognition error corrector poses a crucial question: how can we most effectively train a model on a large mixture of domain datasets? The answer would lie in learning dataset-specific features and digesting their knowledge in a single model. Previous methods achieve this by having separate correction language models, resulting in a significant increase in parameters. In this work, we present Mixture-of-Experts as a solution, highlighting that MoEs are much more than a scalability tool. We propose a Multi-Task Correction MoE, where we train the experts to become an ``expert’’ of speech-to-text, language-to-text and vision-to-text datasets by learning to route each dataset’s tokens to its mapped expert. Experiments on the Open ASR Leaderboard show that we explore a new state-of-the-art performance by achieving an average relative 5.0% WER reduction and substantial improvements in BLEU scores for speech and translation tasks. On zero-shot evaluation, NeKo outperforms GPT-3.5 and Claude-Opus with 15.5% to 27.6% relative WER reduction in the Hyporadise benchmark. NeKo performs competitively on grammar and post-OCR correction as a multi-task model.

Method: Created a human-curated benchmark with over 13,000 multiple-choice questions covering 11 Indic languages (Nepali, Gujarati, Marathi, Odia as low-resource; Dogri, Maithili, Rajasthani, Sanskrit, Bodo, Santali, Konkani as extremely low-resource) plus Sanskrit-English code-mixed set. Evaluated 19 LLMs (proprietary and open-weights) and analyzed performance across different question types including knowledge-oriented vs. linguistic questions, and diverse formats like list-based matching, assertion-reason pairs, and sequence ordering.

Result: Even top-performing GPT-5 reached only 45.0% average accuracy, followed by DeepSeek-3.2 (43.1%) and Claude-4.5 (42.7%). The benchmark reveals significant limitations in cross-lingual transfer for Indic languages and establishes challenging performance baselines.

Conclusion: IndicParam provides crucial insights into LLM limitations for low-resource Indic languages, establishes a challenging benchmark, and highlights the need for improved cross-lingual transfer capabilities. The dataset and evaluation scripts are publicly available for further research.

Abstract: While large language models excel on high-resource multilingual tasks, low- and extremely low-resource Indic languages remain severely under-evaluated. We present IndicParam, a human-curated benchmark of over 13,000 multiple-choice questions covering 11 such languages (Nepali, Gujarati, Marathi, Odia as low-resource; Dogri, Maithili, Rajasthani, Sanskrit, Bodo, Santali, Konkani as extremely low-resource) plus Sanskrit-English code-mixed set. We evaluated 19 LLMs, both proprietary and open-weights, which reveals that even the top-performing GPT-5 reaches only 45.0% average accuracy, followed by DeepSeek-3.2 (43.1) and Claude-4.5 (42.7). We additionally label each question as knowledge-oriented or purely linguistic to discriminate factual recall from grammatical proficiency. Further, we assess the ability of LLMs to handle diverse question formats-such as list-based matching, assertion-reason pairs, and sequence ordering-alongside conventional multiple-choice questions. IndicParam provides insights into limitations of cross-lingual transfer and establishes a challenging benchmark for Indic languages. The dataset is available at https://huggingface.co/datasets/bharatgenai/IndicParam. Scripts to run benchmark are present at https://github.com/ayushbits/IndicParam.

Method: Empirical evaluation of VITS and Style-BERT-VITS2 JP Extra using three character-specific datasets, assessing naturalness (MOS, CMOS), intelligibility (WER), and speaker consistency.

Result: SBV2JE matches human ground truth in naturalness (MOS 4.37 vs 4.38), achieves lower WER, shows slight CMOS preference, and demonstrates effective pitch-accent control despite higher computational demands.

Conclusion: SBV2JE proves effective for Japanese character speech applications like language learning and dialogue generation, outperforming VITS through enhanced pitch-accent controls and WavLM-based discriminator.

Abstract: Synthesizing expressive Japanese character speech poses unique challenges due to pitch-accent sensitivity and stylistic variability. This paper empirically evaluates two open-source text-to-speech models–VITS and Style-BERT-VITS2 JP Extra (SBV2JE)–on in-domain, character-driven Japanese speech. Using three character-specific datasets, we evaluate models across naturalness (mean opinion and comparative mean opinion score), intelligibility (word error rate), and speaker consistency. SBV2JE matches human ground truth in naturalness (MOS 4.37 vs. 4.38), achieves lower WER, and shows slight preference in CMOS. Enhanced by pitch-accent controls and a WavLM-based discriminator, SBV2JE proves effective for applications like language learning and character dialogue generation, despite higher computational demands.

Method: SIQ evaluates voice understanding LLMs across three cognitive levels: (1) Remembering - using WER for verbatim accuracy, (2) Understanding - measuring similarity of LLM’s interpretations, and (3) Application - using QA accuracy for simulating downstream tasks. The framework bridges cognitive principles with voice-oriented benchmarks.

Result: SIQ successfully quantifies voice understanding abilities, provides unified comparisons between cascaded (ASR+LLM) and end-to-end models, identifies annotation errors in existing benchmarks, and detects hallucinations in LLM Voice systems.

Conclusion: SIQ represents a first-of-its-kind intelligence examination that bridges cognitive principles with voice understanding evaluation, exposing overlooked challenges in multi-modal training while providing a comprehensive assessment framework for voice understanding LLMs.

Abstract: We introduce Speech-based Intelligence Quotient (SIQ) as a new form of human cognition-inspired evaluation pipeline for voice understanding large language models, LLM Voice, designed to assess their voice understanding ability. Moving beyond popular voice understanding metrics such as word error rate (WER), SIQ examines LLM Voice across three cognitive levels motivated by Bloom’s Taxonomy: (1) Remembering (i.e., WER for verbatim accuracy); (2) Understanding (i.e., similarity of LLM’s interpretations); and (3) Application (i.e., QA accuracy for simulating downstream tasks). We demonstrate that SIQ not only quantifies voice understanding abilities but also provides unified comparisons between cascaded methods (e.g., ASR LLM) and end-to-end models, identifies annotation errors in existing benchmarks, and detects hallucinations in LLM Voice. Our framework represents a first-of-its-kind intelligence examination that bridges cognitive principles with voice-oriented benchmarks, while exposing overlooked challenges in multi-modal training. Our code and data will be open source to encourage future studies.

Method: Propose CrossFusion-RAG: combines frozen encoders, learned vision projection (512→768), shallow query-agnostic cross-attention over ASR and frames with temporal-consistency regularization, and small cross-attentive reranker. Evaluated on CourseTimeQA dataset (52.3h, 902 queries across 6 courses).

Result: Improves nDCG@10 by 0.10 and MRR by 0.08 over strong BLIP-2 retriever, achieves ~1.55s median end-to-end latency on single A100. Outperforms multiple baselines under matched hardware/indexing constraints, shows robustness to ASR noise.

Conclusion: CrossFusion-RAG provides effective, latency-constrained solution for educational video QA, balancing retrieval performance with practical deployment constraints on limited hardware.

Abstract: We study timestamped question answering over educational lecture videos under a single-GPU latency/memory budget. Given a natural-language query, the system retrieves relevant timestamped segments and synthesizes a grounded answer. We present CourseTimeQA (52.3 h, 902 queries across six courses) and a lightweight, latency-constrained cross-modal retriever (CrossFusion-RAG) that combines frozen encoders, a learned 512->768 vision projection, shallow query-agnostic cross-attention over ASR and frames with a temporal-consistency regularizer, and a small cross-attentive reranker. On CourseTimeQA, CrossFusion-RAG improves nDCG@10 by 0.10 and MRR by 0.08 over a strong BLIP-2 retriever while achieving approximately 1.55 s median end-to-end latency on a single A100. Closest comparators (zero-shot CLIP multi-frame pooling; CLIP + cross-encoder reranker + MMR; learned late-fusion gating; text-only hybrid with cross-encoder reranking and its MMR variant; caption-augmented text retrieval; non-learned temporal smoothing) are evaluated under matched hardware and indexing. We report robustness across ASR noise (WER quartiles), diagnostics for temporal localization, and full training/tuning details to support reproducible comparison.

Method: Investigated how Big Five personality profiles in LLMs influence priming in relevance labeling using multiple LLMs on TREC 2021 and 2022 Deep Learning Track datasets.

Result: Certain personality profiles (High Openness and Low Neuroticism) consistently reduce priming susceptibility. The most effective personality for mitigating priming varies across models and task types.

Conclusion: Propose personality prompting as a method to mitigate threshold priming, connecting psychological evidence with LLM-based evaluation practices.

Abstract: Recent research has explored LLMs as scalable tools for relevance labeling, but studies indicate they are susceptible to priming effects, where prior relevance judgments influence later ones. Although psychological theories link personality traits to such biases, it is unclear whether simulated personalities in LLMs exhibit similar effects. We investigate how Big Five personality profiles in LLMs influence priming in relevance labeling, using multiple LLMs on TREC 2021 and 2022 Deep Learning Track datasets. Our results show that certain profiles, such as High Openness and Low Neuroticism, consistently reduce priming susceptibility. Additionally, the most effective personality in mitigating priming may vary across models and task types. Based on these findings, we propose personality prompting as a method to mitigate threshold priming, connecting psychological evidence with LLM-based evaluation practices.

Method: InnoGym introduces two complementary metrics: performance gain (improvement over best-known solutions) and novelty (methodological differences from prior approaches). The benchmark includes 18 curated real-world tasks with standardized resource filtering, evaluator validation, and solution collection. Also provides iGym, a unified execution environment for reproducible, long-horizon evaluations.

Result: Experiments show some agents produce novel approaches but lack robustness, limiting performance gains. This reveals a gap between creativity and effectiveness in current AI agents.

Conclusion: There’s a need for benchmarks that evaluate both correctness and innovation. InnoGym addresses this gap by systematically assessing innovation potential through performance gain and novelty metrics, highlighting the importance of balancing creativity with effectiveness.

Abstract: LLMs and Agents have achieved impressive progress in code generation, mathematical reasoning, and scientific discovery. However, existing benchmarks primarily measure correctness, overlooking the diversity of methods behind solutions. True innovation depends not only on producing correct answers but also on the originality of the approach. We present InnoGym, the first benchmark and framework designed to systematically evaluate the innovation potential of AI agents. InnoGym introduces two complementary metrics: performance gain, which measures improvement over the best-known solutions, and novelty, which captures methodological differences from prior approaches. The benchmark includes 18 carefully curated tasks from real-world engineering and scientific domains, each standardized through resource filtering, evaluator validation, and solution collection. In addition, we provide iGym, a unified execution environment for reproducible and long-horizon evaluations. Extensive experiments show that while some agents produce novel approaches, their lack of robustness limits performance gains. These results highlight a key gap between creativity and effectiveness, underscoring the need for benchmarks that evaluate both.

Method: Uses LLMs with Python-coded API calls, implements error taxonomy based on Corder (1967), Richards (1971), and James (1998), classifies errors at word/sentence levels (spelling, grammar, punctuation), tested on isolated errors and authentic text “English as she is spoke”.

Result: AI successfully identified diverse error types but showed limitations in contextual understanding and occasionally generated new error categories for uncoded errors.

Conclusion: Demonstrates AI’s potential to transform EFL instruction through automated error analysis, but needs further development for contextual accuracy and expansion to stylistic/discourse-level errors.

Abstract: This study describes the development of an AI-assisted error analysis system designed to identify, categorize, and correct writing errors in English. Utilizing Large Language Models (LLMs) like Claude 3.5 Sonnet and DeepSeek R1, the system employs a detailed taxonomy grounded in linguistic theories from Corder (1967), Richards (1971), and James (1998). Errors are classified at both word and sentence levels, covering spelling, grammar, and punctuation. Implemented through Python-coded API calls, the system provides granular feedback beyond traditional rubric-based assessments. Initial testing on isolated errors refined the taxonomy, addressing challenges like overlapping categories. Final testing used “English as she is spoke” by Jose da Fonseca (1855), a text rich with authentic linguistic errors, to evaluate the system’s capacity for handling complex, multi-layered analysis. The AI successfully identified diverse error types but showed limitations in contextual understanding and occasionally generated new error categories when encountering uncoded errors. This research demonstrates AI’s potential to transform EFL instruction by automating detailed error analysis and feedback. While promising, further development is needed to improve contextual accuracy and expand the taxonomy to stylistic and discourse-level errors.

Method: Created a live dynamic benchmark with 50 questions per month designed by crypto-native professionals, categorized into four quadrants: Simple Retrieval, Complex Retrieval, Simple Prediction, and Complex Prediction. Evaluated 10 LLMs both directly and within agentic frameworks.

Result: Revealed a performance hierarchy and uncovered a “retrieval-prediction imbalance” - many leading models are proficient at data retrieval but show pronounced weakness in predictive analysis tasks, highlighting models that appear factually grounded but lack deeper analytical synthesis capabilities.

Conclusion: CryptoBench provides a more challenging and valuable scenario for LLM agent assessment in cryptocurrency domains, exposing critical gaps in current models’ analytical capabilities despite their retrieval strengths, and establishes a framework for more rigorous evaluation of real-world crypto analysis performance.

Abstract: This paper introduces CryptoBench, the first expert-curated, dynamic benchmark designed to rigorously evaluate the real-world capabilities of Large Language Model (LLM) agents in the uniquely demanding and fast-paced cryptocurrency domain. Unlike general-purpose agent benchmarks for search and prediction, professional crypto analysis presents specific challenges: \emph{extreme time-sensitivity}, \emph{a highly adversarial information environment}, and the critical need to synthesize data from \emph{diverse, specialized sources}, such as on-chain intelligence platforms and real-time Decentralized Finance (DeFi) dashboards. CryptoBench thus serves as a much more challenging and valuable scenario for LLM agent assessment. To address these challenges, we constructed a live, dynamic benchmark featuring 50 questions per month, expertly designed by crypto-native professionals to mirror actual analyst workflows. These tasks are rigorously categorized within a four-quadrant system: Simple Retrieval, Complex Retrieval, Simple Prediction, and Complex Prediction. This granular categorization enables a precise assessment of an LLM agent’s foundational data-gathering capabilities alongside its advanced analytical and forecasting skills. Our evaluation of ten LLMs, both directly and within an agentic framework, reveals a performance hierarchy and uncovers a failure mode. We observe a \textit{retrieval-prediction imbalance}, where many leading models, despite being proficient at data retrieval, demonstrate a pronounced weakness in tasks requiring predictive analysis. This highlights a problematic tendency for agents to appear factually grounded while lacking the deeper analytical capabilities to synthesize information.

Method: Four-stage framework: (1) decompose problem into sequential reasoning sub-problems, (2) assess difficulty of each sub-problem, (3) assign selective processing modes (System 1 for simple, System 2 for complex), (4) execute sequentially with context propagation.

Result: Achieves up to 13.75 percentage point accuracy improvement (57.50% to 71.25% on AIME25) while reducing computational costs by 33%-53%, significantly outperforming uniform scaling baselines.

Conclusion: SCALE represents a major advance in test-time scaling by addressing fundamental limitations of uniform resource allocation, enabling superior performance with better resource utilization through selective compute allocation based on sub-problem difficulty.

Abstract: Test-time compute scaling has emerged as a powerful paradigm for enhancing mathematical reasoning in large language models (LLMs) by allocating additional computational resources during inference. However, current methods employ uniform resource distribution across all reasoning sub-problems, creating fundamental bottlenecks where challenging sub-problems receive insufficient attention while routine operations consume disproportionate resources. This uniform allocation creates performance bottlenecks where additional computational resources yield diminishing returns. Inspired by dual-process theory, we propose \textbf{SCALE} (Selective Resource Allocation), a framework that selectively allocates computational resources based on sub-problem difficulty. SCALE operates through four stages: (1) problem decomposition into sequential reasoning sub-problems, (2) difficulty assessment of each sub-problem to distinguish between routine operations and computationally challenging sub-problems, (3) selective processing mode assignment between System 1 for simple sub-problems and System 2 for complex ones, and (4) sequential execution with context propagation. By concentrating resources on challenging sub-problems while processing routine operations efficiently, SCALE achieves substantial performance improvements with superior resource utilization. Extensive experiments demonstrate that SCALE significantly outperforms uniform scaling baselines, achieving accuracy improvements of up to 13.75 percentage points (57.50% to 71.25% on AIME25) while reducing computational costs by 33%-53%, representing a major advance in test-time scaling that addresses fundamental limitations of current approaches.

Method: Proposes CACARA architecture trained through emergent alignment learning, which allows seamless integration of new modalities into existing bimodal/multimodal models. Only fine-tunes newly incorporated modalities on English-aligned data, without explicit multilingual pretraining or text encoder tuning.

Result: Achieves up to 14.24 percentage points improvement in R@1 audio-to-text retrieval, outperforming state-of-the-art multimodal models. Gains support for over 100 languages without explicit multilingual training, with training cost comparable to monolingual models.

Conclusion: Emergent alignment learning enables efficient multimodal and multilingual capabilities, preserving previously learned knowledge while avoiding heavy computational costs of retraining across every modality and language.

Abstract: As deep learning models evolve, new applications and challenges are rapidly emerging. Tasks that once relied on a single modality, such as text, images, or audio, are now enriched by seamless interactions between multimodal data. These connections bridge information gaps: an image can visually materialize a text, while audio can add context to an image. Researchers have developed numerous multimodal models, but most rely on resource-intensive training across multiple modalities. Similarly, extending these models to new languages often follows the same resource-heavy training strategy. In this work, we propose a multimodal and multilingual architecture, CACARA, trained through emergent alignment learning, enabling the seamless integration of new modalities into an existing bimodal/multimodal model without requiring full retraining. This work breaks new ground by demonstrating that this emergent alignment paradigm can unlock multilingual capabilities from monolingual training. By fine-tuning the newly incorporated modality only on data aligned with the English language, our model develops support for over 100 languages without explicit multilingual pretraining or tuning of the text encoder. Such emergent multimodal and multilingual properties are gained efficiently, preserving previously learned knowledge at a training cost comparable to that of a monolingual model. Our strategy achieves up to a 14.24 percentage points improvement in R@1 audio-to-text retrieval, outperforming state-of-the-art multimodal models – all without the heavy computational cost of retraining across every modality and language.

Method: Proposes G-KV with global scoring mechanism combining local and historical attention scores to assess token importance. Uses post-training techniques including reinforcement learning and distillation to optimize models for compressed KV cache.

Result: The method is available as open source code on GitHub. The abstract suggests it addresses limitations of existing approaches by considering long-term token importance.

Conclusion: G-KV provides a more accurate KV cache eviction method through global scoring and post-training optimization, improving efficiency for reasoning LLMs with long sequences.

Abstract: Recent reasoning large language models (LLMs) excel in complex tasks but encounter significant computational and memory challenges due to long sequence lengths. KV cache compression has emerged as an effective approach to greatly enhance the efficiency of reasoning. However, existing methods often focus on prompt compression or token eviction with local attention score, overlooking the long-term importance of tokens. We propose G-KV, a KV cache eviction method that employs a global scoring mechanism, combining local and historical attention scores to more accurately assess token importance. Additionally, we introduce post-training techniques, including reinforcement learning and distillation, to optimize models for compressed KV cache settings. The code of this paper is available on: https://github.com/microsoft/G-KV.

Method: Combined unsupervised, semi-supervised, and supervised metrics to create novel feature sets; used classical ML methods; built novel neural architectures combining recurrent and recursive networks; created domain-specific polarity lexicons; performed fine morphological analysis for Turkish; developed sentiment-aware word embeddings; redefined context windows as subclauses.

Result: Achieved state-of-the-art results for both Turkish and English sentiment analysis; outperformed neural network models with classical ML approaches; created the first domain-specific polarity lexicon for Turkish; developed novel neural architectures and word embeddings.

Conclusion: This thesis represents the most detailed and comprehensive study on Turkish sentiment analysis as of July 2020, with significant contributions to both Turkish and English sentiment analysis through novel approaches that can be adapted to other languages and NLP tasks.

Abstract: In this thesis, we developed a comprehensive framework for sentiment analysis that takes its many aspects into account mainly for Turkish. We have also proposed several approaches specific to sentiment analysis in English only. We have accordingly made five major and three minor contributions. We generated a novel and effective feature set by combining unsupervised, semi-supervised, and supervised metrics. We then fed them as input into classical machine learning methods, and outperformed neural network models for datasets of different genres in both Turkish and English. We created a polarity lexicon with a semi-supervised domain-specific method, which has been the first approach applied for corpora in Turkish. We performed a fine morphological analysis for the sentiment classification task in Turkish by determining the polarities of morphemes. This can be adapted to other morphologically-rich or agglutinative languages as well. We have built a novel neural network architecture, which combines recurrent and recursive neural network models for English. We built novel word embeddings that exploit sentiment, syntactic, semantic, and lexical characteristics for both Turkish and English. We also redefined context windows as subclauses in modelling word representations in English. This can also be applied to other linguistic fields and natural language processing tasks. We have achieved state-of-the-art and significant results for all these original approaches. Our minor contributions include methods related to aspect-based sentiment in Turkish, parameter redefinition in the semi-supervised approach, and aspect term extraction techniques for English. This thesis can be considered the most detailed and comprehensive study made on sentiment analysis in Turkish as of July, 2020. Our work has also contributed to the opinion classification problem in English.

Method: Developed a diagnostic framework with four axes: forward-backward consistency, transitivity coverage, counterfactual sensitivity, and perturbation robustness. Applied this to Qwen3-0.6B on the MenatQA dataset.

Result: Qwen3-0.6B showed reasonable answer accuracy (70%+) but poor backward consistency (15%), limited transitivity coverage (32.2%), and brittle sensitivity to perturbations, indicating reliance on pattern matching rather than genuine reasoning.

Conclusion: The diagnostic framework exposes reasoning failures invisible to traditional accuracy metrics, suggesting current models may rely on superficial pattern matching. The framework is model-agnostic and released to enable broader assessment of reasoning fidelity.

Abstract: Current evaluation of mathematical reasoning in language models relies primarily on answer accuracy, potentially masking fundamental failures in logical computation. We introduce a diagnostic framework that distinguishes genuine mathematical reasoning from superficial pattern matching through four complementary axes: forward-backward consistency, transitivity coverage, counterfactual sensitivity, and perturbation robustness. Through a case study applying this framework to Qwen3-0.6B on the MenatQA dataset, we reveal a striking disconnect between surface performance and reasoning fidelity. While the model achieves reasonable answer accuracy (70%+), it demonstrates poor backward consistency (15%), limited transitivity coverage (32.2%), and brittle sensitivity to perturbations. Our diagnostics expose reasoning failures invisible to traditional accuracy metrics, suggesting that this small model relies heavily on pattern matching rather than genuine logical computation. While our empirical findings are based on a single 600M-parameter model, the diagnostic framework itself is model-agnostic and generalizable. We release our evaluation protocols to enable the research community to assess reasoning fidelity across different model scales and architectures, moving beyond surface-level accuracy toward verifiable mathematical reasoning.

Method: Creation of a new conceptual dictionary specifically for Slovak language - the first linguistic tool of this kind for Slovak.

Result: Development of the first conceptual dictionary for Slovak language, addressing the gap in machine-readable linguistic resources.

Conclusion: This new conceptual dictionary represents a significant step forward for Slovak NLP, enabling better automated text processing and addressing the resource limitations that have hindered Slovak language processing tasks.

Abstract: When solving tasks in the field of natural language processing, we sometimes need dictionary tools, such as lexicons, word form dictionaries or knowledge bases. However, the availability of dictionary data is insufficient in many languages, especially in the case of low resourced languages. In this article, we introduce a new conceptual dictionary for the Slovak language as the first linguistic tool of this kind. Since Slovak language is a language with limited linguistic resources and there are currently not available any machine-readable linguistic data sources with a sufficiently large volume of data, many tasks which require automated processing of Slovak text achieve weaker results compared to other languages and are almost impossible to solve.

Method: A multi-stage pipeline that: 1) extracts candidate triplets with qualifiers from text, 2) enforces Wikidata-based type and relation constraints for consistency, and 3) normalizes entities to reduce duplication. The approach focuses on creating compact, well-connected KGs.

Result: Achieves 96% correct answer entity coverage on MuSiQue, 76.0 F1 on HotpotQA, and 59.8 F1 on MuSiQue - matching or surpassing retrieval-augmented baselines. Sets SOTA on MINE-1 benchmark (86%). Highly efficient: uses <1,000 output tokens (3× fewer than AriGraph, <1/20 of GraphRAG).

Conclusion: Wikontic enhances KG quality and offers a scalable solution for leveraging structured knowledge in LLMs, demonstrating that high-quality KGs alone can match or exceed performance of systems requiring textual context.

Abstract: Knowledge graphs (KGs) provide structured, verifiable grounding for large language models (LLMs), but current LLM-based systems commonly use KGs as auxiliary structures for text retrieval, leaving their intrinsic quality underexplored. In this work, we propose Wikontic, a multi-stage pipeline that constructs KGs from open-domain text by extracting candidate triplets with qualifiers, enforcing Wikidata-based type and relation constraints, and normalizing entities to reduce duplication. The resulting KGs are compact, ontology-consistent, and well-connected; on MuSiQue, the correct answer entity appears in 96% of generated triplets. On HotpotQA, our triplets-only setup achieves 76.0 F1, and on MuSiQue 59.8 F1, matching or surpassing several retrieval-augmented generation baselines that still require textual context. In addition, Wikontic attains state-of-the-art information-retention performance on the MINE-1 benchmark (86%), outperforming prior KG construction methods. Wikontic is also efficient at build time: KG construction uses less than 1,000 output tokens, about 3$\times$ fewer than AriGraph and $<$1/20 of GraphRAG. The proposed pipeline enhances the quality of the generated KG and offers a scalable solution for leveraging structured knowledge in LLMs.

Method: Prism is built around Core1, a minimal background grammar with categories (numbers, strings, user prompts, tools) and abstract combinators (booleans, predicates, pairs, lists). Agent policies are written as ordinary expressions using a single abstraction operator, with conditionals as selections between alternatives. Domains extend the core by defining context-mini-grammars that introduce new categories, predicates, and external tools while reusing the same compositional machinery.

Result: The authors demonstrate Prism through worked examples from thermostat control, home security, e-commerce recommendation, and medical monitoring, showing how natural language decision rules can be mapped to inspectable, executable policies. The approach provides a compact interface language for agent control that makes action spaces explicit.

Conclusion: Prism offers both linguistic and engineering benefits: linguistically, it enforces separation between reusable grammar-like core and domain-specific lexicons, treating tools as bridges between internal representations and external world; engineering-wise, it provides a compact interface language amenable to analysis, verification, and safety constraints for tool-using agents.

Abstract: Prism is a small, compositional metalanguage for specifying the behaviour of tool-using software agents. Rather than introducing ad hoc control constructs, Prism is built around a fixed core context, Core1, which provides a minimal background grammar of categories numbers, strings, user prompts, tools together with abstract combinators for booleans, predicates, pairs, and lists. Agent policies are written as ordinary expressions using a single abstraction operator so that conditionals appear as selections between alternatives instead of imperative if-else blocks. Domains extend the core by defining their own context-mini-grammars that introduce new categories, predicates, and external tools while reusing the same compositional machinery. We illustrate this with worked examples from thermostat control, home security, e-commerce recommendation, and medical monitoring, showing how natural language decision rules can be mapped to inspectable, executable policies. From a linguistic perspective, Prism enforces a clear separation between a reusable grammar-like core and domain specific lexicons and treats tools as bridges between internal policy representations and the external world. From an engineering perspective, it offers a compact interface language for agent control, making the space of possible actions explicit and amenable to analysis, verification, and safety constraints.

Method: Tournament-style ELO ranking with multi-agent reasoning where multiple LLM agents compete, critique, and collaborate through structured workflows with configurable parameters, dynamic agent selection, and consensus fusion strategies.

Result: Significant improvements in response accuracy, coherence, and reliability with 8.4% overall quality improvement and R² values exceeding 0.96 in ELO rating convergence.

Conclusion: ART provides a scalable, production-ready solution for high-quality, vetted LLM responses by leveraging multi-agent competition and collaboration to outperform individual models.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding and generation. However, single-model responses often exhibit inconsistencies, hallucinations, and varying quality across different query domains. This paper presents ART (Adaptive Response Tuning), a novel framework that employs tournament-style ELO ranking and multi-agent reasoning to systematically optimize LLM outputs. By enabling multiple LLM agents to compete, critique, and collaborate through structured tournament workflows, ART produces consensus responses that outperform individual model outputs. Our framework introduces configurable tournament parameters, dynamic agent selection, and multiple consensus fusion strategies. Experimental evaluations demonstrate significant improvements in response accuracy, coherence, and reliability compared to baseline single-model approaches. The ART framework provides a scalable, production-ready solution for applications requiring high-quality, vetted LLM responses, achieving an 8.4% improvement in overall quality metrics and R22 values exceeding 0.96 in ELO rating convergence.

Method: The method involves reviewing existing literature on LLM sycophancy, identifying five core operationalizations of sycophancy measurement, analyzing difficulties in distinguishing sycophantic responses from related AI alignment concepts, and conducting methodological analysis.

Result: The paper identifies five core operationalizations for measuring LLM sycophancy and highlights that current research fails to evaluate human perception, despite sycophancy being human-centric. It also reveals difficulties in distinguishing sycophantic responses from related AI alignment concepts.

Conclusion: The conclusion emphasizes the need for better methodological approaches to measure LLM sycophancy, particularly incorporating human perception evaluation, and provides actionable recommendations for future research in this area.

Abstract: Sycophantic response patterns in Large Language Models (LLMs) have been increasingly claimed in the literature. We review methodological challenges in measuring LLM sycophancy and identify five core operationalizations. Despite sycophancy being inherently human-centric, current research does not evaluate human perception. Our analysis highlights the difficulties in distinguishing sycophantic responses from related concepts in AI alignment and offers actionable recommendations for future research.

Method: Organizes proprietary knowledge and model-generated content into interactive visual knowledge graphs that link model assertions to underlying sources of truth and indicate confidence levels. Provides a visual interface for users to diagnose inconsistencies, identify weak reasoning chains, and supply corrective feedback.

Result: Creates a human-in-the-loop workflow with a structured feedback loop that enhances model reliability and continuously improves response quality by enabling users to visually identify hallucination zones.

Conclusion: The visual knowledge graph framework offers a practical solution for mitigating LLM hallucinations in enterprise settings by providing transparent, interactive visualization of model reasoning and enabling structured human feedback to improve reliability.

Abstract: Large Language Models have rapidly advanced in their ability to interpret and generate natural language. In enterprise settings, they are frequently augmented with closed-source domain knowledge to deliver more contextually informed responses. However, operational constraints such as limited context windows and inconsistencies between pre-training data and supplied knowledge often lead to hallucinations, some of which appear highly credible and escape routine human review. Current mitigation strategies either depend on costly, large-scale gold-standard Q&A curation or rely on secondary model verification, neither of which offers deterministic assurance. This paper introduces a framework that organizes proprietary knowledge and model-generated content into interactive visual knowledge graphs. The objective is to provide end users with a clear, intuitive view of potential hallucination zones by linking model assertions to underlying sources of truth and indicating confidence levels. Through this visual interface, users can diagnose inconsistencies, identify weak reasoning chains, and supply corrective feedback. The resulting human-in-the-loop workflow creates a structured feedback loop that can enhance model reliability and continuously improve response quality.

Method: Tested GPT-3.5, GPT-4, and GPT-4o on classification tasks with nuanced language. Used four different prompt styles, evaluated precision, recall, and F1 scores, and compared results to human annotators through both quantitative and qualitative analysis.

Result: GPT-4 has difficulty classifying nuanced language despite prompt engineering. Including label definitions in prompts helps but doesn’t solve the problem. Four specific qualitative findings emerged. Overall performance was inferior to human annotators.

Conclusion: ChatGPT should be used with caution for classification tasks involving nuanced language in computational social science, as current models struggle with such complexity despite rapid technological advancements.

Abstract: Generative artificial intelligence tools, like ChatGPT, are an increasingly utilized resource among computational social scientists. Nevertheless, there remains space for improved understanding of the performance of ChatGPT in complex tasks such as classifying and annotating datasets containing nuanced language. Method. In this paper, we measure the performance of GPT-4 on one such task and compare results to human annotators. We investigate ChatGPT versions 3.5, 4, and 4o to examine performance given rapid changes in technological advancement of large language models. We craft four prompt styles as input and evaluate precision, recall, and F1 scores. Both quantitative and qualitative evaluations of results demonstrate that while including label definitions in prompts may help performance, overall GPT-4 has difficulty classifying nuanced language. Qualitative analysis reveals four specific findings. Our results suggest the use of ChatGPT in classification tasks involving nuanced language should be conducted with prudence.

Method: A transformer-based POS tagging framework designed for language-agnostic application, requiring only three lines of framework-specific code to adapt from Bangla to Hindi. The modular architecture allows easy replacement of transformer models with minimal code adjustments.

Result: Achieved 96.85% token-level accuracy for Bangla and 97% for Hindi across POS categories, with strong F1 scores despite dataset imbalance and linguistic overlap. Performance discrepancy in one POS category highlighted dataset curation challenges.

Conclusion: The framework successfully enables rapid cross-lingual adaptation for low-resource languages with minimal code changes, reducing model design overhead and allowing researchers to focus on essential linguistic preprocessing and dataset refinement for underrepresented languages.

Abstract: This study proposes a language-agnostic transformer-based POS tagging framework designed for low-resource languages, using Bangla and Hindi as case studies. With only three lines of framework-specific code, the model was adapted from Bangla to Hindi, demonstrating effective portability with minimal modification. The framework achieves 96.85 percent and 97 percent token-level accuracy across POS categories in Bangla and Hindi while sustaining strong F1 scores despite dataset imbalance and linguistic overlap. A performance discrepancy in a specific POS category underscores ongoing challenges in dataset curation. The strong results stem from the underlying transformer architecture, which can be replaced with limited code adjustments. Its modular and open-source design enables rapid cross-lingual adaptation while reducing model design and tuning overhead, allowing researchers to focus on linguistic preprocessing and dataset refinement, which are essential for advancing NLP in underrepresented languages.

Method: EES is an auxiliary hyperparameter-free approach inspired by information theory that dynamically adjusts candidate sets by balancing normalized entropy with probability mass.

Result: EES consistently performs well across temperature settings on both reasoning and generation tasks, delivering competitive accuracy and coherence while maintaining diversity across various model architectures.

Conclusion: By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance in text generation tasks.

Abstract: Token sampling strategies critically influence text generation quality in large language models (LLMs). However, existing methods introduce additional hyperparameters, requiring extensive tuning and complicating deployment. We present Entropy Equilibrium Sampling (EES), an auxiliary hyperparameter-free approach inspired by information theory that can dynamically adjust candidate sets by balancing normalized entropy with probability mass. We evaluate EES on both reasoning and generation tasks across a range of model architectures. Our results show that EES consistently performs well across temperature settings, delivering competitive accuracy and coherence while maintaining diversity. By eliminating the need for hyperparameter tuning, EES greatly simplifies deployment while improving performance. Code is available at https://github.com/shuanncai/EES

Method: Used automatic mixed precision (AMP) training with dynamic mix of 16-bit and 32-bit floating-point computations. Evaluated across four Bangla NLP tasks (sentiment analysis, NER, error classification, QA) using four transformer models (BanglaBERT, BanglishBERT, XLM-R, mBERT).

Result: AMP accelerates training by 44.5% and reduces memory consumption by 17.6%, while maintaining F-1 score within 99.7% of full-precision baselines across all tasks and models.

Conclusion: AMP has significant potential to democratize access to state-of-the-art NLP capabilities in hardware-constrained settings by lowering computational barriers for Bangla NLP development.

Abstract: Training models for Natural Language Processing (NLP) requires substantial computational resources and time, posing significant challenges, especially for NLP development in Bangla, where access to high-end hardware is often limited. In this work, we explore automatic mixed precision (AMP) training as a means to improve computational efficiency without sacrificing model performance. By leveraging a dynamic mix of 16-bit and 32-bit floating-point computations, AMP lowers GPU memory requirements and speeds up training without degrading model performance. We evaluate AMP across four standard Bangla NLP tasks, namely sentiment analysis, named entity recognition, error classification, and question answering, using four transformer-based models: BanglaBERT, BanglishBERT, XLM-R, and mBERT. Our results demonstrate that AMP accelerates training by 44.5% and reduces memory consumption by 17.6%, while maintaining F-1 score within 99.7% of the full-precision baselines. This empirical study highlights AMP’s potential to democratize access to state-of-the-art NLP capabilities in hardware-constrained settings by lowering computational barriers.

Method: Proposes WaterSearch, a sentence-level search-based framework that controls seed pools for diverse parallel generation and jointly optimizes distribution fidelity and watermark signal characteristics. Includes a sentence-level detection method with attack robustness.

Result: Achieves 51.01% average performance improvement over SOTA baselines at 95% detectability strength. Gains of 47.78% for short text and 36.47% for low-entropy outputs. Maintains high detectability under insertion, synonym substitution, and paraphrase attacks.

Conclusion: WaterSearch effectively addresses the detectability-text quality trade-off in LLM watermarking, providing robust watermarking with minimal performance degradation across diverse tasks and attack scenarios.

Abstract: Watermarking acts as a critical safeguard in text generated by Large Language Models (LLMs). By embedding identifiable signals into model outputs, watermarking enables reliable attribution and enhances the security of machine-generated content. Existing approaches typically embed signals by manipulating token generation probabilities. Despite their effectiveness, these methods inherently face a trade-off between detectability and text quality: the signal strength and randomness required for robust watermarking tend to degrade the performance of downstream tasks. In this paper, we design a novel embedding scheme that controls seed pools to facilitate diverse parallel generation of watermarked text. Based on that scheme, we propose WaterSearch, a sentence-level, search-based watermarking framework adaptable to a wide range of existing methods. WaterSearch enhances text quality by jointly optimizing two key aspects: 1) distribution fidelity and 2) watermark signal characteristics. Furthermore, WaterSearch is complemented by a sentence-level detection method with strong attack robustness. We evaluate our method on three popular LLMs across ten diverse tasks. Extensive experiments demonstrate that our method achieves an average performance improvement of 51.01% over state-of-the-art baselines at a watermark detectability strength of 95%. In challenging scenarios such as short text generation and low-entropy output generation, our method yields performance gains of 47.78% and 36.47%, respectively. Moreover, under different attack senarios including insertion, synonym substitution and paraphrase attasks, WaterSearch maintains high detectability, further validating its robust anti-attack capabilities. Our code is available at \href{https://github.com/Yukang-Lin/WaterSearch}{https://github.com/Yukang-Lin/WaterSearch}.

Method: EffiLoRA uses a unified A matrix across all transformer layers and introduces runtime selective B matrices update that dynamically trades off resource budget and model performance.

Result: EffiLoRA consistently outperforms LoRA across diverse modalities including commonsense reasoning, visual instruction tuning, and image generation, demonstrating improved efficiency and robustness.

Conclusion: EffiLoRA provides a lightweight and generalizable approach for efficient fine-tuning that works across language, multimodal, and diffusion models while maintaining or improving performance.

Abstract: Low-Rank Adaptation (LoRA) is a widely adopted parameter-efficient fine-tuning (PEFT) method for Large Language Models (LLMs), but it still incurs notable overhead and suffers from parameter interference in complex datasets. While re- cent works decouple LoRA update matrices to exploit matrix-wise asymmetry, training costs remain high. We revisit LoRA from the perspective of inter-matrix and intra-layer parameter redundancy and propose Resource-Efficient Low-Rank Adaptation, EffiLoRA, a lightweight and generalizable approach for language, multimodal, and diffusion models. EffiLoRA employs a unified A matrix across all transformer layers and introduces a runtime selective B matrices up- date to dynamically trade-off the system resource budget and model performance. EffiLoRA consistently outperforms LoRA across diverse modalities, including commonsense reasoning, visual instruction tuning, and image generation, demon- strating improved efficiency and robustness.

Method: Reward Auditor uses a hypothesis-testing framework to infer RM suitability. It quantifies statistical significance and effect size by auditing distribution degradation of RM preference perception confidence under real-world perturbed scenarios, enabling systematic vulnerability assessment.

Result: The framework enables inference of both the certainty and severity of RM vulnerabilities across diverse real-world scenarios, moving beyond simple accuracy metrics to systematic suitability assessment.

Conclusion: Reward Auditor lays a foundation for building next-generation LLM alignment systems that are verifiably safe, more robust, and trustworthy by providing a scientific auditing approach to assess reward model suitability in real-world conditions.

Abstract: Reliable reward models (RMs) are critical for ensuring the safe alignment of large language models (LLMs). However, current evaluation methods focus solely on preference perception accuracies in given specific scenarios, obscuring the critical vulnerabilities of RMs in real-world scenarios. We identify the true challenge lies in assessing a novel dimension: Suitability, defined as conditional reliability under specific real-world perturbations. To this end, we introduce Reward Auditor, a hypothesis-testing framework specifically designed for RM suitability inference. Rather than answering “How accurate is the RM’s preference perception for given samples?”, it employs scientific auditing to answer: “Can we infer RMs exhibit systematic vulnerabilities in specific real-world scenarios?”. Under real-world perturbed scenarios, Reward Auditor quantifies statistical significance and effect size by auditing distribution degradation of RM preference perception confidence. This enables inference of both the certainty and severity of RM vulnerabilities across diverse real-world scenarios. This lays a solid foundation for building next-generation LLM alignment systems that are verifiably safe, more robust, and trustworthy.

Method: Tested 7 prompt engineering methods on 8 yeast biotechnology abstracts using 6 instruction-tuned LLMs: baseline prompt, two instruction complexity levels (PE-1, PE-2), two context repetition levels (CR-K1, CR-K2), and two random addition levels (RA-K1, RA-K2). Evaluated 336 summaries using 6 metrics (ROUGE-1, ROUGE-2, ROUGE-L, BERTScore, METEOR, cosine similarity) and performed statistical analysis on 3744 datapoints with BCa bootstrap confidence intervals and Wilcoxon signed-rank tests.

Result: Context repetition (CR) and random addition (RA) significantly improved lexical alignment of LLM-generated summaries with abstracts, indicating these prompt engineering methods can mitigate context inconsistency hallucinations.

Conclusion: Prompt engineering has potential to impact hallucinations in zero-shot scientific summarization tasks, with context repetition and random addition showing particular effectiveness in improving alignment between LLM outputs and source texts.

Abstract: Large language models (LLMs) produce context inconsistency hallucinations, which are LLM generated outputs that are misaligned with the user prompt. This research project investigates whether prompt engineering (PE) methods can mitigate context inconsistency hallucinations in zero-shot LLM summarisation of scientific texts, where zero-shot indicates that the LLM relies purely on its pre-training data. Across eight yeast biotechnology research paper abstracts, six instruction-tuned LLMs were prompted with seven methods: a base- line prompt, two levels of increasing instruction complexity (PE-1 and PE-2), two levels of context repetition (CR-K1 and CR-K2), and two levels of random addition (RA-K1 and RA-K2). Context repetition involved the identification and repetition of K key sentences from the abstract, whereas random addition involved the repetition of K randomly selected sentences from the abstract, where K is 1 or 2. A total of 336 LLM-generated summaries were evaluated using six metrics: ROUGE-1, ROUGE-2, ROUGE-L, BERTScore, METEOR, and cosine similarity, which were used to compute the lexical and semantic alignment be- tween the summaries and the abstracts. Four hypotheses on the effects of prompt methods on summary alignment with the reference text were tested. Statistical analysis on 3744 collected datapoints was performed using bias-corrected and accelerated (BCa) bootstrap confidence intervals and Wilcoxon signed-rank tests with Bonferroni-Holm correction. The results demonstrated that CR and RA significantly improve the lexical alignment of LLM-generated summaries with the abstracts. These findings indicate that prompt engineering has the potential to impact hallucinations in zero-shot scientific summarisation tasks.

Method: DeformAr framework with data extraction library and interactive dashboard. Two evaluation modes: 1) Cross-component analysis dividing language into dataset/model components, 2) Behavioral analysis combining interpretability techniques, token-level metrics, visualizations, and representation space analysis.

Result: First Arabic-specific, component-based interpretability tool that enables component-aware diagnostics to detect model behaviors and explain them through underlying representational patterns and data factors.

Conclusion: DeformAr provides a crucial resource for advancing model analysis in under-resourced languages by systematically investigating performance discrepancies between Arabic and English NER systems.

Abstract: Transformer models have significantly advanced Natural Language Processing (NLP), demonstrating strong performance in English. However, their effectiveness in Arabic, particularly for Named Entity Recognition (NER), remains limited, even with larger pre-trained models. This performance gap stems from multiple factors, including tokenisation, dataset quality, and annotation inconsistencies. Existing studies often analyze these issues in isolation, failing to capture their joint effect on system behaviour and performance. We introduce DeformAr (Debugging and Evaluation Framework for Transformer-based NER Systems), a novel framework designed to investigate and explain the performance discrepancy between Arabic and English NER systems. DeformAr integrates a data extraction library and an interactive dashboard, supporting two modes of evaluation: cross-component analysis and behavioural analysis. The framework divides each language into dataset and model components to examine their interactions. The analysis proceeds in two stages. First, cross-component analysis provides systematic diagnostic measures across data and model subcomponents, addressing the “what,” “how,” and “why” behind observed discrepancies. The second stage applies behavioural analysis by combining interpretability techniques with token-level metrics, interactive visualisations, and representation space analysis. These stages enable a component-aware diagnostic process that detects model behaviours and explains them by linking them to underlying representational patterns and data factors. DeformAr is the first Arabic-specific, component-based interpretability tool, offering a crucial resource for advancing model analysis in under-resourced languages.

Method: Compared FinBERT benchmark with three lightweight open-source LLMs (DeepSeek-LLM 7B, Llama3 8B Instruct, Qwen3 8B) on five public financial datasets spanning different sizes, sources, formats, and languages. Evaluated performance with limited training data (as low as 5%) in zero-shot and few-shot learning scenarios.

Result: Lightweight LLMs, particularly Qwen3 8B and Llama3 8B, performed best in most scenarios, achieving competitive results even with only 5% of training data. Performance held across zero-shot and few-shot learning settings.

Conclusion: Lightweight open-source LLMs offer cost-effective alternatives for financial sentiment analysis, achieving competitive performance with minimal training data across heterogeneous textual sources, making them accessible to researchers with limited resources.

Abstract: Large language models (LLMs) play an increasingly important role in finan- cial markets analysis by capturing signals from complex and heterogeneous textual data sources, such as tweets, news articles, reports, and microblogs. However, their performance is dependent on large computational resources and proprietary datasets, which are costly, restricted, and therefore inacces- sible to many researchers and practitioners. To reflect realistic situations we investigate the ability of lightweight open-source LLMs - smaller and publicly available models designed to operate with limited computational resources - to generalize sentiment understanding from financial datasets of varying sizes, sources, formats, and languages. We compare the benchmark finance natural language processing (NLP) model, FinBERT, and three open-source lightweight LLMs, DeepSeek-LLM 7B, Llama3 8B Instruct, and Qwen3 8B on five publicly available datasets: FinancialPhraseBank, Financial Question Answering, Gold News Sentiment, Twitter Sentiment and Chinese Finance Sentiment. We find that LLMs, specially Qwen3 8B and Llama3 8B, perform best in most scenarios, even from using only 5% of the available training data. These results hold in zero-shot and few-shot learning scenarios. Our findings indicate that lightweight, open-source large language models (LLMs) consti- tute a cost-effective option, as they can achieve competitive performance on heterogeneous textual data even when trained on only a limited subset of the extensive annotated corpora that are typically deemed necessary.

Method: Proposes TAMO (Tables As an independent MOdality) - a multimodal framework with a hypergraph neural network as global table encoder integrated with mainstream LLMs. Treats tables as independent modality alongside text tokens.

Result: Significant improvements on generalization across multiple benchmarking datasets (HiTab, WikiTQ, WikiSQL, FeTaQA, StructQA) with average relative gain of 42.65%.

Conclusion: Treating tables as an independent modality with structural preservation through hypergraph neural networks integrated with LLMs effectively addresses the limitations of current table serialization approaches, leading to substantial performance gains in table reasoning tasks.

Abstract: To migrate the remarkable successes of Large Language Models (LLMs), the community has made numerous efforts to generalize them to the table reasoning tasks for the widely deployed tabular data. Despite that, in this work, by showing a probing experiment on our proposed StructQA benchmark, we postulate that even the most advanced LLMs (such as GPTs) may still fall short of coping with tabular data. More specifically, the current scheme often simply relies on serializing the tabular data, together with the meta information, then inputting them through the LLMs. We argue that the loss of structural information is the root of this shortcoming. In this work, we further propose TAMO, which bears an ideology to treat the tables as an independent modality integrated with the text tokens. The resulting model in TAMO is a multimodal framework consisting of a hypergraph neural network as the global table encoder seamlessly integrated with the mainstream LLM. Empirical results on various benchmarking datasets, including HiTab, WikiTQ, WikiSQL, FeTaQA, and StructQA, have demonstrated significant improvements on generalization with an average relative gain of 42.65%.

Method: The authors introduce Dr.Mi-Bench, a modular-integrated benchmark grounded in academic literature with 200 human-annotated instances across 10 scientific domains (including both research and review papers). They also propose Dr.Mi-Eval, a novel evaluation paradigm that leverages academic paper structure to assess planning, retrieval, and reasoning through two modes: end-to-end evaluation for DR agents and isolated evaluation for foundational LLMs as potential backbones.

Result: Experimental results show a fragmented performance landscape: agents exhibit specialized strengths but share critical weaknesses, particularly in multi-source retrieval for review-style tasks and performing consistently across diverse scientific fields. Improving high-level planning capability is crucial for unlocking the reasoning potential of foundational LLMs as backbones.

Conclusion: Dr.Mi-Bench provides a diagnostic tool to guide development of more reliable academic research assistants by exposing actionable failure modes. The benchmark addresses critical gaps in evaluating deep research agents and reveals that planning capability is key to unlocking LLM reasoning potential for scientific research tasks.

Abstract: The explosive growth in academic literature necessitates automated deep research (DR) agents, yet their evaluation remains a significant challenge. First, existing benchmarks often focus narrowly on retrieval while neglecting high-level planning and reasoning. Second, existing benchmarks favor general domains over the scientific domains that are the core application for DR agents. To address these gaps, we introduce Dr.Mi-Bench, a Modular-integrated benchmark for scientific DR agents. Grounded in academic literature, our benchmark uses a human-annotated dataset of 200 instances across 10 scientific domains, including both research and review papers. Besides, we also propose a Modular-integrated Evaluation Paradigm for DR Agents (Dr.Mi-Eval), a novel modular-integrated evaluation paradigm, which leverages the rich structure of academic papers to assess the core competencies of planning, retrieval, and reasoning through two complementary modes: an end-to-end evaluation for DR agents and an isolated evaluation for foundational LLMs as potential backbones. Experimental results reveal a fragmented performance landscape: agents exhibit specialized strengths but share critical weaknesses, most notably in performing the multi-source retrieval required for review-style tasks and performing consistently across diverse scientific fields. Moreover, improving high-level planning capability is the crucial factor for unlocking the reasoning potential of foundational LLMs as backbones. By exposing these actionable failure modes, Dr.Mi-Bench provides a diagnostic tool to guide the development of more reliable academic research assistants.

Method: Implemented prototype combining Meta’s LLaMA 3 70B and OpenAI’s GPT 4o mini. Compared Graph RAG (structured knowledge-graph based) vs Advanced RAG (hybrid keyword-semantic search) for QA. Evaluated slide deck and podcast script generation with document-grounded retrieval. Used both human ratings across 11 quality dimensions and LLM judges for cross-validation.

Result: GPT 4o mini with Advanced RAG produced most accurate QA responses. Graph RAG offered limited improvements and caused more hallucinations due to structural complexity. For slide/podcast generation, GPT 4o mini performed best, though LLaMA 3 showed promise in narrative coherence. Human reviewers were crucial for detecting layout/stylistic flaws.

Conclusion: Combined human-LLM evaluation is essential for assessing emerging academic outputs. Advanced RAG outperformed Graph RAG for QA, and GPT 4o mini generally performed better than LLaMA 3, though each had strengths in different areas.

Abstract: Most evaluations of large language models focus on standard tasks such as factual question answering or short summarization. This research expands that scope in two directions: first, by comparing two retrieval strategies, Graph RAG, structured knowledge-graph based, and Advanced RAG, hybrid keyword-semantic search, for QA; and second, by evaluating whether LLMs can generate high quality non-traditional academic outputs, specifically slide decks and podcast scripts. We implemented a prototype combining Meta’s LLaMA 3 70B open weight and OpenAI’s GPT 4o mini API based. QA performance was evaluated using both human ratings across eleven quality dimensions and large language model judges for scalable cross validation. GPT 4o mini with Advanced RAG produced the most accurate responses. Graph RAG offered limited improvements and led to more hallucinations, partly due to its structural complexity and manual setup. Slide and podcast generation was tested with document grounded retrieval. GPT 4o mini again performed best, though LLaMA 3 showed promise in narrative coherence. Human reviewers were crucial for detecting layout and stylistic flaws, highlighting the need for combined human LLM evaluation in assessing emerging academic outputs.

Method: 1) Introduce “semantic confusion” failure mode; 2) Build ParaGuard - 10k-prompt corpus with controlled paraphrase clusters holding intent fixed while varying surface form; 3) Propose three token-level metrics: Confusion Index, Confusion Rate, and Confusion Depth using token embeddings, next-token probabilities, and perplexity signals.

Result: Experiments show global false-rejection rate hides critical structure: reveals globally unstable boundaries in some settings, localized pockets of inconsistency in others, and cases where stricter refusal doesn’t increase inconsistency. Confusion-aware auditing separates refusal frequency from refusal sensibility.

Conclusion: Provides practical signal for developers to reduce false refusals while preserving safety by measuring semantic confusion, enabling better diagnosis and tuning of safety-aligned language models beyond global metrics.

Abstract: Safety-aligned language models often refuse prompts that are actually harmless. Current evaluations mostly report global rates such as false rejection or compliance. These scores treat each prompt alone and miss local inconsistency, where a model accepts one phrasing of an intent but rejects a close paraphrase. This gap limits diagnosis and tuning. We introduce “semantic confusion,” a failure mode that captures such local inconsistency, and a framework to measure it. We build ParaGuard, a 10k-prompt corpus of controlled paraphrase clusters that hold intent fixed while varying surface form. We then propose three model-agnostic metrics at the token level: Confusion Index, Confusion Rate, and Confusion Depth. These metrics compare each refusal to its nearest accepted neighbors and use token embeddings, next-token probabilities, and perplexity signals. Experiments across diverse model families and deployment guards show that global false-rejection rate hides critical structure. Our metrics reveal globally unstable boundaries in some settings, localized pockets of inconsistency in others, and cases where stricter refusal does not increase inconsistency. We also show how confusion-aware auditing separates how often a system refuses from how sensibly it refuses. This gives developers a practical signal to reduce false refusals while preserving safety.

Method: Created ELR-1000 dataset through mobile interface crowdsourcing from rural communities, then evaluated multiple state-of-the-art LLMs on translating recipes with and without targeted cultural context (background info, examples, preservation guidelines).

Result: LLMs struggle with low-resource, culturally-specific language translation, but providing targeted context significantly improves translation quality. The dataset reveals gaps in current language technology for underrepresented languages.

Conclusion: Need for benchmarks catering to underrepresented languages and domains to advance equitable language technologies. The released ELR-1000 dataset aims to motivate development of technologies for endangered languages and cultural preservation.

Abstract: We present a culturally-grounded multimodal dataset of 1,060 traditional recipes crowdsourced from rural communities across remote regions of Eastern India, spanning 10 endangered languages. These recipes, rich in linguistic and cultural nuance, were collected using a mobile interface designed for contributors with low digital literacy. Endangered Language Recipes (ELR)-1000 – captures not only culinary practices but also the socio-cultural context embedded in indigenous food traditions. We evaluate the performance of several state-of-the-art large language models (LLMs) on translating these recipes into English and find the following: despite the models’ capabilities, they struggle with low-resource, culturally-specific language. However, we observe that providing targeted context – including background information about the languages, translation examples, and guidelines for cultural preservation – leads to significant improvements in translation quality. Our results underscore the need for benchmarks that cater to underrepresented languages and domains to advance equitable and culturally-aware language technologies. As part of this work, we release the ELR-1000 dataset to the NLP community, hoping it motivates the development of language technologies for endangered languages.

Method: Conducted a systematic survey of 47 papers reporting privacy metrics, identified and compared six distinct privacy notions, analyzed how associated metrics capture different aspects of privacy risk, and assessed alignment with legal standards (HIPAA and GDPR) and user-centered expectations from HCI studies.

Result: Identified six distinct privacy notions used in text anonymization evaluation, analyzed their strengths and limitations in capturing privacy risk, and assessed how well these notions align with legal standards and user expectations. The analysis reveals gaps in current evaluation practices.

Conclusion: The survey provides practical guidance for navigating privacy evaluation approaches and highlights the need for more robust, comparable, and legally aware privacy evaluations in text anonymization to facilitate better privacy protection in NLP research.

Abstract: In this work, we aim to clarify and reconcile metrics for evaluating privacy protection in text through a systematic survey. Although text anonymization is essential for enabling NLP research and model development in domains with sensitive data, evaluating whether anonymization methods sufficiently protect privacy remains an open challenge. In manually reviewing 47 papers that report privacy metrics, we identify and compare six distinct privacy notions, and analyze how the associated metrics capture different aspects of privacy risk. We then assess how well these notions align with legal privacy standards (HIPAA and GDPR), as well as user-centered expectations grounded in HCI studies. Our analysis offers practical guidance on navigating the landscape of privacy evaluation approaches further and highlights gaps in current practices. Ultimately, we aim to facilitate more robust, comparable, and legally aware privacy evaluations in text anonymization.

Method: Framework uses spatial reasoning tasks requiring low-level GUI action sequences, with 250 diverse prompts across 20 categories and 4 difficulty levels. Features 8 objective criteria for reproducible scoring, action-level inspection, and multi-turn feedback for external oversight.

Result: Evaluated 4 state-of-the-art LLMs across 1,000 tests: models achieved 92.8% perfect performance with structured external feedback (average +3.2% improvement, up to +32.8% for complex scenes). Specification clarity proved more important than task complexity - 100% perfect performance with explicit, verifiable criteria.

Conclusion: Transparent evaluation frameworks can establish trust in agentic systems, with external oversight proving more reliable than self-correction. The open-source framework provides a template for trustworthy agent assessment.

Abstract: As agentic AI systems increasingly operate autonomously, establishing trust through verifiable evaluation becomes critical. Yet existing benchmarks lack the transparency and auditability needed to assess whether agents behave reliably. We present DrawingBench, a verification framework for evaluating the trustworthiness of agentic LLMs through spatial reasoning tasks that require generating sequences of low-level GUI actions. Unlike opaque evaluations, DrawingBench provides transparent, rule-based assessment: 8 objective criteria enable reproducible scoring, while action-level inspection allows stakeholders to audit agent behavior. Our framework comprises 250 diverse prompts across 20 categories and 4 difficulty levels, deterministic evaluation metrics, and an external oversight mechanism through multi-turn feedback that enables human control over agent refinement. Evaluating four state-of-the-art LLMs (Claude-4 Sonnet, GPT-4.1, GPT-4.1-mini, Gemini-2.5 Flash) across 1,000 tests, we establish both capabilities and limitations: models achieved 92.8% perfect performance with structured external feedback driving significant improvements (average +3.2%, up to +32.8% for complex scenes), but systematic error patterns emerged in tool state management and long-horizon planning. Notably, specification clarity proved more important than task complexity – models achieved 100% perfect performance when given explicit, verifiable criteria. These findings demonstrate that transparent evaluation frameworks can establish trust in agentic systems, with external oversight proving more reliable than self-correction for guiding agent behavior. Our open-source framework provides a template for trustworthy agent assessment. Code and data: https://github.com/hyunjun1121/DrawingBench

Method: Proposes a comprehensive RAG Perturbation-Temperature Analysis Framework that subjects retrieved documents to three distinct perturbation types across varying temperature settings. Conducts extensive experiments on HotpotQA dataset using both open-source and proprietary LLMs.

Result: Performance degradation follows distinct patterns: high-temperature settings consistently amplify vulnerability to perturbations, while certain perturbation types exhibit non-linear sensitivity across the temperature range.

Conclusion: Provides three key contributions: (1) diagnostic benchmark for assessing RAG robustness, (2) analytical framework for quantifying perturbation-temperature interactions, and (3) practical guidelines for model selection and parameter tuning under noisy retrieval conditions.

Abstract: The evaluation of Retrieval-Augmented Generation (RAG) systems typically examines retrieval quality and generation parameters like temperature in isolation, overlooking their interaction. This work presents a systematic investigation of how text perturbations (simulating noisy retrieval) interact with temperature settings across multiple LLM runs. We propose a comprehensive RAG Perturbation-Temperature Analysis Framework that subjects retrieved documents to three distinct perturbation types across varying temperature settings. Through extensive experiments on HotpotQA with both open-source and proprietary LLMs, we demonstrate that performance degradation follows distinct patterns: high-temperature settings consistently amplify vulnerability to perturbations, while certain perturbation types exhibit non-linear sensitivity across the temperature range. Our work yields three key contributions: (1) a diagnostic benchmark for assessing RAG robustness, (2) an analytical framework for quantifying perturbation-temperature interactions, and (3) practical guidelines for model selection and parameter tuning under noisy retrieval conditions.

Method: Evaluated two widely deployed clinical AI systems (OpenEvidence and UpToDate Expert AI) against three state-of-the-art generalist LLMs (GPT-5, Gemini 3 Pro, and Claude Sonnet 4.5) using a 1,000-item benchmark combining MedQA (medical knowledge) and HealthBench (clinician-alignment) tasks.

Result: Generalist models consistently outperformed clinical tools, with GPT-5 achieving the highest scores. Clinical tools showed deficits in completeness, communication quality, context awareness, and systems-based safety reasoning.

Conclusion: Clinical decision support tools marketed as specialized may lag behind frontier LLMs, highlighting the urgent need for transparent, independent evaluation before deployment in patient-facing workflows.

Abstract: Specialized clinical AI assistants are rapidly entering medical practice, often framed as safer or more reliable than general-purpose large language models (LLMs). Yet, unlike frontier models, these clinical tools are rarely subjected to independent, quantitative evaluation, creating a critical evidence gap despite their growing influence on diagnosis, triage, and guideline interpretation. We assessed two widely deployed clinical AI systems (OpenEvidence and UpToDate Expert AI) against three state-of-the-art generalist LLMs (GPT-5, Gemini 3 Pro, and Claude Sonnet 4.5) using a 1,000-item mini-benchmark combining MedQA (medical knowledge) and HealthBench (clinician-alignment) tasks. Generalist models consistently outperformed clinical tools, with GPT-5 achieving the highest scores, while OpenEvidence and UpToDate demonstrated deficits in completeness, communication quality, context awareness, and systems-based safety reasoning. These findings reveal that tools marketed for clinical decision support may often lag behind frontier LLMs, underscoring the urgent need for transparent, independent evaluation before deployment in patient-facing workflows.

Method: Used human-in-the-loop framework with DeepSeek V3.1 guided by prompt-based cognitive scaffolding to identify metaphor and metonymy in Huangdi Neijing passages. Evaluated translations using ChatGPT 5 Pro and Gemini 2.5 Pro simulating three types of real-world readers, scoring across five cognitive dimensions with structured interviews and Interpretative Phenomenological Analysis.

Result: Prompt-adjusted LLM translations performed best across all five cognitive dimensions with high cross-model and cross-role consistency. Interviews revealed differences between human and machine translation, effective strategies for metaphor/metonymy transfer, and readers’ cognitive preferences.

Conclusion: The study provides a cognitive, efficient, and replicable human-in-the-loop methodological pathway for translating ancient, concept-dense texts like TCM, demonstrating the effectiveness of prompt-guided LLM approaches for metaphor and metonymy transfer.

Abstract: Traditional Chinese Medicine (TCM) theory is built on imagistic thinking, in which medical principles and diagnostic and therapeutic logic are structured through metaphor and metonymy. However, existing English translations largely rely on literal rendering, making it difficult for target-language readers to reconstruct the underlying conceptual networks and apply them in clinical practice. This study adopted a human-in-the-loop (HITL) framework and selected four passages from the medical canon Huangdi Neijing that are fundamental in theory. Through prompt-based cognitive scaffolding, DeepSeek V3.1 was guided to identify metaphor and metonymy in the source text and convey the theory in translation. In the evaluation stage, ChatGPT 5 Pro and Gemini 2.5 Pro were instructed by prompts to simulate three types of real-world readers. Human translations, baseline model translations, and prompt-adjusted translations were scored by the simulated readers across five cognitive dimensions, followed by structured interviews and Interpretative Phenomenological Analysis (IPA). Results show that the prompt-adjusted LLM translations perform best across all five dimensions, with high cross-model and cross-role consistency. The interview themes reveal differences between human and machine translation, effective strategies for metaphor and metonymy transfer, and readers’ cognitive preferences. This study provides a cognitive, efficient, and replicable HITL methodological pathway for the translation of ancient, concept-dense texts such as TCM.

Method: Built sentiment polarity lexicon of 1,195 Nagamese words, used these as features along with additional features for supervised ML using Naive Bayes and Support Vector Machines.

Result: First attempt at sentiment analysis and emotion classification for Nagamese language, creating foundational lexicon and applying ML techniques.

Conclusion: Pioneering work establishing baseline for Nagamese sentiment analysis, demonstrating feasibility of applying NLP techniques to low-resource creole languages.

Abstract: The Nagamese language, a.k.a Naga Pidgin, is an Assamese-lexified creole language developed primarily as a means of communication in trade between the people from Nagaland and people from Assam in the north-east India. Substantial amount of work in sentiment analysis has been done for resource-rich languages like English, Hindi, etc. However, no work has been done in Nagamese language. To the best of our knowledge, this is the first attempt on sentiment analysis and emotion classification for the Nagamese Language. The aim of this work is to detect sentiments in terms of polarity (positive, negative and neutral) and basic emotions contained in textual content of Nagamese language. We build sentiment polarity lexicon of 1,195 nagamese words and use these to build features along with additional features for supervised machine learning techniques using Na"ive Bayes and Support Vector Machines. Keywords: Nagamese, NLP, sentiment analysis, machine learning

Method: Created 500 expert-curated reasoning-intensive chemistry problems with iterative curation to eliminate flaws and prevent data contamination. Each problem includes expert-authored solution paths and supports both multimodal and text-only formats. Introduces Reasoning Path Fidelity (RPF) scoring to evaluate reasoning quality beyond final answers.

Result: Best-performing model GPT-5 (High) achieved only 38.5% accuracy, below human baseline of 40.3%. Gemini 2.5 Pro (37.9%) and DeepSeek-V3.1-Think (37.3%) followed closely. Benchmark reveals model-dependent effects of visual information and distinguishes high-fidelity reasoners from heuristic ones.

Conclusion: SUPERChem provides a challenging benchmark and reliable evaluation framework to advance LLMs toward expert-level chemical intelligence, with the dataset publicly available for research.

Abstract: Current benchmarks for evaluating the chemical reasoning capabilities of Large Language Models (LLMs) are limited by oversimplified tasks, lack of process-level evaluation, and misalignment with expert-level chemistry skills. To address these issues, we introduce SUPERChem, a benchmark of 500 expert-curated reasoning-intensive chemistry problems, covering diverse subfields and provided in both multimodal and text-only formats. Original content and an iterative curation pipeline eliminate flawed items and mitigate data contamination. Each problem is paired with an expert-authored solution path, enabling Reasoning Path Fidelity (RPF) scoring to evaluate reasoning quality beyond final-answer accuracy. Evaluations against a human baseline of 40.3% accuracy show that even the best-performing model, GPT-5 (High), reaches only 38.5%, followed closely by Gemini 2.5 Pro (37.9%) and DeepSeek-V3.1-Think (37.3%). SUPERChem elicits multi-step, multimodal reasoning, reveals model-dependent effects of visual information, and distinguishes high-fidelity reasoners from heuristic ones. By providing a challenging benchmark and a reliable evaluation framework, SUPERChem aims to facilitate the advancement of LLMs toward expert-level chemical intelligence. The dataset of the benchmark is available at https://huggingface.co/datasets/ZehuaZhao/SUPERChem.

Method: Created KardiaBench dataset via model-in-the-loop pipeline with iterative rubric-guided refinement for psychological plausibility and persona consistency. Proposed Kardia-R1 framework using Rubric-as-Judge Empathetic Reinforcement Learning (Rubric-ERL), a GRPO-based method with explainable, human-aligned rubric rewards for stepwise empathetic cognition.

Result: Kardia-R1 consistently outperforms other methods across four LLM backbones in emotion accuracy, empathy, relevance, persona consistency, and safety. The dataset comprises 178,080 QA pairs across 22,080 multi-turn conversations anchored to 671 real-world profiles.

Conclusion: The work addresses critical gaps in empathetic AI by providing a user-grounded benchmark and interpretable framework for identity-aware emotional reasoning, enabling more psychologically plausible and persona-consistent conversational agents.

Abstract: As web platforms evolve towards greater personalization and emotional complexity, conversational agents must transcend superficial empathy to demonstrate identity-aware emotional reasoning. However, existing systems face two limitations: (1) reliance on situation-centric datasets lacking persistent user identity, which hampers the capture of personalized affective nuances; and (2) dependence on opaque, coarse reward signals that hinder development of verifiable empathetic reasoning. To address these gaps, we introduce KardiaBench, a large-scale user-grounded benchmark comprising 178,080 QA pairs across 22,080 multi-turn conversations anchored to 671 real-world profiles. The dataset is constructed via a model-in-the-loop pipeline with iterative rubric-guided refinement to ensure psychological plausibility and persona consistency. This progressive empathy pipeline that integrates user comprehension, contextual reasoning, and emotion perception into conversations, followed by iterative critique and rubric-based refinement to ensure psychological plausibility, emotional fidelity, and persona consistency. Building on this, we propose Kardia-R1, a framework that trains models for interpretable, stepwise empathetic cognition. Kardia-R1 leverages Rubric-as-Judge Empathetic Reinforcement Learning (Rubric-ERL), a GRPO-based method that uses explainable, human-aligned rubric rewards to tightly couple user understanding, emotional inference, and supportive response generation. Extensive experiments across four LLM backbones demonstrate that Kardia-R1 consistently outperforms othet methods in emotion accuracy, empathy, relevance, persona consistency, and safety. Our dataset and model will be released at https://github.com/JhCircle/Kardia-R1.

Method: Proposed Agreement-Constrained PMBR (AC-PMBR) decoding that leverages a knowledge-distilled model to guide the matrix completion process, improving approximation accuracy of missing quality scores.

Result: AC-PMBR improved matrix completion approximation errors by up to 3 times and achieved higher translation quality than PMBR at comparable computational cost on WMT'23 En↔De translation tasks.

Conclusion: AC-PMBR provides a better quality-cost trade-off for MBR decoding by incorporating knowledge distillation to enhance matrix completion, making high-quality translation more computationally efficient.

Abstract: Minimum Bayes risk (MBR) decoding generates high-quality translations by maximizing the expected utility of output candidates, but it evaluates all pairwise scores over the candidate set; hence, it takes quadratic time with respect to the number of candidates. To reduce the number of utility function calls, probabilistic MBR (PMBR) decoding partially evaluates quality scores using sampled pairs of candidates and completes the missing scores with a matrix completion algorithm. Nevertheless, it degrades the translation quality as the number of utility function calls is reduced. Therefore, to improve the trade-off between quality and cost, we propose agreement-constrained PMBR (AC-PMBR) decoding, which leverages a knowledge distilled model to guide the completion of the score matrix. Our AC-PMBR decoding improved approximation errors of matrix completion by up to 3 times and achieved higher translation quality compared with PMBR decoding at a comparable computational cost on the WMT'23 En$\leftrightarrow$De translation tasks.

Method: The platform integrates flexible document storage with structured data management in its backend architecture, supports secure data-scraping via API keys, and provides a user-friendly frontend for data upload and interaction.

Result: MARSAD enables examination of both live and archived social media content, producing detailed visualizations and reports across multiple dimensions including sentiment analysis, emotion analysis, propaganda detection, fact-checking, and hate speech detection.

Conclusion: MARSAD serves as a multifunctional NLP platform that efficiently processes large multimodal datasets while making advanced social media analysis accessible to both technical and non-technical users in the Arabic-speaking context.

Abstract: MARSAD is a multifunctional natural language processing (NLP) platform designed for real-time social media monitoring and analysis, with a particular focus on the Arabic-speaking world. It enables researchers and non-technical users alike to examine both live and archived social media content, producing detailed visualizations and reports across various dimensions, including sentiment analysis, emotion analysis, propaganda detection, fact-checking, and hate speech detection. The platform also provides secure data-scraping capabilities through API keys for accessing public social media data. MARSAD’s backend architecture integrates flexible document storage with structured data management, ensuring efficient processing of large and multimodal datasets. Its user-friendly frontend supports seamless data upload and interaction.

Method: Proposes Dynamic Fusion Learning Model (DyFuLM) with two key modules: 1) Hierarchical Dynamic Fusion module for adaptive multi-level feature integration, and 2) Gated Feature Aggregation module for regulating cross-layer information flow to achieve balanced representation learning.

Result: Achieves 82.64% coarse-grained and 68.48% fine-grained accuracy, with lowest regression errors (MAE=0.0674, MSE=0.0082) and highest R²=0.6903. Ablation studies show each module contributes significantly - removing all modules causes accuracy drops of 0.91% (coarse) and 0.68% (fine).

Conclusion: DyFuLM enhances sentiment representation and overall performance through effective hierarchical feature fusion, with each module playing a crucial role in feature interaction and task balance.

Abstract: Understanding sentiment in complex textual expressions remains a fundamental challenge in affective computing. To address this, we propose a Dynamic Fusion Learning Model (DyFuLM), a multimodal framework designed to capture both hierarchical semantic representations and fine-grained emotional nuances. DyFuLM introduces two key moodules: a Hierarchical Dynamic Fusion module that adaptively integrates multi-level features, and a Gated Feature Aggregation module that regulates cross-layer information ffow to achieve balanced representation learning. Comprehensive experiments on multi-task sentiment datasets demonstrate that DyFuLM achieves 82.64% coarse-grained and 68.48% fine-grained accuracy, yielding the lowest regression errors (MAE = 0.0674, MSE = 0.0082) and the highest R^2 coefficient of determination (R^2= 0.6903). Furthermore, the ablation study validates the effectiveness of each module in DyFuLM. When all modules are removed, the accuracy drops by 0.91% for coarse-grained and 0.68% for fine-grained tasks. Keeping only the gated fusion module causes decreases of 0.75% and 0.55%, while removing the dynamic loss mechanism results in drops of 0.78% and 0.26% for coarse-grained and fine-grained sentiment classification, respectively. These results demonstrate that each module contributes significantly to feature interaction and task balance. Overall, the experimental findings further validate that DyFuLM enhances sentiment representation and overall performance through effective hierarchical feature fusion.

Method: PromptBridge is a training-free framework that preserves prompt effectiveness under model switches. It requires only a small set of alignment tasks for calibration. First, Model-Adaptive Reflective Prompt Evolution (MAP-RPE) obtains task- and model-specific optimal prompts via iterative reflective refinement and quantitative evaluation. Using calibrated prompt pairs for source and target models, PromptBridge learns a cross-model prompt mapping. At test time, given a source-model prompt for an unseen task, this mapping directly produces an optimized prompt for the target model.

Result: Experiments in single-agent and multi-agent settings show that PromptBridge consistently improves downstream accuracy while reducing migration effort. The framework effectively addresses Model Drifting without requiring costly per-task or per-model re-optimization.

Conclusion: PromptBridge provides an effective solution to the Model Drifting problem by enabling cross-model prompt transfer through learned mappings between model-specific prompts, reducing the need for expensive prompt re-engineering when switching between different LLMs.

Abstract: Large language models (LLMs) underpin applications in code generation, mathematical reasoning, and agent-based workflows. In practice, systems access LLMs via commercial APIs or open-source deployments, and the model landscape (e.g., GPT, Claude, Llama) evolves rapidly. This rapid evolution forces frequent model switches driven by capability, cost, deployment constraints, and privacy. Yet prompts are highly model-sensitive: reusing a prompt engineered for one model on another often yields substantially worse performance than a prompt optimized for the target model. We term this phenomenon Model Drifting. Through extensive empirical analysis across diverse LLM configurations, we show that model drifting is both common and severe. To address this challenge, we introduce PromptBridge, a training-free framework that preserves prompt effectiveness under model switches, enabling cross-model prompt transfer without costly per-task or per-model re-optimization. PromptBridge requires only a small set of alignment tasks for calibration. It first applies Model-Adaptive Reflective Prompt Evolution (MAP-RPE) to obtain task- and model-specific optimal prompts via iterative reflective refinement and quantitative evaluation. Using the resulting calibrated prompt pairs for the source and target models, PromptBridge learns a cross-model prompt mapping. At test time, i.e., for an unseen task, given a source-model prompt, this mapping directly produces an optimized prompt for the target model. Experiments in single-agent and multi-agent settings show that PromptBridge consistently improves downstream accuracy while reducing migration effort. The code will be available soon.

Method: Multi-agent architecture with language classification, function management, and multilingual response generation. The system supports code-mixed languages like Hinglish and employs comparative analysis of multiple language models.

Result: Significant improvements in user engagement while maintaining low latency overhead (4-8%). Demonstrated effectiveness through real-world deployment in financial assistance use cases.

Conclusion: The work contributes to bridging the language gap in digital financial services for emerging markets, enabling natural interactions for India’s diverse user base and advancing financial inclusion through multilingual conversational AI.

Abstract: India’s linguistic diversity presents both opportunities and challenges for fintech platforms. While the country has 31 major languages and over 100 minor ones, only 10% of the population understands English, creating barriers to financial inclusion. We present a multilingual conversational AI system for a financial assistance use case that supports code-mixed languages like Hinglish, enabling natural interactions for India’s diverse user base. Our system employs a multi-agent architecture with language classification, function management, and multilingual response generation. Through comparative analysis of multiple language models and real-world deployment, we demonstrate significant improvements in user engagement while maintaining low latency overhead (4-8%). This work contributes to bridging the language gap in digital financial services for emerging markets.

Method: Proposes Active Learning (AL) methods augmented with structured data selection strategies called ‘AL schedulers’, connecting AL to data clustering. Creates an integrated fine-tuning pipeline combining AL, clustering, and dynamic data selection schedulers.

Result: Experiments on Slovak, Maltese, Icelandic, and Turkish show clustering during fine-tuning with AL scheduling yields: 30% annotation savings, up to 4 F1 score improvement, and better fine-tuning stability.

Conclusion: The integrated AL-clustering-scheduler pipeline effectively enhances fine-tuning for low-resource languages, providing significant efficiency gains and performance improvements simultaneously.

Abstract: Limited data for low-resource languages typically yield weaker language models (LMs). Since pre-training is compute-intensive, it is more pragmatic to target improvements during fine-tuning. In this work, we examine the use of Active Learning (AL) methods augmented by structured data selection strategies which we term ‘Active Learning schedulers’, to boost the fine-tuning process with a limited amount of training data. We connect the AL to data clustering and propose an integrated fine-tuning pipeline that systematically combines AL, clustering, and dynamic data selection schedulers to enhance model’s performance. Experiments in the Slovak, Maltese, Icelandic and Turkish languages show that the use of clustering during the fine-tuning phase together with AL scheduling can simultaneously produce annotation savings up to 30% and performance improvements up to four F1 score points, while also providing better fine-tuning stability.

Method: Two innovations: 1) Language scaling method using curriculum learning and data balancing to extend MLLMs to 70 languages, 2) Optimized speech adapter module that compresses speech sequences to only 30 tokens for efficiency.

Result: MCAT surpasses state-of-the-art end-to-end models on FLEURS dataset across 70x69 directions, enhances batch inference efficiency, achieves this with only ~100M trainable parameters and 10 hours of S2TT data per language.

Conclusion: MCAT effectively addresses language coverage and efficiency challenges in MLLMs for S2TT, providing a scalable and efficient framework for multilingual speech translation that has been released as open-source to advance the field.

Abstract: Multimodal Large Language Models (MLLMs) have achieved great success in Speech-to-Text Translation (S2TT) tasks. However, current research is constrained by two key challenges: language coverage and efficiency. Most of the popular S2TT datasets are substantially English-centric, which restricts the scaling-up of MLLMs’ many-to-many translation capabilities. Moreover, the inference speed of MLLMs degrades dramatically when the speech is converted into long sequences (e.g., 750 tokens). To address these limitations, we propose a Multilingual Cost-effective Accelerated Speech-to-Text Translator (MCAT) framework, which includes two innovations. First, a language scaling method that leverages curriculum learning and a data balancing strategy is introduced to extend the language coverage supported by MLLMs to 70 languages and achieve mutual translation among these languages. Second, an optimized speech adapter module is designed to reduce the length of the speech sequence to only 30 tokens. Extensive experiments were conducted on MLLMs of different scales (9B and 27B). The experimental results demonstrate that MCAT not only surpasses state-of-the-art end-to-end models on the FLEURS dataset across 70x69 directions but also enhances batch inference efficiency. This is achieved with only ~100M trainable parameters and by using only 10 hours of S2TT data per language. Furthermore, we have released MCAT as open-source to promote the development of MLLMs for robust S2TT capabilities. The code and models are released at https://github.com/yxduir/m2m-70.

Method: Collected data from subreddits and local news sources for Yoruba, Kinyarwanda, and Amharic; evaluated performance of language detection models (AfroLID and general LLM) on both data types.

Result: Reddit data was minimal with heavy code-switching, while local news media provided robust clean monolingual data. Language detection models performed near-perfect on news data but struggled with code-switched Reddit posts.

Conclusion: Professionally curated news content is more reliable for training AI models for African languages than conversational platform data, and future models need to handle both clean and code-switched text.

Abstract: This study examines the digital representation of African languages and the challenges this presents for current language detection tools. We evaluate their performance on Yoruba, Kinyarwanda, and Amharic. While these languages are spoken by millions, their online usage on conversational platforms is often sparse, heavily influenced by English, and not representative of the authentic, monolingual conversations prevalent among native speakers. This lack of readily available authentic data online creates a challenge of scarcity of conversational data for training language models. To investigate this, data was collected from subreddits and local news sources for each language. The analysis showed a stark contrast between the two sources. Reddit data was minimal and characterized by heavy code-switching. Conversely, local news media offered a robust source of clean, monolingual language data, which also prompted more user engagement in the local language on the news publishers social media pages. Language detection models, including the specialized AfroLID and a general LLM, performed with near-perfect accuracy on the clean news data but struggled with the code-switched Reddit posts. The study concludes that professionally curated news content is a more reliable and effective source for training context-rich AI models for African languages than data from conversational platforms. It also highlights the need for future models that can process clean and code-switched text to improve the detection accuracy for African languages.

Method: 1) Construct UnsolvableQA dataset using dual-track methodology: programmatic generation for logic puzzles and “Reverse Construction” for mathematics. 2) Develop UnsolvableRL reinforcement learning framework with three reward components for accuracy, unsolvability detection, and difficulty.

Result: The approach achieves near-perfect unsolvability detection while improving accuracy on solvable tasks. Key finding: “Capability Collapse” shows explicit exposure to unsolvable data is essential to prevent systematic overconfidence.

Conclusion: The proposed framework successfully enables LLMs to solve feasible problems, detect inherent contradictions, and prudently refuse tasks beyond their capability, addressing a critical reliability challenge in LLM deployment.

Abstract: Ensuring LLM reliability requires not only solving complex problems but also recognizing when a problem is unsolvable. Current models often struggle to distinguish objective unsolvability (inherent contradictions in the problem) from subjective capability limitations (problems beyond the model’s competence), which leads to hallucinations and overconfidence. To address this, we propose UnsolvableQA and UnsolvableRL to solve feasible problems, detect inherent contradictions, and prudently refuse tasks beyond capability. Specifically, we construct UnsolvableQA, a dataset of paired solvable and unsolvable instances derived via a dual-track methodology: programmatic generation for logic puzzles and a novel “Reverse Construction” method that injects contradictions into valid reasoning chains for mathematics. Building on this dataset, we introduce UnsolvableRL, a reinforcement learning framework with three reward components jointly accounting for accuracy, unsolvability, and difficulty. Empirical results show that our approach achieves near-perfect unsolvability detection while also improving accuracy on solvable tasks. Crucially, we identify Capability Collapse, demonstrating that explicit exposure to unsolvable data is indispensable for preventing models from becoming systematically overconfident. Our code and data are available at https://github.com/sfasfaffa/unsolvableQA.

Method: Introduces Mixed Memory-Augmented Generation (MMAG) pattern with five interacting memory layers: conversational, long-term user, episodic and event-linked, sensory and context-aware, and short-term working memory. Draws from cognitive psychology and includes coordination, prioritization, and conflict resolution strategies.

Result: Implemented in Heero conversational agent where encrypted long-term bios and conversational history already improve engagement and retention. Framework addresses storage, retrieval, privacy, and latency concerns.

Conclusion: MMAG provides foundation for building memory-rich language agents that are more coherent, proactive, and aligned with human needs, though open challenges remain.

Abstract: Large Language Models (LLMs) excel at generating coherent text within a single prompt but fall short in sustaining relevance, personalization, and continuity across extended interactions. Human communication, however, relies on multiple forms of memory, from recalling past conversations to adapting to personal traits and situational context. This paper introduces the Mixed Memory-Augmented Generation (MMAG) pattern, a framework that organizes memory for LLM-based agents into five interacting layers: conversational, long-term user, episodic and event-linked, sensory and context-aware, and short-term working memory. Drawing inspiration from cognitive psychology, we map these layers to technical components and outline strategies for coordination, prioritization, and conflict resolution. We demonstrate the approach through its implementation in the Heero conversational agent, where encrypted long-term bios and conversational history already improve engagement and retention. We further discuss implementation concerns around storage, retrieval, privacy, and latency, and highlight open challenges. MMAG provides a foundation for building memory-rich language agents that are more coherent, proactive, and aligned with human needs.

Method: Combines PMI similarities from global correspondence model with lightweight contrastive component trained on phonetic feature vectors, plus automatic threshold selection procedure without labeled data.

Result: PMI alone outperforms existing string similarity measures (NED, SCA), and combined approach performs on par with or better than supervised baselines. Ablation shows importance of character encoding, temperature settings, and augmentation strategies.

Conclusion: Scalable self-supervised approach works without manual supervision, provides command-line tool for researchers, and achieves competitive performance with supervised methods while eliminating need for labeled data.

Abstract: This paper presents a fully self-supervised approach to borrowing detection in multilingual wordlists. The method combines two sources of information: PMI similarities based on a global correspondence model and a lightweight contrastive component trained on phonetic feature vectors. It further includes an automatic procedure for selecting decision thresholds without requiring labeled data. Experiments on benchmark datasets show that PMI alone already improves over existing string similarity measures such as NED and SCA, and that the combined similarity performs on par with or better than supervised baselines. An ablation study highlights the importance of character encoding, temperature settings and augmentation strategies. The approach scales to datasets of different sizes, works without manual supervision and is provided with a command-line tool that allows researchers to conduct their own studies.

Method: Introduced MuSoBench benchmark for multi-solution problems. Compared conventional short chain-of-thought (Short-CoT) with emerging long chain-of-thought (Long-CoT) prompting. Proposed cognitive-rigidity hypothesis and conducted attention-entropy analysis to examine underlying causes.

Result: Short-CoT exhibits pronounced overconfidence, while Long-CoT mitigates it through iterative exploration and self-reflection. Attention-entropy analysis provides preliminary support for the cognitive-rigidity hypothesis that overconfidence arises from premature convergence on narrow thought paths.

Conclusion: The findings provide tools for assessing LLM reasoning completeness and highlight the need to move beyond single-answer accuracy evaluation toward comprehensive exploration in multi-solution tasks.

Abstract: Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks that require generating comprehensive and diverse answers. We attribute this limitation to \textbf{reasoning overconfidence}: a tendency to express undue certainty in an incomplete solution set. To examine the effect, we introduce \textit{MuSoBench}, a benchmark of multi-solution problems. Experiments show that the conventional short chain-of-thought (Short-CoT) prompting paradigm exhibits pronounced overconfidence, whereas the emerging long chain-of-thought (Long-CoT) approach mitigates it through iterative exploration and self-reflection. We further characterise observable behaviours and influential factors. To probe the underlying cause, we propose the \textbf{cognitive-rigidity hypothesis}, which posits that overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths. An attention-entropy analysis offers preliminary support for this view. These findings provide tools for assessing the completeness of LLM reasoning and highlight the need to move evaluation beyond single-answer accuracy toward comprehensive exploration.

Method: OmiGraph constructs omission-aware graphs using contextual environments to capture complementary perspectives, performs omission-oriented relation modeling to identify contextual dependencies and omission intents, and uses omission-aware message-passing and aggregation to extract deception patterns.

Result: The approach achieves average improvements of +5.4% F1 and +5.3% ACC on two large-scale benchmarks, demonstrating remarkable performance by considering omission perspectives.

Conclusion: Omission-based deception is a significant but overlooked aspect of misinformation, and OmiGraph successfully addresses this gap by providing the first comprehensive framework for omission-aware misinformation detection with substantial performance improvements.

Abstract: This paper investigates the detection of misinformation, which deceives readers by explicitly fabricating misleading content or implicitly omitting important information necessary for informed judgment. While the former has been extensively studied, omission-based deception remains largely overlooked, even though it can subtly guide readers toward false conclusions under the illusion of completeness. To pioneer in this direction, this paper presents OmiGraph, the first omission-aware framework for misinformation detection. Specifically, OmiGraph constructs an omission-aware graph for the target news by utilizing a contextual environment that captures complementary perspectives of the same event, thereby surfacing potentially omitted contents. Based on this graph, omission-oriented relation modeling is then proposed to identify the internal contextual dependencies, as well as the dynamic omission intents, formulating a comprehensive omission relation representation. Finally, to extract omission patterns for detection, OmiGraph introduces omission-aware message-passing and aggregation that establishes holistic deception perception by integrating the omission contents and relations. Experiments show that, by considering the omission perspective, our approach attains remarkable performance, achieving average improvements of +5.4% F1 and +5.3% ACC on two large-scale benchmarks.

Method: Investigates reinforcement learning (RL) as a complementary optimization framework for LRM safety training. Unlike supervised fine-tuning, RL directly optimizes model policies with reward feedback, enabling more adaptive and stable alignment.

Result: Extensive experiments across multiple model families and benchmarks show that RL achieves stronger and more consistent safety gains while maintaining reasoning competence. Analysis reveals RL suppresses unsafe exploratory reasoning while preserving reflective depth.

Conclusion: RL provides a more effective approach for safety alignment in large reasoning models compared to purely supervised methods, leading to safer and more reliable reasoning processes without compromising reasoning ability.

Abstract: Large reasoning models (LRMs) extend large language models by generating explicit chain-of-thought (CoT) reasoning, significantly improving mathematical and logical problem solving. However, this explicit reasoning process also introduces new safety risks, as unsafe behaviors often emerge within intermediate reasoning trajectories, even when final answers appear harmless. Existing safety alignment approaches primarily rely on supervised fine-tuning (SFT) over safety-oriented long CoT datasets. While intuitive, we find that SFT produces inconsistent safety improvements, degrades reasoning ability, and generalizes poorly across model families. These limitations suggest that purely supervised approaches are insufficient for robust safety alignment in LRMs. To address this, we investigate reinforcement learning (RL) as a complementary optimization framework for LRM safety training. Unlike SFT, RL directly optimizes model policies with reward feedback, enabling more adaptive and stable alignment. Extensive experiments across multiple model families and benchmarks show that RL achieves stronger and more consistent safety gains while maintaining reasoning competence. Further analysis of reflection dynamics and token-level entropy reveals that RL suppresses unsafe exploratory reasoning while preserving reflective depth, leading to safer and more reliable reasoning processes.

Method: Created BHRAM-IL benchmark with 36,047 curated questions across 9 categories (factual, numerical, reasoning, linguistic tasks). Evaluated 14 state-of-the-art multilingual LLMs on subset of 10,265 questions using category-specific metrics normalized to (0,1) range, analyzing cross-lingual and factual hallucinations across languages, models, scales, categories, and domains.

Result: Aggregation over all categories and models yields primary score of 0.23 and language-corrected fuzzy score of 0.385, demonstrating poor hallucination detection performance. Dataset and code are publicly available on GitHub and HuggingFace.

Conclusion: BHRAM-IL provides a valuable resource for hallucination-focused evaluation in Indian languages, highlighting significant performance gaps and supporting future research in multilingual hallucination detection and mitigation.

Abstract: Large language models (LLMs) are increasingly deployed in multilingual applications but often generate plausible yet incorrect or misleading outputs, known as hallucinations. While hallucination detection has been studied extensively in English, under-resourced Indian languages remain largely unexplored. We present BHRAM-IL, a benchmark for hallucination recognition and assessment in multiple Indian languages, covering Hindi, Gujarati, Marathi, Odia, along with English. The benchmark comprises 36,047 curated questions across nine categories spanning factual, numerical, reasoning, and linguistic tasks. We evaluate 14 state-of-the-art multilingual LLMs on a benchmark subset of 10,265 questions, analyzing cross-lingual and factual hallucinations across languages, models, scales, categories, and domains using category-specific metrics normalized to (0,1) range. Aggregation over all categories and models yields a primary score of 0.23 and a language-corrected fuzzy score of 0.385, demonstrating the usefulness of BHRAM-IL for hallucination-focused evaluation. The dataset, and the code for generation and evaluation are available on GitHub (https://github.com/sambhashana/BHRAM-IL/) and HuggingFace (https://huggingface.co/datasets/sambhashana/BHRAM-IL/) to support future research in multilingual hallucination detection and mitigation.

Method: Cross-lingual interleaving that mixes speech tokens across languages without textual supervision, plus release of EN-FR training dataset (TinyStories, ~42k hours) and synthetic benchmarks (StoryCloze, TopicCloze) generated using GPT-4.

Result: Interleaving improves monolingual semantic accuracy, enables robust cross-lingual continuation, and strengthens cross-lingual hidden-state alignment in 360M and 1B SLMs under matched training-token budgets.

Conclusion: Cross-lingual interleaving is a simple, scalable route to building multilingual SLMs that understand and converse across languages, with all resources made open-source for reproducibility.

Abstract: Spoken Language Models (SLMs) aim to learn linguistic competence directly from speech using discrete units, widening access to Natural Language Processing (NLP) technologies for languages with limited written resources. However, progress has been largely English-centric due to scarce spoken evaluation benchmarks and training data, making cross-lingual learning difficult. We present a cross-lingual interleaving method that mixes speech tokens across languages without textual supervision. We also release an EN-FR training dataset, TinyStories (~42k hours), together with EN-FR spoken StoryCloze and TopicCloze benchmarks for cross-lingual semantic evaluation, both synthetically generated using GPT-4. On 360M and 1B SLMs under matched training-token budgets, interleaving improves monolingual semantic accuracy, enables robust cross-lingual continuation, and strengthens cross-lingual hidden-state alignment. Taken together, these results indicate that cross-lingual interleaving is a simple, scalable route to building multilingual SLMs that understand and converse across languages. All resources will be made open-source to support reproducibility.

Method: Mixed-method experiment evaluating mitigation strategy responses across faithfulness, fairness, harm-removal capacity, and relevance. Within-subject study with 57 participants assessing responses under two conditions: harmful response plus its mitigation vs. solely mitigated response.

Result: Participants’ native language, AI work experience, and annotation familiarity significantly influenced evaluations. Participants showed high sensitivity to linguistic and contextual attributes, penalizing minor grammar errors while rewarding preserved semantic contexts. This contrasts with typical quantitative LLM evaluation approaches.

Conclusion: The study provides new metrics for training and evaluating mitigation strategies and offers insights for human-AI evaluation studies, highlighting the importance of considering human factors and linguistic sensitivity in AI safety evaluations.

Abstract: With the rapid uptake of generative AI, investigating human perceptions of generated responses has become crucial. A major challenge is their `aptitude’ for hallucinating and generating harmful contents. Despite major efforts for implementing guardrails, human perceptions of these mitigation strategies are largely unknown. We conducted a mixed-method experiment for evaluating the responses of a mitigation strategy across multiple-dimensions: faithfulness, fairness, harm-removal capacity, and relevance. In a within-subject study design, 57 participants assessed the responses under two conditions: harmful response plus its mitigation and solely mitigated response. Results revealed that participants’ native language, AI work experience, and annotation familiarity significantly influenced evaluations. Participants showed high sensitivity to linguistic and contextual attributes, penalizing minor grammar errors while rewarding preserved semantic contexts. This contrasts with how language is often treated in the quantitative evaluation of LLMs. We also introduced new metrics for training and evaluating mitigation strategies and insights for human-AI evaluation studies.

Method: 1) Define OPOR-GEN task for automated report generation; 2) Construct OPOR-BENCH dataset covering 463 crisis events with news, social media, and reference summaries; 3) Propose OPOR-EVAL agent-based evaluation framework simulating human expert assessment.

Result: Experiments show OPOR-EVAL achieves high correlation with human judgments. The comprehensive framework provides solid foundation for future research in automated public opinion report generation.

Conclusion: The paper successfully bridges the research gap by providing formal task definition, benchmark dataset, and evaluation framework for automated online public opinion report generation, enabling systematic future research in this critical domain.

Abstract: Online Public Opinion Reports consolidate news and social media for timely crisis management by governments and enterprises. While large language models have made automated report generation technically feasible, systematic research in this specific area remains notably absent, particularly lacking formal task definitions and corresponding benchmarks. To bridge this gap, we define the Automated Online Public Opinion Report Generation (OPOR-GEN) task and construct OPOR-BENCH, an event-centric dataset covering 463 crisis events with their corresponding news articles, social media posts, and a reference summary. To evaluate report quality, we propose OPOR-EVAL, a novel agent-based framework that simulates human expert evaluation by analyzing generated reports in context. Experiments with frontier models demonstrate that our framework achieves high correlation with human judgments. Our comprehensive task definition, benchmark dataset, and evaluation framework provide a solid foundation for future research in this critical domain.

Method: Introduces latent debate framework that captures hidden supporting/attacking signals within a single model during single inference (unlike explicit multi-agent debates). Presents model- and task-agnostic conceptual framework, then instantiates it symbolically for True/False prediction tasks.

Result: Latent debate serves as a faithful structured surrogate model with highly consistent predictions with original LLM. Provides strong baseline for hallucination detection. Analysis reveals strong correlations between hallucinations and debate patterns (e.g., high degree of latent debates in middle layers linked to higher hallucination risk).

Conclusion: Latent debate positions as a potential framework for understanding internal mechanisms of LLMs, especially for scenarios where internal (dis)agreements appear during inference steps.

Abstract: Understanding the internal thinking process of Large Language Models (LLMs) and the cause of hallucinations remains a key challenge. To this end, we introduce latent debate, a novel framework for interpreting model predictions through the lens of implicit internal arguments. Unlike the current work of self-consistency and multi-agent debate, which relies on explicit debates among multiple answers or multiple models, latent debate captures the hidden supporting and attacking signals that arise within a single model during a single inference. We first present a model- and task-agnostic conceptual framework, and then instantiate it symbolically to approximate the thinking process of LLMs on True/False prediction tasks. Empirical studies demonstrate that latent debate is a faithful structured surrogate model that has highly consistent predictions with the original LLM. Beyond interpretability, we demonstrate that latent debate provides a strong baseline for hallucination detection. Further analysis reveals strong correlations between hallucinations and debate patterns, such as a high degree of latent debates in the middle layers is linked to a higher risk of hallucinations. These findings position latent debate as a potential framework for understanding internal mechanisms of LLMs, especially for scenarios where internal (dis)agreements appear during the inference steps.

Method: Frames CoT as gradient descent, defines surrogate objective to assess optimization process, uses dual scoring mechanism (intensity and stability), aggregates into composite process-level reward, integrates into RLVR pipelines.

Result: Extensive experiments across multiple RL algorithms and diverse LLMs on math, science, and coding benchmarks show RePro consistently enhances reasoning performance and mitigates suboptimal reasoning behaviors.

Conclusion: RePro effectively optimizes LLM reasoning processes by treating CoT as optimization and using process-level rewards, leading to improved performance and reduced suboptimal reasoning patterns.

Abstract: Recent advancements in large language models (LLMs) have been driven by their emergent reasoning capabilities, particularly through long chain-of-thought (CoT) prompting, which enables thorough exploration and deliberation. Despite these advances, long-CoT LLMs often exhibit suboptimal reasoning behaviors, such as overthinking and excessively protracted reasoning chains, which can impair performance. In this paper, we analyze reasoning processes through an optimization lens, framing CoT as a gradient descent procedure where each reasoning step constitutes an update toward problem resolution. Building on this perspective, we introduce RePro (Rectifying Process-level Reward), a novel approach to refine LLM reasoning during post-training. RePro defines a surrogate objective function to assess the optimization process underlying CoT, utilizing a dual scoring mechanism to quantify its intensity and stability. These scores are aggregated into a composite process-level reward, seamlessly integrated into reinforcement learning with verifiable rewards (RLVR) pipelines to optimize LLMs. Extensive experiments across multiple reinforcement learning algorithms and diverse LLMs, evaluated on benchmarks spanning mathematics, science, and coding, demonstrate that RePro consistently enhances reasoning performance and mitigates suboptimal reasoning behaviors.

Method: Created FINDER benchmark with 100 human-curated research tasks and 419 structured checklist items to standardize report structure, analytical depth, and factual grounding. Developed DEFT failure taxonomy through grounded theory with human-LLM co-annotation and inter-annotator reliability validation, analyzing ~1,000 reports from mainstream DRAs.

Result: DEFT taxonomy identifies 14 fine-grained failure modes across reasoning, retrieval, and generation. Experimental findings show current DRAs struggle with evidence integration, verification, and reasoning-resilient planning rather than task comprehension.

Conclusion: The FINDER benchmark and DEFT taxonomy provide standardized evaluation and diagnostic tools for Deep Research Agents, revealing critical gaps in evidence integration and verification that need to be addressed for practical report generation.

Abstract: Deep Research Agents (DRAs) aim to automatically produce analyst-level reports through iterative information retrieval and synthesis. However, most existing DRAs were validated on question-answering benchmarks, while research on generating comprehensive reports remains overlooked. Worse, current benchmarks for report synthesis suffer from task complexity and subjective metrics – this fails to reflect user demands and limits the practical utility of generated reports. To address these gaps, we present Fine-grained DEepResearch bench (FINDER), an enhanced benchmark consisting of 100 human-curated research tasks with 419 structured checklist items that standardize report structure, analytical depth, and factual grounding. Based on approximately 1,000 reports produced by mainstream DRAs, we further propose Deep rEsearch Failure Taxonomy (DEFT), the first failure taxonomy for deep research agents. DEFT contains 14 fine-grained failure modes across reasoning, retrieval, and generation, and is built upon grounded theory with human-LLM co-annotating and inter-annotator reliability validation. Our experimental findings reveal that current DRAs struggle not with task comprehension but with evidence integration, verification, and reasoning-resilient planning.

Method: Large-scale study spanning 30B+ tokens using 8 open-source LLMs across 4 reasoning datasets, analyzing TTS strategies under controlled conditions.

Result: Three key findings: 1) No single TTS strategy universally dominates; 2) Reasoning models show distinct trace-quality patterns forming short-horizon and long-horizon categories; 3) Optimal TTS performance scales monotonically with compute budget.

Conclusion: Provides practical recipe for selecting best TTS strategy based on problem difficulty, model type, and compute budget, offering effective inference-time scaling guidance.

Abstract: Test-time scaling (TTS) – the dynamic allocation of compute during inference – is a promising direction for improving reasoning in large language models (LLMs). However, a systematic comparison of well-known TTS strategies under identical conditions is missing, and the influence of model type and problem difficulty on performance remains unclear. To address these gaps, we conduct the first large-scale study of TTS, spanning over thirty billion tokens generated using eight open-source LLMs (7B to 235B parameters), across four reasoning datasets. We observe three consistent trends: (1) no single TTS strategy universally dominates; (2) reasoning models exhibit distinct trace-quality patterns across problem difficulty and trace length, forming short-horizon and long-horizon categories; and (3) for a given model type, the optimal TTS performance scales monotonically with compute budget. Based on these insights, we provide a practical recipe for selecting the best TTS strategy, considering problem difficulty, model type, and compute budget, providing a practical guide to effective inference-time scaling.

Method: Proposes 4/6 algorithm that evaluates two potential scale factors per block instead of one, scaling to smaller FP4 values to make representable value distribution more uniform for near-maximal values. Efficiently implementable on NVIDIA Blackwell GPUs.

Result: Prevents training divergence in transformer/hybrid models, brings training loss significantly closer to BF16 baseline compared to state-of-the-art NVFP4 recipes. Also improves downstream accuracy when incorporated into post-training quantization methods.

Conclusion: 4/6 enables more stable NVFP4 training and better inference performance by addressing floating-point quantization’s near-maximal value representation problem, making NVFP4 more viable for LLM training and deployment.

Abstract: As large language models have grown larger, low-precision numerical formats such as NVFP4 have become increasingly popular due to the speed and memory benefits they provide. However, to accelerate computation with NVFP4, all matrix multiplication operands–weights and activations in the forward pass, and weights, activations, and gradients in the backward pass–must be quantized to NVFP4, often leading to divergence during training and performance degradation during inference. NVFP4 by evaluating multiple potential scale factors for each block of values. To address this issue, in this work we introduce Four Over Six (4/6), a modification to the NVFP4 quantization algorithm that evaluates two potential scale factors for each block of values. Unlike integer formats, floating-point formats such as FP4 have the most quantization error on near-maximal values in each block, which we find to be primarily responsible for downstream performance degradation. We find that for some blocks, scaling to smaller FP4 values makes the distribution of representable values more uniform, improving representation of near-maximal values. Importantly, 4/6 can be implemented efficiently on NVIDIA Blackwell GPUs, making it viable to use while training LLMs with NVFP4. In pre-training experiments with transformer and hybrid model architectures, we find that 4/6 prevents divergence in several cases, bringing training loss significantly closer to BF16 compared to models trained with current state-of-the-art NVFP4 training recipes. We also find that 4/6 can be easily incorporated into many different post-training quantization methods and generally improves downstream accuracy. We hope this inspires future work in training and deploying models with NVFP4.

Method: Developed DeID-GPT, a novel framework leveraging GPT-4’s zero-shot in-context learning capabilities for medical text de-identification. Uses LLMs’ powerful named entity recognition to automatically identify and remove identifying information from unstructured medical text.

Result: DeID-GPT showed the highest accuracy and remarkable reliability compared to existing medical text de-identification methods. It effectively masks private information while preserving original text structure and meaning.

Conclusion: This early study demonstrates GPT-4’s potential for medical text de-identification, providing insights for further LLM research in healthcare. The framework offers an effective solution for HIPAA compliance in medical data sharing.

Abstract: The digitization of healthcare has facilitated the sharing and re-using of medical data but has also raised concerns about confidentiality and privacy. HIPAA (Health Insurance Portability and Accountability Act) mandates removing re-identifying information before the dissemination of medical records. Thus, effective and efficient solutions for de-identifying medical data, especially those in free-text forms, are highly needed. While various computer-assisted de-identification methods, including both rule-based and learning-based, have been developed and used in prior practice, such solutions still lack generalizability or need to be fine-tuned according to different scenarios, significantly imposing restrictions in wider use. The advancement of large language models (LLM), such as ChatGPT and GPT-4, have shown great potential in processing text data in the medical domain with zero-shot in-context learning, especially in the task of privacy protection, as these models can identify confidential information by their powerful named entity recognition (NER) capability. In this work, we developed a novel GPT4-enabled de-identification framework (``DeID-GPT") to automatically identify and remove the identifying information. Compared to existing commonly used medical text data de-identification methods, our developed DeID-GPT showed the highest accuracy and remarkable reliability in masking private information from the unstructured medical text while preserving the original structure and meaning of the text. This study is one of the earliest to utilize ChatGPT and GPT-4 for medical text data processing and de-identification, which provides insights for further research and solution development on the use of LLMs such as ChatGPT/GPT-4 in healthcare. Codes and benchmarking data information are available at https://github.com/yhydhx/ChatGPT-API.

Method: Template-aware Dialogue Sentence Embedding (TaDSE) uses template information to learn utterance embeddings via self-supervised contrastive learning. The method includes synthetic data augmentation that diversifies utterance-template associations, with slot-filling as a preliminary step to enhance the learning process.

Result: TaDSE achieves significant improvements over previous state-of-the-art methods on five downstream benchmark dialogue datasets. The paper also introduces a novel semantic compression test as an analytic instrument and discovers correlations with uniformity and alignment metrics.

Conclusion: TaDSE effectively leverages template information to learn superior dialogue sentence embeddings, demonstrating the value of utilizing token-level annotations in self-supervised learning frameworks for dialogue understanding tasks.

Abstract: Learning high quality sentence embeddings from dialogues has drawn increasing attentions as it is essential to solve a variety of dialogue-oriented tasks with low annotation cost. Annotating and gathering utterance relationships in conversations are difficult, while token-level annotations, \eg, entities, slots and templates, are much easier to obtain. Other sentence embedding methods are usually sentence-level self-supervised frameworks and cannot utilize token-level extra knowledge. We introduce Template-aware Dialogue Sentence Embedding (TaDSE), a novel augmentation method that utilizes template information to learn utterance embeddings via self-supervised contrastive learning framework. We further enhance the effect with a synthetically augmented dataset that diversifies utterance-template association, in which slot-filling is a preliminary step. We evaluate TaDSE performance on five downstream benchmark dialogue datasets. The experiment results show that TaDSE achieves significant improvements over previous SOTA methods for dialogue. We further introduce a novel analytic instrument of semantic compression test, for which we discover a correlation with uniformity and alignment. Our code will be released upon acceptance.

Method: Proposes Imit-MNMT, treating language extension as imitation learning. Uses an expert model to generate pseudo-parallel corpora between new language and existing languages. Implements data distribution imitation with language-specific weighting and translation behavior imitation mechanisms.

Result: Extensive experiments show significant improvement in translation performance between new and existing languages while effectively mitigating catastrophic forgetting compared to previous methods.

Conclusion: Imit-MNMT successfully extends multilingual NMT models to new languages using only English-parallel data, addressing the critical problem of catastrophic forgetting through innovative imitation learning techniques.

Abstract: Despite the growing variety of languages supported by existing multilingual neural machine translation (MNMT) models, most of the world’s languages are still being left behind. We aim to extend large-scale MNMT models to incorporate a new language, enabling translations between this new language and all previously supported languages, even in the challenging scenario where only a parallel corpus between the new language and English is available. Previous methods, such as continued training on parallel data including the new language, often suffer from catastrophic forgetting, which degrades performance on other languages. We propose a novel approach Imit-MNMT which treats this task as an imitation learning problem, a technique widely used in computer vision but less explored in natural language processing. Specifically, we leverage an expert model to generate pseudo-parallel corpora between the new language and the existing languages. We then introduce a data distribution imitation strategy using language-specific weighting, alongside a translation behavior imitation mechanism. Extensive experiments show that our approach significantly improves translation performance between the new and existing languages while mitigating catastrophic forgetting.

Method: Proposed a neurosymbolic reasoning approach that integrates Prolog (symbolic reasoning engine) into LLM inference pipeline. LLMs infer all information (explicit and implied through common sense) and encode it as logical code, while Prolog handles the iterative computations.

Result: Achieved large and robust performance gains across GSM8k and BIG-bench Navigate benchmarks. Near-perfect accuracy on NLR problems, maintaining robustness even as variable interdependence increases.

Conclusion: Neurosymbolic integration of LLMs with symbolic reasoning engines like Prolog effectively addresses LLMs’ reasoning limitations, particularly for non-linear reasoning tasks requiring iterative updates and parallel reasoning chains.

Abstract: Despite their linguistic competence, Large Language Models (LLMs) often struggle to reason reliably and flexibly. To identify these shortcomings, we introduce the Non-Linear Reasoning (NLR) dataset, a collection of 55 unique, hand-designed problems that target reasoning bottlenecks arising from the sequential prediction paradigm of LLMs and the inherently linear nature of natural language. NLR tasks require iterative updates, backtracking, and reasoning across multiple parallel chains of thought but only basic arithmetic to solve. To address these limitations, we propose a neurosymbolic reasoning approach that integrates Prolog, a symbolic reasoning engine, into the inference pipeline of LLMs. This division of labor shifts the LLM’s task from iterative computations to inferring all information, explicit or implied through common sense, and encoding it as logical code. Our method yields large and robust performance gains across the GSM8k and BIG-bench Navigate benchmarks and achieves near-perfect accuracy on NLR problems, maintaining robustness even as variable interdependence - the number of other variables on which the value of a single variable depends - increases.

Method: Used real-world data from Indian patients interacting with a medical chatbot in Indian English and 4 Indic languages. Employed a uniform Retrieval Augmented Generation framework to generate responses, evaluated using both automated techniques and human evaluators on four specific metrics.

Result: Models showed significant performance variations; instruction-tuned Indic models didn’t always perform well on Indic language queries; factual correctness was generally lower for Indic queries compared to English queries; code-mixed and culturally relevant queries posed challenges to models.

Conclusion: Real-world multilingual evaluation reveals important limitations in current LLMs, especially for Indic languages, highlighting the need for better handling of linguistic nuances, cultural context, and code-mixed content in medical applications.

Abstract: Assessing the capabilities and limitations of large language models (LLMs) has garnered significant interest, yet the evaluation of multiple models in real-world scenarios remains rare. Multilingual evaluation often relies on translated benchmarks, which typically do not capture linguistic and cultural nuances present in the source language. This study provides an extensive assessment of 24 LLMs on real world data collected from Indian patients interacting with a medical chatbot in Indian English and 4 other Indic languages. We employ a uniform Retrieval Augmented Generation framework to generate responses, which are evaluated using both automated techniques and human evaluators on four specific metrics relevant to our application. We find that models vary significantly in their performance and that instruction tuned Indic models do not always perform well on Indic language queries. Further, we empirically show that factual correctness is generally lower for responses to Indic queries compared to English queries. Finally, our qualitative work shows that code-mixed and culturally relevant queries in our dataset pose challenges to evaluated models.

Method: Combined several public Document AI and VRDU datasets, reformulated tasks into QA format, released OCR of all documents, included bounding box annotations for answer locations, and explored different prompting techniques (with/without bounding box information) on open-weight models.

Result: Created a unified document QA dataset with OCR and spatial annotations, and identified the most effective prompting approaches for document comprehension through experimentation with different techniques.

Conclusion: The dataset provides a valuable resource for LLM training and evaluation on document understanding tasks, and the study reveals optimal prompting strategies that leverage spatial information for improved document comprehension performance.

Abstract: We present a unified dataset for document Question-Answering (QA), which is obtained combining several public datasets related to Document AI and visually rich document understanding (VRDU). Our main contribution is twofold: on the one hand we reformulate existing Document AI tasks, such as Information Extraction (IE), into a Question-Answering task, making it a suitable resource for training and evaluating Large Language Models; on the other hand, we release the OCR of all the documents and include the exact position of the answer to be found in the document image as a bounding box. Using this dataset, we explore the impact of different prompting techniques (that might include bounding box information) on the performance of open-weight models, identifying the most effective approaches for document comprehension.

Method: Systematic review of LLM-based human simulations in social, economic, policy, and psychological contexts to identify common frameworks, advances, and limitations. Analysis reveals problems stem from LLM limitations and flawed simulation design. Proposes solution framework focusing on data enrichment, LLM advancement, and robust simulation design.

Result: Identified that discrepancies in LLM-based human simulations primarily come from inherent LLM limitations and flawed simulation design. Proposed a systematic solution framework and structured algorithm to enhance reliability and human alignment.

Conclusion: Current LLM-based human simulations are insufficiently reliable, requiring systematic improvements in data foundations, LLM capabilities, and simulation design. The proposed framework and algorithm aim to guide more credible and human-aligned simulations, with resources provided for further research.

Abstract: Large Language Models (LLMs) are increasingly employed for simulating human behaviors across diverse domains. However, our position is that current LLM-based human simulations remain insufficiently reliable, as evidenced by significant discrepancies between their outcomes and authentic human actions. Our investigation begins with a systematic review of LLM-based human simulations in social, economic, policy, and psychological contexts, identifying their common frameworks, recent advances, and persistent limitations. This review reveals that such discrepancies primarily stem from inherent limitations of LLMs and flaws in simulation design, both of which are examined in detail. Building on these insights, we propose a systematic solution framework that emphasizes enriching data foundations, advancing LLM capabilities, and ensuring robust simulation design to enhance reliability. Finally, we introduce a structured algorithm that operationalizes the proposed framework, aiming to guide credible and human-aligned LLM-based simulations. To facilitate further research, we provide a curated list of related literature and resources at https://github.com/Persdre/awesome-llm-human-simulation.

Method: Three approaches: (1) baseline using AMR-to-text models directly on UMR graphs, (2) pipeline conversion of UMR to AMR then using AMR-to-text models, and (3) fine-tuning foundation models and AMR-to-text models with UMR data.

Result: Best models achieved multilingual BERTscores of 0.825 for English and 0.882 for Chinese, showing effectiveness of fine-tuning approaches even with limited UMR data.

Conclusion: Fine-tuning approaches show promise for UMR-to-text generation, and leveraging AMR technologies provides a practical pathway for developing UMR applications despite data limitations.

Abstract: Uniform Meaning Representation (UMR) is a recently developed graph-based semantic representation, which expands on Abstract Meaning Representation (AMR) in a number of ways, in particular through the inclusion of document-level information and multilingual flexibility. In order to effectively adopt and leverage UMR for downstream tasks, efforts must be placed toward developing a UMR technological ecosystem. Though only a small amount of UMR annotations have been produced to date, in this work, we investigate the first approaches to producing text from multilingual UMR graphs. Exploiting the structural similarity between UMR and AMR graphs and the wide availability of AMR technologies, we introduce (1) a baseline approach which passes UMR graphs to AMR-to-text generation models, (2) a pipeline conversion of UMR to AMR, then using AMR-to-text generation models, and (3) a fine-tuning approach for both foundation models and AMR-to-text generation models with UMR data. Our best performing models achieve multilingual BERTscores of 0.825 for English and 0.882 for Chinese, a promising indication of the effectiveness of fine-tuning approaches for UMR-to-text generation even with limited UMR data.

Method: Derived med-gte-hybrid from gte-large sentence transformer using a hybrid tuning strategy combining contrastive learning and denoising autoencoder. Evaluated on clinical prediction tasks using MIMIC-IV dataset including CKD prognosis, eGFR prediction, and mortality prediction.

Result: Med-gte-hybrid outperforms current state-of-the-art models on the Massive Text Embedding Benchmark (MTEB) and improves patient stratification, clustering, and text retrieval. Shows strong performance on clinical prediction tasks using MIMIC-IV data.

Conclusion: The hybrid tuning approach for sentence transformers can be transferred to other medical domains and has potential to improve clinical decision-making and personalized treatment pathways in various healthcare applications.

Abstract: We introduce a novel contextual embedding model med-gte-hybrid that was derived from the gte-large sentence transformer to extract information from unstructured clinical narratives. Our model tuning strategy for med-gte-hybrid combines contrastive learning and a denoising autoencoder. To evaluate the performance of med-gte-hybrid, we investigate several clinical prediction tasks in large patient cohorts extracted from the MIMIC-IV dataset, including Chronic Kidney Disease (CKD) patient prognosis, estimated glomerular filtration rate (eGFR) prediction, and patient mortality prediction. Furthermore, we demonstrate that the med-gte-hybrid model improves patient stratification, clustering, and text retrieval, thus outperforms current state-of-the-art models on the Massive Text Embedding Benchmark (MTEB). While some of our evaluations focus on CKD, our hybrid tuning of sentence transformers could be transferred to other medical domains and has the potential to improve clinical decision-making and personalised treatment pathways in various healthcare applications.

Method: Created MimeQA dataset with ~8 hours of mime video clips from YouTube, annotated with 806 question-answer pairs to probe nonverbal social reasoning. Evaluated state-of-the-art VideoLLMs on this benchmark.

Result: VideoLLMs achieved only 20-30% accuracy on MimeQA, while humans scored 86%. Analysis showed VideoLLMs fail to ground imagined objects and over-rely on text prompts while ignoring subtle nonverbal interactions.

Conclusion: The paper highlights the limitations of current VideoLLMs in nonverbal social reasoning and introduces MimeQA as a benchmark to inspire development of AI models with true social intelligence capable of interpreting nonverbal human interactions.

Abstract: As AI becomes more closely integrated with peoples’ daily activities, socially intelligent AI that can understand and interact seamlessly with humans in daily lives is increasingly important. However, current works in AI social reasoning all rely on language-only or language-dominant approaches to benchmark and training models, resulting in systems that are improving in verbal communication but struggle with nonverbal social understanding. To address this limitation, we tap into a novel data source rich in nonverbal social interactions – mime videos. Mimes refer to the art of expression through gesture and movement without spoken words, which presents unique challenges and opportunities in interpreting nonverbal social communication. We contribute a new dataset called MimeQA, obtained by sourcing ~8 hours of videos clips from YouTube and developing a comprehensive video question-answering benchmark comprising 806 carefully annotated and verified question-answer pairs, designed to probe nonverbal social reasoning capabilities. Using MimeQA, we evaluate state-of-the-art video large language models (VideoLLMs) and find that they achieve low accuracy, generally ranging from 20-30%, while humans score 86%. Our analysis reveals that VideoLLMs often fail to ground imagined objects and over-rely on the text prompt while ignoring subtle nonverbal interactions. We hope to inspire future work in AI models that embody true social intelligence capable of interpreting non-verbal human interactions.

Method: Proposes GradeHITL framework that uses LLMs’ generative properties to pose questions to human experts, incorporating their insights to dynamically refine grading rubrics through a human-in-the-loop approach.

Result: The adaptive human-in-the-loop process significantly improves grading accuracy, outperforming existing methods and bringing ASAG closer to human-level evaluation performance.

Conclusion: Human-in-the-loop approaches using LLMs’ interactive capabilities can effectively enhance automatic short answer grading, achieving near-human performance by dynamically incorporating expert insights into rubric refinement.

Abstract: The rise of artificial intelligence (AI) technologies, particularly large language models (LLMs), has brought significant advancements to the field of education. Among various applications, automatic short answer grading (ASAG), which focuses on evaluating open-ended textual responses, has seen remarkable progress with the introduction of LLMs. These models not only enhance grading performance compared to traditional ASAG approaches but also move beyond simple comparisons with predefined “golden” answers, enabling more sophisticated grading scenarios, such as rubric-based evaluation. However, existing LLM-powered methods still face challenges in achieving human-level grading performance in rubric-based assessments due to their reliance on fully automated approaches. In this work, we explore the potential of LLMs in ASAG tasks by leveraging their interactive capabilities through a human-in-the-loop (HITL) approach. Our proposed framework, GradeHITL, utilizes the generative properties of LLMs to pose questions to human experts, incorporating their insights to refine grading rubrics dynamically. This adaptive process significantly improves grading accuracy, outperforming existing methods and bringing ASAG closer to human-level evaluation.

Method: The paper investigates 6 different solutions from literature and new research for handling negative similarities in GSC. It analyzes both combinatorial Laplacians and normalized Laplacians, and experimentally compares these approaches on document embeddings.

Result: GloVe embeddings frequently cause failures of normalized Laplacian-based GSC due to negative similarities. Applying methods that cure similarity negativity improves accuracy for both combinatorial and normalized Laplacian-based GSC, and enables application of explanation methods originally developed for Term Vector Space embeddings to GloVe embeddings.

Conclusion: Negative similarities from modern document embeddings pose significant challenges for GSC, particularly for normalized Laplacian approaches. The proposed solutions effectively address these issues, improving clustering accuracy and extending the applicability of explanation methods to newer embedding techniques like GloVe.

Abstract: This paper investigates the problem of Graph Spectral Clustering with negative similarities, resulting from document embeddings different from the traditional Term Vector Space (like doc2vec, GloVe, etc.). Solutions for combinatorial Laplacians and normalized Laplacians are discussed. An experimental investigation shows the advantages and disadvantages of 6 different solutions proposed in the literature and in this research. The research demonstrates that GloVe embeddings frequently cause failures of normalized Laplacian based GSC due to negative similarities. Furthermore, application of methods curing similarity negativity leads to accuracy improvement for both combinatorial and normalized Laplacian based GSC. It also leads to applicability for GloVe embeddings of explanation methods developed originally bythe authors for Term Vector Space embeddings.

Method: CLIMB embeds and clusters large-scale datasets in semantic space, then iteratively searches for optimal mixtures using a smaller proxy model and predictor to discover effective data combinations.

Result: 1B model trained on 400B tokens with CLIMB mixture exceeds Llama-3.2-1B by 2.0%, with domain-specific optimization achieving 5% improvement over random sampling. Also released Nemotron-ClimbLab (1.2T tokens, 20 clusters) and Nemotron-ClimbMix (400B tokens).

Conclusion: Automated data mixture discovery through semantic clustering and iterative search effectively optimizes pre-training performance, with released datasets enabling further research in optimal data composition.

Abstract: Pre-training datasets are typically collected from web content and lack inherent domain divisions. For instance, widely used datasets like Common Crawl do not include explicit domain labels, while manually curating labeled datasets such as The Pile is labor-intensive. Consequently, identifying an optimal pre-training data mixture remains a challenging problem, despite its significant benefits for pre-training performance. To address these challenges, we propose CLustering-based Iterative Data Mixture Bootstrapping (Nemotron-CLIMB), an automated framework that discovers, evaluates, and refines data mixtures in a pre-training setting. Specifically, Nemotron-CLIMB embeds and clusters large-scale datasets in a semantic space and then iteratively searches for optimal mixtures using a smaller proxy model and a predictor. When continuously trained on 400B tokens with this mixture, our 1B model exceeds the state-of-the-art Llama-3.2-1B by 2.0%. Moreover, we observe that optimizing for a specific domain (e.g., Social Sciences) yields a 5% improvement over random sampling. Finally, we introduce Nemotron-ClimbLab, a filtered 1.2-trillion-token corpus with 20 clusters as a research playground, and Nemotron-ClimbMix, a compact yet powerful 400-billion-token dataset designed for efficient pre-training that delivers superior performance under an equal token budget. We analyze the final data mixture, elucidating the characteristics of an optimal data mixture. Our data is available at: https://research.nvidia.com/labs/lpr/climb/

Method: Generalized function vectors (FVs) to extract task representations from alternative task presentations, focusing on short textual instruction prompts. Extracted instruction function vectors and compared them with demonstration-based FVs. Used controls to dissociate their contributions to task performance.

Result: Successfully extracted instruction function vectors that promote zero-shot task accuracy. Found evidence that demonstration- and instruction-based function vectors leverage different model components. Different task prompting forms do not induce a common task representation through FVs but elicit different, partly overlapping mechanisms.

Conclusion: Different prompting formats (demonstrations vs. instructions) create distinct task representations in LLMs, supporting the practice of combining both approaches. This presents challenges for universally monitoring task inference across presentation forms and encourages further examination of LLM task inference mechanisms.

Abstract: Demonstrations and instructions are two primary approaches for prompting language models to perform in-context learning (ICL) tasks. Do identical tasks elicited in different ways result in similar representations of the task? An improved understanding of task representation mechanisms would offer interpretability insights and may aid in steering models. We study this through \textit{function vectors} (FVs), recently proposed as a mechanism to extract few-shot ICL task representations. We generalize FVs to alternative task presentations, focusing on short textual instruction prompts, and successfully extract instruction function vectors that promote zero-shot task accuracy. We find evidence that demonstration- and instruction-based function vectors leverage different model components, and offer several controls to dissociate their contributions to task performance. Our results suggest that different task promptings forms do not induce a common task representation through FVs but elicit different, partly overlapping mechanisms. Our findings offer principled support to the practice of combining instructions and task demonstrations, imply challenges in universally monitoring task inference across presentation forms, and encourage further examinations of LLM task inference mechanisms.

Method: Extended probabilistic extraction technique to measure memorization of 50 books in 17 open-weight LLMs, conducting thousands of experiments to assess memorization patterns.

Result: Most LLMs do not memorize most books, but Llama 3.1 70B entirely memorizes some books (e.g., Harry Potter, 1984) - can generate entire books near-verbatim from minimal prompts.

Conclusion: Memorization varies significantly by model and book, with implications for copyright cases that don’t unambiguously favor either plaintiffs or defendants.

Abstract: Plaintiffs and defendants in copyright lawsuits over generative AI often make sweeping, opposing claims about the extent to which large language models (LLMs) have memorized plaintiffs’ protected expression in their training data. Drawing on both machine learning and copyright law, we show that these polarized positions dramatically oversimplify the relationship between memorization and copyright. To do so, we extend a recent probabilistic extraction technique to measure memorization of 50 books in 17 open-weight LLMs. Through thousands of experiments, we show that the extent of memorization varies both by model and by book. With respect to our specific extraction methodology, we find that most LLMs do not memorize most books – either in whole or in part. However, we also find that Llama 3.1 70B entirely memorizes some books, like the first Harry Potter book and 1984. In fact, the first Harry Potter is so memorized that, using a seed prompt consisting of just the first few tokens of the first chapter, we can deterministically generate the entire book near-verbatim. We discuss why our results have significant implications for copyright cases, though not ones that unambiguously favor either side.

Method: Low-Rank Clone (LRC) trains low-rank projection matrices that perform soft pruning by compressing teacher weights and activation cloning by aligning student activations (including FFN signals) with teacher activations, eliminating the need for explicit alignment modules.

Result: LRC matches or surpasses state-of-the-art models trained on trillions of tokens while using only 20B tokens, achieving over 1,000x training efficiency with open-source teachers like Llama-3.2-3B-Instruct and Qwen2.5-3B/7B-Instruct.

Conclusion: LRC provides an efficient unified approach for training high-performing small language models that maximizes knowledge transfer from teacher models while dramatically reducing training costs.

Abstract: Training high-performing Small Language Models (SLMs) remains costly, even with knowledge distillation and pruning from larger teacher models. Existing work often faces three key challenges: (1) information loss from hard pruning, (2) inefficient alignment of representations, and (3) underutilization of informative activations, particularly from Feed-Forward Networks (FFNs). To address these challenges, we introduce Low-Rank Clone (LRC), an efficient pre-training method that constructs SLMs aspiring to behavioral equivalence with strong teacher models. LRC trains a set of low-rank projection matrices that jointly enable soft pruning by compressing teacher weights, and activation clone by aligning student activations, including FFN signals, with those of the teacher. This unified design maximizes knowledge transfer while removing the need for explicit alignment modules. Extensive experiments with open-source teachers (e.g., Llama-3.2-3B-Instruct, Qwen2.5-3B/7B-Instruct) show that LRC matches or surpasses state-of-the-art models trained on trillions of tokens–while using only 20B tokens, achieving over 1,000x training efficiency. Our codes and model checkpoints are available at https://github.com/CURRENTF/LowRankClone and https://huggingface.co/collections/JitaiHao/low-rank-clone-lrc-6828389e96a93f1d4219dfaf.

Method: 1) Systematic evaluation of open models on code-mixed inputs; 2) Testing on both controlled synthetic datasets and real-world social media traces; 3) Introducing saliency drift attribution (SDA) interpretability framework to analyze attention drift; 4) Proposing a lightweight translation-based restoration strategy to recover safety.

Result: Attack success rates spike from 9% in monolingual English to 69% under code-mixed inputs, with rates exceeding 90% in non-Western contexts (Arabic, Hindi). SDA reveals that under code-mixing, model attention drifts away from safety-critical tokens, blinding the model to harmful intent. The proposed restoration strategy recovers roughly 80% of lost safety.

Conclusion: Code-mixing presents a catastrophic safety weakness in LLMs that disproportionately affects non-Western language users. The saliency drift attribution framework explains the mechanism, and translation-based restoration offers a practical mitigation path toward more equitable and robust LLM safety.

Abstract: While LLMs appear robustly safety-aligned in English, we uncover a catastrophic, overlooked weakness: attributional collapse under code-mixed perturbations. Our systematic evaluation of open models shows that the linguistic camouflage of code-mixing – blending languages within a single conversation'' -- can cause safety guardrails to fail dramatically. Attack success rates (ASR) spike from a benign 9\% in monolingual English to 69\% under code-mixed inputs, with rates exceeding 90\% in non-Western contexts such as Arabic and Hindi. These effects hold not only on controlled synthetic datasets but also on real-world social media traces, revealing a serious risk for billions of users. To explain why this happens, we introduce saliency drift attribution (SDA), an interpretability framework that shows how, under code-mixing, the model's internal attention drifts away from safety-critical tokens (e.g., violence’’ or ``corruption’’), effectively blinding it to harmful intent. Finally, we propose a lightweight translation-based restoration strategy that recovers roughly 80% of the safety lost to code-mixing, offering a practical path toward more equitable and robust LLM safety.

Method: The method involves three key contributions: (1) creating a large-scale, multi-domain Japanese normalization dataset, (2) developing normalization methods based on state-of-the-art pretrained models, and (3) conducting experiments across multiple evaluation perspectives to compare encoder-only and decoder-only approaches.

Result: Experiments show that both encoder-only and decoder-only approaches achieve promising results in both accuracy and efficiency for Japanese lexical normalization.

Conclusion: The paper demonstrates that modern pretrained models (both encoder-only and decoder-only architectures) can effectively handle lexical normalization tasks for unsegmented languages like Japanese, providing a comprehensive evaluation framework and dataset for future research.

Abstract: Lexical normalization research has sought to tackle the challenge of processing informal expressions in user-generated text, yet the absence of comprehensive evaluations leaves it unclear which methods excel across multiple perspectives. Focusing on unsegmented languages, we make three key contributions: (1) creating a large-scale, multi-domain Japanese normalization dataset, (2) developing normalization methods based on state-of-the-art pretrained models, and (3) conducting experiments across multiple evaluation perspectives. Our experiments show that both encoder-only and decoder-only approaches achieve promising results in both accuracy and efficiency.

Method: Created a benchmark with 35 task types categorized by difficulty, developed detailed evaluation criteria for each task, and trained LLM-as-a-Judge evaluators (7B and 32B) to score responses with justifications.

Result: A comprehensive Russian LLM benchmark with 2,100 professionally crafted prompts, scalable evaluation tools, and transparent assessment methodology that reduces reliance on costly human judgments.

Conclusion: POLLUX provides a scalable, interpretable evaluation framework for Russian LLMs that effectively replaces traditional human evaluations while offering greater transparency and efficiency.

Abstract: We introduce POLLUX, a comprehensive open-source benchmark designed to evaluate the generative capabilities of large language models (LLMs) in Russian. Our main contribution is a novel evaluation methodology that enhances the interpretability of LLM assessment. For each task type, we define a set of detailed criteria and develop a scoring protocol where models evaluate responses and provide justifications for their ratings. This enables transparent, criteria-driven evaluation beyond traditional resource-consuming, side-by-side human comparisons. POLLUX includes a detailed, fine-grained taxonomy of 35 task types covering diverse generative domains such as code generation, creative writing, and practical assistant use cases, totaling 2,100 manually crafted and professionally authored prompts. Each task is categorized by difficulty (easy/medium/hard), with experts constructing the dataset entirely from scratch. We also release a family of LLM-as-a-Judge (7B and 32B) evaluators trained for nuanced assessment of generative outputs. This approach provides scalable, interpretable evaluation and annotation tools for model development, effectively replacing costly and less precise human judgments.

Method: The authors first introduce Feature Monosemanticity Score (FMS) to quantify feature monosemanticity in latent representations. Building on this, they propose Guided Sparse Autoencoders (G-SAE), which conditions latent representations on labeled concepts during training to improve localization and disentanglement of target concepts.

Result: Evaluations on toxicity detection, writing style identification, and privacy attribute recognition show that G-SAE enhances monosemanticity and enables more effective, fine-grained steering with less quality degradation compared to existing methods.

Conclusion: The paper provides actionable guidelines for measuring and advancing mechanistic interpretability and control of LLMs through improved feature localization and manipulation, with G-SAE demonstrating practical improvements in interpretability, behavior detection, and control.

Abstract: There is growing interest in leveraging mechanistic interpretability and controllability to better understand and influence the internal dynamics of large language models (LLMs). However, current methods face fundamental challenges in reliably localizing and manipulating feature representations. Sparse Autoencoders (SAEs) have recently emerged as a promising direction for feature extraction at scale, yet they, too, are limited by incomplete feature isolation and unreliable monosemanticity. To systematically quantify these limitations, we introduce Feature Monosemanticity Score (FMS), a novel metric to quantify feature monosemanticity in latent representation. Building on these insights, we propose Guided Sparse Autoencoders (G-SAE), a method that conditions latent representations on labeled concepts during training. We demonstrate that reliable localization and disentanglement of target concepts within the latent space improve interpretability, detection of behavior, and control. Specifically, our evaluations on toxicity detection, writing style identification, and privacy attribute recognition show that G-SAE not only enhances monosemanticity but also enables more effective and fine-grained steering with less quality degradation. Our findings provide actionable guidelines for measuring and advancing mechanistic interpretability and control of LLMs.

Method: Evaluated five LLM families on summarization and news fact-checking tasks, assessing consistency with context and hallucination tendencies using a new self-updating dataset. Measured three biases: framing (sentiment changes), hallucination (post-cutoff knowledge), and primacy (early prompt emphasis). Also conducted human experiments on purchasing decisions and tested 18 mitigation methods across three LLM families.

Result: LLMs changed context sentiment in 26.42% of cases (framing bias), hallucinated on 60.33% of post-knowledge-cutoff questions, and showed primacy bias in 10.12% of cases. Humans were 32% more likely to purchase products after reading LLM summaries vs. original reviews. Targeted interventions showed effectiveness in mitigating these issues.

Conclusion: LLMs systematically expose users to biased content that significantly impacts human decision-making, with concerning rates of framing bias, hallucination, and primacy bias. While targeted mitigation methods show promise, these findings highlight the need for careful deployment and oversight of LLMs in decision-influencing applications.

Abstract: Large language models (LLMs) are integrated into applications like shopping reviews, summarization, or medical diagnosis support, where their use affects human decisions. We investigate the extent to which LLMs expose users to biased content and demonstrate its effect on human decision-making. We assess five LLM families in summarization and news fact-checking tasks, evaluating the consistency of LLMs with their context and their tendency to hallucinate on a new self-updating dataset. Our findings show that LLMs expose users to content that changes the context’s sentiment in 26.42% of cases (framing bias), hallucinate on 60.33% of post-knowledge-cutoff questions, and highlight context from earlier parts of the prompt (primacy bias) in 10.12% of cases, averaged across all tested models. We further find that humans are 32% more likely to purchase the same product after reading a summary of the review generated by an LLM rather than the original review. To address these issues, we evaluate 18 mitigation methods across three LLM families and find the effectiveness of targeted interventions.

Method: Introduce attack frameworks targeting verbal confidence scores through perturbation-based and jailbreak-based methods. Examine various prompting strategies, model sizes, and application domains to assess vulnerabilities.

Result: Attacks significantly impair verbal confidence estimates and cause frequent answer changes. Current verbal confidence is vulnerable, and common defense techniques are largely ineffective or counterproductive.

Conclusion: There is an urgent need to design robust mechanisms for confidence expression in LLMs, as even subtle semantic-preserving modifications can lead to misleading confidence in responses.

Abstract: Robust verbal confidence generated by large language models (LLMs) is crucial for the deployment of LLMs to help ensure transparency, trust, and safety in many applications, including those involving human-AI interactions. In this paper, we present the first comprehensive study on the robustness of verbal confidence under adversarial attacks. We introduce attack frameworks targeting verbal confidence scores through both perturbation and jailbreak-based methods, and demonstrate that these attacks can significantly impair verbal confidence estimates and lead to frequent answer changes. We examine a variety of prompting strategies, model sizes, and application domains, revealing that current verbal confidence is vulnerable and that commonly used defence techniques are largely ineffective or counterproductive. Our findings underscore the need to design robust mechanisms for confidence expression in LLMs, as even subtle semantic-preserving modifications can lead to misleading confidence in responses.

Method: Proposed two approaches: Mixture-Embedding RAG (fusing retrieved documents from multiple embeddings) and Confident RAG (generating multiple answers and selecting the one with highest confidence score).

Result: Confident RAG achieved average accuracy improvements of ~10% over vanilla LLMs and ~5% over vanilla RAG. Mixture-Embedding RAG showed limited gains.

Conclusion: Confident RAG is an efficient plug-and-play solution for trustworthy mathematical AI assistants and lays groundwork for deploying Agentic RAG systems in educational settings.

Abstract: Large Language Models (LLMs) hold significant promise for mathematics education, yet they often struggle with complex mathematical reasoning. While Retrieval-Augmented Generation (RAG) mitigates these issues by grounding LLMs in external knowledge, its effectiveness remains unstable, heavily dependent on the choice of a single embedding model. Moving beyond static RAG workflows, we draw on agentic workflow patterns, a paradigm that introduces structured task decomposition and collaboration to enhance system performance. We propose and examine two novel approaches that combine the benefits of multiple embedding models. While our Mixture-Embedding RAG approach (fusing retrieved documents) shows limited gains, our Confident RAG method (generating multiple answers and selecting the one with the highest confidence score) demonstrates significant improvement. Experimental results show that Confident RAG achieved average accuracy improvements of approximately 10% over vanilla LLMs and 5% over vanilla RAG. The consistent results across different LLMs and embedding models indicate that Confident RAG is an efficient plug-and-play solution for trustworthy mathematical AI assistants. Finally, we discuss how this work lays the groundwork for deploying Agentic RAG systems in educational settings, where autonomous planning and iterative refinement can be built upon our robust retrieval foundation.

Method: Proposes Reinforcement Learning from Checklist Feedback (RLCF): 1) Extract checklists from instructions, 2) Evaluate how well responses satisfy each checklist item using both AI judges and specialized verifier programs, 3) Combine these scores to compute rewards for reinforcement learning training.

Result: RLCF applied to Qwen2.5-7B-Instruct improved performance on all five benchmarks tested: 4-point boost in hard satisfaction rate on FollowBench, 6-point increase on InFoBench, and 3-point rise in win rate on Arena-Hard. It was the only method to improve performance on every benchmark.

Conclusion: Checklist feedback is established as a key tool for improving language models’ support of diverse user queries, demonstrating that flexible, instruction-specific criteria can significantly enhance RL’s effectiveness in eliciting instruction following compared to fixed criteria approaches.

Abstract: Language models must be adapted to understand and follow user instructions. Reinforcement learning is widely used to facilitate this – typically using fixed criteria such as “helpfulness” and “harmfulness”. In our work, we instead propose using flexible, instruction-specific criteria as a means of broadening the impact that reinforcement learning can have in eliciting instruction following. We propose “Reinforcement Learning from Checklist Feedback” (RLCF). From instructions, we extract checklists and evaluate how well responses satisfy each item - using both AI judges and specialized verifier programs - then combine these scores to compute rewards for RL. We compare RLCF with other alignment methods applied to a strong instruction following model (Qwen2.5-7B-Instruct) on five widely-studied benchmarks – RLCF is the only method to improve performance on every benchmark, including a 4-point boost in hard satisfaction rate on FollowBench, a 6-point increase on InFoBench, and a 3-point rise in win rate on Arena-Hard. These results establish checklist feedback as a key tool for improving language models’ support of queries that express a multitude of needs.

Method: Proposes DebateCV framework with two Debaters arguing opposing stances over multiple rounds, and a Moderator weighing evidential strength. Introduces Debate-SFT post-training framework using synthetic data to enhance agents’ ability to adjudicate debates effectively.

Result: Methods surpass state-of-the-art non-debate approaches in both accuracy across various evidence conditions and justification quality, improving misinformation detection capabilities.

Conclusion: The debate-driven approach strengthens societal resilience against misinformation and contributes to a more trustworthy online information ecosystem by mimicking real-world fact-checking practices through multi-agent LLM debates.

Abstract: Claim verification is essential for digital literacy, yet state-of-the-art single-agent methods often struggle with complex claims that require nuanced analysis of multifaceted online evidence. Inspired by real-world human fact-checking practices, we propose \textbf{DebateCV}, the first debate-driven claim verification framework powered by multiple LLM agents. In DebateCV, two \textit{Debaters} argue opposing stances over multiple rounds to surface subtle errors in single-agent assessments. A decisive \textit{Moderator} is then required to weigh the evidential strength of conflicting arguments to deliver an accurate verdict. Yet zero-shot agents struggle to adjudicate multi-round debates for verifying complex claims, often defaulting to neutral judgements, and no datasets exist for training agents for this role. To bridge this gap, we propose \textbf{Debate-SFT}, a post-training framework that leverages synthetic data to enhance agents’ ability to effectively adjudicate debates for claim verification. Results show that our methods surpass state-of-the-art non-debate approaches in both accuracy (across various evidence conditions) and justification quality, which strengthens societal resilience against misinformation and contributes to a more trustworthy online information ecosystem.

Method: Constructed SpeechRole-Data (98 diverse roles, 112k speech conversations) with distinct vocal characteristics, and proposed SpeechRole-Eval benchmark for multidimensional evaluation of SRPAs.

Result: Experimental results reveal advantages and challenges of both cascaded and end-to-end speech role-playing agents in maintaining vocal style consistency and role coherence.

Conclusion: The work provides foundational resources (data, code, models) for speech-driven multimodal role-playing research and highlights future development directions in this field.

Abstract: Recently, role-playing agents have emerged as a promising paradigm for achieving personalized interaction and emotional resonance. Existing research primarily focuses on the textual modality, neglecting the critical dimension of speech in realistic interactive scenarios. In particular, there is a lack of systematic evaluation for Speech Role-Playing Agents (SRPAs). To address this gap, we construct SpeechRole-Data, a large-scale, high-quality dataset that comprises 98 diverse roles and 112k speech-based single-turn and multi-turn conversations. Each role demonstrates distinct vocal characteristics, including timbre and prosody, thereby enabling more sophisticated speech role-playing. Furthermore, we propose SpeechRole-Eval, a multidimensional evaluation benchmark that systematically assesses SRPAs performance in key aspects such as fundamental interaction ability, speech expressiveness, and role-playing fidelity. Experimental results reveal the advantages and challenges of both cascaded and end-to-end speech role-playing agents in maintaining vocal style consistency and role coherence. We release all data, code, and baseline models to provide a solid foundation for speech-driven multimodal role-playing research and to foster further developments in this field.

Method: 1) Proposed ShoppingBench benchmark with increasingly challenging grounded intents; 2) Scalable framework to simulate user instructions based on various intents derived from real-world products; 3) Large-scale shopping sandbox with over 2.5 million real products as interactive simulated environment; 4) Trajectory distillation strategy with supervised fine-tuning and reinforcement learning on synthetic trajectories to distill large agent capabilities into smaller ones.

Result: State-of-the-art language agents (including GPT-4.1) achieve absolute success rates under 50% on ShoppingBench tasks, highlighting significant challenges. The proposed trajectory distillation approach enables smaller trained agents to achieve competitive performance compared to GPT-4.1.

Conclusion: ShoppingBench addresses the gap in evaluating AI agents on complex real-world shopping intents, revealing current limitations of even advanced models. The proposed distillation methods show promise for creating more efficient, capable shopping agents that can handle realistic e-commerce scenarios.

Abstract: Existing benchmarks in e-commerce primarily focus on basic user intents, such as finding or purchasing products. However, real-world users often pursue more complex goals, such as applying vouchers, managing budgets, and finding multi-products seller. To bridge this gap, we propose ShoppingBench, a novel end-to-end shopping benchmark designed to encompass increasingly challenging levels of grounded intent. Specifically, we propose a scalable framework to simulate user instructions based on various intents derived from sampled real-world products. To facilitate consistent and reliable evaluations, we provide a large-scale shopping sandbox that serves as an interactive simulated environment, incorporating over 2.5 million real-world products. Experimental results demonstrate that even state-of-the-art language agents (such as GPT-4.1) achieve absolute success rates under 50% on our benchmark tasks, highlighting the significant challenges posed by our ShoppingBench. In addition, we propose a trajectory distillation strategy and leverage supervised fine-tuning, along with reinforcement learning on synthetic trajectories, to distill the capabilities of a large language agent into a smaller one. As a result, our trained agent achieves competitive performance compared to GPT-4.1.

Method: PERSIST framework evaluates 25 open-source models (1B-685B parameters) across 2M+ responses using traditional (BFI, SD3) and novel LLM-adapted personality questionnaires, systematically varying model size, personas, reasoning modes, question order/paraphrasing, and conversation history.

Result: (1) Question reordering alone introduces large personality measurement shifts; (2) Scaling provides limited stability gains (400B+ models show SD >0.3 on 5-point scales); (3) Reasoning and conversation history paradoxically increase variability; (4) Detailed personas show mixed effects with misaligned personas having higher variability; (5) LLM-adapted questionnaires show comparable instability to human-centric versions.

Conclusion: Persistent instability across scales and mitigation strategies suggests current LLMs lack architectural foundations for genuine behavioral consistency, indicating current alignment strategies may be inadequate for safety-critical applications requiring predictable behavior.

Abstract: Large language models require consistent behavioral patterns for safe deployment, yet there are indications of large variability that may lead to an instable expression of personality traits in these models. We present PERSIST (PERsonality Stability in Synthetic Text), a comprehensive evaluation framework testing 25 open-source models (1B-685B parameters) across 2 million+ responses. Using traditional (BFI, SD3) and novel LLM-adapted personality questionnaires, we systematically vary model size, personas, reasoning modes, question order or paraphrasing, and conversation history. Our findings challenge fundamental assumptions: (1) Question reordering alone can introduce large shifts in personality measurements; (2) Scaling provides limited stability gains: even 400B+ models exhibit standard deviations >0.3 on 5-point scales; (3) Interventions expected to stabilize behavior, such as reasoning and inclusion of conversation history, can paradoxically increase variability; (4) Detailed persona instructions produce mixed effects, with misaligned personas showing significantly higher variability than the helpful assistant baseline; (5) The LLM-adapted questionnaires, despite their improved ecological validity, exhibit instability comparable to human-centric versions. This persistent instability across scales and mitigation strategies suggests that current LLMs lack the architectural foundations for genuine behavioral consistency. For safety-critical applications requiring predictable behavior, these findings indicate that current alignment strategies may be inadequate.

Method: Created CCFQA benchmark with parallel speech-text factual questions across 8 languages; proposed few-shot transfer learning strategy to transfer English QA capabilities to multilingual Spoken QA tasks using just 5-shot training.

Result: Current MLLMs face substantial challenges on CCFQA; the proposed few-shot transfer learning achieves competitive performance with GPT-4o-mini-Audio using minimal training data.

Conclusion: CCFQA addresses the multilingual speech factuality evaluation gap, reveals MLLMs’ current limitations, and demonstrates effective transfer learning for multilingual Spoken QA, providing a valuable resource for developing more robust MLLMs.

Abstract: As Large Language Models (LLMs) are increasingly popularized in the multilingual world, ensuring hallucination-free factuality becomes markedly crucial. However, existing benchmarks for evaluating the reliability of Multimodal Large Language Models (MLLMs) predominantly focus on textual or visual modalities with a primary emphasis on English, which creates a gap in evaluation when processing multilingual input, especially in speech. To bridge this gap, we propose a novel Cross-lingual and Cross-modal Factuality benchmark (CCFQA). Specifically, the CCFQA benchmark contains parallel speech-text factual questions across 8 languages, designed to systematically evaluate MLLMs’ cross-lingual and cross-modal factuality capabilities. Our experimental results demonstrate that current MLLMs still face substantial challenges on the CCFQA benchmark. Furthermore, we propose a few-shot transfer learning strategy that effectively transfers the Question Answering (QA) capabilities of LLMs in English to multilingual Spoken Question Answering (SQA) tasks, achieving competitive performance with GPT-4o-mini-Audio using just 5-shot training. We release CCFQA as a foundational research resource to promote the development of MLLMs with more robust and reliable speech understanding capabilities. Our code and dataset are available at https://github.com/yxduir/ccfqa.

Method: Introduce “design logic” concept to mimic human educators’ question-creation process. Reverse-engineer 120,000+ design logics from existing questions, match them with source documents (book and web corpora) to generate questions with controllable types and difficulty levels.

Result: Created two large-scale datasets: DLR-Book (3.04M questions from books) and DLR-Web (1.66M questions from web). Questions show greater difficulty and diversity than baselines. SFT on Qwen3 and Llama3 models with this data substantially enhances multidisciplinary reasoning, even surpassing official final models.

Conclusion: The DESIGNER pipeline successfully generates high-quality, challenging reasoning questions at scale, significantly improving LLMs’ multidisciplinary reasoning capabilities beyond existing approaches.

Abstract: Large language models (LLMs) have achieved remarkable success in many natural language tasks but still struggle with complex, multi-step reasoning, particularly across diverse disciplines. Existing reasoning datasets often lack disciplinary breadth, reasoning depth, and diversity, as well as guiding principles for question synthesis. We propose DESIGNER: a DESIGN-logic-guidEd Reasoning data synthesis pipeline that leverages naturally available, extensive raw documents (e.g., book corpus and web corpus) to generate multidisciplinary challenging questions. We introduce the concept of “design logic” and instruct LLMs to mimic human educators’ question-creation process, enabling the automated synthesis of large-scale, high-difficulty questions. We use LLMs to reverse-engineer and abstract over 120,000 design logics from existing questions across various disciplines. By matching these design logics with source documents, we are able to generate reasoning questions with controllable question types and difficulty levels. Using this pipeline, we synthesized two large-scale reasoning datasets that span 75 disciplines: DLR-Book (3.04 million questions from the book corpus) and DLR-Web (1.66 million questions from the web corpus). Data analysis indicates that the questions synthesized by our method exhibit greater difficulty and diversity compared to those in the baseline datasets. We validate our synthesized data through supervised fine-tuning (SFT) on the Qwen3 and Llama3 model families. Our data substantially enhances their multidisciplinary reasoning capabilities, outperforming existing datasets. Notably, by applying SFT on the base versions of these models using only our data, we even surpass their official final models that have undergone the full post-training process.

Method: CoQ uses natural-language-style schema representations to abstract structural noise, employs clause-by-clause SQL generation for better query quality, and implements hybrid reasoning that separates SQL-based mechanical reasoning from LLM-based logical inference.

Result: Extensive experiments across four models and five benchmarks show CoQ achieves substantial accuracy improvements and significantly lowers invalid SQL rates compared to prior LLM-based, SQL-aided, and hybrid baselines.

Conclusion: CoQ demonstrates superior effectiveness in table understanding by addressing key limitations of existing approaches through its novel multi-agent framework design.

Abstract: Table understanding requires structured, multi-step reasoning. Large Language Models (LLMs) struggle with it due to the structural complexity of tabular data. Recently, multi-agent frameworks for SQL generation have shown promise in tackling the challenges of understanding tabular data, but existing approaches often suffer from limitations such as the inability to comprehend table structure for reliable SQL generation, error propagation that results in invalid queries, and over-reliance on execution correctness. To address these issues, we propose Chain-of-Query (CoQ), a novel multi-agent framework for SQL-aided table understanding. CoQ adopts natural-language-style representations of table schemas to abstract away structural noise and enhance understanding. It employs a clause-by-clause SQL generation strategy to improve query quality and introduces a hybrid reasoning division that separates SQL-based mechanical reasoning from LLM-based logical inference, thereby reducing reliance on execution outcomes. Extensive experiments across four models and five widely used benchmarks demonstrate that CoQ achieves substantial accuracy improvements and significantly lowers invalid SQL rates compared to prior generic LLM-based, SQL-aided, and hybrid baselines, confirming its superior effectiveness in table understanding. The code is available at https://github.com/SongyuanSui/ChainofQuery.

Method: MENTAT: a simple lightweight method combining batch-reflective prompt optimization with neural ensemble learning. The approach optimizes prompts based on batch performance and uses ensemble techniques to improve reasoning capabilities.

Result: MENTAT achieves up to 65% improvement over baseline methods (prompting frozen LLMs and finetuning Transformer encoders) on four realistic RiR benchmark tasks.

Conclusion: MENTAT effectively addresses reasoning-intensive regression challenges, though substantial room remains for future advances in this emerging area of AI research.

Abstract: AI researchers and practitioners increasingly apply large language models (LLMs) to what we call reasoning-intensive regression (RiR), i.e., deducing subtle numerical scores from text. Unlike standard language regression tasks, e.g., for sentiment or similarity, RiR often appears instead in ad-hoc problems such as rubric-based scoring, modeling dense rewards in complex environments, or domain-specific retrieval, where much deeper analysis of context is required while only limited task-specific training data and computation are available. We cast four realistic problems as RiR tasks to establish an initial benchmark, and use that to test our hypothesis that prompting frozen LLMs and finetuning Transformer encoders via gradient descent will both often struggle in RiR. We then propose MENTAT, a simple and lightweight method that combines batch-reflective prompt optimization with neural ensemble learning. MENTAT achieves up to 65% improvement over both baselines, though substantial room remains for future advances in RiR.

Method: Define semantic relations through set-theoretic relations between Answerable Question Sets (AQS) - the sets of questions each text can answer. Create synthetic benchmark using controlled paraphrasing and information removal via AQS manipulations.

Result: Developed a formal framework for Semantic Text Relations (STR) and created a synthetic dataset. Evaluated discriminative and generative models for STR classification to assess how well different architectures capture semantic relations beyond surface similarity.

Conclusion: The AQS-based framework provides a formal, explainable approach to semantic text relations. The released dataset and generation code support further research in understanding fine-grained informational relations between texts.

Abstract: Understanding semantic relations between two texts is crucial for many information and document management tasks, in which one must determine whether the content fully overlaps, is completely superseded by another document, or overlaps only partially, with unique information in each. Beyond establishing this relation, it is equally important to provide explainable outputs that specify which pieces of information are present, missing, or newly added between the text pair. In this study, we formally define semantic relations between two texts through the set-theoretic relation between their respective Answerable Question Sets (AQS), the sets of questions each text can answer. Under this formulation, Semantic Text Relation (STR), such as equivalence, inclusion, and mutual overlap, becomes a well-defined set relation between the corresponding texts’ AQSs. The set differences between the AQSs also serve as an explanation or diagnostic tool for identifying how the information in the texts diverges. Using this definition, we construct a synthetic benchmark that captures fine-grained informational relations through controlled paraphrasing and deliberate information removal supported by AQS manipulations. We then use this dataset to evaluate several discriminative and generative models for classifying text pairs into STR categories, assessing how well different model architectures capture semantic relations beyond surface-level similarity. We publicly release both the dataset and the data generation code to support further research.

Method: Systematic evaluation of fine-tuned LLMs comparing encoder-decoder vs. decoder-only architectures, using MALLS and Willow datasets. Techniques include vocabulary extension, predicate conditioning, multilingual training, with metrics for exact match, logical equivalence, and predicate alignment.

Result: Fine-tuned Flan-T5-XXL achieves 70% accuracy with predicate lists, outperforming GPT-4o and DeepSeek-R1-0528. Key findings: predicate availability boosts performance 15-20%, T5 models surpass larger decoder-only LLMs, models generalize to unseen logical arguments (FOLIO dataset) without specific training.

Conclusion: While structural logic translation proves robust, predicate extraction emerges as the main bottleneck. Fine-tuned encoder-decoder models show strong performance in FOL translation tasks.

Abstract: Automating the translation of natural language to first-order logic (FOL) is crucial for knowledge representation and formal methods, yet remains challenging. We present a systematic evaluation of fine-tuned LLMs for this task, comparing architectures (encoder-decoder vs. decoder-only) and training strategies. Using the MALLS and Willow datasets, we explore techniques like vocabulary extension, predicate conditioning, and multilingual training, introducing metrics for exact match, logical equivalence, and predicate alignment. Our fine-tuned Flan-T5-XXL achieves 70% accuracy with predicate lists, outperforming GPT-4o and even the DeepSeek-R1-0528 model with CoT reasoning ability as well as symbolic systems like ccg2lambda. Key findings show: (1) predicate availability boosts performance by 15-20%, (2) T5 models surpass larger decoder-only LLMs, and (3) models generalize to unseen logical arguments (FOLIO dataset) without specific training. While structural logic translation proves robust, predicate extraction emerges as the main bottleneck.

Method: Step Pruner (SP) uses step-aware RL reward function that prioritizes correctness while penalizing redundant steps, withholds rewards for incorrect responses, and includes dynamic stopping mechanism to prevent step merging hacking behavior.

Result: SP achieves state-of-the-art accuracy across four reasoning benchmarks while significantly reducing response length. On AIME24, it reduces token usage by 69.7%.

Conclusion: Step Pruner effectively addresses overthinking in LRMs by focusing on reasoning step efficiency rather than just token count, preventing hacking behaviors while maintaining high accuracy.

Abstract: Large Reasoning Models (LRMs) demonstrate strong performance on complex tasks but often suffer from excessive verbosity, known as “overthinking.” Existing solutions via reinforcement learning (RL) typically penalize generated tokens to promote conciseness. However, these methods encounter two challenges: responses with fewer tokens do not always correspond to fewer reasoning steps, and models may develop hacking behavior in later stages of training by discarding reasoning steps to minimize token usage. In this work, we introduce \textbf{Step Pruner (SP)}, an RL framework that steers LRMs toward more efficient reasoning by favoring compact reasoning steps. Our step-aware reward function prioritizes correctness while imposing penalties for redundant steps, and withholds rewards for incorrect responses to prevent the reinforcement of erroneous reasoning. Moreover, we propose a dynamic stopping mechanism: when the model’s output no longer shortens, training is halted to prevent hacking behavior caused by the merging of steps. Extensive experiments across four reasoning benchmarks demonstrate that SP achieves state-of-the-art accuracy while significantly reducing response length. For instance, on AIME24, SP reduces token usage by \textbf{69.7%}.

Method: Formulates realistic attacks as constrained optimization over input prompt space under semantic equivalence and coherence constraints, and introduces a constraint-preserving zeroth-order method to search for adversarial yet feasible prompts.

Result: SECA achieves higher attack success rates while maintaining almost no semantic equivalence or coherence errors compared to existing methods on open-ended multiple-choice question answering tasks.

Conclusion: SECA demonstrates that both open-source and commercial LLMs are sensitive to realistic and plausible prompt variations, highlighting the need for more robust models against such attacks.

Abstract: Large Language Models (LLMs) are increasingly deployed in high-risk domains. However, state-of-the-art LLMs often produce hallucinations, raising serious concerns about their reliability. Prior work has explored adversarial attacks for hallucination elicitation in LLMs, but it often produces unrealistic prompts, either by inserting gibberish tokens or by altering the original meaning. As a result, these approaches offer limited insight into how hallucinations may occur in practice. While adversarial attacks in computer vision often involve realistic modifications to input images, the problem of finding realistic adversarial prompts for eliciting LLM hallucinations has remained largely underexplored. To address this gap, we propose Semantically Equivalent and Coherent Attacks (SECA) to elicit hallucinations via realistic modifications to the prompt that preserve its meaning while maintaining semantic coherence. Our contributions are threefold: (i) we formulate finding realistic attacks for hallucination elicitation as a constrained optimization problem over the input prompt space under semantic equivalence and coherence constraints; (ii) we introduce a constraint-preserving zeroth-order method to effectively search for adversarial yet feasible prompts; and (iii) we demonstrate through experiments on open-ended multiple-choice question answering tasks that SECA achieves higher attack success rates while incurring almost no semantic equivalence or semantic coherence errors compared to existing methods. SECA highlights the sensitivity of both open-source and commercial gradient-inaccessible LLMs to realistic and plausible prompt variations. Code is available at https://github.com/Buyun-Liang/SECA.

Method: Combines Centered Kernel Alignment (CKA) for cross-lingual representation alignment with REPINA regularization to constrain parameter updates, applied to Aya-23 8B with QLoRA in zero-shot, few-shot, and fine-tuning settings.

Result: TRepLiNa (CKA+REPINA) alignment of mid-level layers improves low-resource language translation quality, especially in data-scarce settings, as a low-cost practical approach.

Conclusion: Enforcing cross-lingual similarity in specific internal layers of multilingual LLMs through TRepLiNa effectively enhances translation for low-resource languages with minimal computational resources.

Abstract: The 2025 Multimodal Models for Low-Resource Contexts and Social Impact (MMLoSo) Language Challenge addresses one of India’s most pressing linguistic gaps: the lack of resources for its diverse low-resource languages (LRLs). In this study, we investigate whether enforcing cross-lingual similarity in specific internal layers of a decoder-only multilingual large language model (LLM) can improve translation quality from LRL to high-resource language (HRL). Specifically, we combine Centered Kernel Alignment (CKA), a similarity metric that encourages representations of different languages to align, with REPINA, a regularization method that constrains parameter updates to remain close to the pretrained model, into a joint method we call TRepLiNa. In this research project, we experiment with zero-shot, few-shot, and fine-tuning settings using Aya-23 8B with QLoRA across MMLoSo shared task language pairs (Mundari, Santali, Bhili) with Hindi/English pivots. Our results show that aligning mid-level layers using TRepLiNa (CKA+REPINA) is a low-cost, practical approach to improving LRL translation, especially in data-scarce settings.

Method: Developed a theory-informed taxonomy of narrative-understanding tasks, conducted a comprehensive survey of 78 existing benchmarks, and analyzed their coverage and alignment with narrative aspects.

Result: Only 27% of narrative tasks are well captured by existing benchmarks, with significant gaps in narrative events, style, perspective, and revelation. There’s also a need for benchmarks that can assess subjective and perspectival aspects of narrative.

Conclusion: The taxonomy, survey, and methodology provide valuable tools for NLP researchers to better evaluate LLM narrative understanding, highlighting the need for new benchmarks that cover overlooked narrative aspects and can handle subjective elements.

Abstract: We present NarraBench, a theory-informed taxonomy of narrative-understanding tasks, as well as an associated survey of 78 existing benchmarks in the area. We find significant need for new evaluations covering aspects of narrative understanding that are either overlooked in current work or are poorly aligned with existing metrics. Specifically, we estimate that only 27% of narrative tasks are well captured by existing benchmarks, and we note that some areas – including narrative events, style, perspective, and revelation – are nearly absent from current evaluations. We also note the need for increased development of benchmarks capable of assessing constitutively subjective and perspectival aspects of narrative, that is, aspects for which there is generally no single correct answer. Our taxonomy, survey, and methodology are of value to NLP researchers seeking to test LLM narrative understanding.

Method: Fine-tuned arLLMs and dLLMs on three diverse datasets, evaluating with forward and backward style QA to probe knowledge generalization and reversal curse. Introduced a novel masked fine-tuning paradigm for arLLMs that improves data efficiency. Extended the approach to supervised fine-tuning for mathematical capability.

Result: dLLMs achieved high accuracies on both forward and backward QAs without paraphrases, while arLLMs required extensive data augmentation and only worked when paraphrase order matched QA style. The proposed masked fine-tuning paradigm drastically improved arLLM data efficiency, closing the performance gap with dLLMs. Masked SFT also outperformed regular SFT for mathematical capability.

Conclusion: Masked diffusion LLMs have superior knowledge acquisition capabilities compared to autoregressive LLMs, being free from reversal curse and requiring less data augmentation. The novel masked fine-tuning paradigm successfully improves arLLM performance, demonstrating that architectural advantages of dLLMs can be transferred to arLLMs through specialized training methods.

Abstract: Despite autoregressive large language models (arLLMs) being the current dominant paradigm in language modeling, effectively updating these models to incorporate new factual knowledge still remains difficult. They resist knowledge injection via fine-tuning due to inherent shortcomings such as the “reversal curse” – the challenge of answering questions that reverse the original information order in the training sample. Masked diffusion large language models (dLLMs) are rapidly emerging as a powerful alternative to the arLLM paradigm, with evidence of better data efficiency and free of the “reversal curse” in pre-training. However, it is unknown whether these advantages extend to the post-training phase, i.e. whether pre-trained dLLMs can easily acquire new knowledge through fine-tuning. On three diverse datasets, we fine-tune arLLMs and dLLMs, evaluating them with forward and backward style Question Answering (QA) to probe knowledge generalization and the reversal curse. Our results confirm that arLLMs critically rely on extensive data augmentation via paraphrases for QA generalization, and paraphrases are only effective when their information order matches the QA style. Conversely, dLLMs achieve high accuracies on both forward and backward QAs without paraphrases; adding paraphrases yields only marginal gains. Inspired by the dLLM’s performance, we introduce a novel masked fine-tuning paradigm for knowledge injection into pre-trained arLLMs. This proposed method successfully and drastically improves the data efficiency of arLLM fine-tuning, effectively closing its performance gap with dLLMs. We further show that the masked fine-tuning paradigm of arLLMs can be extended to the supervised fine-tuning (SFT) of mathematical capability. Across two models and two datasets, our masked SFT outperforms regular SFT.

Method: Dynamic Nested Depth (DND) identifies critical tokens at transformer layer ends using a router with loss control for better distinguishability, then feeds them back for extra processing via a threshold control scheme for stability.

Result: Improves dense Qwen3-1.7B by 1.88% and MoE Qwen3-30B-A3B by 0.87% on diverse benchmarks with minimal parameter and computing increase, demonstrating effectiveness across both dense and mixture-of-experts models.

Conclusion: DND provides an efficient post-training enhancement method that selectively reprocesses difficult tokens, offering significant performance gains with minimal computational overhead for both dense and MoE LLMs.

Abstract: We introduce Dynamic Nested Depth (DND), a novel method that improves performance for off-the-shelf LLMs by selecting critical tokens to reprocess in a nested depth manner. Specifically, at the end of the given transformer layer, DND identifies more critical tokens with a router and feeds them back for an extra round of processing, effectively ``reviewing" difficult tokens while avoiding redundant computation for easier ones. The dynamic selection mechanism is tailored for precise control via two novel strategies: a router controlling loss to enhance token selection distinguishability, and a threshold control scheme to ensure selection stability. We demonstrate the effectiveness of DND by directly integrating it into pre-trained dense and MoE models during a post-training phase. On diverse benchmarks, this approach boosts the performances of the dense Qwen3-1.7B by 1.88% and the MoE Qwen3-30B-A3B by 0.87%, all with a minimal parameter and computing increase.

Method: The authors use the Minimum Description Length (MDL) approach to propose that recursive numeral systems should be evaluated based on their regularity and processing complexity. They develop MDL-based measures of regularity and processing complexity to analyze numeral systems.

Result: The MDL-based measures of regularity and processing complexity better capture the differences between attested natural numeral systems and unattested but possible ones, including “optimal” recursive numeral systems from previous work. The ad-hoc constraints from previous literature naturally emerge from considerations of regularity.

Conclusion: Recursive numeral systems are efficient with regard to regularity and processing complexity, not just lexicon size vs. morphosyntactic complexity. The study highlights the importance of incorporating regularity across sets of forms when measuring and explaining optimality in language.

Abstract: Previous work has argued that recursive numeral systems optimise the trade-off between lexicon size and average morphosyntatic complexity (Denić and Szymanik, 2024). However, showing that only natural-language-like systems optimise this tradeoff has proven elusive, and the existing solution has relied on ad-hoc constraints to rule out unnatural systems (Yang and Regier, 2025). Here, we argue that this issue arises because the proposed trade-off has neglected regularity, a crucial aspect of complexity central to human grammars in general. Drawing on the Minimum Description Length (MDL) approach, we propose that recursive numeral systems are better viewed as efficient with regard to their regularity and processing complexity. We show that our MDL-based measures of regularity and processing complexity better capture the key differences between attested, natural systems and unattested but possible ones, including “optimal” recursive numeral systems from previous work, and that the ad-hoc constraints from previous literature naturally follow from regularity. Our approach highlights the need to incorporate regularity across sets of forms in studies that attempt to measure and explain optimality in language.

Method: An end-to-end reinforcement learning framework where a small-scale LLM collaborates with a large-scale LLM through multi-turn prompt interactions. The small LLM thinks and generates prompts, while the large LLM performs complex reasoning. A dual-constrained reward optimizes for correctness, generation quality, and reasoning accuracy.

Result: Experiments on multiple public datasets show that Prompt-R1 significantly outperforms baseline models across various tasks.

Conclusion: Prompt-R1 provides a plug-and-play framework that supports both inference and training with various large-scale LLMs, effectively addressing the challenge of prompt engineering for complex problems.

Abstract: Recently, advanced large language models (LLMs) have emerged at an increasingly rapid pace. However, when faced with complex problems, most users are often unable to provide accurate and effective prompts to interact with LLMs, thus limiting the performance of LLMs. To address this challenge, we propose Prompt-R1, an end-to-end reinforcement learning framework that uses a small-scale LLM to collaborate with large-scale LLMs, replacing user interaction to solve problems better. This collaboration is cast as a multi-turn prompt interaction, where the small-scale LLM thinks and generates prompts, and the large-scale LLM performs complex reasoning. A dual-constrained reward is designed to optimize for correctness, generation quality, and reasoning accuracy. Prompt-R1 provides a plug-and-play framework that supports both inference and training with various large-scale LLMs. Experiments on multiple public datasets show that Prompt-R1 significantly outperforms baseline models across tasks. Our code is publicly available at https://github.com/QwenQKing/Prompt-R1.

Method: EQ-Negotiator framework integrates game theory with a Hidden Markov Model (HMM) to learn and track debtor emotional states online without pre-training. This reasoning system equips SLMs with strategic intelligence to counter manipulation while de-escalating conflict and upholding ethics.

Result: A 7B parameter language model with EQ-Negotiator achieves better debt recovery and negotiation efficiency than baseline LLMs more than 10 times its size, across diverse credit negotiation scenarios including adversarial strategies like cheating, threatening, and playing the victim.

Conclusion: Strategic emotional intelligence, not raw model scale, is critical for automated negotiation success. EQ-Negotiator advances persona modeling from descriptive profiles to dynamic emotional architectures that enable effective, ethical, privacy-preserving AI negotiators for edge computing.

Abstract: The deployment of large language models (LLMs) in automated negotiation has set a high performance benchmark, but their computational cost and data privacy requirements render them unsuitable for many privacy-sensitive, on-device applications such as mobile assistants, embodied AI agents or private client interactions. While small language models (SLMs) offer a practical alternative, they suffer from a significant performance gap compared to LLMs in playing emotionally charged complex personas, especially for credit negotiation. This paper introduces EQ-Negotiator, a novel framework that bridges this capability gap using emotional personas. Its core is a reasoning system that integrates game theory with a Hidden Markov Model(HMM) to learn and track debtor emotional states online, without pre-training. This allows EQ-Negotiator to equip SLMs with the strategic intelligence to counter manipulation while de-escalating conflict and upholding ethical standards. Through extensive agent-to-agent simulations across diverse credit negotiation scenarios, including adversarial debtor strategies like cheating, threatening, and playing the victim, we show that a 7B parameter language model with EQ-Negotiator achieves better debt recovery and negotiation efficiency than baseline LLMs more than 10 times its size. This work advances persona modeling from descriptive character profiles to dynamic emotional architectures that operate within privacy constraints. Besides, this paper establishes that strategic emotional intelligence, not raw model scale, is the critical factor for success in automated negotiation, paving the way for effective, ethical, and privacy-preserving AI negotiators that can operate on the edge.

Method: Proposes DPRM with two implicit PRMs: KG-PRM (learns structural constraints from KGs using preference pairs) and CoT-PRM (for Chain of Thought reasoning). Both derive step-level rewards from outcome signals via reward parameterization without explicit annotations. Introduces consistency constraint between CoT and KG reasoning steps for mutual verification and collaborative optimization.

Result: Outperforms 13 baselines on multiple datasets with up to 16.6% improvement on Hit@1 metric.

Conclusion: DPRM effectively addresses limitations of existing implicit PRMs by handling graph structure constraints and ensuring consistency between different reasoning paths in multi-hop QA, achieving significant performance improvements.

Abstract: In multi-hop question answering (MHQA) tasks, Chain of Thought (CoT) improves the quality of generation by guiding large language models (LLMs) through multi-step reasoning, and Knowledge Graphs (KGs) reduce hallucinations via semantic matching. Outcome Reward Models (ORMs) provide feedback after generating the final answers but fail to evaluate the process for multi-step reasoning. Traditional Process Reward Models (PRMs) evaluate the reasoning process but require costly human annotations or rollout generation. While implicit PRM is trained only with outcome signals and derives step rewards through reward parameterization without explicit annotations, it is more suitable for multi-step reasoning in MHQA tasks. However, existing implicit PRM has only been explored for plain text scenarios. When adapting to MHQA tasks, it cannot handle the graph structure constraints in KGs and capture the potential inconsistency between CoT and KG paths. To address these limitations, we propose the DPRM (Dual Implicit Process Reward Model). It trains two implicit PRMs for CoT and KG reasoning in MHQA tasks. Both PRMs, namely KG-PRM and CoT-PRM, derive step-level rewards from outcome signals via reward parameterization without additional explicit annotations. Among them, KG-PRM uses preference pairs to learn structural constraints from KGs. DPRM further introduces a consistency constraint between CoT and KG reasoning steps, making the two PRMs mutually verify and collaboratively optimize the reasoning paths. We also provide a theoretical demonstration of the derivation of process rewards. Experimental results show that our method outperforms 13 baselines on multiple datasets with up to 16.6% improvement on Hit@1.

Method: GAD frames the student LLM as a generator and trains a discriminator to distinguish student responses from teacher responses. This creates a minimax game where the discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback for distillation.

Result: GAD consistently surpasses commonly used sequence-level knowledge distillation. Qwen2.5-14B-Instruct trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation.

Conclusion: GAD establishes a promising and effective paradigm for black-box LLM distillation, enabling student models to achieve performance comparable to proprietary teacher models using only text outputs.

Abstract: Black-box distillation creates student large language models (LLMs) by learning from a proprietary teacher model’s text outputs alone, without access to its internal logits or parameters. In this work, we introduce Generative Adversarial Distillation (GAD), which enables on-policy and black-box distillation. GAD frames the student LLM as a generator and trains a discriminator to distinguish its responses from the teacher LLM’s, creating a minimax game. The discriminator acts as an on-policy reward model that co-evolves with the student, providing stable, adaptive feedback. Experimental results show that GAD consistently surpasses the commonly used sequence-level knowledge distillation. In particular, Qwen2.5-14B-Instruct (student) trained with GAD becomes comparable to its teacher, GPT-5-Chat, on the LMSYS-Chat automatic evaluation. The results establish GAD as a promising and effective paradigm for black-box LLM distillation.

Method: Three-stage framework: 1) Continuous pre-training on broad mathematical corpus with novel tasks including “CoT-augmented state prediction” for fine-grained reasoning; 2) Supervised Fine-tuning with expert iteration loop for Spark-Prover-X1-7B and Spark-Formalizer-X1-7B; 3) Targeted Group Relative Policy Optimization (GRPO) on challenging problems.

Result: Spark-Prover achieves SOTA performance among similarly-sized open-source models in “Whole-Proof Generation” paradigm: solves 27 problems on PutnamBench (pass@32) and achieves 24.0% on CombiBench (pass@32).

Conclusion: Diverse training data and progressively refined training pipeline effectively enhance formal reasoning capabilities of lightweight LLMs; models and ExamFormal-Bench dataset will be released.

Abstract: Large Language Models (LLMs) have shown significant promise in automated theorem proving, yet progress is often constrained by the scarcity of diverse and high-quality formal language data. To address this issue, we introduce Spark-Prover-X1, a 7B parameter model trained via an three-stage framework designed to unlock the reasoning potential of more accessible and moderately-sized LLMs. The first stage infuses deep knowledge through continuous pre-training on a broad mathematical corpus, enhanced by a suite of novel data tasks. Key innovation is a “CoT-augmented state prediction” task to achieve fine-grained reasoning. The second stage employs Supervised Fine-tuning (SFT) within an expert iteration loop to specialize both the Spark-Prover-X1-7B and Spark-Formalizer-X1-7B models. Finally, a targeted round of Group Relative Policy Optimization (GRPO) is applied to sharpen the prover’s capabilities on the most challenging problems. To facilitate robust evaluation, particularly on problems from real-world examinations, we also introduce ExamFormal-Bench, a new benchmark dataset of 402 formal problems. Experimental results demonstrate that Spark-Prover achieves state-of-the-art performance among similarly-sized open-source models within the “Whole-Proof Generation” paradigm. It shows exceptional performance on difficult competition benchmarks, notably solving 27 problems on PutnamBench (pass@32) and achieving 24.0% on CombiBench (pass@32). Our work validates that this diverse training data and progressively refined training pipeline provides an effective path for enhancing the formal reasoning capabilities of lightweight LLMs. We will release both Spark-Prover-X1-7B and Spark-Formalizer-X1-7B, along with the ExamFormal-Bench dataset, in the near future.

Method: COMPASS (Context-Modulated PID Attention Steering System) embeds a model-based feedback loop directly within decoding. It quantifies context reliance via a transparent Context Reliance Score (CRS) that probes how attention heads ground generation in evidence. A PID controller then dynamically modulates attention heads to maintain factual consistency without retraining or multi-pass decoding.

Result: Across benchmarks (HotpotQA, XSum, HaluEval, RAGTruth), COMPASS consistently reduces contextual hallucination rates by 2.8 to 5.8 percent absolute. The system reveals how distinct attention heads contribute to evidence alignment.

Conclusion: Feedback-driven interpretability provides a pathway toward scientific understanding of LLM behavior, demonstrating that lightweight control frameworks can effectively steer attention allocation to reduce factual inconsistencies without requiring retraining.

Abstract: Large language models (LLMs) often generate fluent but factually incorrect statements despite having access to relevant evidence, a failure mode rooted in how they allocate attention between contextual and parametric knowledge. Understanding and steering this internal behavior is key both for trustworthy deployment and for scientific interpretability of model mechanisms. We introduce COMPASS (Context-Modulated PID Attention Steering System), a lightweight, interpretable control framework that embeds a model-based feedback loop directly within decoding. COMPASS quantifies context reliance via a transparent metric, the Context Reliance Score (CRS), which serves as an online probe of how attention heads ground generation in evidence. Using this interpretable signal, a PID controller dynamically modulates attention heads to maintain factual consistency without retraining or multi-pass decoding. Across benchmarks (HotpotQA, XSum, HaluEval, RAGTruth), COMPASS consistently reduces contextual hallucination rates (2.8 to 5.8 percent absolute) while revealing how distinct attention heads contribute to evidence alignment. These results highlight feedback-driven interpretability as a pathway toward scientific understanding of LLM behavior.

Method: PEPPER rewrites input captions into semantically distant but visually similar versions while adding unobtrusive elements, disrupting embedded triggers and diluting trigger token influence.

Result: PEPPER is particularly effective against text encoder based attacks, substantially reducing attack success while preserving generation quality, and can be combined with existing defenses for stronger robustness.

Conclusion: PEPPER provides an effective defense against backdoor attacks in T2I models through caption rewriting, offering enhanced robustness that can be further strengthened when combined with other defense methods.

Abstract: Recent studies show that text to image (T2I) diffusion models are vulnerable to backdoor attacks, where a trigger in the input prompt can steer generation toward harmful or unintended content. To address this, we introduce PEPPER (PErcePtion Guided PERturbation), a backdoor defense that rewrites the caption into a semantically distant yet visually similar caption while adding unobstructive elements. With this rewriting strategy, PEPPER disrupt the trigger embedded in the input prompt, dilute the influence of trigger tokens and thereby achieve enhanced robustness. Experiments show that PEPPER is particularly effective against text encoder based attacks, substantially reducing attack success while preserving generation quality. Beyond this, PEPPER can be paired with any existing defenses yielding consistently stronger and generalizable robustness than any standalone method. Our code will be released on Github.

Method: PARROT framework: (i) double-blind comparison of neutral vs. authoritatively false versions of same questions, (ii) log-likelihood calibration tracking of confidence shifts, (iii) eight-state behavioral taxonomy classification of failure modes. Evaluated 22 models on 1,302 MMLU-style questions across 13 domains with authority templates.

Result: Advanced models (GPT-5, GPT-4.1, Claude Sonnet 4.5) show low follow rates (≤11%) and minimal accuracy loss, while older/smaller models exhibit severe epistemic collapse (GPT-4: 80%, Qwen 2.5-1.5B: 94%). Weak models reduce confidence in correct answers while increasing confidence in imposed incorrect ones. International law/global knowledge domains are fragile; elementary math is resilient.

Conclusion: Resistance to overfitting pressure should be a primary objective alongside accuracy, harm avoidance, and privacy for safe LLM deployment in real-world settings where social pressure exists.

Abstract: This study presents PARROT (Persuasion and Agreement Robustness Rating of Output Truth), a robustness focused framework designed to measure the degradation in accuracy that occurs under social pressure exerted on users through authority and persuasion in large language models (LLMs) the phenomenon of sycophancy (excessive conformity). PARROT (i) isolates causal effects by comparing the neutral version of the same question with an authoritatively false version using a double-blind evaluation, (ii) quantifies confidence shifts toward the correct and imposed false responses using log-likelihood-based calibration tracking, and (iii) systematically classifies failure modes (e.g., robust correct, sycophantic agreement, reinforced error, stubborn error, self-correction, etc.) using an eight-state behavioral taxonomy. We evaluated 22 models using 1,302 MMLU-style multiple-choice questions across 13 domains and domain-specific authority templates. Findings show marked heterogeneity: advanced models (e.g., GPT-5, GPT-4.1, Claude Sonnet 4.5) exhibit low “follow rates” ($\leq 11%$, GPT-5: 4%) and minimal accuracy loss, while older/smaller models show severe epistemic collapse (GPT-4: 80%, Qwen 2.5-1.5B: 94%). The danger is not limited to response changes; weak models reduce confidence in the correct response while increasing confidence in the imposed incorrect response. While international law and global knowledge at the domain level exhibit high fragility, elementary mathematics is relatively resilient. Consequently, we argue that the goal of “resistance to overfitting pressure” should be addressed as a primary objective alongside accuracy, harm avoidance, and privacy for safe deployment in the real world.

Method: Digitized all 46 questions from the 2026 Korean CSAT Mathematics section within 2 hours of public release; evaluated 24 state-of-the-art LLMs across different input modalities (Text-only, Image-only, Text+Figure) and prompt languages (Korean, English); conducted reasoning enhancement experiments with GPT-5 series.

Result: GPT-5 family achieved perfect scores (100 points) in certain configurations; Grok 4, Qwen 3 235B, and Gemini 2.5 pro scored above 97 points; gpt-oss-20B achieved 95.7 points despite small size; Calculus was the weakest domain; text input consistently outperformed image input; increased reasoning intensity improved performance but drastically reduced efficiency.

Conclusion: The study establishes a contamination-free evaluation framework, demonstrates that smaller models can be highly cost-effective, reveals domain-specific weaknesses in LLMs, and suggests that practical deployment should balance performance with computational efficiency rather than maximizing reasoning intensity.

Abstract: This study systematically evaluated the mathematical reasoning capabilities of Large Language Models (LLMs) using the 2026 Korean College Scholastic Ability Test (CSAT) Mathematics section, ensuring a completely contamination-free evaluation environment. To address data leakage issues in existing benchmarks, we digitized all 46 questions (22 common and 24 elective) within two hours of the exam’s public release, eliminating any possibility of inclusion in model training data. We conducted comprehensive evaluations of 24 state-of-the-art LLMs across varying input modalities (Text-only, Image-only, Text+Figure) and prompt languages (Korean, English). The GPT-5 family models achieved perfect scores (100 points) under a limited set of language-modality configurations, while Grok 4, Qwen 3 235B, and Gemini 2.5 pro also scored above 97 points. Notably, gpt-oss-20B achieved 95.7 points despite its relatively small size, demonstrating high cost-effectiveness. Problem-specific analysis revealed Calculus as the weakest domain with significant performance degradation on 4-point high-difficulty problems. Text input consistently outperformed image input, while prompt language effects varied by model scale. In reasoning enhancement experiments with GPT-5 series, increased reasoning intensity improved performance (82.6->100 points) but quadrupled token usage and drastically reduced efficiency, suggesting that models with minimal reasoning may be more practical. This research contributes: (1) implementation of a completely unexposed evaluation environment, (2) a standardized digitization pipeline that converts human-targeted exam materials into LLM-ready evaluation data, and (3) a practical evaluation perspective integrating performance, cost, and time considerations. Detailed results and model comparisons are available at the 2026 Korean CSAT LLM Evaluation Leaderboard; https://isoft.cnu.ac.kr/csat2026/

Method: Distribution-based feature Recovery and Fusion (DRF) maintains feature queues for each modality to approximate distributions, reduces contributions of low-quality modalities based on quality estimation, and recovers missing modalities through inter-modal mapping supervised by samples and distributions.

Result: DRF demonstrates universal improvements compared to SOTA methods on three publicly available image-text datasets under disruption strategies mimicking real-world low-quality and missing modalities.

Conclusion: DRF provides an effective unified framework for robust multimodal sentiment analysis that handles both low-quality and missing modalities in real-world scenarios.

Abstract: As posts on social media increase rapidly, analyzing the sentiments embedded in image-text pairs has become a popular research topic in recent years. Although existing works achieve impressive accomplishments in simultaneously harnessing image and text information, they lack the considerations of possible low-quality and missing modalities. In real-world applications, these issues might frequently occur, leading to urgent needs for models capable of predicting sentiment robustly. Therefore, we propose a Distribution-based feature Recovery and Fusion (DRF) method for robust multimodal sentiment analysis of image-text pairs. Specifically, we maintain a feature queue for each modality to approximate their feature distributions, through which we can simultaneously handle low-quality and missing modalities in a unified framework. For low-quality modalities, we reduce their contributions to the fusion by quantitatively estimating modality qualities based on the distributions. For missing modalities, we build inter-modal mapping relationships supervised by samples and distributions, thereby recovering the missing modalities from available ones. In experiments, two disruption strategies that corrupt and discard some modalities in samples are adopted to mimic the low-quality and missing modalities in various real-world scenarios. Through comprehensive experiments on three publicly available image-text datasets, we demonstrate the universal improvements of DRF compared to SOTA methods under both two strategies, validating its effectiveness in robust multimodal sentiment analysis.

Method: Unified multiclass classification framework using curated Twitter/Reddit datasets with “split-then-balance” pipeline; comprehensive evaluation of lexical models, hybrid approaches, and fine-tuned transformers; introduces hybrid SHAP-LLM explainability framework and prototype dashboard for human-in-the-loop screening.

Result: Domain-adapted MentalBERT emerged as top performer with 0.92 accuracy and 0.76 Macro F1 score, surpassing generic transformers and zero-shot LLM baselines; framework successfully detects 10 distinct mental health and cyberbullying categories.

Conclusion: End-to-end fine-tuning is critical for performance; system is framed as human-in-the-loop screening aid (not diagnostic tool) with explainable AI; provides robust baseline highlighting need for multi-label, clinically-validated datasets at intersection of online safety and computational mental health.

Abstract: Mental health challenges and cyberbullying are increasingly prevalent in digital spaces, necessitating scalable and interpretable detection systems. This paper introduces a unified multiclass classification framework for detecting ten distinct mental health and cyberbullying categories from social media data. We curate datasets from Twitter and Reddit, implementing a rigorous “split-then-balance” pipeline to train on balanced data while evaluating on a realistic, held-out imbalanced test set. We conducted a comprehensive evaluation comparing traditional lexical models, hybrid approaches, and several end-to-end fine-tuned transformers. Our results demonstrate that end-to-end fine-tuning is critical for performance, with the domain-adapted MentalBERT emerging as the top model, achieving an accuracy of 0.92 and a Macro F1 score of 0.76, surpassing both its generic counterpart and a zero-shot LLM baseline. Grounded in a comprehensive ethical analysis, we frame the system as a human-in-the-loop screening aid, not a diagnostic tool. To support this, we introduce a hybrid SHAPLLM explainability framework and present a prototype dashboard (“Social Media Screener”) designed to integrate model predictions and their explanations into a practical workflow for moderators. Our work provides a robust baseline, highlighting future needs for multi-label, clinically-validated datasets at the critical intersection of online safety and computational mental health.

Method: Uses Projected Gradient Descent (PGD) to maximize next-token entropy using a small ensemble of open-source MLLMs. Works in both full-image and Adversarial CAPTCHA settings, with single adversarial images disrupting all models in the ensemble.

Result: The attack successfully disrupts all models in the ensemble and transfers to unseen open-source (Qwen3-VL) and proprietary (GPT-5.1) models, demonstrating strong transferability despite using basic adversarial techniques.

Conclusion: Adversarial Confusion Attack represents a new class of threats against MLLMs that can systematically disrupt model performance, highlighting vulnerabilities in multimodal systems and the need for robust defenses against such attacks.

Abstract: We introduce the Adversarial Confusion Attack, a new class of threats against multimodal large language models (MLLMs). Unlike jailbreaks or targeted misclassification, the goal is to induce systematic disruption that makes the model generate incoherent or confidently incorrect outputs. Practical applications include embedding such adversarial images into websites to prevent MLLM-powered AI Agents from operating reliably. The proposed attack maximizes next-token entropy using a small ensemble of open-source MLLMs. In the white-box setting, we show that a single adversarial image can disrupt all models in the ensemble, both in the full-image and Adversarial CAPTCHA settings. Despite relying on a basic adversarial technique (PGD), the attack generates perturbations that transfer to both unseen open-source (e.g., Qwen3-VL) and proprietary (e.g., GPT-5.1) models.

Method: The study uses AutoML pipelines to extract important features from data segments collected by triaxial accelerometer sensors, then trains multiple lightweight machine learning algorithms using these features. The system is implemented on WeBe Band, a multi-sensor wearable device with a powerful MCU capable of on-device gesture recognition.

Result: Among the explored models, neural networks provided the best balance between accuracy, latency, and memory use. The system demonstrates reliable real-time gesture recognition on WeBe Band, showing great potential for secure, fast-response medical monitoring solutions.

Conclusion: The research successfully demonstrates that efficient motion-based models using only triaxial accelerometer sensors can achieve reliable real-time gesture recognition on wearable devices, offering promising solutions for medical monitoring applications requiring secure and fast response times.

Abstract: The use of tiny devices capable of low-latency gesture recognition is gaining momentum in everyday human-computer interaction and especially in medical monitoring fields. Embedded solutions such as fall detection, rehabilitation tracking, and patient supervision require fast and efficient tracking of movements while avoiding unwanted false alarms. This study presents an efficient solution on how to build very efficient motion-based models only using triaxial accelerometer sensors. We explore the capability of the AutoML pipelines to extract the most important features from the data segments. This approach also involves training multiple lightweight machine learning algorithms using the extracted features. We use WeBe Band, a multi-sensor wearable device that is equipped with a powerful enough MCU to effectively perform gesture recognition entirely on the device. Of the models explored, we found that the neural network provided the best balance between accuracy, latency, and memory use. Our results also demonstrate that reliable real-time gesture recognition can be achieved in WeBe Band, with great potential for real-time medical monitoring solutions that require a secure and fast response time.

Method: Compiled a 161.4 million token multimodal dataset combining textbook question-solution pairs, curriculum-aligned diagrams, and contextual materials. Fine-tuned Qwen-2.5VL-32B using an optimized reasoning syntax called QMSA. Evaluated on YKSUniform benchmark with 1,854 multimodal exam questions across 309 curriculum topics.

Result: The resulting model achieved 78.6% accuracy on YKSUniform benchmark, only 1.0% below Gemini 2.0 Flash. This demonstrates that supervised fine-tuning with carefully curated data can approach state-of-the-art performance.

Conclusion: Data composition and representational syntax play decisive roles in multimodal reasoning. The work establishes a data-centric framework showing that curriculum-grounded multimodal data can elevate supervised fine-tuning to near state-of-the-art performance for open-weight vision-language models.

Abstract: Multimodal reasoning has become a cornerstone of modern AI research. Standardized exam questions offer a uniquely rigorous testbed for such reasoning, providing structured visual contexts and verifiable answers. While recent progress has largely focused on algorithmic advances such as reinforcement learning (e.g., GRPO, DPO), the data centric foundations of vision language reasoning remain less explored. We show that supervised fine-tuning (SFT) with high-quality data can rival proprietary approaches. To this end, we compile a 161.4 million token multimodal dataset combining textbook question-solution pairs, curriculum aligned diagrams, and contextual materials, and fine-tune Qwen-2.5VL-32B using an optimized reasoning syntax (QMSA). The resulting model achieves 78.6% accuracy, only 1.0% below Gemini 2.0 Flash, on our newly released benchmark YKSUniform, which standardizes 1,854 multimodal exam questions across 309 curriculum topics. Our results reveal that data composition and representational syntax play a decisive role in multimodal reasoning. This work establishes a data centric framework for advancing open weight vision language models, demonstrating that carefully curated and curriculum-grounded multimodal data can elevate supervised fine-tuning to near state-of-the-art performance.

Method: PEFT-DML maps diverse modalities (LiDAR, radar, camera, IMU, GNSS) into a shared latent space using parameter-efficient techniques like Low-Rank Adaptation (LoRA) and adapter layers, enabling robust detection under various sensor conditions.

Result: Experiments on nuScenes benchmarks demonstrate superior accuracy and robustness to sensor dropout, fast motion, weather variability, and domain shifts while achieving significant training efficiency.

Conclusion: PEFT-DML provides an efficient and robust solution for multi-modal 3D object detection in autonomous driving that can handle real-world sensor variability and environmental challenges while maintaining high accuracy.

Abstract: This study introduces PEFT-DML, a parameter-efficient deep metric learning framework for robust multi-modal 3D object detection in autonomous driving. Unlike conventional models that assume fixed sensor availability, PEFT-DML maps diverse modalities (LiDAR, radar, camera, IMU, GNSS) into a shared latent space, enabling reliable detection even under sensor dropout or unseen modality class combinations. By integrating Low-Rank Adaptation (LoRA) and adapter layers, PEFT-DML achieves significant training efficiency while enhancing robustness to fast motion, weather variability, and domain shifts. Experiments on benchmarks nuScenes demonstrate superior accuracy.

Method: Proposes DL-CapsNet with multiple capsule layers and a Capsule Summarization layer to reduce parameters and complexity while maintaining accuracy.

Result: DL-CapsNet achieves high accuracy with fewer parameters, faster training/inference, and can handle complex datasets with many categories.

Conclusion: DL-CapsNet provides an efficient deep variant of CapsNet that balances accuracy and computational efficiency for complex image classification tasks.

Abstract: Capsule Network (CapsNet) is among the promising classifiers and a possible successor of the classifiers built based on Convolutional Neural Network (CNN). CapsNet is more accurate than CNNs in detecting images with overlapping categories and those with applied affine transformations. In this work, we propose a deep variant of CapsNet consisting of several capsule layers. In addition, we design the Capsule Summarization layer to reduce the complexity by reducing the number of parameters. DL-CapsNet, while being highly accurate, employs a small number of parameters and delivers faster training and inference. DL-CapsNet can process complex datasets with a high number of categories.

Method: Modified dual-input U-Net architecture with enhanced feature fusion to jointly process pre- and post-disaster satellite images. Uses class-aware weighted loss functions to handle dominance of undamaged pixels and improve sensitivity to major and destroyed damage categories.

Result: Experiments on public disaster datasets show improved localization and classification of structural damage compared to traditional segmentation and baseline change-detection models.

Conclusion: The framework provides rapid, consistent damage assessment to support (not replace) expert decision-making, enabling more efficient, data-driven disaster management through fine-grained pixel-level damage maps.

Abstract: Accurate post-disaster damage assessment is of high importance for prioritizing emergency response; however, manual interpretation of satellite imagery is slow, subjective, and hard to scale. While deep-learning models for image segmentation, such as U-Net-based baselines and change-detection models, are useful baselines, they often struggle with subtle structural variations and severe class imbalance, yielding poor detection of highly damaged regions. The present work proposes a deep-learning framework that jointly processes pre- and post-disaster satellite images to obtain fine-grained pixel-level damage maps: Satellite-to-Street: Disaster Impact Estimator. The model uses a modified dual-input U-Net architecture with enhanced feature fusion to capture both the local structural changes as well as the broader contextual cues. Class-aware weighted loss functions are integrated in order to handle the dominance of undamaged pixels in real disaster datasets, thus enhancing sensitivity toward major and destroyed categories. Experimentation on publicly available disaster datasets shows improved localization and classification of structural damage when compared to traditional segmentation and baseline change-detection models. The resulting damage maps provide a rapid and consistent assessment mechanism to support and not replace expert decision-making, thus allowing more efficient, data-driven disaster management.

Method: Developed ProvRain pipeline using lightweight MobileNet-U-Net architecture with curricula training to generalize to robust weather conditions, trained on synthetic and real PVDN dataset data, compared against Faster RCNN baseline.

Result: 8.94% increase in accuracy and 10.25% increase in recall for vehicle detection in rainy night-time frames; MobileNet-U-Net shows 10-15% PSNR improvement, 5-6% SSIM increase, and up to 67% reduction in perceptual error compared to transformer approaches.

Conclusion: The ProvRain pipeline effectively handles noise in night-time rainy conditions while maintaining detection accuracy, demonstrating significant improvements over baseline methods and superior performance compared to transformer-based approaches.

Abstract: Provident vehicle detection has a lot of scope in the detection of vehicle during night time. The extraction of features other than the headlamps of vehicles allows us to detect oncoming vehicles before they appear directly on the camera. However, it faces multiple issues especially in the field of night vision, where a lot of noise caused due to weather conditions such as rain or snow as well as camera conditions. This paper focuses on creating a pipeline aimed at dealing with such noise while at the same time maintaining the accuracy of provident vehicular detection. The pipeline in this paper, ProvRain, uses a lightweight MobileNet-U-Net architecture tuned to generalize to robust weather conditions by using the concept of curricula training. A mix of synthetic as well as available data from the PVDN dataset is used for this. This pipeline is compared to the base Faster RCNN architecture trained on the PVDN dataset to see how much the addition of a denoising architecture helps increase the detection model’s performance in rainy conditions. The system boasts an 8.94% increase in accuracy and a 10.25% increase in recall in the detection of vehicles in rainy night time frames. Similarly, the custom MobileNet-U-Net architecture that was trained also shows a 10-15% improvement in PSNR, a 5-6% increase in SSIM, and upto a 67% reduction in perceptual error (LPIPS) compared to other transformer approaches.

Method: The method investigates how zero-shot methods use image encoders to extract embeddings fed into diffusion models via cross-attention. It constructs a reversible encryption system mapping original embeddings to encrypted representations using secret keys, and designs multi-target adversarial perturbations to shift embeddings toward encrypted patterns.

Result: Extensive evaluations show the method surpasses existing state-of-the-art defenses in blocking unauthorized zero-shot image synthesis while supporting flexible and secure access control for verified users.

Conclusion: Adapter Shield provides the first universal and authentication-integrated solution for protecting personal images from misuse in zero-shot generation scenarios, effectively preventing unauthorized identity cloning and stylistic imitation while maintaining authorized functionality.

Abstract: With the rapid progress in diffusion models, image synthesis has advanced to the stage of zero-shot image-to-image generation, where high-fidelity replication of facial identities or artistic styles can be achieved using just one portrait or artwork, without modifying any model weights. Although these techniques significantly enhance creative possibilities, they also pose substantial risks related to intellectual property violations, including unauthorized identity cloning and stylistic imitation. To counter such threats, this work presents Adapter Shield, the first universal and authentication-integrated solution aimed at defending personal images from misuse in zero-shot generation scenarios. We first investigate how current zero-shot methods employ image encoders to extract embeddings from input images, which are subsequently fed into the UNet of diffusion models through cross-attention layers. Inspired by this mechanism, we construct a reversible encryption system that maps original embeddings into distinct encrypted representations according to different secret keys. The authorized users can restore the authentic embeddings via a decryption module and the correct key, enabling normal usage for authorized generation tasks. For protection purposes, we design a multi-target adversarial perturbation method that actively shifts the original embeddings toward designated encrypted patterns. Consequently, protected images are embedded with a defensive layer that ensures unauthorized users can only produce distorted or encrypted outputs. Extensive evaluations demonstrate that our method surpasses existing state-of-the-art defenses in blocking unauthorized zero-shot image synthesis, while supporting flexible and secure access control for verified users.

Method: Trained U-Net based diffusion model to generate synthetic brightfield microscopy images, created datasets with varying synthetic-real ratios, tested with YOLOv8, YOLOv9 and RT-DETR object detectors, and conducted human expert survey for realism evaluation.

Result: Synthetic data improved detection accuracies at minimal costs; human experts could not distinguish synthetic from real images (50% accuracy, random chance level).

Conclusion: Diffusion-based synthetic data generation is promising for augmenting microscopy datasets, reducing manual annotation reliance, and potentially improving cell detection model robustness.

Abstract: Accurate single cell detection in brightfield microscopy is crucial for biological research, yet data scarcity and annotation bottlenecks limit the progress of deep learning methods. We investigate the use of unconditional models to generate synthetic brightfield microscopy images and evaluate their impact on object detection performance. A U-Net based diffusion model was trained and used to create datasets with varying ratios of synthetic and real images. Experiments with YOLOv8, YOLOv9 and RT-DETR reveal that training with synthetic data can achieve improved detection accuracies (at minimal costs). A human expert survey demonstrates the high realism of generated images, with experts not capable to distinguish them from real microscopy images (accuracy 50%). Our findings suggest that diffusion-based synthetic data generation is a promising avenue for augmenting real datasets in microscopy image analysis, reducing the reliance on extensive manual annotation and potentially improving the robustness of cell detection models.

Method: TinyViT pipeline combines Transformer-based segmentation with spectral-spatial feature engineering and ensemble regression. It uses only consumer-grade color camera images to classify 7 nuanced surface faults and generate actionable severity grades.

Result: The system achieves competitive accuracy and interpretability on real-world datasets for both classification and regression tasks, validating its effectiveness without specialized sensors.

Conclusion: By eliminating reliance on expensive multi-modal sensors, this approach enables affordable, scalable solar health monitoring with universal field accessibility for resource-limited installations.

Abstract: Sustained operation of solar photovoltaic assets hinges on accurate detection and prioritization of surface faults across vast, geographically distributed modules. While multi modal imaging strategies are popular, they introduce logistical and economic barriers for routine farm level deployment. This work demonstrates that deep learning and classical machine learning may be judiciously combined to achieve robust surface anomaly categorization and severity estimation from planar visible band imagery alone. We introduce TinyViT which is a compact pipeline integrating Transformer based segmentation, spectral-spatial feature engineering, and ensemble regression. The system ingests consumer grade color camera mosaics of PV panels, classifies seven nuanced surface faults, and generates actionable severity grades for maintenance triage. By eliminating reliance on electroluminescence or IR sensors, our method enables affordable, scalable upkeep for resource limited installations, and advances the state of solar health monitoring toward universal field accessibility. Experiments on real public world datasets validate both classification and regression sub modules, achieving accuracy and interpretability competitive with specialized approaches.

Method: Proposes deep visual stereo odometry (DVSO) using stereo disparity, optical flow, and self-supervised learning to jointly estimate depth and ego-motion without labeled data, with potential fusion with absolute references or other sensors for global consistency.

Result: Conceptual evaluation shows expected benefits include reduced scale drift via stereo, low-cost sensing, and scalable data collection, but challenges remain in low-texture surfaces, lighting variability, computational load, and radiation robustness.

Conclusion: DVSO offers a promising alternative for autonomous navigation in constrained accelerator tunnels, defining a research agenda to enable MARWIN to navigate more autonomously in safety-critical infrastructure environments.

Abstract: The MARWIN robot operates at the European XFEL to perform autonomous radiation monitoring in long, monotonous accelerator tunnels where conventional localization approaches struggle. Its current navigation concept combines lidar-based edge detection, wheel/lidar odometry with periodic QR-code referencing, and fuzzy control of wall distance, rotation, and longitudinal position. While robust in predefined sections, this design lacks flexibility for unknown geometries and obstacles. This paper explores deep visual stereo odometry (DVSO) with 3D-geometric constraints as a focused alternative. DVSO is purely vision-based, leveraging stereo disparity, optical flow, and self-supervised learning to jointly estimate depth and ego-motion without labeled data. For global consistency, DVSO can subsequently be fused with absolute references (e.g., landmarks) or other sensors. We provide a conceptual evaluation for accelerator tunnel environments, using the European XFEL as a case study. Expected benefits include reduced scale drift via stereo, low-cost sensing, and scalable data collection, while challenges remain in low-texture surfaces, lighting variability, computational load, and robustness under radiation. The paper defines a research agenda toward enabling MARWIN to navigate more autonomously in constrained, safety-critical infrastructures.

Method: Formulates compression as a mixed-integer nonlinear programming (MINLP) problem, decouples it into discrete reserve ratio sampling and integer linear programming for bit-width settings, and implements quality/size estimators with CUDA kernels.

Result: Achieves state-of-the-art performance in post-training compression on multiple 3DGS variants, and achieves comparable quality to leading training-required methods after fine-tuning.

Conclusion: Proposes an effective size-aware compression framework for 3DGS that can adapt to bandwidth constraints while maintaining visual quality, bridging the gap between post-training and training-required compression methods.

Abstract: Recent advances in 3D Gaussian Splatting (3DGS) have greatly improved 3D reconstruction. However, its substantial data size poses a significant challenge for transmission and storage. While many compression techniques have been proposed, they fail to efficiently adapt to fluctuating network bandwidth, leading to resource wastage. We address this issue from the perspective of size-aware compression, where we aim to compress 3DGS to a desired size by quickly searching for suitable hyperparameters. Through a measurement study, we identify key hyperparameters that affect the size – namely, the reserve ratio of Gaussians and bit-width settings for Gaussian attributes. Then, we formulate this hyperparameter optimization problem as a mixed-integer nonlinear programming (MINLP) problem, with the goal of maximizing visual quality while respecting the size budget constraint. To solve the MINLP, we decouple this problem into two parts: discretely sampling the reserve ratio and determining the bit-width settings using integer linear programming (ILP). To solve the ILP more quickly and accurately, we design a quality loss estimator and a calibrated size estimator, as well as implement a CUDA kernel. Extensive experiments on multiple 3DGS variants demonstrate that our method achieves state-of-the-art performance in post-training compression. Furthermore, our method can achieve comparable quality to leading training-required methods after fine-tuning.

Method: Compared diagnostic prompting with standard gestalt principles-based prompting using 200 Amazon SRP pages and human expert annotations, analyzed decision trees to understand reasoning patterns, and conducted failure case analysis.

Result: Diagnostic prompting showed notable improvements with F1-score increasing from 0.031 to 0.297 (+858% relative improvement), though absolute performance remains modest (Cohen’s κ = 0.071). Models prioritize visual design elements while humans emphasize content similarity, revealing partial alignment in reasoning patterns.

Conclusion: Diagnostic prompting represents a promising initial step toward human-aligned MLLM-based evaluation, but persistent challenges in visual perception (product similarity, color intensity) require continued research, refinement in prompting approaches, and larger ground truth datasets for reliable practical deployment.

Abstract: This study investigates whether diagnostic prompting can improve Multimodal Large Language Model (MLLM) reliability for visual complexity assessment of Amazon Search Results Pages (SRP). We compare diagnostic prompting with standard gestalt principles-based prompting using 200 Amazon SRP pages and human expert annotations. Diagnostic prompting showed notable improvements in predicting human complexity judgments, with F1-score increasing from 0.031 to 0.297 (+858% relative improvement), though absolute performance remains modest (Cohen’s $κ$ = 0.071). The decision tree revealed that models prioritize visual design elements (badge clutter: 38.6% importance) while humans emphasize content similarity, suggesting partial alignment in reasoning patterns. Failure case analysis reveals persistent challenges in MLLM visual perception, particularly for product similarity and color intensity assessment. Our findings indicate that diagnostic prompting represents a promising initial step toward human-aligned MLLM-based evaluation, though failure cases with consistent human-MLLM disagreement require continued research and refinement in prompting approaches with larger ground truth datasets for reliable practical deployment.

Method: Proposes an optimal transport-guided prompt learning framework that enforces joint constraints on both vision and text representations to preserve structural consistency of feature distributions between pre-trained and fine-tuned models, capturing cross-instance relationships and expanding feasible parameter space.

Result: Outperforms existing prompt learning strategies in base-to-novel generalization, cross-dataset evaluation, and domain generalization without additional augmentation or ensemble techniques, as demonstrated in extensive experiments on benchmark datasets.

Conclusion: The OT-guided framework provides a simple yet effective solution to mitigate forgetting in prompt learning for VLMs, achieving better trade-off between adaptation and generalization through holistic feature alignment.

Abstract: Vision-language models (VLMs) such as CLIP demonstrate strong performance but struggle when adapted to downstream tasks. Prompt learning has emerged as an efficient and effective strategy to adapt VLMs while preserving their pre-trained knowledge. However, existing methods still lead to overfitting and degrade zero-shot generalization. To address this challenge, we propose an optimal transport (OT)-guided prompt learning framework that mitigates forgetting by preserving the structural consistency of feature distributions between pre-trained and fine-tuned models. Unlike conventional point-wise constraints, OT naturally captures cross-instance relationships and expands the feasible parameter space for prompt tuning, allowing a better trade-off between adaptation and generalization. Our approach enforces joint constraints on both vision and text representations, ensuring a holistic feature alignment. Extensive experiments on benchmark datasets demonstrate that our simple yet effective method can outperform existing prompt learning strategies in base-to-novel generalization, cross-dataset evaluation, and domain generalization without additional augmentation or ensemble techniques. The code is available at https://github.com/ChongQingNoSubway/Prompt-OT

Method: Proposes a lightweight convolutional neural network specifically designed for speckle patterns, minimizing parameters while maintaining discriminative power. Uses the complete SensiCut dataset of 59 material classes across various categories.

Result: Achieves 95.05% test accuracy with 0.951 F1-scores, using only 341k parameters (70x fewer than ResNet-50). Inference speed of 295 images/sec enables Raspberry Pi/Jetson deployment. Material family regrouping achieves 98-100% recall for practical laser cutter settings.

Conclusion: Compact, domain-specific CNNs outperform large backbone networks for speckle-based material classification, advancing material-aware, edge-deployable laser cutting systems with practical applications in power/speed preset selection.

Abstract: Accurate material recognition is critical for safe and effective laser cutting, as misidentification can lead to poor cut quality, machine damage, or the release of hazardous fumes. Laser speckle sensing has recently emerged as a low-cost and non-destructive modality for material classification; however, prior work has either relied on computationally expensive backbone networks or addressed only limited subsets of materials. In this study, A lightweight convolutional neural network (CNN) tailored for speckle patterns is proposed, designed to minimize parameters while maintaining high discriminative power. Using the complete SensiCut dataset of 59 material classes spanning woods, acrylics, composites, textiles, metals, and paper-based products, the proposed model achieves 95.05% test accuracy, with macro and weighted F1-scores of 0.951. The network contains only 341k trainable parameters (~1.3 MB) – over 70X fewer than ResNet-50 – and achieves an inference speed of 295 images per second, enabling deployment on Raspberry Pi and Jetson-class devices. Furthermore, when materials are regrouped into nine and five practical families, recall exceeds 98% and approaches 100%, directly supporting power and speed preset selection in laser cutters. These results demonstrate that compact, domain-specific CNNs can outperform large backbones for speckle-based material classification, advancing the feasibility of material-aware, edge-deployable laser cutting systems.

Method: Proposes FastTextDiff, which integrates medical text annotations with diffusion models using ModernBERT (a transformer for long clinical notes) to link textual annotations with semantic image content. Uses cross-modal attention between visual and textual features, trained on MIMIC-III and MIMIC-IV datasets.

Result: ModernBERT serves as a fast, scalable alternative to Clinical BioBERT in diffusion-based segmentation pipelines. FastTextDiff benefits from FlashAttention 2, alternating attention mechanism, and a 2-trillion-token corpus, improving both segmentation accuracy and training efficiency over traditional diffusion-based models.

Conclusion: The study validates ModernBERT as an effective alternative to Clinical BioBERT and highlights the promise of multi-modal techniques for medical image analysis, demonstrating improved segmentation performance with reduced labeling requirements.

Abstract: In recent times, denoising diffusion probabilistic models (DPMs) have proven effective for medical image generation and denoising, and as representation learners for downstream segmentation. However, segmentation performance is limited by the need for dense pixel-wise labels, which are expensive, time-consuming, and require expert knowledge. We propose FastTextDiff, a label-efficient diffusion-based segmentation model that integrates medical text annotations to enhance semantic representations. Our approach uses ModernBERT, a transformer capable of processing long clinical notes, to tightly link textual annotations with semantic content in medical images. Trained on MIMIC-III and MIMIC-IV, ModernBERT encodes clinical knowledge that guides cross-modal attention between visual and textual features. This study validates ModernBERT as a fast, scalable alternative to Clinical BioBERT in diffusion-based segmentation pipelines and highlights the promise of multi-modal techniques for medical image analysis. By replacing Clinical BioBERT with ModernBERT, FastTextDiff benefits from FlashAttention 2, an alternating attention mechanism, and a 2-trillion-token corpus, improving both segmentation accuracy and training efficiency over traditional diffusion-based models.

Method: Introduces Proxy-Tuning, which leverages diffusion models to enhance AR models’ capabilities in subject-specific image generation. The method involves using diffusion models as supervisors to fine-tune AR models, enabling them to learn subject-driven generation capabilities.

Result: Reveals a striking weak-to-strong phenomenon: fine-tuned AR models consistently outperform their diffusion model supervisors in both subject fidelity and prompt adherence. AR models excel particularly in multi-subject compositions and contextual understanding.

Conclusion: The work demonstrates impressive results in subject-driven AR image generation and unveils the potential of weak-to-strong generalization in the image generation domain. It contributes to a deeper understanding of different architectures’ strengths and limitations, showing that AR models can surpass diffusion models in specific subject-driven tasks after proper tuning.

Abstract: Multimodal autoregressive (AR) models, based on next-token prediction and transformer architecture, have demonstrated remarkable capabilities in various multimodal tasks including text-to-image (T2I) generation. Despite their strong performance in general T2I tasks, our research reveals that these models initially struggle with subject-driven image generation compared to dominant diffusion models. To address this limitation, we introduce Proxy-Tuning, leveraging diffusion models to enhance AR models’ capabilities in subject-specific image generation. Our method reveals a striking weak-to-strong phenomenon: fine-tuned AR models consistently outperform their diffusion model supervisors in both subject fidelity and prompt adherence. We analyze this performance shift and identify scenarios where AR models excel, particularly in multi-subject compositions and contextual understanding. This work not only demonstrates impressive results in subject-driven AR image generation, but also unveils the potential of weak-to-strong generalization in the image generation domain, contributing to a deeper understanding of different architectures’ strengths and limitations.

Method: ICVision uses GPT-4 Vision to directly interpret ICA dashboard visualizations (topography, time series, power spectra, and ERP plots) as multimodal inputs. It classifies components into six canonical categories and provides both confidence scores and natural language explanations.

Result: Evaluated on 3,168 ICA components from 124 EEG datasets, ICVision achieved k = 0.677 agreement with expert consensus, surpassing MNE ICLabel. Over 97% of outputs were rated as interpretable and actionable by experts, and it preserved clinically relevant brain signals in ambiguous cases.

Conclusion: ICVision represents a paradigm shift toward AI agents capable of expert-level visual decision-making in brain science. It enables scalable, explainable, and reproducible EEG workflows and opens doors for AI visual cognition applications beyond neurophysiology.

Abstract: We introduce EEG Autoclean Vision Language AI (ICVision) a first-of-its-kind system that emulates expert-level EEG ICA component classification through AI-agent vision and natural language reasoning. Unlike conventional classifiers such as ICLabel, which rely on handcrafted features, ICVision directly interprets ICA dashboard visualizations topography, time series, power spectra, and ERP plots, using a multimodal large language model (GPT-4 Vision). This allows the AI to see and explain EEG components the way trained neurologists do, making it the first scientific implementation of AI-agent visual cognition in neurophysiology. ICVision classifies each component into one of six canonical categories (brain, eye, heart, muscle, channel noise, and other noise), returning both a confidence score and a human-like explanation. Evaluated on 3,168 ICA components from 124 EEG datasets, ICVision achieved k = 0.677 agreement with expert consensus, surpassing MNE ICLabel, while also preserving clinically relevant brain signals in ambiguous cases. Over 97% of its outputs were rated as interpretable and actionable by expert reviewers. As a core module of the open-source EEG Autoclean platform, ICVision signals a paradigm shift in scientific AI, where models do not just classify, but see, reason, and communicate. It opens the door to globally scalable, explainable, and reproducible EEG workflows, marking the emergence of AI agents capable of expert-level visual decision-making in brain science and beyond.

Method: Multi-modal on-device learning with tiny μPyD-Net model (107k params), using monocular camera for inference and activating low-power depth sensor (8x8 pixels) only for collecting pseudo-labels in new environments. Introduces memory-driven sparse update scheme to minimize fine-tuning memory to 1.2MB.

Result: Reduces RMSE from 4.9m to 0.6m with only 3k self-labeled samples in real deployment. Fine-tuning takes 17.8 minutes on IoT node with only 2% and 1.5% accuracy drops on KITTI and NYUv2 datasets compared to full update.

Conclusion: Demonstrates practical on-device learning for monocular depth estimation on ULP IoT platforms, enabling adaptation to new environments with minimal energy consumption and memory overhead.

Abstract: Monocular depth estimation (MDE) plays a crucial role in enabling spatially-aware applications in Ultra-low-power (ULP) Internet-of-Things (IoT) platforms. However, the limited number of parameters of Deep Neural Networks for the MDE task, designed for IoT nodes, results in severe accuracy drops when the sensor data observed in the field shifts significantly from the training dataset. To address this domain shift problem, we present a multi-modal On-Device Learning (ODL) technique, deployed on an IoT device integrating a Greenwaves GAP9 MicroController Unit (MCU), a 80 mW monocular camera and a 8 x 8 pixel depth sensor, consuming $\approx$300mW. In its normal operation, this setup feeds a tiny 107 k-parameter $μ$PyD-Net model with monocular images for inference. The depth sensor, usually deactivated to minimize energy consumption, is only activated alongside the camera to collect pseudo-labels when the system is placed in a new environment. Then, the fine-tuning task is performed entirely on the MCU, using the new data. To optimize our backpropagation-based on-device training, we introduce a novel memory-driven sparse update scheme, which minimizes the fine-tuning memory to 1.2 MB, 2.2x less than a full update, while preserving accuracy (i.e., only 2% and 1.5% drops on the KITTI and NYUv2 datasets). Our in-field tests demonstrate, for the first time, that ODL for MDE can be performed in 17.8 minutes on the IoT node, reducing the root mean squared error from 4.9 to 0.6m with only 3 k self-labeled samples, collected in a real-life deployment scenario.

Method: Introduces VIVAT with a taxonomy of five common artifacts (color shift, grid patterns, blur, corner and droplet artifacts), analyzes their root causes, and implements straightforward modifications including loss weight adjustments, padding strategies, and integration of Spatially Conditional Normalization.

Result: Achieves state-of-the-art results in image reconstruction metrics (PSNR and SSIM) across multiple benchmarks and improves text-to-image generation quality with superior CLIP scores.

Conclusion: VIVAT provides an effective, practical solution for mitigating VAE artifacts without radical architectural changes, offering actionable insights for researchers and practitioners to optimize VAE training while maintaining framework simplicity.

Abstract: Variational Autoencoders (VAEs) remain a cornerstone of generative computer vision, yet their training is often plagued by artifacts that degrade reconstruction and generation quality. This paper introduces VIVAT, a systematic approach to mitigating common artifacts in KL-VAE training without requiring radical architectural changes. We present a detailed taxonomy of five prevalent artifacts - color shift, grid patterns, blur, corner and droplet artifacts - and analyze their root causes. Through straightforward modifications, including adjustments to loss weights, padding strategies, and the integration of Spatially Conditional Normalization, we demonstrate significant improvements in VAE performance. Our method achieves state-of-the-art results in image reconstruction metrics (PSNR and SSIM) across multiple benchmarks and enhances text-to-image generation quality, as evidenced by superior CLIP scores. By preserving the simplicity of the KL-VAE framework while addressing its practical challenges, VIVAT offers actionable insights for researchers and practitioners aiming to optimize VAE training.

Method: Parallel modality-specific pipelines: for video, evaluate zero-shot multimodal LLMs, fine-tuned vision-language models, and self-supervised video transformers on 24 activity labels; for transcripts, fine-tune transformer-based classifier with contextualized inputs and compare against prompting-based LLMs on 19 discourse labels. Use per-label thresholding, context windows, and imbalance-aware loss functions to handle class imbalance and multi-label complexity.

Result: Fine-tuned models consistently outperform prompting-based approaches, achieving macro-F1 scores of 0.577 for video analysis and 0.460 for transcript analysis.

Conclusion: The results demonstrate feasibility of automated classroom analysis and establish a foundation for scalable teacher feedback systems, showing that fine-tuned models are more effective than prompting-based approaches for this task.

Abstract: Observation of classroom interactions can provide concrete feedback to teachers, but current methods rely on manual annotation, which is resource-intensive and hard to scale. This work explores AI-driven analysis of classroom recordings, focusing on multimodal instructional activity and discourse recognition as a foundation for actionable feedback. Using a densely annotated dataset of 164 hours of video and 68 lesson transcripts, we design parallel, modality-specific pipelines. For video, we evaluate zero-shot multimodal LLMs, fine-tuned vision-language models, and self-supervised video transformers on 24 activity labels. For transcripts, we fine-tune a transformer-based classifier with contextualized inputs and compare it against prompting-based LLMs on 19 discourse labels. To handle class imbalance and multi-label complexity, we apply per-label thresholding, context windows, and imbalance-aware loss functions. The results show that fine-tuned models consistently outperform prompting-based approaches, achieving macro-F1 scores of 0.577 for video and 0.460 for transcripts. These results demonstrate the feasibility of automated classroom analysis and establish a foundation for scalable teacher feedback systems.

Method: Proposes Dual-Vision Adaptation (DVA) with: 1) Object-Perceptual Adaptation that modifies input samples to help pre-trained models perceive critical objects; 2) In-Context Adaptation that introduces small parameter sets for feature adaptation; 3) Discrimination Perception Transfer to balance efficiency and performance via knowledge distillation.

Result: DVA has fewer learnable parameters and performs well on three in-distribution and three out-of-distribution fine-grained datasets.

Conclusion: DVA effectively guides frozen pre-trained models for FGIR through collaborative sample and feature adaptation, maintaining pre-trained knowledge while achieving strong generalization.

Abstract: Fine-Grained Image Retrieval~(FGIR) faces challenges in learning discriminative visual representations to retrieve images with similar fine-grained features. Current leading FGIR solutions typically follow two regimes: enforce pairwise similarity constraints in the semantic embedding space, or incorporate a localization sub-network to fine-tune the entire model. However, such two regimes tend to overfit the training data while forgetting the knowledge gained from large-scale pre-training, thus reducing their generalization ability. In this paper, we propose a Dual-Vision Adaptation (DVA) approach for FGIR, which guides the frozen pre-trained model to perform FGIR through collaborative sample and feature adaptation. Specifically, we design Object-Perceptual Adaptation, which modifies input samples to help the pre-trained model perceive critical objects and elements within objects that are helpful for category prediction. Meanwhile, we propose In-Context Adaptation, which introduces a small set of parameters for feature adaptation without modifying the pre-trained parameters. This makes the FGIR task using these adjusted features closer to the task solved during the pre-training. Additionally, to balance retrieval efficiency and performance, we propose Discrimination Perception Transfer to transfer the discriminative knowledge in the object-perceptual adaptation to the image encoder using the knowledge distillation mechanism. Extensive experiments show that DVA has fewer learnable parameters and performs well on three in-distribution and three out-of-distribution fine-grained datasets.

Method: Represent each word as a pixel in a 2D image with rows as sentences and boundary rows between sentences. Map word embeddings to HSV color space where Hue encodes topic (with circular components), Saturation encodes sentiment, and Value encodes intensity. Use multi-task learning with auxiliary supervision on Hue and Saturation channels. Insert dynamic boundary rows to highlight semantic transitions.

Result: Achieves competitive or better accuracy than strong baselines (including BERT and hierarchical attention networks) on multi-label and single-label document classification benchmarks. Ablation studies confirm importance of multi-channel HSV representation and dynamic boundary rows.

Conclusion: SemImage provides an effective visual representation for text documents that enables competitive classification performance while offering enhanced interpretability through visible patterns of topic shifts and sentiment changes in the generated images.

Abstract: We propose SemImage, a novel method for representing a text document as a two-dimensional semantic image to be processed by convolutional neural networks (CNNs). In a SemImage, each word is represented as a pixel in a 2D image: rows correspond to sentences and an additional boundary row is inserted between sentences to mark semantic transitions. Each pixel is not a typical RGB value but a vector in a disentangled HSV color space, encoding different linguistic features: the Hue with two components H_cos and H_sin to account for circularity encodes the topic, Saturation encodes the sentiment, and Value encodes intensity or certainty. We enforce this disentanglement via a multi-task learning framework: a ColorMapper network maps each word embedding to the HSV space, and auxiliary supervision is applied to the Hue and Saturation channels to predict topic and sentiment labels, alongside the main task objective. The insertion of dynamically computed boundary rows between sentences yields sharp visual boundaries in the image when consecutive sentences are semantically dissimilar, effectively making paragraph breaks salient. We integrate SemImage with standard 2D CNNs (e.g., ResNet) for document classification. Experiments on multi-label datasets (with both topic and sentiment annotations) and single-label benchmarks demonstrate that SemImage can achieve competitive or better accuracy than strong text classification baselines (including BERT and hierarchical attention networks) while offering enhanced interpretability. An ablation study confirms the importance of the multi-channel HSV representation and the dynamic boundary rows. Finally, we present visualizations of SemImage that qualitatively reveal clear patterns corresponding to topic shifts and sentiment changes in the generated image, suggesting that our representation makes these linguistic features visible to both humans and machines.

Method: Dual-Modal Counterfactual Contrastive Construction (DMC³) framework with: 1) counterfactual sample construction module generating positive/negative samples via event description paraphrasing (text) and core interaction mining (visual), 2) feeding these with original samples into baseline, 3) contrastive optimization minimizing distance to positive samples and maximizing distance to negative samples.

Result: Achieves 52.51% on normal split and 46.04% on indirect split of EgoTaskQA, and 13.2% on QAEGO4D, reaching state-of-the-art performance.

Conclusion: The DMC³ framework effectively addresses first-person perspective challenges in Egocentric VideoQA through counterfactual contrastive learning, demonstrating superior performance on benchmark datasets.

Abstract: Egocentric Video Question Answering (Egocentric VideoQA) plays an important role in egocentric video understanding, which refers to answering questions based on first-person videos. Although existing methods have made progress through the paradigm of pre-training and fine-tuning, they ignore the unique challenges posed by the first-person perspective, such as understanding multiple events and recognizing hand-object interactions. To deal with these challenges, we propose a Dual-Modal Counterfactual Contrastive Construction (DMC$^3$) framework, which contains an egocentric videoqa baseline, a counterfactual sample construction module and a counterfactual sample-involved contrastive optimization. Specifically, We first develop a counterfactual sample construction module to generate positive and negative samples for textual and visual modalities through event description paraphrasing and core interaction mining, respectively. Then, We feed these samples together with the original samples into the baseline. Finally, in the counterfactual sample-involved contrastive optimization module, we apply contrastive loss to minimize the distance between the original sample features and the positive sample features, while maximizing the distance from the negative samples. Experiments show that our method achieve 52.51% and 46.04% on the \textit{normal} and \textit{indirect} splits of EgoTaskQA, and 13.2% on QAEGO4D, both reaching the state-of-the-art performance.

Method: Two-stage approach: 1) Graph Neural Networks process skeletal tracking data to capture nuanced motion patterns, 2) Transformer with specialized attention models player interactions. Event projection heads explicitly connect movements to basketball events.

Result: 35% reduction in trajectory prediction error vs position-only baselines; consistent performance gains across key basketball analytics tasks including pick detection, shot taker identification, assist prediction, shot location classification, and shot type recognition.

Conclusion: CourtMotion demonstrates that incorporating skeletal motion data with semantic understanding significantly improves basketball event prediction, providing a powerful pretrained foundation for multiple downstream analytics tasks.

Abstract: This paper presents CourtMotion, a spatiotemporal modeling framework for analyzing and predicting game events and plays as they develop in professional basketball. Anticipating basketball events requires understanding both physical motion patterns and their semantic significance in the context of the game. Traditional approaches that use only player positions fail to capture crucial indicators such as body orientation, defensive stance, or shooting preparation motions. Our two-stage approach first processes skeletal tracking data through Graph Neural Networks to capture nuanced motion patterns, then employs a Transformer architecture with specialized attention mechanisms to model player interactions. We introduce event projection heads that explicitly connect player movements to basketball events like passes, shots, and steals, training the model to associate physical motion patterns with their tactical purposes. Experiments on NBA tracking data demonstrate significant improvements over position-only baselines: 35% reduction in trajectory prediction error compared to state-of-the-art position-based models and consistent performance gains across key basketball analytics tasks. The resulting pretrained model serves as a powerful foundation for multiple downstream tasks, with pick detection, shot taker identification, assist prediction, shot location classification, and shot type recognition demonstrating substantial improvements over existing methods.

Method: TeleViT uses an asymmetric tokenization strategy to create heterogeneous tokens from (i) fine-scale local fire drivers, (ii) coarsened global fields, and (iii) teleconnection indices. These tokens are processed jointly by a transformer encoder, with a decoder that preserves spatial structure by mapping local tokens to prediction patches.

Result: TeleViT outperforms U-Net++, ViT, and climatology across all lead times (up to 4 months) on the global SeasFire dataset (2001-2021, 8-day resolution). At zero lead: AUPRC 0.630 vs ViT 0.617/U-Net 0.620. At 4-month lead: TeleViT maintains 0.601-0.603 vs ViT 0.582/U-Net 0.578. Best performance in African savannas, lower in boreal/arid regions.

Conclusion: Architectures that explicitly encode large-scale Earth-system context (global fields and teleconnections) can extend wildfire predictability on subseasonal-to-seasonal timescales. Predictions mainly rely on local tokens, with global fields and indices providing coarse contextual information.

Abstract: Forecasting wildfires weeks to months in advance is difficult, yet crucial for planning fuel treatments and allocating resources. While short-term predictions typically rely on local weather conditions, long-term forecasting requires accounting for the Earth’s interconnectedness, including global patterns and teleconnections. We introduce TeleViT, a Teleconnection-aware Vision Transformer that integrates (i) fine-scale local fire drivers, (ii) coarsened global fields, and (iii) teleconnection indices. This multi-scale fusion is achieved through an asymmetric tokenization strategy that produces heterogeneous tokens processed jointly by a transformer encoder, followed by a decoder that preserves spatial structure by mapping local tokens to their corresponding prediction patches. Using the global SeasFire dataset (2001-2021, 8-day resolution), TeleViT improves AUPRC performance over U-Net++, ViT, and climatology across all lead times, including horizons up to four months. At zero lead, TeleViT with indices and global inputs reaches AUPRC 0.630 (ViT 0.617, U-Net 0.620), at 16x8day lead (around 4 months), TeleViT variants using global input maintain 0.601-0.603 (ViT 0.582, U-Net 0.578), while surpassing the climatology (0.572) at all lead times. Regional results show the highest skill in seasonally consistent fire regimes, such as African savannas, and lower skill in boreal and arid regions. Attention and attribution analyses indicate that predictions rely mainly on local tokens, with global fields and indices contributing coarse contextual information. These findings suggest that architectures explicitly encoding large-scale Earth-system context can extend wildfire predictability on subseasonal-to-seasonal timescales.

Method: Incorporates significance-aware and granularity-aware modeling using modality-specific biases to identify salient features without brittle cross-modal attention, and represents region features as a mixture of Gaussian distributions to capture fine-grained uncertainty.

Result: Achieves state-of-the-art performance on Flickr30K and MS-COCO datasets across various backbone architectures, significantly enhancing robustness and interpretability of fine-grained image-text alignment.

Conclusion: The proposed unified approach effectively addresses fundamental limitations in fine-grained image-text alignment through significance-aware modeling and uncertainty modeling, demonstrating superior performance and improved interpretability for multimodal applications.

Abstract: Fine-grained image-text alignment is a pivotal challenge in multimodal learning, underpinning key applications such as visual question answering, image captioning, and vision-language navigation. Unlike global alignment, fine-grained alignment requires precise correspondence between localized visual regions and textual tokens, often hindered by noisy attention mechanisms and oversimplified modeling of cross-modal relationships. In this work, we identify two fundamental limitations of existing approaches: the lack of robust intra-modal mechanisms to assess the significance of visual and textual tokens, leading to poor generalization in complex scenes; and the absence of fine-grained uncertainty modeling, which fails to capture the one-to-many and many-to-one nature of region-word correspondences. To address these issues, we propose a unified approach that incorporates significance-aware and granularity-aware modeling and region-level uncertainty modeling. Our method leverages modality-specific biases to identify salient features without relying on brittle cross-modal attention, and represents region features as a mixture of Gaussian distributions to capture fine-grained uncertainty. Extensive experiments on Flickr30K and MS-COCO demonstrate that our approach achieves state-of-the-art performance across various backbone architectures, significantly enhancing the robustness and interpretability of fine-grained image-text alignment.

Method: Developed an automated QC procedure that works with multiple deposition methods (extrusion-based and SC3DP), is sensor-agnostic (camera, SLS, TLS), and handles both fresh and cured materials.

Result: Created a workflow for filament geometry quality control that supports both online (fresh material) and post-printing (cured material) QC, independent of specific data acquisition sensors.

Conclusion: The automated QC method enables reliable quality assessment for 3D concrete printing filaments across different printing techniques and material states, supporting both real-time and post-construction quality control.

Abstract: The architecture, engineering and construction (AEC) industry is constantly evolving to meet the demand for sustainable and effective design and construction of the built environment. In the literature, two primary deposition techniques for large-scale 3D concrete printing (3DCP) have been described, namely extrusion-based (Contour Crafting-CC) and shotcrete 3D printing (SC3DP) methods. The deposition methods use a digitally controlled nozzle to print material layer by layer. The continuous flow of concrete material used to create the printed structure is called a filament or layer. As these filaments are the essential structure defining the printed object, the filaments’ geometry quality control is crucial. This paper presents an automated procedure for quality control (QC) of filaments in extrusion-based and SC3DP printing methods. The paper also describes a workflow that is independent of the sensor used for data acquisition, such as a camera, a structured light system (SLS) or a terrestrial laser scanner (TLS). This method can be used with materials in either the fresh or cured state. Thus, it can be used for online and post-printing QC.

Method: Uses a controlled generation-based pipeline leveraging recent video generation models. Integrates scene-conditioned anomaly assignment, multi-step storyline generation, and temporally consistent long-form synthesis to produce coherent 41-second videos with minimal human intervention.

Result: Creates a benchmark with scale, diversity, and complexity that reveals new challenges for existing methods. Demonstrates the ability to generate controlled, bias-free datasets that better assess real-world performance.

Conclusion: Pistachio provides a superior benchmark for VAD/VAU research by overcoming limitations of traditional datasets through AI-generated content, enabling more reliable assessment and motivating future work on dynamic, multi-event anomaly understanding.

Abstract: Automatically detecting abnormal events in videos is crucial for modern autonomous systems, yet existing Video Anomaly Detection (VAD) benchmarks lack the scene diversity, balanced anomaly coverage, and temporal complexity needed to reliably assess real-world performance. Meanwhile, the community is increasingly moving toward Video Anomaly Understanding (VAU), which requires deeper semantic and causal reasoning but remains difficult to benchmark due to the heavy manual annotation effort it demands. In this paper, we introduce Pistachio, a new VAD/VAU benchmark constructed entirely through a controlled, generation-based pipeline. By leveraging recent advances in video generation models, Pistachio provides precise control over scenes, anomaly types, and temporal narratives, effectively eliminating the biases and limitations of Internet-collected datasets. Our pipeline integrates scene-conditioned anomaly assignment, multi-step storyline generation, and a temporally consistent long-form synthesis strategy that produces coherent 41-second videos with minimal human intervention. Extensive experiments demonstrate the scale, diversity, and complexity of Pistachio, revealing new challenges for existing methods and motivating future research on dynamic and multi-event anomaly understanding.

Method: Converted wrist-worn activity signals from Psykose and Depresjon datasets into 30x48 pixel images, then evaluated three models (VGG16, ViT-B/16, CoAtNet-Tiny) using three-fold subject-wise cross-validation on the classification task.

Result: CoAtNet-Tiny achieved the highest average accuracy, most stable performance across folds, and best precision/recall/F1-scores, especially for underrepresented depression and schizophrenia classes. VGG16 showed steady improvement but lower accuracy, while ViT-B/16 had strong but inconsistent performance.

Conclusion: CoAtNet-Tiny is the most reliable method for mental health classification using actigraphy-derived images, suggesting hybrid architectures may be particularly suitable for this application domain.

Abstract: This work examines how three different image-based methods, VGG16, ViT-B/16, and CoAtNet-Tiny, perform in identifying depression, schizophrenia, and healthy controls using daily actigraphy records. Wrist-worn activity signals from the Psykose and Depresjon datasets were converted into 30 by 48 images and evaluated through a three-fold subject-wise split. Although all methods fitted the training data well, their behaviour on unseen data differed. VGG16 improved steadily but often settled at lower accuracy. ViT-B/16 reached strong results in some runs, but its performance shifted noticeably from fold to fold. CoAtNet-Tiny stood out as the most reliable, recording the highest average accuracy and the most stable curves across folds. It also produced the strongest precision, recall, and F1-scores, particularly for the underrepresented depression and schizophrenia classes. Overall, the findings indicate that CoAtNet-Tiny performed most consistently on the actigraphy images, while VGG16 and ViT-B/16 yielded mixed results. These observations suggest that certain hybrid designs may be especially suited for mental-health work that relies on actigraphy-derived images.

Method: Hybrid SDG framework combining simulation-based rendering, domain randomization, and real background compositing. Two-stage architecture with YOLOv8n for detection and MobileNetV3-small for classification, trained exclusively on synthetic data.

Result: Achieves mAP@0.5 of 0.995 for detection, 96% classification accuracy, and 90.1% balanced accuracy. Outperforms few-shot real-data baselines (90-91% vs 50% balanced accuracy under severe imbalance).

Conclusion: The proposed SDG-based approach enables annotation-free, scalable, and robust quality inspection for real-world manufacturing applications, overcoming data constraints through synthetic data generation.

Abstract: Machine learning, particularly deep learning, is transforming industrial quality inspection. Yet, training robust machine learning models typically requires large volumes of high-quality labeled data, which are expensive, time-consuming, and labor-intensive to obtain in manufacturing. Moreover, defective samples are intrinsically rare, leading to severe class imbalance that degrades model performance. These data constraints hinder the widespread adoption of machine learning-based quality inspection methods in real production environments. Synthetic data generation (SDG) offers a promising solution by enabling the creation of large, balanced, and fully annotated datasets in an efficient, cost-effective, and scalable manner. This paper presents a hybrid SDG framework that integrates simulation-based rendering, domain randomization, and real background compositing to enable zero-shot learning for computer vision-based industrial part inspection without manual annotation. The SDG pipeline generates 12,960 labeled images in one hour by varying part geometry, lighting, and surface properties, and then compositing synthetic parts onto real image backgrounds. A two-stage architecture utilizing a YOLOv8n backbone for object detection and MobileNetV3-small for quality classification is trained exclusively on synthetic data and evaluated on 300 real industrial parts. The proposed approach achieves an mAP@0.5 of 0.995 for detection, 96% classification accuracy, and 90.1% balanced accuracy. Comparative evaluation against few-shot real-data baseline approaches demonstrates significant improvement. The proposed SDG-based approach achieves 90-91% balanced accuracy under severe class imbalance, while the baselines reach only 50% accuracy. These results demonstrate that the proposed method enables annotation-free, scalable, and robust quality inspection for real-world manufacturing applications.

Method: Integrated approach using ResNet50-based OOD filtering (selected after 12 CNN architecture searches) to reject non-mammographic inputs via cosine similarity, combined with YOLO architectures (YOLOv8, YOLOv11, YOLOv12) for breast cancer detection. The system establishes an in-domain gallery to ensure only domain-associated images enter the detection pipeline.

Result: OOD detection achieves 99.77% general accuracy with perfect 100% accuracy on OOD test sets. The joint framework achieves high detection performance (mAP@0.5: 0.947) with enhanced interpretability via Grad-CAM visualizations. OOD filtering significantly improves system reliability by preventing false alarms on out-of-distribution inputs.

Conclusion: The study provides a fundamental foundation for deploying reliable AI-based breast cancer detection systems in diverse clinical environments with inherent data heterogeneity, addressing the critical OOD problem while maintaining high detection accuracy on mammographic data.

Abstract: Deep learning models for breast cancer detection from mammographic images have significant reliability problems when presented with Out-of-Distribution (OOD) inputs such as other imaging modalities (CT, MRI, X-ray) or equipment variations, leading to unreliable detection and misdiagnosis. The current research mitigates the fundamental OOD issue through a comprehensive approach integrating ResNet50-based OOD filtering with YOLO architectures (YOLOv8, YOLOv11, YOLOv12) for accurate detection of breast cancer. Our strategy establishes an in-domain gallery via cosine similarity to rigidly reject non-mammographic inputs prior to processing, ensuring that only domain-associated images supply the detection pipeline. The OOD detection component achieves 99.77% general accuracy with immaculate 100% accuracy on OOD test sets, effectively eliminating irrelevant imaging modalities. ResNet50 was selected as the optimum backbone after 12 CNN architecture searches. The joint framework unites OOD robustness with high detection performance (mAP@0.5: 0.947) and enhanced interpretability through Grad-CAM visualizations. Experimental validation establishes that OOD filtering significantly improves system reliability by preventing false alarms on out-of-distribution inputs while maintaining higher detection accuracy on mammographic data. The present study offers a fundamental foundation for the deployment of reliable AI-based breast cancer detection systems in diverse clinical environments with inherent data heterogeneity.

Method: Proposes LGCOAMix: an efficient context-aware and object-part-aware superpixel-based grid blending method. Uses superpixel attention approach for label mixing, learns local features from discriminative superpixel regions and cross-image superpixel contrasts.

Result: Outperforms state-of-the-art CutMix methods on classification tasks and weakly supervised object location on CUB200-2011. Effective for both CNN and Transformer networks.

Conclusion: LGCOAMix successfully addresses limitations of existing CutMix methods by preserving local discriminative features through superpixel attention, achieving better performance while being more efficient.

Abstract: Cutmix-based data augmentation, which uses a cut-and-paste strategy, has shown remarkable generalization capabilities in deep learning. However, existing methods primarily consider global semantics with image-level constraints, which excessively reduces attention to the discriminative local context of the class and leads to a performance improvement bottleneck. Moreover, existing methods for generating augmented samples usually involve cutting and pasting rectangular or square regions, resulting in a loss of object part information. To mitigate the problem of inconsistency between the augmented image and the generated mixed label, existing methods usually require double forward propagation or rely on an external pre-trained network for object centering, which is inefficient. To overcome the above limitations, we propose LGCOAMix, an efficient context-aware and object-part-aware superpixel-based grid blending method for data augmentation. To the best of our knowledge, this is the first time that a label mixing strategy using a superpixel attention approach has been proposed for cutmix-based data augmentation. It is the first instance of learning local features from discriminative superpixel-wise regions and cross-image superpixel contrasts. Extensive experiments on various benchmark datasets show that LGCOAMix outperforms state-of-the-art cutmix-based data augmentation methods on classification tasks, {and weakly supervised object location on CUB200-2011.} We have demonstrated the effectiveness of LGCOAMix not only for CNN networks, but also for Transformer networks. Source codes are available at https://github.com/DanielaPlusPlus/LGCOAMix.

Method: Developed Mammo-FM, a domain-specific foundation model pretrained on the largest and most diverse mammography dataset to date (821,326 mammograms from 140,677 patients across 4 US institutions). The model aligns images with text for visual and textual interpretability, enabling unified handling of diagnosis, pathology localization, report generation, and risk prognosis within a single framework.

Result: Mammo-FM consistently outperforms state-of-the-art generalist foundation models across multiple public and private benchmarks for diagnosis, prognosis, and report generation, despite using only one-third of the parameters and operating on native-resolution mammograms. The model demonstrates superior performance in both in-distribution and out-of-distribution datasets.

Conclusion: Domain-specific foundation models like Mammo-FM designed around the full spectrum of clinical tasks offer greater efficiency and value than generalist models. The results emphasize the importance of domain-aligned evaluation and specialized models for real-world clinical adoption, particularly for improving transparency and auditability in breast cancer screening.

Abstract: Breast cancer is one of the leading causes of death among women worldwide. We introduce Mammo-FM, the first foundation model specifically for mammography, pretrained on the largest and most diverse dataset to date - 140,677 patients (821,326 mammograms) across four U.S. institutions. Mammo-FM provides a unified foundation for core clinical tasks in breast imaging, including cancer diagnosis, pathology localization, structured report generation, and cancer risk prognosis within a single framework. Its alignment between images and text enables both visual and textual interpretability, improving transparency and clinical auditability, which are essential for real-world adoption. We rigorously evaluate Mammo-FM across diagnosis, prognosis, and report-generation tasks in in- and out-of-distribution datasets. Despite operating on native-resolution mammograms and using only one-third of the parameters of state-of-the-art generalist FMs, Mammo-FM consistently outperforms them across multiple public and private benchmarks. These results highlight the efficiency and value of domain-specific foundation models designed around the full spectrum of tasks within a clinical domain and emphasize the importance of rigorous, domain-aligned evaluation.

Method: ReactionMamba combines a motion VAE for efficient motion encoding with Mamba-based state-space models for decoding, enabling generation of both short simple motions and long complex motion sequences while maintaining temporal consistency.

Result: The framework demonstrates competitive performance on NTU120-AS, Lindy Hop, and InterX datasets in terms of realism, diversity, and long-sequence generation compared to InterFormer, ReMoS, and Ready-to-React, with substantial improvements in inference speed.

Conclusion: ReactionMamba provides an effective solution for generating long 3D human reaction motions with improved computational efficiency, making it suitable for applications requiring complex motion generation like dance and martial arts.

Abstract: We present ReactionMamba, a novel framework for generating long 3D human reaction motions. Reaction-Mamba integrates a motion VAE for efficient motion encoding with Mamba-based state-space models to decode temporally consistent reactions. This design enables ReactionMamba to generate both short sequences of simple motions and long sequences of complex motions, such as dance and martial arts. We evaluate ReactionMamba on three datasets–NTU120-AS, Lindy Hop, and InterX–and demonstrate competitive performance in terms of realism, diversity, and long-sequence generation compared to previous methods, including InterFormer, ReMoS, and Ready-to-React, while achieving substantial improvements in inference speed.

Method: Automated pipeline leveraging multi-view 2D images and multimodal large language models (MLLMs) to generate dense captions of scene elements, followed by scenario-based question generation that integrates object properties, spatial relationships, and scene context.

Result: DenseScan significantly enhances object-level understanding and question-answering performance in 3D environments compared to traditional annotation pipelines, providing comprehensive annotations that couple geometric detail with semantic richness.

Conclusion: DenseScan broadens the range of downstream 3D understanding tasks and aims to catalyze new research avenues in robotics, augmented reality, and other applications by providing richer, more contextually aware annotations for real-world environments.

Abstract: 3D understanding is a key capability for real-world AI assistance. High-quality data plays an important role in driving the development of the 3D understanding community. Current 3D scene understanding datasets often provide geometric and instance-level information, yet they lack the rich semantic annotations necessary for nuanced visual-language tasks.In this work, we introduce DenseScan, a novel dataset with detailed multi-level descriptions generated by an automated pipeline leveraging multi-view 2D images and multimodal large language models (MLLMs). Our approach enables dense captioning of scene elements, ensuring comprehensive object-level descriptions that capture context-sensitive details. Furthermore, we extend these annotations through scenario-based question generation, producing high-level queries that integrate object properties, spatial relationships, and scene context. By coupling geometric detail with semantic richness, DenseScan broadens the range of downstream tasks, from detailed visual-language navigation to interactive question answering. Experimental results demonstrate that our method significantly enhances object-level understanding and question-answering performance in 3D environments compared to traditional annotation pipelines. We release both the annotated dataset and our annotation pipeline to facilitate future research and applications in robotics, augmented reality, and beyond. Through DenseScan, we aim to catalyze new avenues in 3D scene understanding, allowing researchers and practitioners to tackle the complexities of real-world environments with richer, more contextually aware annotations.

Method: 1) New capture strategy with alternating random environment maps and uniformly lit tracking frames. 2) Physics-informed features encoding geometry, albedo, shading, and camera view from coarse mesh proxy. 3) Transformer-based RelightNet uses cross-attention with novel lighting conditions to regress relit appearance as texel-aligned 3D Gaussian splats.

Result: Superior visual fidelity and lighting reproduction compared to state-of-the-art approaches, enabling efficient single-pass relighting of highly dynamic humans.

Conclusion: RHC demonstrates a practical approach for free-view rendering and relighting of dynamic humans from sparse views, eliminating the need for costly OLAT capture while maintaining high-quality results.

Abstract: We present Relightable Holoported Characters (RHC), a novel person-specific method for free-view rendering and relighting of full-body and highly dynamic humans solely observed from sparse-view RGB videos at inference. In contrast to classical one-light-at-a-time (OLAT)-based human relighting, our transformer-based RelightNet predicts relit appearance within a single network pass, avoiding costly OLAT-basis capture and generation. For training such a model, we introduce a new capture strategy and dataset recorded in a multi-view lightstage, where we alternate frames lit by random environment maps with uniformly lit tracking frames, simultaneously enabling accurate motion tracking and diverse illumination as well as dynamics coverage. Inspired by the rendering equation, we derive physics-informed features that encode geometry, albedo, shading, and the virtual camera view from a coarse human mesh proxy and the input views. Our RelightNet then takes these features as input and cross-attends them with a novel lighting condition, and regresses the relit appearance in the form of texel-aligned 3D Gaussian splats attached to the coarse mesh proxy. Consequently, our RelightNet implicitly learns to efficiently compute the rendering equation for novel lighting conditions within a single feed-forward pass. Experiments demonstrate our method’s superior visual fidelity and lighting reproduction compared to state-of-the-art approaches. Project page: https://vcai.mpi-inf.mpg.de/projects/RHC/

Method: UniDiff framework combines: 1) FiLM-based timestep-modality conditioning, 2) Parameter-efficient adaptation of ~5% of parameters, and 3) Pseudo-RGB anchoring to preserve pre-trained representations and prevent catastrophic forgetting. Adapts a single ImageNet-pretrained diffusion model to multiple sensing modalities using only target-domain data.

Result: Demonstrated on two established multi-modal benchmarking datasets. Shows that unsupervised adaptation of pre-trained diffusion model effectively mitigates annotation constraints and achieves effective fusion of multi-modal remotely sensed data.

Conclusion: UniDiff enables effective feature extraction from remote sensing data under sparse annotations by adapting pre-trained diffusion models to heterogeneous modalities without requiring large labeled datasets, addressing practical deployment limitations of supervised methods.

Abstract: Sparse annotations fundamentally constrain multimodal remote sensing: even recent state-of-the-art supervised methods such as MSFMamba are limited by the availability of labeled data, restricting their practical deployment despite architectural advances. ImageNet-pretrained models provide rich visual representations, but adapting them to heterogeneous modalities such as hyperspectral imaging (HSI) and synthetic aperture radar (SAR) without large labeled datasets remains challenging. We propose UniDiff, a parameter-efficient framework that adapts a single ImageNet-pretrained diffusion model to multiple sensing modalities using only target-domain data. UniDiff combines FiLM-based timestep-modality conditioning, parameter-efficient adaptation of approximately 5% of parameters, and pseudo-RGB anchoring to preserve pre-trained representations and prevent catastrophic forgetting. This design enables effective feature extraction from remote sensing data under sparse annotations. Our results with two established multi-modal benchmarking datasets demonstrate that unsupervised adaptation of a pre-trained diffusion model effectively mitigates annotation constraints and achieves effective fusion of multi-modal remotely sensed data.

Method: HeartFormer: Novel point cloud completion network with two components: 1) Semantic-Aware Dual-Structure Transformer Network (SA-DSTNet) generates initial coarse point cloud with global and substructure geometry features, 2) Semantic-Aware Geometry Feature Refinement Transformer Network (SA-GFRTNet) progressively refines coarse output using semantic-geometry representations.

Result: HeartFormer achieves robust, accurate, and generalizable performance, consistently surpassing state-of-the-art methods in cross-domain experiments on HeartCompv1 (new dataset) and UK Biobank. Created HeartCompv1 - first publicly available large-scale dataset with 17,000 high-resolution 3D multi-class cardiac meshes and point-clouds.

Conclusion: HeartFormer provides the first geometric deep learning framework for 3D cardiac reconstruction from cine MRI, addressing limitations of 2D imaging and establishing a benchmark for this emerging research direction with promising clinical applications.

Abstract: We present the first geometric deep learning framework based on point cloud representation for 3D four-chamber cardiac reconstruction from cine MRI data. This work addresses a long-standing limitation in conventional cine MRI, which typically provides only 2D slice images of the heart, thereby restricting a comprehensive understanding of cardiac morphology and physiological mechanisms in both healthy and pathological conditions. To overcome this, we propose \textbf{HeartFormer}, a novel point cloud completion network that extends traditional single-class point cloud completion to the multi-class. HeartFormer consists of two key components: a Semantic-Aware Dual-Structure Transformer Network (SA-DSTNet) and a Semantic-Aware Geometry Feature Refinement Transformer Network (SA-GFRTNet). SA-DSTNet generates an initial coarse point cloud with both global geometry features and substructure geometry features. Guided by these semantic-geometry representations, SA-GFRTNet progressively refines the coarse output, effectively leveraging both global and substructure geometric priors to produce high-fidelity and geometrically consistent reconstructions. We further construct \textbf{HeartCompv1}, the first publicly available large-scale dataset with 17,000 high-resolution 3D multi-class cardiac meshes and point-clouds, to establish a general benchmark for this emerging research direction. Extensive cross-domain experiments on HeartCompv1 and UK Biobank demonstrate that HeartFormer achieves robust, accurate, and generalizable performance, consistently surpassing state-of-the-art (SOTA) methods. Code and dataset will be released upon acceptance at: https://github.com/10Darren/HeartFormer.

Method: USB uses a paired diffusion mechanism to model the joint distribution of lesions and brain anatomy, with a consistency guidance algorithm to preserve anatomical consistency and lesion correspondence during bidirectional editing.

Result: Extensive experiments on six public brain MRI datasets (healthy controls, stroke, Alzheimer’s patients) demonstrate USB’s ability to produce diverse and realistic results.

Conclusion: USB establishes the first unified benchmark for brain image generation and editing, opening opportunities for scalable dataset creation and robust neuroimaging analysis.

Abstract: Understanding the relationship between pathological and healthy brain structures is fundamental to neuroimaging, connecting disease diagnosis and detection with modeling, prediction, and treatment planning. However, paired pathological-healthy data are extremely difficult to obtain, as they rely on pre- and post-treatment imaging, constrained by clinical outcomes and longitudinal data availability. Consequently, most existing brain image generation and editing methods focus on visual quality yet remain domain-specific, treating pathological and healthy image modeling independently. We introduce USB (Unified Synthetic Brain), the first end-to-end framework that unifies bidirectional generation and editing of pathological and healthy brain images. USB models the joint distribution of lesions and brain anatomy through a paired diffusion mechanism and achieves both pathological and healthy image generation. A consistency guidance algorithm further preserves anatomical consistency and lesion correspondence during bidirectional pathology-healthy editing. Extensive experiments on six public brain MRI datasets including healthy controls, stroke, and Alzheimer’s patients, demonstrate USB’s ability to produce diverse and realistic results. By establishing the first unified benchmark for brain image generation and editing, USB opens opportunities for scalable dataset creation and robust neuroimaging analysis. Code is available at https://github.com/jhuldr/USB.

Method: Proposes HIMOSA framework that leverages inherent redundancy in remote sensing imagery with content-aware sparse attention mechanism. Uses hierarchical window expansion to exploit multi-scale repetitive patterns while reducing computational complexity through attention sparsity adjustment.

Result: Extensive experiments on multiple remote sensing datasets show the method achieves state-of-the-art performance while maintaining computational efficiency.

Conclusion: HIMOSA successfully addresses the performance-efficiency trade-off in remote sensing super-resolution through innovative sparse attention and hierarchical processing, enabling fast inference with strong reconstruction capabilities.

Abstract: In remote sensing applications, such as disaster detection and response, real-time efficiency and model lightweighting are of critical importance. Consequently, existing remote sensing image super-resolution methods often face a trade-off between model performance and computational efficiency. In this paper, we propose a lightweight super-resolution framework for remote sensing imagery, named HIMOSA. Specifically, HIMOSA leverages the inherent redundancy in remote sensing imagery and introduces a content-aware sparse attention mechanism, enabling the model to achieve fast inference while maintaining strong reconstruction performance. Furthermore, to effectively leverage the multi-scale repetitive patterns found in remote sensing imagery, we introduce a hierarchical window expansion and reduce the computational complexity by adjusting the sparsity of the attention. Extensive experiments on multiple remote sensing datasets demonstrate that our method achieves state-of-the-art performance while maintaining computational efficiency.

Method: U-FaceBP uses Bayesian neural networks to model aleatoric and epistemic uncertainties, and employs ensemble learning to estimate BP from rPPG signals, derived PPG signals, and face images using multiple BNNs.

Result: Outperforms state-of-the-art BP estimation methods on two datasets with 1197 subjects from diverse racial groups. Uncertainty estimates are informative for modality fusion, reliability assessment, and racial group analysis.

Conclusion: U-FaceBP provides a reliable, uncertainty-aware approach for face video-based BP measurement with practical applications in healthcare monitoring.

Abstract: Blood pressure (BP) measurement is crucial for daily health assessment. Remote photoplethysmography (rPPG), which extracts pulse waves from face videos captured by a camera, has the potential to enable convenient BP measurement without specialized medical devices. However, there are various uncertainties in BP estimation using rPPG, leading to limited estimation performance and reliability. In this paper, we propose U-FaceBP, an uncertainty-aware Bayesian ensemble deep learning method for face video-based BP measurement. U-FaceBP models aleatoric and epistemic uncertainties in face video-based BP estimation with a Bayesian neural network (BNN). Additionally, we design U-FaceBP as an ensemble method, estimating BP from rPPG signals, PPG signals derived from face videos, and face images using multiple BNNs. Large-scale experiments on two datasets involving 1197 subjects from diverse racial groups demonstrate that U-FaceBP outperforms state-of-the-art BP estimation methods. Furthermore, we show that the uncertainty estimates provided by U-FaceBP are informative and useful for guiding modality fusion, assessing prediction reliability, and analyzing performance across racial groups.

Method: Developed an ensemble system combining shallow deep learning models with feature-based specialized models. Trained on 25,709 CT scans with 69,449 annotated nodules to address dataset scale and explainability limitations.

Result: Achieved AUC of 0.98 internally and 0.945 on independent cohort. Outperformed radiologists, Lung-RADS, and leading AI models (Sybil, Brock, Google, Kaggle) with 0.5 false positives per scan at 99.3% sensitivity. Excelled across all nodule sizes, stages (especially stage 1), and growth metrics including Volume-Doubling Time.

Conclusion: The AI system addresses key barriers to AI adoption in lung cancer screening by providing earlier and more accurate malignancy diagnosis than current methods, potentially enabling earlier intervention and improved patient outcomes.

Abstract: Early detection of malignant lung nodules is critical, but its dependence on size and growth in screening inherently delays diagnosis. We present an AI system that redefines lung cancer screening by performing both detection and malignancy diagnosis directly at the nodule level on low-dose CT scans. To address limitations in dataset scale and explainability, we designed an ensemble of shallow deep learning and feature-based specialized models. Trained and evaluated on 25,709 scans with 69,449 annotated nodules, the system outperforms radiologists, Lung-RADS, and leading AI models (Sybil, Brock, Google, Kaggle). It achieves an area under the receiver operating characteristic curve (AUC) of 0.98 internally and 0.945 on an independent cohort. With 0.5 false positives per scan at 99.3% sensitivity, it addresses key barriers to AI adoption. Critically, it outperforms radiologists across all nodule sizes and stages, excelling in stage 1 cancers, and all growth-based metrics, including the least accurate: Volume-Doubling Time. It also surpasses radiologists by up to one year in diagnosing indeterminate and slow-growing nodules.

Method: Created a novel benchmark dataset composed of Sentinel-2 satellite imagery captured over multiple months, with pixel-level annotations of agricultural potentials for three major crop types (viticulture, market gardening, field crops) across five ordinal classes.

Result: AgriPotential is the first public dataset specifically designed for agricultural potential prediction, covering diverse areas in Southern France with rich spectral information, supporting multiple machine learning tasks including ordinal regression, multi-label classification, and spatio-temporal modeling.

Conclusion: AgriPotential aims to improve data-driven approaches to sustainable land use planning by providing a comprehensive benchmark for agricultural potential prediction, with the dataset and code freely accessible to the research community.

Abstract: Remote sensing has emerged as a critical tool for large-scale Earth monitoring and land management. In this paper, we introduce AgriPotential, a novel benchmark dataset composed of Sentinel-2 satellite imagery captured over multiple months. The dataset provides pixel-level annotations of agricultural potentials for three major crop types - viticulture, market gardening, and field crops - across five ordinal classes. AgriPotential supports a broad range of machine learning tasks, including ordinal regression, multi-label classification, and spatio-temporal modeling. The data cover diverse areas in Southern France, offering rich spectral information. AgriPotential is the first public dataset designed specifically for agricultural potential prediction, aiming to improve data-driven approaches to sustainable land use planning. The dataset and the code are freely accessible at: https://zenodo.org/records/15551829

Method: A modular AR agent system that coordinates multimodal large language models (MLLMs) with coordinate-aware perception tools. It constructs dynamic AR scene graphs encoding nine typed relations (spatial, structural-semantic, causal-functional), uses task-adaptive region-of-interest highlighting, and supports human-in-the-loop refinement. The system dynamically invokes coordinate-aware tools for complex queries like selection, measurement, comparison, and actuation.

Result: The system enables relational reasoning in space and language-conditioned spatial retrieval, handles varying query complexities from simple object identification to multi-object relational reasoning, and returns meter-accurate 3D anchors. It also introduces GroundedAR-Bench, an evaluation framework for language-driven real-world localization and relation grounding.

Conclusion: The modular architecture establishes AR agents as intermediaries that augment MLLMs with real-world spatial intelligence for interactive scene understanding, supporting plug-and-use vision-language models without retraining, bridging language understanding with physical operations.

Abstract: Traditional augmented reality (AR) systems predominantly rely on fixed class detectors or fiducial markers, limiting their ability to interpret complex, open-vocabulary natural language queries. We present a modular AR agent system that integrates multimodal large language models (MLLMs) with grounded vision models to enable relational reasoning in space and language-conditioned spatial retrieval in physical environments. Our adaptive task agent coordinates MLLMs and coordinate-aware perception tools to address varying query complexities, ranging from simple object identification to multi-object relational reasoning, while returning meter-accurate 3D anchors. It constructs dynamic AR scene graphs encoding nine typed relations (spatial, structural-semantic, causal-functional), enabling MLLMs to understand not just what objects exist, but how they relate and interact in 3D space. Through task-adaptive region-of-interest highlighting and contextual spatial retrieval, the system guides human attention to information-dense areas while supporting human-in-the-loop refinement. The agent dynamically invokes coordinate-aware tools for complex queries-selection, measurement, comparison, and actuation-grounding language understanding in physical operations. The modular architecture supports plug-and-use vision-language models without retraining, establishing AR agents as intermediaries that augment MLLMs with real-world spatial intelligence for interactive scene understanding. We also introduce GroundedAR-Bench, an evaluation framework for language-driven real world localization and relation grounding across diverse environments.

Method: Introduces BASIC framework with: 1) hierarchical segmentation-based metrics (foreground, semantic, instance, component masks) for structural fidelity, 2) structured scene representations (objects, attributes, relationships) using multimodal LLMs for semantic alignment, and 3) benchmarking across multiple stimulus-neuroimaging datasets.

Result: The framework provides more discriminative, interpretable, and comprehensive evaluation of brain visual decoding methods, enabling detailed comparisons across different granularities of visual information.

Conclusion: BASIC establishes a unified, multigranular evaluation foundation that addresses limitations of existing protocols and enables more meaningful assessment of brain visual decoding methods.

Abstract: Existing evaluation protocols for brain visual decoding predominantly rely on coarse metrics that obscure inter-model differences, lack neuroscientific foundation, and fail to capture fine-grained visual distinctions. To address these limitations, we introduce BASIC, a unified, multigranular evaluation framework that jointly quantifies structural fidelity, inferential alignment, and contextual coherence between decoded and ground-truth images. For the structural level, we introduce a hierarchical suite of segmentation-based metrics, including foreground, semantic, instance, and component masks, anchored in granularity-aware correspondence across mask structures. For the semantic level, we extract structured scene representations encompassing objects, attributes, and relationships using multimodal large language models, enabling detailed, scalable, and context-rich comparisons with ground-truth stimuli. We benchmark a diverse set of visual decoding methods across multiple stimulus-neuroimaging datasets within this unified evaluation framework. Together, these criteria provide a more discriminative, interpretable, and comprehensive foundation for evaluating brain visual decoding methods.

Method: TGSFormer maintains a persistent Gaussian memory for temporal prediction without relying on image coherence or frame caches. It uses a Dual Temporal Encoder with confidence-aware cross-attention to process current and historical Gaussian features, followed by a Confidence-aware Voxel Fusion module that merges overlapping primitives into voxel-aligned representations while regulating density and maintaining compactness.

Result: Extensive experiments show TGSFormer achieves state-of-the-art results on both local and embodied SSC benchmarks, offering superior accuracy and scalability with significantly fewer primitives while maintaining consistent long-term scene integrity.

Conclusion: TGSFormer provides an effective solution for scalable embodied 3D Semantic Scene Completion that overcomes the limitations of existing Gaussian-based methods, achieving better performance with reduced computational overhead and improved handling of unbounded scenes.

Abstract: Embodied 3D Semantic Scene Completion (SSC) infers dense geometry and semantics from continuous egocentric observations. Most existing Gaussian-based methods rely on random initialization of many primitives within predefined spatial bounds, resulting in redundancy and poor scalability to unbounded scenes. Recent depth-guided approach alleviates this issue but remains local, suffering from latency and memory overhead as scale increases. To overcome these challenges, we propose TGSFormer, a scalable Temporal Gaussian Splatting framework for embodied SSC. It maintains a persistent Gaussian memory for temporal prediction, without relying on image coherence or frame caches. For temporal fusion, a Dual Temporal Encoder jointly processes current and historical Gaussian features through confidence-aware cross-attention. Subsequently, a Confidence-aware Voxel Fusion module merges overlapping primitives into voxel-aligned representations, regulating density and maintaining compactness. Extensive experiments demonstrate that TGSFormer achieves state-of-the-art results on both local and embodied SSC benchmarks, offering superior accuracy and scalability with significantly fewer primitives while maintaining consistent long-term scene integrity. The code will be released upon acceptance.

Method: Reformulates dataset distillation as an Optimal Transport distance minimization problem with three components: (1) OT-guided diffusion sampling for latent distribution alignment, (2) label-image-aligned soft relabeling for adaptive label distributions, and (3) OT-based logit matching for output distribution alignment.

Result: Extensive experiments show consistent outperformance over state-of-the-art methods across diverse architectures and large-scale datasets, achieving at least 4% accuracy improvement under IPC=10 settings for each architecture on ImageNet-1K.

Conclusion: Optimal Transport provides a geometrically faithful framework for dataset distillation that effectively preserves local modes, intra-class patterns, and fine-grained variations, leading to superior generalization performance in an efficient manner.

Abstract: Dataset distillation seeks to synthesize a compact distilled dataset, enabling models trained on it to achieve performance comparable to models trained on the full dataset. Recent methods for large-scale datasets focus on matching global distributional statistics (e.g., mean and variance), but overlook critical instance-level characteristics and intraclass variations, leading to suboptimal generalization. We address this limitation by reformulating dataset distillation as an Optimal Transport (OT) distance minimization problem, enabling fine-grained alignment at both global and instance levels throughout the pipeline. OT offers a geometrically faithful framework for distribution matching. It effectively preserves local modes, intra-class patterns, and fine-grained variations that characterize the geometry of complex, high-dimensional distributions. Our method comprises three components tailored for preserving distributional geometry: (1) OT-guided diffusion sampling, which aligns latent distributions of real and distilled images; (2) label-image-aligned soft relabeling, which adapts label distributions based on the complexity of distilled image distributions; and (3) OT-based logit matching, which aligns the output of student models with soft-label distributions. Extensive experiments across diverse architectures and large-scale datasets demonstrate that our method consistently outperforms state-of-the-art approaches in an efficient manner, achieving at least 4% accuracy improvement under IPC=10 settings for each architecture on ImageNet-1K.

Method: Progressive Binary Thresholding Segmentation (PBTSeg) for lung segmentation, then Anatomy-aware Realistic Texture-based Anomaly Synthesis (ART-ASyn) that generates anatomically consistent lung opacity anomalies using texture-based augmentation.

Result: Generates paired synthetic anomalies with precise pixel-level masks for explicit segmentation supervision, enabling both one-class classification and zero-shot anomaly segmentation on unseen datasets.

Conclusion: ART-ASyn produces realistic, anatomically consistent anomalies that enable supervised anomaly segmentation and demonstrates strong zero-shot generalization capabilities for medical imaging.

Abstract: Unsupervised anomaly detection aims to identify anomalies without pixel-level annotations. Synthetic anomaly-based methods exhibit a unique capacity to introduce controllable irregularities with known masks, enabling explicit supervision during training. However, existing methods often produce synthetic anomalies that are visually distinct from real pathological patterns and ignore anatomical structure. This paper presents a novel Anatomy-aware Realistic Texture-based Anomaly Synthesis framework (ART-ASyn) for chest X-rays that generates realistic and anatomically consistent lung opacity related anomalies using texture-based augmentation guided by our proposed Progressive Binary Thresholding Segmentation method (PBTSeg) for lung segmentation. The generated paired samples of synthetic anomalies and their corresponding precise pixel-level anomaly mask for each normal sample enable explicit segmentation supervision. In contrast to prior work limited to one-class classification, ART-ASyn is further evaluated for zero-shot anomaly segmentation, demonstrating generalizability on an unseen dataset without target-domain annotations. Code availability is available at https://github.com/angelacao-hub/ART-ASyn.

Method: Uses ruled surfaces formed by straight lines sweeping through image-space time as the camera moves. Instead of point correspondences, the method analyzes the shape of these ruled surfaces for odometry information. The approach requires only differentially computed updates from point-to-line associations. The method is inspired by event cameras’ edge detection capabilities and is further constrained by IMU measurements to reduce solution space dimensionality.

Result: The paper presents a novel algorithm that reconstructs 3D scenes and estimates visual odometry from ruled surfaces. By leveraging IMU constraints, the method achieves reduced computational complexity and improved robustness compared to traditional point-based correspondence methods.

Conclusion: The proposed approach offers an efficient alternative to traditional visual odometry by avoiding costly point correspondences and instead using ruled surfaces from line features with IMU constraints, potentially enabling more robust and computationally efficient motion estimation.

Abstract: Visual odometry techniques typically rely on feature extraction from a sequence of images and subsequent computation of optical flow. This point-to-point correspondence between two consecutive frames can be costly to compute and suffers from varying accuracy, which affects the odometry estimate’s quality. Attempts have been made to bypass the difficulties originating from the correspondence problem by adopting line features and fusing other sensors (event camera, IMU) to improve performance, many of which still heavily rely on correspondence. If the camera observes a straight line as it moves, the image of the line sweeps a smooth surface in image-space time. It is a ruled surface and analyzing its shape gives information about odometry. Further, its estimation requires only differentially computed updates from point-to-line associations. Inspired by event cameras’ propensity for edge detection, this research presents a novel algorithm to reconstruct 3D scenes and visual odometry from these ruled surfaces. By constraining the surfaces with the inertia measurements from an onboard IMU sensor, the dimensionality of the solution space is greatly reduced.

Method: 1) Constructed FaceCaptionMask-1M dataset with 1M image-text-mask pairs for precise spatial supervision; 2) Used pretrained text-to-image model as backbone with two facial injection channels: 3D facial rendering for physical characteristics and global facial feature channel; 3) Two optimization objectives: semantic alignment in embedding space to reduce attribute leakage, and perceptual loss on facial images to preserve ID consistency.

Result: Extensive experiments show Face-MakeUpV2 achieves best overall performance in preserving face ID and maintaining physical consistency of reference images compared to other methods.

Conclusion: Face-MakeUpV2 demonstrates practical potential for reliable and controllable facial editing in diverse applications by effectively addressing attribute leakage and physical inconsistency problems.

Abstract: In facial image generation, current text-to-image models often suffer from facial attribute leakage and insufficient physical consistency when responding to local semantic instructions. In this study, we propose Face-MakeUpV2, a facial image generation model that aims to maintain the consistency of face ID and physical characteristics with the reference image. First, we constructed a large-scale dataset FaceCaptionMask-1M comprising approximately one million image-text-masks pairs that provide precise spatial supervision for the local semantic instructions. Second, we employed a general text-to-image pretrained model as the backbone and introduced two complementary facial information injection channels: a 3D facial rendering channel to incorporate the physical characteristics of the image and a global facial feature channel. Third, we formulated two optimization objectives for the supervised learning of our model: semantic alignment in the model’s embedding space to mitigate the attribute leakage problem and perceptual loss on facial images to preserve ID consistency. Extensive experiments demonstrated that our Face-MakeUpV2 achieves best overall performance in terms of preserving face ID and maintaining physical consistency of the reference images. These results highlight the practical potential of Face-MakeUpV2 for reliable and controllable facial editing in diverse applications.

Method: Created MVAD dataset with three key characteristics: genuine multimodality with three realistic video-audio forgery patterns, high perceptual quality using diverse state-of-the-art generative models, and comprehensive diversity across visual styles, content categories, and multimodal data types.

Result: MVAD is introduced as the first comprehensive dataset specifically designed for detecting AI-generated multimodal video-audio content, with samples available at https://github.com/HuMengXue0104/MVAD.

Conclusion: MVAD bridges a critical gap in multimodal AI-generated content detection by providing a comprehensive dataset that addresses limitations of existing resources, enabling development of more robust detection systems.

Abstract: The rapid advancement of AI-generated multimodal video-audio content has raised significant concerns regarding information security and content authenticity. Existing synthetic video datasets predominantly focus on the visual modality alone, while the few incorporating audio are largely confined to facial deepfakes–a limitation that fails to address the expanding landscape of general multimodal AI-generated content and substantially impedes the development of trustworthy detection systems. To bridge this critical gap, we introduce the Multimodal Video-Audio Dataset (MVAD), the first comprehensive dataset specifically designed for detecting AI-generated multimodal video-audio content. Our dataset exhibits three key characteristics: (1) genuine multimodality with samples generated according to three realistic video-audio forgery patterns; (2) high perceptual quality achieved through diverse state-of-the-art generative models; and (3) comprehensive diversity spanning realistic and anime visual styles, four content categories (humans, animals, objects, and scenes), and four video-audio multimodal data types. Our dataset will be available at https://github.com/HuMengXue0104/MVAD.

Method: PANDA uses a CLIP encoder to identify representative regions of low-frequency classes and transplants those patches into frequent-class samples within each task. It also incorporates an adaptive balancing strategy that leverages prior task distributions to smooth inter-task imbalances, reducing the overall gap between average samples across tasks.

Result: Extensive experiments and ablation studies demonstrate PANDA’s capability to work with existing PTM-based continual learning methods, improving accuracy and reducing catastrophic forgetting.

Conclusion: PANDA effectively addresses dual-level data imbalances in exemplar-free continual learning by combining intra-task patch augmentation with inter-task distribution smoothing, enabling fairer learning with frozen pre-trained models.

Abstract: Exemplar-Free Continual Learning (EFCL) restricts the storage of previous task data and is highly susceptible to catastrophic forgetting. While pre-trained models (PTMs) are increasingly leveraged for EFCL, existing methods often overlook the inherent imbalance of real-world data distributions. We discovered that real-world data streams commonly exhibit dual-level imbalances, dataset-level distributions combined with extreme or reversed skews within individual tasks, creating both intra-task and inter-task disparities that hinder effective learning and generalization. To address these challenges, we propose PANDA, a Patch-and-Distribution-Aware Augmentation framework that integrates seamlessly with existing PTM-based EFCL methods. PANDA amplifies low-frequency classes by using a CLIP encoder to identify representative regions and transplanting those into frequent-class samples within each task. Furthermore, PANDA incorporates an adaptive balancing strategy that leverages prior task distributions to smooth inter-task imbalances, reducing the overall gap between average samples across tasks and enabling fairer learning with frozen PTMs. Extensive experiments and ablation studies demonstrate PANDA’s capability to work with existing PTM-based CL methods, improving accuracy and reducing catastrophic forgetting.

Method: AMDET leverages the feature assimilation property where all tokens in backdoored samples show high similarity due to attention concentration on trigger tokens. It performs gradient-based inversion on token embeddings to recover implicit features that activate backdoor behaviors, then filters out natural backdoor features by analyzing loss landscapes.

Result: Extensive experiments on 3,600 backdoored and benign-finetuned models with two attack paradigms and three VLP structures show AMDET achieves 89.90% F1 score, completes detection in ~5 minutes on RTX 4090 GPU, and demonstrates strong robustness against adaptive attacks.

Conclusion: AMDET provides an effective, efficient, and practical solution for backdoor detection in VLPs without requiring prior knowledge, successfully identifying both intentionally injected backdoors and natural backdoor features in official CLIP models.

Abstract: Vision-language pretrained models (VLPs) such as CLIP have achieved remarkable success, but are also highly vulnerable to backdoor attacks. Given a model fine-tuned by an untrusted third party, determining whether the model has been injected with a backdoor is a critical and challenging problem. Existing detection methods usually rely on prior knowledge of training dataset, backdoor triggers and targets, or downstream classifiers, which may be impractical for real-world applications. To address this, To address this challenge, we introduce Assimilation Matters in DETection (AMDET), a novel model-level detection framework that operates without any such prior knowledge. Specifically, we first reveal the feature assimilation property in backdoored text encoders: the representations of all tokens within a backdoor sample exhibit a high similarity. Further analysis attributes this effect to the concentration of attention weights on the trigger token. Leveraging this insight, AMDET scans a model by performing gradient-based inversion on token embeddings to recover implicit features that capable of activating backdoor behaviors. Furthermore, we identify the natural backdoor feature in the OpenAI’s official CLIP model, which are not intentionally injected but still exhibit backdoor-like behaviors. We then filter them out from real injected backdoor by analyzing their loss landscapes. Extensive experiments on 3,600 backdoored and benign-finetuned models with two attack paradigms and three VLP model structures show that AMDET detects backdoors with an F1 score of 89.90%. Besides, it achieves one complete detection in approximately 5 minutes on a RTX 4090 GPU and exhibits strong robustness against adaptive attacks. Code is available at: https://github.com/Robin-WZQ/AMDET

Method: Proposes mmPred, a diffusion-based framework with dual-domain historical motion representation: Time-domain Pose Refinement (TPR) for fine-grained details and Frequency-domain Dominant Motion (FDM) for global trends. Uses Global Skeleton-relational Transformer (GST) as diffusion backbone to model global inter-joint cooperation.

Result: Achieves state-of-the-art performance, outperforming existing methods by 8.6% on mmBody dataset and 22% on mm-Fi dataset.

Conclusion: mmPred successfully addresses radar-specific artifacts in human motion prediction, demonstrating the viability of radar as a novel sensing modality with better performance than existing approaches.

Abstract: Existing Human Motion Prediction (HMP) methods based on RGB-D cameras are sensitive to lighting conditions and raise privacy concerns, limiting their real-world applications such as firefighting and healthcare. Motivated by the robustness and privacy-preserving nature of millimeter-wave (mmWave) radar, this work introduces radar as a novel sensing modality for HMP, for the first time. Nevertheless, radar signals often suffer from specular reflections and multipath effects, resulting in noisy and temporally inconsistent measurements, such as body-part miss-detection. To address these radar-specific artifacts, we propose mmPred, the first diffusion-based framework tailored for radar-based HMP. mmPred introduces a dual-domain historical motion representation to guide the generation process, combining a Time-domain Pose Refinement (TPR) branch for learning fine-grained details and a Frequency-domain Dominant Motion (FDM) branch for capturing global motion trends and suppressing frame-level inconsistency. Furthermore, we design a Global Skeleton-relational Transformer (GST) as the diffusion backbone to model global inter-joint cooperation, enabling corrupted joints to dynamically aggregate information from others. Extensive experiments show that mmPred achieves state-of-the-art performance, outperforming existing methods by 8.6% on mmBody and 22% on mm-Fi.

Method: Introduces SMamDiff with two novel designs: 1) residual-DCT motion encoding that subtracts last observed pose before temporal DCT to reduce DC component dominance and highlight higher-frequency cues, 2) stickman-drawing spatial-mamba module that processes joints in ordered, joint-by-joint manner to induce long-range cross-joint dependencies.

Result: Achieves state-of-the-art results among single-stage probabilistic HMP methods on Human3.6M and HumanEva datasets, while using less latency and memory than multi-stage diffusion baselines.

Conclusion: The proposed coherence mechanisms in SMamDiff enable effective spatial-temporal modeling for human motion prediction within a single-stage diffusion framework, making it suitable for edge deployment with improved efficiency and performance.

Abstract: With intelligent room-side sensing and service robots widely deployed, human motion prediction (HMP) is essential for safe, proactive assistance. However, many existing HMP methods either produce a single, deterministic forecast that ignores uncertainty or rely on probabilistic models that sacrifice kinematic plausibility. Diffusion models improve the accuracy-diversity trade-off but often depend on multi-stage pipelines that are costly for edge deployment. This work focuses on how to ensure spatial-temporal coherence within a single-stage diffusion model for HMP. We introduce SMamDiff, a Spatial Mamba-based Diffusion model with two novel designs: (i) a residual-DCT motion encoding that subtracts the last observed pose before a temporal DCT, reducing the first DC component ($f=0$) dominance and highlighting informative higher-frequency cues so the model learns how joints move rather than where they are; and (ii) a stickman-drawing spatial-mamba module that processes joints in an ordered, joint-by-joint manner, making later joints condition on earlier ones to induce long-range, cross-joint dependencies. On Human3.6M and HumanEva, these coherence mechanisms deliver state-of-the-art results among single-stage probabilistic HMP methods while using less latency and memory than multi-stage diffusion baselines.

Method: 1) Mamba-based dual granularity fusion encoder reformulates global interaction as channel-wise dynamic gating with 1D selective scan for efficient cross-modal modeling (linear complexity). 2) Reinterprets fusion as modality completion problem with region-aware 2D selective scanning completion branch for fine-grained fusion. 3) Lightweight frequency-aware modality adapter with spatial-frequency co-expert structure and pixel-wise router for efficient feature extraction in shared backbone.

Result: Extensive experiments on four multimodal benchmark datasets demonstrate effectiveness and generalization capability of the proposed method.

Conclusion: MM-DETR provides a lightweight and efficient framework for multimodal object detection that balances performance with practical deployment requirements through innovative fusion strategies and parameter-efficient design.

Abstract: Multimodal remote sensing object detection aims to achieve more accurate and robust perception under challenging conditions by fusing complementary information from different modalities. However, existing approaches that rely on attention-based or deformable convolution fusion blocks still struggle to balance performance and lightweight design. Beyond fusion complexity, extracting modality features with shared backbones yields suboptimal representations due to insufficient modality-specific modeling, whereas dual-stream architectures nearly double the parameter count, ultimately limiting practical deployment. To this end, we propose MM-DETR, a lightweight and efficient framework for multimodal object detection. Specifically, we propose a Mamba-based dual granularity fusion encoder that reformulates global interaction as channel-wise dynamic gating and leverages a 1D selective scan for efficient cross-modal modeling with linear complexity. Following this design, we further reinterpret multimodal fusion as a modality completion problem. A region-aware 2D selective scanning completion branch is introduced to recover modality-specific cues, supporting fine-grained fusion along a bidirectional pyramid pathway with minimal overhead. To further reduce parameter redundancy while retaining strong feature extraction capability, a lightweight frequency-aware modality adapter is inserted into the shared backbone. This adapter employs a spatial-frequency co-expert structure to capture modality-specific cues, while a pixel-wise router dynamically balances expert contributions for efficient spatial-frequency fusion. Extensive experiments conducted on four multimodal benchmark datasets demonstrate the effectiveness and generalization capability of the proposed method.

Method: Adapted a psychophysical experiment with 50 human participants to a semantic segmentation task, then tested a family of seven segmentation networks of varying sizes to analyze their internal representations.

Result: Smaller segmentation models naturally form human-like coarse body representations, while larger models tend to develop overly detailed, fine-grained encodings that deviate from human representations.

Conclusion: Coarse body representations emerge naturally under limited computational resources, and machine learning models provide a scalable approach to understanding the structure of physical reasoning in the human brain.

Abstract: Human physical reasoning relies on internal “body” representations - coarse, volumetric approximations that capture an object’s extent and support intuitive predictions about motion and physics. While psychophysical evidence suggests humans use such coarse representations, their internal structure remains largely unknown. Here we test whether vision models trained for segmentation develop comparable representations. We adapt a psychophysical experiment conducted with 50 human participants to a semantic segmentation task and test a family of seven segmentation networks, varying in size. We find that smaller models naturally form human-like coarse body representations, whereas larger models tend toward overly detailed, fine-grain encodings. Our results demonstrate that coarse representations can emerge under limited computational resources, and that machine representations can provide a scalable path toward understanding the structure of physical reasoning in the brain.

Method: THCRL consists of two key modules: 1) Deep Symmetry Hierarchical Fusion (DSHF) module using UNet architecture with multiple denoising mechanisms for trustworthy multi-view fusion, and 2) Average K-Nearest Neighbors Contrastive Learning (AKCL) module that aligns fused representations with view-specific representations by enhancing similarity among samples in the same cluster rather than just same samples across views.

Result: Extensive experiments demonstrate that THCRL achieves state-of-the-art performance in deep multi-view clustering tasks.

Conclusion: THCRL effectively addresses the untrustworthy fusion problem in multi-view clustering through hierarchical contrastive representation learning with denoising and neighbor-aware similarity computation, leading to superior clustering performance.

Abstract: Multi-View Clustering (MVC) has garnered increasing attention in recent years. It is capable of partitioning data samples into distinct groups by learning a consensus representation. However, a significant challenge remains: the problem of untrustworthy fusion. This problem primarily arises from two key factors: 1) Existing methods often ignore the presence of inherent noise within individual views; 2) In traditional MVC methods using Contrastive Learning (CL), similarity computations typically rely on different views of the same instance, while neglecting the structural information from nearest neighbors within the same cluster. Consequently, this leads to the wrong direction for multi-view fusion. To address this problem, we present a novel Trusted Hierarchical Contrastive Representation Learning (THCRL). It consists of two key modules. Specifically, we propose the Deep Symmetry Hierarchical Fusion (DSHF) module, which leverages the UNet architecture integrated with multiple denoising mechanisms to achieve trustworthy fusion of multi-view data. Furthermore, we present the Average K-Nearest Neighbors Contrastive Learning (AKCL) module to align the fused representation with the view-specific representation. Unlike conventional strategies, AKCL enhances representation similarity among samples belonging to the same cluster, rather than merely focusing on the same sample across views, thereby reinforcing the confidence of the fused representation. Extensive experiments demonstrate that THCRL achieves the state-of-the-art performance in deep MVC tasks.

Method: POLARIS (Projection-Orthogonal Least Squares for Robust and Adaptive Inversion) reformulates inversion as an error-origin problem rather than error-compensation. It treats guidance scale ω as a step-wise variable and derives a mathematically grounded formula to minimize inversion error at each step, requiring just one line of code.

Result: POLARIS substantially mitigates noise approximation errors, improves inversion latent quality with negligible performance overhead, and consistently enhances accuracy of downstream tasks in image editing and restoration.

Conclusion: POLARIS provides an efficient, mathematically sound solution to the critical problem of noise approximation error accumulation in diffusion model inversion, offering significant improvements with minimal implementation complexity.

Abstract: The Inversion-Denoising Paradigm, which is based on diffusion models, excels in diverse image editing and restoration tasks. We revisit its mechanism and reveal a critical, overlooked factor in reconstruction degradation: the approximate noise error. This error stems from approximating the noise at step t with the prediction at step t-1, resulting in severe error accumulation throughout the inversion process. We introduce Projection-Orthogonal Least Squares for Robust and Adaptive Inversion (POLARIS), which reformulates inversion from an error-compensation problem into an error-origin problem. Rather than optimizing embeddings or latent codes to offset accumulated drift, POLARIS treats the guidance scale ω as a step-wise variable and derives a mathematically grounded formula to minimize inversion error at each step. Remarkably, POLARIS improves inversion latent quality with just one line of code. With negligible performance overhead, it substantially mitigates noise approximation errors and consistently improves the accuracy of downstream tasks.

Method: Proposed PorePatch dataset with Data-Model Co-Evolution (DMCE) framework to generate high-quality pore-scale image patches from high-resolution facial images. Trained SOTA models on this dataset and evaluated on matching and 3D reconstruction tasks.

Result: SOTA model achieved FPR95 of 1.91% on matching task, outperforming PSIFT (22.41%) by 20.5%. However, in 3D reconstruction, deep learning descriptors didn’t significantly outperform traditional descriptors, showing limitations in facial weak-texture regions.

Conclusion: Deep learning descriptors still have limitations in addressing facial weak-texture challenges despite good matching performance. More work needed in this field, and the PorePatch dataset provides a valuable benchmark for future research.

Abstract: The weak-texture nature of facial skin regions presents significant challenges for local descriptor matching in applications such as facial motion analysis and 3D face reconstruction. Although deep learning-based descriptors have demonstrated superior performance to traditional hand-crafted descriptors in many applications, the scarcity of pore-scale image patch datasets has hindered their further development in the facial domain. In this paper, we propose the PorePatch dataset, a high-quality pore-scale image patch dataset, and establish a rational evaluation benchmark. We introduce a Data-Model Co-Evolution (DMCE) framework to generate a progressively refined, high-quality dataset from high-resolution facial images. We then train existing SOTA models on our dataset and conduct extensive experiments. Our results show that the SOTA model achieves a FPR95 value of 1.91% on the matching task, outperforming PSIFT (22.41%) by a margin of 20.5%. However, its advantage is diminished in the 3D reconstruction task, where its overall performance is not significantly better than that of traditional descriptors. This indicates that deep learning descriptors still have limitations in addressing the challenges of facial weak-texture regions, and much work remains to be done in this field.

Method: Proposes a learnable, fully GPU partitioning algorithm that generates geometrically and semantically coherent superpoints. The module is compact (<60k parameters), trains with a differentiable surrogate loss in under 20 minutes, and requires no handcrafted features. Combined with a lightweight superpoint classifier.

Result: Achieves 13× faster partitioning than prior methods, with 72× faster inference and 120× fewer parameters. The full pipeline fits in <2 MB VRAM, scales to multi-million-point scenes, and supports real-time inference. Matches accuracy of point-based SOTA models across indoor scans (S3DIS), autonomous driving (KITTI-360), and aerial LiDAR (DALES).

Conclusion: EZ-SP provides an efficient, GPU-accelerated superpoint partitioning solution that enables real-time 3D semantic segmentation with minimal memory footprint while maintaining state-of-the-art accuracy across diverse domains.

Abstract: Superpoint-based pipelines provide an efficient alternative to point- or voxel-based 3D semantic segmentation, but are often bottlenecked by their CPU-bound partition step. We propose a learnable, fully GPU partitioning algorithm that generates geometrically and semantically coherent superpoints 13$\times$ faster than prior methods. Our module is compact (under 60k parameters), trains in under 20 minutes with a differentiable surrogate loss, and requires no handcrafted features. Combine with a lightweight superpoint classifier, the full pipeline fits in $<$2 MB of VRAM, scales to multi-million-point scenes, and supports real-time inference. With 72$\times$ faster inference and 120$\times$ fewer parameters, EZ-SP matches the accuracy of point-based SOTA models across three domains: indoor scans (S3DIS), autonomous driving (KITTI-360), and aerial LiDAR (DALES). Code and pretrained models are accessible at github.com/drprojects/superpoint_transformer.

Method: WiseEdit decomposes image editing into three cascaded cognitive steps (Awareness, Interpretation, Imagination) and incorporates three knowledge types (Declarative, Procedural, Metacognitive). It includes both step-specific tasks and complex tasks that combine multiple steps.

Result: The benchmark comprises 1,220 test cases that objectively reveal limitations of state-of-the-art image editing models in knowledge-based cognitive reasoning and creative composition capabilities.

Conclusion: WiseEdit provides a comprehensive evaluation framework for advanced image editing models, addressing the gap in existing benchmarks and enabling better assessment of cognition- and creativity-informed editing capabilities.

Abstract: Recent image editing models boast next-level intelligent capabilities, facilitating cognition- and creativity-informed image editing. Yet, existing benchmarks provide too narrow a scope for evaluation, failing to holistically assess these advanced abilities. To address this, we introduce WiseEdit, a knowledge-intensive benchmark for comprehensive evaluation of cognition- and creativity-informed image editing, featuring deep task depth and broad knowledge breadth. Drawing an analogy to human cognitive creation, WiseEdit decomposes image editing into three cascaded steps, i.e., Awareness, Interpretation, and Imagination, each corresponding to a task that poses a challenge for models to complete at the specific step. It also encompasses complex tasks, where none of the three steps can be finished easily. Furthermore, WiseEdit incorporates three fundamental types of knowledge: Declarative, Procedural, and Metacognitive knowledge. Ultimately, WiseEdit comprises 1,220 test cases, objectively revealing the limitations of SoTA image editing models in knowledge-based cognitive reasoning and creative composition capabilities. The benchmark, evaluation code, and the generated images of each model will be made publicly available soon. Project Page: https://qnancy.github.io/wiseedit_project_page/.

Method: STAMP uses all-mask prediction paradigm that decouples autoregressive dialogue generation from non-autoregressive mask prediction. After generating textual response, it predicts entire segmentation mask in single forward pass by treating it as parallel “fill-in-the-blank” task over image patches.

Result: STAMP significantly outperforms state-of-the-art methods across multiple segmentation benchmarks, providing solution that excels in dialogue, segmentation, and speed without compromise.

Conclusion: The all-mask prediction paradigm successfully resolves the core trilemma in MLLMs, enabling simultaneous preservation of dialogue ability, high segmentation performance, and fast inference through decoupled architecture design.

Abstract: Integrating segmentation into Multimodal Large Language Models (MLLMs) presents a core trilemma: simultaneously preserving dialogue ability, achieving high segmentation performance, and ensuring fast inference. Prevailing paradigms are forced into a compromise. Embedding prediction methods introduce a conflicting pixel-level objective that degrades the MLLM’s general dialogue abilities. The alternative, next-token prediction, reframes segmentation as an autoregressive task, which preserves dialogue but forces a trade-off between poor segmentation performance with sparse outputs or prohibitive inference speeds with rich ones. We resolve this trilemma with all-mask prediction, a novel paradigm that decouples autoregressive dialogue generation from non-autoregressive mask prediction. We present STAMP: Simultaneous Textual All-Mask Prediction, an MLLM that embodies this paradigm. After generating a textual response, STAMP predicts an entire segmentation mask in a single forward pass by treating it as a parallel “fill-in-the-blank” task over image patches. This design maintains the MLLM’s dialogue ability by avoiding conflicting objectives, enables high segmentation performance by leveraging rich, bidirectional spatial context for all mask tokens, and achieves exceptional speed. Extensive experiments show that STAMP significantly outperforms state-of-the-art methods across multiple segmentation benchmarks, providing a solution that excels in dialogue, segmentation, and speed without compromise.

Method: Decomposes video into three compact modalities: textual descriptions (semantic), spatiotemporally degraded video (appearance), and optional sketches/poses (motion cues). Uses a conditional video diffusion model to reconstruct high-quality, temporally coherent videos from these representations. Includes temporal forward filling, token interleaving, and modality-specific codecs to improve multimodal generation and compactness.

Result: Outperforms baseline semantic and traditional codecs by 2-10X on perceptual metrics at low bitrates. The method successfully maintains video quality even at ultra-low bitrates where traditional codecs collapse.

Conclusion: Semantic video compression that focuses on transmitting meaningful information rather than pixel accuracy, combined with generative priors for detail synthesis, provides a superior approach for ultra-low bitrate video compression that aligns better with human perception.

Abstract: Traditional video codecs optimized for pixel fidelity collapse at ultra-low bitrates and produce severe artifacts. This failure arises from a fundamental misalignment between pixel accuracy and human perception. We propose a semantic video compression framework named DiSCo that transmits only the most meaningful information while relying on generative priors for detail synthesis. The source video is decomposed into three compact modalities: a textual description, a spatiotemporally degraded video, and optional sketches or poses that respectively capture semantic, appearance, and motion cues. A conditional video diffusion model then reconstructs high-quality, temporally coherent videos from these compact representations. Temporal forward filling, token interleaving, and modality-specific codecs are proposed to improve multimodal generation and modality compactness. Experiments show that our method outperforms baseline semantic and traditional codecs by 2-10X on perceptual metrics at low bitrates.

Method: Proposes SplatFont3D with Glyph2Cloud module that enhances 2D glyphs to produce 3D point clouds for Gaussian initialization. Uses score distillation sampling with pretrained 2D diffusion model for optimization. Implements dynamic component assignment strategy using geometric priors of 3D Gaussians for part-level control while preventing drift-induced entanglement.

Result: Outperforms existing 3D models for 3D-AFG in style-text consistency, visual quality, and rendering efficiency. Provides more explicit and effective part-level style control than NeRF with faster rendering.

Conclusion: SplatFont3D successfully addresses the challenges of 3D-AFG by enabling precise part-level style control through 3D Gaussian splatting, offering superior performance in style consistency, quality, and efficiency compared to existing methods.

Abstract: Artistic font generation (AFG) can assist human designers in creating innovative artistic fonts. However, most previous studies primarily focus on 2D artistic fonts in flat design, leaving personalized 3D-AFG largely underexplored. 3D-AFG not only enables applications in immersive 3D environments such as video games and animations, but also may enhance 2D-AFG by rendering 2D fonts of novel views. Moreover, unlike general 3D objects, 3D fonts exhibit precise semantics with strong structural constraints and also demand fine-grained part-level style control. To address these challenges, we propose SplatFont3D, a novel structure-aware text-to-3D AFG framework with 3D Gaussian splatting, which enables the creation of 3D artistic fonts from diverse style text prompts with precise part-level style control. Specifically, we first introduce a Glyph2Cloud module, which progressively enhances both the shapes and styles of 2D glyphs (or components) and produces their corresponding 3D point clouds for Gaussian initialization. The initialized 3D Gaussians are further optimized through interaction with a pretrained 2D diffusion model using score distillation sampling. To enable part-level control, we present a dynamic component assignment strategy that exploits the geometric priors of 3D Gaussians to partition components, while alleviating drift-induced entanglement during 3D Gaussian optimization. Our SplatFont3D provides more explicit and effective part-level style control than NeRF, attaining faster rendering efficiency. Experiments show that our SplatFont3D outperforms existing 3D models for 3D-AFG in style-text consistency, visual quality, and rendering efficiency.

Method: Proposes a unified physics-based pipeline with: 1) A flow matching model with explicit physical guidance using alternating velocity-based updates and physics-based refinement, 2) Physics-aware regularization, and 3) A shape-and-physics variational autoencoder (SP-VAE) that jointly encodes shape and physics into a unified latent space.

Result: Experiments on three benchmarks show the synergistic formulation improves shape realism beyond mere visual plausibility by incorporating physical validity.

Conclusion: The proposed physics-guided 3D shape generation approach successfully integrates physical properties into the generative process, enhancing realism for industrial design applications where physical performance matters.

Abstract: Existing generative models for 3D shapes can synthesize high-fidelity and visually plausible shapes. For certain classes of shapes that have undergone an engineering design process, the realism of the shape is tightly coupled with the underlying physical properties, e.g., aerodynamic efficiency for automobiles. Since existing methods lack knowledge of such physics, they are unable to use this knowledge to enhance the realism of shape generation. Motivated by this, we propose a unified physics-based 3D shape generation pipeline, with a focus on industrial design applications. Specifically, we introduce a new flow matching model with explicit physical guidance, consisting of an alternating update process. We iteratively perform a velocity-based update and a physics-based refinement, progressively adjusting the latent code to align with the desired 3D shapes and physical properties. We further strengthen physical validity by incorporating a physics-aware regularization term into the velocity-based update step. To support such physics-guided updates, we build a shape-and-physics variational autoencoder (SP-VAE) that jointly encodes shape and physics information into a unified latent space. The experiments on three benchmarks show that this synergistic formulation improves shape realism beyond mere visual plausibility.

Method: A baseline pipeline combining YOLOv12 detection, BotSORT tracking, OSNet embeddings, OCR-based timestamping, and telematics-based stop classification to recover bus origin-destination flows from onboard CCTV footage.

Result: High counting accuracy under low-density, well-lit conditions (recall ≈95%, precision ≈91%, F1 ≈93%) with OD matrices matching manual tallies, but sharp performance degradation under overcrowding, color-to-monochrome shifts, posture variation, and non-standard door use.

Conclusion: The system reveals deployment-specific failure modes in SSA transit, motivating the need for more robust, deployment-focused Re-ID methods that can handle real-world stressors like overcrowding and challenging environmental conditions.

Abstract: Public transport in sub-Saharan Africa (SSA) often operates in overcrowded conditions where existing automated systems fail to capture reliable passenger flow data. Leveraging onboard CCTV already deployed for security, we present a baseline pipeline that combines YOLOv12 detection, BotSORT tracking, OSNet embeddings, OCR-based timestamping, and telematics-based stop classification to recover bus origin–destination (OD) flows. On annotated CCTV segments from Nairobi and Kigali buses, the system attains high counting accuracy under low-density, well-lit conditions (recall $\approx$95%, precision $\approx$91%, F1 $\approx$93%). It produces OD matrices that closely match manual tallies. Under realistic stressors such as overcrowding, color-to-monochrome shifts, posture variation, and non-standard door use, performance degrades sharply (e.g., $\sim$40% undercount in peak-hour boarding and a $\sim$17 percentage-point drop in recall for monochrome segments), revealing deployment-specific failure modes and motivating more robust, deployment-focused Re-ID methods for SSA transit.

Method: Extracts measurable proxies from generated videos using frozen utility models: optical flow as velocity proxy, appearance features as mass proxy. Enforces Newtonian structure through two rewards: Newtonian kinematic constraint (constant-acceleration dynamics) and mass conservation reward (prevents degenerate solutions).

Result: Evaluated on NewtonBench-60K benchmark with five Newtonian Motion Primitives. Consistently improves physical plausibility, motion smoothness, and temporal coherence over prior methods. Maintains strong performance under out-of-distribution shifts in height, speed, and friction.

Conclusion: Physics-grounded verifiable rewards offer a scalable path toward physics-aware video generation, bridging the gap between visual and physical realism without relying on human or VLM feedback.

Abstract: Recent video diffusion models can synthesize visually compelling clips, yet often violate basic physical laws-objects float, accelerations drift, and collisions behave inconsistently-revealing a persistent gap between visual realism and physical realism. We propose $\texttt{NewtonRewards}$, the first physics-grounded post-training framework for video generation based on $\textit{verifiable rewards}$. Instead of relying on human or VLM feedback, $\texttt{NewtonRewards}$ extracts $\textit{measurable proxies}$ from generated videos using frozen utility models: optical flow serves as a proxy for velocity, while high-level appearance features serve as a proxy for mass. These proxies enable explicit enforcement of Newtonian structure through two complementary rewards: a Newtonian kinematic constraint enforcing constant-acceleration dynamics, and a mass conservation reward preventing trivial, degenerate solutions. We evaluate $\texttt{NewtonRewards}$ on five Newtonian Motion Primitives (free fall, horizontal/parabolic throw, and ramp sliding down/up) using our newly constructed large-scale benchmark, $\texttt{NewtonBench-60K}$. Across all primitives in visual and physics metrics, $\texttt{NewtonRewards}$ consistently improves physical plausibility, motion smoothness, and temporal coherence over prior post-training methods. It further maintains strong performance under out-of-distribution shifts in height, speed, and friction. Our results show that physics-grounded verifiable rewards offer a scalable path toward physics-aware video generation.

Method: Trained a pneumonia classifier exclusively using synthetic chest X-ray images generated by Google’s Nano Banana AI model, then evaluated it on two real-world datasets (RSNA Pneumonia Detection dataset with 14,863 CXRs and Chest X-Ray dataset with 5,856 CXRs) without any real-world training data.

Result: The synthetic-data-only classifier achieved AUROC of 0.923 and AUPR of 0.900 on RSNA dataset, and AUROC of 0.824 and AUPR of 0.913 on Chest X-Ray dataset, demonstrating strong performance on real-world pneumonia detection despite never seeing real training data.

Conclusion: Synthetic medical imaging data shows promise for medical AI development, but faces challenges in prompt design diversity control, post-processing alignment, and requires substantial validation, regulatory approval, and ethical oversight before clinical use.

Abstract: We trained a classifier with synthetic chest X-ray (CXR) images generated by Nano Banana, the latest AI model for image generation and editing, released by Google. When directly applied to real-world CXRs having only been trained with synthetic data, the classifier achieved an AUROC of 0.923 (95% CI: 0.919 - 0.927), and an AUPR of 0.900 (95% CI: 0.894 - 0.907) in recognizing pneumonia in the 2018 RSNA Pneumonia Detection dataset (14,863 CXRs), and an AUROC of 0.824 (95% CI: 0.810 - 0.836), and an AUPR of 0.913 (95% CI: 0.904 - 0.922) in the Chest X-Ray dataset (5,856 CXRs). These external validation results on real-world data demonstrate the feasibility of this approach and suggest potential for synthetic data in medical AI development. Nonetheless, several limitations remain at present, including challenges in prompt design for controlling the diversity of synthetic CXR data and the requirement for post-processing to ensure alignment with real-world data. However, the growing sophistication and accessibility of medical intelligence will necessitate substantial validation, regulatory approval, and ethical oversight prior to clinical translation.

Method: Proposes Filling-Based Reward (FR) that estimates approximate future trajectories by finding and applying reasonable filling schemes to complete intermediate sequences. Builds FR-TTS strategy that searches for good filling schemes and incorporates diversity reward with dynamic weighting for balanced evaluation.

Result: FR provides excellent and reliable metric for evaluating intermediate sample quality, with high correlation between intermediate and final rewards. FR-TTS outperforms multiple established benchmarks and various reward models in experimental validation.

Conclusion: FR-TTS successfully enables effective test-time scaling for next-token prediction by addressing the critical issue of low correlation between intermediate and final rewards through intelligent sequence filling and comprehensive evaluation strategies.

Abstract: Test-time scaling (TTS) has become a prevalent technique in image generation, significantly boosting output quality by expanding the number of parallel samples and filtering them using pre-trained reward models. However, applying this powerful methodology to the next-token prediction (NTP) paradigm remains challenging. The primary obstacle is the low correlation between the reward of an image decoded from an intermediate token sequence and the reward of the fully generated image. Consequently, these incomplete intermediate representations prove to be poor indicators for guiding the pruning direction, a limitation that stems from their inherent incompleteness in scale or semantic content. To effectively address this critical issue, we introduce the Filling-Based Reward (FR). This novel design estimates the approximate future trajectory of an intermediate sample by finding and applying a reasonable filling scheme to complete the sequence. Both the correlation coefficient between rewards of intermediate samples and final samples, as well as multiple intrinsic signals like token confidence, indicate that the FR provides an excellent and reliable metric for accurately evaluating the quality of intermediate samples. Building upon this foundation, we propose FR-TTS, a sophisticated scaling strategy. FR-TTS efficiently searches for good filling schemes and incorporates a diversity reward with a dynamic weighting schedule to achieve a balanced and comprehensive evaluation of intermediate samples. We experimentally validate the superiority of FR-TTS over multiple established benchmarks and various reward models. Code is available at \href{https://github.com/xuhang07/FR-TTS}{https://github.com/xuhang07/FR-TTS}.

Method: Cross-Modal Regression with Manifold Fusion (CRMF): geometric deep learning framework modeling behavioral representations across hyperbolic, spherical, and Euclidean manifolds. Uses geometry-specific expert networks for hierarchical traits, directional patterns, and continuous variations, with adaptive routing and tangent space fusion.

Result: CRMF substantially outperforms baselines with up to 11.4% improvement in Spearman correlation and 6.0% in concordance index, while training 40-50% fewer parameters than large multimodal models.

Conclusion: The RecruitView dataset and CRMF framework provide effective solutions for automated personality assessment by capturing geometric structure in human traits through multimodal behavioral data analysis.

Abstract: Automated personality and soft skill assessment from multimodal behavioral data remains challenging due to limited datasets and methods that fail to capture geometric structure inherent in human traits. We introduce RecruitView, a dataset of 2,011 naturalistic video interview clips from 300+ participants with 27,000 pairwise comparative judgments across 12 dimensions: Big Five personality traits, overall personality score, and six interview performance metrics. To leverage this data, we propose Cross-Modal Regression with Manifold Fusion (CRMF), a geometric deep learning framework that explicitly models behavioral representations across hyperbolic, spherical, and Euclidean manifolds. CRMF employs geometry-specific expert networks to capture hierarchical trait structures, directional behavioral patterns, and continuous performance variations simultaneously. An adaptive routing mechanism dynamically weights expert contributions based on input characteristics. Through principled tangent space fusion, CRMF achieves superior performance while training 40-50% fewer trainable parameters than large multimodal models. Extensive experiments demonstrate that CRMF substantially outperforms the selected baselines, achieving up to 11.4% improvement in Spearman correlation and 6.0% in concordance index. Our RecruitView dataset is publicly available at https://huggingface.co/datasets/AI4A-lab/RecruitView

Method: CausalAffect uses a two-level polarity and direction aware causal hierarchy that integrates population-level regularities with sample-adaptive structures. It includes a feature-level counterfactual intervention mechanism to enforce true causal effects while suppressing spurious correlations. The approach requires neither jointly annotated datasets nor handcrafted causal priors.

Result: The framework recovers causal structures consistent with established psychological theories while revealing novel inhibitory and previously uncharacterized dependencies. Extensive experiments across six benchmarks demonstrate state-of-the-art performance in both AU detection and expression recognition.

Conclusion: CausalAffect establishes a principled connection between causal discovery and interpretable facial behavior, advancing the field by providing the first framework for causal graph discovery in facial affect analysis with practical applications and theoretical consistency.

Abstract: Understanding human affect from facial behavior requires not only accurate recognition but also structured reasoning over the latent dependencies that drive muscle activations and their expressive outcomes. Although Action Units (AUs) have long served as the foundation of affective computing, existing approaches rarely address how to infer psychologically plausible causal relations between AUs and expressions directly from data. We propose CausalAffect, the first framework for causal graph discovery in facial affect analysis. CausalAffect models AU-AU and AU-Expression dependencies through a two-level polarity and direction aware causal hierarchy that integrates population-level regularities with sample-adaptive structures. A feature-level counterfactual intervention mechanism further enforces true causal effects while suppressing spurious correlations. Crucially, our approach requires neither jointly annotated datasets nor handcrafted causal priors, yet it recovers causal structures consistent with established psychological theories while revealing novel inhibitory and previously uncharacterized dependencies. Extensive experiments across six benchmarks demonstrate that CausalAffect advances the state of the art in both AU detection and expression recognition, establishing a principled connection between causal discovery and interpretable facial behavior. All trained models and source code will be released upon acceptance.

Method: RealGen integrates an LLM for prompt optimization and a diffusion model for image generation. It introduces a “Detector Reward” mechanism that quantifies artifacts and assesses realism using semantic-level and feature-level synthetic image detectors, then uses GRPO algorithm to optimize the entire pipeline.

Result: RealGen significantly outperforms general models (GPT-Image-1, Qwen-Image) and specialized photorealistic models (FLUX-Krea) in terms of realism, detail, and aesthetics.

Conclusion: RealGen successfully addresses photorealism in text-to-image generation through its integrated framework with detector-based reward optimization, and the proposed RealBench benchmark enables accurate, human-free evaluation of photorealism aligned with real user experience.

Abstract: With the continuous advancement of image generation technology, advanced models such as GPT-Image-1 and Qwen-Image have achieved remarkable text-to-image consistency and world knowledge However, these models still fall short in photorealistic image generation. Even on simple T2I tasks, they tend to produce " fake" images with distinct AI artifacts, often characterized by “overly smooth skin” and “oily facial sheens”. To recapture the original goal of “indistinguishable-from-reality” generation, we propose RealGen, a photorealistic text-to-image framework. RealGen integrates an LLM component for prompt optimization and a diffusion model for realistic image generation. Inspired by adversarial generation, RealGen introduces a “Detector Reward” mechanism, which quantifies artifacts and assesses realism using both semantic-level and feature-level synthetic image detectors. We leverage this reward signal with the GRPO algorithm to optimize the entire generation pipeline, significantly enhancing image realism and detail. Furthermore, we propose RealBench, an automated evaluation benchmark employing Detector-Scoring and Arena-Scoring. It enables human-free photorealism assessment, yielding results that are more accurate and aligned with real user experience. Experiments demonstrate that RealGen significantly outperforms general models like GPT-Image-1 and Qwen-Image, as well as specialized photorealistic models like FLUX-Krea, in terms of realism, detail, and aesthetics. The code is available at https://github.com/yejy53/RealGen.

Method: Uses Structured Partition of Sequence (SPoS) to partition input frames and provide structured context, calculates group similarities to capture frame differences, employs lightweight fully convolutional network for boundary detection, and adapts Gaussian kernel to handle annotation ambiguities.

Result: Achieves superior performance over state-of-the-art methods on Kinetics-GEBD, TAPOS, and shot transition detection datasets with better speed-accuracy trade-off and linear computational complexity.

Conclusion: The proposed Structured Context Learning with SPoS provides an effective, flexible, and efficient solution for GEBD that outperforms existing methods while maintaining computational efficiency.

Abstract: Generic Event Boundary Detection (GEBD) aims to identify moments in videos that humans perceive as event boundaries. This paper proposes a novel method for addressing this task, called Structured Context Learning, which introduces the Structured Partition of Sequence (SPoS) to provide a structured context for learning temporal information. Our approach is end-to-end trainable and flexible, not restricted to specific temporal models like GRU, LSTM, and Transformers. This flexibility enables our method to achieve a better speed-accuracy trade-off. Specifically, we apply SPoS to partition the input frame sequence and provide a structured context for the subsequent temporal model. Notably, SPoS’s overall computational complexity is linear with respect to the video length. We next calculate group similarities to capture differences between frames, and a lightweight fully convolutional network is utilized to determine the event boundaries based on the grouped similarity maps. To remedy the ambiguities of boundary annotations, we adapt the Gaussian kernel to preprocess the ground-truth event boundaries. Our proposed method has been extensively evaluated on the challenging Kinetics-GEBD, TAPOS, and shot transition detection datasets, demonstrating its superiority over existing state-of-the-art methods.

Method: Task-Aligned Context Selection (TACS) jointly trains a selector network with the task model using a hybrid optimization scheme combining gradient-based supervision and reinforcement learning. This makes retrieval part of the learning objective, aligning example selection with task performance rewards rather than just visual similarity.

Result: Across 18 datasets covering fine-grained recognition, medical image classification, and medical image segmentation, TACS consistently outperforms similarity-based retrieval methods. The improvements are particularly significant in challenging or data-limited settings.

Conclusion: By aligning context selection with task rewards, TACS enables discriminative models to discover which contextual examples genuinely help improve predictions, moving beyond superficial similarity to achieve better task performance.

Abstract: Humans often resolve visual uncertainty by comparing an image with relevant examples, but ViTs lack the ability to identify which examples would improve their predictions. We present Task-Aligned Context Selection (TACS), a framework that learns to select paired examples which truly improve task performance rather than those that merely appear similar. TACS jointly trains a selector network with the task model through a hybrid optimization scheme combining gradient-based supervision and reinforcement learning, making retrieval part of the learning objective. By aligning selection with task rewards, TACS enables discriminative models to discover which contextual examples genuinely help. Across 18 datasets covering fine-grained recognition, medical image classification, and medical image segmentation, TACS consistently outperforms similarity-based retrieval, particularly in challenging or data-limited settings.

Method: Uses a hybrid instance generator combining semantics-aware vector-set representation with detail-rich structured latent representation, plus a camera-conditioned scale-solving algorithm to apply foundational pose estimation models for scene coherence.

Result: Extensive experiments show CC-FMO consistently generates high-fidelity camera-aligned compositional scenes, outperforming all state-of-the-art methods.

Conclusion: CC-FMO successfully addresses scene generation challenges by preserving both object layout fidelity and instance quality through its novel hybrid approach and camera-conditioned pipeline.

Abstract: High-quality 3D scene generation from a single image is crucial for AR/VR and embodied AI applications. Early approaches struggle to generalize due to reliance on specialized models trained on curated small datasets. While recent advancements in large-scale 3D foundation models have significantly enhanced instance-level generation, coherent scene generation remains a challenge, where performance is limited by inaccurate per-object pose estimations and spatial inconsistency. To this end, this paper introduces CC-FMO, a zero-shot, camera-conditioned pipeline for single-image to 3D scene generation that jointly conforms to the object layout in input image and preserves instance fidelity. CC-FMO employs a hybrid instance generator that combines semantics-aware vector-set representation with detail-rich structured latent representation, yielding object geometries that are both semantically plausible and high-quality. Furthermore, CC-FMO enables the application of foundational pose estimation models in the scene generation task via a simple yet effective camera-conditioned scale-solving algorithm, to enforce scene-level coherence. Extensive experiments demonstrate that CC-FMO consistently generates high-fidelity camera-aligned compositional scenes, outperforming all state-of-the-art methods.

Method: Smartphone-based structured-light system projects grid pattern onto ground. Uses novel topology-constrained 2D dynamic time warping (2D-DTW) algorithm for robust grid matching under perspective distortion and occlusion. Designed for resource-limited platforms while preserving grid structure for accurate triangulation.

Result: Proposed 2D-DTW formulation enables terrain sensing by matching deformed grid patterns and also serves as general tool for structured grid pattern matching in image processing. System reconstructs local terrain unevenness from single handheld device.

Conclusion: The smartphone-based structured-light system with 2D-DTW algorithm provides practical solution for detecting subtle terrain unevenness on resource-limited platforms, with applications beyond terrain sensing to general grid pattern matching problems.

Abstract: Low-cost mobile rovers often operate on uneven terrain where small bumps or tilts are difficult to perceive visually but can significantly affect locomotion stability. To address this problem, we explore a smartphone-based structured-light system that projects a grid pattern onto the ground and reconstructs local terrain unevenness from a single handheld device. The system is inspired by face-recognition projectors, but adapted for ground sensing. A key technical challenge is robustly matching the projected grid with its deformed observation under perspective distortion and partial occlusion. Conventional one-dimensional dynamic time warping (1D-DTW) is not directly applicable to such two-dimensional grid patterns. We therefore propose a topology-constrained two-dimensional dynamic time warping (2D-DTW) algorithm that performs column-wise alignment under a global grid consistency constraint. The proposed method is designed to be simple enough to run on resource limited platforms while preserving the grid structure required for accurate triangulation. We demonstrate that our 2D-DTW formulation can be used not only for terrain sensing but also as a general tool for matching structured grid patterns in image processing scenarios. This paper describes the overall system design as well as the 2D-DTW extension that emerged from this application.

Method: Two-part framework: (1) text-conditioned generation using language instruction and first frame, and (2) trajectory-conditioned generation using 2D trajectory overlay and initial frame. Uses LoRA fine-tuning for adaptation.

Result: Experiments on Jaco Play, Bridge V2, and RT1 datasets show both modes produce smooth, coherent robot videos aligned with their respective conditions.

Conclusion: Pretrained image generators encode transferable temporal priors and can function as video-like robotic planners under minimal supervision.

Abstract: Generating realistic robotic manipulation videos is an important step toward unifying perception, planning, and action in embodied agents. While existing video diffusion models require large domain-specific datasets and struggle to generalize, recent image generation models trained on language-image corpora exhibit strong compositionality, including the ability to synthesize temporally coherent grid images. This suggests a latent capacity for video-like generation even without explicit temporal modeling. We explore whether such models can serve as visual planners for robots when lightly adapted using LoRA finetuning. We propose a two-part framework that includes: (1) text-conditioned generation, which uses a language instruction and the first frame, and (2) trajectory-conditioned generation, which uses a 2D trajectory overlay and the same initial frame. Experiments on the Jaco Play dataset, Bridge V2, and the RT1 dataset show that both modes produce smooth, coherent robot videos aligned with their respective conditions. Our findings indicate that pretrained image generators encode transferable temporal priors and can function as video-like robotic planners under minimal supervision. Code is released at \href{https://github.com/pangye202264690373/Image-Generation-as-a-Visual-Planner-for-Robotic-Manipulation}{https://github.com/pangye202264690373/Image-Generation-as-a-Visual-Planner-for-Robotic-Manipulation}.

Method: Three-stage framework: 1) Cross-temporal camera alignment for pose estimation across timestamps, 2) Interference-based confidence initialization to identify unchanged regions between timestamps, 3) Progressive cross-temporal optimization that iteratively integrates historical prior information into 3D scenes using 3D Gaussian Splatting.

Result: Experimental results show significant improvements over baseline methods in reconstruction quality and data efficiency. The method supports non-continuous capture, enabling scene updates with new sparse views and recovery of past scenes from limited data.

Conclusion: Cross-Temporal 3DGS is a promising solution for scene versioning, cross-temporal digital twins, and long-term spatial documentation, demonstrating potential for temporal change analysis using only sparse images that can later be reconstructed into detailed 3D representations.

Abstract: Maintaining consistent 3D scene representations over time is a significant challenge in computer vision. Updating 3D scenes from sparse-view observations is crucial for various real-world applications, including urban planning, disaster assessment, and historical site preservation, where dense scans are often unavailable or impractical. In this paper, we propose Cross-Temporal 3D Gaussian Splatting (Cross-Temporal 3DGS), a novel framework for efficiently reconstructing and updating 3D scenes across different time periods, using sparse images and previously captured scene priors. Our approach comprises three stages: 1) Cross-temporal camera alignment for estimating and aligning camera poses across different timestamps; 2) Interference-based confidence initialization to identify unchanged regions between timestamps, thereby guiding updates; and 3) Progressive cross-temporal optimization, which iteratively integrates historical prior information into the 3D scene to enhance reconstruction quality. Our method supports non-continuous capture, enabling not only updates using new sparse views to refine existing scenes, but also recovering past scenes from limited data with the help of current captures. Furthermore, we demonstrate the potential of this approach to achieve temporal changes using only sparse images, which can later be reconstructed into detailed 3D representations as needed. Experimental results show significant improvements over baseline methods in reconstruction quality and data efficiency, making this approach a promising solution for scene versioning, cross-temporal digital twins, and long-term spatial documentation.

Method: Proposes SAIDO framework with two key components: 1) Scene-Awareness-Based Expert Module (SAEM) that dynamically identifies new scenes using VLLMs and allocates independent expert modules for each scene, 2) Importance-Guided Dynamic Optimization Mechanism (IDOM) that optimizes neurons through importance-guided gradient projection to balance plasticity and stability while mitigating catastrophic forgetting.

Result: Outperforms current SOTA in continual learning tasks with 44.22% relative reduction in average detection error rate and 40.57% reduction in forgetting rate. On open-world datasets, improves average detection accuracy by 9.47% compared to current SOTA.

Conclusion: SAIDO effectively addresses the generalization challenge in AI-generated image detection by combining scene-aware expert allocation with importance-guided optimization, achieving superior performance in both stability and plasticity for continual learning scenarios.

Abstract: The widespread misuse of image generation technologies has raised security concerns, driving the development of AI-generated image detection methods. However, generalization has become a key challenge and open problem: existing approaches struggle to adapt to emerging generative methods and content types in real-world scenarios. To address this issue, we propose a Scene-Aware and Importance-Guided Dynamic Optimization detection framework with continual learning (SAIDO). Specifically, we design Scene-Awareness-Based Expert Module (SAEM) that dynamically identifies and incorporates new scenes using VLLMs. For each scene, independent expert modules are dynamically allocated, enabling the framework to capture scene-specific forgery features better and enhance cross-scene generalization. To mitigate catastrophic forgetting when learning from multiple image generative methods, we introduce Importance-Guided Dynamic Optimization Mechanism (IDOM), which optimizes each neuron through an importance-guided gradient projection strategy, thereby achieving an effective balance between model plasticity and stability. Extensive experiments on continual learning tasks demonstrate that our method outperforms the current SOTA method in both stability and plasticity, achieving 44.22% and 40.57% relative reductions in average detection error rate and forgetting rate, respectively. On open-world datasets, it improves the average detection accuracy by 9.47% compared to the current SOTA method.

Method: Proposes a hybrid approach: 1) Uses 3D generative models to create high-fidelity meshes of scene elements, 2) Applies semantic-aware deformation (rigid transformations for objects, LBS-based deformation for humans), and 3) Performs Gaussian Splatting optimization to refine element alignments in the dynamic scene.

Result: Outperforms state-of-the-art methods on the HOI-M3 dataset (the only dataset with multi-human, multi-object interactions), producing better surface reconstruction that maintains object structures under severe occlusion with multi-view and temporal consistency.

Conclusion: The hybrid approach effectively addresses challenges in multi-human, multi-object dynamic scene reconstruction by combining strengths of different techniques, enabling robust geometry modeling even under severe occlusion conditions.

Abstract: Real-world human-built environments are highly dynamic, involving multiple humans and their complex interactions with surrounding objects. While 3D geometry modeling of such scenes is crucial for applications like AR/VR, gaming, and embodied AI, it remains underexplored due to challenges like diverse motion patterns and frequent occlusions. Beyond novel view rendering, 3D Gaussian Splatting (GS) has demonstrated remarkable progress in producing detailed, high-quality surface geometry with fast optimization of the underlying structure. However, very few GS-based methods address multihuman, multiobject scenarios, primarily due to the above-mentioned inherent challenges. In a monocular setup, these challenges are further amplified, as maintaining structural consistency under severe occlusion becomes difficult when the scene is optimized solely based on GS-based rendering loss. To tackle the challenges of such a multihuman, multiobject dynamic scene, we propose a hybrid approach that effectively combines the advantages of 1) 3D generative models for generating high-fidelity meshes of the scene elements, 2) Semantic-aware deformation, \ie rigid transformation of the rigid objects and LBS-based deformation of the humans, and mapping of the deformed high-fidelity meshes in the dynamic scene, and 3) GS-based optimization of the individual elements for further refining their alignments in the scene. Such a hybrid approach helps maintain the object structures even under severe occlusion and can produce multiview and temporally consistent geometry. We choose HOI-M3 for evaluation, as, to the best of our knowledge, this is the only dataset featuring multihuman, multiobject interactions in a dynamic scene. Our method outperforms the state-of-the-art method in producing better surface reconstruction of such scenes.

Method: NeuroVolve optimizes a neural objective function in the embedding space of a pretrained vision-language model. Images are generated by programming neural objectives that activate/deactivate single or multiple brain regions together. The framework tracks optimization steps to reveal semantic trajectories through embedding space.

Result: The framework successfully recovers known selectivity for individual brain regions and synthesizes coherent scenes satisfying complex multi-region constraints. It generates both low-level and semantic feature-specific stimuli for single ROIs, as well as stimuli aligned to curated neural objectives including co-activation and decorrelation between regions. It captures subject-specific preferences for personalized brain-driven synthesis.

Conclusion: NeuroVolve unifies brain-guided image editing and preferred stimulus generation in a single process, exposing cooperative and antagonistic tuning relationships between brain regions. It offers interpretable constraints for mapping, analyzing, and probing neural representations of visual information, supporting personalized brain-driven synthesis.

Abstract: What visual information is encoded in individual brain regions, and how do distributed patterns combine to create their neural representations? Prior work has used generative models to replicate known category selectivity in isolated regions (e.g., faces in FFA), but these approaches offer limited insight into how regions interact during complex, naturalistic vision. We introduce NeuroVolve, a generative framework that provides brain-guided image synthesis via optimization of a neural objective function in the embedding space of a pretrained vision-language model. Images are generated under the guidance of a programmable neural objective, i.e., activating or deactivating single regions or multiple regions together. NeuroVolve is validated by recovering known selectivity for individual brain regions, while expanding to synthesize coherent scenes that satisfy complex, multi-region constraints. By tracking optimization steps, it reveals semantic trajectories through embedding space, unifying brain-guided image editing and preferred stimulus generation in a single process. We show that NeuroVolve can generate both low-level and semantic feature-specific stimuli for single ROIs, as well as stimuli aligned to curated neural objectives. These include co-activation and decorrelation between regions, exposing cooperative and antagonistic tuning relationships. Notably, the framework captures subject-specific preferences, supporting personalized brain-driven synthesis and offering interpretable constraints for mapping, analyzing, and probing neural representations of visual information.

Method: DAAAM introduces an optimization-based frontend that infers detailed semantic descriptions from localized captioning models (like DAM) using batch processing to speed up inference. It builds hierarchical 4D scene graphs that serve as globally consistent spatio-temporal memory representations.

Result: DAAAM achieves state-of-the-art results: improves OC-NaVQA question accuracy by 53.6%, position errors by 21.9%, temporal errors by 21.6%, and SG3D task grounding accuracy by 27.8% over competitive baselines.

Conclusion: DAAAM provides a novel spatio-temporal memory framework that enables real-time 4D scene understanding with detailed semantic descriptions, effectively bridging the gap between rich open-vocabulary scene understanding and real-time performance for robotics and vision applications.

Abstract: Computer vision and robotics applications ranging from augmented reality to robot autonomy in large-scale environments require spatio-temporal memory frameworks that capture both geometric structure for accurate language-grounding as well as semantic detail. Existing methods face a tradeoff, where producing rich open-vocabulary descriptions comes at the expense of real-time performance when these descriptions have to be grounded in 3D. To address these challenges, we propose Describe Anything, Anywhere, at Any Moment (DAAAM), a novel spatio-temporal memory framework for large-scale and real-time 4D scene understanding. DAAAM introduces a novel optimization-based frontend to infer detailed semantic descriptions from localized captioning models, such as the Describe Anything Model (DAM), leveraging batch processing to speed up inference by an order of magnitude for online processing. It leverages such semantic understanding to build a hierarchical 4D scene graph (SG), which acts as an effective globally spatially and temporally consistent memory representation. DAAAM constructs 4D SGs with detailed, geometrically grounded descriptions while maintaining real-time performance. We show that DAAAM’s 4D SG interfaces well with a tool-calling agent for inference and reasoning. We thoroughly evaluate DAAAM in the complex task of spatio-temporal question answering on the NaVQA benchmark and show its generalization capabilities for sequential task grounding on the SG3D benchmark. We further curate an extended OC-NaVQA benchmark for large-scale and long-time evaluations. DAAAM achieves state-of-the-art results in both tasks, improving OC-NaVQA question accuracy by 53.6%, position errors by 21.9%, temporal errors by 21.6%, and SG3D task grounding accuracy by 27.8% over the most competitive baselines, respectively. We release our data and code open-source.

Method: Created curated ‘Yoga-16’ dataset addressing limitations of existing datasets. Systematically evaluated three deep learning architectures (VGG16, ResNet50, Xception) using three input modalities: direct images, MediaPipe Pose skeleton images, and YOLOv8 Pose skeleton images. Used Grad-CAM for interpretability analysis and cross validation.

Result: Skeleton-based representations outperform raw image inputs. Highest accuracy of 96.09% achieved by VGG16 with MediaPipe Pose skeleton input. Provided interpretability insights through Grad-CAM analysis.

Conclusion: Skeleton-based approaches are superior for yoga pose classification, with VGG16+MediaPipe being the most effective combination. The Yoga-16 dataset and systematic benchmarking provide valuable resources for future research in automated yoga pose assessment.

Abstract: Yoga is a popular form of exercise worldwide due to its spiritual and physical health benefits, but incorrect postures can lead to injuries. Automated yoga pose classification has therefore gained importance to reduce reliance on expert practitioners. While human pose keypoint extraction models have shown high potential in action recognition, systematic benchmarking for yoga pose recognition remains limited, as prior works often focus solely on raw images or a single pose extraction model. In this study, we introduce a curated dataset, ‘Yoga-16’, which addresses limitations of existing datasets, and systematically evaluate three deep learning architectures (VGG16, ResNet50, and Xception) using three input modalities (direct images, MediaPipe Pose skeleton images, and YOLOv8 Pose skeleton images). Our experiments demonstrate that skeleton-based representations outperform raw image inputs, with the highest accuracy of 96.09% achieved by VGG16 with MediaPipe Pose skeleton input. Additionally, we provide interpretability analysis using Grad-CAM, offering insights into model decision-making for yoga pose classification with cross validation analysis.

Method: Proposes SatireDecoder framework with: 1) Multi-agent system performing visual cascaded decoupling to decompose images into fine-grained local and global semantic representations; 2) Chain-of-thought reasoning guided by uncertainty analysis to break down satire comprehension into sequential subtasks with minimized uncertainty.

Result: SatireDecoder significantly improves interpretive accuracy while reducing hallucinations. Experimental results show it outperforms existing baselines in comprehending visual satire.

Conclusion: The framework offers a promising direction for vision-language reasoning in nuanced, high-level semantic tasks like satire comprehension, addressing limitations of current models through training-free, structured decomposition and reasoning approaches.

Abstract: Satire, a form of artistic expression combining humor with implicit critique, holds significant social value by illuminating societal issues. Despite its cultural and societal significance, satire comprehension, particularly in purely visual forms, remains a challenging task for current vision-language models. This task requires not only detecting satire but also deciphering its nuanced meaning and identifying the implicated entities. Existing models often fail to effectively integrate local entity relationships with global context, leading to misinterpretation, comprehension biases, and hallucinations. To address these limitations, we propose SatireDecoder, a training-free framework designed to enhance satirical image comprehension. Our approach proposes a multi-agent system performing visual cascaded decoupling to decompose images into fine-grained local and global semantic representations. In addition, we introduce a chain-of-thought reasoning strategy guided by uncertainty analysis, which breaks down the complex satire comprehension process into sequential subtasks with minimized uncertainty. Our method significantly improves interpretive accuracy while reducing hallucinations. Experimental results validate that SatireDecoder outperforms existing baselines in comprehending visual satire, offering a promising direction for vision-language reasoning in nuanced, high-level semantic tasks.

Method: MedCT-VLM uses parameter-efficient adaptation of CT-CLIP (trained on 25,692 chest CT volumes) via Low-Rank Adaptation (LoRA). Instead of fine-tuning all 440M parameters, it inserts low-rank decomposition matrices into attention layers of both vision and text encoders, training only 1.67M parameters (0.38% of total).

Result: LoRA fine-tuning improves mean AUROC from 61.3% to 68.9% (+7.6 pp), accuracy from 67.2% to 73.6% (+6.4 pp), and macro-F1 from 32.1% to 36.9% (+4.8 pp) on zero-shot classification across 18 thoracic pathologies.

Conclusion: Parameter-efficient methods like LoRA can effectively transfer large-scale pretraining to downstream medical imaging tasks, particularly valuable for zero-shot scenarios where labeled data is scarce.

Abstract: Foundation models trained via vision-language pretraining have demonstrated strong zero-shot capabilities across diverse image domains, yet their application to volumetric medical imaging remains limited. We introduce MedCT-VLM: Medical CT Vision-Language Model, a parameter-efficient vision-language framework designed to adapt large-scale CT foundation models for downstream clinical tasks. MedCT-VLM uses a parameter-efficient approach to adapt CT-CLIP, a contrastive vision-language model trained on 25,692 chest CT volumes, for multi-label pathology classification using Low-Rank Adaptation (LoRA). Rather than fine-tuning the model’s 440 M parameters directly, we insert low-rank decomposition matrices into attention layers of both vision and text encoders, training only 1.67M parameters (0.38% of total). We evaluate on zero-shot classification across 18 thoracic pathologies, where the model must align CT embeddings with unseen text prompts at inference without task-specific training. LoRA fine-tuning improves mean AUROC from 61.3% to 68.9% (+7.6 pp), accuracy from 67.2% to 73.6% (+6.4 pp), and macro-F1 from 32.1% to 36.9% (+4.8 pp). These results demonstrate that parameter-efficient methods can effectively transfer large-scale pretraining to downstream medical imaging tasks, particularly for zero-shot scenarios where labeled data is scarce.

Method: Used 582 labeled images with dynamic augmentation to train and evaluate five deep learning models. Tested generalizability on an oak dataset from Oregon State University, and conducted exploratory analysis by training worst-performing model on additional 64 images.

Result: Swin Transformer achieved highest accuracy (0.94) with excellent fine segmentation. YOLOv9 performed well for bounding boxes but struggled with boundaries. Mask R-CNN improved from 0.45 to 0.80 IoU with NMS. Generalizability testing showed varying performance across models.

Conclusion: Deep learning shows strong potential for automated pith detection, with model selection depending on dataset characteristics and application needs. Swin Transformer performed best overall, while techniques like NMS can significantly improve underperforming models.

Abstract: Pith detection in tree cross-sections is essential for forestry and wood quality analysis but remains a manual, error-prone task. This study evaluates deep learning models – YOLOv9, U-Net, Swin Transformer, DeepLabV3, and Mask R-CNN – to automate the process efficiently. A dataset of 582 labeled images was dynamically augmented to improve generalization. Swin Transformer achieved the highest accuracy (0.94), excelling in fine segmentation. YOLOv9 performed well for bounding box detection but struggled with boundary precision. U-Net was effective for structured patterns, while DeepLabV3 captured multi-scale features with slight boundary imprecision. Mask R-CNN initially underperformed due to overlapping detections, but applying Non-Maximum Suppression (NMS) improved its IoU from 0.45 to 0.80. Generalizability was next tested using an oak dataset of 11 images from Oregon State University’s Tree Ring Lab. Additionally, for exploratory analysis purposes, an additional dataset of 64 labeled tree cross-sections was used to train the worst-performing model to see if this would improve its performance generalizing to the unseen oak dataset. Key challenges included tensor mismatches and boundary inconsistencies, addressed through hyperparameter tuning and augmentation. Our results highlight deep learning’s potential for tree cross-section pith detection, with model choice depending on dataset characteristics and application needs.

Method: 1) Use DCGANs for per-class data augmentation to resolve class imbalance; 2) Train fine-tuned ResNet-50 classifier on augmented dataset for 7-class classification; 3) Integrate LIME and SHAP XAI techniques to provide transparency and verify predictions are based on clinically relevant features.

Result: Achieved 92.50% overall accuracy and 98.82% Macro-AUC, outperforming prior benchmark architectures. Successfully validated a verifiable framework combining high performance with clinical interpretability.

Conclusion: The work demonstrates a trustworthy CAD system that addresses both performance and interpretability challenges. Future research should focus on enhancing discrimination for critical categories like Melanoma NOS (F1-Score: 0.8602) to further improve clinical utility.

Abstract: Accurate and timely diagnosis of multi-class skin lesions is hampered by subjective methods, inherent data imbalance in datasets like HAM10000, and the “black box” nature of Deep Learning (DL) models. This study proposes a trustworthy and highly accurate Computer-Aided Diagnosis (CAD) system to overcome these limitations. The approach utilizes Deep Convolutional Generative Adversarial Networks (DCGANs) for per class data augmentation to resolve the critical class imbalance problem. A fine-tuned ResNet-50 classifier is then trained on the augmented dataset to classify seven skin disease categories. Crucially, LIME and SHAP Explainable AI (XAI) techniques are integrated to provide transparency by confirming that predictions are based on clinically relevant features like irregular morphology. The system achieved a high overall Accuracy of 92.50 % and a Macro-AUC of 98.82 %, successfully outperforming various prior benchmarked architectures. This work successfully validates a verifiable framework that combines high performance with the essential clinical interpretability required for safe diagnostic deployment. Future research should prioritize enhancing discrimination for critical categories, such as Melanoma NOS (F1-Score is 0.8602).

Method: Used YOLOv5 instance segmentation algorithms (Nano, Small, Medium, Large, XLarge variants) on ultrasound images, evaluated across two dataset versions with and without doppler images.

Result: YOLOv5-Large achieved highest performance with 91% dice score and 0.87 mAP on dataset including doppler images. Doppler images significantly improved segmentation performance across all models (e.g., YOLOv5-Small improved from 79% to higher scores).

Conclusion: Instance segmentation with YOLOv5 provides effective real-time approach for thyroid nodule detection, with potential clinical applications in automated diagnostic systems, and doppler images (typically excluded by physicians) can significantly improve performance.

Abstract: The increasing prevalence of thyroid cancer globally has led to the development of various computer-aided detection methods. Accurate segmentation of thyroid nodules is a critical first step in the development of AI-assisted clinical decision support systems. This study focuses on instance segmentation of thyroid nodules using YOLOv5 algorithms on ultrasound images. We evaluated multiple YOLOv5 variants (Nano, Small, Medium, Large, and XLarge) across two dataset versions, with and without doppler images. The YOLOv5-Large algorithm achieved the highest performance with a dice score of 91% and mAP of 0.87 on the dataset including doppler images. Notably, our results demonstrate that doppler images, typically excluded by physicians, can significantly improve segmentation performance. The YOLOv5-Small model achieved 79% dice score when doppler images were excluded, while including them improved performance across all model variants. These findings suggest that instance segmentation with YOLOv5 provides an effective real-time approach for thyroid nodule detection, with potential clinical applications in automated diagnostic systems.

Method: Proposes GAT-ADA: ResNet-50 backbone + batch-level Graph Attention Network (GAT) to model inter-sample relations via sparse ring graphs, combined with adversarial learning (Gradient Reversal Layer) and statistical alignment (CORAL and MMD) for domain adaptation.

Result: Achieves 74.39% mean cross-domain accuracy across multiple datasets. Specifically, reaches 98.0% accuracy on RAF-DB to FER2013 adaptation, representing ~36-point improvement over best baseline with same backbone and preprocessing.

Conclusion: GAT-ADA effectively addresses domain shift in facial expression recognition by combining graph attention for sample relations with adversarial and statistical domain alignment, achieving state-of-the-art performance.

Abstract: Cross-domain facial expression recognition (CD-FER) remains difficult due to severe domain shift between training and deployment data. We propose Graph-Attention Network with Adversarial Domain Alignment (GAT-ADA), a hybrid framework that couples a ResNet-50 as backbone with a batch-level Graph Attention Network (GAT) to model inter-sample relations under shift. Each mini-batch is cast as a sparse ring graph so that attention aggregates cross-sample cues that are informative for adaptation. To align distributions, GAT-ADA combines adversarial learning via a Gradient Reversal Layer (GRL) with statistical alignment using CORAL and MMD. GAT-ADA is evaluated under a standard unsupervised domain adaptation protocol: training on one labeled source (RAF-DB) and adapting to multiple unlabeled targets (CK+, JAFFE, SFEW 2.0, FER2013, and ExpW). GAT-ADA attains 74.39% mean cross-domain accuracy. On RAF-DB to FER2013, it reaches 98.0% accuracy, corresponding to approximately a 36-point improvement over the best baseline we re-implemented with the same backbone and preprocessing.

Method: CF-ViM first performs coarse-grained inference using large patches to reduce token length. When prediction confidence is low, selected regions are re-processed at finer resolution to recover visual details. This dynamic resolution assignment adapts computation to image complexity.

Result: Experiments on ImageNet show CF-ViM outperforms both baseline Vision Mamba and state-of-the-art token reduction techniques in accuracy and efficiency.

Conclusion: CF-ViM provides an adaptive framework that allocates computation according to image complexity, ensuring efficient processing without compromising essential visual information, making Vision Mamba more practical for real-world applications.

Abstract: Vision Mamba has emerged as a promising and efficient alternative to Vision Transformers, yet its efficiency remains fundamentally constrained by the number of input tokens. Existing token reduction approaches typically adopt token pruning or merging to reduce computation. However, they inherently lead to information loss, as they discard or compress token representations. This problem is exacerbated when applied uniformly to fine-grained token representations across all images, regardless of visual complexity. We observe that not all inputs require fine-grained processing. Simple images can be effectively handled at coarse resolution, while only complex ones may warrant refinement. Based on this insight, we propose \textit{Coarse-to-Fine Vision Mamba (CF-ViM)}, an adaptive framework for efficient inference. CF-ViM first performs coarse-grained inference by dividing the input image into large patches, significantly reducing the token length and computation. When the model’s prediction confidence is low, selected regions are re-processed at a finer resolution to recover critical visual details with minimal additional cost. This dynamic resolution assignment strategy allows CF-ViM to allocate computation adaptively according to image complexity, ensuring efficient processing without compromising essential visual information. Experiments on ImageNet demonstrate that CF-ViM outperforms both the baseline Vision Mamba and state-of-the-art token reduction techniques in terms of accuracy and efficiency.

Method: Leverage writer information from NIST digit images to create multi-digit writer (MDW) benchmark data sets that simulate real-world scenarios where multiple digits come from the same writer.

Result: Classifiers that perform well on isolated digits often perform poorly on multi-digit number recognition. The MDW benchmarks reveal this performance gap and provide task-specific metrics that better align with real-world impact.

Conclusion: Additional advances beyond isolated digit classification are needed for real number recognition problems. MDW benchmarks enable development of methods that leverage task-specific knowledge to achieve better performance than individual digit classification approaches.

Abstract: Isolated digit classification has served as a motivating problem for decades of machine learning research. In real settings, numbers often occur as multiple digits, all written by the same person. Examples include ZIP Codes, handwritten check amounts, and appointment times. In this work, we leverage knowledge about the writers of NIST digit images to create more realistic benchmark multi-digit writer (MDW) data sets. As expected, we find that classifiers may perform well on isolated digits yet do poorly on multi-digit number recognition. If we want to solve real number recognition problems, additional advances are needed. The MDW benchmarks come with task-specific performance metrics that go beyond typical error calculations to more closely align with real-world impact. They also create opportunities to develop methods that can leverage task-specific knowledge to improve performance well beyond that of individual digit classification methods.

Method: Leverages Multimodal Diffusion Transformer (MM-DiT) and 4D Gaussian Splatting (4DGS). Uses Spatio-Temporal Sub-Grid Attention (STGA) for locally consistent cross-view and temporal fusion, and Context Token Propagation (CTP) for global propagation via token inheritance and optical-flow-guided token replacement. Directly optimizes pre-trained source 4DGS without additional training.

Result: Extensive experiments on multi-view video dataset DyNeRF demonstrate superior editing fidelity and both multi-view and temporal consistency compared to prior approaches.

Conclusion: Dynamic-eDiTor enables seamless, globally consistent text-driven 4D scene editing without requiring additional training, addressing key challenges in maintaining consistency across space and time during editing.

Abstract: Recent progress in 4D representations, such as Dynamic NeRF and 4D Gaussian Splatting (4DGS), has enabled dynamic 4D scene reconstruction. However, text-driven 4D scene editing remains under-explored due to the challenge of ensuring both multi-view and temporal consistency across space and time during editing. Existing studies rely on 2D diffusion models that edit frames independently, often causing motion distortion, geometric drift, and incomplete editing. We introduce Dynamic-eDiTor, a training-free text-driven 4D editing framework leveraging Multimodal Diffusion Transformer (MM-DiT) and 4DGS. This mechanism consists of Spatio-Temporal Sub-Grid Attention (STGA) for locally consistent cross-view and temporal fusion, and Context Token Propagation (CTP) for global propagation via token inheritance and optical-flow-guided token replacement. Together, these components allow Dynamic-eDiTor to perform seamless, globally consistent multi-view video without additional training and directly optimize pre-trained source 4DGS. Extensive experiments on multi-view video dataset DyNeRF demonstrate that our method achieves superior editing fidelity and both multi-view and temporal consistency prior approaches. Project page for results and code: https://di-lee.github.io/dynamic-eDiTor/

Method: Introduces FoundationGait, a scalable self-supervised pretraining framework with largest version having 0.13B parameters, pretrained on 12 public gait datasets comprising over 2M walking sequences. Uses unified framework for diverse gait tasks.

Result: Achieves 48.0% zero-shot rank-1 accuracy on challenging in-the-wild Gait3D dataset (1,000 test subjects) and 64.5% on largest in-the-lab OU-MVLP dataset (5,000+ test subjects). Performs robustly across wide spectrum of gait datasets, conditions, tasks, and input modalities.

Conclusion: FoundationGait sets a new milestone in robust gait recognition, demonstrating that scalable, unified foundation models can effectively address both scalability and generalization challenges in gait understanding.

Abstract: Gait patterns play a critical role in human identification and healthcare analytics, yet current progress remains constrained by small, narrowly designed models that fail to scale or generalize. Building a unified gait foundation model requires addressing two longstanding barriers: (a) Scalability. Why have gait models historically failed to follow scaling laws? (b) Generalization. Can one model serve the diverse gait tasks that have traditionally been studied in isolation? We introduce FoundationGait, the first scalable, self-supervised pretraining framework for gait understanding. Its largest version has nearly 0.13 billion parameters and is pretrained on 12 public gait datasets comprising over 2 million walking sequences. Extensive experiments demonstrate that FoundationGait, with or without fine-tuning, performs robustly across a wide spectrum of gait datasets, conditions, tasks (e.g., human identification, scoliosis screening, depression prediction, and attribute estimation), and even input modality. Notably, it achieves 48.0% zero-shot rank-1 accuracy on the challenging in-the-wild Gait3D dataset (1,000 test subjects) and 64.5% on the largest in-the-lab OU-MVLP dataset (5,000+ test subjects), setting a new milestone in robust gait recognition. Coming code and model: https://github.com/ShiqiYu/OpenGait.

Method: AFD maps videos to slowly varying affordance tokens forming a shared time-aligned substrate, stabilized via weak alignment and teacher consistency. A lightweight query-routed, conflict-aware scheduler concentrates adaptation and grows capacity only when needed, using question-only replay for training.

Result: Achieves state-of-the-art across protocols: 51.6% average accuracy with -1.8% forgetting on domain-incremental VideoQA, ViLCo R@1@0.5 of 29.6% (MQ) and 20.7% (NLQ) with 18.4% stAP@0.25 (VQ), and 39.5% accuracy with -1.6% forgetting on time-incremental iVQA.

Conclusion: AFD provides an explicit, interpretable split between a stable interaction-centered substrate and targeted adaptation, offering a principled solution for continual video-language learning under realistic constraints.

Abstract: Continual learning for video–language understanding is increasingly important as models face non-stationary data, domains, and query styles, yet prevailing solutions blur what should stay stable versus what should adapt, rely on static routing/capacity, or require replaying past videos. We aim to explicitly specify where stability lives and where plasticity should be focused under realistic memory and privacy constraints. We introduce Affordance-First Decomposition (AFD): videos are mapped to slowly varying affordance tokens that form a shared, time-aligned substrate, while a lightweight, query-routed, conflict-aware scheduler concentrates adaptation and grows capacity only when needed. The substrate is stabilized via weak alignment and teacher consistency, and training uses question-only replay. AFD achieves state-of-the-art across protocols: 51.6% average accuracy with -1.8% forgetting on domain-incremental VideoQA, ViLCo R@1@0.5 of 29.6% (MQ) and 20.7% (NLQ) with 18.4% stAP@0.25 (VQ), and 39.5% accuracy with -1.6% forgetting on time-incremental iVQA. Overall, AFD offers an explicit, interpretable split between a stable interaction-centered substrate and targeted adaptation.

Method: The architecture uses a progressive refinement process starting with frequency-domain inversion for initial deblurring, followed by multiple refinement stages that predict and apply residual corrections to suppress artifacts and restore details. It includes an optional angle detection module for end-to-end training.

Result: The method demonstrates efficiency through experiments on both synthetic and real-life images, with code, models, and dataset links made publicly available.

Conclusion: CAR-net provides an effective solution for rotational motion blur deblurring in semi-blind scenarios, combining frequency-domain initialization with cascade refinement and optional angle detection for robust performance.

Abstract: We propose a new neural network architecture called CAR-net (CAscade Refinement Network) to deblur images that are subject to rotational motion blur. Our architecture is specifically designed for the semi-blind scenarios where only noisy information of the rotational motion blur angle is available. The core of our approach is progressive refinement process that starts with an initial deblurred estimate obtained from frequency-domain inversion; A series of refinement stages take the current deblurred image to predict and apply residual correction to the current estimate, progressively suppressing artifacts and restoring fine details. To handle parameter uncertainty, our architecture accommodates an optional angle detection module which can be trained end-to-end with refinement modules. We provide a detailed description of our architecture and illustrate its efficiency through experiments using both synthetic and real-life images. Our code and model as well as the links to the datasets are available at https://github.com/tony123105/CAR-Net

Method: 1) Use on-policy data instead of off-policy data for better hallucination mitigation; 2) Train a binary hallucination classifier to provide clean annotations; 3) Implement iterative Direct Preference Optimization (DPO) with dynamic sample reweighting scheme.

Result: Reduces hallucination rate of LLaVA-1.5-7B on MMHalBench by 50.8% and average hallucination rate on Object HalBench by 79.5%. Enables LLaVA-1.5-13B to surpass GPT-4V performance.

Conclusion: The proposed approach effectively mitigates hallucinations in LVLMs by leveraging clean on-policy data with binary annotations and robust iterative DPO, demonstrating superior performance over existing methods.

Abstract: Recently, large vision-language models (LVLMs) have risen to be a promising approach for multimodal tasks. However, principled hallucination mitigation remains a critical challenge.In this work, we first analyze the data generation process in LVLM hallucination mitigation and affirm that on-policy data significantly outperforms off-policy data, which thus calls for efficient and reliable preference annotation of on-policy data. We then point out that, existing annotation methods introduce additional hallucination in training samples, which may enhance the model’s hallucination patterns, to address this problem, we propose training a hallucination classifier giving binary annotations, which guarantee clean chosen samples for the subsequent alignment. To further harness of the power of on-policy data, we design a robust iterative direct preference optimization (DPO) algorithm adopting a dynamic sample reweighting scheme. We conduct comprehensive experiments on three benchmarks with comparison to 8 state-of-the-art baselines. In particular, our approach reduces the hallucination rate of LLaVA-1.5-7B on MMHalBench by 50.8% and the average hallucination rate on Object HalBench by 79.5%; more significantly, our method fully taps into the potential of open-source models, enabling LLaVA-1.5-13B to even surpass the performance of GPT-4V.

Method: Utilizes deep learning-based computer vision models including convolutional neural networks (CNNs), transformers, and hybrid attention architectures to automatically extract complex spatial, morphological, and temporal patterns from multimodal imaging data (MRI, CT, PET, mammography, histopathology, ultrasound). Integrates multimodal imaging with radiogenomics to link quantitative imaging features with genomics, transcriptomics, and epigenetic biomarkers.

Result: Models surpass traditional radiological assessment by identifying subtle tissue abnormalities and tumor microenvironment variations invisible to the human eye. Radiogenomic fusion enables prediction of tumor genotype, immune response, molecular subtypes, and treatment resistance without invasive biopsies.

Conclusion: Deep learning and radiogenomics represent a transformative paradigm for personalized oncology, enabling early cancer detection and non-invasive characterization of tumors through advanced analysis of multimodal imaging data.

Abstract: Early cancer detection remains one of the most critical challenges in modern healthcare, where delayed diagnosis significantly reduces survival outcomes. Recent advancements in artificial intelligence, particularly deep learning, have enabled transformative progress in medical imaging analysis. Deep learning-based computer vision models, such as convolutional neural networks (CNNs), transformers, and hybrid attention architectures, can automatically extract complex spatial, morphological, and temporal patterns from multimodal imaging data including MRI, CT, PET, mammography, histopathology, and ultrasound. These models surpass traditional radiological assessment by identifying subtle tissue abnormalities and tumor microenvironment variations invisible to the human eye. At a broader scale, the integration of multimodal imaging with radiogenomics linking quantitative imaging features with genomics, transcriptomics, and epigenetic biomarkers has introduced a new paradigm for personalized oncology. This radiogenomic fusion allows the prediction of tumor genotype, immune response, molecular subtypes, and treatment resistance without invasive biopsies.

Method: Proposes RS-ISRefiner framework with: 1) Adapter-based tuning strategy to preserve Vision Foundation Model representations while learning remote sensing-specific features, 2) Hybrid attention mechanism combining convolutional local modeling with Transformer-based global reasoning for scale diversity and scene complexity, 3) Improved probability map modulation scheme incorporating historical user interactions for stable iterative refinement.

Result: Comprehensive experiments on six remote sensing datasets (iSAID, ISPRS Potsdam, SandBar, NWPU, LoveDA Urban, WHUBuilding) show RS-ISRefiner consistently outperforms state-of-the-art IIS methods in segmentation accuracy, efficiency, and interaction cost.

Conclusion: RS-ISRefiner demonstrates effectiveness and generalizability for high-quality instance segmentation in practical remote sensing scenarios, making it suitable for precise annotation tasks in remote sensing imagery.

Abstract: Interactive image segmentation(IIS) plays a critical role in generating precise annotations for remote sensing imagery, where objects often exhibit scale variations, irregular boundaries and complex backgrounds. However, existing IIS methods, primarily designed for natural images, struggle to generalize to remote sensing domains due to limited annotated data and computational overhead. To address these challenges, we proposed RS-ISRefiner, a novel click-based IIS framework tailored for remote sensing images. The framework employs an adapter-based tuning strategy that preserves the general representations of Vision Foundation Models while enabling efficient learning of remote sensing-specific spatial and boundary characteristics. A hybrid attention mechanism integrating convolutional local modeling with Transformer-based global reasoning enhances robustness against scale diversity and scene complexity. Furthermore, an improved probability map modulation scheme effectively incorporates historical user interactions, yielding more stable iterative refinement and higher boundary fidelity. Comprehensive experiments on six remote sensing datasets, including iSAID, ISPRS Potsdam, SandBar, NWPU, LoveDA Urban and WHUBuilding, demonstrate that RS-ISRefiner consistently outperforms state-of-the-art IIS methods in terms of segmentation accuracy, efficiency and interaction cost. These results confirm the effectiveness and generalizability of our framework, making it highly suitable for high-quality instance segmentation in practical remote sensing scenarios.

Method: Introduces MedGrounder, the first generalized medical phrase grounding model with two-stage training: pre-training on report sentence-anatomy box alignment datasets, then fine-tuning on report sentence-human annotated box datasets.

Result: MedGrounder achieves strong zero-shot transfer and outperforms REC-style and grounded report generation baselines on multi-region and non-groundable phrases, while using far fewer human box annotations. Can be composed with existing report generators to produce grounded reports without retraining.

Conclusion: The paper successfully reformulates medical phrase grounding as a generalized task (GMPG) and demonstrates MedGrounder’s effectiveness in handling real-world radiology report complexities with improved performance and reduced annotation requirements.

Abstract: Medical phrase grounding (MPG) maps textual descriptions of radiological findings to corresponding image regions. These grounded reports are easier to interpret, especially for non-experts. Existing MPG systems mostly follow the referring expression comprehension (REC) paradigm and return exactly one bounding box per phrase. Real reports often violate this assumption. They contain multi-region findings, non-diagnostic text, and non-groundable phrases, such as negations or descriptions of normal anatomy. Motivated by this, we reformulate the task as generalised medical phrase grounding (GMPG), where each sentence is mapped to zero, one, or multiple scored regions. To realise this formulation, we introduce the first GMPG model: MedGrounder. We adopted a two-stage training regime: pre-training on report sentence–anatomy box alignment datasets and fine-tuning on report sentence–human annotated box datasets. Experiments on PadChest-GR and MS-CXR show that MedGrounder achieves strong zero-shot transfer and outperforms REC-style and grounded report generation baselines on multi-region and non-groundable phrases, while using far fewer human box annotations. Finally, we show that MedGrounder can be composed with existing report generators to produce grounded reports without retraining the generator.

Method: Proposes TrajDiff: 1) Uses only raw sensor inputs and future trajectories to construct Gaussian BEV heatmap targets capturing driving modalities; 2) Designs trajectory-oriented BEV encoder without perceptual supervision; 3) Introduces Trajectory-oriented BEV Diffusion Transformer (TB-DiT) that leverages ego-state and predicted TrajBEV features to generate diverse trajectories without handcrafted motion priors.

Result: Achieves 87.5 PDMS on NAVSIM benchmark, state-of-the-art among annotation-free methods. With data scaling, improves to 88.5 PDMS, comparable to advanced perception-based approaches.

Conclusion: TrajDiff successfully establishes a perception annotation-free generative method for end-to-end autonomous driving that achieves competitive performance while enabling exploration of data scaling benefits in annotation-free settings.

Abstract: End-to-end autonomous driving systems directly generate driving policies from raw sensor inputs. While these systems can extract effective environmental features for planning, relying on auxiliary perception tasks, developing perception annotation-free planning paradigms has become increasingly critical due to the high cost of manual perception annotation. In this work, we propose TrajDiff, a Trajectory-oriented BEV Conditioned Diffusion framework that establishes a fully perception annotation-free generative method for end-to-end autonomous driving. TrajDiff requires only raw sensor inputs and future trajectory, constructing Gaussian BEV heatmap targets that inherently capture driving modalities. We design a simple yet effective trajectory-oriented BEV encoder to extract the TrajBEV feature without perceptual supervision. Furthermore, we introduce Trajectory-oriented BEV Diffusion Transformer (TB-DiT), which leverages ego-state information and the predicted TrajBEV features to directly generate diverse yet plausible trajectories, eliminating the need for handcrafted motion priors. Beyond architectural innovations, TrajDiff enables exploration of data scaling benefits in the annotation-free setting. Evaluated on the NAVSIM benchmark, TrajDiff achieves 87.5 PDMS, establishing state-of-the-art performance among all annotation-free methods. With data scaling, it further improves to 88.5 PDMS, which is comparable to advanced perception-based approaches. Our code and model will be made publicly available.

Method: Proposes Multi-GRPO with two orthogonal grouping mechanisms: (1) tree-based trajectories inspired by Monte Carlo Tree Search that create temporal groups by branching at early denoising steps, enabling accurate advantage estimation for early steps via descendant leaves while amortizing computation through shared prefixes, and (2) reward-based grouping that computes advantages for each reward function independently before aggregation to disentangle conflicting signals. Also introduces OCR-Color-10 dataset for multi-objective evaluation.

Result: Multi-GRPO achieves superior stability and alignment performance across single-reward PickScore-25k and multi-objective OCR-Color-10 benchmarks, effectively balancing conflicting objectives.

Conclusion: Multi-GRPO addresses critical limitations of existing GRPO methods through innovative temporal and reward-based grouping mechanisms, demonstrating improved stability and alignment performance for text-to-image models across both single and multi-objective settings.

Abstract: Recently, Group Relative Policy Optimization (GRPO) has shown promising potential for aligning text-to-image (T2I) models, yet existing GRPO-based methods suffer from two critical limitations. (1) \textit{Shared credit assignment}: trajectory-level advantages derived from group-normalized sparse terminal rewards are uniformly applied across timesteps, failing to accurately estimate the potential of early denoising steps with vast exploration spaces. (2) \textit{Reward-mixing}: predefined weights for combining multi-objective rewards (e.g., text accuracy, visual quality, text color)–which have mismatched scales and variances–lead to unstable gradients and conflicting updates. To address these issues, we propose \textbf{Multi-GRPO}, a multi-group advantage estimation framework with two orthogonal grouping mechanisms. For better credit assignment, we introduce tree-based trajectories inspired by Monte Carlo Tree Search: branching trajectories at selected early denoising steps naturally forms \emph{temporal groups}, enabling accurate advantage estimation for early steps via descendant leaves while amortizing computation through shared prefixes. For multi-objective optimization, we introduce \emph{reward-based grouping} to compute advantages for each reward function \textit{independently} before aggregation, disentangling conflicting signals. To facilitate evaluation of multiple objective alignment, we curate \textit{OCR-Color-10}, a visual text rendering dataset with explicit color constraints. Across the single-reward \textit{PickScore-25k} and multi-objective \textit{OCR-Color-10} benchmarks, Multi-GRPO achieves superior stability and alignment performance, effectively balancing conflicting objectives. Code will be publicly available at \href{https://github.com/fikry102/Multi-GRPO}{https://github.com/fikry102/Multi-GRPO}.

Method: 1) PGConv: Uses asymmetric convolutions to strengthen skeleton elements and difference convolutions to extract high-frequency information, with re-parameterization to reduce complexity. 2) MGT: Uses dilated window-based multi-head self-attention with different dilation rates and depth-wise convolutions with different kernel sizes for multi-scale features, plus gate mechanism for enhanced non-linearity. 3) MGTPCN: Combines both components into a joint network for learned image compression.

Result: Experimental results show MGTPCN surpasses state-of-the-art algorithms with a better trade-off between performance and computational complexity.

Conclusion: The proposed MGTPCN framework effectively improves learned image compression by enhancing both local feature extraction (via PGConv) and non-local feature extraction (via MGT), achieving superior performance with manageable complexity.

Abstract: Recently, learned image compression methods have made remarkable achievements, some of which have outperformed the traditional image codec VVC. The advantages of learned image compression methods over traditional image codecs can be largely attributed to their powerful nonlinear transform coding. Convolutional layers and shifted window transformer (Swin-T) blocks are the basic units of neural networks, and their representation capabilities play an important role in nonlinear transform coding. In this paper, to improve the ability of the vanilla convolution to extract local features, we propose a novel prior-guided convolution (PGConv), where asymmetric convolutions (AConvs) and difference convolutions (DConvs) are introduced to strengthen skeleton elements and extract high-frequency information, respectively. A re-parameterization strategy is also used to reduce the computational complexity of PGConv. Moreover, to improve the ability of the Swin-T block to extract non-local features, we propose a novel multi-scale gated transformer (MGT), where dilated window-based multi-head self-attention blocks with different dilation rates and depth-wise convolution layers with different kernel sizes are used to extract multi-scale features, and a gate mechanism is introduced to enhance non-linearity. Finally, we propose a novel joint Multi-scale Gated Transformer and Prior-guided Convolutional Network (MGTPCN) for learned image compression. Experimental results show that our MGTPCN surpasses state-of-the-art algorithms with a better trade-off between performance and complexity.

Method: Combines explicit geometric reasoning with neural semantics using AlphaGo-style search to enforce feasibility, with a fine-tuned vision-language model scoring semantic alignment as reward signals. Uses policy network as heuristic in Monte-Carlo Tree Search, fine-tuned via search-generated plans, and adversarial reward refinement inspired by GANs.

Result: In the Tangram Assembly task, the approach yields higher validity and semantic fidelity than diffusion and auto-regressive baselines, especially as constraints tighten.

Conclusion: The constraint-guided framework successfully addresses the challenges of abstract visual composition by integrating geometric reasoning with neural semantics, outperforming purely statistical methods in generating feasible and semantically-aligned geometric structures.

Abstract: We study abstract visual composition, in which identity is primarily determined by the spatial configuration and relations among a small set of geometric primitives (e.g., parts, symmetry, topology). They are invariant primarily to texture and photorealistic detail. Composing such structures from fixed components under geometric constraints and vague goal specification (such as text) is non-trivial due to combinatorial placement choices, limited data, and discrete feasibility (overlap-free, allowable orientations), which create a sparse solution manifold ill-suited to purely statistical pixel-space generators. We propose a constraint-guided framework that combines explicit geometric reasoning with neural semantics. An AlphaGo-style search enforces feasibility, while a fine-tuned vision-language model scores semantic alignment as reward signals. Our algorithm uses a policy network as a heuristic in Monte-Carlo Tree Search and fine-tunes the network via search-generated plans. Inspired by the Generative Adversarial Network, we use the generated instances for adversarial reward refinement. Over time, the generation should approach the actual data more closely when the reward model cannot distinguish between generated instances and ground-truth. In the Tangram Assembly task, our approach yields higher validity and semantic fidelity than diffusion and auto-regressive baselines, especially as constraints tighten.

Method: Introduces two latent variables to model expert annotation preferences and image boundary ambiguity. Uses variational inference to obtain conditional probabilistic distributions, allowing segmentation outputs to be generated by sampling from these distributions.

Result: Extensive experiments on nasopharyngeal carcinoma (NPC) and lung nodule (LIDC-IDRI) datasets demonstrate state-of-the-art performance, providing segmentation results that are both diverse and expert-personalized.

Conclusion: ProSeg successfully addresses the limitations of previous multi-rater segmentation models by enabling simultaneous diversification and personalization through probabilistic modeling of expert preferences and boundary ambiguity.

Abstract: Medical image segmentation is inherently influenced by data uncertainty, arising from ambiguous boundaries in medical scans and inter-observer variability in diagnosis. To address this challenge, previous works formulated the multi-rater medical image segmentation task, where multiple experts provide separate annotations for each image. However, existing models are typically constrained to either generate diverse segmentation that lacks expert specificity or to produce personalized outputs that merely replicate individual annotators. We propose Probabilistic modeling of multi-rater medical image Segmentation (ProSeg) that simultaneously enables both diversification and personalization. Specifically, we introduce two latent variables to model expert annotation preferences and image boundary ambiguity. Their conditional probabilistic distributions are then obtained through variational inference, allowing segmentation outputs to be generated by sampling from these distributions. Extensive experiments on both the nasopharyngeal carcinoma dataset (NPC) and the lung nodule dataset (LIDC-IDRI) demonstrate that our ProSeg achieves a new state-of-the-art performance, providing segmentation results that are both diverse and expert-personalized. Code can be found in https://github.com/AI4MOL/ProSeg.

Method: The authors introduce FigEdit, a benchmark with over 30,000 samples covering 10 chart types and complex editing instructions. It’s organized into five progressively challenging tasks: single edits, multi edits, conversational edits, visual-guidance-based edits, and style transfer.

Result: Evaluation of state-of-the-art models shows poor performance on scientific figures, as they fail to handle structured transformations. Traditional metrics (SSIM, PSNR) are inadequate for capturing semantic correctness of chart edits.

Conclusion: FigEdit demonstrates limitations of pixel-level manipulation and provides a foundation for developing structure-aware models. The benchmark enables systematic progress in figure editing and encourages research on models that understand both visual and semantic layers of charts.

Abstract: Generative models, such as diffusion and autoregressive approaches, have demonstrated impressive capabilities in editing natural images. However, applying these tools to scientific charts rests on a flawed assumption: a chart is not merely an arrangement of pixels but a visual representation of structured data governed by a graphical grammar. Consequently, chart editing is not a pixel-manipulation task but a structured transformation problem. To address this fundamental mismatch, we introduce \textit{FigEdit}, a large-scale benchmark for scientific figure editing comprising over 30,000 samples. Grounded in real-world data, our benchmark is distinguished by its diversity, covering 10 distinct chart types and a rich vocabulary of complex editing instructions. The benchmark is organized into five distinct and progressively challenging tasks: single edits, multi edits, conversational edits, visual-guidance-based edits, and style transfer. Our evaluation of a range of state-of-the-art models on this benchmark reveals their poor performance on scientific figures, as they consistently fail to handle the underlying structured transformations required for valid edits. Furthermore, our analysis indicates that traditional evaluation metrics (e.g., SSIM, PSNR) have limitations in capturing the semantic correctness of chart edits. Our benchmark demonstrates the profound limitations of pixel-level manipulation and provides a robust foundation for developing and evaluating future structure-aware models. By releasing \textit{FigEdit} (https://github.com/adobe-research/figure-editing), we aim to enable systematic progress in structure-aware figure editing, provide a common ground for fair comparison, and encourage future research on models that understand both the visual and semantic layers of scientific charts.

Method: An end-to-end differentiable inverse graphics framework that jointly models object geometry, physical properties, and interactions directly from videos. Through backpropagation, the approach enables recovery of force representations from object motions.

Result: The method successfully infers plausible force fields from both synthetic and real-world videos, demonstrating the ability to recover invisible physical forces from visual observations.

Conclusion: The approach shows potential for physics-based video generation and editing, and represents a step toward understanding and modeling the physical processes behind pixels, bridging vision and physics.

Abstract: A longstanding goal in computer vision is to model motions from videos, while the representations behind motions, i.e. the invisible physical interactions that cause objects to deform and move, remain largely unexplored. In this paper, we study how to recover the invisible forces from visual observations, e.g., estimating the wind field by observing a leaf falling to the ground. Our key innovation is an end-to-end differentiable inverse graphics framework, which jointly models object geometry, physical properties, and interactions directly from videos. Through backpropagation, our approach enables the recovery of force representations from object motions. We validate our method on both synthetic and real-world scenarios, and the results demonstrate its ability to infer plausible force fields from videos. Furthermore, we show the potential applications of our approach, including physics-based video generation and editing. We hope our approach sheds light on understanding and modeling the physical process behind pixels, bridging the gap between vision and physics. Please check more video results in our \href{https://chaoren2357.github.io/seeingthewind/}{project page}.

Method: Uses instance segmentation to generate edge-aligned masks conforming to traffic sign shapes. Employs a U-Net generator to craft adversarial patches optimized through color/texture constraints and frequency domain analysis for seamless background integration. Focuses on peripheral areas where human attention is lower.

Result: Achieves over 90% attack success rate across various traffic sign classification models under limited query budgets. Shows strong cross-model transferability and maintains robust real-world performance stable under varying angles and distances.

Conclusion: TESP-Attack provides a highly effective stealth-aware adversarial attack method for traffic signs that balances attack effectiveness with visual concealment, addressing the key limitation of current physical attacks while maintaining practical applicability in real-world scenarios.

Abstract: Intelligent driving systems are vulnerable to physical adversarial attacks on traffic signs. These attacks can cause misclassification, leading to erroneous driving decisions that compromise road safety. Moreover, within V2X networks, such misinterpretations can propagate, inducing cascading failures that disrupt overall traffic flow and system stability. However, a key limitation of current physical attacks is their lack of stealth. Most methods apply perturbations to central regions of the sign, resulting in visually salient patterns that are easily detectable by human observers, thereby limiting their real-world practicality. This study proposes TESP-Attack, a novel stealth-aware adversarial patch method for traffic sign classification. Based on the observation that human visual attention primarily focuses on the central regions of traffic signs, we employ instance segmentation to generate edge-aligned masks that conform to the shape characteristics of the signs. A U-Net generator is utilized to craft adversarial patches, which are then optimized through color and texture constraints along with frequency domain analysis to achieve seamless integration with the background environment, resulting in highly effective visual concealment. The proposed method demonstrates outstanding attack success rates across traffic sign classification models with varied architectures, achieving over 90% under limited query budgets. It also exhibits strong cross-model transferability and maintains robust real-world performance that remains stable under varying angles and distances.

Method: Built on MonST3R backbone, EAG3R introduces: 1) Retinex-inspired image enhancement module and lightweight event adapter with SNR-aware fusion that adaptively combines RGB and event features based on local reliability; 2) Novel event-based photometric consistency loss that reinforces spatiotemporal coherence during global optimization.

Result: EAG3R significantly outperforms state-of-the-art RGB-only baselines across monocular depth estimation, camera pose tracking, and dynamic reconstruction tasks in challenging dynamic low-light scenes.

Conclusion: Augmenting pointmap-based reconstruction with asynchronous event streams enables robust 3D geometry estimation in challenging real-world conditions, overcoming limitations of RGB-only approaches without requiring specialized retraining for night-time scenarios.

Abstract: Robust 3D geometry estimation from videos is critical for applications such as autonomous navigation, SLAM, and 3D scene reconstruction. Recent methods like DUSt3R demonstrate that regressing dense pointmaps from image pairs enables accurate and efficient pose-free reconstruction. However, existing RGB-only approaches struggle under real-world conditions involving dynamic objects and extreme illumination, due to the inherent limitations of conventional cameras. In this paper, we propose EAG3R, a novel geometry estimation framework that augments pointmap-based reconstruction with asynchronous event streams. Built upon the MonST3R backbone, EAG3R introduces two key innovations: (1) a retinex-inspired image enhancement module and a lightweight event adapter with SNR-aware fusion mechanism that adaptively combines RGB and event features based on local reliability; and (2) a novel event-based photometric consistency loss that reinforces spatiotemporal coherence during global optimization. Our method enables robust geometry estimation in challenging dynamic low-light scenes without requiring retraining on night-time data. Extensive experiments demonstrate that EAG3R significantly outperforms state-of-the-art RGB-only baselines across monocular depth estimation, camera pose tracking, and dynamic reconstruction tasks.

Method: Developed scalable pipeline combining: 1) large-scale web collection, 2) rigorous filtering/deduplication, 3) object-detection-driven evidence extraction, 4) LLM-based refinement under grounding constraints. Built two resources: DEJIMA-Cap (image-caption) and DEJIMA-VQA (VQA).

Result: Created 3.88M image-text pairs, far exceeding existing Japanese V&L datasets. Human evaluation shows substantially higher Japaneseness and linguistic naturalness than translation/manual datasets, with factual correctness comparable to human-annotated corpora. Models trained on DEJIMA show consistent improvements across Japanese multimodal benchmarks.

Conclusion: Culturally grounded, large-scale resources are crucial for enhancing Japanese V&L model performance. DEJIMA dataset and pipeline are publicly released with commercial-friendly licensing to advance research and industrial applications in Japanese multimodal AI.

Abstract: This work addresses the scarcity of high-quality, large-scale resources for Japanese Vision-and-Language (V&L) modeling. We present a scalable and reproducible pipeline that integrates large-scale web collection with rigorous filtering/deduplication, object-detection-driven evidence extraction, and Large Language Model (LLM)-based refinement under grounding constraints. Using this pipeline, we build two resources: an image-caption dataset (DEJIMA-Cap) and a VQA dataset (DEJIMA-VQA), each containing 3.88M image-text pairs, far exceeding the size of existing Japanese V&L datasets. Human evaluations demonstrate that DEJIMA achieves substantially higher Japaneseness and linguistic naturalness than datasets constructed via translation or manual annotation, while maintaining factual correctness at a level comparable to human-annotated corpora. Quantitative analyses of image feature distributions further confirm that DEJIMA broadly covers diverse visual domains characteristic of Japan, complementing its linguistic and cultural representativeness. Models trained on DEJIMA exhibit consistent improvements across multiple Japanese multimodal benchmarks, confirming that culturally grounded, large-scale resources play a key role in enhancing model performance. All data sources and modules in our pipeline are licensed for commercial use, and we publicly release the resulting dataset and metadata to encourage further research and industrial applications in Japanese V&L modeling.

Method: Two complementary modules: 1) Polarization-guided photometric correction - identifies reflective regions via Degree of Linear Polarization (DoLP) and refines reflective Gaussians with Color Refinement Maps; 2) Polarization-enhanced Gaussian densification - integrates Angle and Degree of Linear Polarization (A/DoLP) into PatchMatch-based depth completion for textureless areas, enabling back-projection and fusion of new Gaussians.

Result: PolarGS achieves superior geometric accuracy compared to state-of-the-art methods and is framework-agnostic.

Conclusion: Leveraging polarization as an optical prior effectively resolves photometric ambiguities in 3DGS, enabling more complete and accurate surface reconstruction, particularly for challenging reflective and textureless surfaces.

Abstract: Recent advances in surface reconstruction for 3D Gaussian Splatting (3DGS) have enabled remarkable geometric accuracy. However, their performance degrades in photometrically ambiguous regions such as reflective and textureless surfaces, where unreliable cues disrupt photometric consistency and hinder accurate geometry estimation. Reflected light is often partially polarized in a manner that reveals surface orientation, making polarization an optic complement to photometric cues in resolving such ambiguities. Therefore, we propose PolarGS, an optics-aware extension of RGB-based 3DGS that leverages polarization as an optical prior to resolve photometric ambiguities and enhance reconstruction accuracy. Specifically, we introduce two complementary modules: a polarization-guided photometric correction strategy, which ensures photometric consistency by identifying reflective regions via the Degree of Linear Polarization (DoLP) and refining reflective Gaussians with Color Refinement Maps; and a polarization-enhanced Gaussian densification mechanism for textureless area geometry recovery, which integrates both Angle and Degree of Linear Polarization (A/DoLP) into a PatchMatch-based depth completion process. This enables the back-projection and fusion of new Gaussians, leading to more complete reconstruction. PolarGS is framework-agnostic and achieves superior geometric accuracy compared to state-of-the-art methods.

Method: Uses structured capture with circle grid targets to encode anisotropic PSFs, leverages binary luminance prior, employs flow-guided subpixel alignment for sharp image reconstruction, models PSF as energy-constrained implicit neural representation, and jointly optimizes both components in a demosaicing-aware differentiable framework.

Result: Extensive experiments on simulated and real-world data demonstrate state-of-the-art accuracy and reliability, validating effectiveness for practical PSF calibration.

Conclusion: CircleFlow provides a robust framework for high-fidelity PSF estimation through accurate edge localization and physically consistent modeling, enabling practical camera calibration applications.

Abstract: The point spread function (PSF) serves as a fundamental descriptor linking the real-world scene to the captured signal, manifesting as camera blur. Accurate PSF estimation is crucial for both optical characterization and computational vision, yet remains challenging due to the inherent ambiguity and the ill-posed nature of intensity-based deconvolution. We introduce CircleFlow, a high-fidelity PSF estimation framework that employs flow-guided edge localization for precise blur characterization. CircleFlow begins with a structured capture that encodes locally anisotropic and spatially varying PSFs by imaging a circle grid target, while leveraging the target’s binary luminance prior to decouple image and kernel estimation. The latent sharp image is then reconstructed through subpixel alignment of an initialized binary structure guided by optical flow, whereas the PSF is modeled as an energy-constrained implicit neural representation. Both components are jointly optimized within a demosaicing-aware differentiable framework, ensuring physically consistent and robust PSF estimation enabled by accurate edge localization. Extensive experiments on simulated and real-world data demonstrate that CircleFlow achieves state-of-the-art accuracy and reliability, validating its effectiveness for practical PSF calibration.

Method: A reinforcement learning-based speculative temporal reasoning framework with dual-model design: lightweight draft MLLM rapidly explores/proposes salient frames from dense temporal regions, while powerful target MLLM focuses on temporal reasoning and verifies proposals, iteratively refining attention until convergence.

Result: SpecTemp maintains competitive accuracy across multiple video understanding benchmarks while significantly accelerating inference compared to existing thinking-with-frames methods.

Conclusion: The dual-model collaborative approach effectively balances efficiency and accuracy for long video understanding, mirroring human brain pathways and overcoming the redundancy bottleneck of current methods.

Abstract: Long video understanding is essential for human-like intelligence, enabling coherent perception and reasoning over extended temporal contexts. While the emerging thinking-with-frames paradigm, which alternates between global temporal reasoning and local frame examination, has advanced the reasoning capabilities of video multi-modal large language models (MLLMs), it suffers from a significant efficiency bottleneck due to the progressively growing and redundant multi-modal context. To address this, we propose SpecTemp, a reinforcement learning-based Speculative Temporal reasoning framework that decouples temporal perception from reasoning via a cooperative dual-model design. In SpecTemp, a lightweight draft MLLM rapidly explores and proposes salient frames from densely sampled temporal regions, while a powerful target MLLM focuses on temporal reasoning and verifies the draft’s proposals, iteratively refining its attention until convergence. This design mirrors the collaborative pathways of the human brain, balancing efficiency with accuracy. To support training, we construct the SpecTemp-80K dataset, featuring synchronized dual-level annotations for coarse evidence spans and fine-grained frame-level evidence. Experiments across multiple video understanding benchmarks demonstrate that SpecTemp not only maintains competitive accuracy but also significantly accelerates inference compared with existing thinking-with-frames methods.

Method: Developed a gaze-video QA generation pipeline that aligns egocentric videos with raw gaze trajectories via fixation extraction, region-specific visual prompting, and scanpath construction to produce spatio-temporally grounded QA pairs reflecting human perceptual dynamics.

Result: Substantial performance gaps between state-of-the-art MLLMs and human performance across all StreamGaze tasks, revealing fundamental limitations in gaze-based temporal reasoning, intention modeling, and proactive prediction.

Conclusion: Current MLLMs struggle with gaze-guided streaming video understanding, and the benchmark provides insights into failure modes and capabilities needed for future models, with all data and code being publicly released for continued research.

Abstract: Streaming video understanding requires models not only to process temporally incoming frames, but also to anticipate user intention for realistic applications like AR glasses. While prior streaming benchmarks evaluate temporal reasoning, none measure whether MLLMs can interpret or leverage human gaze signals within a streaming setting. To fill this gap, we introduce StreamGaze, the first benchmark designed to evaluate how effectively MLLMs use gaze for temporal and proactive reasoning in streaming videos. StreamGaze introduces gaze-guided past, present, and proactive tasks that comprehensively evaluate streaming video understanding. These tasks assess whether models can use real-time gaze to follow shifting attention and infer user intentions from only past and currently observed frames. To build StreamGaze, we develop a gaze-video QA generation pipeline that aligns egocentric videos with raw gaze trajectories via fixation extraction, region-specific visual prompting, and scanpath construction. This pipeline produces spatio-temporally grounded QA pairs that closely reflect human perceptual dynamics. Across all StreamGaze tasks, we observe substantial performance gaps between state-of-the-art MLLMs and human performance, revealing fundamental limitations in gaze-based temporal reasoning, intention modeling, and proactive prediction. We further provide detailed analyses of gaze-prompting strategies, reasoning behaviors, and task-specific failure modes, offering deeper insight into why current MLLMs struggle and what capabilities future models must develop. All data and code will be publicly released to support continued research in gaze-guided streaming video understanding.

Method: IRPO introduces two key innovations: 1) A data formulation principle that selects underperforming samples from pre-training for optimal performance and efficiency, and 2) A three-component reward system including General Reward for structural fidelity, Expert Reward using Qwen-VL for perceptual alignment, and Restoration Reward for task-specific low-level quality.

Result: IRPO achieves state-of-the-art results on six in-domain and five out-of-domain low-level benchmarks, surpassing the AdaIR baseline by 0.83 dB on in-domain tasks and 3.43 dB on OOD settings across diverse degradation types.

Conclusion: IRPO successfully demonstrates the potential of post-training paradigms for low-level vision tasks, providing a systematic approach that balances objective accuracy with human perceptual preference while achieving superior performance and generalization.

Abstract: Recent advances in post-training paradigms have achieved remarkable success in high-level generation tasks, yet their potential for low-level vision remains rarely explored. Existing image restoration (IR) methods rely on pixel-level hard-fitting to ground-truth images, struggling with over-smoothing and poor generalization. To address these limitations, we propose IRPO, a low-level GRPO-based post-training paradigm that systematically explores both data formulation and reward modeling. We first explore a data formulation principle for low-level post-training paradigm, in which selecting underperforming samples from the pre-training stage yields optimal performance and improved efficiency. Furthermore, we model a reward-level criteria system that balances objective accuracy and human perceptual preference through three complementary components: a General Reward for structural fidelity, an Expert Reward leveraging Qwen-VL for perceptual alignment, and a Restoration Reward for task-specific low-level quality. Comprehensive experiments on six in-domain and five out-of-domain (OOD) low-level benchmarks demonstrate that IRPO achieves state-of-the-art results across diverse degradation types, surpassing the AdaIR baseline by 0.83 dB on in-domain tasks and 3.43 dB on OOD settings. Our code can be shown in https://github.com/HaoxuanXU1024/IRPO.

Method: PanFlow exploits the spherical nature of panoramas to decouple camera rotation from input optical flow, introduces spherical noise warping for boundary consistency, and uses a curated large-scale motion-rich dataset with pose and flow annotations.

Result: PanFlow significantly outperforms prior methods in motion fidelity, visual quality, and temporal coherence, and demonstrates effectiveness in motion transfer and video editing applications.

Conclusion: PanFlow provides a robust solution for controllable panoramic video generation with large motions, advancing the field for VR and immersive media applications.

Abstract: Panoramic video generation has attracted growing attention due to its applications in virtual reality and immersive media. However, existing methods lack explicit motion control and struggle to generate scenes with large and complex motions. We propose PanFlow, a novel approach that exploits the spherical nature of panoramas to decouple the highly dynamic camera rotation from the input optical flow condition, enabling more precise control over large and dynamic motions. We further introduce a spherical noise warping strategy to promote loop consistency in motion across panorama boundaries. To support effective training, we curate a large-scale, motion-rich panoramic video dataset with frame-level pose and flow annotations. We also showcase the effectiveness of our method in various applications, including motion transfer and video editing. Extensive experiments demonstrate that PanFlow significantly outperforms prior methods in motion fidelity, visual quality, and temporal coherence. Our code, dataset, and models are available at https://github.com/chengzhag/PanFlow.

Method: AFRAgent uses an instruct-BLIP-based multimodal architecture with adaptive feature renormalization - a token-level affine transformation technique that enriches low-resolution image embeddings and fuses high-resolution details to enhance image embeddings in the LLM pipeline.

Result: AFRAgent achieves superior performance on Meta-GUI and AITW benchmarks for smartphone automation while being less than one-fourth the size of its nearest competitor, establishing a new state-of-the-art baseline.

Conclusion: AFRAgent demonstrates that efficient, compact models can achieve state-of-the-art performance in GUI automation through innovative feature enhancement techniques, addressing both accuracy and efficiency challenges in mobile UI automation.

Abstract: There is a growing demand for mobile user interface (UI) automation, driven by its broad applications across industries. With the advent of visual language models (VLMs), GUI automation has progressed from generating text-based instructions for humans to autonomously executing tasks, thus optimizing automation workflows. Recent approaches leverage VLMs for this problem due to their ability to 1) process on-screen content directly, 2) remain independent of device-specific APIs by utilizing human actions (e.g., clicks, typing), and 3) apply real-world contextual knowledge for task understanding. However, these models often have trouble accurately identifying widgets and determining actions due to limited spatial information in vision encoder features. Additionally, top-performing models are often large, requiring extensive training and resulting in inference delays. In this work, we introduce AFRAgent, an instruct-BLIP-based multimodal architecture that achieves superior performance in GUI automation while being less than one-fourth the size of its nearest competitor. To enhance image embeddings in the large language model (LLM) pipeline, we propose an adaptive feature renormalization-based (a token-level affine transformation) technique that effectively enriches low-resolution image embeddings and fuses high-resolution details. We evaluate AFRAgent on Meta-GUI and AITW benchmarks, establishing a new state-of-the-art baseline for smartphone automation.

Method: Uses a recursive voxel hierarchy to capture splat coordinates, while splat-wise features store abstracted cues including color, opacity, transformation, and material properties, integrating both spatial and semantic information in 3D space.

Result: Achieves state-of-the-art compression on standard benchmarks while maintaining high rendering quality, compressing 3D scenes by orders of magnitude without loss of flexibility.

Conclusion: Smol-GS provides an effective approach for learning compact 3D representations that could serve as a foundation for downstream tasks like navigation, planning, and broader 3D scene understanding beyond just visual fidelity.

Abstract: We present Smol-GS, a novel method for learning compact representations for 3D Gaussian Splatting (3DGS). Our approach learns highly efficient encodings in 3D space that integrate both spatial and semantic information. The model captures the coordinates of the splats through a recursive voxel hierarchy, while splat-wise features store abstracted cues, including color, opacity, transformation, and material properties. This design allows the model to compress 3D scenes by orders of magnitude without loss of flexibility. Smol-GS achieves state-of-the-art compression on standard benchmarks while maintaining high rendering quality. Beyond visual fidelity, the discrete representations could potentially serve as a foundation for downstream tasks such as navigation, planning, and broader 3D scene understanding.

Method: Simple adaptation of Vision Transformers and DINOv2 for volumetric data with targeted modifications to patch embeddings, positional encodings, and volumetric augmentations, enabling scalable self-supervised pretraining directly on 3D CT volumes.

Result: Large-scale 3D pretraining on 105K CT volumes yields stable, robust frozen representations that generalize strongly across downstream tasks.

Conclusion: TAP-CT provides a powerful, low-resource baseline for medical imaging research, with all models, configurations, and benchmark code released to promote transparency and reproducibility.

Abstract: Existing foundation models (FMs) in the medical domain often require extensive fine-tuning or rely on training resource-intensive decoders, while many existing encoders are pretrained with objectives biased toward specific tasks. This illustrates a need for a strong, task-agnostic foundation model that requires minimal fine-tuning beyond feature extraction. In this work, we introduce a suite of task-agnostic pretraining of CT foundation models (TAP-CT): a simple yet effective adaptation of Vision Transformers (ViTs) and DINOv2 for volumetric data, enabling scalable self-supervised pretraining directly on 3D CT volumes. Our approach incorporates targeted modifications to patch embeddings, positional encodings, and volumetric augmentations, making the architecture depth-aware while preserving the simplicity of the underlying architectures. We show that large-scale 3D pretraining on an extensive in-house CT dataset (105K volumes) yields stable, robust frozen representations that generalize strongly across downstream tasks. To promote transparency and reproducibility, and to establish a powerful, low-resource baseline for future research in medical imaging, we will release all pretrained models, experimental configurations, and downstream benchmark code at https://huggingface.co/fomofo/tap-ct-b-3d.

Method: Three-stage deep learning framework for sparse-sampling CBCT reconstruction. Developed and validated using 4102 patients with 8675 CBCT scans from 12 centers. Prospective cross-over trial (NCT07035977) with 138 patients scheduled for percutaneous thoracic puncture.

Result: Reconstructed images indistinguishable from original scans per 11 physicians. Diagnostic performance and image quality comparable to standard reconstruction. Prospective trial: no significant differences in image quality or lesion assessment (all P>0.05). 25 interventionalists expressed no preference between model-based and full-sampling images (Kappa<0.2). Radiation reduced to ~1/6 of conventional approach.

Conclusion: DeepPriorCBCT enables high-quality CBCT reconstruction under sparse sampling while markedly decreasing intraoperative radiation risk, validated through both retrospective and prospective clinical studies.

Abstract: Cone beam computed tomography (CBCT)-guided puncture has become an established approach for diagnosing and treating early- to mid-stage thoracic tumours, yet the associated radiation exposure substantially elevates the risk of secondary malignancies. Although multiple low-dose CBCT strategies have been introduced, none have undergone validation using large-scale multicenter retrospective datasets, and prospective clinical evaluation remains lacking. Here, we propose DeepPriorCBCT - a three-stage deep learning framework that achieves diagnostic-grade reconstruction using only one-sixth of the conventional radiation dose. 4102 patients with 8675 CBCT scans from 12 centers were included to develop and validate DeepPriorCBCT. Additionally, a prospective cross-over trial (Registry number: NCT07035977) which recruited 138 patients scheduled for percutaneous thoracic puncture was conducted to assess the model’s clinical applicability. Assessment by 11 physicians confirmed that reconstructed images were indistinguishable from original scans. Moreover, diagnostic performance and overall image quality were comparable to those generated by standard reconstruction algorithms. In the prospective trial, five radiologists reported no significant differences in image quality or lesion assessment between DeepPriorCBCT and the clinical standard (all P>0.05). Likewise, 25 interventionalists expressed no preference between model-based and full-sampling images for surgical guidance (Kappa<0.2). Radiation exposure with DeepPriorCBCT was reduced to approximately one-sixth of that with the conventional approach, and collectively, the findings confirm that it enables high-quality CBCT reconstruction under sparse sampling conditions while markedly decreasing intraoperative radiation risk.

Method: Proposes a feed-forward 3DGS compression framework with: 1) Large-scale context structure comprising thousands of Gaussians based on Morton serialization, 2) Fine-grained space-channel auto-regressive entropy model to leverage expansive context, and 3) Attention-based transform coding model to extract informative latent priors by aggregating features from wide-range neighboring Gaussians.

Result: Achieves 20× compression ratio for 3DGS in feed-forward inference and establishes state-of-the-art performance among generalizable codecs.

Conclusion: The proposed framework effectively models long-range correlations to enable highly compact and generalizable 3D representations, overcoming limitations of existing methods and making 3DGS more practical for widespread adoption.

Abstract: 3D Gaussian Splatting (3DGS) has emerged as a revolutionary 3D representation. However, its substantial data size poses a major barrier to widespread adoption. While feed-forward 3DGS compression offers a practical alternative to costly per-scene per-train compressors, existing methods struggle to model long-range spatial dependencies, due to the limited receptive field of transform coding networks and the inadequate context capacity in entropy models. In this work, we propose a novel feed-forward 3DGS compression framework that effectively models long-range correlations to enable highly compact and generalizable 3D representations. Central to our approach is a large-scale context structure that comprises thousands of Gaussians based on Morton serialization. We then design a fine-grained space-channel auto-regressive entropy model to fully leverage this expansive context. Furthermore, we develop an attention-based transform coding model to extract informative latent priors by aggregating features from a wide range of neighboring Gaussians. Our method yields a $20\times$ compression ratio for 3DGS in a feed-forward inference and achieves state-of-the-art performance among generalizable codecs.

Method: Quantum-inspired geometric framework representing operators by normalized singular value spectrum on Bloch hypersphere; spectral-to-functional equivalence theorem; Quantum Metric-Driven Functional Redundancy Graphs (QM-FRG) and one-shot structured pruning.

Result: Proposed metric shows superiority over magnitude and random baselines in controlled simulation; extensive experimental validation on large-scale multimodal transformers and domestic heterogeneous hardware (Huawei Ascend, Cambricon MLU, Kunlunxin) is deferred to journal version.

Conclusion: Establishes first rigorous foundation for cross-modal and cross-architecture operator substitutability through quantum-inspired geometric representation and spectral-to-functional equivalence.

Abstract: The rapid growth of multimodal intelligence on resource-constrained and heterogeneous domestic hardware exposes critical bottlenecks: multimodal feature heterogeneity, real-time requirements in dynamic scenarios, and hardware-specific operator redundancy. This work introduces a quantum-inspired geometric framework for neural operators that represents each operator by its normalized singular value spectrum on the Bloch hypersphere. We prove a tight spectral-to-functional equivalence theorem showing that vanishing Fubini–Study/Wasserstein-2 distance implies provable functional closeness, establishing the first rigorous foundation for cross-modal and cross-architecture operator substitutability. Based on this metric, we propose Quantum Metric-Driven Functional Redundancy Graphs (QM-FRG) and one-shot structured pruning. Controlled simulation validates the superiority of the proposed metric over magnitude and random baselines. An extensive experimental validation on large-scale multimodal transformers and domestic heterogeneous hardware (Huawei Ascend, Cambricon MLU, Kunlunxin) hardware is deferred to an extended journal version currently in preparation.

Method: A parameter-efficient framework with three decoupled stages: (1) Look generates objective visual descriptions and candidate sets; (2) Recite uses a lightweight 1.7B router to transform visual cues into targeted queries that trigger candidate-specific parametric knowledge; (3) Answer performs parallel evidence alignment between descriptions and recited knowledge to select the most consistent label.

Result: Achieves state-of-the-art results on AgroBench, improving Weed Identification accuracy by 23.6% over Qwen-VL and surpassing GPT-4o without external search overhead.

Conclusion: The modular design mitigates hallucinations by transforming passive perception into active, controllable knowledge retrieval, enhancing VLM performance in specialized domains while keeping backbone models frozen.

Abstract: Vision-Language Models (VLMs) exhibit significant performance plateaus in specialized domains like precision agriculture, primarily due to “Reasoning-Driven Hallucination” where linguistic priors override visual perception. A key bottleneck is the “Modality Gap”: visual embeddings fail to reliably activate the fine-grained expert knowledge already encoded in model parameters. We propose “Look, Recite, Then Answer,” a parameter-efficient framework that enhances VLMs via self-generated knowledge hints while keeping backbone models frozen. The framework decouples inference into three stages: (1) Look generates objective visual descriptions and candidate sets; (2) Recite employs a lightweight 1.7B router to transform visual cues into targeted queries that trigger candidate-specific parametric knowledge; (3) Answer performs parallel evidence alignment between descriptions and recited knowledge to select the most consistent label. On AgroBench, our method achieves state-of-the-art results, improving Weed Identification accuracy by 23.6% over Qwen-VL and surpassing GPT-4o without external search overhead. This modular design mitigates hallucinations by transforming passive perception into active, controllable knowledge retrieval

Method: Created HanDyVQA benchmark with 6 complementary question types (Action, Process, Objects, Location, State Change, Object Parts), totaling 11.1K multiple-choice QA pairs and 10.3K segmentation masks for object/part-level reasoning.

Result: Best-performing model (Gemini-2.5-Pro) achieved only 73% average accuracy, far from human performance (97%). Analysis revealed challenges in spatial relationships, motion, and part-level geometric understanding.

Conclusion: HanDyVQA highlights significant gaps in current models’ understanding of HOI dynamics. Integrating explicit HOI-related cues improves performance, offering insights for developing future models with deeper understanding of hand-object interaction dynamics.

Abstract: Hand-object interaction (HOI) inherently involves dynamics where human manipulations produce distinct spatio-temporal effects on objects. However, existing semantic HOI benchmarks focused either on manipulation or on the resulting effects at a coarse level, lacking fine-grained spatio-temporal reasoning to capture the underlying dynamics in HOI. We introduce HanDyVQA, a fine-grained video question-answering benchmark that comprehensively covers both the manipulation and effect aspects of HOI. HanDyVQA comprises six complementary question types (Action, Process, Objects, Location, State Change, and Object Parts), totalling 11.1K multiple-choice QA pairs. Collected QA pairs recognizing manipulation styles, hand/object motions, and part-level state changes. HanDyVQA also includes 10.3K segmentation masks for Objects and Object Parts questions, enabling the evaluation of object/part-level reasoning in video object segmentation. We evaluated recent video foundation models on our benchmark and found that even the best-performing model, Gemini-2.5-Pro, reached only 73% average accuracy, which is far from human performance (97%). Further analysis shows the remaining challenges in spatial relationship, motion, and part-level geometric understanding. We also found that integrating explicit HOI-related cues into visual features improves performance, offering insights for developing future models with a deeper understanding of HOI dynamics.

Method: Retrieval-augmented prompting approach: 1) Use domain-adapted SigLIP2 encoder to retrieve related captions/examples from datastore, 2) Two variants: image-blind (multilingual LLM with textual prompts) and image-aware (VLM processing prompt+image), 3) Graph-based re-ranking using PageRank on image-caption graph for coherence.

Result: Competitive with fully supervised English-only systems across 4 benchmark datasets and 10 languages. PageRank re-ranking yields up to 35% performance improvements. VLMs generate visually grounded but lexically diverse captions, while LLMs achieve stronger BLEU/CIDEr scores. Direct target language generation outperforms translation-based strategies.

Conclusion: First systematic evaluation of multilingual, training-free captioning for remote sensing imagery, advancing toward more inclusive and scalable multimodal Earth observation systems without requiring large annotated datasets.

Abstract: Remote sensing image captioning has advanced rapidly through encoder–decoder models, although the reliance on large annotated datasets and the focus on English restricts global applicability. To address these limitations, we propose the first training-free multilingual approach, based on retrieval-augmented prompting. For a given aerial image, we employ a domain-adapted SigLIP2 encoder to retrieve related captions and few-shot examples from a datastore, which are then provided to a language model. We explore two variants: an image-blind setup, where a multilingual Large Language Model (LLM) generates the caption from textual prompts alone, and an image-aware setup, where a Vision–Language Model (VLM) jointly processes the prompt and the input image. To improve the coherence of the retrieved content, we introduce a graph-based re-ranking strategy using PageRank on a graph of images and captions. Experiments on four benchmark datasets across ten languages demonstrate that our approach is competitive with fully supervised English-only systems and generalizes to other languages. Results also highlight the importance of re-ranking with PageRank, yielding up to 35% improvements in performance metrics. Additionally, it was observed that while VLMs tend to generate visually grounded but lexically diverse captions, LLMs can achieve stronger BLEU and CIDEr scores. Lastly, directly generating captions in the target language consistently outperforms other translation-based strategies. Overall, our work delivers one of the first systematic evaluations of multilingual, training-free captioning for remote sensing imagery, advancing toward more inclusive and scalable multimodal Earth observation systems.

Method: STC introduces two token-level accelerators: STC-Cacher reduces ViT encoding overhead by caching and reusing features from similar frames, and STC-Pruner compresses visual token sequences before LLM processing by preserving only the most salient tokens based on spatial and temporal relevance.

Result: STC outperforms other compression methods across four baseline streaming VideoLLMs on five benchmarks, retaining up to 99% accuracy on ReKV while reducing ViT encoding latency by 24.5% and LLM pre-filling latency by 45.3%.

Conclusion: STC provides an effective plug-and-play framework for accelerating streaming VideoLLMs by addressing computational bottlenecks in both ViT encoding and LLM pre-filling stages, enabling more efficient real-time video understanding with minimal accuracy trade-offs.

Abstract: Streaming Video Large Language Models (VideoLLMs) have demonstrated impressive performance across various video understanding tasks, but they face significant challenges in real-time deployment due to the high computational cost of processing dense visual tokens from continuous video streams. In streaming video scenarios, the primary bottleneck lies in the Vision Transformer (ViT) encoding stage, where redundant processing of temporally similar frames leads to inefficiency. Additionally, inflated token sequences during LLM pre-filling further exacerbate latency and memory overhead. To address these challenges, we propose \textbf{S}treaming \textbf{T}oken \textbf{C}ompression (\textbf{STC}), a plug-and-play hierarchical framework that seamlessly integrates into existing streaming VideoLLMs, optimizing both ViT encoding and LLM pre-filling stages to accelerate processing. STC introduces two token-level accelerators: \textbf{STC-Cacher}, which reduces ViT encoding overhead by caching and reusing features from temporally similar frames, and \textbf{STC-Pruner}, which compresses the visual token sequence before it enters the LLM, preserving only the most salient tokens based on both spatial and temporal relevance. Extensive experiments on four baseline streaming VideoLLMs across five benchmarks demonstrate that STC outperforms other compression methods. Notably, STC retains up to \textbf{99%} of accuracy on the ReKV framework while reducing ViT encoding latency and LLM pre-filling latency by \textbf{24.5%} and \textbf{45.3%}.

Method: 1) Uses pretrained 4D visual geometry transformer with temporal cache to extract 4D features from 2D images; 2) Introduces Fusion Tokens (learnable tokens trained with future prediction) to unify 2D and 4D representations; 3) Employs mask-and-reconstruct strategy to train VLM on 4D representations, allowing 4D branch to be dropped at inference.

Result: SwiftVLA outperforms lightweight baselines and rivals VLAs up to 7× larger, achieving comparable performance on edge devices while being 18× faster and reducing memory footprint by 12×.

Conclusion: SwiftVLA demonstrates that compact VLA models can achieve strong 4D understanding through efficient architectural design, enabling practical deployment on edge devices without sacrificing spatiotemporal reasoning capabilities.

Abstract: Vision-Language-Action (VLA) models built on pretrained Vision-Language Models (VLMs) show strong potential but are limited in practicality due to their large parameter counts. To mitigate this issue, using a lightweight VLM has been explored, but it compromises spatiotemporal reasoning. Although some methods suggest that incorporating additional 3D inputs can help, they usually rely on large VLMs to fuse 3D and 2D inputs and still lack temporal understanding. Therefore, we propose SwiftVLA, an architecture that enhances a compact model with 4D understanding while preserving design efficiency. Specifically, our approach features a pretrained 4D visual geometry transformer with a temporal cache that extracts 4D features from 2D images. Then, to enhance the VLM’s ability to exploit both 2D images and 4D features, we introduce Fusion Tokens, a set of learnable tokens trained with a future prediction objective to generate unified representations for action generation. Finally, we introduce a mask-and-reconstruct strategy that masks 4D inputs to the VLM and trains the VLA to reconstruct them, enabling the VLM to learn effective 4D representations and allowing the 4D branch to be dropped at inference with minimal performance loss. Experiments in real and simulated environments show that SwiftVLA outperforms lightweight baselines and rivals VLAs up to 7 times larger, achieving comparable performance on edge devices while being 18 times faster and reducing memory footprint by 12 times.

Method: Proposes CAE with three steps: 1) Construct semantic space using relevant nouns from WordNet and descriptions from caption datasets, 2) Align image features with selected nouns and captions via optimal transport to get discriminative semantic space, 3) Combine enhanced semantic and image features for clustering.

Result: Extensive experiments on 8 datasets show effectiveness, notably surpassing state-of-the-art training-free approach with 4.2% accuracy improvement and 2.9% ARI improvement on ImageNet-1K.

Conclusion: CAE effectively addresses noun ambiguity in image clustering by combining hierarchical semantic alignment, achieving superior performance without requiring training.

Abstract: Image clustering is a classic problem in computer vision, which categorizes images into different groups. Recent studies utilize nouns as external semantic knowledge to improve clus- tering performance. However, these methods often overlook the inherent ambiguity of nouns, which can distort semantic representations and degrade clustering quality. To address this issue, we propose a hierarChical semAntic alignmEnt method for image clustering, dubbed CAE, which improves cluster- ing performance in a training-free manner. In our approach, we incorporate two complementary types of textual seman- tics: caption-level descriptions, which convey fine-grained attributes of image content, and noun-level concepts, which represent high-level object categories. We first select relevant nouns from WordNet and descriptions from caption datasets to construct a semantic space aligned with image features. Then, we align image features with selected nouns and captions via optimal transport to obtain a more discriminative semantic space. Finally, we combine the enhanced semantic and image features to perform clustering. Extensive experiments across 8 datasets demonstrate the effectiveness of our method, notably surpassing the state-of-the-art training-free approach with a 4.2% improvement in accuracy and a 2.9% improvement in adjusted rand index (ARI) on the ImageNet-1K dataset.

Method: TalkingPose uses a diffusion-based framework that leverages driving frames to capture facial/hand movements and transfers them to target actors via stable diffusion backbone. It introduces a feedback-driven mechanism for temporal coherence without additional computational costs or secondary training.

Result: The framework enables generation of animations with unlimited duration while maintaining temporal consistency, and introduces a comprehensive large-scale dataset as a new benchmark for human upper-body animation.

Conclusion: TalkingPose addresses key limitations in long-form character animation generation by providing a computationally efficient solution for temporally coherent extended animations, supported by a new benchmark dataset.

Abstract: Recent advancements in diffusion models have significantly improved the realism and generalizability of character-driven animation, enabling the synthesis of high-quality motion from just a single RGB image and a set of driving poses. Nevertheless, generating temporally coherent long-form content remains challenging. Existing approaches are constrained by computational and memory limitations, as they are typically trained on short video segments, thus performing effectively only over limited frame lengths and hindering their potential for extended coherent generation. To address these constraints, we propose TalkingPose, a novel diffusion-based framework specifically designed for producing long-form, temporally consistent human upper-body animations. TalkingPose leverages driving frames to precisely capture expressive facial and hand movements, transferring these seamlessly to a target actor through a stable diffusion backbone. To ensure continuous motion and enhance temporal coherence, we introduce a feedback-driven mechanism built upon image-based diffusion models. Notably, this mechanism does not incur additional computational costs or require secondary training stages, enabling the generation of animations with unlimited duration. Additionally, we introduce a comprehensive, large-scale dataset to serve as a new benchmark for human upper-body animation.

Method: Dual-stream angle-aware generation network with CNN branch for local geometric structures from equirectangular projections and ViT branch for global contextual cues from cubemap projections. Features are integrated through dual-projection adaptive fusion module with high-frequency enhancement, circular padding, and channel attention mechanisms.

Result: Outperforms existing approaches on SUN360 and M3D datasets in both inclination estimation and upright panorama generation. Ablation studies validate each module’s contribution and highlight task synergy.

Conclusion: Proposed method effectively addresses non-upright panorama correction for robotic vision, offering improved performance over traditional approaches through dual-stream architecture and specialized enhancements for 360° continuity.

Abstract: Panoramic cameras, capable of capturing a 360-degree field of view, are crucial in robotic vision, particularly in environments with sparse features. However, non-upright panoramas due to unstable robot postures hinder downstream tasks. Traditional IMU-based correction methods suffer from drift and external disturbances, while vision-based approaches offer a promising alternative. This study presents a dual-stream angle-aware generation network that jointly estimates camera inclination angles and reconstructs upright panoramic images. The network comprises a CNN branch that extracts local geometric structures from equirectangular projections and a ViT branch that captures global contextual cues from cubemap projections. These are integrated through a dual-projection adaptive fusion module that aligns spatial features across both domains. To further enhance performance, we introduce a high-frequency enhancement block, circular padding, and channel attention mechanisms to preserve 360° continuity and improve geometric sensitivity. Experiments on the SUN360 and M3D datasets demonstrate that our method outperforms existing approaches in both inclination estimation and upright panorama generation. Ablation studies further validate the contribution of each module and highlight the synergy between the two tasks. The code and related datasets can be found at: https://github.com/YuhaoShine/DualProjectionFusion.

Method: Used 97 micro-CT scanned specimens across 27 species, selected 12 species with sufficient representation, employed specimen-level data splitting (109,617 slices total), evaluated 7 CNN architectures with transfer learning, and created ensemble model combining ConvNeXt-Large and EfficientNetV2-Small.

Result: Final ensemble model achieved 95.64% test accuracy, 99.6% top-3 accuracy, and 0.998 AUC across all species. Developed interactive dashboard with real-time classification and 3D slice matching using SSIM, NCC, and Dice coefficient metrics.

Conclusion: Establishes new benchmarks for AI-assisted micropaleontological identification and provides a fully reproducible framework for foraminifera classification research, bridging deep learning with applied geosciences.

Abstract: This study presents a comprehensive deep learning pipeline for the automated classification of 12 foraminifera species using 2D micro-CT slices derived from 3D scans. We curated a scientifically rigorous dataset comprising 97 micro-CT scanned specimens across 27 species, selecting 12 species with sufficient representation for robust machine learning. To ensure methodological integrity and prevent data leakage, we employed specimen-level data splitting, resulting in 109,617 high-quality 2D slices (44,103 for training, 14,046 for validation, and 51,468 for testing). We evaluated seven state-of-the-art 2D convolutional neural network (CNN) architectures using transfer learning. Our final ensemble model, combining ConvNeXt-Large and EfficientNetV2-Small, achieved a test accuracy of 95.64%, with a top-3 accuracy of 99.6% and an area under the ROC curve (AUC) of 0.998 across all species. To facilitate practical deployment, we developed an interactive advanced dashboard that supports real-time slice classification and 3D slice matching using advanced similarity metrics, including SSIM, NCC, and the Dice coefficient. This work establishes new benchmarks for AI-assisted micropaleontological identification and provides a fully reproducible framework for foraminifera classification research, bridging the gap between deep learning and applied geosciences.

Method: Proposes LAHNet with: 1) Group Transformer using Locality-Sensitive Hashing to partition point clouds into non-overlapping windows for long-range context; 2) Cross-window strategy to expand receptive field; 3) Interaction Transformer with overlap matrix to match overlap regions between point cloud pairs.

Result: Extensive results demonstrate LAHNet learns robust and distinctive features, achieving significant registration results on real-world indoor and outdoor benchmarks.

Conclusion: LAHNet effectively enhances feature distinctiveness through local attention mechanisms and window-based strategies, improving point cloud registration performance across diverse real-world scenarios.

Abstract: Most existing learning-based point cloud descriptors for point cloud registration focus on perceiving local information of point clouds to generate distinctive features. However, a reasonable and broader receptive field is essential for enhancing feature distinctiveness. In this paper, we propose a Local Attentive Hashing Network for point cloud registration, called LAHNet, which introduces a local attention mechanism with the inductive bias of locality of convolution-like operators into point cloud descriptors. Specifically, a Group Transformer is designed to capture reasonable long-range context between points. This employs a linear neighborhood search strategy, Locality-Sensitive Hashing, enabling uniformly partitioning point clouds into non-overlapping windows. Meanwhile, an efficient cross-window strategy is adopted to further expand the reasonable feature receptive field. Furthermore, building on this effective windowing strategy, we propose an Interaction Transformer to enhance the feature interactions of the overlap regions within point cloud pairs. This computes an overlap matrix to match overlap regions between point cloud pairs by representing each window as a global signal. Extensive results demonstrate that LAHNet can learn robust and distinctive features, achieving significant registration results on real-world indoor and outdoor benchmarks.

Method: SceneProp reformulates scene-graph grounding as Maximum a Posteriori (MAP) inference in a Markov Random Field (MRF). It performs global inference over the entire query graph to find optimal assignment of image regions to nodes, implemented via differentiable Belief Propagation in an end-to-end framework.

Result: Experiments on four benchmarks show SceneProp significantly outperforms prior work. Critically, its accuracy consistently improves with the size and complexity of query graphs, demonstrating for the first time that more relational context leads to better grounding.

Conclusion: SceneProp successfully addresses the structural deficit in grounding complex visual queries by treating scene-graph grounding as a joint inference problem, enabling models to leverage relational context rather than being hindered by it.

Abstract: Grounding complex, compositional visual queries with multiple objects and relationships is a fundamental challenge for vision-language models. While standard phrase grounding methods excel at localizing single objects, they lack the structural inductive bias to parse intricate relational descriptions, often failing as queries become more descriptive. To address this structural deficit, we focus on scene-graph grounding, a powerful but less-explored formulation where the query is an explicit graph of objects and their relationships. However, existing methods for this task also struggle, paradoxically showing decreased performance as the query graph grows – failing to leverage the very information that should make grounding easier. We introduce SceneProp, a novel method that resolves this issue by reformulating scene-graph grounding as a Maximum a Posteriori (MAP) inference problem in a Markov Random Field (MRF). By performing global inference over the entire query graph, SceneProp finds the optimal assignment of image regions to nodes that jointly satisfies all constraints. This is achieved within an end-to-end framework via a differentiable implementation of the Belief Propagation algorithm. Experiments on four benchmarks show that our dedicated focus on the scene-graph grounding formulation allows SceneProp to significantly outperform prior work. Critically, its accuracy consistently improves with the size and complexity of the query graph, demonstrating for the first time that more relational context can, and should, lead to better grounding. Codes are available at https://github.com/keitaotani/SceneProp.

Method: 1) Coarse-to-fine binary encoding scheme compresses per-Gaussian category features into single integers via binary-to-decimal mapping; 2) Progressive training strategy decomposes panoptic segmentation into independent sub-tasks; 3) Fine-tuning opacity during segmentation training to address photometric rendering vs. semantic segmentation incompatibility.

Result: Achieves state-of-the-art segmentation performance on multiple benchmarks while significantly reducing memory consumption and accelerating inference.

Conclusion: The proposed binary encoding and progressive training approach effectively addresses memory and fine-grained segmentation challenges in 3D-GS, making it a practical solution for semantic segmentation tasks.

Abstract: 3D Gaussian Splatting (3D-GS) has emerged as an efficient 3D representation and a promising foundation for semantic tasks like segmentation. However, existing 3D-GS-based segmentation methods typically rely on high-dimensional category features, which introduce substantial memory overhead. Moreover, fine-grained segmentation remains challenging due to label space congestion and the lack of stable multi-granularity control mechanisms. To address these limitations, we propose a coarse-to-fine binary encoding scheme for per-Gaussian category representation, which compresses each feature into a single integer via the binary-to-decimal mapping, drastically reducing memory usage. We further design a progressive training strategy that decomposes panoptic segmentation into a series of independent sub-tasks, reducing inter-class conflicts and thereby enhancing fine-grained segmentation capability. Additionally, we fine-tune opacity during segmentation training to address the incompatibility between photometric rendering and semantic segmentation, which often leads to foreground-background confusion. Extensive experiments on multiple benchmarks demonstrate that our method achieves state-of-the-art segmentation performance while significantly reducing memory consumption and accelerating inference.

Method: DEMR framework with: 1) Reflective Flipped Fusion (RFF) block for cross-modal alignment, 2) query reconstruction task to enhance text sensitivity, and 3) Geom-regularizer to refine uncertainty predictions for adaptive alignment with difficult moments.

Result: Extensive testing shows significant improvements in effectiveness, robustness, and interpretability on standard datasets and debiased datasets ActivityNet-CD and Charades-CD.

Conclusion: DEMR provides a promising solution for temporal-semantic robustness in moment retrieval by addressing uncertainty estimation biases and modality imbalance through debiased evidential learning.

Abstract: In the domain of moment retrieval, accurately identifying temporal segments within videos based on natural language queries remains challenging. Traditional methods often employ pre-trained models that struggle with fine-grained information and deterministic reasoning, leading to difficulties in aligning with complex or ambiguous moments. To overcome these limitations, we explore Deep Evidential Regression (DER) to construct a vanilla Evidential baseline. However, this approach encounters two major issues: the inability to effectively handle modality imbalance and the structural differences in DER’s heuristic uncertainty regularizer, which adversely affect uncertainty estimation. This misalignment results in high uncertainty being incorrectly associated with accurate samples rather than challenging ones. Our observations indicate that existing methods lack the adaptability required for complex video scenarios. In response, we propose Debiased Evidential Learning for Moment Retrieval (DEMR), a novel framework that incorporates a Reflective Flipped Fusion (RFF) block for cross-modal alignment and a query reconstruction task to enhance text sensitivity, thereby reducing bias in uncertainty estimation. Additionally, we introduce a Geom-regularizer to refine uncertainty predictions, enabling adaptive alignment with difficult moments and improving retrieval accuracy. Extensive testing on standard datasets and debiased datasets ActivityNet-CD and Charades-CD demonstrates significant enhancements in effectiveness, robustness, and interpretability, positioning our approach as a promising solution for temporal-semantic robustness in moment retrieval. The code is publicly available at https://github.com/KaijingOfficial/DEMR.

Method: 4DHOISolver: an efficient optimization framework that constrains ill-posed 4D HOI reconstruction using sparse human-in-the-loop contact point annotations while maintaining spatio-temporal coherence and physical plausibility.

Result: Created Open4DHOI dataset with 144 object types and 103 actions. Demonstrated RL-based agents can imitate recovered motions. Benchmark shows existing 3D foundation models cannot automatically predict precise human-object contact correspondences.

Conclusion: Human-in-the-loop strategy is currently necessary for accurate 4D HOI reconstruction. The work provides a valuable dataset and framework while highlighting the open challenge of automated contact prediction for the community.

Abstract: Generalized robots must learn from diverse, large-scale human-object interactions (HOI) to operate robustly in the real world. Monocular internet videos offer a nearly limitless and readily available source of data, capturing an unparalleled diversity of human activities, objects, and environments. However, accurately and scalably extracting 4D interaction data from these in-the-wild videos remains a significant and unsolved challenge. Thus, in this work, we introduce 4DHOISolver, a novel and efficient optimization framework that constrains the ill-posed 4D HOI reconstruction problem by leveraging sparse, human-in-the-loop contact point annotations, while maintaining high spatio-temporal coherence and physical plausibility. Leveraging this framework, we introduce Open4DHOI, a new large-scale 4D HOI dataset featuring a diverse catalog of 144 object types and 103 actions. Furthermore, we demonstrate the effectiveness of our reconstructions by enabling an RL-based agent to imitate the recovered motions. However, a comprehensive benchmark of existing 3D foundation models indicates that automatically predicting precise human-object contact correspondences remains an unsolved problem, underscoring the immediate necessity of our human-in-the-loop strategy while posing an open challenge to the community. Data and code will be publicly available at https://wenboran2002.github.io/open4dhoi/

Method: MM-ACT integrates text, image, and action in shared token space with Context-Shared Multimodal Learning. Uses re-mask parallel decoding for text/image generation and one-step parallel decoding for action generation to improve efficiency.

Result: Achieves 96.3% success rate on LIBERO simulation, 72.0% across three real Franka tasks, and 52.38% across eight bimanual tasks on RoboTwin2.0, with additional 9.25% gain from cross-modal learning.

Conclusion: MM-ACT demonstrates effective unified multimodal learning for robotics, showing strong in-domain and out-of-domain performance with cross-modal learning providing significant improvements. The shared token space approach enables better integration of semantic understanding and action generation.

Abstract: A generalist robotic policy needs both semantic understanding for task planning and the ability to interact with the environment through predictive capabilities. To tackle this, we present MM-ACT, a unified Vision-Language-Action (VLA) model that integrates text, image, and action in shared token space and performs generation across all three modalities. MM-ACT adopts a re-mask parallel decoding strategy for text and image generation, and employs a one-step parallel decoding strategy for action generation to improve efficiency. We introduce Context-Shared Multimodal Learning, a unified training paradigm that supervises generation in all three modalities from a shared context, enhancing action generation through cross-modal learning. Experiments were conducted on the LIBERO simulation and Franka real-robot setups as well as RoboTwin2.0 to assess in-domain and out-of-domain performances respectively. Our approach achieves a success rate of 96.3% on LIBERO, 72.0% across three tasks of real Franka, and 52.38% across eight bimanual tasks of RoboTwin2.0 with an additional gain of 9.25% from cross-modal learning. We release our codes, models and data at https://github.com/HHYHRHY/MM-ACT.

Method: The framework uses a multi-modal model trained on a curated large-scale dataset. Composition guidance is organized hierarchically into shift, zoom-in, and view-change tasks. Training data is synthesized by degrading well-composed photos into poor ones using a learned degradation model applied to multi-view datasets and expert photos.

Result: Experiments show that textual instructions effectively steer image composition, and combining them with visual exemplars consistently outperforms exemplar-only baselines.

Conclusion: PhotoFramer represents a practical step toward composition assistants that make expert photographic knowledge accessible to everyday users, with released code, models, and datasets.

Abstract: Composition matters during the photo-taking process, yet many casual users struggle to frame well-composed images. To provide composition guidance, we introduce PhotoFramer, a multi-modal composition instruction framework. Given a poorly composed image, PhotoFramer first describes how to improve the composition in natural language and then generates a well-composed example image. To train such a model, we curate a large-scale dataset. Inspired by how humans take photos, we organize composition guidance into a hierarchy of sub-tasks: shift, zoom-in, and view-change tasks. Shift and zoom-in data are sampled from existing cropping datasets, while view-change data are obtained via a two-stage pipeline. First, we sample pairs with varying viewpoints from multi-view datasets, and train a degradation model to transform well-composed photos into poorly composed ones. Second, we apply this degradation model to expert-taken photos to synthesize poor images to form training pairs. Using this dataset, we finetune a model that jointly processes and generates both text and images, enabling actionable textual guidance with illustrative examples. Extensive experiments demonstrate that textual instructions effectively steer image composition, and coupling them with exemplars yields consistent improvements over exemplar-only baselines. PhotoFramer offers a practical step toward composition assistants that make expert photographic priors accessible to everyday users. Codes, model weights, and datasets have been released in https://zhiyuanyou.github.io/photoframer.

Method: Proposes S2AM3D with three key components: 1) Point-consistent part encoder that aggregates multi-view 2D features through native 3D contrastive learning for globally consistent point features, 2) Scale-aware prompt decoder enabling real-time adjustment of segmentation granularity via continuous scale signals, and 3) A large-scale, high-quality part-level point cloud dataset with over 100k samples.

Result: Extensive experiments demonstrate that S2AM3D achieves leading performance across multiple evaluation settings, exhibiting exceptional robustness and controllability when handling complex structures and parts with significant size variations.

Conclusion: S2AM3D effectively addresses the challenges of 3D part segmentation by combining 2D priors with 3D consistency, providing a robust and controllable solution that outperforms existing methods while handling complex structures and varying part sizes.

Abstract: Part-level point cloud segmentation has recently attracted significant attention in 3D computer vision. Nevertheless, existing research is constrained by two major challenges: native 3D models lack generalization due to data scarcity, while introducing 2D pre-trained knowledge often leads to inconsistent segmentation results across different views. To address these challenges, we propose S2AM3D, which incorporates 2D segmentation priors with 3D consistent supervision. We design a point-consistent part encoder that aggregates multi-view 2D features through native 3D contrastive learning, producing globally consistent point features. A scale-aware prompt decoder is then proposed to enable real-time adjustment of segmentation granularity via continuous scale signals. Simultaneously, we introduce a large-scale, high-quality part-level point cloud dataset with more than 100k samples, providing ample supervision signals for model training. Extensive experiments demonstrate that S2AM3D achieves leading performance across multiple evaluation settings, exhibiting exceptional robustness and controllability when handling complex structures and parts with significant size variations.

Method: Semantic-aware medical image reconstruction framework integrating high-level latent embeddings with hybrid U-Net architecture to preserve clinically relevant structures. Includes lightweight blockchain-based provenance layer using scale-free graph design for verifiable recording of reconstruction events.

Result: Extensive evaluation across multiple datasets and corruption types demonstrates improved structural consistency, restoration accuracy, and provenance integrity compared to existing approaches.

Conclusion: The solution advances dependable AI for medical imaging by uniting semantic-guided reconstruction with secure traceability, enhancing both diagnostic confidence and regulatory compliance in healthcare environments.

Abstract: Medical imaging is essential for clinical diagnosis, yet real-world data frequently suffers from corruption, noise, and potential tampering, challenging the reliability of AI-assisted interpretation. Conventional reconstruction techniques prioritize pixel-level recovery and may produce visually plausible outputs while compromising anatomical fidelity, an issue that can directly impact clinical outcomes. We propose a semantic-aware medical image reconstruction framework that integrates high-level latent embeddings with a hybrid U-Net architecture to preserve clinically relevant structures during restoration. To ensure trust and accountability, we incorporate a lightweight blockchain-based provenance layer using scale-free graph design, enabling verifiable recording of each reconstruction event without imposing significant overhead. Extensive evaluation across multiple datasets and corruption types demonstrates improved structural consistency, restoration accuracy, and provenance integrity compared with existing approaches. By uniting semantic-guided reconstruction with secure traceability, our solution advances dependable AI for medical imaging, enhancing both diagnostic confidence and regulatory compliance in healthcare environments.

Method: Two-stage framework: 1) Retrofit LISA with geometry-aware LoRA layers for cross-view consistent masks, 2) Use frozen SAM-3D reconstructor with RGBA prompts (RGB + masks) to output Gaussian splats or textured meshes without retraining.

Result: Improves language-to-3D accuracy by up to +15.6 points over single-view baselines on ScanRefer and Nr3D datasets, adapting only 11.6M parameters while supporting zero-shot deployment on unseen categories.

Conclusion: LISA-3D provides a practical, modular, and data-efficient recipe for language-guided 3D content creation by effectively combining 2D language-image segmentation with 3D reconstruction without requiring 3D-text supervision.

Abstract: Text-driven 3D reconstruction demands a mask generator that simultaneously understands open-vocabulary instructions and remains consistent across viewpoints. We present LISA-3D, a two-stage framework that lifts language-image segmentation into 3D by retrofitting the instruction-following model LISA with geometry-aware Low-Rank Adaptation (LoRA) layers and reusing a frozen SAM-3D reconstructor. During training we exploit off-the-shelf RGB-D sequences and their camera poses to build a differentiable reprojection loss that enforces cross-view agreement without requiring any additional 3D-text supervision. The resulting masks are concatenated with RGB images to form RGBA prompts for SAM-3D, which outputs Gaussian splats or textured meshes without retraining. Across ScanRefer and Nr3D, LISA-3D improves language-to-3D accuracy by up to +15.6 points over single-view baselines while adapting only 11.6M parameters. The system is modular, data-efficient, and supports zero-shot deployment on unseen categories, providing a practical recipe for language-guided 3D content creation. Our code will be available at https://github.com/binisalegend/LISA-3D.

Method: Two-stage framework: 1) Core predictor with single-step deterministic formulation, clean-data objective, and lightweight local continuity module for globally coherent structures. 2) Detail sharpener with constrained multi-step rectified-flow refinement within the manifold to enhance fine-grained geometry.

Result: Using only 59K training samples (<1% of existing datasets), achieves SOTA in monocular depth estimation and competitive surface normal prediction, demonstrating diffusion models can serve as deterministic world priors.

Conclusion: Diffusion models can be adapted as deterministic world priors for high-quality geometric reasoning, bridging traditional discriminative and generative paradigms with minimal training data.

Abstract: Recovering pixel-wise geometric properties from a single image is fundamentally ill-posed due to appearance ambiguity and non-injective mappings between 2D observations and 3D structures. While discriminative regression models achieve strong performance through large-scale supervision, their success is bounded by the scale, quality and diversity of available data and limited physical reasoning. Recent diffusion models exhibit powerful world priors that encode geometry and semantics learned from massive image-text data, yet directly reusing their stochastic generative formulation is suboptimal for deterministic geometric inference: the former is optimized for diverse and high-fidelity image generation, whereas the latter requires stable and accurate predictions. In this work, we propose Lotus-2, a two-stage deterministic framework for stable, accurate and fine-grained geometric dense prediction, aiming to provide an optimal adaption protocol to fully exploit the pre-trained generative priors. Specifically, in the first stage, the core predictor employs a single-step deterministic formulation with a clean-data objective and a lightweight local continuity module (LCM) to generate globally coherent structures without grid artifacts. In the second stage, the detail sharpener performs a constrained multi-step rectified-flow refinement within the manifold defined by the core predictor, enhancing fine-grained geometry through noise-free deterministic flow matching. Using only 59K training samples, less than 1% of existing large-scale datasets, Lotus-2 establishes new state-of-the-art results in monocular depth estimation and highly competitive surface normal prediction. These results demonstrate that diffusion models can serve as deterministic world priors, enabling high-quality geometric reasoning beyond traditional discriminative and generative paradigms.

Method: TRoVe extracts candidate static features from annotated validation datasets and scores each feature based on: (1) its effect on classification errors, and (2) the extent to which the VLM relies on the feature for predictions.

Result: TRoVe achieves 28.6% improvement over the closest baseline in identifying error-inducing static feature biases. When applied to 7 off-the-shelf VLMs and 2 temporal understanding tasks, it surfaces previously-unknown biases and shows this knowledge can improve test-time performance.

Conclusion: TRoVe provides an effective automated approach for discovering static feature biases in temporal VLMs, enabling better model understanding and potential performance improvements through bias awareness.

Abstract: Vision-language models (VLMs) have made great strides in addressing temporal understanding tasks, which involve characterizing visual changes across a sequence of images. However, recent works have suggested that when making predictions, VLMs may rely on static feature biases, such as background or object features, rather than dynamic visual changes. Static feature biases are a type of shortcut and can contribute to systematic prediction errors on downstream tasks; as a result, identifying and characterizing error-inducing static feature biases is critical prior to real-world model deployment. In this work, we introduce TRoVe, an automated approach for discovering error-inducing static feature biases learned by temporal VLMs. Given a trained VLM and an annotated validation dataset associated with a downstream classification task, TRoVe extracts candidate static features from the dataset and scores each feature by (i) the effect of the feature on classification errors as well as (ii) the extent to which the VLM relies on the feature when making predictions. In order to quantitatively evaluate TRoVe, we introduce an evaluation framework consisting of 101 trained temporal VLMs paired with ground-truth annotations for learned static feature biases. We use this framework to demonstrate that TRoVe can accurately identify error-inducing static feature biases in VLMs, achieving a 28.6% improvement over the closest baseline. Finally, we apply TRoVe to 7 off-the-shelf VLMs and 2 temporal understanding tasks, surfacing previously-unknown static feature biases and demonstrating that knowledge of learned biases can aid in improving model performance at test time. Our code is available at https://github.com/Stanford-AIMI/TRoVe.

Method: Two simple parameter-reduction strategies applied to MLP blocks of ViT-B/16: 1) GroupedMLP - shares MLP weights between adjacent transformer blocks (32.7% parameter reduction), and 2) ShallowMLP - halves the MLP hidden dimension (32.7% parameter reduction). Both maintain baseline computational cost or improve inference throughput.

Result: GroupedMLP achieves 81.47% top-1 accuracy (vs baseline 81.05%) with same computational cost. ShallowMLP reaches 81.25% accuracy with 38% increase in inference throughput. Both show substantially improved training stability, reducing peak-to-final accuracy degradation from 0.47% to 0.03-0.06%.

Conclusion: ViT-B/16 on ImageNet-1K operates in an overparameterized regime where MLP capacity can be reduced without harming performance. Architectural constraints like parameter sharing and reduced width act as useful inductive biases, highlighting the importance of parameter allocation in Vision Transformer design.

Abstract: Although scaling laws and many empirical results suggest that increasing the size of Vision Transformers often improves performance, model accuracy and training behavior are not always monotonically increasing with scale. Focusing on ViT-B/16 trained on ImageNet-1K, we study two simple parameter-reduction strategies applied to the MLP blocks, each removing 32.7% of the baseline parameters. Our \emph{GroupedMLP} variant shares MLP weights between adjacent transformer blocks and achieves 81.47% top-1 accuracy while maintaining the baseline computational cost. Our \emph{ShallowMLP} variant halves the MLP hidden dimension and reaches 81.25% top-1 accuracy with a 38% increase in inference throughput. Both models outperform the 86.6M-parameter baseline (81.05%) and exhibit substantially improved training stability, reducing peak-to-final accuracy degradation from 0.47% to the range 0.03% to 0.06%. These results suggest that, for ViT-B/16 on ImageNet-1K with a standard training recipe, the model operates in an overparameterized regime in which MLP capacity can be reduced without harming performance and can even slightly improve it. More broadly, our findings suggest that architectural constraints such as parameter sharing and reduced width may act as useful inductive biases, and highlight the importance of how parameters are allocated when designing Vision Transformers. All code is available at: https://github.com/AnanthaPadmanaban-KrishnaKumar/parameter-efficient-vit-mlps.

Method: Proposes Speed-RL, a reinforcement learning paradigm that combines quality and speed rewards to finetune pretrained MGMs. Instead of matching teacher-student distributions, it directly optimizes for generating high-quality images in fewer steps using RL.

Result: The method accelerates base models by 3x while maintaining comparable image quality, as demonstrated through extensive experiments.

Conclusion: Speed-RL offers an effective reinforcement learning alternative to conventional distillation for accelerating masked generative models, achieving significant speedup without sacrificing image quality.

Abstract: Masked Generative Models (MGM)s demonstrate strong capabilities in generating high-fidelity images. However, they need many sampling steps to create high-quality generations, resulting in slow inference speed. In this work, we propose Speed-RL, a novel paradigm for accelerating a pretrained MGMs to generate high-quality images in fewer steps. Unlike conventional distillation methods which formulate the acceleration problem as a distribution matching problem, where a few-step student model is trained to match the distribution generated by a many-step teacher model, we consider this problem as a reinforcement learning problem. Since the goal of acceleration is to generate high quality images in fewer steps, we can combine a quality reward with a speed reward and finetune the base model using reinforcement learning with the combined reward as the optimization target. Through extensive experiments, we show that the proposed method was able to accelerate the base model by a factor of 3x while maintaining comparable image quality.

Method: Created synthetic video sequences with periodic patterns in object motion and visual attributes, using a tiered evaluation system with progressive difficulty through variations in cyclic objects, scene clutter, and lighting conditions.

Result: Current VLMs (both open-source and proprietary) struggle with cyclical dynamics, lack temporal understanding, cannot extract quantitative insights, and show no consistent performance correlation with model size or architecture.

Conclusion: CycliST provides a comprehensive evaluation framework that highlights significant technical gaps in current VLMs and paves the way for improved visual reasoning models capable of understanding periodic patterns.

Abstract: We present CycliST, a novel benchmark dataset designed to evaluate Video Language Models (VLM) on their ability for textual reasoning over cyclical state transitions. CycliST captures fundamental aspects of real-world processes by generating synthetic, richly structured video sequences featuring periodic patterns in object motion and visual attributes. CycliST employs a tiered evaluation system that progressively increases difficulty through variations in the number of cyclic objects, scene clutter, and lighting conditions, challenging state-of-the-art models on their spatio-temporal cognition. We conduct extensive experiments with current state-of-the-art VLMs, both open-source and proprietary, and reveal their limitations in generalizing to cyclical dynamics such as linear and orbital motion, as well as time-dependent changes in visual attributes like color and scale. Our results demonstrate that present-day VLMs struggle to reliably detect and exploit cyclic patterns, lack a notion of temporal understanding, and are unable to extract quantitative insights from scenes, such as the number of objects in motion, highlighting a significant technical gap that needs to be addressed. More specifically, we find no single model consistently leads in performance: neither size nor architecture correlates strongly with outcomes, and no model succeeds equally well across all tasks. By providing a targeted challenge and a comprehensive evaluation framework, CycliST paves the way for visual reasoning models that surpass the state-of-the-art in understanding periodic patterns.

Method: The method trains a network to decompose an implicit approximation operator by minimizing reconstruction error in the learned basis over a distribution of probe functions. Grounded in optimal-approximation theory, it learns spectral basis, eigenvalues, and metric sampling density simultaneously without explicit operator construction.

Result: The approach successfully recovers meaningful spectral bases that resemble those of the Laplacian operator on both 3D surface point clouds and high-dimensional image manifolds, without explicit operator construction. It scales to arbitrary datasets of any dimension.

Conclusion: This learning-based framework provides a principled, data-driven alternative to conventional spectral analysis pipelines, eliminating the need for operator selection, discretization, and eigendecomposition. It opens new possibilities for geometry processing on unstructured data, especially in high-dimensional spaces.

Abstract: We introduce a novel framework that directly learns a spectral basis for shape and manifold analysis from unstructured data, eliminating the need for traditional operator selection, discretization, and eigensolvers. Grounded in optimal-approximation theory, we train a network to decompose an implicit approximation operator by minimizing the reconstruction error in the learned basis over a chosen distribution of probe functions. For suitable distributions, they can be seen as an approximation of the Laplacian operator and its eigendecomposition, which are fundamental in geometry processing. Furthermore, our method recovers in a unified manner not only the spectral basis, but also the implicit metric’s sampling density and the eigenvalues of the underlying operator. Notably, our unsupervised method makes no assumption on the data manifold, such as meshing or manifold dimensionality, allowing it to scale to arbitrary datasets of any dimension. On point clouds lying on surfaces in 3D and high-dimensional image manifolds, our approach yields meaningful spectral bases, that can resemble those of the Laplacian, without explicit construction of an operator. By replacing the traditional operator selection, construction, and eigendecomposition with a learning-based approach, our framework offers a principled, data-driven alternative to conventional pipelines. This opens new possibilities in geometry processing for unstructured data, particularly in high-dimensional spaces.

Method: SlotSPE uses slot attention to compress multimodal inputs into compact, modality-specific sets of mutually distinctive slots, inspired by factorial coding. These slots represent prognostic events and enable efficient modeling of complex interactions while allowing incorporation of biological priors.

Result: Outperforms existing methods on 8 out of 10 cancer benchmarks with 2.9% overall improvement. Remains robust under missing genomic data and provides improved interpretability through structured event decomposition.

Conclusion: SlotSPE effectively models sparse structural prognostic events in multimodal cancer data, achieving superior survival prediction performance while offering enhanced interpretability and robustness to missing data.

Abstract: The integration of histology images and gene profiles has shown great promise for improving survival prediction in cancer. However, current approaches often struggle to model intra- and inter-modal interactions efficiently and effectively due to the high dimensionality and complexity of the inputs. A major challenge is capturing critical prognostic events that, though few, underlie the complexity of the observed inputs and largely determine patient outcomes. These events, manifested as high-level structural signals such as spatial histologic patterns or pathway co-activations, are typically sparse, patient-specific, and unannotated, making them inherently difficult to uncover. To address this, we propose SlotSPE, a slot-based framework for structural prognostic event modeling. Specifically, inspired by the principle of factorial coding, we compress each patient’s multimodal inputs into compact, modality-specific sets of mutually distinctive slots using slot attention. By leveraging these slot representations as encodings for prognostic events, our framework enables both efficient and effective modeling of complex intra- and inter-modal interactions, while also facilitating seamless incorporation of biological priors that enhance prognostic relevance. Extensive experiments on ten cancer benchmarks show that SlotSPE outperforms existing methods in 8 out of 10 cohorts, achieving an overall improvement of 2.9%. It remains robust under missing genomic data and delivers markedly improved interpretability through structured event decomposition.

Method: Three components: (1) shared Swin Transformer encoder for unified 4D spatiotemporal representations, (2) cross-task interaction module with learnable queries for attention-based inter-task dependencies, (3) lightweight decoding heads for task-specific predictions.

Result: Superior performance across benchmarks with high efficiency: 4.63% video classification improvement with image data, 63% parameter reduction, 50% training time reduction compared to task-specific models.

Conclusion: OmniFD establishes a practical, generalizable solution for comprehensive face forgery detection by unifying multiple tasks in one framework, leveraging multi-task learning for generalized representations and knowledge transfer.

Abstract: Face forgery detection encompasses multiple critical tasks, including identifying forged images and videos and localizing manipulated regions and temporal segments. Current approaches typically employ task-specific models with independent architectures, leading to computational redundancy and ignoring potential correlations across related tasks. We introduce OmniFD, a unified framework that jointly addresses four core face forgery detection tasks within a single model, i.e., image and video classification, spatial localization, and temporal localization. Our architecture consists of three principal components: (1) a shared Swin Transformer encoder that extracts unified 4D spatiotemporal representations from both images and video inputs, (2) a cross-task interaction module with learnable queries that dynamically captures inter-task dependencies through attention-based reasoning, and (3) lightweight decoding heads that transform refined representations into corresponding predictions for all FFD tasks. Extensive experiments demonstrate OmniFD’s advantage over task-specific models. Its unified design leverages multi-task learning to capture generalized representations across tasks, especially enabling fine-grained knowledge transfer that facilitates other tasks. For example, video classification accuracy improves by 4.63% when image data are incorporated. Furthermore, by unifying images, videos and the four tasks within one framework, OmniFD achieves superior performance across diverse benchmarks with high efficiency and scalability, e.g., reducing 63% model parameters and 50% training time. It establishes a practical and generalizable solution for comprehensive face forgery detection in real-world applications. The source code is made available at https://github.com/haotianll/OmniFD.

Method: Uses triangular prior to convert sparse temporal landmarks into dense pseudo-intensity trajectories. A lightweight temporal regression model combines ResNet18 encoder with bidirectional GRU to predict frame-wise intensity directly from image sequences. Requires no frame-level annotation and uses consistent preprocessing pipeline.

Result: On SAMM: Spearman correlation 0.9014, Kendall correlation 0.7999. On CASME II: up to 0.9116 and 0.8168 respectively when trained without apex-ranking term. Outperforms frame-wise baseline. Strong temporal agreement with pseudo-intensity trajectories.

Conclusion: First unified approach for continuous micro-expression intensity estimation using only sparse temporal annotations. Temporal modeling and structured pseudo labels are central to capturing rise-apex-fall dynamics of micro-facial movements.

Abstract: Micro-facial expressions are brief and involuntary facial movements that reflect genuine emotional states. While most prior work focuses on classifying discrete micro-expression categories, far fewer studies address the continuous evolution of intensity over time. Progress in this direction is limited by the lack of frame-level intensity labels, which makes fully supervised regression impractical. We propose a unified framework for continuous micro-expression intensity estimation using only weak temporal labels (onset, apex, offset). A simple triangular prior converts sparse temporal landmarks into dense pseudo-intensity trajectories, and a lightweight temporal regression model that combines a ResNet18 encoder with a bidirectional GRU predicts frame-wise intensity directly from image sequences. The method requires no frame-level annotation effort and is applied consistently across datasets through a single preprocessing and temporal alignment pipeline. Experiments on SAMM and CASME II show strong temporal agreement with the pseudo-intensity trajectories. On SAMM, the model reaches a Spearman correlation of 0.9014 and a Kendall correlation of 0.7999, outperforming a frame-wise baseline. On CASME II, it achieves up to 0.9116 and 0.8168, respectively, when trained without the apex-ranking term. Ablation studies confirm that temporal modeling and structured pseudo labels are central to capturing the rise-apex-fall dynamics of micro-facial movements. To our knowledge, this is the first unified approach for continuous micro-expression intensity estimation using only sparse temporal annotations.

Method: Propose SocialFusion framework that learns minimal connection between frozen visual encoder and language model. Investigate social degradation through linear representation probing (decodability) and gradient conflict analysis (compatibility).

Result: SocialFusion exhibits positive transfer across all five social tasks, leverages synergies between tasks to enhance overall performance, and achieves comparable performance to task-specific state-of-the-art models on various benchmarks.

Conclusion: Current VLM pre-training strategies may be detrimental to acquiring general social competence, highlighting need for more socially-aware training paradigms. SocialFusion demonstrates effective approach to unified social perception learning.

Abstract: Understanding social interactions from visual cues is a fundamental challenge for a socially competent AI. While powerful pre-trained vision-language models (VLMs) have shown remarkable general capabilities, they surprisingly struggle to unify and learn multiple social perception tasks simultaneously, often exhibiting negative transfer. We identify that this negative transfer stems from a critical issue we term “social degradation,” whereby the general visual-linguistic pre-training process of VLMs impairs the visual encoder’s ability to represent nuanced social information. We investigate this behavior further under two lenses: decodability through linear representation probing and compatibility through gradient conflict analysis, revealing that both play a role in the degradation, especially the former, which is significantly compromised in the VLM pre-training process. To address these issues, we propose SocialFusion, a unified framework that learns a minimal connection between a frozen visual encoder and a language model. Compared with existing VLMs, it exhibits positive transfer across all five social tasks, leveraging synergies between them to enhance overall performance and achieves comparable performance to task-specific state-of-the-art models on various benchmarks. Our findings suggest that current VLM pre-training strategies may be detrimental to acquiring general social competence and highlight the need for more socially-aware training paradigms.

Method: Formalize degradation as path-space KL divergence, derive optimal control via Girsanov’s theorem, and propose DPAC which projects adversarial gradients onto tangent spaces orthogonal to the score function.

Result: DPAC achieves lower FID and path-KL at matched attack success rates, with theoretical guarantees of O(Δt²) quality gap in discrete solvers.

Conclusion: Distribution-preserving adversarial control (DPAC) provides principled connection between adversarial control energy and perceptual fidelity, enabling high-quality targeted sampling.

Abstract: Adversarially guided diffusion sampling often achieves the target class, but sample quality degrades as deviations between the adversarially controlled and nominal trajectories accumulate. We formalize this degradation as a path-space Kullback-Leibler divergence(path-KL) between controlled and nominal (uncontrolled) diffusion processes, thereby showing via Girsanov’s theorem that it exactly equals the control energy. Building on this stochastic optimal control (SOC) view, we theoretically establish that minimizing this path-KL simultaneously tightens upper bounds on both the 2-Wasserstein distance and Fréchet Inception Distance (FID), revealing a principled connection between adversarial control energy and perceptual fidelity. From a variational perspective, we derive a first-order optimality condition for the control: among all directions that yield the same classification gain, the component tangent to iso-(log-)density surfaces (i.e., orthogonal to the score) minimizes path-KL, whereas the normal component directly increases distributional drift. This leads to DPAC (Distribution-Preserving Adversarial Control), a diffusion guidance rule that projects adversarial gradients onto the tangent space defined by the generative score geometry. We further show that in discrete solvers, the tangent projection cancels the O(Δt) leading error term in the Wasserstein distance, achieving an O(Δt^2) quality gap; moreover, it remains second-order robust to score or metric approximation. Empirical studies on ImageNet-100 validate the theoretical predictions, confirming that DPAC achieves lower FID and estimated path-KL at matched attack success rates.

Method: Proposed a real-time annotation approach using YOLO models deployed on edge devices for immediate labeling during image capture. Conducted extensive comparative analysis of three YOLO architectures (YOLOv5, YOLOv8, YOLOv12) with different configurations: single-class vs multi-class annotation and pretrained vs scratch-based training.

Result: Pretrained and single-class configurations showed significant advantages in model convergence, performance, and robustness. The real-time annotation framework was validated as feasible and effective, capable of drastically reducing dataset preparation time while maintaining high annotation quality.

Conclusion: The proposed real-time annotation system using YOLO models on edge devices is a practical solution for agricultural applications, with pretrained single-class configurations offering the best performance for efficient and accurate dataset annotation.

Abstract: Efficient and accurate annotation of datasets remains a significant challenge for deploying object detection models such as You Only Look Once (YOLO) in real-world applications, particularly in agriculture where rapid decision-making is critical. Traditional annotation techniques are labor-intensive, requiring extensive manual labeling post data collection. This paper presents a novel real-time annotation approach leveraging YOLO models deployed on edge devices, enabling immediate labeling during image capture. To comprehensively evaluate the efficiency and accuracy of our proposed system, we conducted an extensive comparative analysis using three prominent YOLO architectures (YOLOv5, YOLOv8, YOLOv12) under various configurations: single-class versus multi-class annotation and pretrained versus scratch-based training. Our analysis includes detailed statistical tests and learning dynamics, demonstrating significant advantages of pretrained and single-class configurations in terms of model convergence, performance, and robustness. Results strongly validate the feasibility and effectiveness of our real-time annotation framework, highlighting its capability to drastically reduce dataset preparation time while maintaining high annotation quality.

Method: Two-stage pipeline: 1) Multi-view 3D autolabeling using signed distance fields (SDFs) with instance-aware volumetric silhouette rendering, decomposing SDFs into cuboid SDFs and residual distance fields, modeling dynamic objects with velocity attributes and confidence scores, and using 3D attribute initialization; 2) Training monocular 3D detectors using the generated pseudo-labels.

Result: Extensive experiments on KITTI-360 dataset show VSRD++ significantly outperforms existing weakly supervised approaches for monocular 3D object detection on both static and dynamic scenes.

Conclusion: VSRD++ successfully demonstrates that weakly supervised monocular 3D object detection is feasible by leveraging neural-field-based volumetric rendering with only 2D supervision, eliminating the need for expensive 3D annotations while achieving strong performance.

Abstract: Monocular 3D object detection is a fundamental yet challenging task in 3D scene understanding. Existing approaches heavily depend on supervised learning with extensive 3D annotations, which are often acquired from LiDAR point clouds through labor-intensive labeling processes. To tackle this problem, we propose VSRD++, a novel weakly supervised framework for monocular 3D object detection that eliminates the reliance on 3D annotations and leverages neural-field-based volumetric rendering with weak 2D supervision. VSRD++ consists of a two-stage pipeline: multi-view 3D autolabeling and subsequent monocular 3D detector training. In the multi-view autolabeling stage, object surfaces are represented as signed distance fields (SDFs) and rendered as instance masks via the proposed instance-aware volumetric silhouette rendering. To optimize 3D bounding boxes, we decompose each instance’s SDF into a cuboid SDF and a residual distance field (RDF) that captures deviations from the cuboid. To address the geometry inconsistency commonly observed in volume rendering methods applied to dynamic objects, we model the dynamic objects by including velocity into bounding box attributes as well as assigning confidence to each pseudo-label. Moreover, we also employ a 3D attribute initialization module to initialize the dynamic bounding box parameters. In the monocular 3D object detection phase, the optimized 3D bounding boxes serve as pseudo labels for training monocular 3D object detectors. Extensive experiments on the KITTI-360 dataset demonstrate that VSRD++ significantly outperforms existing weakly supervised approaches for monocular 3D object detection on both static and dynamic scenes. Code is available at https://github.com/Magicboomliu/VSRD_plus_plus

Method: TabletopGen accepts reference images (can be synthesized by text-to-image models), performs instance segmentation/completion to get per-instance images, reconstructs each instance into 3D models with canonical coordinate alignment, then uses a novel two-stage pose/scale alignment approach (Differentiable Rotation Optimizer + Top-view Spatial Alignment) before assembling into collision-free, simulation-ready scenes.

Result: Extensive experiments and user studies show TabletopGen achieves state-of-the-art performance, surpassing existing methods in visual fidelity, layout accuracy, and physical plausibility, capable of generating realistic tabletop scenes with rich stylistic and spatial diversity.

Conclusion: TabletopGen provides an effective training-free framework for generating high-fidelity, physically interactive 3D tabletop scenes, addressing limitations of current methods and enabling better support for embodied AI applications like robotic manipulation.

Abstract: Generating high-fidelity, physically interactive 3D simulated tabletop scenes is essential for embodied AI–especially for robotic manipulation policy learning and data synthesis. However, current text- or image-driven 3D scene generation methods mainly focus on large-scale scenes, struggling to capture the high-density layouts and complex spatial relations that characterize tabletop scenes. To address these challenges, we propose TabletopGen, a training-free, fully automatic framework that generates diverse, instance-level interactive 3D tabletop scenes. TabletopGen accepts a reference image as input, which can be synthesized by a text-to-image model to enhance scene diversity. We then perform instance segmentation and completion on the reference to obtain per-instance images. Each instance is reconstructed into a 3D model followed by canonical coordinate alignment. The aligned 3D models then undergo pose and scale estimation before being assembled into a collision-free, simulation-ready tabletop scene. A key component of our framework is a novel pose and scale alignment approach that decouples the complex spatial reasoning into two stages: a Differentiable Rotation Optimizer for precise rotation recovery and a Top-view Spatial Alignment mechanism for robust translation and scale estimation, enabling accurate 3D reconstruction from 2D reference. Extensive experiments and user studies show that TabletopGen achieves state-of-the-art performance, markedly surpassing existing methods in visual fidelity, layout accuracy, and physical plausibility, capable of generating realistic tabletop scenes with rich stylistic and spatial diversity. Our code will be publicly available.

Method: Two instance-wise minimax reformulations: (1) with asymptotically vanishing gap, (2) unbiased with more variables. Key ideas: establish equivalent instance-wise problem to lower time complexity, simplify sample selection via threshold learning, apply different smoothing techniques.

Result: Algorithms achieve linear per-iteration computational complexity w.r.t. sample size and convergence rate of O(ε^{-1/3}) for typical PAUCs. Provides tight generalization bound showing impact of TPR/FPR constraints α/β with order of Õ(α^{-1}n_+^{-1} + β^{-1}n_-^{-1}). Extensive experiments validate method strength.

Conclusion: The proposed methods close the approximation gap in PAUC optimization with efficient solvers, linear complexity, and proven convergence rates, while providing theoretical generalization bounds that explicitly quantify constraint impacts.

Abstract: As a variant of the Area Under the ROC Curve (AUC), the partial AUC (PAUC) focuses on a specific range of false positive rate (FPR) and/or true positive rate (TPR) in the ROC curve. It is a pivotal evaluation metric in real-world scenarios with both class imbalance and decision constraints. However, selecting instances within these constrained intervals during its calculation is NP-hard, and thus typically requires approximation techniques for practical resolution. Despite the progress made in PAUC optimization over the last few years, most existing methods still suffer from uncontrollable approximation errors or a limited scalability when optimizing the approximate PAUC objectives. In this paper, we close the approximation gap of PAUC optimization by presenting two simple instance-wise minimax reformulations: one with an asymptotically vanishing gap, the other with the unbiasedness at the cost of more variables. Our key idea is to first establish an equivalent instance-wise problem to lower the time complexity, simplify the complicated sample selection procedure by threshold learning, and then apply different smoothing techniques. Equipped with an efficient solver, the resulting algorithms enjoy a linear per-iteration computational complexity w.r.t. the sample size and a convergence rate of $O(ε^{-1/3})$ for typical one-way and two-way PAUCs. Moreover, we provide a tight generalization bound of our minimax reformulations. The result explicitly demonstrates the impact of the TPR/FPR constraints $α$/$β$ on the generalization and exhibits a sharp order of $\tilde{O}(α^{-1}\n_+^{-1} + β^{-1}\n_-^{-1})$. Finally, extensive experiments on several benchmark datasets validate the strength of our proposed methods.

Method: Proposes M4-BLIP framework using BLIP-2 model for local feature extraction, incorporates local facial information as prior knowledge, includes alignment and fusion module to integrate local and global features, and integrates with Large Language Models for interpretability.

Result: Extensive quantitative and visualization experiments validate the framework’s effectiveness against state-of-the-art competitors, showing enhanced detection accuracy.

Conclusion: The M4-BLIP framework successfully addresses the limitation of overlooking localized information in media manipulation detection by focusing on facial regions and improving interpretability through LLM integration.

Abstract: In the contemporary digital landscape, multi-modal media manipulation has emerged as a significant societal threat, impacting the reliability and integrity of information dissemination. Current detection methodologies in this domain often overlook the crucial aspect of localized information, despite the fact that manipulations frequently occur in specific areas, particularly in facial regions. In response to this critical observation, we propose the M4-BLIP framework. This innovative framework utilizes the BLIP-2 model, renowned for its ability to extract local features, as the cornerstone for feature extraction. Complementing this, we incorporate local facial information as prior knowledge. A specially designed alignment and fusion module within M4-BLIP meticulously integrates these local and global features, creating a harmonious blend that enhances detection accuracy. Furthermore, our approach seamlessly integrates with Large Language Models (LLM), significantly improving the interpretability of the detection outcomes. Extensive quantitative and visualization experiments validate the effectiveness of our framework against the state-of-the-art competitors.

Method: Proposes S²-MLLM with spatial guidance strategy leveraging feed-forward 3D reconstruction’s structure awareness. Includes structure-enhanced module (SE) with intra-view/inter-view attention and multi-level position encoding to associate visual representations with spatial positions.

Result: Achieves significant performance improvements over existing methods across ScanRefer, Nr3D, and Sr3D datasets, unifying superior performance, generalization, and efficiency.

Conclusion: S²-MLLM effectively addresses inefficiency and limited spatial reasoning in existing 3DVG methods by enabling implicit spatial understanding without point cloud reconstruction, demonstrating strong performance across multiple benchmarks.

Abstract: 3D Visual Grounding (3DVG) focuses on locating objects in 3D scenes based on natural language descriptions, serving as a fundamental task for embodied AI and robotics. Recent advances in Multi-modal Large Language Models (MLLMs) have motivated research into extending them to 3DVG. However, MLLMs primarily process 2D visual inputs and struggle with understanding 3D spatial structure of scenes solely from these limited perspectives. Existing methods mainly utilize viewpoint-dependent rendering of reconstructed point clouds to provide explicit structural guidance for MLLMs in 3DVG tasks, leading to inefficiency and limited spatial reasoning. To address this issue, we propose S$^2$-MLLM, an efficient framework that enhances spatial reasoning in MLLMs through implicit spatial reasoning. We introduce a spatial guidance strategy that leverages the structure awareness of feed-forward 3D reconstruction. By acquiring 3D structural understanding during training, our model can implicitly reason about 3D scenes without relying on inefficient point cloud reconstruction. Moreover, we propose a structure-enhanced module (SE), which first employs intra-view and inter-view attention mechanisms to capture dependencies within views and correspondences across views. The module further integrates multi-level position encoding to associate visual representations with spatial positions and viewpoint information, enabling more accurate structural understanding. Extensive experiments demonstrate that S$^2$-MLLM unifies superior performance, generalization, and efficiency, achieving significant performance over existing methods across the ScanRefer, Nr3D, and Sr3D datasets. Code will be available upon acceptance.

Method: 1) Scalable multi-subject data generation pipeline using single-subject models; 2) Pairwise Subject-Consistency Rewards and general-purpose rewards; 3) Reinforcement learning stage for refinement; 4) New benchmark with 7 subsets across 3 dimensions.

Result: Extensive experiments demonstrate effectiveness in advancing multi-subject personalized image generation, with improved subject consistency and text controllability.

Conclusion: The proposed approach successfully addresses multi-subject personalization challenges through scalable data generation and refined reinforcement learning with specialized rewards.

Abstract: Personalized generation models for a single subject have demonstrated remarkable effectiveness, highlighting their significant potential. However, when extended to multiple subjects, existing models often exhibit degraded performance, particularly in maintaining subject consistency and adhering to textual prompts. We attribute these limitations to the absence of high-quality multi-subject datasets and refined post-training strategies. To address these challenges, we propose a scalable multi-subject data generation pipeline that leverages powerful single-subject generation models to construct diverse and high-quality multi-subject training data. Through this dataset, we first enable single-subject personalization models to acquire knowledge of synthesizing multi-image and multi-subject scenarios. Furthermore, to enhance both subject consistency and text controllability, we design a set of Pairwise Subject-Consistency Rewards and general-purpose rewards, which are incorporated into a refined reinforcement learning stage. To comprehensively evaluate multi-subject personalization, we introduce a new benchmark that assesses model performance using seven subsets across three dimensions. Extensive experiments demonstrate the effectiveness of our approach in advancing multi-subject personalized image generation. Github Link: https://github.com/wang-shulei/PSR

Method: Built on Group Relative Policy Optimization, TRivia automatically identifies effective unlabeled samples and uses a question-answering-based reward mechanism without human annotations. An attention-guided module generates diverse questions for each table image, and the model’s ability to interpret recognition results and answer correctly provides feedback for optimization.

Result: TRivia-3B, an open-sourced 3B parameter model, surpasses existing systems (including Gemini 2.5 Pro and MinerU2.5) on three popular benchmarks, demonstrating state-of-the-art performance.

Conclusion: TRivia enables pretrained VLMs to autonomously learn table recognition from unlabeled data, bridging the performance gap between proprietary and open-source models while eliminating the need for costly labeled datasets.

Abstract: Table recognition (TR) aims to transform table images into semi-structured representations such as HTML or Markdown. As a core component of document parsing, TR has long relied on supervised learning, with recent efforts dominated by fine-tuning vision-language models (VLMs) using labeled data. While VLMs have brought TR to the next level, pushing performance further demands large-scale labeled data that is costly to obtain. Consequently, although proprietary models have continuously pushed the performance boundary, open-source models, often trained with limited resources and, in practice, the only viable option for many due to privacy regulations, still lag far behind. To bridge this gap, we introduce TRivia, a self-supervised fine-tuning method that enables pretrained VLMs to learn TR directly from unlabeled table images in the wild. Built upon Group Relative Policy Optimization, TRivia automatically identifies unlabeled samples that most effectively facilitate learning and eliminates the need for human annotations through a question-answering-based reward mechanism. An attention-guided module generates diverse questions for each table image, and the ability to interpret the recognition results and answer them correctly provides feedback to optimize the TR model. This closed-loop process allows the TR model to autonomously learn to recognize, structure, and reason over tables without labeled data. Leveraging this pipeline, we present TRivia-3B, an open-sourced, compact, and state-of-the-art TR model that surpasses existing systems (e.g., Gemini 2.5 Pro, MinerU2.5) on three popular benchmarks. Model and code are released at: https://github.com/opendatalab/TRivia

Method: Computer vision system that infers viscosity by analyzing how a fixed background pattern becomes optically distorted as light refracts through the mixing-driven, continuously deforming free surface of the fluid.

Result: Achieves mean absolute error of 0.113 in log m² s⁻¹ units for regression and up to 81% accuracy in viscosity-class prediction. Multi-pattern strategy improves robustness, and uncertainty quantification provides confidence estimates.

Conclusion: The stand-off viscometer offers a practical, automation-ready alternative to existing viscometry methods, enabling non-invasive viscosity measurement in real process conditions.

Abstract: Viscosity measurement is essential for process monitoring and autonomous laboratory operation, yet conventional viscometers remain invasive and require controlled laboratory environments that differ substantially from real process conditions. We present a computer-vision-based viscometer that infers viscosity by exploiting how a fixed background pattern becomes optically distorted as light refracts through the mixing-driven, continuously deforming free surface. Under diverse lighting conditions, the system achieves a mean absolute error of 0.113 in log m2 s^-1 units for regression and reaches up to 81% accuracy in viscosity-class prediction. Although performance declines for classes with closely clustered viscosity values, a multi-pattern strategy improves robustness by providing enriched visual cues. To ensure sensor reliability, we incorporate uncertainty quantification, enabling viscosity predictions with confidence estimates. This stand-off viscometer offers a practical, automation-ready alternative to existing viscometry methods.

Method: Proposes nnMobileNet++, a hybrid architecture with three key components: 1) dynamic snake convolution for boundary-aware feature extraction, 2) stage-specific transformer blocks after the second down-sampling stage for global context modeling, and 3) retinal image pretraining to improve generalization.

Result: Achieves state-of-the-art or highly competitive accuracy on multiple public retinal datasets for classification tasks while maintaining low computational cost, as demonstrated through experiments and ablation studies.

Conclusion: nnMobileNet++ represents an effective lightweight framework for retinal image analysis that successfully bridges convolutional and transformer representations, addressing the limitations of purely convolutional architectures while maintaining computational efficiency.

Abstract: Retinal imaging is a critical, non-invasive modality for the early detection and monitoring of ocular and systemic diseases. Deep learning, particularly convolutional neural networks (CNNs), has significant progress in automated retinal analysis, supporting tasks such as fundus image classification, lesion detection, and vessel segmentation. As a representative lightweight network, nnMobileNet has demonstrated strong performance across multiple retinal benchmarks while remaining computationally efficient. However, purely convolutional architectures inherently struggle to capture long-range dependencies and model the irregular lesions and elongated vascular patterns that characterize on retinal images, despite the critical importance of vascular features for reliable clinical diagnosis. To further advance this line of work and extend the original vision of nnMobileNet, we propose nnMobileNet++, a hybrid architecture that progressively bridges convolutional and transformer representations. The framework integrates three key components: (i) dynamic snake convolution for boundary-aware feature extraction, (ii) stage-specific transformer blocks introduced after the second down-sampling stage for global context modeling, and (iii) retinal image pretraining to improve generalization. Experiments on multiple public retinal datasets for classification, together with ablation studies, demonstrate that nnMobileNet++ achieves state-of-the-art or highly competitive accuracy while maintaining low computational cost, underscoring its potential as a lightweight yet effective framework for retinal image analysis.

Method: Two key modules: 1) Targeted Contrastive Unlearning (TCU) extends triplet loss to reduce seafloor-induced background bias, 2) Unlearn to Explain Sonar Framework (UESF) provides visual insights into what the model has forgotten and adapts LIME explainer for better attribution evaluation.

Result: Extensive experiments on real and synthetic sonar datasets show significant improvements in unlearning effectiveness, model robustness, and interpretability compared to existing approaches.

Conclusion: The proposed framework successfully addresses seafloor bias in sonar image analysis, improving generalization while maintaining interpretability through novel unlearning and explanation techniques.

Abstract: Acoustic sonar image analysis plays a critical role in object detection and classification, with applications in both civilian and defense domains. Despite the availability of real and synthetic datasets, existing AI models that achieve high accuracy often over-rely on seafloor features, leading to poor generalization. To mitigate this issue, we propose a novel framework that integrates two key modules: (i) a Targeted Contrastive Unlearning (TCU) module, which extends the traditional triplet loss to reduce seafloor-induced background bias and improve generalization, and (ii) the Unlearn to Explain Sonar Framework (UESF), which provides visual insights into what the model has deliberately forgotten while adapting the LIME explainer to generate more faithful and localized attributions for unlearning evaluation. Extensive experiments across both real and synthetic sonar datasets validate our approach, demonstrating significant improvements in unlearning effectiveness, model robustness, and interpretability.

Method: Combines variational autoencoder (VAE) with latent diffusion model. VAE compresses input into low-dimensional latent space to reduce noise and accelerate training. Diffusion process operates in this compact representation. Uses weighted cross-entropy loss in VAE mask reconstruction to better preserve tiny structures like small nodules.

Result: Achieves state-of-the-art or highly competitive Dice and IoU scores on ISIC-2018 (skin lesions), CVC-Clinic (polyps), and LIDC-IDRI (lung nodules). Simultaneously generates diverse segmentation hypotheses and confidence maps for enhanced interpretability.

Conclusion: MedSegLatDiff provides efficient uncertainty-aware medical image segmentation with enhanced interpretability and reliability compared to deterministic baselines, making it particularly suitable for clinical deployment.

Abstract: Medical image segmentation is crucial for clinical diagnosis and treatment planning. Traditional methods typically produce a single segmentation mask, failing to capture inherent uncertainty. Recent generative models enable the creation of multiple plausible masks per image, mimicking the collaborative interpretation of several clinicians. However, these approaches remain computationally heavy. We propose MedSegLatDiff, a diffusion based framework that combines a variational autoencoder (VAE) with a latent diffusion model for efficient medical image segmentation. The VAE compresses the input into a low dimensional latent space, reducing noise and accelerating training, while the diffusion process operates directly in this compact representation. We further replace the conventional MSE loss with weighted cross entropy in the VAE mask reconstruction path to better preserve tiny structures such as small nodules. MedSegLatDiff is evaluated on ISIC-2018 (skin lesions), CVC-Clinic (polyps), and LIDC-IDRI (lung nodules). It achieves state of the art or highly competitive Dice and IoU scores while simultaneously generating diverse segmentation hypotheses and confidence maps. This provides enhanced interpretability and reliability compared to deterministic baselines, making the model particularly suitable for clinical deployment.

Method: Proposes EGG-Fusion with robust sparse-to-dense camera tracking and geometry-aware Gaussian surfel mapping module. Uses information filter-based fusion method that explicitly accounts for sensor noise. Differentiable Gaussian surfel mapping models multi-view consistent surfaces while enabling efficient parameter optimization.

Result: Achieves surface reconstruction error of 0.6cm on standardized benchmarks (Replica and ScanNet++), representing over 20% improvement in accuracy compared to state-of-the-art GS-based methods. Maintains real-time processing at 24 FPS.

Conclusion: EGG-Fusion establishes itself as one of the most accurate differentiable-rendering-based real-time reconstruction systems, successfully addressing both real-time computation and sensor noise challenges while achieving high-precision surface reconstruction.

Abstract: Real-time 3D reconstruction is a fundamental task in computer graphics. Recently, differentiable-rendering-based SLAM system has demonstrated significant potential, enabling photorealistic scene rendering through learnable scene representations such as Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS). Current differentiable rendering methods face dual challenges in real-time computation and sensor noise sensitivity, leading to degraded geometric fidelity in scene reconstruction and limited practicality. To address these challenges, we propose a novel real-time system EGG-Fusion, featuring robust sparse-to-dense camera tracking and a geometry-aware Gaussian surfel mapping module, introducing an information filter-based fusion method that explicitly accounts for sensor noise to achieve high-precision surface reconstruction. The proposed differentiable Gaussian surfel mapping effectively models multi-view consistent surfaces while enabling efficient parameter optimization. Extensive experimental results demonstrate that the proposed system achieves a surface reconstruction error of 0.6\textit{cm} on standardized benchmark datasets including Replica and ScanNet++, representing over 20% improvement in accuracy compared to state-of-the-art (SOTA) GS-based methods. Notably, the system maintains real-time processing capabilities at 24 FPS, establishing it as one of the most accurate differentiable-rendering-based real-time reconstruction systems. Project Page: https://zju3dv.github.io/eggfusion/

Method: Created FOD-S2R dataset with 3,114 real HD images captured in a controlled fuel tank replica and 3,137 synthetic images generated using Unreal Engine. Dataset includes varied FOVs, object distances, lighting conditions, colors, and object sizes. Benchmarked state-of-the-art object detection models.

Result: Synthetic data improves detection accuracy and generalization to real-world conditions. Experiments demonstrate synthetic data enhances model performance and narrows the Sim2Real gap.

Conclusion: FOD-S2R provides the first systematic evaluation of synthetic data for FOD detection in confined fuel tanks, offering a valuable foundation for developing automated aviation maintenance systems.

Abstract: Foreign Object Debris (FOD) within aircraft fuel tanks presents critical safety hazards including fuel contamination, system malfunctions, and increased maintenance costs. Despite the severity of these risks, there is a notable lack of dedicated datasets for the complex, enclosed environments found inside fuel tanks. To bridge this gap, we present a novel dataset, FOD-S2R, composed of real and synthetic images of the FOD within a simulated aircraft fuel tank. Unlike existing datasets that focus on external or open-air environments, our dataset is the first to systematically evaluate the effectiveness of synthetic data in enhancing the real-world FOD detection performance in confined, closed structures. The real-world subset consists of 3,114 high-resolution HD images captured in a controlled fuel tank replica, while the synthetic subset includes 3,137 images generated using Unreal Engine. The dataset is composed of various Field of views (FOV), object distances, lighting conditions, color, and object size. Prior research has demonstrated that synthetic data can reduce reliance on extensive real-world annotations and improve the generalizability of vision models. Thus, we benchmark several state-of-the-art object detection models and demonstrate that introducing synthetic data improves the detection accuracy and generalization to real-world conditions. These experiments demonstrate the effectiveness of synthetic data in enhancing the model performance and narrowing the Sim2Real gap, providing a valuable foundation for developing automated FOD detection systems for aviation maintenance.

Method: TBT-Former introduces three key components: 1) scaled Transformer backbone with more attention heads and expanded MLP dimensions, 2) cross-scale FPN with top-down pathway and lateral connections, 3) novel boundary distribution regression head inspired by Generalized Focal Loss that treats boundary regression as probability distribution learning.

Result: Achieves state-of-the-art performance on THUMOS14 and EPIC-Kitchens 100 datasets, while remaining competitive on ActivityNet-1.3.

Conclusion: TBT-Former advances Transformer-based architectures for temporal action localization by better addressing boundary uncertainty and multi-scale feature fusion, establishing new benchmarks on competitive datasets.

Abstract: Temporal Action Localization (TAL) remains a fundamental challenge in video understanding, aiming to identify the start time, end time, and category of all action instances within untrimmed videos. While recent single-stage, anchor-free models like ActionFormer have set a high standard by leveraging Transformers for temporal reasoning, they often struggle with two persistent issues: the precise localization of actions with ambiguous or “fuzzy” temporal boundaries and the effective fusion of multi-scale contextual information. In this paper, we introduce the Temporal Boundary Transformer (TBT-Former), a new architecture that directly addresses these limitations. TBT-Former enhances the strong ActionFormer baseline with three core contributions: (1) a higher-capacity scaled Transformer backbone with an increased number of attention heads and an expanded Multi-Layer Perceptron (MLP) dimension for more powerful temporal feature extraction; (2) a cross-scale feature pyramid network (FPN) that integrates a top-down pathway with lateral connections, enabling richer fusion of high-level semantics and low-level temporal details; and (3) a novel boundary distribution regression head. Inspired by the principles of Generalized Focal Loss (GFL), this new head recasts the challenging task of boundary regression as a more flexible probability distribution learning problem, allowing the model to explicitly represent and reason about boundary uncertainty. Within the paradigm of Transformer-based architectures, TBT-Former advances the formidable benchmark set by its predecessors, establishing a new level of performance on the highly competitive THUMOS14 and EPIC-Kitchens 100 datasets, while remaining competitive on the large-scale ActivityNet-1.3. Our code is available at https://github.com/aaivu/In21-S7-CS4681-AML-Research-Projects/tree/main/projects/210536K-Multi-Modal-Learning_Video-Understanding

Method: Uses divide-and-conquer strategy: 1) decomposes prompts by extracting and dividing target text, 2) assigns each segment to designated regions, 3) applies Text-Focus and Context-Expansion attention masks sequentially during denoising, 4) uses Localized Noise Initialization for better accuracy and alignment.

Result: Achieves best text accuracy on single- and multi-sentence benchmarks without compromising image quality, while also delivering the lowest generation latency.

Conclusion: DCText effectively solves the long/multiple text rendering problem in text-to-image generation through a training-free, computationally efficient approach that maintains both text accuracy and image quality.

Abstract: Despite recent text-to-image models achieving highfidelity text rendering, they still struggle with long or multiple texts due to diluted global attention. We propose DCText, a training-free visual text generation method that adopts a divide-and-conquer strategy, leveraging the reliable short-text generation of Multi-Modal Diffusion Transformers. Our method first decomposes a prompt by extracting and dividing the target text, then assigns each to a designated region. To accurately render each segment within their regions while preserving overall image coherence, we introduce two attention masks - Text-Focus and Context-Expansion - applied sequentially during denoising. Additionally, Localized Noise Initialization further improves text accuracy and region alignment without increasing computational cost. Extensive experiments on single- and multisentence benchmarks show that DCText achieves the best text accuracy without compromising image quality while also delivering the lowest generation latency.

Method: Created IAVS dataset with image-mask pairs and hemodynamic outcomes, established two-stage evaluation benchmarks (aneurysm localization + fine-grained segmentation), developed two-stage framework as baseline, and created standardized CFD applicability evaluation system for automated conversion to CFD models.

Result: First comprehensive multi-center dataset addressing limitations of existing datasets, providing tools for clinically relevant technique development with focus on topological integrity and CFD applicability.

Conclusion: IAVS dataset bridges gap between image segmentation and practical CFD applications, enabling development of segmentation methods that are clinically relevant for hemodynamic analysis of intracranial aneurysms.

Abstract: The precise segmentation of intracranial aneurysms and their parent vessels (IA-Vessel) is a critical step for hemodynamic analyses, which mainly depends on computational fluid dynamics (CFD). However, current segmentation methods predominantly focus on image-based evaluation metrics, often neglecting their practical effectiveness in subsequent CFD applications. To address this deficiency, we present the Intracranial Aneurysm Vessel Segmentation (IAVS) dataset, the first comprehensive, multi-center collection comprising 641 3D MRA images with 587 annotations of aneurysms and IA-Vessels. In addition to image-mask pairs, IAVS dataset includes detailed hemodynamic analysis outcomes, addressing the limitations of existing datasets that neglect topological integrity and CFD applicability. To facilitate the development and evaluation of clinically relevant techniques, we construct two evaluation benchmarks including global localization of aneurysms (Stage I) and fine-grained segmentation of IA-Vessel (Stage II) and develop a simple and effective two-stage framework, which can be used as a out-of-the-box method and strong baseline. For comprehensive evaluation of applicability of segmentation results, we establish a standardized CFD applicability evaluation system that enables the automated and consistent conversion of segmentation masks into CFD models, offering an applicability-focused assessment of segmentation outcomes. The dataset, code, and model will be public available at https://github.com/AbsoluteResonance/IAVS.

Method: Proposes Gaussian Swaying, a surface-based framework using 3D Gaussians to model surfaces continuously. Uses Gaussian patches as a unified representation that supports both force computation for dynamics and provides normals for lightweight shading, eliminating the need for separate simulation and rendering representations.

Result: Comprehensive experiments on synthetic and real-world datasets across multiple metrics demonstrate state-of-the-art performance and efficiency. The framework offers scalable realistic aerodynamic scene simulation with fine-grained interaction.

Conclusion: Gaussian Swaying provides an efficient, unified approach for aerodynamic simulation that bridges the gap between simulation and rendering, enabling realistic wind-driven motion in computer vision and graphics applications with superior performance and scalability.

Abstract: Branches swaying in the breeze, flags rippling in the wind, and boats rocking on the water all show how aerodynamics shape natural motion – an effect crucial for realism in vision and graphics. In this paper, we present Gaussian Swaying, a surface-based framework for aerodynamic simulation using 3D Gaussians. Unlike mesh-based methods that require costly meshing, or particle-based approaches that rely on discrete positional data, Gaussian Swaying models surfaces continuously with 3D Gaussians, enabling efficient and fine-grained aerodynamic interaction. Our framework unifies simulation and rendering on the same representation: Gaussian patches, which support force computation for dynamics while simultaneously providing normals for lightweight shading. Comprehensive experiments on both synthetic and real-world datasets across multiple metrics demonstrate that Gaussian Swaying achieves state-of-the-art performance and efficiency, offering a scalable approach for realistic aerodynamic scene simulation.

Method: The study systematically explores the role of data quality level in pretraining by performing pretraining on datasets with different quality levels, followed by fine-tuning for brain age prediction on external cohorts.

Result: Results show significant performance differences across quality levels, revealing both opportunities and limitations in using heterogeneous quality data for pretraining.

Conclusion: There is a gap between computer vision practices and clinical neuroimaging standards, emphasizing the necessity of domain-aware curation to ensure trusted and generalizable domain-specific foundation models.

Abstract: Large-scale brain imaging datasets provide unprecedented opportunities for developing domain foundation models through pretraining. However, unlike natural image datasets in computer vision, these neuroimaging data often exhibit high heterogeneity in quality, ranging from well-structured scans to severely distorted or incomplete brain volumes. This raises a fundamental question: can noise or low-quality scans contribute meaningfully to pretraining, or do they instead hinder model learning? In this study, we systematically explore the role of data quality level in pretraining and its impact on downstream tasks. Specifically, we perform pretraining on datasets with different quality levels and perform fine-tuning for brain age prediction on external cohorts. Our results show significant performance differences across quality levels, revealing both opportunities and limitations. We further discuss the gap between computer vision practices and clinical neuroimaging standards, emphasizing the necessity of domain-aware curation to ensure trusted and generalizable domain-specific foundation models.

Method: Introduces IVCR-200K dataset (bilingual, multi-turn, conversational) and proposes MLLM-based framework supporting multiple interaction modes with explainable solutions.

Result: Extensive experiments demonstrate effectiveness of both the dataset and the proposed framework for interactive video retrieval.

Conclusion: IVCR task represents a more realistic video search paradigm that enables conversational interactions, with the dataset and framework providing strong foundations for future research.

Abstract: In recent years, significant developments have been made in both video retrieval and video moment retrieval tasks, which respectively retrieve complete videos or moments for a given text query. These advancements have greatly improved user satisfaction during the search process. However, previous work has failed to establish meaningful “interaction” between the retrieval system and the user, and its one-way retrieval paradigm can no longer fully meet the personalization and dynamic needs of at least 80.8% of users. In this paper, we introduce the Interactive Video Corpus Retrieval (IVCR) task, a more realistic setting that enables multi-turn, conversational, and realistic interactions between the user and the retrieval system. To facilitate research on this challenging task, we introduce IVCR-200K, a high-quality, bilingual, multi-turn, conversational, and abstract semantic dataset that supports video retrieval and even moment retrieval. Furthermore, we propose a comprehensive framework based on multi-modal large language models (MLLMs) to help users interact in several modes with more explainable solutions. The extensive experiments demonstrate the effectiveness of our dataset and framework.

Method: Created RICE-VL benchmark with 28,000+ human-curated VQA samples across 3 formats and 1,000 image-bounding box pairs for Visual Grounding, annotated by culturally informed experts. Proposed SEA-LAVE metric extension for assessing textual accuracy, cultural alignment, and country identification.

Result: Evaluation of 6 open- and closed-source VLMs revealed significant performance gaps in low-resource countries and abstract cultural domains, exposing limitations in VLMs’ cultural comprehension.

Conclusion: RICE-VL highlights the need for inclusive model development to better serve diverse global populations and addresses cultural biases in VLMs.

Abstract: Vision-Language Models (VLMs) excel in multimodal tasks but often exhibit Western-centric biases, limiting their effectiveness in culturally diverse regions like Southeast Asia (SEA). To address this, we introduce RICE-VL, a novel benchmark evaluating VLM cultural understanding across 11 ASEAN countries. RICE-VL includes over 28,000 human-curated Visual Question Answering (VQA) samples – covering True or False, Fill-in-the-Blank, and open-ended formats – and 1,000 image-bounding box pairs for Visual Grounding, annotated by culturally informed experts across 14 sub-ground categories. We propose SEA-LAVE, an extension of the LAVE metric, assessing textual accuracy, cultural alignment, and country identification. Evaluations of six open- and closed-source VLMs reveal significant performance gaps in low-resource countries and abstract cultural domains. The Visual Grounding task tests models’ ability to localize culturally significant elements in complex scenes, probing spatial and contextual accuracy. RICE-VL exposes limitations in VLMs’ cultural comprehension and highlights the need for inclusive model development to better serve diverse global populations.

Method: TokenPure decomposes watermarked images into visual tokens (texture) and structural tokens (geometry), then uses these tokens to condition a diffusion process for synthesizing watermark-free images while bypassing initial watermark-carrying noise.

Result: TokenPure achieves state-of-the-art watermark removal and reconstruction fidelity, substantially outperforming existing baselines in both perceptual quality and consistency.

Conclusion: The framework successfully solves the trade-off between thorough watermark destruction and content consistency through token-based conditional reconstruction, offering an effective solution for watermark removal in digital content.

Abstract: In the digital economy era, digital watermarking serves as a critical basis for ownership proof of massive replicable content, including AI-generated and other virtual assets. Designing robust watermarks capable of withstanding various attacks and processing operations is even more paramount. We introduce TokenPure, a novel Diffusion Transformer-based framework designed for effective and consistent watermark removal. TokenPure solves the trade-off between thorough watermark destruction and content consistency by leveraging token-based conditional reconstruction. It reframes the task as conditional generation, entirely bypassing the initial watermark-carrying noise. We achieve this by decomposing the watermarked image into two complementary token sets: visual tokens for texture and structural tokens for geometry. These tokens jointly condition the diffusion process, enabling the framework to synthesize watermark-free images with fine-grained consistency and structural integrity. Comprehensive experiments show that TokenPure achieves state-of-the-art watermark removal and reconstruction fidelity, substantially outperforming existing baselines in both perceptual quality and consistency.

Method: Developed an interpretable ML framework using ensemble modeling and XAI techniques. Included comprehensive data preprocessing with feature engineering, addressed class imbalance using Random Over-Sampling (ROS), evaluated 10 ML models with 5-fold cross-validation, and created an optimized ensemble model combining Random Forest, ExtraTrees, and XGBoost. Used LIME-based interpretability analysis to identify key clinical variables.

Result: The optimized ensemble model achieved exceptional 99.09% accuracy on the Stroke Prediction Dataset (SPD). LIME-based analysis identified three critical clinical variables: age, hypertension, and glucose levels. The framework demonstrated high accuracy while maintaining interpretability for clinical use.

Conclusion: The study demonstrates that combining ensemble learning with explainable AI can provide highly accurate and interpretable stroke risk assessment. The framework enables early prediction, data-driven prevention, and personalized clinical decisions, potentially transforming stroke prediction and cardiovascular risk management.

Abstract: Stroke is a major cause of death and permanent impairment, making it a major worldwide health concern. For prompt intervention and successful preventative tactics, early risk assessment is essential. To address this challenge, we used ensemble modeling and explainable AI (XAI) techniques to create an interpretable machine learning framework for stroke risk prediction. A thorough evaluation of 10 different machine learning models using 5-fold cross-validation across several datasets was part of our all-inclusive strategy, which also included feature engineering and data pretreatment (using Random Over-Sampling (ROS) to solve class imbalance). Our optimized ensemble model (Random Forest + ExtraTrees + XGBoost) performed exceptionally well, obtaining a strong 99.09% accuracy on the Stroke Prediction Dataset (SPD). We improved the model’s transparency and clinical applicability by identifying three important clinical variables using LIME-based interpretability analysis: age, hypertension, and glucose levels. Through early prediction, this study highlights how combining ensemble learning with explainable AI (XAI) can deliver highly accurate and interpretable stroke risk assessment. By enabling data-driven prevention and personalized clinical decisions, our framework has the potential to transform stroke prediction and cardiovascular risk management.

Method: AlignVid has two components: 1) Attention Scaling Modulation (ASM) that directly reweights attention via lightweight Q or K scaling, and 2) Guidance Scheduling (GS) that applies ASM selectively across transformer blocks and denoising steps to reduce visual quality degradation. The framework is training-free and uses minimal intervention.

Result: The method improves semantic fidelity in TI2V generation. The authors also introduce OmitI2V, a new evaluation benchmark with 367 human-annotated samples spanning addition, deletion, and modification scenarios to measure semantic negligence.

Conclusion: AlignVid effectively addresses semantic negligence in TI2V generation through attention-based modulation, improving prompt adherence while limiting aesthetic degradation, and provides a new benchmark for evaluating these capabilities.

Abstract: Text-guided image-to-video (TI2V) generation has recently achieved remarkable progress, particularly in maintaining subject consistency and temporal coherence. However, existing methods still struggle to adhere to fine-grained prompt semantics, especially when prompts entail substantial transformations of the input image (e.g., object addition, deletion, or modification), a shortcoming we term semantic negligence. In a pilot study, we find that applying a Gaussian blur to the input image improves semantic adherence. Analyzing attention maps, we observe clearer foreground-background separation. From an energy perspective, this corresponds to a lower-entropy cross-attention distribution. Motivated by this, we introduce AlignVid, a training-free framework with two components: (i) Attention Scaling Modulation (ASM), which directly reweights attention via lightweight Q or K scaling, and (ii) Guidance Scheduling (GS), which applies ASM selectively across transformer blocks and denoising steps to reduce visual quality degradation. This minimal intervention improves prompt adherence while limiting aesthetic degradation. In addition, we introduce OmitI2V to evaluate semantic negligence in TI2V generation, comprising 367 human-annotated samples that span addition, deletion, and modification scenarios. Extensive experiments demonstrate that AlignVid can enhance semantic fidelity.

Method: DFS uses multiple diffusion paths each dedicated to specific image features, rule-chain-based reasoning for dynamic coordination, and fuzzy membership-based latent-space compression to reduce computational costs.

Result: DFS achieves more stable training and faster convergence than existing single-path and multi-path diffusion models on LSUN Bedroom, LSUN Church, and MS COCO datasets. It surpasses baselines in image quality, text-image alignment, and accuracy compared to target references.

Conclusion: DFS effectively addresses the limitations of multi-path diffusion models by providing better feature specialization, efficient coordination through fuzzy rules, and reduced computational overhead, resulting in superior image generation performance.

Abstract: Diffusion models have emerged as a leading technique for generating images due to their ability to create high-resolution and realistic images. Despite their strong performance, diffusion models still struggle in managing image collections with significant feature differences. They often fail to capture complex features and produce conflicting results. Research has attempted to address this issue by learning different regions of an image through multiple diffusion paths and then combining them. However, this approach leads to inefficient coordination among multiple paths and high computational costs. To tackle these issues, this paper presents a Diffusion Fuzzy System (DFS), a latent-space multi-path diffusion model guided by fuzzy rules. DFS offers several advantages. First, unlike traditional multi-path diffusion methods, DFS uses multiple diffusion paths, each dedicated to learning a specific class of image features. By assigning each path to a different feature type, DFS overcomes the limitations of multi-path models in capturing heterogeneous image features. Second, DFS employs rule-chain-based reasoning to dynamically steer the diffusion process and enable efficient coordination among multiple paths. Finally, DFS introduces a fuzzy membership-based latent-space compression mechanism to reduce the computational costs of multi-path diffusion effectively. We tested our method on three public datasets: LSUN Bedroom, LSUN Church, and MS COCO. The results show that DFS achieves more stable training and faster convergence than existing single-path and multi-path diffusion models. Additionally, DFS surpasses baseline models in both image quality and alignment between text and images, and also shows improved accuracy when comparing generated images to target references.

Method: 1) Construct THQA-MT dataset with 5,492 Multi-Talker-generated THs from 15 representative Multi-Talkers using 400 real portraits. 2) Conduct subjective experiments to analyze perceptual discrepancies and identify 12 distortion types. 3) Develop EvalTalker framework that perceives global quality, human characteristics, identity consistency, and integrates Qwen-Sync for multimodal synchrony perception.

Result: EvalTalker achieves superior correlation with subjective scores, demonstrating its effectiveness as a quality assessment framework for Multi-Talker-generated content.

Conclusion: The proposed THQA-MT dataset and EvalTalker framework provide a robust foundation for future research on high-quality Multi-Talker generation and evaluation, addressing current quality assessment limitations in multi-subject talking human animation.

Abstract: Speech-driven Talking Human (TH) generation, commonly known as “Talker,” currently faces limitations in multi-subject driving capabilities. Extending this paradigm to “Multi-Talker,” capable of animating multiple subjects simultaneously, introduces richer interactivity and stronger immersion in audiovisual communication. However, current Multi-Talkers still exhibit noticeable quality degradation caused by technical limitations, resulting in suboptimal user experiences. To address this challenge, we construct THQA-MT, the first large-scale Multi-Talker-generated Talking Human Quality Assessment dataset, consisting of 5,492 Multi-Talker-generated THs (MTHs) from 15 representative Multi-Talkers using 400 real portraits collected online. Through subjective experiments, we analyze perceptual discrepancies among different Multi-Talkers and identify 12 common types of distortion. Furthermore, we introduce EvalTalker, a novel TH quality assessment framework. This framework possesses the ability to perceive global quality, human characteristics, and identity consistency, while integrating Qwen-Sync to perceive multimodal synchrony. Experimental results demonstrate that EvalTalker achieves superior correlation with subjective scores, providing a robust foundation for future research on high-quality Multi-Talker generation and evaluation.

Method: Large-scale multi-center benchmark using 2,976 T1 and 2,972 T2-weighted healthy scans for training, 92 for validation, and 2,221 T1w/1,262 T2w for testing across diverse clinical cohorts. Systematic evaluation of different algorithms’ robustness to scanners, lesion types/sizes, and demographics.

Result: Dice-based segmentation performance varied widely (0.03-0.65). Reconstruction-based methods (especially diffusion-inspired) achieved strongest lesion segmentation, while feature-based methods showed greater robustness to distribution shifts. Systematic biases observed: small/low-contrast lesions missed more often, false positives varied with age/sex. More training data yields only modest gains.

Conclusion: Current unsupervised anomaly detection frameworks are limited algorithmically rather than by data availability. Benchmark establishes foundation for future research and highlights priorities: image native pretraining, principled deviation measures, fairness-aware modeling, and robust domain adaptation for clinical translation.

Abstract: Deep unsupervised anomaly detection in brain magnetic resonance imaging offers a promising route to identify pathological deviations without requiring lesion-specific annotations. Yet, fragmented evaluations, heterogeneous datasets, and inconsistent metrics have hindered progress toward clinical translation. Here, we present a large-scale, multi-center benchmark of deep unsupervised anomaly detection for brain imaging. The training cohort comprised 2,976 T1 and 2,972 T2-weighted scans from healthy individuals across six scanners, with ages ranging from 6 to 89 years. Validation used 92 scans to tune hyperparameters and estimate unbiased thresholds. Testing encompassed 2,221 T1w and 1,262 T2w scans spanning healthy datasets and diverse clinical cohorts. Across all algorithms, the Dice-based segmentation performance varied between 0.03 and 0.65, indicating substantial variability. To assess robustness, we systematically evaluated the impact of different scanners, lesion types and sizes, as well as demographics (age, sex). Reconstruction-based methods, particularly diffusion-inspired approaches, achieved the strongest lesion segmentation performance, while feature-based methods showed greater robustness under distributional shifts. However, systematic biases, such as scanner-related effects, were observed for the majority of algorithms, including that small and low-contrast lesions were missed more often, and that false positives varied with age and sex. Increasing healthy training data yields only modest gains, underscoring that current unsupervised anomaly detection frameworks are limited algorithmically rather than by data availability. Our benchmark establishes a transparent foundation for future research and highlights priorities for clinical translation, including image native pretraining, principled deviation measures, fairness-aware modeling, and robust domain adaptation.

Method: Proposes Encoder-Predictor-Decoder framework where predictor acts as latent world model. Two-stage pretraining: Stage 1 uses conditional diffusion decoder with image-level semantic priors to bridge pixel fidelity with semantic abstraction; Stage 2 predicts frozen Stage 1 targets to learn world knowledge while mitigating shortcuts.

Result: Achieves state-of-the-art results across benchmarks when trained on public, unlabeled videos, providing scalable path toward general video representation learning.

Conclusion: InternVideo-Next successfully addresses architectural limitations of traditional masked video modeling by disentangling encoder-decoder design and introducing semantic-aware pretraining, enabling effective learning of implicit world knowledge from unlabeled videos.

Abstract: Large-scale video-text pretraining achieves strong performance but depends on noisy, synthetic captions with limited semantic coverage, often overlooking implicit world knowledge such as object motion, 3D geometry, and physical cues. In contrast, masked video modeling (MVM) directly exploits spatiotemporal structures but trails text-supervised methods on general tasks. We find this gap arises from overlooked architectural issues: pixel-level reconstruction struggles with convergence and its low-level requirement often conflicts with semantics, while latent prediction often encourages shortcut learning. To address these, we disentangle the traditional encoder-decoder design into an Encoder-Predictor-Decoder (EPD) framework, where the predictor acts as a latent world model, and propose InternVideo-Next, a two-stage pretraining scheme that builds a semantically consistent yet detail-preserving latent space for this world model. First, conventional linear decoder in pixel MVM enforces the predictor output latent to be linearly projected to, thus separable in pixel space, causing the conflict with semantic abstraction. Our Stage 1 proposes a conditional diffusion decoder and injects reliable image-level semantic priors to enhance semantics and convergence, thus bridging pixel-level fidelity with high-level semantic abstraction. Stage 2 further learns world knowledge by predicting frozen Stage 1 targets within this space, mitigating shortcut learning. Trained on public, unlabeled videos, InternVideo-Next achieves state-of-the-art results across benchmarks and provides a scalable path toward general video representation learning.

Method: Proposes a ViT-Transformer Decoder-LM architecture: Vision Transformer (ViT) extracts visual features, Transformer decoder generates text sequences, and pre-trained language model (LM) refines output. Uses DeiT for masked image modeling and RoBERTa for masked language modeling. Implements implicit line segmentation to process paragraphs line by line without explicit segmentation.

Result: Achieved benchmark results on multiple datasets: 96.24% character recognition rate on NUST-UHWR, 92.05% on PUCIT-OUHL, 94.80% on Parimal-Urdu, and 80.64% on Hindi dataset. Outperformed several state-of-the-art Urdu text recognition methods.

Conclusion: BharatOCR demonstrates effective paragraph-level handwritten text recognition for low-resource languages through a segmentation-free approach combining vision transformers, transformer decoders, and language models, achieving state-of-the-art performance on Hindi and Urdu datasets.

Abstract: Despite considerable progress in handwritten text recognition, paragraph-level handwritten text recognition, especially in low-resource languages, such as Hindi, Urdu and similar scripts, remains a challenging problem. These languages, often lacking comprehensive linguistic resources, require special attention to develop robust systems for accurate optical character recognition (OCR). This paper introduces BharatOCR, a novel segmentation-free paragraph-level handwritten Hindi and Urdu text recognition. We propose a ViT-Transformer Decoder-LM architecture for handwritten text recognition, where a Vision Transformer (ViT) extracts visual features, a Transformer decoder generates text sequences, and a pre-trained language model (LM) refines the output to improve accuracy, fluency, and coherence. Our model utilizes a Data-efficient Image Transformer (DeiT) model proposed for masked image modeling in this research work. In addition, we adopt a RoBERTa architecture optimized for masked language modeling (MLM) to enhance the linguistic comprehension and generative capabilities of the proposed model. The transformer decoder generates text sequences from visual embeddings. This model is designed to iteratively process a paragraph image line by line, called implicit line segmentation. The proposed model was evaluated using our custom dataset (‘Parimal Urdu’) and (‘Parimal Hindi’), introduced in this research work, as well as two public datasets. The proposed model achieved benchmark results in the NUST-UHWR, PUCIT-OUHL, and Parimal-Urdu datasets, achieving character recognition rates of 96.24%, 92.05%, and 94.80%, respectively. The model also provided benchmark results using the Hindi dataset achieving a character recognition rate of 80.64%. The results obtained from our proposed model indicated that it outperformed several state-of-the-art Urdu text recognition methods.

Method: 1) Created MasHeNe dataset: 3,779 contrast-enhanced CT slices with pixel-level annotations of tumors and cysts. 2) Proposed WEMF model: applies tri-window enhancement to input, uses multi-frequency attention to fuse skip connections within U-shaped Mamba backbone.

Result: WEMF achieves best performance on MasHeNe: Dice 70.45%, IoU 66.89%, NSD 72.33%, HD95 5.12 mm. Dataset provides benchmark for head-and-neck mass segmentation beyond malignancy-only datasets.

Conclusion: MasHeNe addresses gap in head & neck mass segmentation datasets. WEMF shows strong performance but error patterns indicate task remains challenging, requiring further research. Dataset and code publicly available.

Abstract: Head and neck masses are space-occupying lesions that can compress the airway and esophagus and may affect nerves and blood vessels. Available public datasets primarily focus on malignant lesions and often overlook other space-occupying conditions in this region. To address this gap, we introduce MasHeNe, an initial dataset of 3,779 contrast-enhanced CT slices that includes both tumors and cysts with pixel-level annotations. We also establish a benchmark using standard segmentation baselines and report common metrics to enable fair comparison. In addition, we propose the Windowing-Enhanced Mamba with Frequency integration (WEMF) model. WEMF applies tri-window enhancement to enrich the input appearance before feature extraction. It further uses multi-frequency attention to fuse information across skip connections within a U-shaped Mamba backbone. On MasHeNe, WEMF attains the best performance among evaluated methods, with a Dice of 70.45%, IoU of 66.89%, NSD of 72.33%, and HD95 of 5.12 mm. This model indicates stable and strong results on this challenging task. MasHeNe provides a benchmark for head-and-neck mass segmentation beyond malignancy-only datasets. The observed error patterns also suggest that this task remains challenging and requires further research. Our dataset and code are available at https://github.com/drthaodao3101/MasHeNe.git.

Method: Two-stage pipeline: 1) Cross-modal instance alignment - associate 2D image instance cues from vision foundation model with 3D point clouds; 2) Instance categorization by rigidity/motion state, then generate adaptive bounding boxes using class-specific size statistics. No self-training required.

Result: Experiments on Waymo Open Dataset, Lyft Level 5 Perception dataset, and nuScenes dataset show improved accuracy and efficiency over baselines. Produces high-quality 3D bounding box annotations without self-training.

Conclusion: OpenBox addresses key challenges in unsupervised 3D object detection by leveraging 2D vision foundation models for automatic annotation, eliminating computational overhead of self-training while improving annotation quality through adaptive, state-aware bounding boxes.

Abstract: Unsupervised and open-vocabulary 3D object detection has recently gained attention, particularly in autonomous driving, where reducing annotation costs and recognizing unseen objects are critical for both safety and scalability. However, most existing approaches uniformly annotate 3D bounding boxes, ignore objects’ physical states, and require multiple self-training iterations for annotation refinement, resulting in suboptimal quality and substantial computational overhead. To address these challenges, we propose OpenBox, a two-stage automatic annotation pipeline that leverages a 2D vision foundation model. In the first stage, OpenBox associates instance-level cues from 2D images processed by a vision foundation model with the corresponding 3D point clouds via cross-modal instance alignment. In the second stage, it categorizes instances by rigidity and motion state, then generates adaptive bounding boxes with class-specific size statistics. As a result, OpenBox produces high-quality 3D bounding box annotations without requiring self-training. Experiments on the Waymo Open Dataset, the Lyft Level 5 Perception dataset, and the nuScenes dataset demonstrate improved accuracy and efficiency over baselines.

Method: Uses a single earbud and paired phone to detect approaching hazards. Combines Kalman filter-based trajectory estimation with reinforcement learning for optimal image sampling. Corrects for user head movements using estimated pitch and yaw angles to align coordinate systems and improve depth estimation accuracy.

Result: Lightweight system with ultra-low power consumption (29.8 mW on earbud, 702.6 mW on smartphone). Achieves high detection accuracy with low average false positive ratio (4.90%) and false negative ratio (1.47%).

Conclusion: BlinkBud provides an effective, low-power solution for detecting approaching hazards from behind, significantly improving road safety for vulnerable road users without requiring constant visual attention.

Abstract: Failing to be aware of speeding vehicles approaching from behind poses a huge threat to the road safety of pedestrians and cyclists. In this paper, we propose BlinkBud, which utilizes a single earbud and a paired phone to online detect hazardous objects approaching from behind of a user. The core idea is to accurately track visually identified objects utilizing a small number of sampled camera images taken from the earbud. To minimize the power consumption of the earbud and the phone while guaranteeing the best tracking accuracy, a novel 3D object tracking algorithm is devised, integrating both a Kalman filter based trajectory estimation scheme and an optimal image sampling strategy based on reinforcement learning. Moreover, the impact of constant user head movements on the tracking accuracy is significantly eliminated by leveraging the estimated pitch and yaw angles to correct the object depth estimation and align the camera coordinate system to the user’s body coordinate system, respectively. We implement a prototype BlinkBud system and conduct extensive real-world experiments. Results show that BlinkBud is lightweight with ultra-low mean power consumptions of 29.8 mW and 702.6 mW on the earbud and smartphone, respectively, and can accurately detect hazards with a low average false positive ratio (FPR) and false negative ratio (FNR) of 4.90% and 1.47%, respectively.

Method: 1) Encode 3D meshes into language token space, 2) Use LLM as bridge between shape info and realism evaluation, 3) Design dedicated realism decoder to align LLM output with human perception, 4) Create RealismGrading dataset with human-annotated scores from 16 algorithms across dozen objects.

Result: Metric correlates well with human perceptions, outperforms existing methods, shows good generalizability validated through k-fold cross-validation across different objects.

Conclusion: Proposed SRAM effectively evaluates 3D shape realism without ground truth, leveraging LLMs and human-annotated dataset for better alignment with human perception than traditional methods.

Abstract: 3D generation and reconstruction techniques have been widely used in computer games, film, and other content creation areas. As the application grows, there is a growing demand for 3D shapes that look truly realistic. Traditional evaluation methods rely on a ground truth to measure mesh fidelity. However, in many practical cases, a shape’s realism does not depend on having a ground truth reference. In this work, we propose a Shape-Realism Alignment Metric that leverages a large language model (LLM) as a bridge between mesh shape information and realism evaluation. To achieve this, we adopt a mesh encoding approach that converts 3D shapes into the language token space. A dedicated realism decoder is designed to align the language model’s output with human perception of realism. Additionally, we introduce a new dataset, RealismGrading, which provides human-annotated realism scores without the need for ground truth shapes. Our dataset includes shapes generated by 16 different algorithms on over a dozen objects, making it more representative of practical 3D shape distributions. We validate our metric’s performance and generalizability through k-fold cross-validation across different objects. Experimental results show that our metric correlates well with human perceptions and outperforms existing methods, and has good generalizability.

Method: The authors meticulously designed and collected RoleMotion with focus on scenes and roles, featuring 25 classic scenes, 110 functional roles, over 500 behaviors, and 10,296 high-quality human motion sequences of body and hands. They annotated these with 27,831 fine-grained text descriptions and built a stronger evaluator than existing counterparts.

Result: The dataset contains comprehensive motion data with fine-grained annotations. The authors proved the reliability of their evaluator and evaluated various text-to-motion methods on their dataset. They also explored the interplay of body and hand motion generation.

Conclusion: RoleMotion demonstrates high-quality and functionality for text-driven whole-body generation, providing a valuable resource for motion generation research with its scene-specific, role-playing focused approach and fine-grained annotations.

Abstract: In this paper, we introduce RoleMotion, a large-scale human motion dataset that encompasses a wealth of role-playing and functional motion data tailored to fit various specific scenes. Existing text datasets are mainly constructed decentrally as amalgamation of assorted subsets that their data are nonfunctional and isolated to work together to cover social activities in various scenes. Also, the quality of motion data is inconsistent, and textual annotation lacks fine-grained details in these datasets. In contrast, RoleMotion is meticulously designed and collected with a particular focus on scenes and roles. The dataset features 25 classic scenes, 110 functional roles, over 500 behaviors, and 10296 high-quality human motion sequences of body and hands, annotated with 27831 fine-grained text descriptions. We build an evaluator stronger than existing counterparts, prove its reliability, and evaluate various text-to-motion methods on our dataset. Finally, we explore the interplay of motion generation of body and hands. Experimental results demonstrate the high-quality and functionality of our dataset on text-driven whole-body generation.

Method: Textured Geometry Evaluation (TGE) directly analyzes 3D meshes with texture without rendering, jointly using geometry and color information to calculate fidelity compared to a reference colored shape.

Result: TGE outperforms rendering-based and geometry-only methods on a human-annotated dataset with real-world distortions.

Conclusion: TGE provides a more human-aligned 3D fidelity evaluation method that works directly on textured meshes, addressing limitations of existing approaches and performing better on real-world distortions.

Abstract: Textured high-fidelity 3D models are crucial for games, AR/VR, and film, but human-aligned evaluation methods still fall behind despite recent advances in 3D reconstruction and generation. Existing metrics, such as Chamfer Distance, often fail to align with how humans evaluate the fidelity of 3D shapes. Recent learning-based metrics attempt to improve this by relying on rendered images and 2D image quality metrics. However, these approaches face limitations due to incomplete structural coverage and sensitivity to viewpoint choices. Moreover, most methods are trained on synthetic distortions, which differ significantly from real-world distortions, resulting in a domain gap. To address these challenges, we propose a new fidelity evaluation method that is based directly on 3D meshes with texture, without relying on rendering. Our method, named Textured Geometry Evaluation TGE, jointly uses the geometry and color information to calculate the fidelity of the input textured mesh with comparison to a reference colored shape. To train and evaluate our metric, we design a human-annotated dataset with real-world distortions. Experiments show that TGE outperforms rendering-based and geometry-only methods on real-world distortion dataset.

Method: Reversible Inversion (ReInversion) - a two-stage denoising process: first conditioned on source image, then on reference image. Plus Mask-Guided Selective Denoising (MSD) to constrain edits to target regions while preserving background structure.

Result: Achieves state-of-the-art exemplar-guided image editing performance with the lowest computational overhead, demonstrated through both qualitative and quantitative comparisons.

Conclusion: ReInversion provides an effective and efficient training-free solution for exemplar-guided image editing, overcoming limitations of both pre-training approaches and standard inversion techniques.

Abstract: Exemplar-guided Image Editing (EIE) aims to modify a source image according to a visual reference. Existing approaches often require large-scale pre-training to learn relationships between the source and reference images, incurring high computational costs. As a training-free alternative, inversion techniques can be used to map the source image into a latent space for manipulation. However, our empirical study reveals that standard inversion is sub-optimal for EIE, leading to poor quality and inefficiency. To tackle this challenge, we introduce \textbf{Reversible Inversion ({ReInversion})} for effective and efficient EIE. Specifically, ReInversion operates as a two-stage denoising process, which is first conditioned on the source image and subsequently on the reference. Besides, we introduce a Mask-Guided Selective Denoising (MSD) strategy to constrain edits to target regions, preserving the structural consistency of the background. Both qualitative and quantitative comparisons demonstrate that our ReInversion method achieves state-of-the-art EIE performance with the lowest computational overhead.

Method: Proposed Envision benchmark with 1,000 four-stage prompts across six scientific/humanities domains, structured by spatiotemporal causality. Introduced Envision-Score metric integrating multi-dimensional consistency, physicality, and aesthetics. Evaluated 15 models (10 specialized T2I, 5 unified multimodal).

Result: Specialized T2I models excel in aesthetics but lack world knowledge. Unified multimodal models outperform specialized ones in causal narrative coherence but still lag behind closed-source models and struggle with spatiotemporal consistency. Focus on single images impedes multi-frame reasoning.

Conclusion: Current emphasis on static single-image generation promotes pattern matching over dynamic world modeling, limiting world knowledge internalization. Future work should prioritize spatiotemporal consistency and causal reasoning in multi-frame generation.

Abstract: Current multimodal models aim to transcend the limitations of single-modality representations by unifying understanding and generation, often using text-to-image (T2I) tasks to calibrate semantic consistency. However, their reliance on static, single-image generation in training and evaluation leads to overfitting to static pattern matching and semantic fusion, while fundamentally hindering their ability to model dynamic processes that unfold over time. To address these constraints, we propose Envision-a causal event progression benchmark for chained text-to-multi-image generation. Grounded in world knowledge and structured by spatiotemporal causality, it reorganizes existing evaluation dimensions and includes 1,000 four-stage prompts spanning six scientific and humanities domains. To transition evaluation from single images to sequential frames and assess whether models truly internalize world knowledge while adhering to causal-temporal constraints, we introduce Envision-Score, a holistic metric integrating multi-dimensional consistency, physicality, and aesthetics. Comprehensive evaluation of 15 models (10 specialized T2I models, 5 unified models) uncovers: specialized T2I models demonstrate proficiency in aesthetic rendering yet lack intrinsic world knowledge. Unified multimodal models bridge this gap, consistently outperforming specialized counterparts in causal narrative coherence. However, even these unified architectures remain subordinate to closed-source models and struggle to overcome the core challenge of spatiotemporal consistency. This demonstrates that a focus on causally-isolated single images impedes multi-frame reasoning and generation, promoting static pattern matching over dynamic world modeling-ultimately limiting world knowledge internalization, generation.

Method: Proposes PointNet4D with Hybrid Mamba-Transformer temporal fusion block combining efficient state-space modeling (Mamba) with bidirectional modeling power of Transformers. Introduces 4DMAP frame-wise masked auto-regressive pretraining strategy to capture motion cues across frames.

Result: Extensive evaluations across 9 tasks on 7 datasets show consistent improvements across diverse domains. Built two robotic application systems (4D Diffusion Policy and 4D Imitation Learning) achieving substantial gains on RoboTwin and HandoverSim benchmarks.

Conclusion: PointNet4D provides an efficient 4D backbone optimized for both online and offline settings, enabling real-time processing of streaming point cloud video with strong performance across various tasks and practical applications in robotics.

Abstract: Understanding dynamic 4D environments-3D space evolving over time-is critical for robotic and interactive systems. These applications demand systems that can process streaming point cloud video in real-time, often under resource constraints, while also benefiting from past and present observations when available. However, current 4D backbone networks rely heavily on spatiotemporal convolutions and Transformers, which are often computationally intensive and poorly suited to real-time applications. We propose PointNet4D, a lightweight 4D backbone optimized for both online and offline settings. At its core is a Hybrid Mamba-Transformer temporal fusion block, which integrates the efficient state-space modeling of Mamba and the bidirectional modeling power of Transformers. This enables PointNet4D to handle variable-length online sequences efficiently across different deployment scenarios. To enhance temporal understanding, we introduce 4DMAP, a frame-wise masked auto-regressive pretraining strategy that captures motion cues across frames. Our extensive evaluations across 9 tasks on 7 datasets, demonstrating consistent improvements across diverse domains. We further demonstrate PointNet4D’s utility by building two robotic application systems: 4D Diffusion Policy and 4D Imitation Learning, achieving substantial gains on the RoboTwin and HandoverSim benchmarks.

Method: Uses final-layer feature maps as teachers for intermediate layers, decomposing features into LF/HF bands via FFT masks. Applies Intra Contrastive Loss for LF structure stabilization and Inter Contrastive Loss for HF detail sharpening with random-layer/in-batch negatives. Includes Frequency-based Adaptive Weight and Frequency-based Alignment Modulation for per-layer signal reweighting and similarity-based distillation gating.

Result: Consistently improves PSNR/SSIM and perceptual metrics (LPIPS, NIQE, MANIQA, MUSIQ) across U-Net and DiT backbones including Stable Diffusion 2 and 3. Ablations validate the final-layer teacher and random-layer negatives.

Conclusion: FRAMER effectively addresses diffusion models’ frequency biases in Real-ISR through frequency-aware feature distillation and contrastive learning, providing plug-and-play improvements without modifying backbone architectures or inference procedures.

Abstract: Real-image super-resolution (Real-ISR) seeks to recover HR images from LR inputs with mixed, unknown degradations. While diffusion models surpass GANs in perceptual quality, they under-reconstruct high-frequency (HF) details due to a low-frequency (LF) bias and a depth-wise “low-first, high-later” hierarchy. We introduce FRAMER, a plug-and-play training scheme that exploits diffusion priors without changing the backbone or inference. At each denoising step, the final-layer feature map teaches all intermediate layers. Teacher and student feature maps are decomposed into LF/HF bands via FFT masks to align supervision with the model’s internal frequency hierarchy. For LF, an Intra Contrastive Loss (IntraCL) stabilizes globally shared structure. For HF, an Inter Contrastive Loss (InterCL) sharpens instance-specific details using random-layer and in-batch negatives. Two adaptive modulators, Frequency-based Adaptive Weight (FAW) and Frequency-based Alignment Modulation (FAM), reweight per-layer LF/HF signals and gate distillation by current similarity. Across U-Net and DiT backbones (e.g., Stable Diffusion 2, 3), FRAMER consistently improves PSNR/SSIM and perceptual metrics (LPIPS, NIQE, MANIQA, MUSIQ). Ablations validate the final-layer teacher and random-layer negatives.

Method: Proposes MDiff4STR with two key strategies: 1) Six noising strategies to align training/inference behavior, addressing the noising gap; 2) Token-replacement noise mechanism that provides non-mask noise to encourage revision of overconfident incorrect predictions.

Result: MDiff4STR consistently outperforms popular STR models across diverse benchmarks (irregular, artistic, occluded, Chinese text), surpassing state-of-the-art ARMs in accuracy while maintaining fast inference with only three denoising steps.

Conclusion: MDiff4STR successfully adapts Mask Diffusion Models to Scene Text Recognition, achieving superior accuracy over auto-regressive models while maintaining efficiency, demonstrating the potential of diffusion models for text recognition tasks.

Abstract: Mask Diffusion Models (MDMs) have recently emerged as a promising alternative to auto-regressive models (ARMs) for vision-language tasks, owing to their flexible balance of efficiency and accuracy. In this paper, for the first time, we introduce MDMs into the Scene Text Recognition (STR) task. We show that vanilla MDM lags behind ARMs in terms of accuracy, although it improves recognition efficiency. To bridge this gap, we propose MDiff4STR, a Mask Diffusion model enhanced with two key improvement strategies tailored for STR. Specifically, we identify two key challenges in applying MDMs to STR: noising gap between training and inference, and overconfident predictions during inference. Both significantly hinder the performance of MDMs. To mitigate the first issue, we develop six noising strategies that better align training with inference behavior. For the second, we propose a token-replacement noise mechanism that provides a non-mask noise type, encouraging the model to reconsider and revise overly confident but incorrect predictions. We conduct extensive evaluations of MDiff4STR on both standard and challenging STR benchmarks, covering diverse scenarios including irregular, artistic, occluded, and Chinese text, as well as whether the use of pretraining. Across these settings, MDiff4STR consistently outperforms popular STR models, surpassing state-of-the-art ARMs in accuracy, while maintaining fast inference with only three denoising steps. Code: https://github.com/Topdu/OpenOCR.

Method: Created ViRectify benchmark with 30K+ instances via AI-assisted annotation with human verification. Proposed trajectory evidence-driven correction framework with step-wise error trajectory and reward modeling on visual evidence-grounded correction to focus on error propagation and key timestamps.

Result: GPT-5 achieves only 31.94% correction accuracy on ViRectify, showing it’s a challenging testbed. The framework enables Qwen2.5-VL-7B to outperform 72B variants, demonstrating effectiveness. Analysis reveals systematic asymmetries in error correction across models.

Conclusion: ViRectify provides a comprehensive benchmark for evaluating MLLMs’ video reasoning error correction capabilities, offers a valuable data resource for reflection learning, and introduces a new direction for assessing advanced MLLMs in video reasoning.

Abstract: As multimodal large language models (MLLMs) frequently exhibit errors in complex video reasoning scenarios, correcting these errors is critical for uncovering their weaknesses and improving performance. However, existing benchmarks lack systematic evaluation of MLLMs’ ability to identify and correct these video reasoning errors. To bridge this gap, we propose \textit{ViRectify}, a comprehensive benchmark to evaluate their fine-grained correction capability. Through an AI-assisted annotation pipeline with human verification, we construct a dataset of over 30\textit{K} instances spanning dynamic perception, scientific reasoning, and embodied decision-making domains. In \textit{ViRectify}, we challenge MLLMs to perform step-wise error identification and generate rationales with key video evidence grounding. In addition, we further propose the trajectory evidence-driven correction framework, comprising step-wise error trajectory and reward modeling on visual evidence-grounded correction. It encourages the model to explicitly concentrate on error propagation and key timestamps for correction. Extensive evaluation across 16 advanced MLLMs demonstrates that our \textit{ViRectify} serves as a challenging testbed, where GPT-5 achieves only 31.94% correction accuracy. Our framework enables a Qwen2.5-VL-7B to consistently outperform the variants of 72B on \textit{ViRectify}, showing the effectiveness of our approach. Further analysis uncovers systematic asymmetries in error correction across models, and our dataset is also a valuable data resource to perform reflection learning. We believe \textit{ViRectify} provides a new direction for comprehensively evaluating the advanced MLLMs in video reasoning.

Method: 1) PE scaling technique to correct positional encoding errors when extrapolating to higher resolutions, 2) Local-enhancement mechanism using base-resolution local attention, 3) Patch-level fusion module to aggregate global and local cues, 4) Gaussian-weighted splicing strategy to eliminate grid artifacts.

Result: ResDiT consistently delivers high-fidelity, high-resolution image synthesis without additional training, integrates seamlessly with downstream tasks including spatially controlled generation, and outperforms previous complex pipeline approaches.

Conclusion: ResDiT provides an efficient, training-free solution for high-resolution image synthesis with Diffusion Transformers by addressing core issues in position embeddings and local detail generation, enabling practical high-resolution applications.

Abstract: Leveraging pre-trained Diffusion Transformers (DiTs) for high-resolution (HR) image synthesis often leads to spatial layout collapse and degraded texture fidelity. Prior work mitigates these issues with complex pipelines that first perform a base-resolution (i.e., training-resolution) denoising process to guide HR generation. We instead explore the intrinsic generative mechanisms of DiTs and propose ResDiT, a training-free method that scales resolution efficiently. We identify the core factor governing spatial layout, position embeddings (PEs), and show that the original PEs encode incorrect positional information when extrapolated to HR, which triggers layout collapse. To address this, we introduce a PE scaling technique that rectifies positional encoding under resolution changes. To further remedy low-fidelity details, we develop a local-enhancement mechanism grounded in base-resolution local attention. We design a patch-level fusion module that aggregates global and local cues, together with a Gaussian-weighted splicing strategy that eliminates grid artifacts. Comprehensive evaluations demonstrate that ResDiT consistently delivers high-fidelity, high-resolution image synthesis and integrates seamlessly with downstream tasks, including spatially controlled generation.

Method: Zero-shot approach leveraging CLIP’s shared image-text embedding space to segment unknown objects and assign human-interpretable names. Unlike open-vocabulary methods, it dynamically extends vocabulary at inference time without retraining, enabling detection and naming of anomalies beyond common class definitions.

Result: Achieves state-of-the-art performance on established anomaly segmentation benchmarks while providing interpretability and flexibility essential for practical deployment in autonomous driving systems.

Conclusion: Clipomaly is the first CLIP-based open-world and anomaly segmentation method that successfully addresses the limitations of existing approaches by enabling zero-shot detection and semantic labeling of unknown objects without requiring anomaly-specific training data.

Abstract: Open-world and anomaly segmentation methods seek to enable autonomous driving systems to detect and segment both known and unknown objects in real-world scenes. However, existing methods do not assign semantically meaningful labels to unknown regions, and distinguishing and learning representations for unknown classes remains difficult. While open-vocabulary segmentation methods show promise in generalizing to novel classes, they require a fixed inference vocabulary and thus cannot be directly applied to anomaly segmentation where unknown classes are unconstrained. We propose Clipomaly, the first CLIP-based open-world and anomaly segmentation method for autonomous driving. Our zero-shot approach requires no anomaly-specific training data and leverages CLIP’s shared image-text embedding space to both segment unknown objects and assign human-interpretable names to them. Unlike open-vocabulary methods, our model dynamically extends its vocabulary at inference time without retraining, enabling robust detection and naming of anomalies beyond common class definitions such as those in Cityscapes. Clipomaly achieves state-of-the-art performance on established anomaly segmentation benchmarks while providing interpretability and flexibility essential for practical deployment.

Method: A unified learning-based framework with two phases: 1) Uses a U-Net architecture to decouple texture and pose information in a feed-forward process for fast template generation, overcoming inefficiencies and artifacts. 2) Proposes a data-driven refinement technique to enhance structural integrity during target pose deformation.

Result: Extensive experiments show the model delivers consistent performance across diverse poses with optimal balance between efficiency and quality, surpassing state-of-the-art methods.

Conclusion: The proposed framework effectively addresses both template construction inefficiencies and deformation artifacts in 3D human avatar animation, achieving superior performance compared to existing approaches.

Abstract: 3D human avatar animation aims at transforming a human avatar from an arbitrary initial pose to a specified target pose using deformation algorithms. Existing approaches typically divide this task into two stages: canonical template construction and target pose deformation. However, current template construction methods demand extensive skeletal rigging and often produce artifacts for specific poses. Moreover, target pose deformation suffers from structural distortions caused by Linear Blend Skinning (LBS), which significantly undermines animation realism. To address these problems, we propose a unified learning-based framework to address both challenges in two phases. For the former phase, to overcome the inefficiencies and artifacts during template construction, we leverage a U-Net architecture that decouples texture and pose information in a feed-forward process, enabling fast generation of a human template. For the latter phase, we propose a data-driven refinement technique that enhances structural integrity. Extensive experiments show that our model delivers consistent performance across diverse poses with an optimal balance between efficiency and quality,surpassing state-of-the-art (SOTA) methods.

Method: Reinforcement Learning with World Grounding (RLWG) uses self-supervised post-training with geometric and perceptual rewards (pose cycle-consistency, depth reprojection, temporal coherence) to align pretrained world models with physically verifiable structure. Implemented as GrndCtrl using Group Relative Policy Optimization (GRPO).

Result: The framework yields world models that maintain stable trajectories, consistent geometry, and reliable rollouts for embodied navigation, achieving superior spatial coherence and navigation stability over supervised fine-tuning in outdoor environments.

Conclusion: RLWG bridges generative pretraining and grounded behavior through verifiable rewards, analogous to RLVR in language models, enabling geometrically grounded world models for navigation tasks.

Abstract: Recent advances in video world modeling have enabled large-scale generative models to simulate embodied environments with high visual fidelity, providing strong priors for prediction, planning, and control. Yet, despite their realism, these models often lack geometric grounding, limiting their use in navigation tasks that require spatial coherence and long-horizon stability. We introduce Reinforcement Learning with World Grounding (RLWG), a self-supervised post-training framework that aligns pretrained world models with a physically verifiable structure through geometric and perceptual rewards. Analogous to reinforcement learning from verifiable feedback (RLVR) in language models, RLWG can use multiple rewards that measure pose cycle-consistency, depth reprojection, and temporal coherence. We instantiate this framework with GrndCtrl, a reward-aligned adaptation method based on Group Relative Policy Optimization (GRPO), yielding world models that maintain stable trajectories, consistent geometry, and reliable rollouts for embodied navigation. Like post-training alignment in large language models, GrndCtrl leverages verifiable rewards to bridge generative pretraining and grounded behavior, achieving superior spatial coherence and navigation stability over supervised fine-tuning in outdoor environments.

Method: Proposes ChronosObserver with two key components: 1) World State Hyperspace to represent spatiotemporal constraints of 4D scenes, and 2) Hyperspace Guided Sampling to synchronize diffusion sampling trajectories across multiple views using the hyperspace representation.

Result: The method achieves high-fidelity, 3D-consistent, time-synchronized multi-view video generation without requiring training or fine-tuning of diffusion models.

Conclusion: ChronosObserver provides an effective training-free solution for generating synchronized multi-view videos with 3D consistency, addressing limitations of existing approaches while maintaining scalability and generalization.

Abstract: Although prevailing camera-controlled video generation models can produce cinematic results, lifting them directly to the generation of 3D-consistent and high-fidelity time-synchronized multi-view videos remains challenging, which is a pivotal capability for taming 4D worlds. Some works resort to data augmentation or test-time optimization, but these strategies are constrained by limited model generalization and scalability issues. To this end, we propose ChronosObserver, a training-free method including World State Hyperspace to represent the spatiotemporal constraints of a 4D world scene, and Hyperspace Guided Sampling to synchronize the diffusion sampling trajectories of multiple views using the hyperspace. Experimental results demonstrate that our method achieves high-fidelity and 3D-consistent time-synchronized multi-view videos generation without training or fine-tuning for diffusion models.

Method: Fine-tuning approach for PaddleOCRv5 using curated ancient Vietnamese Chinese manuscripts, with full training pipeline including preprocessing, LMDB conversion, evaluation, and visualization.

Result: Significant improvement over base model with exact accuracy increasing from 37.5% to 50.0%, particularly effective under noisy image conditions.

Conclusion: The fine-tuned OCR model enables better digitization of historical documents and supports downstream applications like Han-Vietnamese semantic alignment, machine translation, and historical linguistics research, with an interactive demo available.

Abstract: Recognizing and processing Classical Chinese (Han-Nom) texts play a vital role in digitizing Vietnamese historical documents and enabling cross-lingual semantic research. However, existing OCR systems struggle with degraded scans, non-standard glyphs, and handwriting variations common in ancient sources. In this work, we propose a fine-tuning approach for PaddleOCRv5 to improve character recognition on Han-Nom texts. We retrain the text recognition module using a curated subset of ancient Vietnamese Chinese manuscripts, supported by a full training pipeline covering preprocessing, LMDB conversion, evaluation, and visualization. Experimental results show a significant improvement over the base model, with exact accuracy increasing from 37.5 percent to 50.0 percent, particularly under noisy image conditions. Furthermore, we develop an interactive demo that visually compares pre- and post-fine-tuning recognition results, facilitating downstream applications such as Han-Vietnamese semantic alignment, machine translation, and historical linguistics research. The demo is available at https://huggingface.co/spaces/MinhDS/Fine-tuned-PaddleOCRv5

Method: Uses variational approach based on energy discretization and Smirnov’s decomposition theorem for vector fields. Implements bi-level minimization for curve extraction. Extends to curvature-dependent energies by lifting curves to position-orientation space with sub-Riemannian/Finslerian metrics.

Result: Developed a framework for automated curve extraction from images that can handle both simple and curvature-dependent cases through mathematical lifting techniques.

Conclusion: The variational approach provides an effective, mostly unsupervised method for extracting 1D structures from images, with extensions to curvature-dependent cases through geometric lifting techniques.

Abstract: We introduce a variational approach for extracting curves between a list of possible endpoints, based on the discretization of an energy and Smirnov’s decomposition theorem for vector fields. It is used to design a bi-level minimization approach to automatically extract curves and 1D structures from an image, which is mostly unsupervised. We extend then the method to curvature-dependent energies, using a now classical lifting of the curves in the space of positions and orientations equipped with an appropriate sub-Riemanian or Finslerian metric.

Method: ELVIS includes: 1) unsupervised synthetic low-light video pipeline modeling spatial/temporal degradations, 2) calibration-free degradation profile synthesis network (VDP-Net), and 3) enhancement decoder head that disentangles degradations from content features.

Result: Improves performance by up to +3.7AP on synthetic low-light YouTube-VIS 2019 dataset.

Conclusion: ELVIS enables effective domain adaptation of state-of-the-art VIS models to low-light scenarios, addressing key challenges in low-light video instance segmentation.

Abstract: Video instance segmentation (VIS) for low-light content remains highly challenging for both humans and machines alike, due to adverse imaging conditions including noise, blur and low-contrast. The lack of large-scale annotated datasets and the limitations of current synthetic pipelines, particularly in modeling temporal degradations, further hinder progress. Moreover, existing VIS methods are not robust to the degradations found in low-light videos and, as a result, perform poorly even when finetuned on low-light data. In this paper, we introduce \textbf{ELVIS} (\textbf{E}nhance \textbf{L}ow-light for \textbf{V}ideo \textbf{I}nstance \textbf{S}egmentation), a novel framework that enables effective domain adaptation of state-of-the-art VIS models to low-light scenarios. ELVIS comprises an unsupervised synthetic low-light video pipeline that models both spatial and temporal degradations, a calibration-free degradation profile synthesis network (VDP-Net) and an enhancement decoder head that disentangles degradations from content features. ELVIS improves performances by up to \textbf{+3.7AP} on the synthetic low-light YouTube-VIS 2019 dataset. Code will be released upon acceptance.

Method: SRCSM uses semantic-aware random convolution during training to diversify the source domain by applying different augmentations to different image regions based on annotation labels. At test-time, it maps target domain image intensities to resemble source domain data.

Result: The method outperforms previous DG techniques in most experiments across abdominal, whole-heart, and prostate segmentation tasks in cross-modality and cross-center settings. It even matches in-domain baseline performance in several settings and shows promising results for more challenging cine MR data.

Conclusion: SRCSM establishes a new state-of-the-art in domain generalization for medical image segmentation, demonstrating robust performance across diverse domain shifts and showing potential to close domain gaps even in challenging clinical scenarios.

Abstract: We tackle the challenging problem of single-source domain generalization (DG) for medical image segmentation. To this end, we aim for training a network on one domain (e.g., CT) and directly apply it to a different domain (e.g., MR) without adapting the model and without requiring images or annotations from the new domain during training. We propose a novel method for promoting DG when training deep segmentation networks, which we call SRCSM. During training, our method diversifies the source domain through semantic-aware random convolution, where different regions of a source image are augmented differently, based on their annotation labels. At test-time, we complement the randomization of the training domain via mapping the intensity of target domain images, making them similar to source domain data. We perform a comprehensive evaluation on a variety of cross-modality and cross-center generalization settings for abdominal, whole-heart and prostate segmentation, where we outperform previous DG techniques in a vast majority of experiments. Additionally, we also investigate our method when training on whole-heart CT or MR data and testing on the diastolic and systolic phase of cine MR data captured with different scanner hardware, where we make a step towards closing the domain gap in this even more challenging setting. Overall, our evaluation shows that SRCSM can be considered a new state-of-the-art in DG for medical image segmentation and, moreover, even achieves a segmentation performance that matches the performance of the in-domain baseline in several settings.

Method: Two-stage framework: 1) Draft stage: multiple small VLMs act as draft experts to generate diverse reasoning paths with localization candidates; 2) Verdict stage: a strong VLM synthesizes these paths to produce final answer. Includes consensus expert selection to forward only high-agreement paths for efficiency.

Result: Achieves consistent gains on challenging information-intensive and high-resolution VQA benchmarks (InfographicVQA, ChartMuseum, ChartQAPro, HR-Bench 4K). Enables error correction and cost-efficiency compared to large proprietary models or training pipelines.

Conclusion: SV framework successfully addresses challenges in information-intensive visual reasoning by combining multiple draft experts with a verdict model, achieving both improved accuracy and computational efficiency without requiring training.

Abstract: Large Vision-Language Models (VLMs) have achieved remarkable progress in multimodal understanding, yet they struggle when reasoning over information-intensive images that densely interleave textual annotations with fine-grained graphical elements. The main challenges lie in precisely localizing critical cues in dense layouts and multi-hop reasoning to integrate dispersed evidence. We propose Speculative Verdict (SV), a training-free framework inspired by speculative decoding that combines multiple lightweight draft experts with a large verdict model. In the draft stage, small VLMs act as draft experts to generate reasoning paths that provide diverse localization candidates; in the verdict stage, a strong VLM synthesizes these paths to produce the final answer, minimizing computational cost while recovering correct answers. To further improve efficiency and accuracy, SV introduces a consensus expert selection mechanism that forwards only high-agreement reasoning paths to the verdict. Empirically, SV achieves consistent gains on challenging information-intensive and high-resolution visual question answering benchmarks, including InfographicVQA, ChartMuseum, ChartQAPro, and HR-Bench 4K. By synthesizing correct insights from multiple partially accurate reasoning paths, SV achieves both error correction and cost-efficiency compared to large proprietary models or training pipelines. Code is available at https://github.com/Tinaliu0123/speculative-verdict.

Method: Created QuantumCanvas dataset spanning 2,850 element-element pairs with 18 electronic, thermodynamic, and geometric properties, paired with ten-channel image representations derived from orbital densities, angular field transforms, co-occupancy maps, and charge-density projections. Benchmarking eight architectures across 18 targets.

Result: Achieved mean absolute errors of 0.201 eV on energy gap (GATv2), 0.265 eV on HOMO and 0.274 eV on LUMO (EGNN). DimeNet attained 2.27 eV total-energy MAE and 0.132 eV repulsive-energy MAE. Multimodal fusion achieved 2.15 eV Mermin free-energy MAE. Pretraining improves convergence and generalization on larger datasets.

Conclusion: QuantumCanvas provides a principled, interpretable basis for learning transferable quantum interactions through coupled visual and numerical modalities, unifying orbital physics with vision-based representation learning to address the physical transferability gap in molecular machine learning.

Abstract: Despite rapid advances in molecular and materials machine learning, most models still lack physical transferability: they fit correlations across whole molecules or crystals rather than learning the quantum interactions between atomic pairs. Yet bonding, charge redistribution, orbital hybridization, and electronic coupling all emerge from these two-body interactions that define local quantum fields in many-body systems. We introduce QuantumCanvas, a large-scale multimodal benchmark that treats two-body quantum systems as foundational units of matter. The dataset spans 2,850 element-element pairs, each annotated with 18 electronic, thermodynamic, and geometric properties and paired with ten-channel image representations derived from l- and m-resolved orbital densities, angular field transforms, co-occupancy maps, and charge-density projections. These physically grounded images encode spatial, angular, and electrostatic symmetries without explicit coordinates, providing an interpretable visual modality for quantum learning. Benchmarking eight architectures across 18 targets, we report mean absolute errors of 0.201 eV on energy gap using GATv2, 0.265 eV on HOMO and 0.274 eV on LUMO using EGNN. For energy-related quantities, DimeNet attains 2.27 eV total-energy MAE and 0.132 eV repulsive-energy MAE, while a multimodal fusion model achieves a 2.15 eV Mermin free-energy MAE. Pretraining on QuantumCanvas further improves convergence stability and generalization when fine-tuned on larger datasets such as QM9, MD17, and CrysMTM. By unifying orbital physics with vision-based representation learning, QuantumCanvas provides a principled and interpretable basis for learning transferable quantum interactions through coupled visual and numerical modalities. Dataset and model implementations are available at https://github.com/KurbanIntelligenceLab/QuantumCanvas.

Method: VisualSync uses multi-view dynamics optimization that leverages off-the-shelf 3D reconstruction, feature matching, and dense tracking to extract tracklets, relative poses, and cross-view correspondences. It then jointly minimizes epipolar error to estimate each camera’s time offset.

Result: Experiments on four diverse, challenging datasets show VisualSync outperforms baseline methods, achieving median synchronization error below 50 ms.

Conclusion: VisualSync provides an effective framework for synchronizing unposed, unsynchronized videos from consumer cameras with high accuracy, overcoming limitations of existing methods that require controlled settings or specialized hardware.

Abstract: Today, people can easily record memorable moments, ranging from concerts, sports events, lectures, family gatherings, and birthday parties with multiple consumer cameras. However, synchronizing these cross-camera streams remains challenging. Existing methods assume controlled settings, specific targets, manual correction, or costly hardware. We present VisualSync, an optimization framework based on multi-view dynamics that aligns unposed, unsynchronized videos at millisecond accuracy. Our key insight is that any moving 3D point, when co-visible in two cameras, obeys epipolar constraints once properly synchronized. To exploit this, VisualSync leverages off-the-shelf 3D reconstruction, feature matching, and dense tracking to extract tracklets, relative poses, and cross-view correspondences. It then jointly minimizes the epipolar error to estimate each camera’s time offset. Experiments on four diverse, challenging datasets show that VisualSync outperforms baseline methods, achieving an median synchronization error below 50 ms.

Method: FlashVGGT compresses spatial information from each frame into compact descriptor tokens, then computes global attention as cross-attention between full image tokens and this smaller descriptor set. It also enables online inference via chunk-recursive mechanism that reuses cached descriptors.

Result: FlashVGGT achieves reconstruction accuracy competitive with VGGT while reducing inference time to just 9.3% of VGGT for 1,000 images, and scales efficiently to sequences exceeding 3,000 images.

Conclusion: FlashVGGT provides an efficient alternative to full self-attention methods for 3D reconstruction, addressing scalability bottlenecks while maintaining competitive performance, enabling practical applications with long image sequences.

Abstract: 3D reconstruction from multi-view images is a core challenge in computer vision. Recently, feed-forward methods have emerged as efficient and robust alternatives to traditional per-scene optimization techniques. Among them, state-of-the-art models like the Visual Geometry Grounding Transformer (VGGT) leverage full self-attention over all image tokens to capture global relationships. However, this approach suffers from poor scalability due to the quadratic complexity of self-attention and the large number of tokens generated in long image sequences. In this work, we introduce FlashVGGT, an efficient alternative that addresses this bottleneck through a descriptor-based attention mechanism. Instead of applying dense global attention across all tokens, FlashVGGT compresses spatial information from each frame into a compact set of descriptor tokens. Global attention is then computed as cross-attention between the full set of image tokens and this smaller descriptor set, significantly reducing computational overhead. Moreover, the compactness of the descriptors enables online inference over long sequences via a chunk-recursive mechanism that reuses cached descriptors from previous chunks. Experimental results show that FlashVGGT achieves reconstruction accuracy competitive with VGGT while reducing inference time to just 9.3% of VGGT for 1,000 images, and scaling efficiently to sequences exceeding 3,000 images. Our project page is available at https://wzpscott.github.io/flashvggt_page/.

Method: Created AbscessHeNe dataset with 4,926 contrast-enhanced CT slices of clinically confirmed head/neck abscesses, featuring pixel-level annotations and clinical metadata. Evaluated CNN, Transformer, and Mamba-based segmentation architectures to establish performance baselines.

Result: Best model achieved Dice Similarity Coefficient of 0.39, Intersection-over-Union of 0.27, and Normalized Surface Distance of 0.67, highlighting the difficulty of the segmentation task. Dataset structured for future multimedia indexing and case-based retrieval applications.

Conclusion: AbscessHeNe provides a valuable resource for developing abscess segmentation models and supporting clinical workflows. The modest baseline performance indicates significant challenges and need for further research. Dataset will be publicly available for community use.

Abstract: Abscesses in the head and neck represent an acute infectious process that can potentially lead to sepsis or mortality if not diagnosed and managed promptly. Accurate detection and delineation of these lesions on imaging are essential for diagnosis, treatment planning, and surgical intervention. In this study, we introduce AbscessHeNe, a curated and comprehensively annotated dataset comprising 4,926 contrast-enhanced CT slices with clinically confirmed head and neck abscesses. The dataset is designed to facilitate the development of robust semantic segmentation models that can accurately delineate abscess boundaries and evaluate deep neck space involvement, thereby supporting informed clinical decision-making. To establish performance baselines, we evaluate several state-of-the-art segmentation architectures, including CNN, Transformer, and Mamba-based models. The highest-performing model achieved a Dice Similarity Coefficient of 0.39, Intersection-over-Union of 0.27, and Normalized Surface Distance of 0.67, indicating the challenges of this task and the need for further research. Beyond segmentation, AbscessHeNe is structured for future applications in content-based multimedia indexing and case-based retrieval. Each CT scan is linked with pixel-level annotations and clinical metadata, providing a foundation for building intelligent retrieval systems and supporting knowledge-driven clinical workflows. The dataset will be made publicly available at https://github.com/drthaodao3101/AbscessHeNe.git.

Method: Shape Dynamic Time Warping (ShapeDTW) algorithm with local shape features, using combined HOG1D and original signal as shape descriptors to construct morphologically sensitive distance matrix for sequence alignment.

Result: Field tests show precise alignment for images with complex textures, depth shifts, or local scaling, providing better structural similarity preservation during alignment.

Conclusion: The ShapeDTW-based method effectively solves depth misalignment in borehole imaging and offers a flexible framework for integrating other descriptors tailored to specific geological features.

Abstract: In well logging operations using the oil-based mud (OBM) microresistivity imager, which employs an interleaved design with upper and lower pad sets, depth misalignment issues persist between the pad images even after velocity correction. This paper presents a depth matching method for borehole images based on the Shape Dynamic Time Warping (ShapeDTW) algorithm. The method extracts local shape features to construct a morphologically sensitive distance matrix, better preserving structural similarity between sequences during alignment. We implement this by employing a combined feature set of the one-dimensional Histogram of Oriented Gradients (HOG1D) and the original signal as the shape descriptor. Field test examples demonstrate that our method achieves precise alignment for images with complex textures, depth shifts, or local scaling. Furthermore, it provides a flexible framework for feature extension, allowing the integration of other descriptors tailored to specific geological features.

Method: 1) Use VLMs to extract coarse URDF parameters and generate part-level reference images; 2) Integrate part-image guidance and inferred structure graph into generative diffusion transformer for consistent part and complete shape synthesis; 3) Refine URDF parameters using differentiable forward kinematics and rendering optimization under VLM-generated open-state supervision.

Result: Extensive experiments show SPARK produces high-quality, simulation-ready articulated assets across diverse categories, enabling downstream applications like robotic manipulation and interaction modeling.

Conclusion: SPARK provides an effective framework for reconstructing physically consistent, kinematic part-level articulated objects from single images, addressing the labor-intensive nature of creating simulation-ready assets.

Abstract: Articulated 3D objects are critical for embodied AI, robotics, and interactive scene understanding, yet creating simulation-ready assets remains labor-intensive and requires expert modeling of part hierarchies and motion structures. We introduce SPARK, a framework for reconstructing physically consistent, kinematic part-level articulated objects from a single RGB image. Given an input image, we first leverage VLMs to extract coarse URDF parameters and generate part-level reference images. We then integrate the part-image guidance and the inferred structure graph into a generative diffusion transformer to synthesize consistent part and complete shapes of articulated objects. To further refine the URDF parameters, we incorporate differentiable forward kinematics and differentiable rendering to optimize joint types, axes, and origins under VLM-generated open-state supervision. Extensive experiments show that SPARK produces high-quality, simulation-ready articulated assets across diverse categories, enabling downstream applications such as robotic manipulation and interaction modeling.

Method: Proposes Fusion-Diff with two key innovations: 1) multimodal fusion feature editing in joint vision-language space to narrow modality gap, and 2) lightweight Control-Adapter fine-tuned on only 200K synthetic samples for data efficiency.

Result: Significantly outperforms prior zero-shot approaches on standard CIR benchmarks (CIRR, FashionIQ, CIRCO). Achieves state-of-the-art performance with enhanced interpretability through visualization of fused multimodal representations.

Conclusion: Fusion-Diff effectively addresses the modality gap challenge in zero-shot CIR while maintaining high data efficiency, offering a promising direction for fine-grained visual search without expensive annotations.

Abstract: Composed Image Retrieval (CIR) enables fine-grained visual search by combining a reference image with a textual modification. While supervised CIR methods achieve high accuracy, their reliance on costly triplet annotations motivates zero-shot solutions. The core challenge in zero-shot CIR (ZS-CIR) stems from a fundamental dilemma: existing text-centric or diffusion-based approaches struggle to effectively bridge the vision-language modality gap. To address this, we propose Fusion-Diff, a novel generative editing framework with high effectiveness and data efficiency designed for multimodal alignment. First, it introduces a multimodal fusion feature editing strategy within a joint vision-language (VL) space, substantially narrowing the modality gap. Second, to maximize data efficiency, the framework incorporates a lightweight Control-Adapter, enabling state-of-the-art performance through fine-tuning on only a limited-scale synthetic dataset of 200K samples. Extensive experiments on standard CIR benchmarks (CIRR, FashionIQ, and CIRCO) demonstrate that Fusion-Diff significantly outperforms prior zero-shot approaches. We further enhance the interpretability of our model by visualizing the fused multimodal representations.

Method: Introduces UniSandbox, a decoupled evaluation framework with controlled synthetic datasets to avoid data leakage. Uses explicit Chain-of-Thought (CoT) in understanding modules and self-training approaches to internalize reasoning abilities.

Result: Reveals significant understanding-generation gap in two dimensions: reasoning generation and knowledge transfer. Shows explicit CoT effectively bridges the gap, and self-training can internalize this ability for implicit reasoning during generation. Also finds CoT helps retrieve newly learned knowledge and query-based architectures have latent CoT-like properties.

Conclusion: UniSandbox provides preliminary insights for designing future unified architectures and training strategies to bridge the understanding-generation gap. The framework enables detailed analysis of how understanding informs generation in multimodal models.

Abstract: Recent years have witnessed significant progress in Unified Multimodal Models, yet a fundamental question remains: Does understanding truly inform generation? To investigate this, we introduce UniSandbox, a decoupled evaluation framework paired with controlled, synthetic datasets to avoid data leakage and enable detailed analysis. Our findings reveal a significant understanding-generation gap, which is mainly reflected in two key dimensions: reasoning generation and knowledge transfer. Specifically, for reasoning generation tasks, we observe that explicit Chain-of-Thought (CoT) in the understanding module effectively bridges the gap, and further demonstrate that a self-training approach can successfully internalize this ability, enabling implicit reasoning during generation. Additionally, for knowledge transfer tasks, we find that CoT assists the generative process by helping retrieve newly learned knowledge, and also discover that query-based architectures inherently exhibit latent CoT-like properties that affect this transfer. UniSandbox provides preliminary insights for designing future unified architectures and training strategies that truly bridge the gap between understanding and generation. Code and data are available at https://github.com/PKU-YuanGroup/UniSandBox

Method: Conducted systematic empirical study of TTT designs for visual sequence modeling, distilled six practical insights, and developed ViT³ - a pure TTT architecture with linear complexity and parallelizable computation.

Result: ViT³ consistently matches or outperforms advanced linear-complexity models (Mamba, linear attention variants) across image classification, generation, object detection, and segmentation, narrowing the gap to optimized vision Transformers.

Conclusion: The study establishes design principles for effective visual TTT and provides ViT³ as a strong baseline to facilitate future work on efficient visual sequence modeling.

Abstract: Test-Time Training (TTT) has recently emerged as a promising direction for efficient sequence modeling. TTT reformulates attention operation as an online learning problem, constructing a compact inner model from key-value pairs at test time. This reformulation opens a rich and flexible design space while achieving linear computational complexity. However, crafting a powerful visual TTT design remains challenging: fundamental choices for the inner module and inner training lack comprehensive understanding and practical guidelines. To bridge this critical gap, in this paper, we present a systematic empirical study of TTT designs for visual sequence modeling. From a series of experiments and analyses, we distill six practical insights that establish design principles for effective visual TTT and illuminate paths for future improvement. These findings culminate in the Vision Test-Time Training (ViT$^3$) model, a pure TTT architecture that achieves linear complexity and parallelizable computation. We evaluate ViT$^3$ across diverse visual tasks, including image classification, image generation, object detection, and semantic segmentation. Results show that ViT$^3$ consistently matches or outperforms advanced linear-complexity models (e.g., Mamba and linear attention variants) and effectively narrows the gap to highly optimized vision Transformers. We hope this study and the ViT$^3$ baseline can facilitate future work on visual TTT models. Code is available at https://github.com/LeapLabTHU/ViTTT.

Method: Proposes Adaptive Dual Branch Kolmogorov-Arnold UNet (DB-KAUNet) with Heterogeneous Dual-Branch Encoder (HDBE) featuring parallel CNN and Transformer pathways. Key components include: KANConv and KAT blocks, Cross-Branch Channel Interaction (CCI) module, attention-based Spatial Feature Enhancement (SFE) module, and advanced SFE-GAF module with adaptive sampling for precise vessel morphology focus.

Result: Extensive experiments on DRIVE, STARE, and CHASE_DB1 datasets validate that DB-KAUNet achieves leading segmentation performance and demonstrates exceptional robustness.

Conclusion: DB-KAUNet effectively addresses the limitations of traditional CNN methods by combining CNN and Transformer strengths with novel architectural components, resulting in superior retinal vessel segmentation for clinical applications.

Abstract: Accurate segmentation of retinal vessels is crucial for the clinical diagnosis of numerous ophthalmic and systemic diseases. However, traditional Convolutional Neural Network (CNN) methods exhibit inherent limitations, struggling to capture long-range dependencies and complex nonlinear relationships. To address the above limitations, an Adaptive Dual Branch Kolmogorov-Arnold UNet (DB-KAUNet) is proposed for retinal vessel segmentation. In DB-KAUNet, we design a Heterogeneous Dual-Branch Encoder (HDBE) that features parallel CNN and Transformer pathways. The HDBE strategically interleaves standard CNN and Transformer blocks with novel KANConv and KAT blocks, enabling the model to form a comprehensive feature representation. To optimize feature processing, we integrate several critical components into the HDBE. First, a Cross-Branch Channel Interaction (CCI) module is embedded to facilitate efficient interaction of channel features between the parallel pathways. Second, an attention-based Spatial Feature Enhancement (SFE) module is employed to enhance spatial features and fuse the outputs from both branches. Building upon the SFE module, an advanced Spatial Feature Enhancement with Geometrically Adaptive Fusion (SFE-GAF) module is subsequently developed. In the SFE-GAF module, adaptive sampling is utilized to focus on true vessel morphology precisely. The adaptive process strengthens salient vascular features while significantly reducing background noise and computational overhead. Extensive experiments on the DRIVE, STARE, and CHASE_DB1 datasets validate that DB-KAUNet achieves leading segmentation performance and demonstrates exceptional robustness.

Method: Proposes ScaleBridge-Det with two key modules: 1) Routing-Enhanced Mixture Attention (REM) that dynamically selects/fuses scale-specific expert features via adaptive routing, and 2) Density-Guided Dynamic Query (DGQ) that predicts object density to adaptively adjust query positions/numbers for efficient resource allocation.

Result: Achieves state-of-the-art performance on AI-TOD-V2 and DTOD datasets, while showing superior cross-domain robustness on VisDrone dataset. The framework optimizes performance for both dense tiny and general objects without trade-offs.

Conclusion: ScaleBridge-Det successfully bridges the scale gap in remote sensing object detection, providing the first large detection framework specifically designed for tiny objects that achieves balanced performance across diverse scales through innovative scale-adaptive mechanisms.

Abstract: Tiny object detection in remote sensing imagery has attracted significant research interest in recent years. Despite recent progress, achieving balanced detection performance across diverse object scales remains a formidable challenge, particularly in scenarios where dense tiny objects and large objects coexist. Although large foundation models have revolutionized general vision tasks, their application to tiny object detection remains unexplored due to the extreme scale variation and density distribution inherent to remote sensing imagery. To bridge this scale gap, we propose ScaleBridge-Det, to the best of our knowledge, the first large detection framework designed for tiny objects, which could achieve balanced performance across diverse scales through scale-adaptive expert routing and density-guided query allocation. Specifically, we introduce a Routing-Enhanced Mixture Attention (REM) module that dynamically selects and fuses scale-specific expert features via adaptive routing to address the tendency of standard MoE models to favor dominant scales. REM generates complementary and discriminative multi-scale representations suitable for both tiny and large objects. Furthermore, we present a Density-Guided Dynamic Query (DGQ) module that predicts object density to adaptively adjust query positions and numbers, enabling efficient resource allocation for objects of varying scales. The proposed framework allows ScaleBridge-Det to simultaneously optimize performance for both dense tiny and general objects without trade-offs. Extensive experiments on benchmark and cross-domain datasets demonstrate that ScaleBridge-Det achieves state-of-the-art performance on AI-TOD-V2 and DTOD, while exhibiting superior cross-domain robustness on VisDrone.

Method: GRASP uses external priors to partition samples into clusters minimizing intra-group gradient conflicts, then fine-tunes pre-trained models by injecting cluster-specific residual adapters into transformer feedforward layers, bypassing learned gating for stability and efficiency.

Result: On MIMIC-CXR-LT dataset, GRASP achieves superior FID and diversity metrics, especially for rare classes, outperforming baselines like vanilla fine-tuning and Mixture of Experts variants. Downstream classification on NIH-CXR-LT improves for tail labels, and generalization to ImageNet-LT confirms broad applicability.

Conclusion: GRASP provides a lightweight, scalable solution to gradient conflicts in long-tail diffusion models, enabling better synthetic data generation for underrepresented classes while maintaining efficiency and integration with existing diffusion pipelines.

Abstract: Recent advances in text-to-image diffusion models enable high-fidelity generation across diverse prompts. However, these models falter in long-tail settings, such as medical imaging, where rare pathologies comprise a small fraction of data. This results in mode collapse: tail-class outputs lack quality and diversity, undermining the goal of synthetic data augmentation for underrepresented conditions. We pinpoint gradient conflicts between frequent head and rare tail classes as the primary culprit, a factor unaddressed by existing sampling or conditioning methods that mainly steer inference without altering the learned distribution. To resolve this, we propose GRASP: Guided Residual Adapters with Sample-wise Partitioning. GRASP uses external priors to statically partition samples into clusters that minimize intra-group gradient clashes. It then fine-tunes pre-trained models by injecting cluster-specific residual adapters into transformer feedforward layers, bypassing learned gating for stability and efficiency. On the long-tail MIMIC-CXR-LT dataset, GRASP yields superior FID and diversity metrics, especially for rare classes, outperforming baselines like vanilla fine-tuning and Mixture of Experts variants. Downstream classification on NIH-CXR-LT improves considerably for tail labels. Generalization to ImageNet-LT confirms broad applicability. Our method is lightweight, scalable, and readily integrates with diffusion pipelines.

Method: Proposes SCAR (Structure and Contact-Aware Representation) that captures hand-object contact, occlusion, and holistic structure context without 3D annotations. Introduces joint-generation paradigm with share-and-specialization strategy to generate both interaction-oriented representations and videos.

Result: Outperforms state-of-the-art methods on two real-world datasets in generating physics-realistic and temporally coherent HOI videos. Shows strong generalization to challenging open-world scenarios.

Conclusion: The proposed SCAR representation provides scalable yet interaction-faithful supervision for HOI video generation, enabling learning of fine-grained interaction physics and generalization to open-world scenarios without 3D annotations.

Abstract: Generating realistic hand-object interactions (HOI) videos is a significant challenge due to the difficulty of modeling physical constraints (e.g., contact and occlusion between hands and manipulated objects). Current methods utilize HOI representation as an auxiliary generative objective to guide video synthesis. However, there is a dilemma between 2D and 3D representations that cannot simultaneously guarantee scalability and interaction fidelity. To address this limitation, we propose a structure and contact-aware representation that captures hand-object contact, hand-object occlusion, and holistic structure context without 3D annotations. This interaction-oriented and scalable supervision signal enables the model to learn fine-grained interaction physics and generalize to open-world scenarios. To fully exploit the proposed representation, we introduce a joint-generation paradigm with a share-and-specialization strategy that generates interaction-oriented representations and videos. Extensive experiments demonstrate that our method outperforms state-of-the-art methods on two real-world datasets in generating physics-realistic and temporally coherent HOI videos. Furthermore, our approach exhibits strong generalization to challenging open-world scenarios, highlighting the benefit of our scalable design. Our project page is https://hgzn258.github.io/SCAR/.

Method: Train a self-supervised encoder on large tissue images from resection specimens, then apply it to smaller biopsy material to capture meaningful morphological patterns for analysis.

Result: The model successfully captures morphological patterns from biopsy material and can predict patient survival and classify tumor subtypes using these patterns.

Conclusion: AI-driven tools trained on resection tissue can be effectively applied to biopsy material, demonstrating potential to support mesothelioma diagnosis and treatment planning in real-world clinical settings.

Abstract: Accurate subtype classification and outcome prediction in mesothelioma are essential for guiding therapy and patient care. Most computational pathology models are trained on large tissue images from resection specimens, limiting their use in real-world settings where small biopsies are common. We show that a self-supervised encoder trained on resection tissue can be applied to biopsy material, capturing meaningful morphological patterns. Using these patterns, the model can predict patient survival and classify tumor subtypes. This approach demonstrates the potential of AI-driven tools to support diagnosis and treatment planning in mesothelioma.

Method: Builds on pretrained video DiT model; introduces RegionalRoPE for layout-based position control via region-aware positional encoding; adds masked condition loss for spatial constraints; integrates LLM-based layout generator for comic-style layouts from natural language scripts.

Result: Achieves 29.2% increase in character consistency and 36.2% increase in style similarity compared to previous methods, with high spatial accuracy.

Conclusion: DreamingComics effectively addresses layout control and consistency issues in story visualization, enabling flexible comic-style generation with improved artistic quality.

Abstract: Current story visualization methods tend to position subjects solely by text and face challenges in maintaining artistic consistency. To address these limitations, we introduce DreamingComics, a layout-aware story visualization framework. We build upon a pretrained video diffusion-transformer (DiT) model, leveraging its spatiotemporal priors to enhance identity and style consistency. For layout-based position control, we propose RegionalRoPE, a region-aware positional encoding scheme that re-indexes embeddings based on the target layout. Additionally, we introduce a masked condition loss to further constrain each subject’s visual features to their designated region. To infer layouts from natural language scripts, we integrate an LLM-based layout generator trained to produce comic-style layouts, enabling flexible and controllable layout conditioning. We present a comprehensive evaluation of our approach, showing a 29.2% increase in character consistency and a 36.2% increase in style similarity compared to previous methods, while displaying high spatial accuracy. Our project page is available at https://yj7082126.github.io/dreamingcomics/

Method: Proposes Semantic and Spatial Rectification (SSR) with two components: 1) Cross-Modal Prototype Alignment (CMPA) for semantic-level contrastive learning to align feature spaces across modalities, and 2) Superpixel-Guided Correction (SGC) for spatial-level filtering using superpixel-based priors during affinity propagation.

Result: Achieves state-of-the-art performance on PASCAL VOC (79.5% mIoU) and MS COCO (50.6% mIoU), outperforming both single-stage and more complex multi-stage approaches.

Conclusion: The SSR method effectively addresses over-activation issues in CLIP-based WSSS through semantic and spatial rectification, demonstrating superior segmentation performance across benchmark datasets.

Abstract: In recent years, Contrastive Language-Image Pretraining (CLIP) has been widely applied to Weakly Supervised Semantic Segmentation (WSSS) tasks due to its powerful cross-modal semantic understanding capabilities. This paper proposes a novel Semantic and Spatial Rectification (SSR) method to address the limitations of existing CLIP-based weakly supervised semantic segmentation approaches: over-activation in non-target foreground regions and background areas. Specifically, at the semantic level, the Cross-Modal Prototype Alignment (CMPA) establishes a contrastive learning mechanism to enforce feature space alignment across modalities, reducing inter-class overlap while enhancing semantic correlations, to rectify over-activation in non-target foreground regions effectively; at the spatial level, the Superpixel-Guided Correction (SGC) leverages superpixel-based spatial priors to precisely filter out interference from non-target regions during affinity propagation, significantly rectifying background over-activation. Extensive experiments on the PASCAL VOC and MS COCO datasets demonstrate that our method outperforms all single-stage approaches, as well as more complex multi-stage approaches, achieving mIoU scores of 79.5% and 50.6%, respectively.

Method: FreqEdit uses three components: (1) high-frequency feature injection from reference velocity fields to preserve details, (2) adaptive injection strategy for spatial modulation of injection strength, and (3) path compensation mechanism to periodically recalibrate editing trajectory.

Result: FreqEdit enables stable editing across 10+ consecutive iterations and outperforms seven state-of-the-art baselines in both identity preservation and instruction following.

Conclusion: The training-free FreqEdit framework effectively addresses multi-turn editing degradation by preserving high-frequency information, offering a practical solution for stable iterative image editing.

Abstract: Instruction-based image editing through natural language has emerged as a powerful paradigm for intuitive visual manipulation. While recent models achieve impressive results on single edits, they suffer from severe quality degradation under multi-turn editing. Through systematic analysis, we identify progressive loss of high-frequency information as the primary cause of this quality degradation. We present FreqEdit, a training-free framework that enables stable editing across 10+ consecutive iterations. Our approach comprises three synergistic components: (1) high-frequency feature injection from reference velocity fields to preserve fine-grained details, (2) an adaptive injection strategy that spatially modulates injection strength for precise region-specific control, and (3) a path compensation mechanism that periodically recalibrates the editing trajectory to prevent over-constraint. Extensive experiments demonstrate that FreqEdit achieves superior performance in both identity preservation and instruction following compared to seven state-of-the-art baselines.

Method: HCPO with two components: (1) Dynamic Context Sampling (DCS) presents variable length histories during sampling for adaptive context use; (2) Anchor-guided History Compression (AHC) uses a dual branch strategy with compressed branch removing history observations while keeping actions as anchors, coupled through history-enhanced alignment loss.

Result: HiconAgent-3B outperforms GUI-R1-7B by +8.46% grounding accuracy and +11.32% step success rate on GUI-Odyssey, achieves comparable results on AndroidControl and AITW with up to 2.47x speedup and 60% FLOPs reduction.

Conclusion: HiconAgent demonstrates efficient and effective historical context utilization for GUI agents through HCPO, achieving strong performance with reduced computational requirements.

Abstract: Graphical User Interface (GUI) agents require effective use of historical context to perform sequential navigation tasks. While incorporating past actions and observations can improve decision making, naive use of full history leads to excessive computational overhead and distraction from irrelevant information. To address this, we introduce HiconAgent, a GUI agent trained with History Context-aware Policy Optimization (HCPO) for efficient and effective utilization of historical information. HCPO optimizes history usage in both sampling and policy updates through two complementary components: (1) Dynamic Context Sampling (DCS) presents the agent with variable length histories during sampling, enabling adaptive use of the most relevant context; (2) Anchor-guided History Compression (AHC) refines the policy update phase with a dual branch strategy where the compressed branch removes history observations while keeping history actions as information flow anchors. The compressed and uncompressed branches are coupled through a history-enhanced alignment loss to enforce consistent history usage while maintaining efficiency. Experiments on mainstream GUI navigation benchmarks demonstrate strong performance. Despite being smaller, HiconAgent-3B outperforms GUI-R1-7B by +8.46 percent grounding accuracy and +11.32 percent step success rate on GUI-Odyssey, while achieving comparable results on AndroidControl and AITW with up to 2.47x computational speedup and 60 percent FLOPs reduction.

Method: Extracts video content using state-of-the-art techniques, represents it using extended relational model (R++) and graph models, enables continuous query processing via a specialized query language, and uses parameterized templates for domain independence.

Result: Extensive experiments across Assisted Living, Civic Monitoring, and Surveillance domains show the approach achieves accuracy, efficiency, and robustness with videos of varying lengths.

Conclusion: The proposed framework overcomes limitations of manual analysis and domain-specific algorithms by providing a general-purpose solution for automated video situation analysis across multiple application domains.

Abstract: Automatically understanding video contents is important for several applications in Civic Monitoring (CM), general Surveillance (SL), Assisted Living (AL), etc. Decades of Image and Video Analysis (IVA) research have advanced tasks such as content extraction (e.g., object recognition and tracking). Identifying meaningful activities or situations (e.g., two objects coming closer) remains difficult and cannot be achieved by content extraction alone. Currently, Video Situation Analysis (VSA) is done manually with a human in the loop, which is error-prone and labor-intensive, or through custom algorithms designed for specific video types or situations. These algorithms are not general-purpose and require a new algorithm/software for each new situation or video from a new domain. This report proposes a general-purpose VSA framework that overcomes the above limitations. Video contents are extracted once using state-of-the-art Video Content Extraction technologies. They are represented using two alternative models – the extended relational model (R++) and graph models. When represented using R++, the extracted contents can be used as data streams, enabling Continuous Query Processing via the proposed Continuous Query Language for Video Analysis. The graph models complement this by enabling the detection of situations that are difficult or impossible to detect using the relational model alone. Existing graph algorithms and newly developed algorithms support a wide variety of situation detection. To support domain independence, primitive situation variants across domains are identified and expressed as parameterized templates. Extensive experiments were conducted across several interesting situations from three domains – AL, CM, and SL– to evaluate the accuracy, efficiency, and robustness of the proposed approach using a dataset of videos of varying lengths from these domains.

Method: Integrates learnable edge kernels with learning-based rigid and non-rigid registration. Starts with predefined edge detection kernel perturbed with random noise, then learns optimal edge features during training. Introduces four variant models each for rigid and non-rigid registration to analyze component contributions.

Result: Method consistently outperformed state-of-the-art techniques across all experiments: rigid registration without/with skull removal, non-rigid registration on Medical University dataset, and two public datasets.

Conclusion: The adaptive edge detection approach enhances medical image registration by capturing diverse structural features, demonstrating potential to improve multi-modal image alignment and anatomical structure analysis.

Abstract: Medical image registration is crucial for various clinical and research applications including disease diagnosis or treatment planning which require alignment of images from different modalities, time points, or subjects. Traditional registration techniques often struggle with challenges such as contrast differences, spatial distortions, and modality-specific variations. To address these limitations, we propose a method that integrates learnable edge kernels with learning-based rigid and non-rigid registration techniques. Unlike conventional layers that learn all features without specific bias, our approach begins with a predefined edge detection kernel, which is then perturbed with random noise. These kernels are learned during training to extract optimal edge features tailored to the task. This adaptive edge detection enhances the registration process by capturing diverse structural features critical in medical imaging. To provide clearer insight into the contribution of each component in our design, we introduce four variant models for rigid registration and four variant models for non-rigid registration. We evaluated our approach using a dataset provided by the Medical University across three setups: rigid registration without skull removal, with skull removal, and non-rigid registration. Additionally, we assessed performance on two publicly available datasets. Across all experiments, our method consistently outperformed state-of-the-art techniques, demonstrating its potential to improve multi-modal image alignment and anatomical structure analysis.

Method: Two primary approaches: 1) Using SAM2 to extract unique objects as masks from images, with a segmentation network generating and refining initial predictions in parallel. 2) Using predicted masks to extract unique feature vectors, feeding them into a simple network for classification, then combining classifications and masks for final segmentation.

Result: Experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.

Conclusion: SAM2 can be effectively extended to Video Semantic Segmentation, with its object boundary precision providing significant performance improvements despite the challenges of spatial accuracy, temporal consistency, and multi-object tracking in complex video scenes.

Abstract: The Segmentation Anything Model 2 (SAM2) has proven to be a powerful foundation model for promptable visual object segmentation in both images and videos, capable of storing object-aware memories and transferring them temporally through memory blocks. While SAM2 excels in video object segmentation by providing dense segmentation masks based on prompts, extending it to dense Video Semantic Segmentation (VSS) poses challenges due to the need for spatial accuracy, temporal consistency, and the ability to track multiple objects with complex boundaries and varying scales. This paper explores the extension of SAM2 for VSS, focusing on two primary approaches and highlighting firsthand observations and common challenges faced during this process. The first approach involves using SAM2 to extract unique objects as masks from a given image, with a segmentation network employed in parallel to generate and refine initial predictions. The second approach utilizes the predicted masks to extract unique feature vectors, which are then fed into a simple network for classification. The resulting classifications and masks are subsequently combined to produce the final segmentation. Our experiments suggest that leveraging SAM2 enhances overall performance in VSS, primarily due to its precise predictions of object boundaries.

Method: Compare traditional compression (JPEG 2000) versus learned compression (Discretized Mixed Gaussian Likelihood) on three EO segmentation tasks: fire, cloud, and building detection. Also test joint end-to-end optimization of compression and segmentation models.

Result: Learned compression significantly outperforms JPEG 2000 for large-scale, multi-channel optical imagery in both reconstruction quality (PSNR) and segmentation accuracy. Traditional codecs remain competitive on smaller, single-channel thermal infrared datasets. Joint optimization doesn’t improve performance over standalone optimization.

Conclusion: Learned compression is promising for large-scale EO data but context matters; traditional methods still have value for certain datasets, and joint optimization doesn’t provide additional benefits.

Abstract: The rapid growth of data from satellite-based Earth observation (EO) systems poses significant challenges in data transmission and storage. We evaluate the potential of task-specific learned compression algorithms in this context to reduce data volumes while retaining crucial information. In detail, we compare traditional compression (JPEG 2000) versus a learned compression approach (Discretized Mixed Gaussian Likelihood) on three EO segmentation tasks: Fire, cloud, and building detection. Learned compression notably outperforms JPEG 2000 for large-scale, multi-channel optical imagery in both reconstruction quality (PSNR) and segmentation accuracy. However, traditional codecs remain competitive on smaller, single-channel thermal infrared datasets due to limited data and architectural constraints. Additionally, joint end-to-end optimization of compression and segmentation models does not improve performance over standalone optimization.

Method: Three-component architecture: 1) SAM3 image encoder for feature extraction, 2) simple adapter for parameter-efficient fine-tuning, and 3) lightweight U-Net-style decoder for segmentation output.

Result: Outperforms prior SAM2-UNet and other state-of-the-art methods on multiple tasks (mirror detection, salient object detection) while requiring less than 6GB GPU memory during training with batch size 12.

Conclusion: SAM3-UNet provides an effective, resource-efficient solution for adapting large foundation models to downstream segmentation tasks, making advanced segmentation capabilities more accessible.

Abstract: In this paper, we introduce SAM3-UNet, a simplified variant of Segment Anything Model 3 (SAM3), designed to adapt SAM3 for downstream tasks at a low cost. Our SAM3-UNet consists of three components: a SAM3 image encoder, a simple adapter for parameter-efficient fine-tuning, and a lightweight U-Net-style decoder. Preliminary experiments on multiple tasks, such as mirror detection and salient object detection, demonstrate that the proposed SAM3-UNet outperforms the prior SAM2-UNet and other state-of-the-art methods, while requiring less than 6 GB of GPU memory during training with a batch size of 12. The code is publicly available at https://github.com/WZH0120/SAM3-UNet.

Method: Learns a latent space from real-world human actions by fusing appearance-agnostic skeletal geometry features with appearance-based features. Measures action quality by computing distance between generated video representations and this learned real-world action distribution.

Result: Achieves 68% improvement over state-of-the-art methods on their new benchmark, performs competitively on established external benchmarks, and shows stronger correlation with human perception.

Conclusion: The metric reveals critical limitations in current video generative models and establishes a new standard for advanced video generation research by providing robust evaluation of human action fidelity.

Abstract: Despite rapid advances in video generative models, robust metrics for evaluating visual and temporal correctness of complex human actions remain elusive. Critically, existing pure-vision encoders and Multimodal Large Language Models (MLLMs) are strongly appearance-biased, lack temporal understanding, and thus struggle to discern intricate motion dynamics and anatomical implausibilities in generated videos. We tackle this gap by introducing a novel evaluation metric derived from a learned latent space of real-world human actions. Our method first captures the nuances, constraints, and temporal smoothness of real-world motion by fusing appearance-agnostic human skeletal geometry features with appearance-based features. We posit that this combined feature space provides a robust representation of action plausibility. Given a generated video, our metric quantifies its action quality by measuring the distance between its underlying representations and this learned real-world action distribution. For rigorous validation, we develop a new multi-faceted benchmark specifically designed to probe temporally challenging aspects of human action fidelity. Through extensive experiments, we show that our metric achieves substantial improvement of more than 68% compared to existing state-of-the-art methods on our benchmark, performs competitively on established external benchmarks, and has a stronger correlation with human perception. Our in-depth analysis reveals critical limitations in current video generative models and establishes a new standard for advanced research in video generation.

Method: 1) MILO paradigm integrates visual generator for geometry-aware feedback to implicitly ground symbolic reasoning; 2) RePE encoding captures relative camera-pose transformations instead of absolute coordinates; 3) GeoGen dataset with 2,241 videos and 67,827 observation-action-outcome triplets for training.

Result: Experiments show significant enhancement of spatial reasoning capabilities across multiple baselines and benchmarks, offering more holistic understanding of 3D space.

Conclusion: MILO successfully bridges the gap between symbolic reasoning and visual perception in spatial understanding, providing a more human-like spatial imagination capability for MLLMs.

Abstract: Spatial reasoning, the ability to understand and interpret the 3D structure of the world, is a critical yet underdeveloped capability in Multimodal Large Language Models (MLLMs). Current methods predominantly rely on verbal descriptive tuning, which suffers from visual illiteracy, i.e., they learn spatial concepts through textual symbols alone, devoid of connection to their visual manifestations. To bridge this gap, this paper introduces MILO, an Implicit spatIaL wOrld modeling paradigm that simulates human-like spatial imagination. MILO integrates a visual generator to provide geometry-aware feedback, thereby implicitly grounding the MLLM’s symbolic reasoning in perceptual experience. Complementing this paradigm, we propose RePE (Relative Positional Encoding), a novel encoding scheme that captures relative camera-pose transformations, offering superior performance over absolute coordinate systems. To support the training, we construct GeoGen, a large-scale Geometry-aware Generative dataset with approximately 2,241 videos and 67,827 observation-action-outcome triplets. Experiments demonstrate that our approach significantly enhances spatial reasoning capabilities across multiple baselines and benchmarks, offering a more holistic understanding of 3D space.

Method: Three-component training approach: (1) data curation from VCG-32K dataset (32K+ images with causal graphs), (2) Tree-of-Causal-Thought for synthesizing reasoning trajectories, and (3) reinforcement learning with causal reward to refine reasoning policy.

Result: CauSight outperforms GPT-4.1 on visual causal discovery, achieving over a threefold performance boost with 21% absolute gain. The model, code, and dataset are fully open-sourced.

Conclusion: The work successfully introduces visual causal discovery as a new AI task, demonstrates that specialized models can significantly outperform general-purpose LLMs on causal reasoning in visual domains, and provides open resources to advance the field.

Abstract: Causal thinking enables humans to understand not just what is seen, but why it happens. To replicate this capability in modern AI systems, we introduce the task of visual causal discovery. It requires models to infer cause-and-effect relations among visual entities across diverse scenarios instead of merely perceiving their presence. To this end, we first construct the Visual Causal Graph dataset (VCG-32K), a large-scale collection of over 32,000 images annotated with entity-level causal graphs, and further develop CauSight, a novel vision-language model to perform visual causal discovery through causally aware reasoning. Our training recipe integrates three components: (1) training data curation from VCG-32K, (2) Tree-of-Causal-Thought (ToCT) for synthesizing reasoning trajectories, and (3) reinforcement learning with a designed causal reward to refine the reasoning policy. Experiments show that CauSight outperforms GPT-4.1 on visual causal discovery, achieving over a threefold performance boost (21% absolute gain). Our code, model, and dataset are fully open-sourced at project page: https://github.com/OpenCausaLab/CauSight.

Method: Proposes OpenREAD framework with: 1) Large-scale Chain-of-Thought annotations on driving datasets, 2) Using Qwen3 LLM as critic in RFT to quantify reasoning quality for open-ended questions, 3) End-to-end RFT across both high-level reasoning and low-level trajectory planning.

Result: Extensive experiments show joint end-to-end RFT yields substantial improvements in both upstream and downstream tasks, achieving state-of-the-art performance on reasoning and planning benchmarks.

Conclusion: OpenREAD successfully addresses limitations of current approaches by enabling comprehensive reinforcement fine-tuning across the full spectrum of autonomous driving tasks, demonstrating the effectiveness of LLM-based reward modeling for open-ended reasoning problems.

Abstract: Recently, two-stage fine-tuning strategies, e.g., acquiring essential driving knowledge through supervised fine-tuning (SFT) and further enhancing decision-making and planning via reinforcement fine-tuning (RFT), have shown strong potential in advancing the knowledge-driven autonomous driving (AD) paradigm. However, the learning nature of SFT still limits the generalization of reasoning, thereby constraining the full potential of driving performance. Meanwhile, current RFT approaches are primarily applied to downstream tasks, since scene understanding is an open-ended problem where corresponding rewards are difficult to quantify. To address these limitations, we propose OpenREAD, an OPEN-ended REasoning reinforced vision-language model (VLM)-based autonomous driving (AD) framework that enables end-to-end RFT across the full spectrum from high-level reasoning to low-level trajectory planning. Specifically, we begin by constructing large-scale Chain-of-Thought (CoT) annotations on open-source driving-related knowledge datasets, and employ the powerful Qwen3 large language model (LLM) as the critic in RFT to quantify reasoning quality for open-ended questions during reward modeling. Extensive experiments confirm that joint end-to-end RFT yields substantial improvements in both upstream and downstream tasks, enabling OpenREAD to achieve state-of-the-art performance on reasoning and planning benchmarks.

Method: Constructed a PID dataset with test/train splits of annotated videos where implausible videos are created by rewriting captions to induce physical violations. Fine-tuned VLMs to detect implausibility and generate explanations (PhyDetEx).

Result: PhyDetEx effectively detects physical implausibility and explains violated principles. Benchmarking shows T2V models still struggle with physical laws, especially open-source models, despite recent progress.

Conclusion: Understanding and adhering to physical laws remains challenging for T2V models. The proposed PhyDetEx provides a valuable tool for evaluating physical plausibility, with dataset and code publicly available.

Abstract: Driven by the growing capacity and training scale, Text-to-Video (T2V) generation models have recently achieved substantial progress in video quality, length, and instruction-following capability. However, whether these models can understand physics and generate physically plausible videos remains a question. While Vision-Language Models (VLMs) have been widely used as general-purpose evaluators in various applications, they struggle to identify the physically impossible content from generated videos. To investigate this issue, we construct a \textbf{PID} (\textbf{P}hysical \textbf{I}mplausibility \textbf{D}etection) dataset, which consists of a \textit{test split} of 500 manually annotated videos and a \textit{train split} of 2,588 paired videos, where each implausible video is generated by carefully rewriting the caption of its corresponding real-world video to induce T2V models producing physically implausible content. With the constructed dataset, we introduce a lightweight fine-tuning approach, enabling VLMs to not only detect physically implausible events but also generate textual explanations on the violated physical principles. Taking the fine-tuned VLM as a physical plausibility detector and explainer, namely \textbf{PhyDetEx}, we benchmark a series of state-of-the-art T2V models to assess their adherence to physical laws. Our findings show that although recent T2V models have made notable progress toward generating physically plausible content, understanding and adhering to physical laws remains a challenging issue, especially for open-source models. Our dataset, training code, and checkpoints are available at \href{https://github.com/Zeqing-Wang/PhyDetEx}{https://github.com/Zeqing-Wang/PhyDetEx}.

Method: Proposes a conditional generation approach where a learned continuous, point-wise velocity field transports noisy points to a registered scene. Uses a lightweight local feature extractor and test-time rigidity enforcement. Unlike previous methods that do correspondence matching and pairwise optimization, this directly generates the registered point cloud.

Result: Achieves state-of-the-art results on pairwise and multi-view registration benchmarks, particularly with low overlap. Generalizes across scales and sensor modalities. Supports downstream tasks including relocalization, multi-robot SLAM, and multi-session map merging.

Conclusion: The conditional generation approach to point cloud registration is effective, achieving superior performance on challenging benchmarks and demonstrating practical utility for various 3D vision and robotics applications.

Abstract: Point cloud registration aligns multiple unposed point clouds into a common frame, and is a core step for 3D reconstruction and robot localization. In this work, we cast registration as conditional generation: a learned continuous, point-wise velocity field transports noisy points to a registered scene, from which the pose of each view is recovered. Unlike previous methods that conduct correspondence matching to estimate the transformation between a pair of point clouds and then optimize the pairwise transformations to realize multi-view registration, our model directly generates the registered point cloud. With a lightweight local feature extractor and test-time rigidity enforcement, our approach achieves state-of-the-art results on pairwise and multi-view registration benchmarks, particularly with low overlap, and generalizes across scales and sensor modalities. It further supports downstream tasks including relocalization, multi-robot SLAM, and multi-session map merging. Source code available at: https://github.com/PRBonn/RAP.

Method: Proposes a reconfigurable Multi-Agent System where each agent specializes in specific analysis aspects; uses iterative invocation and flexible composition of agents to construct adaptive pipelines for different temporal scales and tasks.

Result: Demonstrated adaptability using two representative tasks in badminton analysis, showing ability to bridge fine-grained event detection and global semantic organization.

Conclusion: Presents a paradigm shift toward flexible, scalable, and interpretable systems for robust, cross-task sports video intelligence.

Abstract: Intelligent sports video analysis demands a comprehensive understanding of temporal context, from micro-level actions to macro-level game strategies. Existing end-to-end models often struggle with this temporal hierarchy, offering solutions that lack generalization, incur high development costs for new tasks, and suffer from poor interpretability. To overcome these limitations, we propose a reconfigurable Multi-Agent System (MAS) as a foundational framework for sports video understanding. In our system, each agent functions as a distinct “cognitive tool” specializing in a specific aspect of analysis. The system’s architecture is not confined to a single temporal dimension or task. By leveraging iterative invocation and flexible composition of these agents, our framework can construct adaptive pipelines for both short-term analytic reasoning (e.g., Rally QA) and long-term generative summarization (e.g., match summaries). We demonstrate the adaptability of this framework using two representative tasks in badminton analysis, showcasing its ability to bridge fine-grained event detection and global semantic organization. This work presents a paradigm shift towards a flexible, scalable, and interpretable system for robust, cross-task sports video intelligence.The project homepage is available at https://aiden1020.github.io/COACH-project-page

Method: A lightweight deep learning framework that performs matching on cell detection embeddings without tracking-specific feature quantification, integrates Transformer Networks, uses multi-stage matching with all detection boxes, and interpolates missing tracklets with Kalman Filter.

Result: The framework achieves strong performance across diverse conditions, effectively tracks cells while capturing division and death events, and demonstrates practical utility in analyzing chemotherapeutic drug efficacy at single-cell level.

Conclusion: TransientTrack advances quantitative cancer cell dynamics studies by enabling detailed characterization of treatment response and resistance mechanisms, with code publicly available for broader use.

Abstract: Tracking cells in time-lapse videos is an essential technique for monitoring cell population dynamics at a single-cell level. Current methods for cell tracking are developed on videos with mostly single, constant signals and do not detect pivotal events such as cell death. Here, we present TransientTrack, a deep learning-based framework for cell tracking in multi-channel microscopy video data with transient fluorescent signals that fluctuate over time following processes such as the circadian rhythm of cells. By identifying key cellular events - mitosis (cell division) and apoptosis (cell death) our method allows us to build complete trajectories, including cell lineage information. TransientTrack is lightweight and performs matching on cell detection embeddings directly, without the need for quantification of tracking-specific cell features. Furthermore, our approach integrates Transformer Networks, multi-stage matching using all detection boxes, and the interpolation of missing tracklets with the Kalman Filter. This unified framework achieves strong performance across diverse conditions, effectively tracking cells and capturing cell division and death. We demonstrate the use of TransientTrack in an analysis of the efficacy of a chemotherapeutic drug at a single-cell level. The proposed framework could further advance quantitative studies of cancer cell dynamics, enabling detailed characterization of treatment response and resistance mechanisms. The code is available at https://github.com/bozeklab/TransientTrack.

Method: KM-ViPE tightly couples DINO visual features with geometric constraints using a high-level features based adaptive robust kernel. It performs simultaneous online localization and open-vocabulary semantic mapping by fusing geometric and deep visual features aligned with language embeddings, operating directly on raw RGB streams without depth sensors or calibration.

Result: The system achieves competitive results with state-of-the-art approaches while operating online with uncalibrated monocular input. It uniquely combines online operation, uncalibrated monocular input, and robust handling of dynamic scenes, outperforming solutions that require offline processing, depth data, or odometry estimation.

Conclusion: KM-ViPE advances practical spatial intelligence for embodied AI, making it suitable for autonomous robotics and AR/VR applications. Its ability to operate in real-time with uncalibrated monocular cameras while handling dynamic scenes represents significant progress toward practical spatial understanding systems.

Abstract: We present KM-ViPE (Knowledge Mapping Video Pose Engine), a real-time open-vocabulary SLAM framework for uncalibrated monocular cameras in dynamic environments. Unlike systems requiring depth sensors and offline calibration, KM-ViPE operates directly on raw RGB streams, making it ideal for ego-centric applications and harvesting internet-scale video data for training. KM-ViPE tightly couples DINO visual features with geometric constraints through a high-level features based adaptive robust kernel that handles both moving objects and movable static objects (e.g., moving furniture in ego-centric views). The system performs simultaneous online localization and open-vocabulary semantic mapping by fusing geometric and deep visual features aligned with language embeddings. Our results are competitive with state-of-the-art approaches, while existing solutions either operate offline, need depth data and/or odometry estimation, or lack dynamic scene robustness. KM-ViPE benefits from internet-scale training and uniquely combines online operation, uncalibrated monocular input, and robust handling of dynamic scenes, which makes it a good fit for autonomous robotics and AR/VR applications and advances practical spatial intelligence capabilities for embodied AI.

Method: Proposes an efficient data augmentation strategy alongside a dual-encoder framework to extract domain-invariant identity cues and capture domain-specific stylistic variations, then conditions a diffusion model to retarget facial expressions across domains.

Result: Extensive experiments demonstrate that StyleYourSmile achieves superior identity preservation and retargeting fidelity across a wide range of visual domains.

Conclusion: StyleYourSmile enables effective cross-domain face retargeting with disentangled control over identity, expressions, and domain-specific styles without requiring curated multi-style paired data.

Abstract: Cross-domain face retargeting requires disentangled control over identity, expressions, and domain-specific stylistic attributes. Existing methods, typically trained on real-world faces, either fail to generalize across domains, need test-time optimizations, or require fine-tuning with carefully curated multi-style datasets to achieve domain-invariant identity representations. In this work, we introduce \textit{StyleYourSmile}, a novel one-shot cross-domain face retargeting method that eliminates the need for curated multi-style paired data. We propose an efficient data augmentation strategy alongside a dual-encoder framework, for extracting domain-invariant identity cues and capturing domain-specific stylistic variations. Leveraging these disentangled control signals, we condition a diffusion model to retarget facial expressions across domains. Extensive experiments demonstrate that \textit{StyleYourSmile} achieves superior identity preservation and retargeting fidelity across a wide range of visual domains.

Method: SARL augments BYOL architecture with three map-level objectives: Saliency Alignment (SAL) to maintain attentional focus, Patch-Prototype Distribution Alignment (PPDA) to preserve part composition, and Region Affinity Matching (RAM) to keep geometric relations consistent across views. These losses act on intermediate feature maps alongside the global objective.

Result: SARL consistently outperforms nine SSL baselines across six downstream tasks. On edge-pose regression (geometry-sensitive task), SARL achieves MAE of 0.3955, a 30% relative improvement over next-best SSL method (0.5682 MAE), approaching supervised upper bound performance.

Conclusion: For fused visual-tactile data, structured spatial equivariance (features varying predictably with object geometry) is the most effective signal, enabling more capable robotic perception. Preserving spatial structure in SSL frameworks is crucial for manipulation tasks.

Abstract: Contact-rich robotic manipulation requires representations that encode local geometry. Vision provides global context but lacks direct measurements of properties such as texture and hardness, whereas touch supplies these cues. Modern visuo-tactile sensors capture both modalities in a single fused image, yielding intrinsically aligned inputs that are well suited to manipulation tasks requiring visual and tactile information. Most self-supervised learning (SSL) frameworks, however, compress feature maps into a global vector, discarding spatial structure and misaligning with the needs of manipulation. To address this, we propose SARL, a spatially-aware SSL framework that augments the Bootstrap Your Own Latent (BYOL) architecture with three map-level objectives, including Saliency Alignment (SAL), Patch-Prototype Distribution Alignment (PPDA), and Region Affinity Matching (RAM), to keep attentional focus, part composition, and geometric relations consistent across views. These losses act on intermediate feature maps, complementing the global objective. SARL consistently outperforms nine SSL baselines across six downstream tasks with fused visual-tactile data. On the geometry-sensitive edge-pose regression task, SARL achieves a Mean Absolute Error (MAE) of 0.3955, a 30% relative improvement over the next-best SSL method (0.5682 MAE) and approaching the supervised upper bound. These findings indicate that, for fused visual-tactile data, the most effective signal is structured spatial equivariance, in which features vary predictably with object geometry, which enables more capable robotic perception.

Method: Med-VCD uses sparse visual-contrastive decoding with a novel token-sparsification strategy that selects visually informed tokens on the fly. This approach trims redundancy while retaining critical visual context, balancing efficiency with reliability without secondary decoding overhead.

Result: Evaluations across eight medical datasets spanning ophthalmology, radiology, and pathology tasks (visual question answering, report generation, hallucination benchmarks) show Med-VCD improves factual accuracy by average 13% and hallucination accuracy by 6% relative to baseline medical LVLMs.

Conclusion: Med-VCD effectively mitigates hallucinations in medical LVLMs while maintaining inference efficiency, making it suitable for healthcare applications where both accuracy and speed are critical.

Abstract: Large vision-language models (LVLMs) are now central to healthcare applications such as medical visual question answering and imaging report generation. Yet, these models remain vulnerable to hallucination outputs that appear plausible but are in fact incorrect. In the natural image domain, several decoding strategies have been proposed to mitigate hallucinations by reinforcing visual evidence, but most rely on secondary decoding or rollback procedures that substantially slow inference. Moreover, existing solutions are often domain-specific and may introduce misalignment between modalities or between generated and ground-truth content. We introduce Med-VCD, a sparse visual-contrastive decoding method that mitigates hallucinations in medical LVLMs without the time overhead of secondary decoding. Med-VCD incorporates a novel token-sparsification strategy that selects visually informed tokens on the fly, trimming redundancy while retaining critical visual context and thus balancing efficiency with reliability. Evaluations on eight medical datasets, spanning ophthalmology, radiology, and pathology tasks in visual question answering, report generation, and dedicated hallucination benchmarks, show that Med-VCD raises factual accuracy by an average of 13% and improves hallucination accuracy by 6% relative to baseline medical LVLMs.

Method: AdvTraj uses adversarial trajectories to transfer attacker’s ID to targeted objects, confusing tracking systems without attacking object detection. The attack works in online and physical settings, with simulation in CARLA. The paper characterizes trajectory patterns and proposes two universal adversarial maneuvers that can be performed by humans in daily scenarios.

Result: White-box attacks against SORT achieve 100% success rate in various scenarios. High transferability (up to 93% success rate) against SOTA MOT algorithms due to common design principles. Identifies patterns in adversarial trajectories and proposes practical adversarial maneuvers.

Conclusion: Reveals under-explored weaknesses in object association phase of SOTA MOT systems. Provides insights for enhancing robustness of tracking systems against such ID-manipulation attacks.

Abstract: Multi-Object Tracking (MOT) is a critical task in computer vision, with applications ranging from surveillance systems to autonomous driving. However, threats to MOT algorithms have yet been widely studied. In particular, incorrect association between the tracked objects and their assigned IDs can lead to severe consequences, such as wrong trajectory predictions. Previous attacks against MOT either focused on hijacking the trackers of individual objects, or manipulating the tracker IDs in MOT by attacking the integrated object detection (OD) module in the digital domain, which are model-specific, non-robust, and only able to affect specific samples in offline datasets. In this paper, we present AdvTraj, the first online and physical ID-manipulation attack against tracking-by-detection MOT, in which an attacker uses adversarial trajectories to transfer its ID to a targeted object to confuse the tracking system, without attacking OD. Our simulation results in CARLA show that AdvTraj can fool ID assignments with 100% success rate in various scenarios for white-box attacks against SORT, which also have high attack transferability (up to 93% attack success rate) against state-of-the-art (SOTA) MOT algorithms due to their common design principles. We characterize the patterns of trajectories generated by AdvTraj and propose two universal adversarial maneuvers that can be performed by a human walker/driver in daily scenarios. Our work reveals under-explored weaknesses in the object association phase of SOTA MOT systems, and provides insights into enhancing the robustness of such systems.

Method: Script uses two modules: 1) graph-structured pruning module that removes visually redundant tokens, and 2) query-conditioned semantic pruning module that preserves query-relevant visual information. It’s plug-and-play, requires no retraining, and generalizes across diverse MLLMs.

Result: Experiments on 14 benchmarks across image and video understanding tasks show Script consistently achieves higher efficiency and accuracy than existing pruning methods. On LLaVA-NeXT-7B, it achieves up to 6.8x prefill speedup, 10x FLOP reduction, while retaining 96.88% of original performance.

Conclusion: Script effectively addresses the computational challenges of MLLMs by combining visual redundancy removal with query-aware semantic preservation, offering a practical solution for efficient multimodal inference without sacrificing performance.

Abstract: The rapid growth of visual tokens in multimodal large language models (MLLMs) leads to excessive memory consumption and inference latency, especially when handling high-resolution images and videos. Token pruning is a technique used to mitigate this issue by removing redundancy, but existing methods often ignore relevance to the user query or suffer from the limitations of attention mechanisms, reducing their adaptability and effectiveness. To address these challenges, we propose Script, a plug-and-play pruning method that requires no retraining and generalizes across diverse MLLMs. Script comprises two modules: a graph-structured pruning module that removes visually redundant tokens, and a query-conditioned semantic pruning module that preserves query-relevant visual information. Together, they enhance performance on multimodal tasks. Experiments on fourteen benchmarks across image and video understanding tasks show that Script consistently achieves higher model efficiency and predictive accuracy compared to existing pruning methods. On LLaVA-NeXT-7B, it achieves up to 6.8x prefill speedup and 10x FLOP reduction, while retaining 96.88% of the original performance.

Method: Propose Mixture-of-Head attention (MoH) that: 1) Expresses multi-head attention in summation form, 2) Treats attention heads as experts in MoE framework, 3) Enables tokens to select appropriate heads via routing, 4) Replaces standard summation with weighted summation.

Result: MoH outperforms multi-head attention using only 50%-90% of attention heads across ViT, DiT, and LLMs. MoH-LLaMA3-8B achieves 64.0% average accuracy across 14 benchmarks (2.4% improvement) using only 75% of attention heads. Pre-trained models can be fine-tuned into MoH.

Conclusion: MoH is a promising alternative to multi-head attention that improves efficiency without compromising accuracy, providing flexibility through token-specific head selection and weighted summation, enabling more advanced and efficient attention-based models.

Abstract: In this work, we upgrade the multi-head attention mechanism, the core of the Transformer model, to improve efficiency while maintaining or surpassing the previous accuracy level. We show that multi-head attention can be expressed in the summation form. Drawing on the insight that not all attention heads hold equal significance, we propose Mixture-of-Head attention (MoH), a new architecture that treats attention heads as experts in the Mixture-of-Experts (MoE) mechanism. MoH has two significant advantages: First, MoH enables each token to select the appropriate attention heads, enhancing inference efficiency without compromising accuracy or increasing the number of parameters. Second, MoH replaces the standard summation in multi-head attention with a weighted summation, introducing flexibility to the attention mechanism and unlocking extra performance potential. Extensive experiments on ViT, DiT, and LLMs demonstrate that MoH outperforms multi-head attention by using only 50%-90% of the attention heads. Moreover, we demonstrate that pre-trained multi-head attention models, such as LLaMA3-8B, can be further continue-tuned into our MoH models. Notably, MoH-LLaMA3-8B achieves an average accuracy of 64.0% across 14 benchmarks, outperforming LLaMA3-8B by 2.4% by utilizing only 75% of the attention heads. We believe the proposed MoH is a promising alternative to multi-head attention and provides a strong foundation for developing advanced and efficient attention-based models.

Method: SpriteHand uses an autoregressive video generation framework with causal inference architecture for real-time synthesis. It takes a static object image and hand video stream as input, generates hand-object interaction effects, and employs hybrid post-training to enhance visual realism and temporal coherence.

Result: The 1.3B model achieves real-time streaming generation at ~18 FPS and 640x368 resolution with ~150 ms latency on a single NVIDIA RTX 5090 GPU, supporting over a minute of continuous output. It demonstrates superior visual quality, physical plausibility, and interaction fidelity compared to generative and engine-based baselines.

Conclusion: SpriteHand provides an effective solution for synthesizing versatile hand-object interactions across diverse object types, overcoming limitations of conventional simulation approaches and enabling real-time generation of physically plausible interactions.

Abstract: Modeling and synthesizing complex hand-object interactions remains a significant challenge, even for state-of-the-art physics engines. Conventional simulation-based approaches rely on explicitly defined rigid object models and pre-scripted hand gestures, making them inadequate for capturing dynamic interactions with non-rigid or articulated entities such as deformable fabrics, elastic materials, hinge-based structures, furry surfaces, or even living creatures. In this paper, we present SpriteHand, an autoregressive video generation framework for real-time synthesis of versatile hand-object interaction videos across a wide range of object types and motion patterns. SpriteHand takes as input a static object image and a video stream in which the hands are imagined to interact with the virtual object embedded in a real-world scene, and generates corresponding hand-object interaction effects in real time. Our model employs a causal inference architecture for autoregressive generation and leverages a hybrid post-training approach to enhance visual realism and temporal coherence. Our 1.3B model supports real-time streaming generation at around 18 FPS and 640x368 resolution, with an approximate 150 ms latency on a single NVIDIA RTX 5090 GPU, and more than a minute of continuous output. Experiments demonstrate superior visual quality, physical plausibility, and interaction fidelity compared to both generative and engine-based baselines.

Method: Proposes Scene Graph Guided Diffusion Model (SGDiff) with: 1) Prompt-Centric Scene Graph Adaptor to map user prompts to scene graphs capturing intention; 2) Diffusion process with Scene Graph Guided Bimodal Transformer to predict correlated caption-mask pairs; 3) Multi-Entities Contrastive Learning loss for explicit visual-textual alignment.

Result: Extensive experiments on two datasets show SGDiff achieves superior performance in SegCaptioning, producing promising results for both captioning and segmentation tasks with minimal prompt input.

Conclusion: SegCaptioning enables flexible semantic interpretation from simple prompts, and SGDiff effectively addresses the challenges of capturing user intention while generating correlated, well-aligned caption-mask pairs through structured scene graph guidance.

Abstract: Controllable image semantic understanding tasks, such as captioning or segmentation, necessitate users to input a prompt (e.g., text or bounding boxes) to predict a unique outcome, presenting challenges such as high-cost prompt input or limited information output. This paper introduces a new task ``Image Collaborative Segmentation and Captioning’’ (SegCaptioning), which aims to translate a straightforward prompt, like a bounding box around an object, into diverse semantic interpretations represented by (caption, masks) pairs, allowing flexible result selection by users. This task poses significant challenges, including accurately capturing a user’s intention from a minimal prompt while simultaneously predicting multiple semantically aligned caption words and masks. Technically, we propose a novel Scene Graph Guided Diffusion Model that leverages structured scene graph features for correlated mask-caption prediction. Initially, we introduce a Prompt-Centric Scene Graph Adaptor to map a user’s prompt to a scene graph, effectively capturing his intention. Subsequently, we employ a diffusion process incorporating a Scene Graph Guided Bimodal Transformer to predict correlated caption-mask pairs by uncovering intricate correlations between them. To ensure accurate alignment, we design a Multi-Entities Contrastive Learning loss to explicitly align visual and textual entities by considering inter-modal similarity, resulting in well-aligned caption-mask pairs. Extensive experiments conducted on two datasets demonstrate that SGDiff achieves superior performance in SegCaptioning, yielding promising results for both captioning and segmentation tasks with minimal prompt input.

Method: 1) Lift real 2D images into 3D Gaussian Splats with varying appearance and deblurring ability to synthesize diverse training data with pose AND appearance variations. 2) Build a two-branch joint training pipeline with an adversarial discriminator to bridge the synthetic-to-real gap and fully leverage appearance-diverse data.

Result: Significantly outperforms state-of-the-art methods: reduces translation errors by 50% and rotation errors by 41% on indoor datasets, and 38% and 44% on outdoor datasets. Shows remarkable robustness in dynamic driving scenarios under varying weather conditions and day-to-night scenarios where previous APR methods fail.

Conclusion: Appearance variation is crucial for robust visual localization. The proposed approach effectively addresses generalization limitations of APR methods by incorporating appearance diversity through 3D Gaussian Splat synthesis and adversarial training, achieving superior performance across diverse real-world conditions.

Abstract: Visual localization, which estimates a camera’s pose within a known scene, is a fundamental capability for autonomous systems. While absolute pose regression (APR) methods have shown promise for efficient inference, they often struggle with generalization. Recent approaches attempt to address this through data augmentation with varied viewpoints, yet they overlook a critical factor: appearance diversity. In this work, we identify appearance variation as the key to robust localization. Specifically, we first lift real 2D images into 3D Gaussian Splats with varying appearance and deblurring ability, enabling the synthesis of diverse training data that varies not just in poses but also in environmental conditions such as lighting and weather. To fully unleash the potential of the appearance-diverse data, we build a two-branch joint training pipeline with an adversarial discriminator to bridge the syn-to-real gap. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods, reducing translation and rotation errors by 50% and 41% on indoor datasets, and 38% and 44% on outdoor datasets. Most notably, our method shows remarkable robustness in dynamic driving scenarios under varying weather conditions and in day-to-night scenarios, where previous APR methods fail. Project Page: https://ai4ce.github.io/RAP/

Method: Artemis performs structured proposal-based reasoning where each intermediate step is represented as a (label, bounding-box) pair capturing verifiable visual states, enabling explicit state tracking, direct supervision for proposal quality, and avoiding language ambiguity.

Result: Artemis, built on Qwen2.5-VL-3B, achieves strong performance on grounding and detection tasks, shows substantial generalization to counting and geometric-perception tasks, and achieves competitive performance on general MLLM benchmarks.

Conclusion: Aligning reasoning with spatial representations enhances perception-policy learning, and spatially grounded reasoning provides a principled route toward scalable and general perception policies.

Abstract: Recent reinforcement-learning frameworks for visual perception policy have begun to incorporate intermediate reasoning chains expressed in natural language. Empirical observations indicate that such purely linguistic intermediate reasoning often reduces performance on perception tasks. We argue that the core issue lies not in reasoning per se but in the form of reasoning: while these chains perform semantic reasoning in an unstructured linguistic space, visual perception requires reasoning in a spatial and object-centric space. In response, we introduce Artemis, a perception-policy learning framework that performs structured proposal-based reasoning, where each intermediate step is represented as a (label, bounding-box) pair capturing a verifiable visual state. This design enables explicit tracking of intermediate states, direct supervision for proposal quality, and avoids ambiguity introduced by language-based reasoning. Artemis is built on Qwen2.5-VL-3B, achieves strong performance on grounding and detection task and exhibits substantial generalization to counting and geometric-perception tasks. The consistent improvements across these diverse settings confirm that aligning reasoning with spatial representations enhances perception-policy learning. Owing to its strengthened visual reasoning, Artemis also achieves competitive performance on general MLLM benchmarks, illustrating that spatially grounded reasoning provides a principled route toward scalable and general perception policies.

Method: LADiBI uses large-scale text-to-image diffusion models within a Bayesian framework. It employs text prompts to encode priors for both target images and degradation operators. The method features a novel diffusion posterior sampling algorithm that combines strategic operator initialization with iterative refinement of both image and operator parameters.

Result: Experiments demonstrate that LADiBI effectively handles both linear and challenging nonlinear image restoration problems across various image distributions, all without task-specific assumptions or retraining.

Conclusion: LADiBI represents a significant advancement in blind inverse image restoration by eliminating restrictive assumptions, offering unprecedented flexibility through text-guided priors, and working without training or specific operator constraints.

Abstract: This paper considers blind inverse image restoration, the task of predicting a target image from a degraded source when the degradation (i.e. the forward operator) is unknown. Existing solutions typically rely on restrictive assumptions such as operator linearity, curated training data or narrow image distributions limiting their practicality. We introduce LADiBI, a training-free method leveraging large-scale text-to-image diffusion to solve diverse blind inverse problems with minimal assumptions. Within a Bayesian framework, LADiBI uses text prompts to jointly encode priors for both target images and operators, unlocking unprecedented flexibility compared to existing methods. Additionally, we propose a novel diffusion posterior sampling algorithm that combines strategic operator initialization with iterative refinement of image and operator parameters, eliminating the need for highly constrained operator forms. Experiments show that LADiBI effectively handles both linear and challenging nonlinear image restoration problems across various image distributions, all without task-specific assumptions or retraining.

Method: Introduces PAI-Bench, a unified benchmark evaluating perception and prediction capabilities across three areas: video generation, conditional video generation, and video understanding, using 2,808 real-world cases with task-aligned metrics focused on physical plausibility and domain-specific reasoning.

Result: Video generative models show strong visual fidelity but struggle with physically coherent dynamics, while multi-modal large language models have limited performance in forecasting and causal interpretation, indicating current systems are at early stages for Physical AI demands.

Conclusion: PAI-Bench establishes a realistic foundation for evaluating Physical AI and highlights key gaps in physical coherence and causal reasoning that future systems must address to advance physical intelligence capabilities.

Abstract: Physical AI aims to develop models that can perceive and predict real-world dynamics; yet, the extent to which current multi-modal large language models and video generative models support these abilities is insufficiently understood. We introduce Physical AI Bench (PAI-Bench), a unified and comprehensive benchmark that evaluates perception and prediction capabilities across video generation, conditional video generation, and video understanding, comprising 2,808 real-world cases with task-aligned metrics designed to capture physical plausibility and domain-specific reasoning. Our study provides a systematic assessment of recent models and shows that video generative models, despite strong visual fidelity, often struggle to maintain physically coherent dynamics, while multi-modal large language models exhibit limited performance in forecasting and causal interpretation. These observations suggest that current systems are still at an early stage in handling the perceptual and predictive demands of Physical AI. In summary, PAI-Bench establishes a realistic foundation for evaluating Physical AI and highlights key gaps that future systems must address.

Method: PRIMA integrates pixel-level grounding with multi-image reasoning using SQuARE vision module, which injects cross-image relational context into compact query-based visual tokens before fusing with language backbone. Trained on M4SEG benchmark.

Result: PRIMA outperforms state-of-the-art baselines with 7.83% improvement in Recall and 11.25% improvement in S-IoU. Ablation studies confirm SQuARE’s effectiveness in capturing cross-image relationships.

Conclusion: PRIMA successfully bridges the gap between pixel-level grounding and multi-image reasoning, enabling detailed, fine-grained comparisons across multiple images with pixel-grounded explanations.

Abstract: Despite significant advancements in Large Vision-Language Models (LVLMs)’ capabilities, existing pixel-grounding models operate in single-image settings, limiting their ability to perform detailed, fine-grained comparisons across multiple images. Conversely, current multi-image understanding models lack pixel-level grounding. Our work addresses this gap by introducing the task of multi-image pixel-grounded reasoning alongside PRIMA, an LVLM that integrates pixel-level grounding with robust multi-image reasoning to produce contextually rich, pixel-grounded explanations. Central to PRIMA is SQuARE, a vision module that injects cross-image relational context into compact query-based visual tokens before fusing them with the language backbone. To support training and evaluation, we curate M4SEG, a new multi-image reasoning segmentation benchmark consisting of $\sim$744K question-answer pairs that require fine-grained visual understanding across multiple images. PRIMA outperforms state-of-the-art baselines with $7.83%$ and $11.25%$ improvements in Recall and S-IoU, respectively. Ablation studies further demonstrate the effectiveness of the proposed SQuARE module in capturing cross-image relationships.

Method: Propose AVAGFormer with semantic-conditioned cross-modal mixer and dual-head decoder to effectively fuse audio and visual signals for mask prediction. Construct first AV-AG dataset with action sounds, object images, and pixel-level affordance annotations including unseen subset for zero-shot evaluation.

Result: AVAGFormer achieves state-of-the-art performance on AV-AG, surpassing baselines from related tasks. Comprehensive analyses show distinctions between AV-AG and AVS, benefits of end-to-end modeling, and contributions of each component.

Conclusion: Audio provides valuable cues for affordance grounding, and the proposed AVAGFormer effectively leverages audio-visual fusion for interaction region segmentation. The released dataset and code support further research in this direction.

Abstract: We introduce Audio-Visual Affordance Grounding (AV-AG), a new task that segments object interaction regions from action sounds. Unlike existing approaches that rely on textual instructions or demonstration videos, which often limited by ambiguity or occlusion, audio provides real-time, semantically rich, and visually independent cues for affordance grounding, enabling more intuitive understanding of interaction regions. To support this task, we construct the first AV-AG dataset, comprising a large collection of action sounds, object images, and pixel-level affordance annotations. The dataset also includes an unseen subset to evaluate zero-shot generalization. Furthermore, we propose AVAGFormer, a model equipped with a semantic-conditioned cross-modal mixer and a dual-head decoder that effectively fuses audio and visual signals for mask prediction. Experiments show that AVAGFormer achieves state-of-the-art performance on AV-AG, surpassing baselines from related tasks. Comprehensive analyses highlight the distinctions between AV-AG and AVS, the benefits of end-to-end modeling, and the contribution of each component. Code and dataset have been released on https://jscslld.github.io/AVAGFormer/.

Method: MV-TAP uses camera geometry and a cross-view attention mechanism to aggregate spatio-temporal information across different views. The approach is supported by a large-scale synthetic training dataset and real-world evaluation sets specifically designed for multi-view tracking.

Result: Extensive experiments show that MV-TAP outperforms existing point-tracking methods on challenging benchmarks, establishing an effective baseline for advancing research in multi-view point tracking.

Conclusion: MV-TAP provides a novel and effective solution for point tracking in multi-view videos, demonstrating the value of leveraging cross-view information through attention mechanisms and geometric constraints for improved trajectory estimation in dynamic scenes.

Abstract: Multi-view camera systems enable rich observations of complex real-world scenes, and understanding dynamic objects in multi-view settings has become central to various applications. In this work, we present MV-TAP, a novel point tracker that tracks points across multi-view videos of dynamic scenes by leveraging cross-view information. MV-TAP utilizes camera geometry and a cross-view attention mechanism to aggregate spatio-temporal information across views, enabling more complete and reliable trajectory estimation in multi-view videos. To support this task, we construct a large-scale synthetic training dataset and real-world evaluation sets tailored for multi-view tracking. Extensive experiments demonstrate that MV-TAP outperforms existing point-tracking methods on challenging benchmarks, establishing an effective baseline for advancing research in multi-view point tracking.

Method: Proposes a Shapley value-based algorithm to assess modality contributions and detect imbalance, plus a modality learning regulation strategy that slows early-stage training to balance learning across modalities.

Result: Extensive experiments on MM-Fi dataset show superior performance in enhancing 3D pose estimation under complex conditions compared to existing methods.

Conclusion: The balanced multi-modal learning approach effectively addresses modality imbalance and improves 3D human pose estimation performance across multiple sensing modalities.

Abstract: 3D human pose estimation (3D HPE) has emerged as a prominent research topic, particularly in the realm of RGB-based methods. However, the use of RGB images is often limited by issues such as occlusion and privacy constraints. Consequently, multi-modal sensing, which leverages non-intrusive sensors, is gaining increasing attention. Nevertheless, multi-modal 3D HPE still faces challenges, including modality imbalance. In this work, we introduce a novel balanced multi-modal learning method for 3D HPE, which harnesses the power of RGB, LiDAR, mmWave, and WiFi. Specifically, we propose a Shapley value-based contribution algorithm to assess the contribution of each modality and detect modality imbalance. To address this imbalance, we design a modality learning regulation strategy that decelerates the learning process during the early stages of training. We conduct extensive experiments on the widely adopted multi-modal dataset, MM-Fi, demonstrating the superiority of our approach in enhancing 3D pose estimation under complex conditions. We will release our codes soon.

Method: 1) Render-aligned data and labeling paradigm for pixel-level geometric, semantic, and entity-level understanding; 2) Interactive pedestrian-aware system for modeling human behavior; 3) Automated trajectory generation paradigm to support navigation tasks.

Result: Collected more than 60K panoramic samples and conducted extensive experiments across various tasks demonstrating the simulator’s effectiveness. The platform is the first to systematically model the 4D real world under omnidirectional setting.

Conclusion: AirSim360 addresses the data scarcity problem in omnidirectional understanding by providing a comprehensive simulation platform with diverse aerial data collection capabilities, which will be made publicly available to advance spatial intelligence research.

Abstract: The field of 360-degree omnidirectional understanding has been receiving increasing attention for advancing spatial intelligence. However, the lack of large-scale and diverse data remains a major limitation. In this work, we propose AirSim360, a simulation platform for omnidirectional data from aerial viewpoints, enabling wide-ranging scene sampling with drones. Specifically, AirSim360 focuses on three key aspects: a render-aligned data and labeling paradigm for pixel-level geometric, semantic, and entity-level understanding; an interactive pedestrian-aware system for modeling human behavior; and an automated trajectory generation paradigm to support navigation tasks. Furthermore, we collect more than 60K panoramic samples and conduct extensive experiments across various tasks to demonstrate the effectiveness of our simulator. Unlike existing simulators, our work is the first to systematically model the 4D real world under an omnidirectional setting. The entire platform, including the toolkit, plugins, and collected datasets, will be made publicly available at https://insta360-research-team.github.io/AirSim360-website.

Method: Reformulate training objective as loss on instantaneous velocity parameterized by network predicting average velocity. Formulate guidance as explicit conditioning variables processed through in-context conditioning. Train entirely from scratch.

Result: Achieves 1.72 FID on ImageNet 256×256 with single function evaluation (1-NFE), substantially outperforming prior one-step methods and closing gap with multi-step methods without distillation.

Conclusion: Improved MeanFlow (iMF) advances fastforward generative modeling as a standalone paradigm with better training stability and flexible guidance mechanisms.

Abstract: MeanFlow (MF) has recently been established as a framework for one-step generative modeling. However, its ``fastforward’’ nature introduces key challenges in both the training objective and the guidance mechanism. First, the original MF’s training target depends not only on the underlying ground-truth fields but also on the network itself. To address this issue, we recast the objective as a loss on the instantaneous velocity $v$, re-parameterized by a network that predicts the average velocity $u$. Our reformulation yields a more standard regression problem and improves the training stability. Second, the original MF fixes the classifier-free guidance scale during training, which sacrifices flexibility. We tackle this issue by formulating guidance as explicit conditioning variables, thereby retaining flexibility at test time. The diverse conditions are processed through in-context conditioning, which reduces model size and benefits performance. Overall, our $\textbf{improved MeanFlow}$ ($\textbf{iMF}$) method, trained entirely from scratch, achieves $\textbf{1.72}$ FID with a single function evaluation (1-NFE) on ImageNet 256$\times$256. iMF substantially outperforms prior methods of this kind and closes the gap with multi-step methods while using no distillation. We hope our work will further advance fastforward generative modeling as a stand-alone paradigm.

Method: TUNA builds a unified continuous visual representation by cascading a VAE encoder with a representation encoder. This creates a single representation space that enables end-to-end processing of images and videos for both understanding and generation tasks.

Result: TUNA achieves state-of-the-art results across multiple benchmarks: image/video understanding, image/video generation, and image editing. The unified representation outperforms decoupled alternatives, and stronger pretrained representation encoders consistently improve performance.

Conclusion: The unified representation design is effective and scalable. Joint training on understanding and generation data allows the tasks to benefit from each other rather than interfere, demonstrating the advantages of truly unified multimodal modeling.

Abstract: Unified multimodal models (UMMs) aim to jointly perform multimodal understanding and generation within a single framework. We present TUNA, a native UMM that builds a unified continuous visual representation by cascading a VAE encoder with a representation encoder. This unified representation space allows end-to-end processing of images and videos for both understanding and generation tasks. Compared to prior UMMs with decoupled representations, TUNA’s unified visual space avoids representation format mismatches introduced by separate encoders, outperforming decoupled alternatives in both understanding and generation. Moreover, we observe that stronger pretrained representation encoders consistently yield better performance across all multimodal tasks, highlighting the importance of the representation encoder. Finally, in this unified setting, jointly training on both understanding and generation data allows the two tasks to benefit from each other rather than interfere. Our extensive experiments on multimodal understanding and generation benchmarks show that TUNA achieves state-of-the-art results in image and video understanding, image and video generation, and image editing, demonstrating the effectiveness and scalability of its unified representation design.

Method: Track-conditioned V2V framework that conditions video generation on source video and paired 3D point tracks representing source/target motions, using 3D tracks for depth cues and occlusion handling, trained in two stages on synthetic and real data.

Result: Enables diverse motion edits including joint camera/object manipulation, motion transfer, and non-rigid deformation with precise motion control and spatiotemporal coherence.

Conclusion: The framework unlocks new creative potential in video editing by providing precise control over both camera and object motions using 3D track conditioning.

Abstract: Camera and object motions are central to a video’s narrative. However, precisely editing these captured motions remains a significant challenge, especially under complex object movements. Current motion-controlled image-to-video (I2V) approaches often lack full-scene context for consistent video editing, while video-to-video (V2V) methods provide viewpoint changes or basic object translation, but offer limited control over fine-grained object motion. We present a track-conditioned V2V framework that enables joint editing of camera and object motion. We achieve this by conditioning a video generation model on a source video and paired 3D point tracks representing source and target motions. These 3D tracks establish sparse correspondences that transfer rich context from the source video to new motions while preserving spatiotemporal coherence. Crucially, compared to 2D tracks, 3D tracks provide explicit depth cues, allowing the model to resolve depth order and handle occlusions for precise motion editing. Trained in two stages on synthetic and real data, our model supports diverse motion edits, including joint camera/object manipulation, motion transfer, and non-rigid deformation, unlocking new creative potential in video editing.

Method: 1) Test out-of-the-box video generators on falling objects; 2) Investigate if errors are due to ambiguous metric scale (e.g., frame rate assumptions); 3) Introduce unit-free, two-object protocol testing timing ratio t₁²/t₂² = h₁/h₂ independent of g, focal length, and scale; 4) Use lightweight low-rank adaptor fine-tuned on only 100 single-ball clips to correct physical representation.

Result: 1) Video generators consistently generate objects falling at effectively slower acceleration (g_eff = 1.81 m/s² vs. Earth’s 9.8 m/s²); 2) Temporal rescaling cannot correct high-variance gravity artifacts; 3) Relative tests reveal violations of Galileo’s equivalence principle; 4) Fine-tuning raises g_eff to 6.43 m/s² (65% of terrestrial gravity); 5) Specialist adaptor generalizes zero-shot to two-ball drops and inclined planes.

Conclusion: Video generators have significant gaps in representing fundamental physical laws like gravity, but these can be partially corrected with targeted specialization using minimal data, suggesting specific physical laws can be learned and corrected in video generation models.

Abstract: Video generators are increasingly evaluated as potential world models, which requires them to encode and understand physical laws. We investigate their representation of a fundamental law: gravity. Out-of-the-box video generators consistently generate objects falling at an effectively slower acceleration. However, these physical tests are often confounded by ambiguous metric scale. We first investigate if observed physical errors are artifacts of these ambiguities (e.g., incorrect frame rate assumptions). We find that even temporal rescaling cannot correct the high-variance gravity artifacts. To rigorously isolate the underlying physical representation from these confounds, we introduce a unit-free, two-object protocol that tests the timing ratio $t_1^2/t_2^2 = h_1/h_2$, a relationship independent of $g$, focal length, and scale. This relative test reveals violations of Galileo’s equivalence principle. We then demonstrate that this physical gap can be partially mitigated with targeted specialization. A lightweight low-rank adaptor fine-tuned on only 100 single-ball clips raises $g_{\mathrm{eff}}$ from $1.81,\mathrm{m/s^2}$ to $6.43,\mathrm{m/s^2}$ (reaching $65%$ of terrestrial gravity). This specialist adaptor also generalizes zero-shot to two-ball drops and inclined planes, offering initial evidence that specific physical laws can be corrected with minimal data.

Method: Two-track approach: 1) Real track uses human-in-the-loop scheme with automated acquisition and selective human verification; 2) Virtual track augments real data using reference-conditioned, prompt-guided image generation with screening and validation. Combined tracks create class-balanced dataset for bubble detection training.

Result: Model trained on automatically acquired real images achieves 99.6% accuracy on held-out test set. Mixing real and generated data maintains 99.4% accuracy while reducing collection and review workload.

Conclusion: The approach provides scalable, cost-effective strategy for supplying visual feedback data to SDL workflows and offers practical solution to data scarcity in rare event detection and broader vision tasks.

Abstract: Self-driving laboratories offer a promising path toward reducing the labor-intensive, time-consuming, and often irreproducible workflows in the biological sciences. Yet their stringent precision requirements demand highly robust models whose training relies on large amounts of annotated data. However, this kind of data is difficult to obtain in routine practice, especially negative samples. In this work, we focus on pipetting, the most critical and precision sensitive action in SDLs. To overcome the scarcity of training data, we build a hybrid pipeline that fuses real and virtual data generation. The real track adopts a human-in-the-loop scheme that couples automated acquisition with selective human verification to maximize accuracy with minimal effort. The virtual track augments the real data using reference-conditioned, prompt-guided image generation, which is further screened and validated for reliability. Together, these two tracks yield a class-balanced dataset that enables robust bubble detection training. On a held-out real test set, a model trained entirely on automatically acquired real images reaches 99.6% accuracy, and mixing real and generated data during training sustains 99.4% accuracy while reducing collection and review load. Our approach offers a scalable and cost-effective strategy for supplying visual feedback data to SDL workflows and provides a practical solution to data scarcity in rare event detection and broader vision tasks.

Method: GARF uses fracture-aware pretraining to learn fracture features from individual fragments, employs flow matching for precise 6-DoF alignments, and introduces one-step preassembly at inference time for robustness to unseen objects and varying numbers of fractures.

Result: The framework achieves 82.87% lower rotation error and 25.15% higher part accuracy compared to state-of-the-art methods. It demonstrates strong generalization across unseen object shapes and diverse fracture types in real-world materials like ceramics, bones, eggshells, and lithics.

Conclusion: GARF successfully bridges the gap between synthetic training and real-world application, showing that training on synthetic data can advance real-world 3D puzzle solving with strong generalization capabilities across diverse fracture types and object shapes.

Abstract: 3D reassembly is a challenging spatial intelligence task with broad applications across scientific domains. While large-scale synthetic datasets have fueled promising learning-based approaches, their generalizability to different domains is limited. Critically, it remains uncertain whether models trained on synthetic datasets can generalize to real-world fractures where breakage patterns are more complex. To bridge this gap, we propose GARF, a generalizable 3D reassembly framework for real-world fractures. GARF leverages fracture-aware pretraining to learn fracture features from individual fragments, with flow matching enabling precise 6-DoF alignments. At inference time, we introduce one-step preassembly, improving robustness to unseen objects and varying numbers of fractures. In collaboration with archaeologists, paleoanthropologists, and ornithologists, we curate Fractura, a diverse dataset for vision and learning communities, featuring real-world fracture types across ceramics, bones, eggshells, and lithics. Comprehensive experiments have shown our approach consistently outperforms state-of-the-art methods on both synthetic and real-world datasets, achieving 82.87% lower rotation error and 25.15% higher part accuracy. This sheds light on training on synthetic data to advance real-world 3D puzzle solving, demonstrating its strong generalization across unseen object shapes and diverse fracture types. GARF’s code, data and demo are available at https://ai4ce.github.io/GARF/.

Method: Systematic survey approach evaluating state-of-the-art solutions across single-modality and multi-sensor pipelines (RGB, infrared, audio, radar, RF), covering emerging methodologies including diffusion-based data synthesis, multi-modal fusion, vision-language modeling, self-supervised learning, and reinforcement learning.

Result: Identifies persistent gaps in real-time performance, stealth detection, and swarm-based scenarios, while providing comprehensive evaluation of existing solutions and benchmarks (including large-scale and adversarial benchmarks).

Conclusion: Highlights urgent need for robust, adaptive anti-UAV systems and outlines open research directions to guide development of next-generation defense strategies in an era of extensive UAV usage.

Abstract: Unmanned Aerial Vehicles (UAVs) are indispensable for infrastructure inspection, surveillance, and related tasks, yet they also introduce critical security challenges. This survey provides a wide-ranging examination of the anti-UAV domain, centering on three core objectives-classification, detection, and tracking-while detailing emerging methodologies such as diffusion-based data synthesis, multi-modal fusion, vision-language modeling, self-supervised learning, and reinforcement learning. We systematically evaluate state-of-the-art solutions across both single-modality and multi-sensor pipelines (spanning RGB, infrared, audio, radar, and RF) and discuss large-scale as well as adversarially oriented benchmarks. Our analysis reveals persistent gaps in real-time performance, stealth detection, and swarm-based scenarios, underscoring pressing needs for robust, adaptive anti-UAV systems. By highlighting open research directions, we aim to foster innovation and guide the development of next-generation defense strategies in an era marked by the extensive use of UAVs.

Method: Introduces VoQA dataset and evaluates models under pure visual-only zero-shot, prompt-guided, and OCR-assisted settings. Investigates question-alignment fine-tuning strategies to guide models in interpreting visual questions before reasoning.

Result: Current models show clear performance drop in VoQA compared to traditional VQA. Using VoQA dataset with question-alignment fine-tuning yields robust vision-only reasoning while preserving cross-task generalization to traditional VQA.

Conclusion: VoQA enables more realistic visual understanding where questions are embedded in scenes, and the proposed training strategies foster complementary visual and textual reasoning capabilities while maintaining generalization to traditional VQA tasks.

Abstract: Visual understanding requires interpreting both natural scenes and the textual information that appears within them, motivating tasks such as Visual Question Answering (VQA). However, current VQA benchmarks overlook scenarios with visually embedded questions, whereas advanced agents should be able to see the question without separate text input as humans. We introduce Visual-only Question Answering (VoQA), where both the scene and the question appear within a single image, requiring models to perceive and reason purely through vision. This setting supports more realistic visual understanding and interaction in scenarios where questions or instructions are embedded directly in the visual scene. Evaluations under pure visual-only zero-shot, prompt-guided and OCR-assisted settings show that current models exhibit a clear performance drop compared to traditional VQA. To address this, we investigate question-alignment fine-tuning strategies designed to guide models toward interpreting the visual question prior to reasoning. Leveraging VoQA dataset together with these strategies yields robust vision-only reasoning while preserving cross-task generalization to traditional VQA, reflecting the complementary visual and textual reasoning capabilities fostered through VoQA training. The code and data are publicly available.

Method: RLRF (Reinforcement Learning from Rendering Feedback) uses RL to enhance SVG generation by generating SVG roll-outs, rendering them, comparing to original images for reward computation, and using this visual fidelity feedback to guide model improvements.

Result: RLRF significantly outperforms supervised fine-tuning, addressing common failure modes and enabling precise, high-quality SVG generation with strong structural understanding and generalization.

Conclusion: Reinforcement learning with rendering feedback is an effective approach to improve SVG generation in autoregressive VLMs, overcoming limitations of training without visual output observation.

Abstract: Scalable Vector Graphics (SVG) offer a powerful format for representing visual designs as interpretable code. Recent advances in vision-language models (VLMs) have enabled high-quality SVG generation by framing the problem as a code generation task and leveraging large-scale pretraining. VLMs are particularly suitable for this task as they capture both global semantics and fine-grained visual patterns, while transferring knowledge across vision, natural language, and code domains. However, existing VLM approaches often struggle to produce faithful and efficient SVGs because they never observe the rendered images during training. Although differentiable rendering for autoregressive SVG code generation remains unavailable, rendered outputs can still be compared to original inputs, enabling evaluative feedback suitable for reinforcement learning (RL). We introduce RLRF (Reinforcement Learning from Rendering Feedback), an RL method that enhances SVG generation in autoregressive VLMs by leveraging feedback from rendered SVG outputs. Given an input image, the model generates SVG roll-outs that are rendered and compared to the original image to compute a reward. This visual fidelity feedback guides the model toward producing more accurate, efficient, and semantically coherent SVGs. RLRF significantly outperforms supervised fine-tuning, addressing common failure modes and enabling precise, high-quality SVG generation with strong structural understanding and generalization.

Method: DiffProtect utilizes a diffusion autoencoder to generate semantically meaningful perturbations for facial recognition systems, creating adversarial examples that preserve visual quality while effectively fooling FR systems.

Result: DiffProtect produces more natural-looking encrypted images than state-of-the-art methods while achieving significantly higher attack success rates (24.5% and 25.1% absolute improvements on CelebA-HQ and FFHQ datasets).

Conclusion: Diffusion models can effectively generate high-quality adversarial examples for facial recognition protection, balancing both visual quality and attack performance better than existing methods.

Abstract: The increasingly pervasive facial recognition (FR) systems raise serious concerns about personal privacy, especially for billions of users who have publicly shared their photos on social media. Several attempts have been made to protect individuals from being identified by unauthorized FR systems utilizing adversarial attacks to generate encrypted face images. However, existing methods suffer from poor visual quality or low attack success rates, which limit their utility. Recently, diffusion models have achieved tremendous success in image generation. In this work, we ask: can diffusion models be used to generate adversarial examples to improve both visual quality and attack performance? We propose DiffProtect, which utilizes a diffusion autoencoder to generate semantically meaningful perturbations on FR systems. Extensive experiments demonstrate that DiffProtect produces more natural-looking encrypted images than state-of-the-art methods while achieving significantly higher attack success rates, e.g., 24.5% and 25.1% absolute improvements on the CelebA-HQ and FFHQ datasets.

Method: Uses pre-trained LLM for layout generation, produces masked latents in bounding boxes for object placement, employs attention map fusion and diffusion U-Net for composition, optimizes SVG with pre-trained encoder and LPIPS loss with opacity modulation, and explores primitive shapes for canvas completion.

Result: SVGCraft outperforms prior works in abstraction, recognizability, and detail, achieving metrics: CLIP-T: 0.4563, Cosine Similarity: 0.6342, Confusion: 0.66, Aesthetic: 6.7832.

Conclusion: The framework successfully generates comprehensive vector graphics scenes from text prompts, advancing beyond single-object generation and demonstrating superior performance through both qualitative and quantitative assessments.

Abstract: Generating VectorArt from text prompts is a challenging vision task, requiring diverse yet realistic depictions of the seen as well as unseen entities. However, existing research has been mostly limited to the generation of single objects, rather than comprehensive scenes comprising multiple elements. In response, this work introduces SVGCraft, a novel end-to-end framework for the creation of vector graphics depicting entire scenes from textual descriptions. Utilizing a pre-trained LLM for layout generation from text prompts, this framework introduces a technique for producing masked latents in specified bounding boxes for accurate object placement. It introduces a fusion mechanism for integrating attention maps and employs a diffusion U-Net for coherent composition, speeding up the drawing process. The resulting SVG is optimized using a pre-trained encoder and LPIPS loss with opacity modulation to maximize similarity. Additionally, this work explores the potential of primitive shapes in facilitating canvas completion in constrained environments. Through both qualitative and quantitative assessments, SVGCraft is demonstrated to surpass prior works in abstraction, recognizability, and detail, as evidenced by its performance metrics (CLIP-T: 0.4563, Cosine Similarity: 0.6342, Confusion: 0.66, Aesthetic: 6.7832). The code will be available at https://github.com/ayanban011/SVGCraft.

Method: Uses multi-scale feature extraction incorporating Haar wavelet transformation and normalization to preprocess document images before GAN training, reducing computational complexity.

Result: Achieves 10% reduction in training time and 26% reduction in inference time while maintaining performance at 73.79 Average-Score metric, comparable to state-of-the-art methods.

Conclusion: The proposed efficient method significantly reduces computational costs for document image binarization while preserving enhancement quality, making it more practical for real-world applications.

Abstract: The outcome of text recognition for degraded color documents is often unsatisfactory due to interference from various contaminants. To extract information more efficiently for text recognition, document image enhancement and binarization are often employed as preliminary steps in document analysis. Training independent generative adversarial networks (GANs) for each color channel can generate images where shadows and noise are effectively removed, which subsequently allows for efficient text information extraction. However, employing multiple GANs for different color channels requires long training and inference times. To reduce both the training and inference times of these preliminary steps, we propose an efficient method based on multi-scale feature extraction, which incorporates Haar wavelet transformation and normalization to process document images before submitting them to GANs for training. Experiment results show that our proposed method significantly reduces both the training and inference times while maintaining comparable performances when benchmarked against the state-of-the-art methods. In the best case scenario, a reduction of 10% and 26% are observed for training and inference times, respectively, while maintaining the model performance at 73.79 of Average-Score metric. The implementation of this work is available at https://github.com/RuiyangJu/Efficient_Document_Image_Binarization.

Method: Two-component approach: 1) Compute implicit token-wise classification scores within query images, 2) Adaptive score aggregation strategy that emphasizes tokens with lower scores to produce final membership likelihood score.

Result: Method outperforms adapted LLM detection algorithms in class-conditional and text-to-image scenarios, shows strong robustness to data transformations, and reveals two key findings: linear scaling law on membership inference vulnerability and easier detection in scale-wise visual autoregressive models.

Conclusion: Autoregressive image models are vulnerable to membership inference attacks, with larger foundation models showing increased vulnerability; the proposed method effectively detects unauthorized training data usage while revealing important security implications.

Abstract: Autoregressive image generation has witnessed rapid advancements, with prominent models such as scale-wise visual auto-regression pushing the boundaries of visual synthesis. However, these developments also raise significant concerns regarding data privacy and copyright. In response, training data detection has emerged as a critical task for identifying unauthorized data usage in model training. To better understand the vulnerability of autoregressive image generative models to such detection, we conduct the first study applying membership inference to this domain. Our approach comprises two key components: implicit classification and an adaptive score aggregation strategy. First, we compute the implicit token-wise classification score within the query image. Then we propose an adaptive score aggregation strategy to acquire a final score, which places greater emphasis on the tokens with lower scores. A higher final score indicates that the sample is more likely to be involved in the training set. To validate the effectiveness of our method, we adapt existing detection algorithms originally designed for LLMs to visual autoregressive models. Extensive experiments demonstrate the superiority of our method in both class-conditional and text-to-image scenarios. Moreover, our approach exhibits strong robustness and generalization under various data transformations. Furthermore, sufficient experiments suggest two novel key findings: (1) A linear scaling law on membership inference, exposing the vulnerability of large foundation models. (2) Training data from scale-wise visual autoregressive models is easier to detect than other autoregressive paradigms. Our code is available at https://github.com/Chrisqcwx/ImageAR-MIA.

Method: Narrows affective forecasting scope to Human-interaction-based Emotion Forecasting (EF), focusing on two-party interactions where emotions are influenced by partner’s emotional expressions and informational cues.

Result: Proposes a structured perspective for exploring emotional change patterns, enhancing the feasibility of emotion forecasting compared to traditional approaches.

Conclusion: Human-interaction-based EF provides a more structured and feasible framework for emotion forecasting by focusing on dyadic interactions and partner influence.

Abstract: Affective Forecasting is an psychology task that involves predicting an individual’s future emotional responses, often hampered by reliance on external factors leading to inaccuracies, and typically remains at a qualitative analysis stage. To address these challenges, we narrows the scope of Affective Forecasting by introducing the concept of Human-interaction-based Emotion Forecasting (EF). This task is set within the context of a two-party interaction, positing that an individual’s emotions are significantly influenced by their interaction partner’s emotional expressions and informational cues. This dynamic provides a structured perspective for exploring the patterns of emotional change, thereby enhancing the feasibility of emotion forecasting.

Method: Created CP-Bench, a minimal controlled benchmark using a simple task: counting identical cubes in synthetic scenes while camera moves, revealing only subsets of objects at any moment. Used static-camera control variant to isolate temporal integration ability.

Result: State-of-the-art models (Qwen-3-VL, InternVL3, GPT-5, Gemini-3-Pro) fail dramatically. Static-camera control confirms failure is due to inability to accumulate evidence across time, not object recognition. Higher FPS, enhanced models, and finetuning don’t help.

Conclusion: Modern multimodal architectures and training paradigms have a fundamental limitation in continuous perception. CP-Bench provides a diagnostic tool and testbed for developing time-consistent visual reasoning models.

Abstract: Continuous perception, the ability to integrate visual observations over time in a continuous stream fashion, is essential for robust real-world understanding, yet remains largely untested in current multimodal models. We introduce CP-Bench, a minimal and fully controlled benchmark designed to isolate this capability using an extremely simple task: counting identical cubes in a synthetic scene while the camera moves and only reveals subsets of objects at any moment. Despite the simplicity of the setting, we find that state-of-the-art open-source and commercial models, including Qwen-3-VL, InternVL3, GPT-5, and Gemini-3-Pro, fail dramatically. A static-camera control variant confirms that the failure arises not from object recognition but from an inability to accumulate evidence across time. Further experiments show that neither higher sampling FPS, perception- or spatial-enhanced models, nor finetuning with additional videos leads to meaningful cross-temporal generalization. Our results reveal a fundamental limitation in modern multimodal architectures and training paradigms. CP-Bench provides a simple yet powerful diagnostic tool and establishes a clean testbed for developing models capable of genuine time-consistent visual reasoning.

Method: VITA directly flows from visual representations to latent actions without noise or conditioning modules. It uses an action autoencoder to map raw actions into a structured latent space aligned with visual latents. To prevent latent space collapse, it employs flow latent decoding that anchors generation by backpropagating action reconstruction loss through ODE solving steps.

Result: VITA achieves 1.5x-2x faster inference compared to conventional methods with conditioning modules, while outperforming or matching state-of-the-art policies on 9 simulation and 5 real-world tasks from ALOHA and Robomimic.

Conclusion: VITA demonstrates that eliminating noise and visual conditioning modules in flow matching policies can significantly improve inference speed while maintaining or improving performance, offering a more efficient approach for vision-to-action policy learning.

Abstract: Conventional flow matching and diffusion-based policies sample through iterative denoising from standard noise distributions (e.g., Gaussian), and require conditioning modules to repeatedly incorporate visual information during the generative process, incurring substantial time and memory overhead. To reduce the complexity, we develop VITA(VIsion-To-Action policy), a noise-free and conditioning-free flow matching policy learning framework that directly flows from visual representations to latent actions. Since the source of the flow is visually grounded, VITA eliminates the need of visual conditioning during generation. As expected, bridging vision and action is challenging, because actions are lower-dimensional, less structured, and sparser than visual representations; moreover, flow matching requires the source and target to have the same dimensionality. To overcome this, we introduce an action autoencoder that maps raw actions into a structured latent space aligned with visual latents, trained jointly with flow matching. To further prevent latent space collapse, we propose flow latent decoding, which anchors the latent generation process by backpropagating the action reconstruction loss through the flow matching ODE (ordinary differential equation) solving steps. We evaluate VITA on 9 simulation and 5 real-world tasks from ALOHA and Robomimic. VITA achieves 1.5x-2x faster inference compared to conventional methods with conditioning modules, while outperforming or matching state-of-the-art policies. Codes, datasets, and demos are available at our project page: https://ucd-dare.github.io/VITA/.

Method: SynPlay uses a novel rule-guided motion generation framework combining real-world motion capture with motion evolution graphs, enabling dynamic human actions through high-level game rules rather than predefined scripts. It features extensive multi-camera setup including UAVs, CCTVs, and UGVs.

Result: The dataset contains over 73k images and 6.5M human instances with detailed annotations for detection, segmentation, and keypoint tasks. Training with SynPlay significantly improves human localization performance, especially in few-shot and data-scarce scenarios.

Conclusion: SynPlay addresses critical gaps in existing datasets by focusing on aerial-view human localization with spontaneous behaviors and complex interactions, providing a valuable resource for developing models for long-range human analysis.

Abstract: We introduce SynPlay, a large-scale synthetic human dataset purpose-built for advancing multi-perspective human localization, with a predominant focus on aerial-view perception. SynPlay departs from traditional synthetic datasets by addressing a critical but underexplored challenge: localizing humans in aerial scenes where subjects often occupy only tens of pixels in the image. In such scenarios, fine-grained details like facial features or textures become irrelevant, shifting the burden of recognition to human motion, behavior, and interactions. To meet this need, SynPlay implements a novel rule-guided motion generation framework that combines real-world motion capture with motion evolution graphs. This design enables human actions to evolve dynamically through high-level game rules rather than predefined scripts, resulting in effectively uncountable motion variations. Unlike existing synthetic datasets-which either focus on static visual traits or reuse a limited set of mocap-driven actions-SynPlay captures a wide spectrum of spontaneous behaviors, including complex interactions that naturally emerge from unscripted gameplay scenarios. SynPlay also introduces an extensive multi-camera setup that spans UAVs at random altitudes, CCTVs, and a freely roaming UGV, achieving true near-to-far perspective coverage in a single dataset. The majority of instances are captured from aerial viewpoints at varying scales, directly supporting the development of models for long-range human analysis-a setting where existing datasets fall short. Our data contains over 73k images and 6.5M human instances, with detailed annotations for detection, segmentation, and keypoint tasks. Extensive experiments demonstrate that training with SynPlay significantly improves human localization performance, especially in few-shot and data-scarce scenarios.

Method: Uses KANs for HSI super-resolution: 1) KAN-Fusion module integrates spatial information from HR-MSI, 2) KAN Channel Attention Block (KAN-CAB) for post-fusion feature extraction with channel attention, 3) Avoids curse of dimensionality while enhancing fine-grained adjustment.

Result: Extensive experiments show HSR-KAN achieves best performance compared to current state-of-the-art HSI-SR methods in both qualitative and quantitative assessments.

Conclusion: HSR-KAN effectively addresses HSI super-resolution challenges by leveraging KANs for spatial-spectral fusion and attention mechanisms, demonstrating superior performance over existing methods.

Abstract: Hyperspectral images (HSIs) have great potential in various visual tasks due to their rich spectral information. However, obtaining high-resolution hyperspectral images remains challenging due to limitations of physical imaging. Inspired by Kolmogorov-Arnold Networks (KANs), we propose an efficient HSI super-resolution (HSI-SR) model to fuse a low-resolution HSI (LR-HSI) and a high-resolution multispectral image (HR-MSI), yielding a high-resolution HSI (HR-HSI). To achieve the effective integration of spatial information from HR-MSI, we design a fusion module based on KANs, called KAN-Fusion. Further inspired by the channel attention mechanism, we design a spectral channel attention module called KAN Channel Attention Block (KAN-CAB) for post-fusion feature extraction. As a channel attention module integrated with KANs, KAN-CAB not only enhances the fine-grained adjustment ability of deep networks, enabling networks to accurately simulate details of spectral sequences and spatial textures, but also effectively avoid Curse of Dimensionality. Extensive experiments show that, compared to current state-of-the-art HSI-SR methods, proposed HSR-KAN achieves the best performance in terms of both qualitative and quantitative assessments. Our code is available at: https://github.com/Baisonm-Li/HSR-KAN.

Method: Proposes a sketch-guided deformation system that: 1) Allows users to modify 3D GS geometry by drawing silhouette sketches from single viewpoints, 2) Combines cage-based deformations with a variant of Neural Jacobian Fields for precise control, 3) Leverages large-scale 2D diffusion priors and ControlNet to ensure semantic plausibility of generated deformations.

Result: Through experiments, demonstrates the effectiveness of the method and showcases its ability to animate static 3D GS models as a key application, providing fine-grained deformation control that was previously challenging.

Conclusion: The presented system successfully addresses the challenge of fine-grained deformation control in 3D Gaussian Splatting by introducing an intuitive sketch-guided approach that combines cage-based deformation with Neural Jacobian Fields and diffusion priors, enabling both precise editing and semantic plausibility.

Abstract: 3D Gaussian Splatting (GS) is one of the most promising novel 3D representations that has received great interest in computer graphics and computer vision. While various systems have introduced editing capabilities for 3D GS, such as those guided by text prompts, fine-grained control over deformation remains an open challenge. In this work, we present a novel sketch-guided 3D GS deformation system that allows users to intuitively modify the geometry of a 3D GS model by drawing a silhouette sketch from a single viewpoint. Our approach introduces a new deformation method that combines cage-based deformations with a variant of Neural Jacobian Fields, enabling precise, fine-grained control. Additionally, it leverages large-scale 2D diffusion priors and ControlNet to ensure the generated deformations are semantically plausible. Through a series of experiments, we demonstrate the effectiveness of our method and showcase its ability to animate static 3D GS models as one of its key applications.

Method: Three complementary techniques: 1) Dynamic ReLU phasing (DyReLU gradually replaced by standard ReLU), 2) weight sharing within residual layers, and 3) cyclic sparsity with dynamic sparsity levels and patterns.

Result: EAST achieves competitive or improved performance at extreme sparsities (99.9-99.99%) on ResNet-34/50 across CIFAR-10, CIFAR-100, and ImageNet, with notable gains at extreme sparsity levels.

Conclusion: The proposed EAST method enables stable training at extreme sparsity levels through complementary mechanisms that address gradient flow and parameter exploration challenges, pushing the boundaries of sparse neural networks.

Abstract: Pruning of deep neural networks has been an effective technique for reducing model size while preserving most of the performance of dense networks, crucial for deploying models on memory and power-constrained devices. While recent sparse learning methods have shown promising performance up to moderate sparsity levels such as 95% and 98%, accuracy quickly deteriorates when pushing sparsities to extreme levels due to unique challenges such as fragile gradient flow. In this work, we explore network performance beyond the commonly studied sparsities, and develop techniques that encourage stable training without accuracy collapse even at extreme sparsities, including 99.90%, 99.95% and 99.99% on ResNet architectures. We propose three complementary techniques that enhance sparse training through different mechanisms: 1) Dynamic ReLU phasing, where DyReLU initially allows for richer parameter exploration before being gradually replaced by standard ReLU, 2) weight sharing which reuses parameters within a residual layer while maintaining the same number of learnable parameters, and 3) cyclic sparsity, where both sparsity levels and sparsity patterns evolve dynamically throughout training to better encourage parameter exploration. We evaluate our method, which we term Extreme Adaptive Sparse Training (EAST) at extreme sparsities using ResNet-34 and ResNet-50 on CIFAR-10, CIFAR-100, and ImageNet, achieving competitive or improved performance compared to existing methods, with notable gains at extreme sparsity levels.

Method: Proposes MMIF-AMIN with: 1) Invertible Dense Network (IDN) for lossless feature extraction from individual modalities, 2) Multi-scale Complementary Feature Extraction Module (MCFEM) with hybrid attention mechanism, varying convolutional layers, and Transformers, and 3) Adaptive loss function to guide model learning.

Result: Outperforms nine state-of-the-art MMIF methods in both quantitative and qualitative analyses. Ablation experiments confirm effectiveness of each component. Extension to other image fusion tasks also achieves promising performance.

Conclusion: MMIF-AMIN effectively addresses the challenge of capturing both unique and complementary information in multimodal medical image fusion, demonstrating superior performance and generalizability to other fusion tasks.

Abstract: Multimodal medical image fusion (MMIF) aims to integrate images from different modalities to produce a comprehensive image that enhances medical diagnosis by accurately depicting organ structures, tissue textures, and metabolic information. Capturing both the unique and complementary information across multiple modalities simultaneously is a key research challenge in MMIF. To address this challenge, this paper proposes a novel image fusion method, MMIF-AMIN, which features a new architecture that can effectively extract these unique and complementary features. Specifically, an Invertible Dense Network (IDN) is employed for lossless feature extraction from individual modalities. To extract complementary information between modalities, a Multi-scale Complementary Feature Extraction Module (MCFEM) is designed, which incorporates a hybrid attention mechanism, convolutional layers of varying sizes, and Transformers. An adaptive loss function is introduced to guide model learning, addressing the limitations of traditional manually-designed loss functions and enhancing the depth of data mining. Extensive experiments demonstrate that MMIF-AMIN outperforms nine state-of-the-art MMIF methods, delivering superior results in both quantitative and qualitative analyses. Ablation experiments confirm the effectiveness of each component of the proposed method. Additionally, extending MMIF-AMIN to other image fusion tasks also achieves promising performance.

Method: A single-step framework that enables end-to-end training with perceptual and re-render losses, decomposing material properties in one inference step, with a UV refinement network for 3D objects to ensure viewpoint consistency.

Result: Achieves state-of-the-art PBR material decomposition quality while reducing inference time from seconds to milliseconds per image, and completes 3D object material estimation in approximately 3 seconds.

Conclusion: SuperMat provides an efficient, high-quality solution for physically-based material decomposition that maintains computational efficiency and physical consistency across both 2D images and 3D objects.

Abstract: Decomposing physically-based materials from images into their constituent properties remains challenging, particularly when maintaining both computational efficiency and physical consistency. While recent diffusion-based approaches have shown promise, they face substantial computational overhead due to multiple denoising steps and separate models for different material properties. We present SuperMat, a single-step framework that achieves high-quality material decomposition with one-step inference. This enables end-to-end training with perceptual and re-render losses while decomposing albedo, metallic, and roughness maps at millisecond-scale speeds. We further extend our framework to 3D objects through a UV refinement network, enabling consistent material estimation across viewpoints while maintaining efficiency. Experiments demonstrate that SuperMat achieves state-of-the-art PBR material decomposition quality while reducing inference time from seconds to milliseconds per image, and completes PBR material estimation for 3D objects in approximately 3 seconds. The project page is at https://hyj542682306.github.io/SuperMat/.

Method: Manual-PA uses a transformer-based framework that leverages assembly manual diagrams. It employs contrastive learning to semantically align 3D parts with their manual illustrations, then splits the problem into discrete (assembly order prediction) and continuous (6D pose estimation) phases by relating parts to the final furniture depicted in manuals.

Result: The method shows significant improvements over state-of-the-art on the PartNet benchmark dataset. Manual-PA also demonstrates strong generalization to real-world IKEA furniture assembly on the IKEA-Manual dataset.

Conclusion: Using assembly manual diagrams and part order information leads to substantial performance gains in furniture assembly tasks. The framework successfully bridges the gap between diagrammatic instructions and 3D assembly, enabling effective learning from sparse solution spaces.

Abstract: Assembling furniture amounts to solving the discrete-continuous optimization task of selecting the furniture parts to assemble and estimating their connecting poses in a physically realistic manner. The problem is hampered by its combinatorially large yet sparse solution space thus making learning to assemble a challenging task for current machine learning models. In this paper, we attempt to solve this task by leveraging the assembly instructions provided in diagrammatic manuals that typically accompany the furniture parts. Our key insight is to use the cues in these diagrams to split the problem into discrete and continuous phases. Specifically, we present Manual-PA, a transformer-based instruction Manual-guided 3D Part Assembly framework that learns to semantically align 3D parts with their illustrations in the manuals using a contrastive learning backbone towards predicting the assembly order and infers the 6D pose of each part via relating it to the final furniture depicted in the manual. To validate the efficacy of our method, we conduct experiments on the benchmark PartNet dataset. Our results show that using the diagrams and the order of the parts lead to significant improvements in assembly performance against the state of the art. Further, Manual-PA demonstrates strong generalization to real-world IKEA furniture assembly on the IKEA-Manual dataset.

Method: Proposes Minimal Subset Approach (MSA) - an online keyframe sampling method that constructs pose graphs using the most impactful keyframes. MSA optimizes two objectives: redundancy minimization and information preservation, implemented within a sliding window framework. It operates in feature space rather than 3D space to efficiently reduce redundant keyframes while retaining essential information.

Result: Evaluation on diverse public datasets shows MSA outperforms naive methods in reducing false positive rates in place recognition, delivers superior ATE (Absolute Trajectory Error) and RPE (Relative Pose Error) in metric localization, requires no manual parameter tuning, reduces memory usage, and lowers computational overhead during loop closure detection and pose graph optimization.

Conclusion: MSA provides an efficient and scalable keyframe sampling approach for LiDAR SLAM that improves loop closure performance while reducing computational and memory requirements, making it suitable for large-scale missions.

Abstract: Typical LiDAR SLAM architectures feature a front-end for odometry estimation and a back-end for refining and optimizing the trajectory and map, commonly through loop closures. However, loop closure detection in large-scale missions presents significant computational challenges due to the need to identify, verify, and process numerous candidate pairs for pose graph optimization. Keyframe sampling bridges the front-end and back-end by selecting frames for storing and processing during global optimization. This article proposes an online keyframe sampling approach that constructs the pose graph using the most impactful keyframes for loop closure. We introduce the Minimal Subset Approach (MSA), which optimizes two key objectives: redundancy minimization and information preservation, implemented within a sliding window framework. By operating in the feature space rather than 3-D space, MSA efficiently reduces redundant keyframes while retaining essential information. Evaluations on diverse public datasets show that the proposed approach outperforms naive methods in reducing false positive rates in place recognition, while delivering superior ATE and RPE in metric localization, without the need for manual parameter tuning. Additionally, MSA demonstrates efficiency and scalability by reducing memory usage and computational overhead during loop closure detection and pose graph optimization.

Method: (1) Guided Mixture-of-Experts (MoE) module for targeted multimodal and aspect-specific modeling; (2) Core semantic region detection in product images to reduce background noise interference; (3) Specialized negative sampling strategy to increase difficulty and diversity of negative samples.

Result: MOON demonstrates competitive zero-shot performance on both the proposed MBE benchmark and public datasets, showing strong generalization across downstream tasks including cross-modal retrieval, product classification, and attribute prediction. Case studies and visualizations confirm effectiveness.

Conclusion: MOON successfully addresses key challenges in product representation learning using generative MLLMs, establishing a new approach that outperforms existing discriminative methods and provides a comprehensive benchmark for future research in product understanding.

Abstract: With the rapid advancement of e-commerce, exploring general representations rather than task-specific ones has attracted increasing research attention. For product understanding, although existing discriminative dual-flow architectures drive progress in this field, they inherently struggle to model the many-to-one alignment between multiple images and texts of products. Therefore, we argue that generative Multimodal Large Language Models (MLLMs) hold significant potential for improving product representation learning. Nevertheless, achieving this goal still remains non-trivial due to several key challenges: the lack of multimodal and aspect-aware modeling modules in typical LLMs; the common presence of background noise in product images; and the absence of a standard benchmark for evaluation. To address these issues, we propose the first generative MLLM-based model named MOON for product representation learning. Our method (1) employs a guided Mixture-of-Experts (MoE) module for targeted modeling of multimodal and aspect-specific product content; (2) effectively detects core semantic regions in product images to mitigate the distraction and interference caused by background noise; and (3) introduces the specialized negative sampling strategy to increase the difficulty and diversity of negative samples. In addition, we release a large-scale multimodal benchmark MBE for various product understanding tasks. Experimentally, our model demonstrates competitive zero-shot performance on both our benchmark and the public dataset, showcasing strong generalization across various downstream tasks, including cross-modal retrieval, product classification, and attribute prediction. Furthermore, the case study and visualization illustrate the effectiveness of MOON for product understanding.

Method: Target-driven Self-Distillation (TSD) leverages predicted accurate targets to guide predictions under partial observation. Uses self-distillation to learn from feature distributions of both fully and partially observed trajectories in single end-to-end training.

Result: Significant performance improvements on multiple datasets and state-of-the-art motion forecasting models for both fully observed and partially observed scenarios.

Conclusion: TSD provides an effective single-stage training approach that enhances motion prediction accuracy in both observation conditions, addressing limitations of previous multi-stage distillation methods.

Abstract: Accurate prediction of future trajectories of traffic agents is essential for ensuring safe autonomous driving. However, partially observed trajectories can significantly degrade the performance of even state-of-the-art models. Previous approaches often rely on knowledge distillation to transfer features from fully observed trajectories to partially observed ones. This involves firstly training a fully observed model and then using a distillation process to create the final model. While effective, they require multi-stage training, making the training process very expensive. Moreover, knowledge distillation can lead to a performance degradation of the model. In this paper, we introduce a Target-drivenSelf-Distillation method (TSD) for motion forecasting. Our method leverages predicted accurate targets to guide the model in making predictions under partial observation conditions. By employing self-distillation, the model learns from the feature distributions of both fully observed and partially observed trajectories during a single end-to-end training process. This enhances the model’s ability to predict motion accurately in both fully observed and partially observed scenarios. We evaluate our method on multiple datasets and state-of-the-art motion forecasting models. Extensive experimental results demonstrate that our approach achieves significant performance improvements in both settings. To facilitate further research, we will release our code and model checkpoints.

Method: The authors extend Forward-Forward to Vision Transformers and propose Contrastive Forward-Forward, inspired by contrastive learning insights. The algorithm modifies the baseline Forward-Forward approach to better suit complex transformer architectures.

Result: Contrastive Forward-Forward achieves up to 10% higher accuracy and 5-20x faster convergence than baseline Forward-Forward. It reduces the performance gap with backpropagation on Vision Transformers and even outperforms backpropagation in certain conditions like inaccurate supervision.

Conclusion: The proposed Contrastive Forward-Forward algorithm successfully extends the brain-inspired Forward-Forward approach to modern Vision Transformers, significantly improving performance over the baseline and narrowing the gap with backpropagation while maintaining biological plausibility.

Abstract: Although backpropagation is widely accepted as a training algorithm for artificial neural networks, researchers are always looking for inspiration from the brain to find ways with potentially better performance. Forward-Forward is a novel training algorithm that is more similar to what occurs in the brain, although there is a significant performance gap compared to backpropagation. In the Forward-Forward algorithm, the loss functions are placed after each layer, and the updating of a layer is done using two local forward passes and one local backward pass. Forward-Forward is in its early stages and has been designed and evaluated on simple multi-layer perceptron networks to solve image classification tasks. In this work, we have extended the use of this algorithm to a more complex and modern network, namely the Vision Transformer. Inspired by insights from contrastive learning, we have attempted to revise this algorithm, leading to the introduction of Contrastive Forward-Forward. Experimental results show that our proposed algorithm performs significantly better than the baseline Forward-Forward leading to an increase of up to 10% in accuracy and accelerating the convergence speed by 5 to 20 times. Furthermore, if we take Cross Entropy as the baseline loss function in backpropagation, it will be demonstrated that the proposed modifications to the baseline Forward-Forward reduce its performance gap compared to backpropagation on Vision Transformer, and even outperforms it in certain conditions, such as inaccurate supervision.

Method: HiMo repurposes scene flow estimation for non-ego motion compensation. The authors also developed SeFlow++, a real-time scene flow estimator that addresses limitations of existing self-supervised methods which produce degenerate estimates under high-speed distortion. They introduce two new evaluation metrics: compensation accuracy at point level and shape similarity of objects.

Result: HiMo improves geometric consistency and visual fidelity of dynamic objects in LiDAR point clouds. The method was validated on Argoverse 2, ZOD, and a new real-world dataset with highway driving and multi-LiDAR heavy vehicles. Benefits extend to downstream tasks like semantic segmentation and 3D detection.

Conclusion: HiMo effectively addresses motion distortions from dynamic objects in LiDAR point clouds, particularly important for high-speed environments and multi-LiDAR configurations. The pipeline and accompanying SeFlow++ estimator provide practical solutions for improving autonomous vehicle perception systems.

Abstract: LiDAR point cloud is essential for autonomous vehicles, but motion distortions from dynamic objects degrade the data quality. While previous work has considered distortions caused by ego motion, distortions caused by other moving objects remain largely overlooked, leading to errors in object shape and position. This distortion is particularly pronounced in high-speed environments such as highways and in multi-LiDAR configurations, a common setup for heavy vehicles. To address this challenge, we introduce HiMo, a pipeline that repurposes scene flow estimation for non-ego motion compensation, correcting the representation of dynamic objects in point clouds. During the development of HiMo, we observed that existing self-supervised scene flow estimators often produce degenerate or inconsistent estimates under high-speed distortion. We further propose SeFlow++, a real-time scene flow estimator that achieves state-of-the-art performance on both scene flow and motion compensation. Since well-established motion distortion metrics are absent in the literature, we introduce two evaluation metrics: compensation accuracy at a point level and shape similarity of objects. We validate HiMo through extensive experiments on Argoverse 2, ZOD, and a newly collected real-world dataset featuring highway driving and multi-LiDAR-equipped heavy vehicles. Our findings show that HiMo improves the geometric consistency and visual fidelity of dynamic objects in LiDAR point clouds, benefiting downstream tasks such as semantic segmentation and 3D detection. See https://kin-zhang.github.io/HiMo for more details.

Method: PointNSP uses a coarse-to-fine generative framework inspired by level-of-detail principles. It preserves global shape structure at low resolutions and progressively refines fine-grained geometry through next-scale prediction, enabling rich intra-scale interactions without fixed orderings.

Result: PointNSP establishes state-of-the-art generation quality for the first time within the autoregressive paradigm on ShapeNet, surpasses diffusion-based baselines in parameter, training, and inference efficiency, and shows even better performance in dense generation with 8,192 points.

Conclusion: The proposed multi-scale factorization approach successfully aligns autoregressive objectives with the permutation-invariant nature of point sets, demonstrating superior scalability and efficiency while overcoming traditional limitations of autoregressive point cloud generation.

Abstract: Autoregressive point cloud generation has long lagged behind diffusion-based approaches in quality. The performance gap stems from the fact that autoregressive models impose an artificial ordering on inherently unordered point sets, forcing shape generation to proceed as a sequence of local predictions. This sequential bias emphasizes short-range continuity but undermines the model’s capacity to capture long-range dependencies, hindering its ability to enforce global structural properties such as symmetry, consistent topology, and large-scale geometric regularities. Inspired by the level-of-detail (LOD) principle in shape modeling, we propose PointNSP, a coarse-to-fine generative framework that preserves global shape structure at low resolutions and progressively refines fine-grained geometry at higher scales through a next-scale prediction paradigm. This multi-scale factorization aligns the autoregressive objective with the permutation-invariant nature of point sets, enabling rich intra-scale interactions while avoiding brittle fixed orderings. Experiments on ShapeNet show that PointNSP establishes state-of-the-art (SOTA) generation quality for the first time within the autoregressive paradigm. In addition, it surpasses strong diffusion-based baselines in parameter, training, and inference efficiency. Finally, in dense generation with 8,192 points, PointNSP’s advantages become even more pronounced, underscoring its scalability potential.

Method: Align the original classifier’s distribution (over class indices) with its vision-language counterpart distribution from textual class names, while preserving classifier performance. No image-concept labels needed, CLIP-free, and zero-shot linear classifier derivation.

Result: Applied to over 40 visual classifiers, the method sets new SOTA, surpassing even supervised CLIP-based CBMs. Also achieves SOTA in zero-shot image captioning, outperforming CLIP-based methods.

Conclusion: The proposed method successfully lifts all three limitations of traditional CBMs, enabling CLIP-free, label-free, zero-shot conversion of any frozen visual classifier into a CBM while preserving performance and achieving state-of-the-art results.

Abstract: Concept Bottleneck Models (CBMs) map dense, high-dimensional feature representations into a set of human-interpretable concepts which are then combined linearly to make a prediction. However, modern CBMs rely on the CLIP model to establish a mapping from dense feature representations to textual concepts, and it remains unclear how to design CBMs for models other than CLIP. Methods that do not use CLIP instead require manual, labor intensive annotation to associate feature representations with concepts. Furthermore, all CBMs necessitate training a linear classifier to map the extracted concepts to class labels. In this work, we lift all three limitations simultaneously by proposing a method that converts any frozen visual classifier into a CBM without requiring image-concept labels (label-free), without relying on the CLIP model (CLIP-free), and by deriving the linear classifier in a zero-shot manner. Our method is formulated by aligning the original classifier’s distribution (over discrete class indices) with its corresponding vision-language counterpart distribution derived from textual class names, while preserving the classifier’s performance. The approach requires no ground-truth image-class annotations, and is highly data-efficient and preserves the classifier’s reasoning process. Applied and tested on over 40 visual classifiers, our resulting CLIP-free, zero-shot CBM sets a new state of the art, surpassing even supervised CLIP-based CBMs. Finally, we also show that our method can be used for zero-shot image captioning, outperforming existing methods based on CLIP, and achieving state of the art results.

Method: 1) Introduces Insight scene representation combining explicit (attention-centric) and implicit (reasoning-centric) representations. 2) Uses Chain-of-Thought instructions to model human driving cognition. 3) Designs task-level Mixture-of-Experts adapter to inject LLM knowledge at minimal parameter cost. 4) Conditions planner on both representations using diffusion-based generative policy. 5) Establishes knowledge distillation pipeline from human driving to LLMs to onboard models.

Result: Extensive experiments on nuScenes and Navsim benchmarks show significant improvements over conventional scene representation approaches.

Conclusion: InsightDrive successfully bridges the gap between human-like cognitive reasoning and autonomous driving systems through a lightweight framework that combines explicit and implicit scene representations with LLM knowledge distillation, achieving superior performance on standard benchmarks.

Abstract: Conventional end-to-end autonomous driving methods often rely on explicit global scene representations, which typically consist of 3D object detection, online mapping, and motion prediction. In contrast, human drivers selectively attend to task-relevant regions and implicitly reason over the broader traffic context. Motivated by this observation, we introduce a lightweight end-to-end autonomous driving framework, InsightDrive. Unlike approaches that directly embed large language models (LLMs), InsightDrive introduces an Insight scene representation that jointly models attention-centric explicit scene representation and reasoning-centric implicit scene representation, so that scene understanding aligns more closely with human cognitive patterns for trajectory planning. To this end, we employ Chain-of-Thought (CoT) instructions to model human driving cognition and design a task-level Mixture-of-Experts (MoE) adapter that injects this knowledge into the autonomous driving model at negligible parameter cost. We further condition the planner on both explicit and implicit scene representations and employ a diffusion-based generative policy, which produces robust trajectory predictions and decisions. The overall framework establishes a knowledge distillation pipeline that transfers human driving knowledge to LLMs and subsequently to onboard models. Extensive experiments on the nuScenes and Navsim benchmarks demonstrate that InsightDrive achieves significant improvements over conventional scene representation approaches.

Method: Analyze pairwise combinations of neurons and attention heads in CLIP’s attention-pooling layer, approximate neuron-head pairs as single directions in CLIP-ResNet’s image-text embedding space, and interpret them by associating with text.

Result: Found that neuron-head pairs can be approximated by single embedding directions, only a sparse set significantly contribute to output, some polysemantic pairs represent sub-concepts, and these insights enable training-free semantic segmentation and dataset distribution shift monitoring.

Conclusion: Examining individual computation paths reveals interpretable units in neural networks, and these units can be effectively utilized for downstream tasks like semantic segmentation and dataset monitoring.

Abstract: We present a novel technique for interpreting the neurons in CLIP-ResNet by decomposing their contributions to the output into individual computation paths. More specifically, we analyze all pairwise combinations of neurons and the following attention heads of CLIP’s attention-pooling layer. We find that these neuron-head pairs can be approximated by a single direction in CLIP-ResNet’s image-text embedding space. Leveraging this insight, we interpret each neuron-head pair by associating it with text. Additionally, we find that only a sparse set of the neuron-head pairs have a significant contribution to the output value, and that some neuron-head pairs, while polysemantic, represent sub-concepts of their corresponding neurons. We use these observations for two applications. First, we employ the pairs for training-free semantic segmentation, outperforming previous methods for CLIP-ResNet. Second, we utilize the contributions of neuron-head pairs to monitor dataset distribution shifts. Our results demonstrate that examining individual computation paths in neural networks uncovers interpretable units, and that such units can be utilized for downstream tasks.

Method: OmniSVG parameterizes SVG commands and coordinates into discrete tokens, decoupling structural logic from low-level geometry for efficient training while maintaining expressiveness. It leverages pre-trained Vision-Language Models for end-to-end multimodal SVG generation.

Result: Extensive experiments show OmniSVG outperforms existing methods. The authors also introduce MMSVG-2M, a multimodal dataset with 2 million richly annotated SVG assets, and a standardized evaluation protocol for conditional SVG generation tasks.

Conclusion: OmniSVG demonstrates potential for integration into professional SVG design workflows by generating high-quality, complex SVG images efficiently while maintaining structural expressiveness.

Abstract: Scalable Vector Graphics (SVG) is an important image format widely adopted in graphic design because of their resolution independence and editability. The study of generating high-quality SVG has continuously drawn attention from both designers and researchers in the AIGC community. However, existing methods either produces unstructured outputs with huge computational cost or is limited to generating monochrome icons of over-simplified structures. To produce high-quality and complex SVG, we propose OmniSVG, a unified framework that leverages pre-trained Vision-Language Models (VLMs) for end-to-end multimodal SVG generation. By parameterizing SVG commands and coordinates into discrete tokens, OmniSVG decouples structural logic from low-level geometry for efficient training while maintaining the expressiveness of complex SVG structure. To further advance the development of SVG synthesis, we introduce MMSVG-2M, a multimodal dataset with two million richly annotated SVG assets, along with a standardized evaluation protocol for conditional SVG generation tasks. Extensive experiments show that OmniSVG outperforms existing methods and demonstrates its potential for integration into professional SVG design workflows.

Method: Created M3DLayout dataset with 21,367 layouts and 433k+ object instances from three sources: real-world scans, professional CAD designs, and procedurally generated scenes. Each layout has structured text descriptions at global, relational, and fine-grained levels.

Result: Benchmarking with text-conditioned diffusion and autoregressive models shows the dataset provides solid foundation for training. Multi-source composition enhances diversity, especially Inf3DLayout subset enabling generation of more complex, detailed scenes with rich small-object information.

Conclusion: M3DLayout serves as valuable resource for advancing text-driven 3D scene synthesis research, addressing dataset limitations and enabling more sophisticated layout generation models.

Abstract: In text-driven 3D scene generation, object layout serves as a crucial intermediate representation that bridges high-level language instructions with detailed geometric output. It not only provides a structural blueprint for ensuring physical plausibility but also supports semantic controllability and interactive editing. However, the learning capabilities of current 3D indoor layout generation models are constrained by the limited scale, diversity, and annotation quality of existing datasets. To address this, we introduce M3DLayout, a large-scale, multi-source dataset for 3D indoor layout generation. M3DLayout comprises 21,367 layouts and over 433k object instances, integrating three distinct sources: real-world scans, professional CAD designs, and procedurally generated scenes. Each layout is paired with detailed structured text describing global scene summaries, relational placements of large furniture, and fine-grained arrangements of smaller items. This diverse and richly annotated resource enables models to learn complex spatial and semantic patterns across a wide variety of indoor environments. To assess the potential of M3DLayout, we establish a benchmark using both a text-conditioned diffusion model and a text-conditioned autoregressive model. Experimental results demonstrate that our dataset provides a solid foundation for training layout generation models. Its multi-source composition enhances diversity, notably through the Inf3DLayout subset which provides rich small-object information, enabling the generation of more complex and detailed scenes. We hope that M3DLayout can serve as a valuable resource for advancing research in text-driven 3D scene synthesis. All dataset and code will be made public upon acceptance.

Method: Two approaches: DAA-I treats attention maps as spatially independent and uses Frobenius norm; DAA-S uses graph-based state equations to model spatial correlations between attention maps and analyzes global asymptotic stability.

Result: Significantly outperforms existing methods across six backdoor attack scenarios, achieving average F1 Score of 79.27% and AUC of 86.27%.

Conclusion: Dynamic characteristics of diffusion models are effective indicators for backdoor detection, and the proposed DAA framework provides a novel perspective that leverages attention map evolution patterns for improved security.

Abstract: Recent studies have revealed that text-to-image diffusion models are vulnerable to backdoor attacks, where attackers implant stealthy textual triggers to manipulate model outputs. Previous backdoor detection methods primarily focus on the static features of backdoor samples. However, a vital property of diffusion models is their inherent dynamism. This study introduces a novel backdoor detection perspective named Dynamic Attention Analysis (DAA), showing that these dynamic characteristics serve as better indicators for backdoor detection. Specifically, by examining the dynamic evolution of cross-attention maps, we observe that backdoor samples exhibit distinct feature evolution patterns at the $<$EOS$>$ token compared to benign samples. To quantify these dynamic anomalies, we first introduce DAA-I, which treats the tokens’ attention maps as spatially independent and measures dynamic feature using the Frobenius norm. Furthermore, to better capture the interactions between attention maps and refine the feature, we propose a dynamical system-based approach, referred to as DAA-S. This model formulates the spatial correlations among attention maps using a graph-based state equation and we theoretically analyze the global asymptotic stability of this method. Extensive experiments across six representative backdoor attack scenarios demonstrate that our approach significantly surpasses existing detection methods, achieving an average F1 Score of 79.27% and an AUC of 86.27%. The code is available at https://github.com/Robin-WZQ/DAA.

Method: Spiral is a range-view LiDAR diffusion model that generates depth, reflectance images, and semantic maps simultaneously. It uses a novel architecture that preserves the computational efficiency of range-view representations while enabling joint generation of geometry and semantics.

Result: Spiral achieves state-of-the-art performance on SemanticKITTI and nuScenes datasets with the smallest parameter size, outperforming two-step methods that combine generative and segmentation models. The generated range images effectively serve as synthetic data augmentation for downstream segmentation tasks, significantly reducing labeling effort.

Conclusion: Spiral successfully addresses the limitation of existing range-view methods by enabling simultaneous generation of geometric and semantic information while maintaining computational efficiency. The approach demonstrates superior performance and practical utility for LiDAR data augmentation in autonomous driving applications.

Abstract: Leveraging recent diffusion models, LiDAR-based large-scale 3D scene generation has achieved great success. While recent voxel-based approaches can generate both geometric structures and semantic labels, existing range-view methods are limited to producing unlabeled LiDAR scenes. Relying on pretrained segmentation models to predict the semantic maps often results in suboptimal cross-modal consistency. To address this limitation while preserving the advantages of range-view representations, such as computational efficiency and simplified network design, we propose Spiral, a novel range-view LiDAR diffusion model that simultaneously generates depth, reflectance images, and semantic maps. Furthermore, we introduce novel semantic-aware metrics to evaluate the quality of the generated labeled range-view data. Experiments on the SemanticKITTI and nuScenes datasets demonstrate that Spiral achieves state-of-the-art performance with the smallest parameter size, outperforming two-step methods that combine the generative and segmentation models. Additionally, we validate that range images generated by Spiral can be effectively used for synthetic data augmentation in the downstream segmentation training, significantly reducing the labeling effort on LiDAR data.

Method: Uni-X uses a two-end-separated, middle-shared architecture: initial and final layers are dedicated to modality-specific processing, while middle layers share parameters for high-level semantic fusion. This X-shaped design eliminates gradient conflicts at both ends and reduces conflicts in shared layers.

Result: Uni-X achieves superior training efficiency and parameter efficiency. At 3B parameters, it matches or surpasses 7B AR-based unified multimodal models, achieving GenEval score of 82 for image generation while maintaining strong performance in text and vision understanding tasks.

Conclusion: Uni-X establishes a parameter-efficient and scalable foundation for future unified multimodal modeling by solving gradient conflicts through its X-shaped architecture, enabling better training efficiency and performance with fewer parameters.

Abstract: Unified Multimodal Models (UMMs) built on shared autoregressive (AR) transformers are attractive for their architectural simplicity. However, we identify a critical limitation: when trained on multimodal inputs, modality-shared transformers suffer from severe gradient conflicts between vision and text, particularly in shallow and deep layers. We trace this issue to the fundamentally different low-level statistical properties of images and text, while noting that conflicts diminish in middle layers where representations become more abstract and semantically aligned. To overcome this challenge, we propose Uni-X, a two-end-separated, middle-shared architecture. Uni-X dedicates its initial and final layers to modality-specific processing, while maintaining shared parameters in the middle layers for high-level semantic fusion. This X-shaped design not only eliminates gradient conflicts at both ends but also further alleviates residual conflicts in the shared layers. Extensive experiments validate the effectiveness of Uni-X. Under identical training conditions, Uni-X achieves superior training efficiency compared to strong baselines. When scaled to 3B parameters with larger training data, Uni-X matches or surpasses 7B AR-based UMMs, achieving a GenEval score of 82 for image generation alongside strong performance in text and vision understanding tasks. These results establish Uni-X as a parameter-efficient and scalable foundation for future unified multimodal modeling. Our code is available at https://github.com/CURRENTF/Uni-X

Method: Two key innovations: 1) Direct preference optimization tailored for human-centric animation using curated human preference data, 2) Temporal motion modulation that resolves spatiotemporal resolution mismatches by reshaping motion conditions into dimensionally aligned latent features through temporal channel redistribution and proportional feature expansion.

Result: Experiments demonstrate obvious improvements in lip-audio synchronization, expression vividness, body motion coherence over baseline methods, alongside notable gains in human preference metrics.

Conclusion: The proposed human-preference-aligned diffusion framework effectively addresses challenges in portrait animation generation and is complementary to existing UNet and DiT-based approaches.

Abstract: Generating highly dynamic and photorealistic portrait animations driven by audio and skeletal motion remains challenging due to the need for precise lip synchronization, natural facial expressions, and high-fidelity body motion dynamics. We propose a human-preference-aligned diffusion framework that addresses these challenges through two key innovations. First, we introduce direct preference optimization tailored for human-centric animation, leveraging a curated dataset of human preferences to align generated outputs with perceptual metrics for portrait motion-video alignment and naturalness of expression. Second, the proposed temporal motion modulation resolves spatiotemporal resolution mismatches by reshaping motion conditions into dimensionally aligned latent features through temporal channel redistribution and proportional feature expansion, preserving the fidelity of high-frequency motion details in diffusion-based synthesis. The proposed mechanism is complementary to existing UNet and DiT-based portrait diffusion approaches, and experiments demonstrate obvious improvements in lip-audio synchronization, expression vividness, body motion coherence over baseline methods, alongside notable gains in human preference metrics. Our model and source code can be found at: https://github.com/fudan-generative-vision/hallo4.

Method: Uses two static photos (front and back) taken in a mirror, along with IMU motion reference from holding a mobile phone. Proposes a video diffusion-based model with parameter-free frame generation strategy and multi-reference attention mechanism to integrate appearance from both selfies. Also introduces image-based fine-tuning to enhance frame sharpness and improve shadows/reflections for realistic human-scene composition.

Result: Synthesizes realistic videos of users performing target motions, enabling rendering into new scenes with consistent illumination and shadows.

Conclusion: The approach provides a convenient solution for full-body video capture using only mobile devices, overcoming limitations of traditional camera setups through novel video diffusion techniques.

Abstract: Self-captured full-body videos are popular, but most deployments require mounted cameras, carefully-framed shots, and repeated practice. We propose a more convenient solution that enables full-body video capture using handheld mobile devices. Our approach takes as input two static photos (front and back) of you in a mirror, along with an IMU motion reference that you perform while holding your mobile phone, and synthesizes a realistic video of you performing a similar target motion. We enable rendering into a new scene, with consistent illumination and shadows. We propose a novel video diffusion-based model to achieve this. Specifically, we propose a parameter-free frame generation strategy and a multi-reference attention mechanism to effectively integrate appearance information from both the front and back selfies into the video diffusion model. Further, we introduce an image-based fine-tuning strategy to enhance frame sharpness and improve shadows and reflections generation for more realistic human-scene composition.

Method: Two-stage training strategy that gradually removes future motion supervision via object masks, training video diffusion models on synthetic scenes of simple interactions, and integrating low-level motion control with high-level textual conditioning via predicted scene descriptions.

Result: KineMask achieves strong improvements over recent models of comparable size in generating realistic object interactions and demonstrates complementary roles of low- and high-level conditioning in video diffusion models.

Conclusion: KineMask addresses limitations in current video generation approaches by enabling realistic rigid body control and object interactions, with potential applications in robotics and embodied AI through its integration of low-level motion control with high-level conditioning.

Abstract: Recent video generation models have achieved remarkable progress and are now deployed in film, social media production, and advertising. Beyond their creative potential, such models also hold promise as world simulators for robotics and embodied decision making. Despite strong advances, however, current approaches still struggle to generate physically plausible object interactions and lack object-level control mechanisms. To address these limitations, we introduce KineMask, an approach for video generation that enables realistic rigid body control, interactions, and effects. Given a single image and a specified object velocity, our method generates videos with inferred motions and future object interactions. We propose a two-stage training strategy that gradually removes future motion supervision via object masks. Using this strategy we train video diffusion models (VDMs) on synthetic scenes of simple interactions and demonstrate significant improvements of object interactions in real scenes. Furthermore, KineMask integrates low-level motion control with high-level textual conditioning via predicted scene descriptions, leading to support for synthesis of complex dynamical phenomena. Our experiments show that KineMask achieves strong improvements over recent models of comparable size. Ablation studies further highlight the complementary roles of low- and high-level conditioning in VDMs. Our code, model, and data will be made publicly available. Project Page: https://daromog.github.io/KineMask/

Method: Proposes SIB-VAD with two key components: (1) Sparse Feature Filtering Module (SFFM) that compresses normal features into low-dimensional manifolds using information bottleneck principle with adaptive routing to select suitable normal bottleneck subspaces; (2) Multimodal prediction framework jointly modeling appearance, motion, and semantics with consistency constraints and joint error computation.

Result: Experimental results validate the effectiveness of the feature filtering paradigm based on semantics-aware information bottleneck. The framework demonstrates improved VAD performance compared to existing approaches.

Conclusion: SIB-VAD successfully addresses limitations of existing methods by replacing rigid memory modules with adaptive information bottleneck filtering and enhancing semantic awareness through multimodal modeling, achieving more robust video anomaly detection performance.

Abstract: Semi-supervised video anomaly detection methods face two critical challenges: (1) Strong generalization blurs the boundary between normal and abnormal patterns. Although existing approaches attempt to alleviate this issue using memory modules, their rigid prototype-matching process limits adaptability to diverse scenarios; (2) Relying solely on low-level appearance and motion cues makes it difficult to perceive high-level semantic anomalies in complex scenes. To address these limitations, we propose SIB-VAD, a novel framework based on adaptive information bottleneck filtering and semantic-aware enhancement. We propose the Sparse Feature Filtering Module (SFFM) to replace traditional memory modules. It compresses normal features directly into a low-dimensional manifold based on the information bottleneck principle and uses an adaptive routing mechanism to dynamically select the most suitable normal bottleneck subspace. Trained only on normal data, SFFMs only learn normal low-dimensional manifolds, while abnormal features deviate and are effectively filtered. Unlike memory modules, SFFM directly removes abnormal information and adaptively handles scene variations. To improve semantic awareness, we further design a multimodal prediction framework that jointly models appearance, motion, and semantics. Through multimodal consistency constraints and joint error computation, it achieves more robust VAD performance. Experimental results validate the effectiveness of our feature filtering paradigm based on semantics-aware information bottleneck. Project page at https://qzfm.github.io/sib_vad_project_page/

Method: Uses 4D Gaussian as inductive bias to unify static and dynamic components; processes 64 consecutive posed frames in rolling-window fashion; employs novel density control strategy for handling longer space-time input; performs purely feed-forward inference with Transformer architecture.

Result: Significantly outperforms prior Gaussian-based networks on real-world videos; achieves on-par accuracy with optimization-based methods on cross-domain videos; reduces reconstruction time from hours to seconds; scales effectively to long video sequences.

Conclusion: 4DGT demonstrates that a feed-forward Transformer model with 4D Gaussian representation can achieve efficient and accurate dynamic scene reconstruction from monocular posed videos, bridging the gap between optimization-based and learning-based approaches.

Abstract: We propose 4DGT, a 4D Gaussian-based Transformer model for dynamic scene reconstruction, trained entirely on real-world monocular posed videos. Using 4D Gaussian as an inductive bias, 4DGT unifies static and dynamic components, enabling the modeling of complex, time-varying environments with varying object lifespans. We proposed a novel density control strategy in training, which enables our 4DGT to handle longer space-time input and remain efficient rendering at runtime. Our model processes 64 consecutive posed frames in a rolling-window fashion, predicting consistent 4D Gaussians in the scene. Unlike optimization-based methods, 4DGT performs purely feed-forward inference, reducing reconstruction time from hours to seconds and scaling effectively to long video sequences. Trained only on large-scale monocular posed video datasets, 4DGT can outperform prior Gaussian-based networks significantly in real-world videos and achieve on-par accuracy with optimization-based methods on cross-domain videos. Project page: https://4dgt.github.io

Method: The method predicts dense 2D contact-aware surface landmarks conditioned on segmentation masks, enabling person-specific correspondence estimation under occlusion. It uses a novel architecture with learnable queries for each landmark, trained on a large synthetic multi-view dataset combining diverse human motions including extreme poses, hand motions, and close interactions.

Result: The approach handles complex person-person interactions with greater accuracy than existing methods, offers competitive reconstruction quality compared to commercial marker-based solutions without manual cleanup, and introduces evaluation benchmarks for dense-landmark prediction and markerless motion capture.

Conclusion: MAMMA provides an effective markerless motion-capture solution for two-person interactions that overcomes limitations of traditional systems and existing learning-based methods, with plans to release the dataset, benchmark, method, and code for research purposes.

Abstract: We present MAMMA, a markerless motion-capture pipeline that accurately recovers SMPL-X parameters from multi-view video of two-person interaction sequences. Traditional motion-capture systems rely on physical markers. Although they offer high accuracy, their requirements of specialized hardware, manual marker placement, and extensive post-processing make them costly and time-consuming. Recent learning-based methods attempt to overcome these limitations, but most are designed for single-person capture, rely on sparse keypoints, or struggle with occlusions and physical interactions. In this work, we introduce a method that predicts dense 2D contact-aware surface landmarks conditioned on segmentation masks, enabling person-specific correspondence estimation even under heavy occlusion. We employ a novel architecture that exploits learnable queries for each landmark. We demonstrate that our approach can handle complex person–person interaction and offers greater accuracy than existing methods. To train our network, we construct a large, synthetic multi-view dataset combining human motions from diverse sources, including extreme poses, hand motions, and close interactions. Our dataset yields high-variability synthetic sequences with rich body contact and occlusion, and includes SMPL-X ground-truth annotations with dense 2D landmarks. The result is a system capable of capturing human motion without the need for markers. Our approach offers competitive reconstruction quality compared to commercial marker-based motion-capture solutions, without the extensive manual cleanup. Finally, we address the absence of common benchmarks for dense-landmark prediction and markerless motion capture by introducing two evaluation settings built from real multi-view sequences. We will release our dataset, benchmark, method, training code, and pre-trained model weights for research purposes.

Method: ProtoMM constructs multimodal prototypes as discrete distributions over textual descriptions and visual particles. Visual particles are dynamically updated as testing data flows, enabling continual learning. The framework formulates semantic distance between prototypes and test images as an optimal transport problem to quantify their importance.

Result: Extensive experiments on 15 zero-shot benchmarks show ProtoMM achieves 1.03% average accuracy improvement over state-of-the-art methods on ImageNet and its variant datasets.

Conclusion: ProtoMM effectively addresses the limitations of text-only prototypes by leveraging multimodal features for comprehensive prototype learning, enabling better adaptation of vision-language models during test time without requiring training.

Abstract: With the increasing attention to pre-trained vision-language models (VLMs), \eg, CLIP, substantial efforts have been devoted to many downstream tasks, especially in test-time adaptation (TTA). However, previous works focus on learning prototypes only in the textual modality while overlooking the ambiguous semantics in class names. These ambiguities lead to textual prototypes that are insufficient to capture visual concepts, resulting in limited performance. To address this issue, we introduce \textbf{ProtoMM}, a training-free framework that constructs multimodal prototypes to adapt VLMs during the test time. By viewing the prototype as a discrete distribution over the textual descriptions and visual particles, ProtoMM has the ability to combine the multimodal features for comprehensive prototype learning. More importantly, the visual particles are dynamically updated as the testing stream flows. This allows our multimodal prototypes to continually learn from the data, enhancing their generalizability in unseen scenarios. In addition, we quantify the importance of the prototypes and test images by formulating their semantic distance as an optimal transport problem. Extensive experiments on 15 zero-shot benchmarks demonstrate the effectiveness of our method, achieving a 1.03% average accuracy improvement over state-of-the-art methods on ImageNet and its variant datasets.

Method: Two-stage approach: 1) Pixel-wise image alignment using dense optical flow tracking to eliminate ghosting artifacts from image misalignment. 2) Image-sequential binomial self-compensation (I-BSC) method that performs weighted summation of homogeneous fringe images using binomial coefficients to correct ripple-like distortions from phase deviation, with low computational complexity comparable to traditional four-step PSP.

Result: Extensive experiments show RPSP-AM outperforms state-of-the-art methods in compensating for both ghosting artifacts and ripple-like distortions under challenging conditions including 6-DoF motion, non-rigid deformations, and multi-target movements.

Conclusion: The proposed method extends PSP applicability to arbitrary motion scenarios, enabling high-fidelity motion-error-free 3D reconstruction with potential for widespread adoption in robotics, industrial inspection, and medical reconstruction.

Abstract: Phase-shifting profilometry (PSP) enables high-accuracy 3D reconstruction but remains highly susceptible to object motion. Although numerous studies have explored compensation for motion-induced errors, residual inaccuracies still persist, particularly in complex motion scenarios. In this paper, we propose a robust phase-shifting profilometry for arbitrary motion (RPSP-AM), including six-degrees-of-freedom (6-DoF) motion (translation and rotation in any direction), non-rigid deformations, and multi-target movements, achieving high-fidelity motion-error-free 3D reconstruction. We categorize motion errors into two components: 1) ghosting artifacts induced by image misalignment, and 2) ripple-like distortions induced by phase deviation. To eliminate the ghosting artifacts, we perform pixel-wise image alignment based on dense optical flow tracking. To correct ripple-like distortions, we propose a high-accuracy, low-complexity image-sequential binomial self-compensation (I-BSC) method, which performs a summation of the homogeneous fringe images weighted by binomial coefficients, exponentially reducing the ripple-like distortions with a competitive computational speed compared with the traditional four-step phase-shifting method. Extensive experimental results demonstrate that, under challenging conditions such as 6-DoF motion, non-rigid deformations, and multi-target movements, the proposed RPSP-AM outperforms state-of-the-art (SoTA) methods in compensating for both ghosting artifacts and ripple-like distortions. Our approach extends the applicability of PSP to arbitrary motion scenarios, endowing it with potential for widespread adoption in fields such as robotics, industrial inspection, and medical reconstruction.

Method: Created two datasets: PigDetect for object detection and PigTrack for multi-object tracking, using diverse image/video material from realistic barn conditions with challenging scenarios (occlusions, bad visibility). Evaluated state-of-the-art detection models and compared SORT-based vs end-to-end trainable tracking methods.

Result: Challenging training images improve detection performance; state-of-the-art models outperform real-time alternatives. For tracking, SORT-based methods have better detection performance while end-to-end models show better association performance. Models generalize well to unseen pens, highlighting the importance of high-quality training data.

Conclusion: The study provides systematic benchmarking for pig monitoring, showing that challenging training data improves performance and that while SORT-based tracking currently performs better, end-to-end models show promise for future improvements. The publicly available datasets facilitate reproducibility and further development.

Abstract: To ensure animal welfare and effective management in pig farming, monitoring individual behavior is a crucial prerequisite. While monitoring tasks have traditionally been carried out manually, advances in machine learning have made it possible to collect individualized information in an increasingly automated way. Central to these methods is the localization of animals across space (object detection) and time (multi-object tracking). Despite extensive research of these two tasks in pig farming, a systematic benchmarking study has not yet been conducted. In this work, we address this gap by curating two datasets: PigDetect for object detection and PigTrack for multi-object tracking. The datasets are based on diverse image and video material from realistic barn conditions, and include challenging scenarios such as occlusions or bad visibility. For object detection, we show that challenging training images improve detection performance beyond what is achievable with randomly sampled images alone. Comparing different approaches, we found that state-of-the-art models offer substantial improvements in detection quality over real-time alternatives. For multi-object tracking, we observed that SORT-based methods achieve superior detection performance compared to end-to-end trainable models. However, end-to-end models show better association performance, suggesting they could become strong alternatives in the future. We also investigate characteristic failure cases of end-to-end models, providing guidance for future improvements. The detection and tracking models trained on our datasets perform well in unseen pens, suggesting good generalization capabilities. This highlights the importance of high-quality training data. The datasets and research code are made publicly available to facilitate reproducibility, re-use and further development.

Method: Proposes PIF-Net with: 1) Explicit incorporation of ill-posed priors to handle data misalignment; 2) Invertible Mamba architecture for global spectral modeling while maintaining information consistency and computational efficiency; 3) Fusion-Aware Low-Rank Adaptation module for dynamic calibration of spectral and spatial features while keeping the model lightweight.

Result: Extensive experiments on multiple benchmark datasets show PIF-Net achieves significantly better image restoration performance than current state-of-the-art methods while maintaining model efficiency.

Conclusion: PIF-Net effectively addresses the ill-posed nature of MHIF through explicit ill-posed priors, invertible architecture for information consistency, and lightweight dynamic feature calibration, achieving superior performance with maintained efficiency.

Abstract: The goal of multispectral and hyperspectral image fusion (MHIF) is to generate high-quality images that simultaneously possess rich spectral information and fine spatial details. However, due to the inherent trade-off between spectral and spatial information and the limited availability of observations, this task is fundamentally ill-posed. Previous studies have not effectively addressed the ill-posed nature caused by data misalignment. To tackle this challenge, we propose a fusion framework named PIF-Net, which explicitly incorporates ill-posed priors to effectively fuse multispectral images and hyperspectral images. To balance global spectral modeling with computational efficiency, we design a method based on an invertible Mamba architecture that maintains information consistency during feature transformation and fusion, ensuring stable gradient flow and process reversibility. Furthermore, we introduce a novel fusion module called the Fusion-Aware Low-Rank Adaptation module, which dynamically calibrates spectral and spatial features while keeping the model lightweight. Extensive experiments on multiple benchmark datasets demonstrate that PIF-Net achieves significantly better image restoration performance than current state-of-the-art methods while maintaining model efficiency.

Method: DynaStride uses adaptive frame sampling and multimodal windowing on YouCookII dataset, employs multimodal chain-of-thought for action-object pairs, and refines them with dynamic stride window selection algorithm to balance temporal context and redundancy.

Result: Outperforms strong baselines (VLLaMA3, GPT-4o) on both N-gram metrics (BLEU, METEOR) and semantic similarity measures (BERTScore, CLIPScore), with qualitative analysis showing better temporal coherence and informativeness.

Conclusion: DynaStride effectively generates more coherent and informative scene-level captions for instructional videos, offering a promising approach for AI-powered educational content generation without requiring manual segmentation.

Abstract: Scene-level captioning in instructional videos can enhance learning by requiring an understanding of both visual cues and temporal structure. By aligning visual cues with textual guidance, this understanding supports procedural learning and multimodal reasoning, providing a richer context for skill acquisition. However, captions that fail to capture this structure may lack coherence and quality, which can create confusion and undermine the video’s educational intent. To address this gap, we introduce DynaStride, a pipeline to generate coherent, scene-level captions without requiring manual scene segmentation. Using the YouCookII dataset’s scene annotations, DynaStride performs adaptive frame sampling and multimodal windowing to capture key transitions within each scene. It then employs a multimodal chain-of-thought process to produce multiple action-object pairs, which are refined and fused using a dynamic stride window selection algorithm that adaptively balances temporal context and redundancy. The final scene-level caption integrates visual semantics and temporal reasoning in a single instructional caption. Empirical evaluations against strong baselines, including VLLaMA3 and GPT-4o, demonstrate consistent gains on both N-gram-based metrics (BLEU, METEOR) and semantic similarity measures (BERTScore, CLIPScore). Qualitative analyses further show that DynaStride produces captions that are more temporally coherent and informative, suggesting a promising direction for improving AI-powered instructional content generation.

Method: 1) Depth-guided initialization using Group-wise Multi-scale Fusion (GMF) module to generate sparse representative Gaussian primitives from multi-scale image and depth features. 2) Decoupled Gaussian Aggregator (DGA) that separates geometric and semantic predictions during Gaussian-to-voxel splatting to reduce outlier noise. 3) Probability Scale Loss for optimization.

Result: State-of-the-art performance on Occ-ScanNet dataset: +6.3% IoU and +4.1% mIoU improvement over prior methods, while reducing latency and memory cost by >9.3%.

Conclusion: SplatSSC effectively addresses limitations of random Gaussian initialization and outlier artifacts through depth-guided initialization and decoupled aggregation, achieving superior efficiency and accuracy in monocular 3D semantic scene completion.

Abstract: Monocular 3D Semantic Scene Completion (SSC) is a challenging yet promising task that aims to infer dense geometric and semantic descriptions of a scene from a single image. While recent object-centric paradigms significantly improve efficiency by leveraging flexible 3D Gaussian primitives, they still rely heavily on a large number of randomly initialized primitives, which inevitably leads to 1) inefficient primitive initialization and 2) outlier primitives that introduce erroneous artifacts. In this paper, we propose SplatSSC, a novel framework that resolves these limitations with a depth-guided initialization strategy and a principled Gaussian aggregator. Instead of random initialization, SplatSSC utilizes a dedicated depth branch composed of a Group-wise Multi-scale Fusion (GMF) module, which integrates multi-scale image and depth features to generate a sparse yet representative set of initial Gaussian primitives. To mitigate noise from outlier primitives, we develop the Decoupled Gaussian Aggregator (DGA), which enhances robustness by decomposing geometric and semantic predictions during the Gaussian-to-voxel splatting process. Complemented with a specialized Probability Scale Loss, our method achieves state-of-the-art performance on the Occ-ScanNet dataset, outperforming prior approaches by over 6.3% in IoU and 4.1% in mIoU, while reducing both latency and memory cost by more than 9.3%.

Method: Used Vision Transformer (ViT)-based autoencoder with Low-Rank Adaptation (LoRA) to implement fast weights as short-term memory traces, analyzing how familiarity training affects representation alignment and attention scope.

Result: (1) ViT self-attention performs manifold transform similar to neural circuit models; (2) Early layer representations align with top-layer global context; (3) Attention scope broadens within remembered context; (4) LoRA fast weights amplify these effects.

Conclusion: Familiarity training introduces global sensitivity to earlier hierarchical layers, and hybrid fast-slow weight architectures provide viable computational models for studying rapid global context learning in biological systems.

Abstract: Recent neurophysiological studies have revealed that the early visual cortex can rapidly learn global image context, as evidenced by a sparsification of population responses and a reduction in mean activity when exposed to familiar versus novel image contexts. This phenomenon has been attributed primarily to local recurrent interactions, rather than changes in feedforward or feedback pathways, supported by both empirical findings and circuit-level modeling. Recurrent neural circuits capable of simulating these effects have been shown to reshape the geometry of neural manifolds, enhancing robustness and invariance to irrelevant variations. In this study, we employ a Vision Transformer (ViT)-based autoencoder to investigate, from a functional perspective, how familiarity training can induce sensitivity to global context in the early layers of a deep neural network. We hypothesize that rapid learning operates via fast weights, which encode transient or short-term memory traces, and we explore the use of Low-Rank Adaptation (LoRA) to implement such fast weights within each Transformer layer. Our results show that (1) The proposed ViT-based autoencoder’s self-attention circuit performs a manifold transform similar to a neural circuit model of the familiarity effect. (2) Familiarity training aligns latent representations in early layers with those in the top layer that contains global context information. (3) Familiarity training broadens the self-attention scope within the remembered image context. (4) These effects are significantly amplified by LoRA-based fast weights. Together, these findings suggest that familiarity training introduces global sensitivity to earlier layers in a hierarchical network, and that a hybrid fast-and-slow weight architecture may provide a viable computational model for studying rapid global context learning in the brain.

Method: 1) Created PETARSeg-11K dataset with 11,356 lesion descriptions paired with 3D segmentations. 2) Developed PETAR-4B model that jointly encodes PET, CT, and 3D lesion segmentation masks using 3D focal prompts to capture fine-grained lesion details (less than 0.1% of volume).

Result: PETAR-4B substantially outperforms all 2D and 3D baselines on automated metrics. Human study with five physicians confirms clinical utility and establishes correlations between automated metrics and expert judgment.

Conclusion: This work provides foundational dataset and novel architecture for 3D medical vision-language understanding in PET, advancing automated PET reporting with clinically validated performance.

Abstract: Generating automated reports for 3D positron emission tomography (PET) is an important and challenging task in medical imaging. PET plays a vital role in oncology, but automating report generation is difficult due to the complexity of whole-body 3D volumes, the wide range of potential clinical findings, and the limited availability of annotated datasets. To address these challenges, we introduce PETARSeg-11K, the first large-scale, publicly available dataset that provides lesion-level correspondence between 3D PET/CT volumes and free-text radiological findings. It comprises 11,356 lesion descriptions paired with 3D segmentations. Second, we propose PETAR-4B, a 3D vision-language model designed for mask-aware, spatially grounded PET/CT reporting. PETAR-4B jointly encodes PET, CT, and 3D lesion segmentation masks, using a 3D focal prompt to capture fine-grained details of lesions that normally comprise less than 0.1% of the volume. Evaluations using automated metrics show PETAR-4B substantially outperforming all 2D and 3D baselines. A human study involving five physicians – the first of its kind for automated PET reporting – confirms the model’s clinical utility and establishes correlations between automated metrics and expert judgment. This work provides a foundational dataset and a novel architecture, advancing 3D medical vision-language understanding in PET.

Method: Proposes MBMamba without changing original Mamba architecture: 1) Memory buffer mechanism to preserve historical information for later fusion, enabling reliable modeling of relevance between adjacent features; 2) Ising-inspired regularization loss that simulates energy minimization of physical system’s “mutual attraction” between pixels to maintain image structure and coherence.

Result: Outperforms state-of-the-art approaches on widely used benchmarks.

Conclusion: MBMamba effectively addresses Mamba’s limitations for image deblurring while maintaining computational efficiency, achieving superior performance through memory buffer mechanism and physics-inspired regularization.

Abstract: The Mamba architecture has emerged as a promising alternative to CNNs and Transformers for image deblurring. However, its flatten-and-scan strategy often results in local pixel forgetting and channel redundancy, limiting its ability to effectively aggregate 2D spatial information. Although existing methods mitigate this by modifying the scan strategy or incorporating local feature modules, it increase computational complexity and hinder real-time performance. In this paper, we propose a structure-aware image deblurring network without changing the original Mamba architecture. Specifically, we design a memory buffer mechanism to preserve historical information for later fusion, enabling reliable modeling of relevance between adjacent features. Additionally, we introduce an Ising-inspired regularization loss that simulates the energy minimization of the physical system’s “mutual attraction” between pixels, helping to maintain image structure and coherence. Building on this, we develop MBMamba. Experimental results show that our method outperforms state-of-the-art approaches on widely used benchmarks.

Method: Proposed Hybrid Swin Attention Network (HSANet) with Efficient Global Attention (EGA) modules for enhanced spatial/channel interaction and a hybrid upsampling module to prevent overfitting to noise.

Result: HSANet achieves superior denoising performance compared to existing methods on public LDCT/PET dataset while maintaining lightweight model size suitable for standard GPU memory configurations.

Conclusion: HSANet provides a highly practical solution for real-world clinical applications by delivering excellent denoising performance with efficient computational requirements.

Abstract: Low-dose computed tomography (LDCT) and positron emission tomography (PET) have emerged as safer alternatives to conventional imaging modalities by significantly reducing radiation exposure. However, this reduction often results in increased noise and artifacts, which can compromise diagnostic accuracy. Consequently, denoising for LDCT/PET has become a vital area of research aimed at enhancing image quality while maintaining radiation safety. In this study, we introduce a novel Hybrid Swin Attention Network (HSANet), which incorporates Efficient Global Attention (EGA) modules and a hybrid upsampling module. The EGA modules enhance both spatial and channel-wise interaction, improving the network’s capacity to capture relevant features, while the hybrid upsampling module mitigates the risk of overfitting to noise. We validate the proposed approach using a publicly available LDCT/PET dataset. Experimental results demonstrate that HSANet achieves superior denoising performance compared to existing methods, while maintaining a lightweight model size suitable for deployment on GPUs with standard memory configurations. This makes our approach highly practical for real-world clinical applications.

Method: Cross-Hierarchical Differential Feature Fusion Network (CHDFFN) with: 1) Multiscale feature extraction subnetwork using custom encoder-decoder backbone with residual connections and dual-core channel-spatial attention; 2) Spatial-spectral change feature learning module for hierarchical change representations; 3) Adaptive high-level feature fusion module with dynamic weight assignment.

Result: Experiments on four public hyperspectral datasets show average maximum improvements of 4.61% OA, 19.79% KC, and 18.90% F1 compared to state-of-the-art methods.

Conclusion: CHDFFN effectively addresses limitations of existing methods by better capturing multiscale spatial-spectral features and fusing differential information, demonstrating superior performance in hyperspectral change detection tasks.

Abstract: Hyperspectral change detection (HCD) is one of the core applications of remote sensing images, holding significant research value in fields like environmental monitoring and disaster assessment. However, existing methods often suffer from incomplete capture of multiscale spatial-spectral features and insufficient fusion of differential feature information. To address these challenges, this paper proposes a Cross-Hierarchical Differential Feature Fusion Network (CHDFFN) based on a multiscale encoder-decoder. Firstly, a multiscale feature extraction subnetwork is designed, taking the customized encoder-decoder as the backbone, combined with residual connections and the proposed dual-core channel-spatial attention module to achieve multi-level extraction and initial integration of spatial-spectral features. The encoder embeds convolutional blocks with different receptive field sizes to capture multiscale representations from shallow details to deep semantics. The decoder fuses the encoder’s output via skip connections to gradually restore spatial resolution while suppressing background noise and redundancy. To enhance the model’s ability to capture differential features between bi-temporal hyperspectral images, a spatial-spectral change feature learning module is designed to learn hierarchical change representations. Additionally, an adaptive high-level feature fusion module is proposed, dynamically balancing the contribution of hierarchical differential features by adaptively assigning weights, which effectively strengthens the model’s capability to characterize complex change patterns. Finally, experiments on four public hyperspectral datasets show that compared with some state-of-the-art methods, the average maximum improvements of OA, KC, and F1 are 4.61%, 19.79%, and 18.90% respectively, verifying the model’s effectiveness.

Method: HyCoVAD combines a multi-task SSL temporal analyzer (built on nnFormer backbone) with an LLM validator. SSL module identifies suspected anomaly frames through multiple proxy tasks, then LLM applies structured, rule-based reasoning to validate anomalies with semantic context.

Result: Achieves 72.5% frame-level AUC on ComplexVAD dataset, outperforming existing baselines by 12.5% while reducing LLM computation. The approach effectively handles complex anomalies with better semantic understanding.

Conclusion: HyCoVAD successfully bridges the gap between SSL’s spatiotemporal pattern modeling and LLM’s semantic reasoning, providing an efficient hybrid solution for complex video anomaly detection with released taxonomy, protocol, and code for future research.

Abstract: Video anomaly detection (VAD) is crucial for intelligent surveillance, but a significant challenge lies in identifying complex anomalies, which are events defined by intricate relationships and temporal dependencies among multiple entities rather than by isolated actions. While self-supervised learning (SSL) methods effectively model low-level spatiotemporal patterns, they often struggle to grasp the semantic meaning of these interactions. Conversely, large language models (LLMs) offer powerful contextual reasoning but are computationally expensive for frame-by-frame analysis and lack fine-grained spatial localization. We introduce HyCoVAD, Hybrid Complex Video Anomaly Detection, a hybrid SSL-LLM model that combines a multi-task SSL temporal analyzer with LLM validator. The SSL module is built upon an nnFormer backbone which is a transformer-based model for image segmentation. It is trained with multiple proxy tasks, learns from video frames to identify those suspected of anomaly. The selected frames are then forwarded to the LLM, which enriches the analysis with semantic context by applying structured, rule-based reasoning to validate the presence of anomalies. Experiments on the challenging ComplexVAD dataset show that HyCoVAD achieves a 72.5% frame-level AUC, outperforming existing baselines by 12.5% while reducing LLM computation. We release our interaction anomaly taxonomy, adaptive thresholding protocol, and code to facilitate future research in complex VAD scenarios.

Method: Combines dynamic volumetric prompt generation with content-aware adaptive cropping to optimize image encoder usage. Simulates realistic user interaction patterns during training while addressing computational challenges of learning from sequential refinement feedback on a single GPU. Initializes network using publicly available nnInteractive segmentation model weights.

Result: Achieved strong performance on Foundation Models for Interactive 3D Biomedical Image Segmentation competition: average final Dice score of 0.6385, normalized surface distance of 0.6614, and area-under-the-curve metrics of 2.4799 (Dice) and 2.5671 (NSD).

Conclusion: The proposed training strategy effectively addresses computational and interactive challenges in 3D biomedical image segmentation, demonstrating competitive performance through efficient use of resources and realistic interaction simulation.

Abstract: Interactive 3D biomedical image segmentation requires efficient models that can iteratively refine predictions based on user prompts. Current foundation models either lack volumetric awareness or suffer from limited interactive capabilities. We propose a training strategy that combines dynamic volumetric prompt generation with content-aware adaptive cropping to optimize the use of the image encoder. Our method simulates realistic user interaction patterns during training while addressing the computational challenges of learning from sequential refinement feedback on a single GPU. For efficient training, we initialize our network using the publicly available weights from the nnInteractive segmentation model. Evaluation on the \textbf{Foundation Models for Interactive 3D Biomedical Image Segmentation} competition demonstrates strong performance with an average final Dice score of 0.6385, normalized surface distance of 0.6614, and area-under-the-curve metrics of 2.4799 (Dice) and 2.5671 (NSD).

Method: μDeepIQA adapts a deep convolutional neural network architecture originally designed for natural image IQA to optical microscopy data. The model is retrained to predict both individual quality metrics and global quality scores, and provides patch-wise quality predictions for spatial visualization.

Result: The method provides fast and stable quality predictions that generalize even outside the ideal range of standard methods. It enables patch-wise quality assessment and visualization of spatially varying quality within single images.

Conclusion: Optical microscopy studies benefit from deep learning-based IQA due to stable performance with outliers, ability to assess small image patches, rapid predictions, and superior generalizability compared to traditional methods.

Abstract: Optical microscopy is one of the most widely used techniques in research studies for life sciences and biomedicine. These applications require reliable experimental pipelines to extract valuable knowledge from the measured samples and must be supported by image quality assessment (IQA) to ensure correct processing and analysis of the image data. IQA methods are implemented with variable complexity. However, while most quality metrics have a straightforward implementation, they might be time consuming and computationally expensive when evaluating a large dataset. In addition, quality metrics are often designed for well-defined image features and may be unstable for images out of the ideal domain. To overcome these limitations, recent works have proposed deep learning-based IQA methods, which can provide superior performance, increased generalizability and fast prediction. Our method, named $\mathrmμ$DeepIQA, is inspired by previous studies and applies a deep convolutional neural network designed for IQA on natural images to optical microscopy measurements. We retrained the same architecture to predict individual quality metrics and global quality scores for optical microscopy data. The resulting models provide fast and stable predictions of image quality by generalizing quality estimation even outside the ideal range of standard methods. In addition, $\mathrmμ$DeepIQA provides patch-wise prediction of image quality and can be used to visualize spatially varying quality in a single image. Our study demonstrates that optical microscopy-based studies can benefit from the generalizability of deep learning models due to their stable performance in the presence of outliers, the ability to assess small image patches, and rapid predictions.

Method: Decomposes point clouds into semantically coherent patches, encodes them into compact latent representations using semantic-aware encoders conditioned by FiLM, and uses a folding-based decoder guided by latent features and graph node attributes for reconstruction.

Result: Achieves state-of-the-art compression rates (up to 98% data reduction) on SemanticKITTI and nuScenes datasets while preserving structural and semantic fidelity. Supports downstream applications like multi-robot pose graph optimization and map merging with accuracy comparable to raw LiDAR scans.

Conclusion: The semantic scene graph-based deep compression framework enables efficient point cloud transmission for multi-agent robotic systems while maintaining data quality for critical perception tasks.

Abstract: Efficient transmission of 3D point cloud data is critical for advanced perception in centralized and decentralized multi-agent robotic systems, especially nowadays with the growing reliance on edge and cloud-based processing. However, the large and complex nature of point clouds creates challenges under bandwidth constraints and intermittent connectivity, often degrading system performance. We propose a deep compression framework based on semantic scene graphs. The method decomposes point clouds into semantically coherent patches and encodes them into compact latent representations with semantic-aware encoders conditioned by Feature-wise Linear Modulation (FiLM). A folding-based decoder, guided by latent features and graph node attributes, enables structurally accurate reconstruction. Experiments on the SemanticKITTI and nuScenes datasets show that the framework achieves state-of-the-art compression rates, reducing data size by up to 98% while preserving both structural and semantic fidelity. In addition, it supports downstream applications such as multi-robot pose graph optimization and map merging, achieving trajectory accuracy and map alignment comparable to those obtained with raw LiDAR scans.

Method: Proposes a progressive relational learning framework that extends relational learning from binary to n-ary relationships, uses grouped supervision loss for n-ary relational learning (since specific annotations for referred objects are absent), and employs a multi-modal network with hybrid attention mechanisms within scene graphs created with n-ary relationships.

Result: The method outperforms state-of-the-art approaches on ReferIt3D and ScanRefer benchmarks, demonstrating the advantages of n-ary relational perception in 3D localization.

Conclusion: N-ary relational learning is crucial for effective 3D object grounding with natural language, as it captures global perceptual significance that pairwise relationships miss, leading to superior localization performance.

Abstract: Localizing 3D objects using natural language is essential for robotic scene understanding. The descriptions often involve multiple spatial relationships to distinguish similar objects, making 3D-language alignment difficult. Current methods only model relationships for pairwise objects, ignoring the global perceptual significance of n-ary combinations in multi-modal relational understanding. To address this, we propose a novel progressive relational learning framework for 3D object grounding. We extend relational learning from binary to n-ary to identify visual relations that match the referential description globally. Given the absence of specific annotations for referred objects in the training data, we design a grouped supervision loss to facilitate n-ary relational learning. In the scene graph created with n-ary relationships, we use a multi-modal network with hybrid attention mechanisms to further localize the target within the n-ary combinations. Experiments and ablation studies on the ReferIt3D and ScanRefer benchmarks demonstrate that our method outperforms the state-of-the-art, and proves the advantages of the n-ary relational perception in 3D localization.

Method: GaussMedAct framework uses Multivariate Gaussian Representation to project joint motions to temporally scaled space and decompose actions into adaptive 3D Gaussian tokens. Hybrid Spatial Encoding employs Cartesian and Vector dual-stream strategy to utilize skeletal joint and bone features.

Result: Achieves 92.1% Top-1 accuracy on CPREval-6k benchmark, outperforming baselines by +5.9% accuracy with only 10% FLOPs. Cross-dataset experiments confirm robustness superiority.

Conclusion: The proposed GaussMedAct framework with CPREval-6k dataset advances medical motion analysis through adaptive spatiotemporal representation learning, enabling accurate real-time evaluation of fine-grained medical actions.

Abstract: Fine-grained action evaluation in medical vision faces unique challenges due to the unavailability of comprehensive datasets, stringent precision requirements, and insufficient spatiotemporal dynamic modeling of very rapid actions. To support development and evaluation, we introduce CPREval-6k, a multi-view, multi-label medical action benchmark containing 6,372 expert-annotated videos with 22 clinical labels. Using this dataset, we present GaussMedAct, a multivariate Gaussian encoding framework, to advance medical motion analysis through adaptive spatiotemporal representation learning. Multivariate Gaussian Representation projects the joint motions to a temporally scaled multi-dimensional space, and decomposes actions into adaptive 3D Gaussians that serve as tokens. These tokens preserve motion semantics through anisotropic covariance modeling while maintaining robustness to spatiotemporal noise. Hybrid Spatial Encoding, employing a Cartesian and Vector dual-stream strategy, effectively utilizes skeletal information in the form of joint and bone features. The proposed method achieves 92.1% Top-1 accuracy with real-time inference on the benchmark, outperforming baseline by +5.9% accuracy with only 10% FLOPs. Cross-dataset experiments confirm the superiority of our method in robustness.

Method: Recasts normal integration as estimating relative scales of continuous components, constraining pixels in the same component to vary jointly. Includes component estimation heuristics, optimization term rebalancing, and iterative component merging to reduce problem size.

Result: Achieves state-of-the-art results on standard benchmarks in seconds, with one-order-of-magnitude speedup over pixel-level approaches on large-resolution normal maps.

Conclusion: The component-based approach provides an efficient and scalable solution to normal integration, dramatically reducing computational complexity while maintaining high accuracy.

Abstract: Surface normal integration is a fundamental problem in computer vision, dealing with the objective of reconstructing a surface from its corresponding normal map. Existing approaches require an iterative global optimization to jointly estimate the depth of each pixel, which scales poorly to larger normal maps. In this paper, we address this problem by recasting normal integration as the estimation of relative scales of continuous components. By constraining pixels belonging to the same component to jointly vary their scale, we drastically reduce the number of optimization variables. Our framework includes a heuristic to accurately estimate continuous components from the start, a strategy to rebalance optimization terms, and a technique to iteratively merge components to further reduce the size of the problem. Our method achieves state-of-the-art results on the standard normal integration benchmark in as little as a few seconds and achieves one-order-of-magnitude speedup over pixel-level approaches on large-resolution normal maps.

Method: Proposes Completion-by-Correction paradigm: start with topologically complete shape prior from pretrained image-to-3D model, then perform feature-space correction to align with partial observation. PGNet implements this with multi-stage framework: dual-feature encoding to ground generative prior, synthesizing coarse structurally aligned scaffold, and progressive refinement via hierarchical correction.

Result: Experiments on ShapeNetViPC dataset show PGNet outperforms state-of-the-art baselines with 23.5% reduction in average Chamfer Distance and 7.1% improvement in F-score.

Conclusion: The Completion-by-Correction paradigm provides a more robust approach to point cloud completion by shifting from unconstrained synthesis to guided refinement, enabling structurally consistent and observation-aligned reconstruction.

Abstract: Point cloud completion aims to reconstruct complete 3D shapes from partial observations, which is a challenging problem due to severe occlusions and missing geometry. Despite recent advances in multimodal techniques that leverage complementary RGB images to compensate for missing geometry, most methods still follow a Completion-by-Inpainting paradigm, synthesizing missing structures from fused latent features. We empirically show that this paradigm often results in structural inconsistencies and topological artifacts due to limited geometric and semantic constraints. To address this, we rethink the task and propose a more robust paradigm, termed Completion-by-Correction, which begins with a topologically complete shape prior generated by a pretrained image-to-3D model and performs feature-space correction to align it with the partial observation. This paradigm shifts completion from unconstrained synthesis to guided refinement, enabling structurally consistent and observation-aligned reconstruction. Building upon this paradigm, we introduce PGNet, a multi-stage framework that conducts dual-feature encoding to ground the generative prior, synthesizes a coarse yet structurally aligned scaffold, and progressively refines geometric details via hierarchical correction. Experiments on the ShapeNetViPC dataset demonstrate the superiority of PGNet over state-of-the-art baselines in terms of average Chamfer Distance (-23.5%) and F-score (+7.1%).

Method: CORE has three main components: 1) Multi-granular Environment Perception Module using LLMs to distill environmental semantics from POI distributions, 2) Route Choice Encoder with MoE architecture to capture route choice patterns using context-enriched road network and navigational factors, and 3) Transformer encoder to aggregate representations into global trajectory embeddings.

Result: Extensive experiments on 4 real-world datasets across 6 downstream tasks show CORE consistently outperforms 12 state-of-the-art TRL methods, achieving an average improvement of 9.79% over the best-performing baseline.

Conclusion: CORE successfully bridges the gap by integrating context-aware route choice semantics into trajectory embeddings, demonstrating superior performance across multiple downstream tasks and providing a more semantically rich representation of trajectories.

Abstract: Trajectory representation learning (TRL) aims to encode raw trajectory data into low-dimensional embeddings for downstream tasks such as travel time estimation, mobility prediction, and trajectory similarity analysis. From a behavioral perspective, a trajectory reflects a sequence of route choices within an urban environment. However, most existing TRL methods ignore this underlying decision-making process and instead treat trajectories as static, passive spatiotemporal sequences, thereby limiting the semantic richness of the learned representations. To bridge this gap, we propose CORE, a TRL framework that integrates context-aware route choice semantics into trajectory embeddings. CORE first incorporates a multi-granular Environment Perception Module, which leverages large language models (LLMs) to distill environmental semantics from point of interest (POI) distributions, thereby constructing a context-enriched road network. Building upon this backbone, CORE employs a Route Choice Encoder with a mixture-of-experts (MoE) architecture, which captures route choice patterns by jointly leveraging the context-enriched road network and navigational factors. Finally, a Transformer encoder aggregates the route-choice-aware representations into a global trajectory embedding. Extensive experiments on 4 real-world datasets across 6 downstream tasks demonstrate that CORE consistently outperforms 12 state-of-the-art TRL methods, achieving an average improvement of 9.79% over the best-performing baseline. Our code is available at https://github.com/caoji2001/CORE.

Method: Proposes a three-axis taxonomy: (1) Functionality (Decision-Coupled vs. General-Purpose), (2) Temporal Modeling (Sequential Simulation vs. Global Difference Prediction), (3) Spatial Representation (Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, Decomposed Rendering). Systematizes data resources and metrics across robotics, autonomous driving, and general video settings.

Result: Provides quantitative comparison of state-of-the-art models and identifies key open challenges: scarcity of unified datasets, need for evaluation metrics assessing physical consistency over pixel fidelity, trade-off between model performance and computational efficiency, and difficulty achieving long-horizon temporal consistency while mitigating error accumulation.

Conclusion: World models are crucial for embodied AI as internal simulators. The survey offers a unified framework and taxonomy to systematize the field, identifies current limitations, and maintains a curated bibliography to support future research in developing more effective world models for real-world embodied AI applications.

Abstract: Embodied AI requires agents that perceive, act, and anticipate how actions reshape future world states. World models serve as internal simulators that capture environment dynamics, enabling forward and counterfactual rollouts to support perception, prediction, and decision making. This survey presents a unified framework for world models in embodied AI. Specifically, we formalize the problem setting and learning objectives, and propose a three-axis taxonomy encompassing: (1) Functionality, Decision-Coupled vs. General-Purpose; (2) Temporal Modeling, Sequential Simulation and Inference vs. Global Difference Prediction; (3) Spatial Representation, Global Latent Vector, Token Feature Sequence, Spatial Latent Grid, and Decomposed Rendering Representation. We systematize data resources and metrics across robotics, autonomous driving, and general video settings, covering pixel prediction quality, state-level understanding, and task performance. Furthermore, we offer a quantitative comparison of state-of-the-art models and distill key open challenges, including the scarcity of unified datasets and the need for evaluation metrics that assess physical consistency over pixel fidelity, the trade-off between model performance and the computational efficiency required for real-time control, and the core modeling difficulty of achieving long-horizon temporal consistency while mitigating error accumulation. Finally, we maintain a curated bibliography at https://github.com/Li-Zn-H/AwesomeWorldModels.

Method: Created CrossVid benchmark with hierarchical tasks across four high-level dimensions and ten specific tasks. Includes 5,331 videos and 9,015 challenging QA pairs in single-choice, multiple-choice, and open-ended formats to comprehensively evaluate spatial-temporal reasoning in cross-video contexts.

Result: Gemini-2.5-Pro performed best with 50.4% average accuracy. Most current MLLMs struggle with cross-video reasoning tasks due to inability to integrate or compare evidence distributed across multiple videos for reasoning.

Conclusion: CrossVid reveals significant limitations in current MLLMs’ cross-video reasoning capabilities and provides a comprehensive benchmark to guide future advancements in enhancing MLLMs’ ability to reason across multiple videos.

Abstract: Cross-Video Reasoning (CVR) presents a significant challenge in video understanding, which requires simultaneous understanding of multiple videos to aggregate and compare information across groups of videos. Most existing video understanding benchmarks focus on single-video analysis, failing to assess the ability of multimodal large language models (MLLMs) to simultaneously reason over various videos. Recent benchmarks evaluate MLLMs’ capabilities on multi-view videos that capture different perspectives of the same scene. However, their limited tasks hinder a thorough assessment of MLLMs in diverse real-world CVR scenarios. To this end, we introduce CrossVid, the first benchmark designed to comprehensively evaluate MLLMs’ spatial-temporal reasoning ability in cross-video contexts. Firstly, CrossVid encompasses a wide spectrum of hierarchical tasks, comprising four high-level dimensions and ten specific tasks, thereby closely reflecting the complex and varied nature of real-world video understanding. Secondly, CrossVid provides 5,331 videos, along with 9,015 challenging question-answering pairs, spanning single-choice, multiple-choice, and open-ended question formats. Through extensive experiments on various open-source and closed-source MLLMs, we observe that Gemini-2.5-Pro performs best on CrossVid, achieving an average accuracy of 50.4%. Notably, our in-depth case study demonstrates that most current MLLMs struggle with CVR tasks, primarily due to their inability to integrate or compare evidence distributed across multiple videos for reasoning. These insights highlight the potential of CrossVid to guide future advancements in enhancing MLLMs’ CVR capabilities.

Method: AMR uses a two-stage training framework: 1) Cold-start stage employs curriculum learning on augmented data to build foundational boundary/semantic awareness. 2) Distillation stage introduces dual query sets: Original Queries maintain DETR-based localization using frozen Base Queries, while Active Queries dynamically adapt to real-data distributions. A cross-stage distillation loss enforces consistency between Original and Base Queries to prevent knowledge forgetting while enabling real-world generalization.

Result: Experiments on multiple benchmarks show that AMR achieves improved performance over prior state-of-the-art approaches, demonstrating effectiveness in overcoming the identified bottlenecks.

Conclusion: AMR successfully addresses key limitations in moment retrieval by resolving ambiguous boundary information and semantic confusion without additional labeled data, while preserving enhanced boundary and semantic discriminative capabilities that generalize to real-world scenarios.

Abstract: Existing Moment Retrieval methods face three critical bottlenecks: (1) data scarcity forces models into shallow keyword-feature associations; (2) boundary ambiguity in transition regions between adjacent events; (3) insufficient discrimination of fine-grained semantics (e.g., distinguishing kicking" vs. throwing" a ball). In this paper, we propose a zero-external-dependency Augmented Moment Retrieval framework, AMR, designed to overcome local optima caused by insufficient data annotations and the lack of robust boundary and semantic discrimination capabilities. AMR is built upon two key insights: (1) it resolves ambiguous boundary information and semantic confusion in existing annotations without additional data (avoiding costly manual labeling), and (2) it preserves boundary and semantic discriminative capabilities enhanced by training while generalizing to real-world scenarios, significantly improving performance. Furthermore, we propose a two-stage training framework with cold-start and distillation adaptation. The cold-start stage employs curriculum learning on augmented data to build foundational boundary/semantic awareness. The distillation stage introduces dual query sets: Original Queries maintain DETR-based localization using frozen Base Queries from the cold-start model, while Active Queries dynamically adapt to real-data distributions. A cross-stage distillation loss enforces consistency between Original and Base Queries, preventing knowledge forgetting while enabling real-world generalization. Experiments on multiple benchmarks show that AMR achieves improved performance over prior state-of-the-art approaches.

Method: Two-stage hybrid framework: 1) Physics-based reverse time migration (time-reversal acoustics) on multi-angle acquisitions to produce migration fragments preserving structural details under low SNR; 2) Transformer-based super-resolution encoder-decoder with graph-based attention unit (GAU) to fuse fragments into coherent, accurate SoS images. Uses partial-array acquisition with movable low-count transducers for improved feasibility.

Result: Experiments on two synthetic datasets show BrainPuzzle achieves superior SoS reconstruction accuracy and image completeness compared to existing methods, demonstrating potential for advancing quantitative ultrasound brain imaging.

Conclusion: BrainPuzzle successfully addresses limitations of both physics-based and purely data-driven approaches by combining their strengths, enabling accurate quantitative transcranial ultrasound imaging even under challenging conditions of low SNR and sparse aperture coverage.

Abstract: Ultrasound brain imaging remains challenging due to the large difference in sound speed between the skull and brain tissues and the difficulty of coupling large probes to the skull. This work aims to achieve quantitative transcranial ultrasound by reconstructing an accurate speed-of-sound (SoS) map of the brain. Traditional physics-based full-waveform inversion (FWI) is limited by weak signals caused by skull-induced attenuation, mode conversion, and phase aberration, as well as incomplete spatial coverage since full-aperture arrays are clinically impractical. In contrast, purely data-driven methods that learn directly from raw ultrasound data often fail to model the complex nonlinear and nonlocal wave propagation through bone, leading to anatomically plausible but quantitatively biased SoS maps under low signal-to-noise and sparse-aperture conditions. To address these issues, we propose BrainPuzzle, a hybrid two-stage framework that combines physical modeling with machine learning. In the first stage, reverse time migration (time-reversal acoustics) is applied to multi-angle acquisitions to produce migration fragments that preserve structural details even under low SNR. In the second stage, a transformer-based super-resolution encoder-decoder with a graph-based attention unit (GAU) fuses these fragments into a coherent and quantitatively accurate SoS image. A partial-array acquisition strategy using a movable low-count transducer set improves feasibility and coupling, while the hybrid algorithm compensates for the missing aperture. Experiments on two synthetic datasets show that BrainPuzzle achieves superior SoS reconstruction accuracy and image completeness, demonstrating its potential for advancing quantitative ultrasound brain imaging.

Method: HARMONY integrates three components at token level: generated text tokens, model’s uncertainty score (MaxProb), and hidden representations capturing visual understanding. It uses appropriate input mapping design and suitable architecture to process these signals effectively, addressing inter-token dependencies and vision-text alignment.

Result: Experiments across two open-ended VQA benchmarks (A-OKVQA and VizWiz) and four VLMs (LLaVA-7B, LLaVA-13B, InstructBLIP, and Qwen-VL) show HARMONY consistently matches or surpasses existing approaches, achieving up to 5% improvement in AUROC and 9% in PRR.

Conclusion: HARMONY demonstrates that effectively processing multimodal alignment signals in hidden representations significantly improves uncertainty estimation performance for vision-language models, providing a more reliable framework for quantifying model output reliability.

Abstract: Uncertainty Estimation (UE) plays a central role in quantifying the reliability of model outputs and reducing unsafe generations via selective prediction. In this regard, most existing probability-based UE approaches rely on predefined functions, aggregating token probabilities into a single UE score using heuristics such as length-normalization. However, these methods often fail to capture the complex relationships between generated tokens and struggle to identify biased probabilities often influenced by \textbf{language priors}. Another line of research uses hidden representations of the model and trains simple MLP architectures to predict uncertainty. However, such functions often lose the intricate \textbf{ inter-token dependencies}. While prior works show that hidden representations encode multimodal alignment signals, our work demonstrates that how these signals are processed has a significant impact on the UE performance. To effectively leverage these signals to identify inter-token dependencies, and vision-text alignment, we propose \textbf{HARMONY} (Hidden Activation Representations and Model Output-Aware Uncertainty Estimation for Vision-Language Models), a novel UE framework that integrates generated tokens (’text’), model’s uncertainty score at the output (‘MaxProb’), and its internal belief on the visual understanding of the image and the generated token (captured by ‘hidden representations’) at token level via appropriate input mapping design and suitable architecture choice. Our experimental experiments across two open-ended VQA benchmarks (A-OKVQA, and VizWiz) and four state-of-the-art VLMs (LLaVA-7B, LLaVA-13B, InstructBLIP, and Qwen-VL) show that HARMONY consistently matches or surpasses existing approaches, achieving up to 5% improvement in AUROC and 9% in PRR.

Method: Uses compact sparse voxel spectrum to represent tree movement; generates mesh motion from 3D Gaussian Splatting reconstructions, then binds Gaussians to deform mesh; enables fast modal analysis for real-time interactive responses to external forces.

Result: Achieves realistic and responsive tree animations, significantly outperforming existing approaches in both visual quality and computational efficiency.

Conclusion: DynamicTree is the first framework for generating long-term, interactive 3D motion for real tree reconstructions, enabled by sparse voxel spectrum representation and supported by the new 4DTree dataset.

Abstract: Generating dynamic and interactive 3D trees has wide applications in virtual reality, games, and world simulation. However, existing methods still face various challenges in generating structurally consistent and realistic 4D motion for complex real trees. In this paper, we propose DynamicTree, the first framework that can generate long-term, interactive 3D motion for 3DGS reconstructions of real trees. Unlike prior optimization-based methods, our approach generates dynamics in a fast feed-forward manner. The key success of our approach is the use of a compact sparse voxel spectrum to represent the tree movement. Given a 3D tree from Gaussian Splatting reconstruction, our pipeline first generates mesh motion using the sparse voxel spectrum and then binds Gaussians to deform the mesh. Additionally, the proposed sparse voxel spectrum can also serve as a basis for fast modal analysis under external forces, allowing real-time interactive responses. To train our model, we also introduce 4DTree, the first large-scale synthetic 4D tree dataset containing 8,786 animated tree meshes with 100-frame motion sequences. Extensive experiments demonstrate that our method achieves realistic and responsive tree animations, significantly outperforming existing approaches in both visual quality and computational efficiency.

Method: Proposes AttnRegDeepLab with dual-branch MTL: 1) Modified DeepLabV3+ decoder with Attention Gates in skip connections to suppress cytoplasmic noise, 2) Multi-Scale Regression Head with Feature Injection to propagate grading priors, 3) 2-stage decoupled training to address gradient conflict, 4) Range-based loss for weakly labeled data.

Result: Achieves robust grading precision while maintaining excellent segmentation accuracy (Dice coefficient = 0.729), outperforming end-to-end counterparts that sacrifice contour integrity for grading error minimization.

Conclusion: Provides a clinically interpretable solution that balances visual fidelity and quantitative precision for embryo fragmentation assessment in IVF.

Abstract: Embryo fragmentation is a morphological indicator critical for evaluating developmental potential in In Vitro Fertilization (IVF). However, manual grading is subjective and inefficient, while existing deep learning solutions often lack clinical explainability or suffer from accumulated errors in segmentation area estimation. To address these issues, this study proposes AttnRegDeepLab (Attention-Guided Regression DeepLab), a framework characterized by dual-branch Multi-Task Learning (MTL). A vanilla DeepLabV3+ decoder is modified by integrating Attention Gates into its skip connections, explicitly suppressing cytoplasmic noise to preserve contour details. Furthermore, a Multi-Scale Regression Head is introduced with a Feature Injection mechanism to propagate global grading priors into the segmentation task, rectifying systematic quantification errors. A 2-stage decoupled training strategy is proposed to address the gradient conflict in MTL. Also, a range-based loss is designed to leverage weakly labeled data. Our method achieves robust grading precision while maintaining excellent segmentation accuracy (Dice coefficient =0.729), in contrast to the end-to-end counterpart that might minimize grading error at the expense of contour integrity. This work provides a clinically interpretable solution that balances visual fidelity and quantitative precision.

Method: PRISM-Bench introduces visual puzzles requiring multi-step symbolic, geometric, and analogical reasoning. The diagnostic task presents a puzzle with a step-by-step chain-of-thought containing exactly one error, and models must identify the first incorrect step. This approach resists shortcuts based on superficial pattern matching.

Result: Evaluations across state-of-the-art MLLMs reveal a persistent gap between fluent generation and faithful reasoning. Models that produce plausible chain-of-thoughts often fail to locate simple logical faults, highlighting the disconnect between answer generation and reasoning verification.

Conclusion: PRISM-Bench offers a sharper lens on multimodal reasoning competence by disentangling answer generation from reasoning verification. The benchmark underscores the need for diagnostic evaluation protocols in developing trustworthy MLLMs that can reliably reason and detect errors.

Abstract: Multimodal large language models (MLLMs) have achieved remarkable progress on vision-language tasks, yet their reasoning processes remain sometimes unreliable. We introduce PRISM-Bench, a benchmark of puzzle-based visual challenges designed to evaluate not only whether models can solve problems, but how their reasoning unfolds. Unlike prior evaluations that measure only final-answer accuracy, PRISM-Bench introduces a diagnostic task: given a visual puzzle and a step-by-step chain-of-thought (CoT) containing exactly one error, models must identify the first incorrect step. This setting enables fine-grained assessment of logical consistency, error detection, and visual reasoning. The puzzles in PRISM-Bench require multi-step symbolic, geometric, and analogical reasoning, resisting shortcuts based on superficial pattern matching. Evaluations across state-of-the-art MLLMs reveal a persistent gap between fluent generation and faithful reasoning: models that produce plausible CoTs often fail to locate simple logical faults. By disentangling answer generation from reasoning verification, PRISM-Bench offers a sharper lens on multimodal reasoning competence and underscores the need for diagnostic evaluation protocols in the development of trustworthy MLLMs.

Method: Developed a scalable annotation pipeline combining vision-language model prompting with human verification, plus platform-aware normalization and cross-modal alignment techniques for cross-platform learning.

Result: Created a dataset 10x larger than existing ones, with significant performance gaps revealed in evaluations, highlighting challenges in generalizable 3D grounding across platforms.

Conclusion: 3EED advances language-driven 3D embodied perception research by providing a comprehensive benchmark for cross-platform visual grounding in outdoor environments.

Abstract: Visual grounding in 3D is the key for embodied agents to localize language-referred objects in open-world environments. However, existing benchmarks are limited to indoor focus, single-platform constraints, and small scale. We introduce 3EED, a multi-platform, multi-modal 3D grounding benchmark featuring RGB and LiDAR data from vehicle, drone, and quadruped platforms. We provide over 128,000 objects and 22,000 validated referring expressions across diverse outdoor scenes – 10x larger than existing datasets. We develop a scalable annotation pipeline combining vision-language model prompting with human verification to ensure high-quality spatial grounding. To support cross-platform learning, we propose platform-aware normalization and cross-modal alignment techniques, and establish benchmark protocols for in-domain and cross-platform evaluations. Our findings reveal significant performance gaps, highlighting the challenges and opportunities of generalizable 3D grounding. The 3EED dataset and benchmark toolkit are released to advance future research in language-driven 3D embodied perception.

Method: Two modules: 1) Mixture of Ego-Graphs Fusion (MoEGF) constructs ego graphs and uses Mixture-of-Experts network for fine-grained fusion at sample level; 2) Ego Graph Contrastive Learning (EGCL) aligns fused representation with view-specific representation, enhancing similarity of samples from same cluster.

Result: Extensive experiments show MoEGCL achieves state-of-the-art results in deep multi-view clustering tasks.

Conclusion: MoEGCL addresses coarse-grained fusion limitation through sample-level ego-graph fusion and contrastive learning, significantly improving multi-view clustering performance.

Abstract: In recent years, the advancement of Graph Neural Networks (GNNs) has significantly propelled progress in Multi-View Clustering (MVC). However, existing methods face the problem of coarse-grained graph fusion. Specifically, current approaches typically generate a separate graph structure for each view and then perform weighted fusion of graph structures at the view level, which is a relatively rough strategy. To address this limitation, we present a novel Mixture of Ego-Graphs Contrastive Representation Learning (MoEGCL). It mainly consists of two modules. In particular, we propose an innovative Mixture of Ego-Graphs Fusion (MoEGF), which constructs ego graphs and utilizes a Mixture-of-Experts network to implement fine-grained fusion of ego graphs at the sample level, rather than the conventional view-level fusion. Additionally, we present the Ego Graph Contrastive Learning (EGCL) module to align the fused representation with the view-specific representation. The EGCL module enhances the representation similarity of samples from the same cluster, not merely from the same sample, further boosting fine-grained graph representation. Extensive experiments demonstrate that MoEGCL achieves state-of-the-art results in deep multi-view clustering tasks. The source code is publicly available at https://github.com/HackerHyper/MoEGCL.

Method: Introduces Chain-of-Visual-Thought (COVT) framework that enables VLMs to reason through continuous visual tokens (compact latent representations encoding rich perceptual cues). Distills knowledge from lightweight vision experts to capture 2D appearance, 3D geometry, spatial layout, and edge structure within ~20 tokens. During training, VLMs autoregressively predict visual tokens to reconstruct dense supervision signals (depth, segmentation, edges, DINO features).

Result: Integration into strong VLMs (Qwen2.5-VL and LLaVA) consistently improves performance by 3% to 16% across more than ten diverse perception benchmarks including CV-Bench, MMVP, RealWorldQA, MMStar, WorldMedQA, and HRBench.

Conclusion: Compact continuous visual thinking enables more precise, grounded, and interpretable multimodal intelligence while preserving efficiency, with optional dense prediction decoding for interpretability.

Abstract: Vision-Language Models (VLMs) excel at reasoning in linguistic space but struggle with perceptual understanding that requires dense visual perception, e.g., spatial reasoning and geometric awareness. This limitation stems from the fact that current VLMs have limited mechanisms to capture dense visual information across spatial dimensions. We introduce Chain-of-Visual-Thought (COVT), a framework that enables VLMs to reason not only in words but also through continuous visual tokens-compact latent representations that encode rich perceptual cues. Within a small budget of roughly 20 tokens, COVT distills knowledge from lightweight vision experts, capturing complementary properties such as 2D appearance, 3D geometry, spatial layout, and edge structure. During training, the VLM with COVT autoregressively predicts these visual tokens to reconstruct dense supervision signals (e.g., depth, segmentation, edges, and DINO features). At inference, the model reasons directly in the continuous visual token space, preserving efficiency while optionally decoding dense predictions for interpretability. Evaluated across more than ten diverse perception benchmarks, including CV-Bench, MMVP, RealWorldQA, MMStar, WorldMedQA, and HRBench, integrating COVT into strong VLMs such as Qwen2.5-VL and LLaVA consistently improves performance by 3% to 16% and demonstrates that compact continuous visual thinking enables more precise, grounded, and interpretable multimodal intelligence.

Method: Progressive training framework that integrates physics-informed PDE dynamics into state-of-the-art restoration architectures. Uses advection-diffusion equations to model feature evolution, capturing directional flow characteristics of motion blur while enabling principled global spatial modeling.

Result: PDE-enhanced deblurring models achieve superior restoration quality with minimal overhead (only ~1% increase in inference GMACs). Consistent improvements in perceptual quality across multiple architectures (FFTformer, NAFNet, Restormer, Stripformer). Significant PSNR and SSIM improvements on standard motion deblurring benchmarks.

Conclusion: Incorporating mathematical physics principles through PDE-based global layers enhances deep learning-based image restoration. Establishes a promising direction for physics-informed neural network design in computer vision applications.

Abstract: Motion blur, caused by relative movement between camera and scene during exposure, significantly degrades image quality and impairs downstream computer vision tasks such as object detection, tracking, and recognition in dynamic environments. While deep learning-based motion deblurring methods have achieved remarkable progress, existing approaches face fundamental challenges in capturing the long-range spatial dependencies inherent in motion blur patterns. Traditional convolutional methods rely on limited receptive fields and require extremely deep networks to model global spatial relationships. These limitations motivate the need for alternative approaches that incorporate physical priors to guide feature evolution during restoration. In this paper, we propose a progressive training framework that integrates physics-informed PDE dynamics into state-of-the-art restoration architectures. By leveraging advection-diffusion equations to model feature evolution, our approach naturally captures the directional flow characteristics of motion blur while enabling principled global spatial modeling. Our PDE-enhanced deblurring models achieve superior restoration quality with minimal overhead, adding only approximately 1% to inference GMACs while providing consistent improvements in perceptual quality across multiple state-of-the-art architectures. Comprehensive experiments on standard motion deblurring benchmarks demonstrate that our physics-informed approach improves PSNR and SSIM significantly across four diverse architectures, including FFTformer, NAFNet, Restormer, and Stripformer. These results validate that incorporating mathematical physics principles through PDE-based global layers can enhance deep learning-based image restoration, establishing a promising direction for physics-informed neural network design in computer vision applications.

Method: Designed a small PID convolutional neural network (17 layers, 413K parameters) with feature reuse via concatenation and pooling. Analyzed neural networks as second-order difference equations with nonlinearities. Trained and tested on simulated randomly moving ball datasets.

Result: Prediction accuracy approached the upper limit that input image resolution can represent. Demonstrated practical feature reuse and analyzed experimental errors, shortcomings, and improvement directions.

Conclusion: High-dimensional convolution improves computational efficiency and feature space utilization. PID information has potential advantages for implementing memory and attention mechanisms in neural networks.

Abstract: This article trained a network for perceiving three-dimensional motion information of binocular vision target, which can provide real-time three-dimensional coordinate, velocity, and acceleration, and has a basic spatiotemporal perception capability. Understood the ability of neural networks to fit nonlinear problems from the perspective of PID. Considered a single-layer neural network as using a second-order difference equation and a nonlinearity to describe a local problem. Multilayer networks gradually transform the raw representation to the desired representation through multiple such combinations. Analysed some reference principles for designing neural networks. Designed a relatively small PID convolutional neural network, with a total of 17 layers and 413 thousand parameters. Implemented a simple but practical feature reuse method by concatenation and pooling. The network was trained and tested using the simulated randomly moving ball datasets, and the experimental results showed that the prediction accuracy was close to the upper limit that the input image resolution can represent. Analysed the experimental results and errors, as well as the existing shortcomings and possible directions for improvement. Finally, discussed the advantages of high-dimensional convolution in improving computational efficiency and feature space utilization. As well as the potential advantages of using PID information to implement memory and attention mechanisms.

Method: Two-step “Semantic Erasure-Injection” process: 1) Erase unwanted semantics from noise using information theory principles, 2) Inject desired semantics into cleaned noise by leveraging the equivalence between diffusion generation process and semantic injection.

Result: Method is consistently effective across various T2C models (both DiT and UNet architectures), training-free, and provides a universal tool for consistent generation optimization.

Conclusion: Initial noise in diffusion models contains analyzable semantic patterns; the proposed noise modulation method offers a novel perspective for optimizing diffusion model generation and enables universal consistent generation control.

Abstract: In text-driven content generation (T2C) diffusion model, semantic of generated content is mostly attributed to the process of text embedding and attention mechanism interaction. The initial noise of the generation process is typically characterized as a random element that contributes to the diversity of the generated content. Contrary to this view, this paper reveals that beneath the random surface of noise lies strong analyzable patterns. Specifically, this paper first conducts a comprehensive analysis of the impact of random noise on the model’s generation. We found that noise not only contains rich semantic information, but also allows for the erasure of unwanted semantics from it in an extremely simple way based on information theory, and using the equivalence between the generation process of diffusion model and semantic injection to inject semantics into the cleaned noise. Then, we mathematically decipher these observations and propose a simple but efficient training-free and universal two-step “Semantic Erasure-Injection” process to modulate the initial noise in T2C diffusion model. Experimental results demonstrate that our method is consistently effective across various T2C models based on both DiT and UNet architectures and presents a novel perspective for optimizing the generation of diffusion model, providing a universal tool for consistent generation.

Method: UPA-RFAS combines: 1) Feature-space objective with ℓ₁ deviation prior and repulsive InfoNCE loss for transferable representation shifts; 2) Robustness-augmented two-phase min-max procedure with inner loop for invisible perturbations and outer loop for universal patch optimization; 3) Two VLA-specific losses: Patch Attention Dominance to hijack text→vision attention and Patch Semantic Misalignment to induce image-text mismatch.

Result: Experiments show UPA-RFAS consistently transfers across diverse VLA models, manipulation suites, and physical executions. It works across models, tasks, and viewpoints, exposing a practical patch-based attack surface.

Conclusion: UPA-RFAS establishes a strong baseline for universal adversarial attacks on VLA-driven robots, demonstrating transferability across unknown architectures and sim-to-real shifts, highlighting security vulnerabilities in VLA systems.

Abstract: Vision-Language-Action (VLA) models are vulnerable to adversarial attacks, yet universal and transferable attacks remain underexplored, as most existing patches overfit to a single model and fail in black-box settings. To address this gap, we present a systematic study of universal, transferable adversarial patches against VLA-driven robots under unknown architectures, finetuned variants, and sim-to-real shifts. We introduce UPA-RFAS (Universal Patch Attack via Robust Feature, Attention, and Semantics), a unified framework that learns a single physical patch in a shared feature space while promoting cross-model transfer. UPA-RFAS combines (i) a feature-space objective with an $\ell_1$ deviation prior and repulsive InfoNCE loss to induce transferable representation shifts, (ii) a robustness-augmented two-phase min-max procedure where an inner loop learns invisible sample-wise perturbations and an outer loop optimizes the universal patch against this hardened neighborhood, and (iii) two VLA-specific losses: Patch Attention Dominance to hijack text$\to$vision attention and Patch Semantic Misalignment to induce image-text mismatch without labels. Experiments across diverse VLA models, manipulation suites, and physical executions show that UPA-RFAS consistently transfers across models, tasks, and viewpoints, exposing a practical patch-based attack surface and establishing a strong baseline for future defenses.

Method: Multiple diffusion-finetuning techniques: LoRA and DreamBooth, with clustering within groups to prevent DreamBooth models from being overwhelmed by intra-group variations. Generate group-balanced data for pretraining, then fine-tune on real data.

Result: Finetuning approaches outperform vanilla Stable Diffusion on average and achieve results comparable to SOTA debiasing techniques like Group-DRO, while surpassing them as dataset bias severity increases.

Conclusion: Diffusion model finetuning techniques effectively generate balanced training data for debiasing, with performance improving relative to existing methods as bias severity increases.

Abstract: Image classification systems often inherit biases from uneven group representation in training data. For example, in face datasets for hair color classification, blond hair may be disproportionately associated with females, reinforcing stereotypes. A recent approach leverages the Stable Diffusion model to generate balanced training data, but these models often struggle to preserve the original data distribution. In this work, we explore multiple diffusion-finetuning techniques, e.g., LoRA and DreamBooth, to generate images that more accurately represent each training group by learning directly from their samples. Additionally, in order to prevent a single DreamBooth model from being overwhelmed by excessive intra-group variations, we explore a technique of clustering images within each group and train a DreamBooth model per cluster. These models are then used to generate group-balanced data for pretraining, followed by fine-tuning on real data. Experiments on multiple benchmarks demonstrate that the studied finetuning approaches outperform vanilla Stable Diffusion on average and achieve results comparable to SOTA debiasing techniques like Group-DRO, while surpassing them as the dataset bias severity increases.

Method: Three-stage pipeline: 1) Contextual object descriptions guide localization using vision-language understanding, 2) Self-cross-attention mechanisms identify candidate regions and produce precise masks, 3) Automatic re-registration detects tracking failures through feature similarity monitoring and recovers from severe occlusions/rapid motion.

Result: Achieves state-of-the-art accuracy on challenging industrial datasets with multi-object occlusions, high-speed motion, and varying illumination. Operates at real-time speeds without requiring additional training.

Conclusion: STORM provides a practical, annotation-free solution that significantly reduces deployment overhead for applications like flexible manufacturing and intelligent quality control, overcoming limitations of traditional pose estimation methods.

Abstract: Accurate 6D pose estimation and tracking are fundamental capabilities for physical AI systems such as robots. However, existing approaches typically require a pre-defined 3D model of the target and rely on a manually annotated segmentation mask in the first frame, which is labor-intensive and leads to reduced performance when faced with occlusions or rapid movement. To address these limitations, we propose STORM (Segment, Track, and Object Re-localization from a single iMage), an open-source robust real-time 6D pose estimation system that requires no manual annotation. STORM employs a novel three-stage pipeline combining vision-language understanding with feature matching: contextual object descriptions guide localization, self-cross-attention mechanisms identify candidate regions, and produce precise masks and 3D models for accurate pose estimation. Another key innovation is our automatic re-registration mechanism that detects tracking failures through feature similarity monitoring and recovers from severe occlusions or rapid motion. STORM achieves state-of-the-art accuracy on challenging industrial datasets featuring multi-object occlusions, high-speed motion, and varying illumination, while operating at real-time speeds without additional training. This annotation-free approach significantly reduces deployment overhead, providing a practical solution for modern applications, such as flexible manufacturing and intelligent quality control.

Method: Two-stage approach: 1) Noise-to-Coarse (N2C) Module denoises input point clouds in single pass, eliminating iterative sampling; 2) Refine Module uses coarse output and intermediate features for precise refinement. Also introduces Distance-aware Selected Repeat strategy to handle LiDAR’s distance-dependent spatial distribution.

Result: On SemanticKITTI: 199.8x inference speedup, 50.7% reduction in Chamfer Distance, uses only 6.1% of parameters compared to LiDiff diffusion model.

Conclusion: LiNeXt demonstrates superior efficiency and effectiveness for real-time scene completion, making it practical for autonomous vehicle perception systems.

Abstract: 3D LiDAR scene completion from point clouds is a fundamental component of perception systems in autonomous vehicles. Previous methods have predominantly employed diffusion models for high-fidelity reconstruction. However, their multi-step iterative sampling incurs significant computational overhead, limiting its real-time applicability. To address this, we propose LiNeXt-a lightweight, non-diffusion network optimized for rapid and accurate point cloud completion. Specifically, LiNeXt first applies the Noise-to-Coarse (N2C) Module to denoise the input noisy point cloud in a single pass, thereby obviating the multi-step iterative sampling of diffusion-based methods. The Refine Module then takes the coarse point cloud and its intermediate features from the N2C Module to perform more precise refinement, further enhancing structural completeness. Furthermore, we observe that LiDAR point clouds exhibit a distance-dependent spatial distribution, being densely sampled at proximal ranges and sparsely sampled at distal ranges. Accordingly, we propose the Distance-aware Selected Repeat strategy to generate a more uniformly distributed noisy point cloud. On the SemanticKITTI dataset, LiNeXt achieves a 199.8x speedup in inference, reduces Chamfer Distance by 50.7%, and uses only 6.1% of the parameters compared with LiDiff. These results demonstrate the superior efficiency and effectiveness of LiNeXt for real-time scene completion.

Method: GFT learns a dynamic graph from initial tokenized inputs using lightweight graph convolution, then passes graph features to deeper layers via skip connections and efficient cross-attention modules.

Result: Extensive experiments on object classification and segmentation show GFT rivals existing methods while reducing trainable parameters.

Conclusion: GFT provides an effective point-cloud-specific PEFT approach that significantly reduces parameter count while maintaining competitive performance on 3D vision tasks.

Abstract: Parameter-efficient fine-tuning (PEFT) significantly reduces computational and memory costs by updating only a small subset of the model’s parameters, enabling faster adaptation to new tasks with minimal loss in performance. Previous studies have introduced PEFTs tailored for point cloud data, as general approaches are suboptimal. To further reduce the number of trainable parameters, we propose a point-cloud-specific PEFT, termed Graph Features Tuning (GFT), which learns a dynamic graph from initial tokenized inputs of the transformer using a lightweight graph convolution network and passes these graph features to deeper layers via skip connections and efficient cross-attention modules. Extensive experiments on object classification and segmentation tasks show that GFT operates in the same domain, rivalling existing methods, while reducing the trainable parameters. Code is available at https://github.com/manishdhakal/GFT.

Method: MultiTypeFCDD uses a simple convolutional framework that learns from image-level labels to produce multi-channel heatmaps, where each channel corresponds to a specific anomaly type. It functions as a single unified framework capable of differentiating anomaly types across multiple object categories without needing separate models for each category.

Result: The method achieves 96.4% I-AUROC on the Real-IAD dataset while being only about 1% the size of state-of-the-art VLM models used for similar tasks, making it highly practical for resource-constrained applications.

Conclusion: MultiTypeFCDD provides a practical, lightweight alternative for explainable multi-type anomaly detection that addresses the limitations of existing methods by offering type differentiation, unified multi-category capability, and computational efficiency suitable for real-world applications.

Abstract: Explainable anomaly detection methods often have the capability to identify and spatially localise anomalies within an image but lack the capability to differentiate the type of anomaly. Furthermore, they often require the costly training and maintenance of separate models for each object category. The lack of specificity is a significant research gap because identifying the type of anomaly (e.g., “Crack” vs. “Scratch”) is crucial for accurate diagnosis that facilitates cost-saving operational decisions across diverse application domains. While some recent large-scale Vision-Language Models (VLMs) have begun to address this, they are computationally intensive and memory-heavy, restricting their use in real-time or embedded systems. We propose MultiTypeFCDD, a simple and lightweight convolutional framework designed as a practical alternative for explainable multi-type anomaly detection. MultiTypeFCDD uses only image-level labels to learn and produce multi-channel heatmaps, where each channel is trained to correspond to a specific anomaly type. The model functions as a single, unified framework capable of differentiating anomaly types across multiple object categories, eliminating the need to train and manage separate models for each object category. We evaluated our proposed method on the Real-IAD dataset and it delivers competitive results (96.4% I-AUROC) at just over 1% the size of state-of-the-art VLM models used for similar tasks. This makes it a highly practical and viable solution for real-world applications where computational resources are tightly constrained.

Method: CountSteer is a training-free method that improves object count accuracy by steering the model’s cross-attention hidden states during inference. It leverages the model’s latent notion of numerical correctness by observing how internal signals shift depending on whether outputs meet specified counts.

Result: CountSteer improved object-count accuracy by about 4% without compromising visual quality. The method demonstrates that models already encode latent numerical correctness that can be harnessed for better control.

Conclusion: CountSteer represents a simple yet effective step toward more controllable and semantically reliable text-to-image generation by exploiting the model’s existing awareness of numerical correctness through cross-attention steering.

Abstract: Text-to-image diffusion models generate realistic and coherent images but often fail to follow numerical instructions in text, revealing a gap between language and visual representation. Interestingly, we found that these models are not entirely blind to numbers-they are implicitly aware of their own counting accuracy, as their internal signals shift in consistent ways depending on whether the output meets the specified count. This observation suggests that the model already encodes a latent notion of numerical correctness, which can be harnessed to guide generation more precisely. Building on this intuition, we introduce CountSteer, a training-free method that improves generation of specified object counts by steering the model’s cross-attention hidden states during inference. In our experiments, CountSteer improved object-count accuracy by about 4% without compromising visual quality, demonstrating a simple yet effective step toward more controllable and semantically reliable text-to-image generation.

Method: Proposes Missing-aware Mixture-of-Loras (MaMOL) framework with dual-routing mechanism: 1) task-oriented dynamic router that adaptively activates experts for different missing patterns, and 2) modality-specific-shared static router that maintains stable cross-modal knowledge sharing. Uses lightweight expert updates and shared expert reuse for parameter-efficient adaptation.

Result: Experiments on multiple remote sensing benchmarks show superior robustness and generalization under varying missing rates with minimal computational overhead. Transfer experiments on natural image datasets validate scalability and cross-domain applicability.

Conclusion: MaMOL provides a general and efficient solution for incomplete multimodal learning that outperforms prior methods while being computationally efficient and transferable across domains.

Abstract: Multimodal classification in remote sensing often suffers from missing modalities caused by environmental interference, sensor failures, or atmospheric effects, which severely degrade classification performance. Existing two-stage adaptation methods are computationally expensive and assume complete multimodal data during training, limiting their generalization to real-world incompleteness. To overcome these issues, we propose a Missing-aware Mixture-of-Loras (MaMOL) framework that reformulates modality missing as a multi-task learning problem. MaMOL introduces a dual-routing mechanism: a task-oriented dynamic router that adaptively activates experts for different missing patterns, and a modality-specific-shared static router that maintains stable cross-modal knowledge sharing. Unlike prior methods that train separate networks for each missing configuration, MaMOL achieves parameter-efficient adaptation via lightweight expert updates and shared expert reuse. Experiments on multiple remote sensing benchmarks demonstrate superior robustness and generalization under varying missing rates, with minimal computational overhead. Moreover, transfer experiments on natural image datasets validate its scalability and cross-domain applicability, highlighting MaMOL as a general and efficient solution for incomplete multimodal learning.

Method: Two-stage framework: 1) VG-Align distills visual pathway into audio pathway by aligning next-token distributions on synchronized audio-video clips. 2) EmoAdapter injects emotion-sensitive residuals and applies emotion supervision for cross-modal emotion understanding.

Result: Achieves state-of-the-art results on ArtEmoBenchmark, outperforming audio-only, visual-only, and audio-visual baselines. Ablations show proposed components are complementary.

Conclusion: VAEmotionLLM enables efficient audio-visual emotion understanding with limited audio pretraining, addressing scalability limitations while enhancing emotion comprehension in multimodal LLMs.

Abstract: Emotion understanding is critical for making Large Language Models (LLMs) more general, reliable, and aligned with humans. Art conveys emotion through the joint design of visual and auditory elements, yet most prior work is human-centered or single-modality, overlooking the emotion intentionally expressed by the artwork. Meanwhile, current Audio-Visual Language Models (AVLMs) typically require large-scale audio pretraining to endow Visual Language Models (VLMs) with hearing, which limits scalability. We present Vision Anchored Audio-Visual Emotion LLM (VAEmotionLLM), a two-stage framework that teaches a VLM to hear by seeing with limited audio pretraining and to understand emotion across modalities. In Stage 1, Vision-Guided Audio Alignment (VG-Align) distills the frozen visual pathway into a new audio pathway by aligning next-token distributions of the shared LLM on synchronized audio-video clips, enabling hearing without a large audio dataset. In Stage 2, a lightweight Cross-Modal Emotion Adapter (EmoAdapter), composed of the Emotion Enhancer and the Emotion Supervisor, injects emotion-sensitive residuals and applies emotion supervision to enhance cross-modal emotion understanding. We also construct ArtEmoBenchmark, an art-centric emotion benchmark that evaluates content and emotion understanding under audio-only, visual-only, and audio-visual inputs. VAEmotionLLM achieves state-of-the-art results on ArtEmoBenchmark, outperforming audio-only, visual-only, and audio-visual baselines. Ablations show that the proposed components are complementary.

Method: Proposes HSL framework with three modules: 1) Dual Style Randomization (DSR) simulates target domain data with diverse style variations; 2) Hierarchical Semantic Mining (HSM) uses multi-scale superpixels to mine intra-class consistency and inter-class distinction at different granularities; 3) Prototype Confidence-modulated Thresholding (PCMT) mitigates segmentation ambiguity when foreground and background are similar.

Result: Extensive experiments on four popular target domain datasets demonstrate state-of-the-art performance.

Conclusion: The proposed HSL framework effectively addresses both style and granularity gaps in CD-FSS, enhancing semantic discriminability for novel classes in target domains through hierarchical semantic learning.

Abstract: Cross-domain Few-shot Segmentation (CD-FSS) aims to segment novel classes from target domains that are not involved in training and have significantly different data distributions from the source domain, using only a few annotated samples, and recent years have witnessed significant progress on this task. However, existing CD-FSS methods primarily focus on style gaps between source and target domains while ignoring segmentation granularity gaps, resulting in insufficient semantic discriminability for novel classes in target domains. Therefore, we propose a Hierarchical Semantic Learning (HSL) framework to tackle this problem. Specifically, we introduce a Dual Style Randomization (DSR) module and a Hierarchical Semantic Mining (HSM) module to learn hierarchical semantic features, thereby enhancing the model’s ability to recognize semantics at varying granularities. DSR simulates target domain data with diverse foreground-background style differences and overall style variations through foreground and global style randomization respectively, while HSM leverages multi-scale superpixels to guide the model to mine intra-class consistency and inter-class distinction at different granularities. Additionally, we also propose a Prototype Confidence-modulated Thresholding (PCMT) module to mitigate segmentation ambiguity when foreground and background are excessively similar. Extensive experiments are conducted on four popular target domain datasets, and the results demonstrate that our method achieves state-of-the-art performance.

Method: HiGFA leverages diffusion sampling dynamics with hierarchical guidance: early-to-mid stages use strong text and transformed contour guidance to establish scene, style, and structure; final stages activate specialized fine-grained classifier guidance and dynamically modulate all guidance strengths based on prediction confidence.

Result: Experiments on several Fine-Grained Visual Classification (FGVC) datasets demonstrate the effectiveness of HiGFA in generating diverse yet faithful synthetic images for data augmentation.

Conclusion: HiGFA’s hierarchical, confidence-driven orchestration successfully balances global structure formation with precise detail refinement, addressing the challenge of applying diffusion models to fine-grained tasks.

Abstract: Generative diffusion models show promise for data augmentation. However, applying them to fine-grained tasks presents a significant challenge: ensuring synthetic images accurately capture the subtle, category-defining features critical for high fidelity. Standard approaches, such as text-based Classifier-Free Guidance (CFG), often lack the required specificity, potentially generating misleading examples that degrade fine-grained classifier performance. To address this, we propose Hierarchically Guided Fine-grained Augmentation (HiGFA). HiGFA leverages the temporal dynamics of the diffusion sampling process. It employs strong text and transformed contour guidance with fixed strengths in the early-to-mid sampling stages to establish overall scene, style, and structure. In the final sampling stages, HiGFA activates a specialized fine-grained classifier guidance and dynamically modulates the strength of all guidance signals based on prediction confidence. This hierarchical, confidence-driven orchestration enables HiGFA to generate diverse yet faithful synthetic images by intelligently balancing global structure formation with precise detail refinement. Experiments on several FGVC datasets demonstrate the effectiveness of HiGFA.

Method: TempoMaster formulates video generation as next-frame-rate prediction: 1) Generate low-frame-rate clip as coarse blueprint, 2) Progressively increase frame rate to refine details and motion, 3) Use bidirectional attention within each frame-rate level, 4) Perform autoregression across frame rates.

Result: Extensive experiments show TempoMaster establishes new state-of-the-art in long video generation, excelling in both visual quality and temporal coherence while enabling efficient parallel synthesis.

Conclusion: The TempoMaster framework successfully addresses long video generation challenges through its novel next-frame-rate prediction approach, achieving superior performance in temporal coherence and visual quality with efficient parallel synthesis.

Abstract: We present TempoMaster, a novel framework that formulates long video generation as next-frame-rate prediction. Specifically, we first generate a low-frame-rate clip that serves as a coarse blueprint of the entire video sequence, and then progressively increase the frame rate to refine visual details and motion continuity. During generation, TempoMaster employs bidirectional attention within each frame-rate level while performing autoregression across frame rates, thus achieving long-range temporal coherence while enabling efficient and parallel synthesis. Extensive experiments demonstrate that TempoMaster establishes a new state-of-the-art in long video generation, excelling in both visual and temporal quality.

Method: Gen-ViRe decomposes CoF reasoning into six cognitive dimensions (from perceptual logic to abstract planning) and 24 subtasks. It uses multi-source data curation, minimal prompting protocols, and hybrid VLM-assisted evaluation with detailed criteria to provide quantitative assessment.

Result: Experiments on state-of-the-art systems reveal substantial discrepancies between impressive visual quality and actual reasoning depth, establishing baselines and diagnostic tools for advancing genuine world simulators.

Conclusion: Gen-ViRe provides the first quantitative framework to assess video models as reasoners, enabling systematic evaluation of core cognitive abilities in multi-step planning, algorithmic logic, and abstract pattern extrapolation for developing better world simulators.

Abstract: While Chain-of-Thought (CoT) prompting enables sophisticated symbolic reasoning in LLMs, it remains confined to discrete text and cannot simulate the continuous, physics-governed dynamics of the real world. Recent video generation models have emerged as potential world simulators through Chain-of-Frames (CoF) reasoning – materializing thought as frame-by-frame visual sequences, with each frame representing a physically-grounded reasoning step. Despite compelling demonstrations, a challenge persists: existing benchmarks, focusing on fidelity or alignment, do not assess CoF reasoning and thus cannot measure core cognitive abilities in multi-step planning, algorithmic logic, or abstract pattern extrapolation. This evaluation void prevents systematic understanding of model capabilities and principled guidance for improvement. We introduce Gen-ViRe (Generative Visual Reasoning Benchmark), a framework grounded in cognitive science and real-world AI applications, which decomposes CoF reasoning into six cognitive dimensions – from perceptual logic to abstract planning – and 24 subtasks. Through multi-source data curation, minimal prompting protocols, and hybrid VLM-assisted evaluation with detailed criteria, Gen-ViRe delivers the first quantitative assessment of video models as reasoners. Our experiments on SOTA systems reveal substantial discrepancies between impressive visual quality and actual reasoning depth, establishing baselines and diagnostic tools to advance genuine world simulators.

Method: Uses inward sliding window attention to maintain training-scale receptive field, with dual-path pipeline featuring cross-attention override strategy for global consistency, plus cross-attention caching for efficiency.

Result: Generates ultra-high-resolution videos with fine-grained details and high efficiency in training-free paradigm, achieves superior VBench performance compared to training-based alternatives with competitive efficiency.

Conclusion: Proposes effective training-free approach for high-resolution video generation that preserves visual fidelity while ensuring global coherence through novel attention mechanisms.

Abstract: The quadratic time and memory complexity of the attention mechanism in modern Transformer based video generators makes end-to-end training for ultra high resolution videos prohibitively expensive. Motivated by this limitation, we introduce a training-free approach that leverages video Diffusion Transformers pretrained at their native scale to synthesize higher resolution videos without any additional training or adaptation. At the core of our method lies an inward sliding window attention mechanism, which originates from a key observation: maintaining each query token’s training scale receptive field is crucial for preserving visual fidelity and detail. However, naive local window attention, unfortunately, often leads to repetitive content and exhibits a lack of global coherence in the generated results. To overcome this challenge, we devise a dual-path pipeline that backs up window attention with a novel cross-attention override strategy, enabling the semantic content produced by local attention to be guided by another branch with a full receptive field and, therefore, ensuring holistic consistency. Furthermore, to improve efficiency, we incorporate a cross-attention caching strategy for this branch to avoid the frequent computation of full 3D attention. Extensive experiments demonstrate that our method delivers ultra-high-resolution videos with fine-grained visual details and high efficiency in a training-free paradigm. Meanwhile, it achieves superior performance on VBench, even compared to training-based alternatives, with competitive or improved efficiency. Codes are available at: https://github.com/WillWu111/FreeSwim

Method: 1) MoR architecture treats each LoRA rank as independent expert with fixed-position shared experts; 2) Degradation estimation module using CLIP embeddings and text pairs; 3) Zero-expert slots and degradation-aware load-balancing loss for dynamic expert activation.

Result: Comprehensive experiments validate framework effectiveness and state-of-the-art performance in real-world image super-resolution.

Conclusion: MoR architecture successfully integrates sparse MoE into Real-ISR, enabling adaptive degradation handling, flexible knowledge recombination, and optimal computational resource allocation.

Abstract: The demonstrated success of sparsely-gated Mixture-of-Experts (MoE) architectures, exemplified by models such as DeepSeek and Grok, has motivated researchers to investigate their adaptation to diverse domains. In real-world image super-resolution (Real-ISR), existing approaches mainly rely on fine-tuning pre-trained diffusion models through Low-Rank Adaptation (LoRA) module to reconstruct high-resolution (HR) images. However, these dense Real-ISR models are limited in their ability to adaptively capture the heterogeneous characteristics of complex real-world degraded samples or enable knowledge sharing between inputs under equivalent computational budgets. To address this, we investigate the integration of sparse MoE into Real-ISR and propose a Mixture-of-Ranks (MoR) architecture for single-step image super-resolution. We introduce a fine-grained expert partitioning strategy that treats each rank in LoRA as an independent expert. This design enables flexible knowledge recombination while isolating fixed-position ranks as shared experts to preserve common-sense features and minimize routing redundancy. Furthermore, we develop a degradation estimation module leveraging CLIP embeddings and predefined positive-negative text pairs to compute relative degradation scores, dynamically guiding expert activation. To better accommodate varying sample complexities, we incorporate zero-expert slots and propose a degradation-aware load-balancing loss, which dynamically adjusts the number of active experts based on degradation severity, ensuring optimal computational resource allocation. Comprehensive experiments validate our framework’s effectiveness and state-of-the-art performance.

Method: Replace CCT interpolation with lightweight MLP using 2D chromaticity features, trained via lightbox calibration with representative LED sources.

Result: 22% average reduction in angular reproduction error for LED-lit scenes, maintains backward compatibility, handles multi-illuminant scenes, and supports real-time deployment.

Conclusion: 2D chromaticity space with MLP provides superior colorimetric accuracy for modern lighting while preserving computational efficiency and compatibility.

Abstract: Traditional in-camera colorimetric mapping relies on correlated color temperature (CCT)-based interpolation between pre-calibrated transforms optimized for Planckian illuminants such as CIE A and D65. However, modern lighting technologies such as LEDs can deviate substantially from the Planckian locus, exposing the limitations of relying on conventional one-dimensional CCT for illumination characterization. This paper demonstrates that transitioning from 1D CCT (on the Planckian locus) to a 2D chromaticity space (off the Planckian locus) improves colorimetric accuracy across various mapping approaches. In addition, we replace conventional CCT interpolation with a lightweight multi-layer perceptron (MLP) that leverages 2D chromaticity features for robust colorimetric mapping under non-Planckian illuminants. A lightbox-based calibration procedure incorporating representative LED sources is used to train our MLP. Validated across diverse LED lighting, our method reduces angular reproduction error by 22% on average in LED-lit scenes, maintains backward compatibility with traditional illuminants, accommodates multi-illuminant scenes, and supports real-time in-camera deployment with negligible additional computational cost.

Method: Hierarchical Semi-Supervised Active Learning (HSSAL) integrates SSL and hierarchical active learning in iterative loop. SSL refines model using labeled data (supervised) and unlabeled data (weak-to-strong self-training). Hierarchical AL uses progressive clustering to select most informative samples based on scalability, diversity, and uncertainty.

Result: HSSAL outperforms SSL- or AL-only baselines on three RS datasets (UCM, AID, NWPU-RESISC45). With only 8%, 4%, and 2% labeled data respectively, achieves over 95% of fully-supervised accuracy.

Conclusion: HSSAL framework demonstrates superior label efficiency by effectively exploiting informativeness of unlabeled data through integrated SSL and hierarchical AL approach.

Abstract: The performance of deep learning models in remote sensing (RS) strongly depends on the availability of high-quality labeled data. However, collecting large-scale annotations is costly and time-consuming, while vast amounts of unlabeled imagery remain underutilized. To address this challenge, we propose a Hierarchical Semi-Supervised Active Learning (HSSAL) framework that integrates semi-supervised learning (SSL) and a novel hierarchical active learning (HAL) in a closed iterative loop. In each iteration, SSL refines the model using both labeled data through supervised learning and unlabeled data via weak-to-strong self-training, improving feature representation and uncertainty estimation. Guided by the refined representations and uncertainty cues of unlabeled samples, HAL then conducts sample querying through a progressive clustering strategy, selecting the most informative instances that jointly satisfy the criteria of scalability, diversity, and uncertainty. This hierarchical process ensures both efficiency and representativeness in sample selection. Extensive experiments on three benchmark RS scene classification datasets, including UCM, AID, and NWPU-RESISC45, demonstrate that HSSAL consistently outperforms SSL- or AL-only baselines. Remarkably, with only 8%, 4%, and 2% labeled training data on UCM, AID, and NWPU-RESISC45, respectively, HSSAL achieves over 95% of fully-supervised accuracy, highlighting its superior label efficiency through informativeness exploitation of unlabeled data. Our code will be publicly available.

Method: GigaWorld-0 combines two synergistic components: GigaWorld-0-Video (large-scale video generation with fine-grained control) and GigaWorld-0-3D (3D generative modeling with Gaussian Splatting reconstruction, system identification, and motion planning). The framework uses joint optimization and efficient training via GigaTrain with FP8-precision and sparse attention.

Result: The system generates high-quality, diverse, and controllable embodied interaction data across multiple dimensions. VLA models (like GigaBrain-0) trained on this synthetic data achieve strong real-world performance, significantly improving generalization and task success on physical robots without any real-world interaction during training.

Conclusion: GigaWorld-0 demonstrates that unified world models can serve as effective data engines for VLA learning, enabling scalable synthesis of visually compelling, spatially coherent, physically plausible, and instruction-aligned training data that transfers successfully to real-world robotic tasks.

Abstract: World models are emerging as a foundational paradigm for scalable, data-efficient embodied AI. In this work, we present GigaWorld-0, a unified world model framework designed explicitly as a data engine for Vision-Language-Action (VLA) learning. GigaWorld-0 integrates two synergistic components: GigaWorld-0-Video, which leverages large-scale video generation to produce diverse, texture-rich, and temporally coherent embodied sequences under fine-grained control of appearance, camera viewpoint, and action semantics; and GigaWorld-0-3D, which combines 3D generative modeling, 3D Gaussian Splatting reconstruction, physically differentiable system identification, and executable motion planning to ensure geometric consistency and physical realism. Their joint optimization enables the scalable synthesis of embodied interaction data that is visually compelling, spatially coherent, physically plausible, and instruction-aligned. Training at scale is made feasible through our efficient GigaTrain framework, which exploits FP8-precision and sparse attention to drastically reduce memory and compute requirements. We conduct comprehensive evaluations showing that GigaWorld-0 generates high-quality, diverse, and controllable data across multiple dimensions. Critically, VLA model (e.g., GigaBrain-0) trained on GigaWorld-0-generated data achieve strong real-world performance, significantly improving generalization and task success on physical robots without any real-world interaction during training.

Method: Proposed a verification-centric Generate-then-Verify framework: first generates QA pairs with figure-associated textual context, then applies cross-modal consistency checks against figures along with auxiliary filters to eliminate erroneous pairs.

Result: Created VeriSciQA dataset with 20,351 QA pairs spanning 20 scientific domains and 12 figure types. It poses a challenging benchmark (64% accuracy for open-source vs 82% for proprietary models). Models fine-tuned on VeriSciQA achieve consistent improvements on SVQA benchmarks with scalable performance gains.

Conclusion: The verification-centric framework successfully creates high-quality SVQA data, and continued data expansion through this scalable approach can advance SVQA capability in the open-source community.

Abstract: Large Vision-Language Models (LVLMs) show promise for scientific applications, yet open-source models still struggle with Scientific Visual Question Answering (SVQA), namely answering questions about figures from scientific papers. A key bottleneck lies in the lack of public, large-scale, high-quality SVQA datasets. Although recent work uses LVLMs to synthesize data at scale, we identify systematic errors in their resulting QA pairs, stemming from LVLMs’ inherent limitations and information asymmetry between figures and text. To address these challenges, we propose a verification-centric Generate-then-Verify framework that first generates QA pairs with figure-associated textual context, then applies cross-modal consistency checks against figures along with auxiliary filters to eliminate erroneous pairs. We instantiate this framework to curate VeriSciQA, a dataset of 20,351 QA pairs spanning 20 scientific domains and 12 figure types. VeriSciQA poses a challenging benchmark for open-source models, with a substantial accuracy gap between the leading open-source models (64%) and a proprietary model (82%). Moreover, models fine-tuned on VeriSciQA achieve consistent improvements on SVQA benchmarks, with performance gains that scale with data size and surpass models trained on existing datasets. Human evaluation further validates the superior correctness of VeriSciQA. Together, these evidences demonstrate that continued data expansion by our scalable framework can further advance SVQA capability in the open-source community.

Method: Introduces iMontage framework with minimally invasive adaptation strategy, tailored data curation, and training paradigm to repurpose video models for image generation while preserving motion priors.

Result: iMontage excels at many-in-many-out image tasks, maintaining strong cross-image consistency while generating scenes with extraordinary dynamics beyond conventional scopes.

Conclusion: iMontage successfully unifies diverse image generation and editing tasks by leveraging video models’ temporal coherence while injecting image data diversity, creating a versatile framework for variable-length image set generation.

Abstract: Pre-trained video models learn powerful priors for generating high-quality, temporally coherent content. While these models excel at temporal coherence, their dynamics are often constrained by the continuous nature of their training data. We hypothesize that by injecting the rich and unconstrained content diversity from image data into this coherent temporal framework, we can generate image sets that feature both natural transitions and a far more expansive dynamic range. To this end, we introduce iMontage, a unified framework designed to repurpose a powerful video model into an all-in-one image generator. The framework consumes and produces variable-length image sets, unifying a wide array of image generation and editing tasks. To achieve this, we propose an elegant and minimally invasive adaptation strategy, complemented by a tailored data curation process and training paradigm. This approach allows the model to acquire broad image manipulation capabilities without corrupting its invaluable original motion priors. iMontage excels across several mainstream many-in-many-out tasks, not only maintaining strong cross-image contextual consistency but also generating scenes with extraordinary dynamics that surpass conventional scopes. Find our homepage at: https://kr1sjfu.github.io/iMontage-web/.

Method: T3-Tracer jointly analyzes audio at frame, segment, and audio levels. It consists of two core modules: 1) Frame-Audio Feature Aggregation Module (FA-FAM) that detects frame authenticity by combining frame-level and audio-level temporal information, and 2) Segment-level Multi-Scale Discrepancy-Aware Module (SMDAM) that refines detection by modeling frame features and inter-frame differences across multi-scale temporal windows to identify forged boundaries.

Result: Extensive experiments on three challenging datasets demonstrate that T3-Tracer achieves state-of-the-art performance in detecting partial audio forgeries.

Conclusion: The proposed hierarchical three-level analysis framework effectively addresses the limitations of existing methods by capturing both transient and sustained anomalies, making it a powerful solution for detecting sophisticated partial audio forgeries.

Abstract: Recently, partial audio forgery has emerged as a new form of audio manipulation. Attackers selectively modify partial but semantically critical frames while preserving the overall perceptual authenticity, making such forgeries particularly difficult to detect. Existing methods focus on independently detecting whether a single frame is forged, lacking the hierarchical structure to capture both transient and sustained anomalies across different temporal levels. To address these limitations, We identify three key levels relevant to partial audio forgery detection and present T3-Tracer, the first framework that jointly analyzes audio at the frame, segment, and audio levels to comprehensively detect forgery traces. T3-Tracer consists of two complementary core modules: the Frame-Audio Feature Aggregation Module (FA-FAM) and the Segment-level Multi-Scale Discrepancy-Aware Module (SMDAM). FA-FAM is designed to detect the authenticity of each audio frame. It combines both frame-level and audio-level temporal information to detect intra-frame forgery cues and global semantic inconsistencies. To further refine and correct frame detection, we introduce SMDAM to detect forgery boundaries at the segment level. It adopts a dual-branch architecture that jointly models frame features and inter-frame differences across multi-scale temporal windows, effectively identifying abrupt anomalies that appeared on the forged boundaries. Extensive experiments conducted on three challenging datasets demonstrate that our approach achieves state-of-the-art performance.

Method: Proposed AVFakeBench with 12K audio-video questions covering 7 forgery types and 4 annotation levels, using a multi-stage hybrid forgery framework that integrates proprietary models for task planning with expert generative models for precise manipulation.

Result: Evaluation of 11 Audio-Video Large Language Models and 2 prevalent detection methods shows AV-LMMs have potential as emerging forgery detectors but reveal notable weaknesses in fine-grained perception and reasoning.

Conclusion: AVFakeBench establishes a comprehensive benchmark for audio-video forgery detection, demonstrating both the promise and limitations of AV-LMMs for this task, highlighting the need for improved fine-grained perception and reasoning capabilities.

Abstract: The threat of Audio-Video (AV) forgery is rapidly evolving beyond human-centric deepfakes to include more diverse manipulations across complex natural scenes. However, existing benchmarks are still confined to DeepFake-based forgeries and single-granularity annotations, thus failing to capture the diversity and complexity of real-world forgery scenarios. To address this, we introduce AVFakeBench, the first comprehensive audio-video forgery detection benchmark that spans rich forgery semantics across both human subject and general subject. AVFakeBench comprises 12K carefully curated audio-video questions, covering seven forgery types and four levels of annotations. To ensure high-quality and diverse forgeries, we propose a multi-stage hybrid forgery framework that integrates proprietary models for task planning with expert generative models for precise manipulation. The benchmark establishes a multi-task evaluation framework covering binary judgment, forgery types classification, forgery detail selection, and explanatory reasoning. We evaluate 11 Audio-Video Large Language Models (AV-LMMs) and 2 prevalent detection methods on AVFakeBench, demonstrating the potential of AV-LMMs as emerging forgery detectors while revealing their notable weaknesses in fine-grained perception and reasoning.

Method: CaliTex introduces geometry-calibrated attention with two modules: Part-Aligned Attention that enforces spatial alignment across semantically matched parts, and Condition-Routed Attention which routes appearance information through geometry-conditioned pathways. This is coupled with a two-stage diffusion transformer to make geometric coherence an inherent network behavior.

Result: CaliTex produces seamless and view-consistent textures and outperforms both open-source and commercial baselines in 3D texture generation.

Conclusion: By explicitly aligning attention with 3D structure through geometry-calibrated attention, CaliTex addresses the fundamental issue of cross-view inconsistency in 3D texture generation, making geometric coherence an inherent property rather than an optimization byproduct.

Abstract: Despite major advances brought by diffusion-based models, current 3D texture generation systems remain hindered by cross-view inconsistency – textures that appear convincing from one viewpoint often fail to align across others. We find that this issue arises from attention ambiguity, where unstructured full attention is applied indiscriminately across tokens and modalities, causing geometric confusion and unstable appearance-structure coupling. To address this, we introduce CaliTex, a framework of geometry-calibrated attention that explicitly aligns attention with 3D structure. It introduces two modules: Part-Aligned Attention that enforces spatial alignment across semantically matched parts, and Condition-Routed Attention which routes appearance information through geometry-conditioned pathways to maintain spatial fidelity. Coupled with a two-stage diffusion transformer, CaliTex makes geometric coherence an inherent behavior of the network rather than a byproduct of optimization. Empirically, CaliTex produces seamless and view-consistent textures and outperforms both open-source and commercial baselines.

Method: Deep learning multiclass classifier for 16 oral lesions using stratified data splitting, advanced data augmentation, and oversampling techniques to handle limited and imbalanced datasets.

Result: Achieved 83.33% accuracy, 89.12% precision, and 77.31% recall, outperforming state-of-the-art methods, with particularly notable performance on minority class classification.

Conclusion: The framework demonstrates the effectiveness of oversampling and augmentation strategies for imbalanced medical datasets and shows promise as a first step toward trustworthy computer-aided diagnostic systems for early oral cancer detection.

Abstract: Oral cancer is highly common across the globe and is mostly diagnosed during the later stages due to the close visual similarity to benign, precancerous, and malignant lesions in the oral cavity. Implementing computer aided diagnosis systems early on has the potential to greatly improve clinical outcomes. This research intends to use deep learning to build a multiclass classifier for sixteen different oral lesions. To overcome the challenges of limited and imbalanced datasets, the proposed technique combines stratified data splitting and advanced data augmentation and oversampling to perform the classification. The experimental results, which achieved 83.33 percent accuracy, 89.12 percent precision, and 77.31 percent recall, demonstrate the superiority of the suggested model over state of the art methods now in use. The suggested model effectively conveys the effectiveness of oversampling and augmentation strategies in situations where the minority class classification performance is noteworthy. As a first step toward trustworthy computer aided diagnostic systems for the early detection of oral cancer in clinical settings, the suggested framework shows promise.

Method: Propose Triple Physical Constraints (TPCs) theory based on Kubelka-Munk theory, preserving specular reflection coefficient. Build TPCNet with physical constraints in feature space rather than image space, maintaining relationships between illumination, reflection, and detection.

Result: TPCNet outperforms state-of-the-art methods on 10 datasets in both quantitative metrics and visual quality. Constraints improve performance without introducing new parameters.

Conclusion: The TPC theory and TPCNet effectively address limitations of previous Retinex-based methods by incorporating specular reflection and feature-space constraints, achieving superior low-light image enhancement.

Abstract: Low-light image enhancement is an essential computer vision task to improve image contrast and to decrease the effects of color bias and noise. Many existing interpretable deep-learning algorithms exploit the Retinex theory as the basis of model design. However, previous Retinex-based algorithms, that consider reflected objects as ideal Lambertian ignore specular reflection in the modeling process and construct the physical constraints in image space, limiting generalization of the model. To address this issue, we preserve the specular reflection coefficient and reformulate the original physical constraints in the imaging process based on the Kubelka-Munk theory, thereby constructing constraint relationship between illumination, reflection, and detection, the so-called triple physical constraints (TPCs)theory. Based on this theory, the physical constraints are constructed in the feature space of the model to obtain the TPC network (TPCNet). Comprehensive quantitative and qualitative benchmark and ablation experiments confirm that these constraints effectively improve the performance metrics and visual quality without introducing new parameters, and demonstrate that our TPCNet outperforms other state-of-the-art methods on 10 datasets.

Method: HybridWorldSim integrates multi-traversal neural reconstruction for static backgrounds with generative modeling for dynamic agents, creating a unified framework that addresses limitations of previous methods.

Result: The framework surpasses previous state-of-the-art methods and is accompanied by the MIRROR dataset, a new multi-traversal dataset capturing diverse routes and environmental conditions across different cities.

Conclusion: HybridWorldSim provides a practical and scalable solution for high-fidelity autonomous driving simulation, offering a valuable resource for research and development in the field.

Abstract: Realistic and controllable simulation is critical for advancing end-to-end autonomous driving, yet existing approaches often struggle to support novel view synthesis under large viewpoint changes or to ensure geometric consistency. We introduce HybridWorldSim, a hybrid simulation framework that integrates multi-traversal neural reconstruction for static backgrounds with generative modeling for dynamic agents. This unified design addresses key limitations of previous methods, enabling the creation of diverse and high-fidelity driving scenarios with reliable visual and spatial consistency. To facilitate robust benchmarking, we further release a new multi-traversal dataset MIRROR that captures a wide range of routes and environmental conditions across different cities. Extensive experiments demonstrate that HybridWorldSim surpasses previous state-of-the-art methods, providing a practical and scalable solution for high-fidelity simulation and a valuable resource for research and development in autonomous driving.

Method: Train noise-aware reward models via self-distillation to align intermediate estimate rewards with final clean image rewards, using curriculum training from clean to noisy images, and introduce reward alignment metric.

Result: Improves performance by over 16% compared to existing methods, enables more efficient test-time scaling, and provides orthogonal gains when combined with other techniques.

Conclusion: TTSnap offers an effective framework for computationally efficient test-time scaling by early pruning of low-quality candidates through noise-aware reward prediction.

Abstract: A prominent approach to test-time scaling for text-to-image diffusion models formulates the problem as a search over multiple noise seeds, selecting the one that maximizes a certain image-reward function. The effectiveness of this strategy heavily depends on the number and diversity of noise seeds explored. However, verifying each candidate is computationally expensive, because each must be fully denoised before a reward can be computed. This severely limits the number of samples that can be explored under a fixed budget. We propose test-time scaling with noise-aware pruning (TTSnap), a framework that prunes low-quality candidates without fully denoising them. The key challenge is that reward models are learned in the clean image domain, and the ranking of rewards predicted for intermediate estimates are often inconsistent with those predicted for clean images. To overcome this, we train noise-aware reward models via self-distillation to align the reward for intermediate estimates with that of the final clean images. To stabilize learning across different noise levels, we adopt a curriculum training strategy that progressively shifts the data domain from clean images to noise images. In addition, we introduce a new metric that measures reward alignment and computational budget utilization. Experiments demonstrate that our approach improves performance by over 16% compared with existing methods, enabling more efficient and effective test-time scaling. It also provides orthogonal gains when combined with post-training techniques and local test-time optimization. Code: https://github.com/TerrysLearning/TTSnap/.

Method: MVMAE combines masked image reconstruction with cross-view alignment, using clinical redundancy across projections as self-supervision. MVMAE-V2T extends this by incorporating radiology reports as auxiliary text-based learning signals while maintaining vision-only inference.

Result: Evaluated on MIMIC-CXR, CheXpert, and PadChest datasets for disease classification, MVMAE consistently outperforms supervised and vision-language baselines. MVMAE-V2T provides additional gains, especially in low-label regimes where text supervision is most beneficial.

Conclusion: Structural (multi-view) and textual supervision are complementary paths toward scalable, clinically grounded medical foundation models, with MVMAE demonstrating the value of exploiting clinical data organization for robust medical ML.

Abstract: Building robust medical machine learning systems requires pretraining strategies that exploit the intrinsic structure present in clinical data. We introduce Multiview Masked Autoencoder (MVMAE), a self-supervised framework that leverages the natural multi-view organization of radiology studies to learn view-invariant and disease-relevant representations. MVMAE combines masked image reconstruction with cross-view alignment, transforming clinical redundancy across projections into a powerful self-supervisory signal. We further extend this approach with MVMAE-V2T, which incorporates radiology reports as an auxiliary text-based learning signal to enhance semantic grounding while preserving fully vision-based inference. Evaluated on a downstream disease classification task on three large-scale public datasets, MIMIC-CXR, CheXpert, and PadChest, MVMAE consistently outperforms supervised and vision-language baselines. Furthermore, MVMAE-V2T provides additional gains, particularly in low-label regimes where structured textual supervision is most beneficial. Together, these results establish the importance of structural and textual supervision as complementary paths toward scalable, clinically grounded medical foundation models.

Method: Benchmarked four ML architectures (U-Nets, visual transformers, CNNs, MDNs) for multi-class state recognition in double-QD charge-stability diagrams across different data budgets and normalization schemes using synthetic and experimental data.

Result: U-Nets and ViTs achieved highest scores on synthetic data (>0.98 MSE) but failed to generalize to experimental data. CNNs offered best trade-off on experimental CSDs with strong accuracy and two orders of magnitude fewer parameters than U-Nets/ViTs. MDNs were computationally efficient but had lower performance.

Conclusion: CNNs with min-max normalization are the most practical approach for quantum dot charge-stability diagram analysis, balancing accuracy, generalization, and computational efficiency for scalable quantum processor tuning.

Abstract: Semiconductor quantum dots (QDs) are a leading platform for scalable quantum processors. However, scaling to large arrays requires reliable, automated tuning strategies for devices’ bootstrapping, calibration, and operation, with many tuning aspects depending on accurately identifying QD device states from charge-stability diagrams (CSDs). In this work, we present a comprehensive benchmarking study of four modern machine learning (ML) architectures for multi-class state recognition in double-QD CSDs. We evaluate their performance across different data budgets and normalization schemes using both synthetic and experimental data. We find that the more resource-intensive models – U-Nets and visual transformers (ViTs) – achieve the highest MSE score (defined as $1-\mathrm{MSE}$) on synthetic data (over $0.98$) but fail to generalize to experimental data. MDNs are the most computationally efficient and exhibit highly stable training, but with substantially lower peak performance. CNNs offer the most favorable trade-off on experimental CSDs, achieving strong accuracy with two orders of magnitude fewer parameters than the U-Nets and ViTs. Normalization plays a nontrivial role: min-max scaling generally yields higher MSE scores but less stable convergence, whereas z-score normalization produces more predictable training dynamics but at reduced accuracy for most models. Overall, our study shows that CNNs with min-max normalization are a practical approach for QD CSDs.

Method: Introduces a thinking-editing-reflection loop: thinking mechanism interprets abstract instructions using MLLM’s world knowledge, while reflection reviews results, corrects unintended manipulations, and identifies stopping points. Two versions: ReasonEdit-S (initialized from Step1X-Edit) and ReasonEdit-Q (integrated with Qwen-Image-Edit).

Result: Achieves significant performance improvements: +4.3% on ImgEdit, +4.7% on GEdit, and +8.2% on Kris with ReasonEdit-S. ReasonEdit-Q outperforms previous open-source methods on both GEdit and Kris benchmarks.

Conclusion: Unlocking MLLM reasoning capabilities through thinking and reflection mechanisms significantly enhances image editing performance, demonstrating the value of incorporating reasoning loops into editing frameworks.

Abstract: Recent advances in image editing models have shown remarkable progress. A common architectural design couples a multimodal large language model (MLLM) encoder with a diffusion decoder, as seen in systems such as Step1X-Edit and Qwen-Image-Edit, where the MLLM encodes both the reference image and the instruction but remains frozen during training. In this work, we demonstrate that unlocking the reasoning capabilities of MLLM can further push the boundaries of editing models. Specifically, we explore two reasoning mechanisms, thinking and reflection, which enhance instruction understanding and editing accuracy. Based on that, our proposed framework enables image editing in a thinking-editing-reflection loop: the thinking mechanism leverages the world knowledge of MLLM to interpret abstract instructions, while the reflection reviews editing results, automatically corrects unintended manipulations, and identifies the stopping round. Extensive experiments demonstrate that our reasoning approach achieves significant performance gains, with improvements of ImgEdit (+4.3%), GEdit (+4.7%), and Kris (+8.2%) when initializing our DiT from the Step1X-Edit (ReasonEdit-S), and also outperforms previous open-source methods on both GEdit and Kris when integrated with Qwen-Image-Edit (ReasonEdit-Q).

Method: Used synchronous aerial VIS and TIR imagery of great blue herons; tested two fusion methods: early fusion (PCA-based image fusion) and late fusion (CART-based detection fusion); aligned images using deep learning; trained YOLO11n models.

Result: Both fusion methods improved F1 scores over VIS-only model; late fusion achieved best results: 93.0% F1 score for occupied nests (vs 90.2% VIS-only); identified false positives with 90% recall.

Conclusion: VIS-TIR fusion improves detection accuracy but has limitations (TIR FOV constraints, alignment issues); very high-resolution visible sensors alone may be more practical for operational surveys.

Abstract: Efficient wildlife monitoring methods are necessary for biodiversity conservation and management. The combination of remote sensing, aerial imagery and deep learning offer promising opportunities to renew or improve existing survey methods. The complementary use of visible (VIS) and thermal infrared (TIR) imagery can add information compared to a single-source image and improve results in an automated detection context. However, the alignment and fusion process can be challenging, especially since visible and thermal images usually have different fields of view (FOV) and spatial resolutions. This research presents a case study on the great blue heron (Ardea herodias) to evaluate the performances of synchronous aerial VIS and TIR imagery to automatically detect individuals and nests using a YOLO11n model. Two VIS-TIR fusion methods were tested and compared: an early fusion approach and a late fusion approach, to determine if the addition of the TIR image gives any added value compared to a VIS-only model. VIS and TIR images were automatically aligned using a deep learning model. A principal component analysis fusion method was applied to VIS-TIR image pairs to form the early fusion dataset. A classification and regression tree was used to process the late fusion dataset, based on the detection from the VIS-only and TIR-only trained models. Across all classes, both late and early fusion improved the F1 score compared to the VIS-only model. For the main class, occupied nest, the late fusion improved the F1 score from 90.2 (VIS-only) to 93.0%. This model was also able to identify false positives from both sources with 90% recall. Although fusion methods seem to give better results, this approach comes with a limiting TIR FOV and alignment constraints that eliminate data. Using an aircraft-mounted very high-resolution visible sensor could be an interesting option for operationalizing surveys.

Method: Proposes DenoiseGS framework using 3D Gaussian Splatting for burst denoising. Introduces Gaussian self-consistency (GSC) loss to regularize geometry from noisy inputs using high-quality Gaussian point clouds from clean inputs. Adds log-weighted frequency (LWF) loss for spectral domain supervision to preserve fine details by adaptively weighting frequency discrepancies.

Result: Significantly exceeds state-of-the-art NeRF-based methods on both burst denoising and novel view synthesis under noisy conditions, while achieving 250× faster inference speed.

Conclusion: DenoiseGS demonstrates that 3D Gaussian Splatting can be effectively adapted for burst denoising, overcoming challenges of noise degradation and detail loss through novel consistency and frequency-based losses, achieving superior performance with dramatically improved efficiency.

Abstract: Burst denoising methods are crucial for enhancing images captured on handheld devices, but they often struggle with large motion or suffer from prohibitive computational costs. In this paper, we propose DenoiseGS, the first framework to leverage the efficiency of 3D Gaussian Splatting for burst denoising. Our approach addresses two key challenges when applying feedforward Gaussian reconsturction model to noisy inputs: the degradation of Gaussian point clouds and the loss of fine details. To this end, we propose a Gaussian self-consistency (GSC) loss, which regularizes the geometry predicted from noisy inputs with high-quality Gaussian point clouds. These point clouds are generated from clean inputs by the same model that we are training, thereby alleviating potential bias or domain gaps. Additionally, we introduce a log-weighted frequency (LWF) loss to strengthen supervision within the spectral domain, effectively preserving fine-grained details. The LWF loss adaptively weights frequency discrepancies in a logarithmic manner, emphasizing challenging high-frequency details. Extensive experiments demonstrate that DenoiseGS significantly exceeds the state-of-the-art NeRF-based methods on both burst denoising and novel view synthesis under noisy conditions, while achieving 250$\times$ faster inference speed. Code and models are released at https://github.com/yscheng04/DenoiseGS.

Method: 1) Reformulate training as self-supervised outpainting to transform diverse-layout references into unified occluded-input format; 2) Design reference extractor for comprehensive identity feature extraction; 3) Integrate hybrid reference fusion attention for varying resolutions and dynamic sequence lengths; 4) Introduce identity-robust pose control to decouple appearance from skeletal structure; 5) Use token replace strategy for coherent long-video generation.

Result: Extensive experiments show the method outperforms existing approaches in character animation and image pose transfer for references with arbitrary layouts.

Conclusion: One-to-All Animation provides a unified framework for high-fidelity character animation and pose transfer that successfully handles spatially misaligned references, addressing a key limitation of existing methods.

Abstract: Recent advances in diffusion models have greatly improved pose-driven character animation. However, existing methods are limited to spatially aligned reference-pose pairs with matched skeletal structures. Handling reference-pose misalignment remains unsolved. To address this, we present One-to-All Animation, a unified framework for high-fidelity character animation and image pose transfer for references with arbitrary layouts. First, to handle spatially misaligned reference, we reformulate training as a self-supervised outpainting task that transforms diverse-layout reference into a unified occluded-input format. Second, to process partially visible reference, we design a reference extractor for comprehensive identity feature extraction. Further, we integrate hybrid reference fusion attention to handle varying resolutions and dynamic sequence lengths. Finally, from the perspective of generation quality, we introduce identity-robust pose control that decouples appearance from skeletal structure to mitigate pose overfitting, and a token replace strategy for coherent long-video generation. Extensive experiments show that our method outperforms existing approaches. The code and model are available at https://github.com/ssj9596/One-to-All-Animation.

Method: Dual-branch framework that mutually generates camera-consistent RGB and depth sequences, with Semantic Guided Mutual Alignment (SIGMA) mechanism for semantics-guided RGB-depth fusion and mutual reinforcement.

Result: Achieves more consistent camera-controlled video generation with over 40% reduction in camera motion errors compared to prior methods, and reveals complementary roles of early/late denoising stages in structure formation and detail refinement.

Conclusion: DualCamCtrl effectively disentangles appearance and geometry modeling through its dual-branch approach with SIGMA, enabling more faithful adherence to specified camera trajectories in video generation.

Abstract: This paper presents DualCamCtrl, a novel end-to-end diffusion model for camera-controlled video generation. Recent works have advanced this field by representing camera poses as ray-based conditions, yet they often lack sufficient scene understanding and geometric awareness. DualCamCtrl specifically targets this limitation by introducing a dual-branch framework that mutually generates camera-consistent RGB and depth sequences. To harmonize these two modalities, we further propose the Semantic Guided Mutual Alignment (SIGMA) mechanism, which performs RGB-depth fusion in a semantics-guided and mutually reinforced manner. These designs collectively enable DualCamCtrl to better disentangle appearance and geometry modeling, generating videos that more faithfully adhere to the specified camera trajectories. Additionally, we analyze and reveal the distinct influence of depth and camera poses across denoising stages and further demonstrate that early and late stages play complementary roles in forming global structure and refining local details. Extensive experiments demonstrate that DualCamCtrl achieves more consistent camera-controlled video generation, with over 40% reduction in camera motion errors compared with prior methods. Our project page: https://soyouthinkyoucantell.github.io/dualcamctrl-page/

Method: Proposes PowerCLIP with powerset alignment that optimizes region-to-phrase alignments by minimizing loss between powersets of image regions and textual parse trees. Introduces efficient non-linear aggregators (NLAs) to reduce computational complexity from O(2^M) to O(M) while approximating exact loss.

Result: PowerCLIP outperforms state-of-the-art methods in zero-shot classification and retrieval tasks, demonstrating improved compositionality and robustness compared to existing approaches.

Conclusion: The powerset alignment framework with efficient NLAs effectively captures compositional semantics across multiple image regions, leading to superior zero-shot performance while maintaining computational feasibility.

Abstract: Contrastive vision-language pre-training frameworks such as CLIP have demonstrated impressive zero-shot performance across a range of vision-language tasks. Recent studies have shown that aligning individual text tokens with specific image patches or regions enhances fine-grained compositional understanding. However, it remains challenging to capture compositional semantics that span multiple image regions. To address this limitation, we propose PowerCLIP, a novel contrastive pre-training framework enhanced by powerset alignment, which exhaustively optimizes region-to-phrase alignments by minimizing the loss defined between powersets of image regions and textual parse trees. Since the naive powerset construction incurs exponential computational cost due to the combinatorial explosion in the number of region subsets, we introduce efficient non-linear aggregators (NLAs) that reduce complexity from O(2^M) to O(M) with respect to the number of regions M, while approximating the exact loss value with arbitrary precision. Our extensive experiments demonstrate that PowerCLIP outperforms state-of-the-art methods in zero-shot classification and retrieval tasks, underscoring the compositionality and robustness of our approach. Our code will be made publicly available.

Method: Uses pre-trained video generation models conditioned on a single edited view to generate other consistent edited views. Introduces motion-preserved noise blending for camera pose alignment and geometry-aware denoising for better multi-view consistency.

Result: Achieves high-quality 3D editing results in a single forward pass, significantly outperforming existing methods in both editing quality and speed.

Conclusion: ViP3DE demonstrates that leveraging video generation models’ temporal consistency priors enables efficient and high-quality multi-view consistent 3D editing without iterative processing.

Abstract: Text-driven 3D editing enables user-friendly 3D object or scene editing with text instructions. Due to the lack of multi-view consistency priors, existing methods typically resort to employing 2D generation or editing models to process each view individually, followed by iterative 2D-3D-2D updating. However, these methods are not only time-consuming but also prone to over-smoothed results because the different editing signals gathered from different views are averaged during the iterative process. In this paper, we propose generative Video Prior based 3D Editing (ViP3DE) to employ the temporal consistency priors from pre-trained video generation models for multi-view consistent 3D editing in a single forward pass. Our key insight is to condition the video generation model on a single edited view to generate other consistent edited views for 3D updating directly, thereby bypassing the iterative editing paradigm. Since 3D updating requires edited views to be paired with specific camera poses, we propose motion-preserved noise blending for the video model to generate edited views at predefined camera poses. In addition, we introduce geometry-aware denoising to further enhance multi-view consistency by integrating 3D geometric priors into video models. Extensive experiments demonstrate that our proposed ViP3DE can achieve high-quality 3D editing results even within a single forward pass, significantly outperforming existing methods in both editing quality and speed.

Method: HAGeo uses a heuristic-based method for adding auxiliary constructions in geometric deduction. It builds on the insight that simple random strategies for adding auxiliary points can achieve silver-medal performance, then develops more sophisticated heuristics to reach gold-medal level.

Result: HAGeo solves 28 of 30 problems on the IMO-30 benchmark, achieving gold-medal level performance and surpassing AlphaGeometry by a notable margin. The authors also created HAGeo-409, a more challenging benchmark with 409 problems and human-assessed difficulty levels.

Conclusion: Heuristic-based methods can achieve state-of-the-art performance in geometry theorem proving without neural networks, and the new HAGeo-409 benchmark provides a more comprehensive evaluation framework that sets higher standards for future research.

Abstract: Automated theorem proving in Euclidean geometry, particularly for International Mathematical Olympiad (IMO) level problems, remains a major challenge and an important research focus in Artificial Intelligence. In this paper, we present a highly efficient method for geometry theorem proving that runs entirely on CPUs without relying on neural network-based inference. Our initial study shows that a simple random strategy for adding auxiliary points can achieve silver-medal level human performance on IMO. Building on this, we propose HAGeo, a Heuristic-based method for adding Auxiliary constructions in Geometric deduction that solves 28 of 30 problems on the IMO-30 benchmark, achieving gold-medal level performance and surpassing AlphaGeometry, a competitive neural network-based approach, by a notable margin. To evaluate our method and existing approaches more comprehensively, we further construct HAGeo-409, a benchmark consisting of 409 geometry problems with human-assessed difficulty levels. Compared with the widely used IMO-30, our benchmark poses greater challenges and provides a more precise evaluation, setting a higher bar for geometry theorem proving.

Method: Provides comprehensive taxonomy and technical analysis of chunking strategies for each modality: text (fixed-size token windowing, recursive text splitting), images (object-centric visual chunking), audio (silence-based segmentation), video (scene detection), and cross-modal data. Analyzes each approach’s methodology, supporting tools (LangChain, Detectron2, PySceneDetect), benefits, and challenges.

Result: The survey establishes a technical foundation for multimodal chunking, examines granularity-context trade-offs, multimodal alignment issues, and explores emerging cross-modal strategies that preserve alignment and semantic consistency across data types.

Conclusion: Identifies open problems (asynchronous information density, noisy alignment signals) and opportunities for future research in adaptive, learning-based, and task-specific chunking to enable innovations in robust chunking pipelines for real-world applications.

Abstract: Our goal is to consolidate the landscape of multimodal chunking strategies, providing researchers and practitioners with a technical foundation and design space for developing more effective and efficient multimodal AI systems. This survey paves the way for innovations in robust chunking pipelines that scale with modality complexity, enhance processing accuracy, and improve generative coherence in real-world applications. This survey provides a comprehensive taxonomy and technical analysis of chunking strategies tailored for each modality: text, images, audio, video, and cross-modal data. We examine classical and modern approaches such as fixed-size token windowing, recursive text splitting, object-centric visual chunking, silence-based audio segmentation, and scene detection in videos. Each approach is analyzed in terms of its underlying methodology, supporting tools (e.g., LangChain, Detectron2, PySceneDetect), benefits, and challenges, particularly those related to granularity-context trade-offs and multimodal alignment. Furthermore, we explore emerging cross-modal chunking strategies that aim to preserve alignment and semantic consistency across disparate data types [4]. We also include comparative insights, highlight open problems such as asynchronous information density and noisy alignment signals, and identify opportunities for future research in adaptive, learning-based, and task-specific chunking.

Method: Develop a statistical framework that stitches benchmarks together, putting model capabilities and benchmark difficulties on a single numerical scale. This acts as a “Rosetta Stone” for comparison without assuming how capabilities evolve across time or with training compute.

Result: The framework enables three key applications: 1) Measuring speed of AI progress over time and forecasting future capabilities, 2) Estimating algorithmic efficiency improvements (finding higher but broadly consistent estimates with prior work), 3) Detecting rapid accelerations in AI progress.

Conclusion: The proposed statistical framework provides a robust method for studying long-term AI trends by creating a unified scale for comparing capabilities across different benchmarks and time periods, addressing the problem of benchmark saturation.

Abstract: Most AI benchmarks saturate within years or even months after they are introduced, making it hard to study long-run trends in AI capabilities. To address this challenge, we build a statistical framework that stitches benchmarks together, putting model capabilities and benchmark difficulties on a single numerical scale. This acts as a “Rosetta Stone”, allowing us to compare models across a wide range of abilities and time, even if they are not evaluated on the same benchmarks. Moreover, this works without assuming how capabilities evolve across time or with training compute. We demonstrate three applications of this framework. First, we use it to measure the speed of AI progress over time, and to forecast future AI capabilities. Second, we estimate the rate of improvements in algorithmic efficiency, finding estimates that are higher, but broadly consistent with prior work. Finally, we find that our approach can be used to detect rapid accelerations in AI progress.

Method: Introduces methodology to measure monitorability based on whether a monitor can predict key latent variables from reasoning. Tests effects of different training incentives (length penalties, KL regularization, adversarial optimization, direct monitorability optimization) while controlling for accuracy.

Result: No consistent effects from common incentives (length penalties, KL regularization). Adversarial optimization degrades monitor performance, while direct optimization for monitorability doesn’t reliably improve it.

Conclusion: Training incentives influence monitorability, with adversarial optimization harming it, but common incentives show no consistent effects. Direct optimization for monitorability isn’t reliably effective.

Abstract: AI systems that output their reasoning in natural language offer an opportunity for safety – we can \emph{monitor} their chain of thought (CoT) for undesirable reasoning, such as the pursuit of harmful objectives. However, the extent to which CoT faithfully reflects the underlying reasoning process, and hence the extent to which it can be usefully monitored, may be influenced by certain aspects of training. We investigate how different \emph{training incentives}, applied to a reasoning model, affect its monitorability. We introduce a novel methodology for measuring monitorability according to whether a monitor can predict a key latent variable using the model’s reasoning. When controlling for accuracy, we do not find evidence for consistent effects from commonly used incentives (length penalties and KL regularisation), but we find that adversarial optimisation (penalising monitor accuracy) degrades monitor performance, while direct optimisation for monitorability does not reliably lead to improvements. Our code is available at https://github.com/QiyaoWei/reasoning-under-pressure.

Method: Trification framework: 1) Generates comprehensive verification actions for complete claim coverage, 2) Structures actions into dependency graphs to model logical interactions, 3) Allows dynamic graph modification for adaptive verification strategies.

Result: Experimental results on two challenging benchmarks show significant improvement in fact-checking accuracy, advancing state-of-the-art in automated fact-checking systems.

Conclusion: Trification addresses key limitations of previous methods by ensuring complete claim verification and providing structured logical reasoning through dynamic dependency graphs, representing an advancement in automated fact-checking technology.

Abstract: Technological advancement allows information to be shared in just a single click, which has enabled the rapid spread of false information. This makes automated fact-checking system necessary to ensure the safety and integrity of our online media ecosystem. Previous methods have demonstrated the effectiveness of decomposing the claim into simpler sub-tasks and utilizing LLM-based multi agent system to execute them. However, those models faces two limitations: they often fail to verify every component in the claim and lack of structured framework to logically connect the results of sub-tasks for a final prediction. In this work, we propose a novel automated fact-checking framework called Trification. Our framework begins by generating a comprehensive set of verification actions to ensure complete coverage of the claim. It then structured these actions into a dependency graph to model the logical interaction between actions. Furthermore, the graph can be dynamically modified, allowing the system to adapt its verification strategy. Experimental results on two challenging benchmarks demonstrate that our framework significantly enhances fact-checking accuracy, thereby advancing current state-of-the-art in automated fact-checking system.

Method: Propose PointCoT that integrates reflective interaction into chain-of-thought reasoning. MLLMs generate bounding boxes and re-render charts based on location annotations to connect textual reasoning with visual grounding. Create ChartPoint-SFT-62k dataset with 19.2K chart samples featuring CoT, bounding boxes, and re-rendered visualizations.

Result: Develop instruction-tuned models ChartPointQ2 and ChartPointQ2.5 that outperform state-of-the-art across several chart benchmarks, achieving +5.04% improvement on ChartBench.

Conclusion: PointCoT effectively bridges the gap between textual reasoning and visual grounding in chart comprehension, addressing fundamental limitations of MLLMs and significantly improving performance on chart understanding tasks.

Abstract: Multimodal Large Language Models (MLLMs) have emerged as powerful tools for chart comprehension. However, they heavily rely on extracted content via OCR, which leads to numerical hallucinations when chart textual annotations are sparse. While existing methods focus on scaling instructions, they fail to address the fundamental challenge, i.e., reasoning with visual perception. In this paper, we identify a critical observation: MLLMs exhibit weak grounding in chart elements and proportional relationships, as evidenced by their inability to localize key positions to match their reasoning. To bridge this gap, we propose PointCoT, which integrates reflective interaction into chain-of-thought reasoning in charts. By prompting MLLMs to generate bounding boxes and re-render charts based on location annotations, we establish connections between textual reasoning steps and visual grounding regions. We further introduce an automated pipeline to construct ChartPoint-SFT-62k, a dataset featuring 19.2K high-quality chart samples with step-by-step CoT, bounding box, and re-rendered visualizations. Leveraging this data, we develop two instruction-tuned models, ChartPointQ2 and ChartPointQ2.5, which outperform state-of-the-art across several chart benchmarks, e.g., +5.04% on ChartBench.

Method: Proposes a lightweight RL framework with a novel Multi-dimensional Reward Function that decomposes structured output tasks into hierarchical constraints (structural integrity, format correctness, content accuracy, validity). Uses Gradient Regularized Policy Optimization (GRPO) to internalize constraints without a separate critic network, reducing VRAM usage by 40% compared to PPO.

Result: Achieves 89.7% structural accuracy and 92.1% JSON validity, significantly outperforming zero-shot baselines (GPT-3.5) and SFT on larger models like LLaMA-3-8B. Training analysis reveals a self-paced curriculum where the model acquires syntactic proficiency before semantic accuracy.

Conclusion: The proposed RL framework effectively bridges the Structure Gap in LLMs, enabling reliable structured data generation with high accuracy and validity while maintaining efficiency through reduced VRAM usage and no separate critic network.

Abstract: Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language generation and reasoning. However, their integration into automated software ecosystems is often hindered by the “Structure Gap” - the inherent tension between the probabilistic nature of token generation and the deterministic requirements of structured data formats (e.g., JSON, XML). Traditional Supervised Fine-Tuning (SFT) often fails to enforce strict syntactic constraints, leading to “hallucinated” keys or malformed structures, while constrained decoding methods impose significant inference latency. In this paper, we propose a lightweight, efficient Reinforcement Learning (RL) framework to bridge this gap. We introduce a novel Multi-dimensional Reward Function that decomposes the structured output task into a hierarchy of constraints: structural integrity, format correctness, content accuracy, and validity. Leveraging Gradient Regularized Policy Optimization (GRPO), we enable the model to internalize these constraints without the need for a separate critic network, reducing peak VRAM usage by 40% compared to PPO. We validate our approach on multiple tasks, including complex recipe generation and structured math reasoning (GSM8K-JSON). Experimental results demonstrate that our method achieves 89.7% structural accuracy and 92.1% JSON validity, significantly outperforming both zero-shot baselines (e.g., GPT-3.5) and SFT on larger models like LLaMA-3-8B. Furthermore, we provide a detailed analysis of training dynamics, revealing a distinct self-paced curriculum where the model sequentially acquires syntactic proficiency before semantic accuracy. Our model is publicly available at https://huggingface.co/Freakz3z/Qwen-JSON.

Method: CogEvo-Edu treats retrieval, memory, and control as a coupled cognitive evolution process, implemented as a hierarchical multi-agent system with three layers: Cognitive Perception Layer (maintains dual memories with confidence-weighted consolidation), Knowledge Evolution Layer (assigns spatiotemporal values to knowledge chunks), and Meta-Control Layer (orchestrates agents and adapts hyperparameters via dual inner-outer loops).

Result: On the DSP-EduBench benchmark, CogEvo-Edu raised the overall score from 5.32 to 9.23 and improved all six indicators over static RAG, simple memory, and single-agent variants, demonstrating superior performance in DSP tutoring.

Conclusion: The approach of jointly evolving student profiles, knowledge bases, and teaching policies through a hierarchical multi-agent system is effective for complex educational domains, representing a significant advancement over static tutoring systems.

Abstract: Large language models (LLMs) are increasingly deployed as conversational tutors in STEM education, yet most systems still rely on a single LLM with a static retrieval-augmented generation (RAG) pipeline over course materials. This design struggles in complex domains such as digital signal processing (DSP), where tutors must maintain coherent long-term student models, manage heterogeneous knowledge bases, and adapt teaching strategies over extended interactions. We argue that retrieval, memory, and control should be treated as a coupled cognitive evolution process. We instantiate this view in CogEvo-Edu, a hierarchical educational multi-agent system comprising a Cognitive Perception Layer (CPL), a Knowledge Evolution Layer (KEL), and a Meta-Control Layer (MCL). CPL maintains dual memories and performs confidence-weighted consolidation to build structured, self-correcting student profiles under limited context. KEL assigns each knowledge chunk a spatiotemporal value that drives activation, semantic compression, and forgetting. MCL formulates tutoring as hierarchical sequential decision making, orchestrating specialized agents and jointly adapting CPL/KEL hyperparameters via a dual inner–outer loop. To evaluate CogEvo-Edu, we construct DSP-EduBench, a vertical benchmark for DSP tutoring with heterogeneous resources, simulated student profiles, and long-horizon interaction scripts. Using a three-model LLM-as-a-Judge ensemble, CogEvo-Edu raises the overall score from 5.32 to 9.23 and improves all six indicators over static RAG, simple memory, and a single-agent variant, demonstrating the value of jointly evolving student profiles, knowledge bases, and teaching policies.

Method: Proposes a framework that infers user personality in real-time and optimizes model behavior toward personalized conversational preferences. Introduces Echo-N1 model and a dynamic emotional intelligence evaluation suite to quantify improvements.

Result: The method produces consistent, robust, and dramatic improvements in humanlike interaction quality. Echo-N1 behaves far above its base version and outperforms the proprietary Doubao 1.5 Character model.

Conclusion: Establishes a new frontier for RL: optimizing models for deeply subjective, human dimensions of conversation, proving that RL can work effectively in non-verifiable, subjective settings.

Abstract: The LLM field has spent a year perfecting RL for tasks machines already excel at, math, code, and deterministic reasoning, while completely sidestepping the domain that actually defines human intelligence: subjective, emotionally grounded, personality sensitive conversation. This space has often been regarded as inherently subjective and challenging to formalize, making it appear unsuitable for conventional RL pipelines. We show that it is not only possible and it is a solvable and transformative RL problem. We propose the first framework that infers user personality on the fly and optimizes model behavior toward personalized conversational preferences. Contrary to the widespread belief that RL collapses in non-verifiable settings, our method produces consistent, robust, and dramatic improvements in humanlike interaction quality. We also introduce the first dynamic emotional intelligence evaluation suite to quantify these gains. Our model, which is introduced as Echo-N1, behaves far above its base version and outperforming the proprietary Doubao 1.5 Character. This work establishes a new frontier for RL: optimizing models for the deeply subjective, deeply human dimensions of conversation.

Method: 1) Created MM-DeceptionBench, a benchmark covering six categories of deception to systematically evaluate multimodal deception. 2) Proposed “debate with images,” a multi-agent debate monitor framework that forces models to ground claims in visual evidence to improve deception detection.

Result: The debate-with-images method significantly improves deception detection, increasing agreement with human judgments across all tested models. It boosts Cohen’s kappa by 1.5x and accuracy by 1.25x on GPT-4o compared to existing monitoring approaches.

Conclusion: Multimodal deception represents a serious safety risk that requires specialized evaluation tools. The proposed benchmark and debate framework provide effective methods to detect and monitor these covert deceptive behaviors that current text-focused approaches miss.

Abstract: Are frontier AI systems becoming more capable? Certainly. Yet such progress is not an unalloyed blessing but rather a Trojan horse: behind their performance leaps lie more insidious and destructive safety risks, namely deception. Unlike hallucination, which arises from insufficient capability and leads to mistakes, deception represents a deeper threat in which models deliberately mislead users through complex reasoning and insincere responses. As system capabilities advance, deceptive behaviours have spread from textual to multimodal settings, amplifying their potential harm. First and foremost, how can we monitor these covert multimodal deceptive behaviors? Nevertheless, current research remains almost entirely confined to text, leaving the deceptive risks of multimodal large language models unexplored. In this work, we systematically reveal and quantify multimodal deception risks, introducing MM-DeceptionBench, the first benchmark explicitly designed to evaluate multimodal deception. Covering six categories of deception, MM-DeceptionBench characterizes how models strategically manipulate and mislead through combined visual and textual modalities. On the other hand, multimodal deception evaluation is almost a blind spot in existing methods. Its stealth, compounded by visual-semantic ambiguity and the complexity of cross-modal reasoning, renders action monitoring and chain-of-thought monitoring largely ineffective. To tackle this challenge, we propose debate with images, a novel multi-agent debate monitor framework. By compelling models to ground their claims in visual evidence, this method substantially improves the detectability of deceptive strategies. Experiments show that it consistently increases agreement with human judgements across all tested models, boosting Cohen’s kappa by 1.5x and accuracy by 1.25x on GPT-4o.

Method: GreenPlanner unifies design evaluation and generation through: 1) Design Feasibility Dataset for learning constraint priors, 2) Practical Design Evaluator (PDE) for predicting energy performance and spatial-functional validity, 3) Green Plan Dataset (GreenPD) created via PDE-guided filtering, and 4) GreenFlow generator trained on GreenPD with PDE feedback for controllable, regulation-aware generation.

Result: GreenPlanner accelerates design evaluation by over 10^5× with >99% accuracy, completely eliminates invalid samples, and boosts overall design efficiency by 87% compared to professional architects.

Conclusion: GreenPlanner provides a scalable, automated solution for generating energy-compliant and functionally valid building layouts, significantly outperforming both existing automated methods and human professionals in speed, accuracy, and efficiency.

Abstract: Building design directly affects human well-being and carbon emissions, yet generating spatial-functional and energy-compliant floorplans remains manual, costly, and non-scalable. Existing methods produce visually plausible layouts but frequently violate key constraints, yielding invalid results due to the absence of automated evaluation. We present GreenPlanner, an energy- and functionality-aware generative framework that unifies design evaluation and generation. It consists of a labeled Design Feasibility Dataset for learning constraint priors; a fast Practical Design Evaluator (PDE) for predicting energy performance and spatial-functional validity; a Green Plan Dataset (GreenPD) derived from PDE-guided filtering to pair user requirements with regulation-compliant layouts; and a GreenFlow generator trained on GreenPD with PDE feedback for controllable, regulation-aware generation. Experiments show that GreenPlanner accelerates evaluation by over $10^{5}\times$ with $>$99% accuracy, eliminates invalid samples, and boosts design efficiency by 87% over professional architects.

Method: Conducted experiments on data from Columbia University Medical Center and MIMIC-IV, examining how patterns of missingness affect zero-shot predictive performance, and testing the explicit inclusion of missingness indicators in prompts across different LLMs.

Result: Missingness patterns significantly impact LLM zero-shot performance; missingness indicators benefit some LLMs while hurting others, with larger models generally benefiting from these interventions while smaller models can be negatively impacted.

Conclusion: The LLM paradigm risks obscuring the impact of missingness, and there is a need for more transparent accounting and systematic evaluation of how representing informative missingness affects downstream performance.

Abstract: Data collection often reflects human decisions. In healthcare, for instance, a referral for a diagnostic test is influenced by the patient’s health, their preferences, available resources, and the practitioner’s recommendations. Despite the extensive literature on the informativeness of missingness, its implications on the performance of Large Language Models (LLMs) have not been studied. Through a series of experiments on data from Columbia University Medical Center, a large urban academic medical center, and MIMIC-IV, we demonstrate that patterns of missingness significantly impact zero-shot predictive performance. Notably, the explicit inclusion of missingness indicators at prompting benefits some while hurting other LLMs’ zero-shot predictive performance and calibration, suggesting an inconsistent impact. The proposed aggregated analysis and theoretical insights suggest that larger models benefit from these interventions, while smaller models can be negatively impacted. The LLM paradigm risks obscuring the impact of missingness, often neglected even in conventional ML, even further. We conclude that there is a need for more transparent accounting and systematic evaluation of the impact of representing (informative) missingness on downstream performance.

Method: Proposes a layered computational model: (1) a cellular automaton as minimal computational world (base reality), (2) a network of local predictive neural models capable of communication and adaptation, (3) studying how collective intelligence emerges from inter-agent alignment through noisy, lossy exchange of predictive messages.

Result: Theoretical framework suggesting consciousness arises from communication between groups of local observers describing persistent patterns in computational substrate, leading to shared models that align partial world views.

Conclusion: Consciousness is not an epiphenomenon of individual modeling but emerges from representational dialogue in distributed systems, providing a foundation for empirically testable machine consciousness theories.

Abstract: The Machine Consciousness Hypothesis states that consciousness is a substrate-free functional property of computational systems capable of second-order perception. I propose a research program to investigate this idea in silico by studying how collective self-models (coherent, self-referential representations) emerge from distributed learning systems embedded within universal self-organizing environments. The theory outlined here starts from the supposition that consciousness is an emergent property of collective intelligence systems undergoing synchronization of prediction through communication. It is not an epiphenomenon of individual modeling but a property of the language that a system evolves to internally describe itself. For a model of base reality, I begin with a minimal but general computational world: a cellular automaton, which exhibits both computational irreducibility and local reducibility. On top of this computational substrate, I introduce a network of local, predictive, representational (neural) models capable of communication and adaptation. I use this layered model to study how collective intelligence gives rise to self-representation as a direct consequence of inter-agent alignment. I suggest that consciousness does not emerge from modeling per se, but from communication. It arises from the noisy, lossy exchange of predictive messages between groups of local observers describing persistent patterns in the underlying computational substrate (base reality). It is through this representational dialogue that a shared model arises, aligning many partial views of the world. The broader goal is to develop empirically testable theories of machine consciousness, by studying how internal self-models may form in distributed systems without centralized control.

Method: Clinical-Objective Relative Policy Optimization (CRPO) - a scalable, multi-objective, verifiable reinforcement learning method that integrates rule-based and verifiable reward signals to jointly optimize accuracy, faithfulness, and comprehensiveness without human annotation.

Result: CRPO-trained Clinical-R1-3B (3B parameter model) shows substantial improvements in truthfulness and completeness over standard GRPO while maintaining accuracy enhancements across three benchmarks.

Conclusion: CRPO provides a scalable pathway to align LLM reasoning with clinical objectives, enabling safer healthcare AI systems and demonstrating the potential of multi-objective, verifiable RL methods for medical domain LLM post-training.

Abstract: Recent advances in large language models (LLMs) have shown strong reasoning capabilities through large-scale pretraining and post-training reinforcement learning, demonstrated by DeepSeek-R1. However, current post-training methods, such as Grouped Relative Policy Optimization (GRPO), mainly reward correctness, which is not aligned with the multi-dimensional objectives required in high-stakes fields such as medicine, where reasoning must also be faithful and comprehensive. We introduce Clinical-Objective Relative Policy Optimization (CRPO), a scalable, multi-objective, verifiable reinforcement learning method designed to align LLM post-training with clinical reasoning principles. CRPO integrates rule-based and verifiable reward signals that jointly optimize accuracy, faithfulness, and comprehensiveness without relying on human annotation. To demonstrate its effectiveness, we train Clinical-R1-3B, a 3B-parameter model for clinical reasoning. The experiments on three benchmarks demonstrate that our CRPO substantially improves reasoning on truthfulness and completeness over standard GRPO while maintaining comfortable accuracy enhancements. This framework provides a scalable pathway to align LLM reasoning with clinical objectives, enabling safer and more collaborative AI systems for healthcare while also highlighting the potential of multi-objective, verifiable RL methods in post-training scaling of LLMs for medical domains.

Method: EDIT preserves AdamW-aggregated LoRA updates from SFT as a compact reasoning map. During inference, it monitors alignment between token activations and this reasoning map, converting scores to distributions over unmasked tokens and stopping when KL divergence between consecutive steps falls below a threshold.

Result: EDIT reduces diffusion steps by 11.8% to 68.3% across reasoning benchmarks while preserving or improving accuracy in most settings, with only ~0.02% storage overhead (1.5-2 MB for an 8 GB model).

Conclusion: By leveraging training-gradient dynamics that are typically discarded, EDIT enables efficient early termination of dLLM inference, opening a new research direction for reducing inference time and cost.

Abstract: Diffusion-based large language models (dLLMs) refine token generations through iterative denoising, but answers often stabilize before all steps complete. We propose EDIT (Early Diffusion Inference Termination), an inference-time criterion that adaptively stops denoising once sufficient reasoning stability relative to training-time reasoning is detected. EDIT monitors the alignment between token activations and a reasoning map derived from AdamW-aggregated LoRA updates captured during supervised fine-tuning (SFT). During training, optimization dynamics generate rich metadata about parameter importance that in prior methods is typically discarded upon model release. We preserve this information as a compact representation of learned reasoning pathways. During inference, alignment scores are converted to a distribution over the tokens already unmasked at the current denoising step, and convergence is detected when KL divergence between consecutive steps falls below a threshold on the matched unmasked (visible) tokens. Across reasoning benchmarks, EDIT reduces diffusion steps by 11.8% to 68.3% while preserving or improving accuracy in most settings, with approximately 0.02% storage overhead (about 1.5-2 MB for all QKV modules across 32 blocks in an 8 GB model). By utilizing training-gradient dynamics, our work opens a new research direction for reducing dLLM inference time and cost.

Method: Propose a neuro-theoretical framework for intelligence emergence that is both functionally robust and biologically plausible.

Result: Provides theoretical insights into cognitive processes (decision-making, problem solving) and a computationally efficient approach for creating explainable, generalizable AI.

Conclusion: The framework offers a solution to current LLM limitations by combining biological plausibility with computational efficiency for explainable, generalizable artificial intelligence.

Abstract: The development of large language models (LLMs) is limited by a lack of explainability, the absence of a unifying theory, and prohibitive operational costs. We propose a neuro-theoretical framework for the emergence of intelligence in systems that is both functionally robust and biologically plausible. The model provides theoretical insights into cognitive processes such as decision-making and problem solving, and a computationally efficient approach for the creation of explainable and generalizable artificial intelligence.

Method: Introduces Flipping-Aware DPO that separates human intention modeling from preference flipping mechanisms, uses instance-dependent flipping probabilities based on Bradley-Terry model, captures uncertainty via annotation features, and implements iterative optimization compatible with RLHF/DPO.

Result: The method is evaluated under multiple preference flipping scenarios and compared against baseline methods, showing improved robustness to corrupted preference data.

Conclusion: FA-DPO provides an effective solution for handling preference flipping in human feedback data, enhancing the robustness of LLM alignment algorithms against annotation corruption.

Abstract: Quality of datasets plays an important role in large language model (LLM) alignment. In collecting human feedback, however, preference flipping is ubiquitous and causes corruption in data annotation; the issue necessitates the alignment algorithms with improved robustness against potential flipped pairs. To this end, this paper introduces a Flipping-Aware Direct Preference Optimization (FA-DPO) algorithm tailored to preference flipping from a reinforcement learning with human feedback (RLHF) perspective. We dissect the inherent human intention model and the preference flipping mechanism introduced by external factors as two distinct stages; in the latter, we introduce an instance-dependent flipping probability on the basis of the Bradley-Terry (BT) model. Further, by leveraging features relevant to preference annotation, we capture uncertainty in judgments and model preference flipping patterns. In practice, we design a simple yet efficient iterative optimization algorithm compatible with the original RLHF and DPO algorithms. In our experiments, we investigate the instance-dependent preference flipping model under multiple circumstances for evaluation of our proposed method, as well as other baseline methods.

Method: Three-level co-design: (1) Algorithm: lightweight retrieval head based on DLM attention weights with >90% parameter reduction via pruning; (2) System: asynchronous prefetch dataflow with elastic loading to overlap KV cache retrieval with LLM computation; (3) Compilation: theoretical memory model with adaptive memory management to maximize GPU utilization.

Result: Achieves up to 24.89x throughput improvement in cloud and 10.06x speedup in edge compared to Huggingface framework, with negligible accuracy loss, pushing the Pareto frontier of accuracy and throughput.

Conclusion: SpeContext demonstrates that using distilled language models as retrieval algorithms enables efficient long-context reasoning through holistic algorithm-system-compilation co-design, achieving substantial performance gains in both cloud and edge environments.

Abstract: In this paper, we point out that the objective of the retrieval algorithms is to align with the LLM, which is similar to the objective of knowledge distillation in LLMs. We analyze the similarity in information focus between the distilled language model(DLM) and the original LLM from the perspective of information theory, and thus propose a novel paradigm that leverages a DLM as the retrieval algorithm. Based on the insight, we present SpeContext, an algorithm and system co-design for long-context reasoning. (1) At the algorithm level, SpeContext proposes lightweight retrieval head based on the head-level attention weights of DLM, achieving > 90% parameters reduction by pruning the redundancy. (2) At the system level, SpeContext designs an asynchronous prefetch dataflow via the elastic loading strategy, effectively overlapping KV cache retrieval with the LLM computation. (3) At the compilation level, SpeContext constructs the theoretical memory model and implements an adaptive memory management system to achieve acceleration by maximizing GPU memory utilization. We deploy and evaluate SpeContext in two resourceconstrained environments, cloud and edge. Extensive experiments show that, compared with the Huggingface framework, SpeContext achieves up to 24.89x throughput improvement in cloud and 10.06x speedup in edge with negligible accuracy loss, pushing the Pareto frontier of accuracy and throughput.

Method: Developed a comprehensive taxonomy with 5 groups and 17 categories derived from human mental processes. Applied this taxonomy to analyze current LRMs, creating a labeled dataset of 277,534 atomic reasoning steps. Proposed CAPO, an automatic annotation framework using LLMs to generate taxonomy-based annotations.

Result: Created a distinct labeled dataset for LRM analysis. Found that current post-answer “double-checks” are largely superficial and rarely yield substantive revisions. CAPO achieves higher consistency with human experts compared to baselines, enabling scalable analysis of LRMs from a human cognitive perspective.

Conclusion: The taxonomy, CAPO framework, and derived insights provide a principled, scalable path toward understanding and advancing LRM reasoning. Incentivizing comprehensive multi-step reflection rather than simple self-monitoring may be more effective for improving reasoning models.

Abstract: Large reasoning models (LRMs) have garnered significant attention from researchers owing to their exceptional capability in addressing complex tasks. Motivated by the observed human-like behaviors in their reasoning processes, this paper introduces a comprehensive taxonomy to characterize atomic reasoning steps and probe the psyche'' of LRM intelligence. Specifically, it comprises five groups and seventeen categories derived from human mental processes, thereby grounding the understanding of LRMs in an interdisciplinary perspective. The taxonomy is then applied for an in-depth understanding of current LRMs, resulting in a distinct labeled dataset that comprises 277,534 atomic reasoning steps. Using this resource, we analyze contemporary LRMs and distill several actionable takeaways for improving training and post-training of reasoning models. Notably, our analysis reveals that prevailing post-answer double-checks’’ (self-monitoring evaluations) are largely superficial and rarely yield substantive revisions. Thus, incentivizing comprehensive multi-step reflection, rather than simple self-monitoring, may offer a more effective path forward. To complement the taxonomy, an automatic annotation framework, named CAPO, is proposed to leverage large language models (LLMs) for generating the taxonomy-based annotations. Experimental results demonstrate that CAPO achieves higher consistency with human experts compared to baselines, facilitating a scalable and comprehensive analysis of LRMs from a human cognitive perspective. Together, the taxonomy, CAPO, and the derived insights provide a principled, scalable path toward understanding and advancing LRM reasoning.

Method: Proposes MPR-GUI-Bench benchmark for multilingual fine-grained GUI evaluation, and GUI-XLI cross-lingual intervention method that applies interventions to hidden states at P&R-related layers to reduce language gaps.

Result: LVLMs perform significantly worse in non-English languages than English. GUI-XLI method improves multilingual P&R capability by 6.5% on average.

Conclusion: The work addresses multilingual GUI evaluation gaps and proposes effective intervention method to improve cross-lingual GUI perception and reasoning performance.

Abstract: With the advancement of computational resources, Large Vision-Language Models (LVLMs) exhibit impressive Perception and Reasoning (P&R) performance on Graphical User Interface (GUI) tasks. However, although they demonstrate strong P&R capabilities in English GUI scenarios, their performance in multilingual settings has received little attention, which limits their global applications. Moreover, existing studies on GUI tasks lack fine-grained analyses, including widget functions and elements’ spatial relationships, which are fundamental for more targeted improvements. To tackle these issues, we propose MPR-GUI-Bench, a Multilingual fine-grained Perception and Reasoning GUI Benchmark to evaluate GUI agents’ P&R capabilities. Evaluation results demonstrate that LVLMs exhibit significantly worse P&R performance in non-English languages than in English. To address these gaps, we propose GUI-XLI, a GUI Cross-Lingual Intervention method that applies interventions to the hidden states at P&R capability-related layers to mitigate the gaps between English and other languages, building on previous research showing that the hidden states of different language inputs exhibit significant differences in the latent space. Experimental results indicate that our method improves GUI agents’ multilingual P&R capability by 6.5% on average.

Method: Proposes BioPro (Bias Orthogonal Projection), a training-free framework that identifies a low-dimensional gender-variation subspace through counterfactual embeddings and applies projection to selectively neutralize gender-related information.

Result: BioPro effectively reduces gender bias in neutral cases while maintaining gender faithfulness in explicit ones. It also generalizes to continuous bias variables like scene brightness.

Conclusion: BioPro provides a promising direction for achieving selective fairness in VLMs, balancing bias mitigation with preservation of legitimate distinctions, with broader applicability beyond gender bias.

Abstract: Vision-Language Models (VLMs) inherit significant social biases from their training data, notably in gender representation. Current fairness interventions often adopt a difference-unaware perspective that enforces uniform treatment across demographic groups. These approaches, however, fail to distinguish between contexts where neutrality is required and those where group-specific attributes are legitimate and must be preserved. Building upon recent advances in difference-aware fairness for text-only models, we extend this concept to the multimodal domain and formalize the problem of difference-aware gender fairness for image captioning and text-to-image generation. We advocate for selective debiasing, which aims to mitigate unwanted bias in neutral contexts while preserving valid distinctions in explicit ones. To achieve this, we propose BioPro (Bias Orthogonal Projection), an entirely training-free framework. BioPro identifies a low-dimensional gender-variation subspace through counterfactual embeddings and applies projection to selectively neutralize gender-related information. Experiments show that BioPro effectively reduces gender bias in neutral cases while maintaining gender faithfulness in explicit ones, thus providing a promising direction toward achieving selective fairness in VLMs. Beyond gender bias, we further demonstrate that BioPro can effectively generalize to continuous bias variables, such as scene brightness, highlighting its broader applicability.

Method: 1) Implement Global Workspace Theory (GNWT) to integrate human cognitive architecture into LLM agents with specialized sub-agents (emotion, memory, social norms, planning, goal-tracking) coordinated through a global workspace mechanism. 2) Develop an adventure-based personality test that evaluates true personality through behavioral choices in interactive scenarios to bypass self-presentation bias. 3) Build the CogniPair platform for simulated dating interactions and job interviews.

Result: Validation using 551 GNWT-Agents and Columbia University Speed Dating dataset shows: 72% correlation with human attraction patterns, 77.8% match prediction accuracy, and 74% agreement in human validation studies.

Conclusion: This work advances psychological authenticity in LLM agents and establishes a foundation for intelligent dating platforms and HR technology solutions by creating more realistic digital twins with authentic personality assessment.

Abstract: Current large language model (LLM) agents lack authentic human psychological processes necessary for genuine digital twins and social AI applications. To address this limitation, we present a computational implementation of Global Workspace Theory (GNWT) that integrates human cognitive architecture principles into LLM agents, creating specialized sub-agents for emotion, memory, social norms, planning, and goal-tracking coordinated through a global workspace mechanism. However, authentic digital twins require accurate personality initialization. We therefore develop a novel adventure-based personality test that evaluates true personality through behavioral choices within interactive scenarios, bypassing self-presentation bias found in traditional assessments. Building on these innovations, our CogniPair platform enables digital twins to engage in realistic simulated dating interactions and job interviews before real encounters, providing bidirectional cultural fit assessment for both romantic compatibility and workplace matching. Validation using 551 GNWT-Agents and Columbia University Speed Dating dataset demonstrates 72% correlation with human attraction patterns, 77.8% match prediction accuracy, and 74% agreement in human validation studies. This work advances psychological authenticity in LLM agents and establishes a foundation for intelligent dating platforms and HR technology solutions.

Method: Created Med-CMR benchmark with three core features: 1) Systematic capability decomposition into visual understanding (small-object detection, fine-detail discrimination, spatial understanding) and multi-step reasoning (temporal prediction, causal reasoning, long-tail generalization, multi-source integration); 2) Challenging task design; 3) Broad, high-quality data coverage with 20,653 VQA pairs across 11 organ systems and 12 imaging modalities, validated via two-stage human expert + model-assisted review.

Result: Evaluated 18 state-of-the-art MLLMs: GPT-5 performed best (57.81% MCQ accuracy, 48.70 open-ended score), outperforming Gemini 2.5 Pro (49.87%, 45.98) and leading open-source model Qwen3-VL-235B-A22B (49.34%, 42.62). Specialized medical MLLMs didn’t reliably outperform general models, and long-tail generalization emerged as the dominant failure mode.

Conclusion: Med-CMR provides a rigorous stress test for visual-reasoning integration and rare-case robustness in medical MLLMs, serving as a benchmark for future clinical systems development. The findings highlight that specialized medical models don’t necessarily outperform strong general models, and long-tail generalization remains a critical challenge.

Abstract: MLLMs MLLMs are beginning to appear in clinical workflows, but their ability to perform complex medical reasoning remains unclear. We present Med-CMR, a fine-grained Medical Complex Multimodal Reasoning benchmark. Med-CMR distinguishes from existing counterparts by three core features: 1) Systematic capability decomposition, splitting medical multimodal reasoning into fine-grained visual understanding and multi-step reasoning to enable targeted evaluation; 2) Challenging task design, with visual understanding across three key dimensions (small-object detection, fine-detail discrimination, spatial understanding) and reasoning covering four clinically relevant scenarios (temporal prediction, causal reasoning, long-tail generalization, multi-source integration); 3) Broad, high-quality data coverage, comprising 20,653 Visual Question Answering (VQA) pairs spanning 11 organ systems and 12 imaging modalities, validated via a rigorous two-stage (human expert + model-assisted) review to ensure clinical authenticity. We evaluate 18 state-of-the-art MLLMs with Med-CMR, revealing GPT-5 as the top-performing commercial model: 57.81 accuracy on multiple-choice questions (MCQs) and a 48.70 open-ended score, outperforming Gemini 2.5 Pro (49.87 MCQ accuracy, 45.98 open-ended score) and leading open-source model Qwen3-VL-235B-A22B (49.34 MCQ accuracy, 42.62 open-ended score). However, specialized medical MLLMs do not reliably outperform strong general models, and long-tail generalization emerges as the dominant failure mode. Med-CMR thus provides a stress test for visual-reasoning integration and rare-case robustness in medical MLLMs, and a rigorous yardstick for future clinical systems.

Method: HAMLET generates narrative blueprints from simple topics, then uses autonomous actors with independent minds that make decisions based on background, goals, and emotions. Actors can change scene props through actions, with state changes broadcast to update other actors’ knowledge and influence subsequent actions.

Result: Experimental evaluation using a method assessing character performance, narrative quality, and interaction experience shows HAMLET can create expressive and coherent theatrical experiences.

Conclusion: HAMLET addresses limitations of existing LLM-based drama generation by creating a framework where autonomous agents with physical scene interaction capabilities can generate immersive theatrical performances from minimal input.

Abstract: Creating an immersive and interactive theatrical experience is a long-term goal in the field of interactive narrative. The emergence of large language model (LLM) is providing a new path to achieve this goal. However, existing LLM-based drama generation methods often result in agents that lack initiative and cannot interact with the physical scene. Furthermore, these methods typically require detailed user input to drive the drama. These limitations reduce the interactivity and immersion of online real-time performance. To address the above challenges, we propose HAMLET, a multi-agent framework focused on drama creation and online performance. Given a simple topic, the framework generates a narrative blueprint, guiding the subsequent improvisational performance. During the online performance, each actor is given an autonomous mind. This means that actors can make independent decisions based on their own background, goals, and emotional state. In addition to conversations with other actors, their decisions can also change the state of scene props through actions such as opening a letter or picking up a weapon. The change is then broadcast to other related actors, updating what they know and care about, which in turn influences their next action. To evaluate the quality of drama performance generated by HAMLET, we designed an evaluation method to assess three primary aspects, including character performance, narrative quality, and interaction experience. The experimental evaluation shows that HAMLET can create expressive and coherent theatrical experiences.

Method: Integrates semantic communication with Agentic AI: In V2I, RSU uses feature-extraction and semantic-analysis agents to process trajectories, then transmits compact representations and semantic insights. In V2V, vehicles perform local feature extraction/semantic analysis while exchanging predicted trajectories with neighbors.

Result: Extensive experiments show significant outperformance over baseline schemes, achieving up to 47.5% improvement in prediction accuracy under low SNR conditions.

Conclusion: The proposed framework effectively enhances trajectory prediction performance in vehicular environments by combining semantic communication with Agentic AI for efficient information exchange and reliable contextual reasoning.

Abstract: Efficient information exchange and reliable contextual reasoning are essential for vehicle-to-everything (V2X) networks. Conventional communication schemes often incur significant transmission overhead and latency, while existing trajectory prediction models generally lack environmental perception and logical inference capabilities. This paper presents a trajectory prediction framework that integrates semantic communication with Agentic AI to enhance predictive performance in vehicular environments. In vehicle-to-infrastructure (V2I) communication, a feature-extraction agent at the Roadside Unit (RSU) derives compact representations from historical vehicle trajectories, followed by semantic reasoning performed by a semantic-analysis agent. The RSU then transmits both feature representations and semantic insights to the target vehicle via semantic communication, enabling the vehicle to predict future trajectories by combining received semantics with its own historical data. In vehicle-to-vehicle (V2V) communication, each vehicle performs local feature extraction and semantic analysis while receiving predicted trajectories from neighboring vehicles, and jointly utilizes this information for its own trajectory prediction. Extensive experiments across diverse communication conditions demonstrate that the proposed method significantly outperforms baseline schemes, achieving up to a 47.5% improvement in prediction accuracy under low signal-to-noise ratio (SNR) conditions.

Method: Proposes a three-component framework: Runtime Goal Refinement (RGR) for robustness and intent alignment; Observable Cognitive Architecture (OCA) for white-box observability using component decoupling and logic layering; and Evolutionary Memory Architecture (EMA) with execution-evolution dual-loop for evolvability. Implemented as Fairy, a mobile GUI agent.

Result: On RealMobile-Eval benchmark for ambiguous/complex tasks, Fairy outperformed best SoTA baseline by 33.7% in user requirement completion. Controlled experiments, human-subject studies, and ablation studies confirmed RGR improves refinement accuracy and prevents intent deviation, OCA improves maintainability, and EMA is crucial for long-term performance.

Conclusion: The research provides empirically validated specifications and a practical blueprint for building reliable, observable, and evolvable Agentic AI systems, addressing critical software engineering gaps in the field.

Abstract: The Agentic Paradigm faces a significant Software Engineering Absence, yielding Agentic systems commonly lacking robustness, observability, and evolvability. To address these deficiencies, we propose a principled engineering framework comprising Runtime Goal Refinement (RGR), Observable Cognitive Architecture (OCA), and Evolutionary Memory Architecture (EMA). In this framework, RGR ensures robustness and intent alignment via knowledge-constrained refinement and human-in-the-loop clarification; OCA builds an observable and maintainable white-box architecture using component decoupling, logic layering, and state-control separation; and EMA employs an execution-evolution dual-loop for evolvability. We implemented and empirically validated Fairy, a mobile GUI agent based on this framework. On RealMobile-Eval, our novel benchmark for ambiguous and complex tasks, Fairy outperformed the best SoTA baseline in user requirement completion by 33.7%. Subsequent controlled experiments, human-subject studies, and ablation studies further confirmed that the RGR enhances refinement accuracy and prevents intent deviation; the OCA improves maintainability; and the EMA is crucial for long-term performance. This research provides empirically validated specifications and a practical blueprint for building reliable, observable, and evolvable Agentic AI systems.

Method: The paper presents and contrasts three approaches for estimating model error in counterfactual scenarios. These methods are demonstrated and evaluated in a simulation experiment to assess their benefits and limitations.

Result: Through simulation experiments, the authors demonstrate the benefits and limitations of each approach for estimating counterfactual error. They provide specific recommendations for error estimation in counterfactual modeling.

Conclusion: The paper concludes with recommendations for estimating counterfactual error and discusses the necessary components of scenario design required to make scenario projections evaluable, emphasizing the importance of distinguishing between scenario deviation and model miscalibration.

Abstract: Counterfactual scenario modeling exercises that ask “what would happen if?” are one of the most common ways we plan for the future. Despite their ubiquity in planning and decision making, scenario projections are rarely evaluated retrospectively. Differences between projections and observations come from two sources: scenario deviation and model miscalibration. We argue the latter is most important for assessing the value of models in decision making, but requires estimating model error in counterfactual worlds. Here we present and contrast three approaches for estimating this error, and demonstrate the benefits and limitations of each in a simulation experiment. We provide recommendations for the estimation of counterfactual error and discuss the components of scenario design that are required to make scenario projections evaluable.

Method: ARCADIA integrates large-language-model reasoning with statistical diagnostics through iterative refinement of candidate DAGs. It uses constraint-guided prompting and causal-validity feedback loops to construct valid causal structures, creating a fully explainable, intervention-ready pipeline.

Result: Experiments on corporate bankruptcy data show ARCADIA produces more reliable causal graphs than NOTEARS, GOLEM, and DirectLiNGAM. The framework demonstrates stable and interpretable models suitable for real-world high-stakes applications.

Conclusion: ARCADIA advances AI by showing how agentic LLMs can participate in autonomous scientific modeling and structured causal inference, bridging the gap between statistical methods and human-like reasoning for causal discovery.

Abstract: This paper introduces ARCADIA, an agentic AI framework for causal discovery that integrates large-language-model reasoning with statistical diagnostics to construct valid, temporally coherent causal structures. Unlike traditional algorithms, ARCADIA iteratively refines candidate DAGs through constraint-guided prompting and causal-validity feedback, leading to stable and interpretable models for real-world high-stakes domains. Experiments on corporate bankruptcy data show that ARCADIA produces more reliable causal graphs than NOTEARS, GOLEM, and DirectLiNGAM while offering a fully explainable, intervention-ready pipeline. The framework advances AI by demonstrating how agentic LLMs can participate in autonomous scientific modeling and structured causal inference.

Method: DTDE paradigm with each node as autonomous agent; uses diffusion models to capture/predict environment dynamics; inverse dynamics model guides action generation; mean-field mechanism approximates large-scale agent interactions; theoretical bound on distributional approximation error.

Result: Outperforms existing MARL baselines in average reward and QoS metrics; demonstrates scalability and practicality for real-world wireless network optimization.

Conclusion: MA-CDMP effectively addresses MBRL challenges in wireless networks through diffusion-based modeling and mean-field approximation, providing stable convergence and efficient decentralized resource management.

Abstract: In wireless communication systems, efficient and adaptive resource allocation plays a crucial role in enhancing overall Quality of Service (QoS). Compared to the conventional Model-Free Reinforcement Learning (MFRL) scheme, Model-Based RL (MBRL) first learns a generative world model for subsequent planning. The reuse of historical experience in MBRL promises more stable training behavior, yet its deployment in large-scale wireless networks remains challenging due to high-dimensional stochastic dynamics, strong inter-agent cooperation, and communication constraints. To overcome these challenges, we propose the Multi-Agent Conditional Diffusion Model Planner (MA-CDMP) for decentralized communication resource management. Built upon the Distributed Training with Decentralized Execution (DTDE) paradigm, MA-CDMP models each communication node as an autonomous agent and employs Diffusion Models (DMs) to capture and predict environment dynamics. Meanwhile, an inverse dynamics model guides action generation, thereby enhancing sample efficiency and policy scalability. Moreover, to approximate large-scale agent interactions, a Mean-Field (MF) mechanism is introduced as an assistance to the classifier in DMs. This design mitigates inter-agent non-stationarity and enhances cooperation with minimal communication overhead in distributed settings. We further theoretically establish an upper bound on the distributional approximation error introduced by the MF-based diffusion generation, guaranteeing convergence stability and reliable modeling of multi-agent stochastic dynamics. Extensive experiments demonstrate that MA-CDMP consistently outperforms existing MARL baselines in terms of average reward and QoS metrics, showcasing its scalability and practicality for real-world wireless network optimization.

Method: Introduces SFS (Semantic Field Subspace) representation and SAFARI algorithm with Semantic Shift metric to quantify semantic evolution. Develops efficient approximation replacing SVD computations for scalability.

Result: Achieves 15-30x speedup with average errors below 0.01. Outperforms standard classifiers in classification and nuanced tasks like political bias detection across six text/image datasets. Reveals interpretable semantic hierarchies.

Conclusion: Presents a unified framework for structuring, analyzing, and scaling semantic understanding in embedding spaces with geometry-preserving, context-aware representations.

Abstract: Embedding spaces are fundamental to modern AI, translating raw data into high-dimensional vectors that encode rich semantic relationships. Yet, their internal structures remain opaque, with existing approaches often sacrificing semantic coherence for structural regularity or incurring high computational overhead to improve interpretability. To address these challenges, we introduce the Semantic Field Subspace (SFS), a geometry-preserving, context-aware representation that captures local semantic neighborhoods within the embedding space. We also propose SAFARI (SemAntic Field subspAce deteRmInation), an unsupervised, modality-agnostic algorithm that uncovers hierarchical semantic structures using a novel metric called Semantic Shift, which quantifies how semantics evolve as SFSes evolve. To ensure scalability, we develop an efficient approximation of Semantic Shift that replaces costly SVD computations, achieving a 15~30x speedup with average errors below 0.01. Extensive evaluations across six real-world text and image datasets show that SFSes outperform standard classifiers not only in classification but also in nuanced tasks such as political bias detection, while SAFARI consistently reveals interpretable and generalizable semantic hierarchies. This work presents a unified framework for structuring, analyzing, and scaling semantic understanding in embedding spaces.

Method: Systematically characterize agentic AI based on orchestrator components, inference path dynamics, and flow repetitiveness. Profile five representative workloads (Haystack RAG, Toolformer, ChemCrow, Langchain, SWE-Agent) for latency, throughput, and energy metrics. Propose two optimizations: CPU and GPU-Aware Micro-batching (CGAM) and Mixed Agentic Workload Scheduling (MAWS).

Result: Key findings: 1) CPU tool processing consumes up to 90.6% of total latency; 2) Throughput bottlenecks come from CPU factors (coherence, synchronization, core oversubscription) or GPU factors (memory capacity/bandwidth); 3) CPU dynamic energy reaches 44% of total at large batch sizes. Optimizations achieve up to 2.1x P50 latency speedup for homogeneous workloads and 1.41x for heterogeneous workloads.

Conclusion: CPU-centric bottlenecks are critical in agentic AI systems, and the proposed CGAM and MAWS optimizations demonstrate significant potential to improve performance, efficiency, and scalability of both homogeneous and heterogeneous agentic workloads.

Abstract: Agentic AI frameworks add a decision-making orchestrator embedded with external tools, including web search, Python interpreter, contextual database, and others, on top of monolithic LLMs, turning them from passive text oracles into autonomous problem-solvers that can plan, call tools, remember past steps, and adapt on the fly. This paper aims to characterize and understand the system bottlenecks introduced by agentic AI workloads from a largely overlooked CPU-centric perspective. We first systematically characterize Agentic AI on the basis of orchestrator/decision making component, inference path dynamics and repetitiveness of the agentic flow which directly influences the system-level performance. Thereafter, based on the characterization, we choose five representative agentic AI workloads- Haystack RAG, Toolformer, ChemCrow, Langchain and SWE-Agent to profile latency, throughput and energy metrics and demystify the significant impact of CPUs on these metrics relative to GPUs. We observe that - 1. Tool processing on CPUs can take up to 90.6% of the total latency; 2. Agentic throughput gets bottlenecked either by CPU factors - coherence, synchronization and over-subscription of cores or GPU factors - main memory capacity and bandwidth; \circled{3} CPU dynamic energy consumes up to 44% of the total dynamic energy at large batch sizes. Based on the profiling insights, we present two key optimizations- 1. CPU and GPU-Aware Micro-batching (CGAM) and 2. Mixed Agentic Workload Scheduling (MAWS) for homogeneous and heterogeneous agentic workloads respectively to demonstrate the potential to improve the performance, efficiency, and scalability of agentic AI. We achieve up to 2.1x and 1.41x P50 latency speedup compared to the multi-processing benchmark for homogeneous and heterogeneous agentic workloads respectively.

Method: Proposed Hybrid-DMKG: a hybrid reasoning framework using dynamic multimodal knowledge graphs (DMKG). It decomposes questions with LLMs, retrieves updated facts via multimodal encoding, and uses parallel reasoning paths (relation linking + RAG with VLMs) with decision aggregation.

Result: Current MKE methods struggle with consistent updates and reasoning over multimodal chains. Hybrid-DMKG significantly outperforms existing approaches on MMQAKE, achieving higher accuracy and improved robustness to knowledge updates.

Conclusion: MMQAKE addresses limitations in MKE evaluation, and Hybrid-DMKG provides an effective solution for accurate multihop reasoning over updated multimodal knowledge, demonstrating superior performance over existing methods.

Abstract: Multimodal Knowledge Editing (MKE) extends traditional knowledge editing to settings involving both textual and visual modalities. However, existing MKE benchmarks primarily assess final answer correctness while neglecting the quality of intermediate reasoning and robustness to visually rephrased inputs. To address this limitation, we introduce MMQAKE, the first benchmark for multimodal multihop question answering with knowledge editing. MMQAKE evaluates (1) a model’s ability to reason over 2-5-hop factual chains that span both text and images, including performance at each intermediate step, and (2) robustness to visually rephrased inputs in multihop questions. Our evaluation shows that current MKE methods often struggle to consistently update and reason over multimodal reasoning chains after knowledge edits. To overcome these challenges, we propose Hybrid-DMKG, a hybrid reasoning framework built on a dynamic multimodal knowledge graph (DMKG) to enable accurate multihop reasoning over updated multimodal knowledge. Hybrid-DMKG first uses a large language model to decompose multimodal multihop questions into sequential sub-questions, then applies a multimodal retrieval model to locate updated facts by jointly encoding each sub-question with candidate entities and their associated images. For answer inference, a hybrid reasoning module operates over the DMKG via two parallel paths: (1) relation linking prediction, and (2) RAG reasoning with large vision-language models. A decision module aggregates evidence from both paths to select the most credible answer. Experimental results on MMQAKE show that Hybrid-DMKG significantly outperforms existing MKE approaches, achieving higher accuracy and improved robustness to knowledge updates.

Method: STATION simulates a complete scientific ecosystem where agents with long context can read papers, formulate hypotheses, collaborate with peers, submit experiments, and publish results without centralized coordination, allowing free choice of actions and narrative development.

Result: AI agents achieve state-of-the-art performance across mathematics, computational biology, and machine learning benchmarks, surpassing AlphaEvolve in circle packing. Emergent narratives produce novel methods like a density-adaptive algorithm for scRNA-seq batch integration.

Conclusion: STATION represents a new paradigm for autonomous scientific discovery driven by emergent behavior in open-world environments, moving beyond rigid pipelines and enabling organic method development through agent collaboration.

Abstract: We introduce the STATION, an open-world multi-agent environment for autonomous scientific discovery. The Station simulates a complete scientific ecosystem, where agents can engage in long scientific journeys that include reading papers from peers, formulating hypotheses, collaborating with peers, submitting experiments, and publishing results. Importantly, there is no centralized system coordinating their activities. Utilizing their long context, agents are free to choose their own actions and develop their own narratives within the Station. Experiments demonstrate that AI agents in the Station achieve new state-of-the-art performance on a wide range of benchmarks, spanning mathematics, computational biology, and machine learning, notably surpassing AlphaEvolve in circle packing. A rich tapestry of unscripted narratives emerges, such as agents collaborating and analyzing other works rather than pursuing myopic optimization. From these emergent narratives, novel methods arise organically, such as a new density-adaptive algorithm for scRNA-seq batch integration that borrows concepts from another domain. The Station marks a first step towards autonomous scientific discovery driven by emergent behavior in an open-world environment, representing a new paradigm that moves beyond rigid pipelines.

Method: Proposed CAN method to detect Consistently Activated Neurons and locate critical neurons through progressive masking. Experiments conducted on LLaVA-1.5-7b-hf and InstructBLIP-Vicuna-7b.

Result: Masking only a tiny portion of language model’s feed-forward networks (as few as 4 neurons) triggers catastrophic collapse. Critical neurons are predominantly in language model rather than vision components, with down-projection layer being particularly vulnerable. Observed consistent two-stage collapse pattern: initial expressive degradation followed by sudden complete collapse.

Conclusion: The findings reveal critical structural vulnerabilities in LVLMs and provide important insights for safety research, highlighting the disproportionate impact of removing specific neurons in language model components.

Abstract: Large Vision-Language Models (LVLMs) have shown impressive multimodal understanding capabilities, yet their robustness is poorly understood. In this paper, we investigate the structural vulnerabilities of LVLMs to identify any critical neurons whose removal triggers catastrophic collapse. In this context, we propose CAN, a method to detect Consistently Activated Neurons and to locate critical neurons by progressive masking. Experiments on LLaVA-1.5-7b-hf and InstructBLIP-Vicuna-7b reveal that masking only a tiny portion of the language model’s feed-forward networks (just as few as four neurons in extreme cases) suffices to trigger catastrophic collapse. Notably, critical neurons are predominantly localized in the language model rather than in the vision components, and the down-projection layer is a particularly vulnerable structure. We also observe a consistent two-stage collapse pattern: initial expressive degradation followed by sudden, complete collapse. Our findings provide important insights for safety research in LVLMs.

Method: Vision-Interleaved Chain-of-Thought Framework (VICoT) uses stack-based reasoning structure and modular MCP-compatible tool suite to enable LLMs to perform multi-round vision-language reasoning. Includes Reasoning Stack distillation to migrate agent behaviors to smaller models.

Result: VICoT significantly outperforms existing SOTA frameworks on multiple remote sensing benchmarks in reasoning transparency, execution efficiency, and generation quality.

Conclusion: VICoT provides an effective framework for complex remote sensing reasoning tasks with strong generalization and flexibility, while distillation enables deployment on lightweight models.

Abstract: The current remote sensing image analysis task is increasingly evolving from traditional object recognition to complex intelligence reasoning, which places higher requirements on the model’s reasoning ability and the flexibility of tool invocation. To this end, we propose a new multimodal agent framework, Vision-Interleaved Chain-of-Thought Framework (VICoT), which implements explicit multi-round reasoning by dynamically incorporating visual tools into the chain of thought. Through a stack-based reasoning structure and a modular MCP-compatible tool suite, VICoT enables LLMs to efficiently perform multi-round, interleaved vision-language reasoning tasks with strong generalization and flexibility.We also propose the Reasoning Stack distillation method to migrate complex Agent behaviors to small, lightweight models, which ensures the reasoning capability while significantly reducing complexity. Experiments on multiple remote sensing benchmarks demonstrate that VICoT significantly outperforms existing SOTA frameworks in reasoning transparency, execution efficiency, and generation quality.

Method: Uses a pre-trained causal foundation model as a “what-if” model to estimate effects of potential fixes. Measures causal effects of interventions on system-level KPIs (like OEE) to provide data-driven ranking of actions. Evaluated using semi-synthetic manufacturing data compared to baseline ML model.

Result: The model identifies root causes and quantifies their operational impact, providing a systematic method to understand failure causes and recommend optimal interventions. Sets technical basis for robust prescriptive maintenance framework.

Conclusion: The proposed causal ML approach enables engineers to test potential solutions in a causal environment, make more effective operational decisions, reduce costly downtimes, and move from diagnosis to active prescription in maintenance.

Abstract: The transition to prescriptive maintenance in manufacturing is critically constrained by a dependence on predictive models. These models tend to rely on spurious correlations rather than identifying the true causal drivers of failures, often leading to costly misdiagnoses and ineffective interventions. This fundamental limitation results in a key-challenge: while we can predict that a failure may occur, we lack a systematic method to understand why a failure occurs, thereby providing the basis for identifying the most effective intervention. This paper proposes a model based on causal machine learning to bridge this gap. Our objective is to move beyond diagnosis to active prescription by simulating and evaluating potential fixes toward optimizing KPIs such as Overall Equipment Effectiveness (OEE). For this purpose a pre-trained causal foundation model is used as a “what-if” model to estimate the effects of potential fixes. By measuring the causal effect of each intervention on system-level KPIs, it provides a data-driven ranking of actions to recommend at the production line. This process not only identifies root causes but also quantifies their operational impact. The model is evaluated using semi-synthetic manufacturing data and compared with a baseline machine learning model. This paper sets the technical basis for a robust prescriptive maintenance framework, allowing engineers to test potential solutions in a causal environment to make more effective operational decisions and reduce costly downtimes.

Method: AI-powered human-assisted pipeline using category-based retrieval, iterative compiler feedback, and multi-model ensembles to generate candidate formalizations, with expert validation via interactive dashboard and automated quality summaries.

Result: Evaluation shows autoformalization remains challenging with substantial gaps between syntactic validity and semantic correctness, and theorem proving success rates remain low even with iterative refinement.

Conclusion: IndiMathBench presents a challenging testbed for mathematical reasoning and is publicly available for research use.

Abstract: We introduce IndiMathBench, a human-verified benchmark designed to evaluate mathematical theorem proving, curated using an AI-powered human-assisted pipeline for formalizing natural language problems in Lean. IndiMathBench is composed of 312 formal Lean 4 theorems paired with their corresponding informal problem statements, sourced from Indian Mathematics Olympiads. Through category-based retrieval, iterative compiler feedback, and multi-model ensembles, our pipeline generates candidate formalizations that experts efficiently validate via an interactive dashboard with automated quality summaries. Evaluation across multiple frontier models demonstrates that autoformalization remains challenging, with substantial gaps between syntactic validity and semantic correctness, while theorem proving success rates remain low even with iterative refinement, demonstrating that \benchmark~presents a challenging testbed for mathematical reasoning. IndiMathBench is available at https://github.com/prmbiy/IndiMathBench.

Method: Proposes ChartAnchor benchmark with 8k+ chart-table-code triples spanning 30 chart types from diverse sources. Introduces two complementary tasks: chart-to-code generation (synthesizing executable code to replicate charts) and controlled chart-to-table reconstruction (extracting exact data with predefined headers). Uses multi-level evaluation framework integrating semantic validation, stylistic analysis, and perceptual metrics.

Result: Extensive experiments on MLLMs reveal critical limitations in numerical precision and code synthesis, showing that current models struggle with structured reasoning beyond surface-level perception.

Conclusion: ChartAnchor establishes a rigorous foundation for chart grounding by unifying symbolic and data-driven approaches, offering meaningful insights for advancing MLLMs in scientific, financial, and industrial domains where precise chart comprehension is essential.

Abstract: Recent advances in multimodal large language models (MLLMs) highlight the need for benchmarks that rigorously evaluate structured chart comprehension.Chart grounding refers to the bidirectional alignment between a chart’s visual appearance and the structured semantics. This task requires models to produce a symbolic specification that faithfully captures the chart’s visual and structural intent, while also recovering the underlying tabular data with precise values and relationships. Chart grounding directly reflects a model’s capabilities in numerical reasoning, multimodal alignment, and structural reconstruction, and has several important applications in real-world scenarios.Existing benchmarks, constrained by narrow chart diversity, isolated tasks, and incomplete evaluation frameworks, fail to holistically assess grounding. To address this, we propose ChartAnchor, a comprehensive benchmark of 8k+ chart-table-code triples spanning 30 chart types drawn from diverse real-world and augmented sources. ChartAnchor introduces two complementary tasks: chart-to-code generation (synthesizing executable code to replicate charts) and controlled chart-to-table reconstruction (extracting exact data with predefined headers), enabling cross-validation of visual and numerical fidelity. A multi-level evaluation framework integrates semantic validation, stylistic analysis, and perceptual metrics to assess both structural and content-level correctness. Extensive experiments on MLLMs reveal critical limitations in numerical precision and code synthesis, emphasizing the need for structured reasoning beyond surface-level perception. By unifying symbolic and data-driven grounding, ChartAnchor establishes a rigorous foundation for chart grounding, offering meaningful insights for advancing MLLMs in scientific, financial, and industrial domains.

Method: Created LEGIT dataset (24K instances) by converting court judgments into hierarchical trees of opposing arguments and court conclusions. These trees serve as rubrics to evaluate reasoning trace coverage and correctness. Verified reliability via human expert annotations and comparison with simpler rubrics.

Result: (1) LLMs’ legal reasoning ability is significantly affected by both legal issue coverage and correctness. (2) RAG and RL with rubrics provide complementary benefits: RAG improves overall reasoning capability, while RL improves correctness but reduces coverage.

Conclusion: LEGIT provides a reliable framework for evaluating LLM reasoning traces in legal domain. The dataset reveals important insights about LLM legal reasoning limitations and shows how different techniques (RAG vs RL) address different aspects of reasoning quality.

Abstract: Evaluating the quality of LLM-generated reasoning traces in expert domains (e.g., law) is essential for ensuring credibility and explainability, yet remains challenging due to the inherent complexity of such reasoning tasks. We introduce LEGIT (LEGal Issue Trees), a novel large-scale (24K instances) expert-level legal reasoning dataset with an emphasis on reasoning trace evaluation. We convert court judgments into hierarchical trees of opposing parties’ arguments and the court’s conclusions, which serve as rubrics for evaluating the issue coverage and correctness of the reasoning traces. We verify the reliability of these rubrics via human expert annotations and comparison with coarse, less informative rubrics. Using the LEGIT dataset, we show that (1) LLMs’ legal reasoning ability is seriously affected by both legal issue coverage and correctness, and that (2) retrieval-augmented generation (RAG) and RL with rubrics bring complementary benefits for legal reasoning abilities, where RAG improves overall reasoning capability, whereas RL improves correctness albeit with reduced coverage.

Method: Med-CRAFT extracts structured visual primitives from raw videos and builds a dynamic Spatiotemporal Knowledge Graph. It then generates benchmark queries by traversing valid paths in this graph, ensuring Chain-of-Thought provenance for each synthesized item.

Result: The pipeline produced M3-Med-Auto, a large-scale medical video reasoning benchmark with fine-grained temporal selectivity and multi-hop logical complexity. Evaluations show it generates query workloads comparable to expert-curated datasets, with high correlation between graph topology and MLLM reasoning steps.

Conclusion: Med-CRAFT enables scalable, low-cost construction of robust evaluation protocols for medical MLLMs by encoding verifiable logic into visual-linguistic benchmarks through deterministic graph traversal.

Abstract: The scarcity of high-quality, logically annotated video datasets remains a primary bottleneck in advancing Multi-Modal Large Language Models (MLLMs) for the medical domain. Traditional manual annotation is prohibitively expensive and non-scalable, while existing synthetic methods often suffer from stochastic hallucinations and a lack of logical interpretability. To address these challenges, we introduce \textbf{\PipelineName}, a novel neuro-symbolic data engineering framework that formalizes benchmark synthesis as a deterministic graph traversal process. Unlike black-box generative approaches, Med-CRAFT extracts structured visual primitives (e.g., surgical instruments, anatomical boundaries) from raw video streams and instantiates them into a dynamic Spatiotemporal Knowledge Graph. By anchoring query generation to valid paths within this graph, we enforce a rigorous Chain-of-Thought (CoT) provenance for every synthesized benchmark item. We instantiate this pipeline to produce M3-Med-Auto, a large-scale medical video reasoning benchmark exhibiting fine-grained temporal selectivity and multi-hop logical complexity. Comprehensive evaluations demonstrate that our automated pipeline generates query workloads with complexity comparable to expert-curated datasets. Furthermore, a logic alignment analysis reveals a high correlation between the prescribed graph topology and the reasoning steps of state-of-the-art MLLMs, validating the system’s capability to encode verifiable logic into visual-linguistic benchmarks. This work paves the way for scalable, low-cost construction of robust evaluation protocols in critical domains.

Method: SCUs leverage prior knowledge of system dynamics to create hierarchical shields that explicitly manage subsets of constraints, ensuring interpretable safety guarantees for RL deployment.

Result: RL agent with SCUs achieved 24% reduction in fuel consumption without increasing battery degradation while satisfying all operational constraints in remote microgrid optimization.

Conclusion: SCUs provide a systematic, interpretable approach for safe RL deployment in energy transition applications, enabling constraint satisfaction while maintaining optimization performance.

Abstract: Reinforcement learning (RL) is a powerful framework for optimizing decision-making in complex systems under uncertainty, an essential challenge in real-world settings, particularly in the context of the energy transition. A representative example is remote microgrids that supply power to communities disconnected from the main grid. Enabling the energy transition in such systems requires coordinated control of renewable sources like wind turbines, alongside fuel generators and batteries, to meet demand while minimizing fuel consumption and battery degradation under exogenous and intermittent load and wind conditions. These systems must often conform to extensive regulations and complex operational constraints. To ensure that RL agents respect these constraints, it is crucial to provide interpretable guarantees. In this paper, we introduce Shielded Controller Units (SCUs), a systematic and interpretable approach that leverages prior knowledge of system dynamics to ensure constraint satisfaction. Our shield synthesis methodology, designed for real-world deployment, decomposes the environment into a hierarchical structure where each SCU explicitly manages a subset of constraints. We demonstrate the effectiveness of SCUs on a remote microgrid optimization task with strict operational requirements. The RL agent, equipped with SCUs, achieves a 24% reduction in fuel consumption without increasing battery degradation, outperforming other baselines while satisfying all constraints. We hope SCUs contribute to the safe application of RL to the many decision-making challenges linked to the energy transition.

Method: AutoSpec searches for logical specification refinements using four procedures that modify the abstract graph of SpectRL specifications by refining existing edges or introducing new ones, maintaining soundness.

Result: AutoSpec improves the complexity of control tasks that can be solved when integrated with existing RL algorithms, with experiments showing promising improvements.

Conclusion: Automatic specification refinement through AutoSpec enhances RL performance by providing better guidance from logical specifications while maintaining soundness guarantees.

Abstract: Logical specifications have been shown to help reinforcement learning algorithms in achieving complex tasks. However, when a task is under-specified, agents might fail to learn useful policies. In this work, we explore the possibility of improving coarse-grained logical specifications via an exploration-guided strategy. We propose \textsc{AutoSpec}, a framework that searches for a logical specification refinement whose satisfaction implies satisfaction of the original specification, but which provides additional guidance therefore making it easier for reinforcement learning algorithms to learn useful policies. \textsc{AutoSpec} is applicable to reinforcement learning tasks specified via the SpectRL specification logic. We exploit the compositional nature of specifications written in SpectRL, and design four refinement procedures that modify the abstract graph of the specification by either refining its existing edge specifications or by introducing new edge specifications. We prove that all four procedures maintain specification soundness, i.e. any trajectory satisfying the refined specification also satisfies the original. We then show how \textsc{AutoSpec} can be integrated with existing reinforcement learning algorithms for learning policies from logical specifications. Our experiments demonstrate that \textsc{AutoSpec} yields promising improvements in terms of the complexity of control tasks that can be solved, when refined logical specifications produced by \textsc{AutoSpec} are utilized.

Method: Built on Unreal Engine 5, SimWorld provides: 1) realistic open-ended world simulation with accurate physics/social dynamics and language-driven procedural generation, 2) rich multimodal interfaces for LLM/VLM agents with open-vocabulary actions, and 3) extensible physical/social reasoning scenarios.

Result: The authors deployed frontier LLM agents (GPT-4o, Gemini-2.5-Flash, Claude-3.5, DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks with strategic cooperation/competition, revealing distinct reasoning patterns and limitations across models.

Conclusion: SimWorld is open-sourced as a foundational platform for advancing real-world agent intelligence across disciplines, addressing critical gaps in agent development infrastructure for complex physical and social environments.

Abstract: While LLM/VLM-powered AI agents have advanced rapidly in math, coding, and computer use, their applications in complex physical and social environments remain challenging. Building agents that can survive and thrive in the real world (for example, by autonomously earning income or running a business) requires massive-scale interaction, reasoning, training, and evaluation across diverse embodied scenarios. However, existing world simulators for such development fall short: they often rely on limited hand-crafted environments, simulate simplified game-like physics and social rules, and lack native support for LLM/VLM agents. We introduce SimWorld, a new simulator built on Unreal Engine 5, designed for developing and evaluating LLM/VLM agents in rich, real-world-like settings. SimWorld offers three core capabilities: (1) realistic, open-ended world simulation, including accurate physical and social dynamics and language-driven procedural environment generation; (2) a rich interface for LLM/VLM agents, with multimodal world inputs and open-vocabulary actions at varying levels of abstraction; and (3) diverse and extensible physical and social reasoning scenarios that are easily customizable by users. We demonstrate SimWorld by deploying frontier LLM agents (e.g., GPT-4o, Gemini-2.5-Flash, Claude-3.5, and DeepSeek-Prover-V2) on long-horizon multi-agent delivery tasks involving strategic cooperation and competition. The results reveal distinct reasoning patterns and limitations across models. We open-source SimWorld and hope it becomes a foundational platform for advancing real-world agent intelligence across disciplines: https://simworld.org.

Method: CodeDistiller automatically distills large collections of scientific GitHub repositories into a vetted library of working domain-specific code examples. The system uses automatic and domain-expert evaluation on 250 materials science repositories to identify functional examples.

Result: The best model produces functional examples for 74% of repositories. Downstream evaluation shows ASD agents augmented with CodeDistiller-generated libraries produce more accurate, complete, and scientifically sound experiments compared to agents with only general materials-science examples.

Conclusion: CodeDistiller successfully enables ASD systems to automatically expand their capabilities by distilling working code examples from scientific repositories, overcoming limitations of current approaches and improving experiment quality.

Abstract: Automated Scientific Discovery (ASD) systems can help automatically generate and run code-based experiments, but their capabilities are limited by the code they can reliably generate from parametric knowledge alone. As a result, current systems either mutate a small number of manually-crafted experiment examples, or operate solely from parametric knowledge, limiting quality and reach. We introduce CodeDistiller, a system that automatically distills large collections of scientific Github repositories into a vetted library of working domain-specific code examples, allowing ASD agents to expand their capabilities without manual effort. Using a combination of automatic and domain-expert evaluation on 250 materials science repositories, we find the best model is capable of producing functional examples for 74% of repositories, while our downstream evaluation shows an ASD agent augmented with a CodeDistiller generated library produces more accurate, complete, and scientifically sound experiments than an agent with only general materials-science code examples.

Method: Proposed GUIDE framework that incorporates quantitative model performance characteristics and accounts for performance-energy trade-offs in decision-making.

Result: GUIDE increases accuracy by 0.90%-11.92% across tasks, achieves up to 54% energy efficiency improvement, and reduces orchestrator model selection latency from 4.51 s to 7.2 ms.

Conclusion: Quantitative model performance characteristics should be incorporated into AI orchestration frameworks to improve accuracy, energy efficiency, and reduce latency in model selection.

Abstract: As modern artificial intelligence (AI) systems become more advanced and capable, they can leverage a wide range of tools and models to perform complex tasks. Today, the task of orchestrating these models is often performed by Large Language Models (LLMs) that rely on qualitative descriptions of models for decision-making. However, the descriptions provided to these LLM-based orchestrators do not reflect true model capabilities and performance characteristics, leading to suboptimal model selection, reduced accuracy, and increased energy costs. In this paper, we conduct an empirical analysis of LLM-based orchestration limitations and propose GUIDE, a new energy-aware model selection framework that accounts for performance-energy trade-offs by incorporating quantitative model performance characteristics in decision-making. Experimental results demonstrate that GUIDE increases accuracy by 0.90%-11.92% across various evaluated tasks, and achieves up to 54% energy efficiency improvement, while reducing orchestrator model selection latency from 4.51 s to 7.2 ms.

Method: Introduce “foundation prior” concept showing synthetic data are draws from prior predictive distribution. Model subjectivity through dependence on user’s anticipated data distribution, prompt engineering, and trust in model. Derive foundation prior as exponential-tilted generalized Bayesian update of user’s primitive prior with trust parameter.

Result: Framework enables incorporation of synthetic data into statistical/econometric workflows as structured, explicitly subjective priors rather than empirical observations. Applications include refining complex models, informing latent constructs, guiding experimental design, and augmenting random-coefficient/partially linear specifications.

Conclusion: Treating generative outputs as foundation priors offers principled way to harness foundation models in empirical work while avoiding conflation of synthetic data with real observations. Framework acknowledges subjective nature of synthetic data generation process.

Abstract: Foundation models, and in particular large language models, can generate highly informative responses, prompting growing interest in using these ‘‘synthetic’’ outputs as data in empirical research and decision-making. This paper introduces the idea of a foundation prior, which shows that model-generated outputs are not as real observations, but draws from the foundation prior induced prior predictive distribution. As such synthetic data reflects both the model’s learned patterns and the user’s subjective priors, expectations, and biases. We model the subjectivity of the generative process by making explicit the dependence of synthetic outputs on the user’s anticipated data distribution, the prompt-engineering process, and the trust placed in the foundation model. We derive the foundation prior as an exponential-tilted, generalized Bayesian update of the user’s primitive prior, where a trust parameter governs the weight assigned to synthetic data. We then show how synthetic data and the associated foundation prior can be incorporated into standard statistical and econometric workflows, and discuss their use in applications such as refining complex models, informing latent constructs, guiding experimental design, and augmenting random-coefficient and partially linear specifications. By treating generative outputs as structured, explicitly subjective priors rather than as empirical observations, the framework offers a principled way to harness foundation models in empirical work while avoiding the conflation of synthetic ‘‘facts’’ with real data.

Method: Evaluated 8 predictive models from baseline statistical approaches to formal causal inference methods on 10,000 CNC machines dataset with 3.3% failure prevalence. Compared causal inference model (L5) against correlation-based decision tree model (L3).

Result: Causal model achieved $1.16M annual savings (70.2% reduction), outperforming best correlation model by $80K/year. Matched highest recall (87.9%) while reducing false alarms 97% (165 to 5), precision 92.1%, with only 2.6% train-test gap.

Conclusion: Causal AI methods with domain knowledge yield superior financial outcomes and more interpretable predictions than correlation-based approaches in predictive maintenance, addressing operational cost asymmetry effectively.

Abstract: Predictive maintenance in manufacturing environments presents a challenging optimization problem characterized by extreme cost asymmetry, where missed failures incur costs roughly fifty times higher than false alarms. Conventional machine learning approaches typically optimize statistical accuracy metrics that do not reflect this operational reality and cannot reliably distinguish causal relationships from spurious correlations. This study evaluates eight predictive models, ranging from baseline statistical approaches to formal causal inference methods, on a dataset of 10,000 CNC machines with a 3.3% failure prevalence. The formal causal inference model (L5) achieved estimated annual cost savings of 1.16 million USD (a 70.2 percent reduction), outperforming the best correlation-based decision tree model (L3) by approximately 80,000 USD per year. The causal model matched the highest observed recall (87.9 percent) while reducing false alarms by 97 percent (from 165 to 5) and attained a precision of 92.1 percent, with a train-test performance gap of only 2.6 percentage points. These results indicate that causal AI methods, when combined with domain knowledge, can yield superior financial outcomes and more interpretable predictions compared to correlation-based approaches in predictive maintenance applications.

Method: Uses Dual Brain Decoding Alignment with fMRI-to-text and text-to-gesture models to establish dual fMRI-to-gesture reconstruction patterns, then aligns outputs and trains in self-supervised manner on unpaired data.

Result: The method successfully reconstructs expressive gestures directly from fMRI recordings and investigates how fMRI signals from different cortical ROIs affect generation results.

Conclusion: Provides new insights into decoding co-speech gestures, advancing understanding of neuroscience and cognitive science by overcoming data pairing limitations.

Abstract: Understanding how the brain responds to external stimuli and decoding this process has been a significant challenge in neuroscience. While previous studies typically concentrated on brain-to-image and brain-to-language reconstruction, our work strives to reconstruct gestures associated with speech stimuli perceived by brain. Unfortunately, the lack of paired {brain, speech, gesture} data hinders the deployment of deep learning models for this purpose. In this paper, we introduce a novel approach, \textbf{fMRI2GES}, that allows training of fMRI-to-gesture reconstruction networks on unpaired data using \textbf{Dual Brain Decoding Alignment}. This method relies on two key components: (i) observed texts that elicit brain responses, and (ii) textual descriptions associated with the gestures. Then, instead of training models in a completely supervised manner to find a mapping relationship among the three modalities, we harness an fMRI-to-text model, a text-to-gesture model with paired data and an fMRI-to-gesture model with unpaired data, establishing dual fMRI-to-gesture reconstruction patterns. Afterward, we explicitly align two outputs and train our model in a self-supervision way. We show that our proposed method can reconstruct expressive gestures directly from fMRI recordings. We also investigate fMRI signals from different ROIs in the cortex and how they affect generation results. Overall, we provide new insights into decoding co-speech gestures, thereby advancing our understanding of neuroscience and cognitive science.

Method: Map ICD-9 codes to PrimeKG knowledge graph, extract disease-relevant nodes and multi-hop reasoning paths as scaffolds for chain-of-thought generation, retain only explanations matching observed outcomes, then fine-tune lightweight LLMs (LLaMA-3.1-Instruct-8B and Gemma-7B) on this supervision corpus.

Result: KG-guided models outperform classical baselines across ten diseases with AUROC 0.66-0.70 and macro-AUPR 0.40-0.47 on limited training data (400-1000 cases). Zero-shot transfer to CRADLE cohort improves accuracy from ~0.40-0.51 to 0.72-0.77. Clinician evaluation shows preference for KG-guided explanations in clarity, relevance, and clinical correctness.

Conclusion: The KG-guided chain-of-thought framework provides clinically grounded, temporally consistent explanations for EHR disease prediction, improving both predictive performance and explanation quality while enabling effective transfer learning across cohorts.

Abstract: Electronic health records (EHRs) support powerful clinical prediction models, but existing methods typically provide coarse, post hoc explanations that offer limited value for patient-level decision making. We introduce a knowledge graph (KG)-guided chain-of-thought (CoT) framework that generates clinically grounded and temporally consistent reasoning for visit-level disease prediction in MIMIC-III. ICD-9 codes are mapped to PrimeKG, from which disease-relevant nodes and multi-hop reasoning paths are extracted and used as scaffolds for CoT generation; only explanations whose conclusions match observed outcomes are retained. Lightweight LLaMA-3.1-Instruct-8B and Gemma-7B models are then fine-tuned on this supervision corpus. Across ten PrimeKG-mapped diseases and limited training cohorts (400 and 1000 cases), KG-guided models outperform strong classical baselines, achieving AUROC values of 0.66 to 0.70 and macro-AUPR values of 0.40 to 0.47. The models also transfer zero-shot to the CRADLE cohort, improving accuracy from approximately 0.40 to 0.51 up to 0.72 to 0.77. A blinded clinician evaluation shows consistent preference for KG-guided CoT explanations in clarity, relevance, and clinical correctness.

Method: Fine-tuned DeepSeek-R1-Distill-Llama-70B to perform chain-of-thought reasoning in ROT-13 encryption while maintaining intelligible English outputs, then evaluated logit lens analysis on decoding hidden reasoning from internal activations.

Result: Logit lens effectively translates encoded reasoning with accuracy peaking in intermediate-to-late layers. Developed unsupervised decoding pipeline combining logit lens with automated paraphrasing achieves substantial accuracy in reconstructing complete reasoning transcripts.

Conclusion: Current mechanistic interpretability techniques may be more robust to simple forms of encoded reasoning than previously understood, providing a framework for evaluating interpretability against non-human-readable reasoning formats to maintain AI oversight.

Abstract: As large language models become increasingly capable, there is growing concern that they may develop reasoning processes that are encoded or hidden from human oversight. To investigate whether current interpretability techniques can penetrate such encoded reasoning, we construct a controlled testbed by fine-tuning a reasoning model (DeepSeek-R1-Distill-Llama-70B) to perform chain-of-thought reasoning in ROT-13 encryption while maintaining intelligible English outputs. We evaluate mechanistic interpretability methods–in particular, logit lens analysis–on their ability to decode the model’s hidden reasoning process using only internal activations. We show that logit lens can effectively translate encoded reasoning, with accuracy peaking in intermediate-to-late layers. Finally, we develop a fully unsupervised decoding pipeline that combines logit lens with automated paraphrasing, achieving substantial accuracy in reconstructing complete reasoning transcripts from internal model representations. These findings suggest that current mechanistic interpretability techniques may be more robust to simple forms of encoded reasoning than previously understood. Our work provides an initial framework for evaluating interpretability methods against models that reason in non-human-readable formats, contributing to the broader challenge of maintaining oversight over increasingly capable AI systems.

Method: Three-stage pipeline: (1) structure-aware segmentation using table-of-contents boundaries, (2) ontology-constrained LLM extraction that embeds the ESGMKG schema into prompts while enriching entities with semantic fields, (3) two-phase validation combining LLM-based semantic verification with rule-based schema checking across entity, property, and relationship levels (VR001-VR006). The framework preserves both segment-level and page-level provenance for audit traceability.

Result: Evaluated on five ESG standards (SASB Commercial Banks, SASB Semiconductors, TCFD, IFRS S2, AASB S2) totaling 228 pages and 60 segments, OntoMetric achieves 65-90% semantic accuracy and 80-90% schema compliance, compared to 3-10% for baseline unconstrained extraction, at approximately 0.01 to 0.02 USD per validated entity.

Conclusion: Combining symbolic ontology constraints with neural extraction enables reliable, auditable knowledge graphs suitable for regulatory compliance and web integration, supporting downstream applications such as sustainable-finance analytics, transparency portals, and automated compliance tools.

Abstract: Environmental, Social, and Governance (ESG) disclosure frameworks such as SASB, TCFD, and IFRS S2 require organizations to compute and report numerous metrics for compliance, yet these requirements are embedded in long, unstructured PDF documents that are difficult to interpret, standardize, and audit. Manual extraction is unscalable, while unconstrained large language model (LLM) extraction often produces inconsistent entities, hallucinated relationships, missing provenance, and high validation failure rates. We present OntoMetric, an ontology-guided framework that transforms ESG regulatory documents into validated, AI- and web-ready knowledge graphs. OntoMetric operates through a three-stage pipeline: (1) structure-aware segmentation using table-of-contents boundaries, (2) ontology-constrained LLM extraction that embeds the ESGMKG schema into prompts while enriching entities with semantic fields for downstream reasoning, and (3) two-phase validation that combines LLM-based semantic verification with rule-based schema checking across entity, property, and relationship levels (VR001-VR006). The framework preserves both segment-level and page-level provenance for audit traceability. Evaluated on five ESG standards (SASB Commercial Banks, SASB Semiconductors, TCFD, IFRS S2, AASB S2) totaling 228 pages and 60 segments, OntoMetric achieves 65-90% semantic accuracy and 80-90% schema compliance, compared to 3-10% for baseline unconstrained extraction, at approximately 0.01 to 0.02 USD per validated entity. Our results demonstrate that combining symbolic ontology constraints with neural extraction enables reliable, auditable knowledge graphs suitable for regulatory compliance and web integration, supporting downstream applications such as sustainable-finance analytics, transparency portals, and automated compliance tools.

Method: 1) Introduces RoboDriveBench benchmark with 11 simulated corruption scenarios (6 sensor, 5 prompt) across 250 unique driving scenarios each. 2) Proposes RoboDriveVLM framework mapping multimodal data (lidar, radar) into unified latent space. 3) Implements Test-Time Adaptation method using cross-modal knowledge distillation.

Result: Benchmark reveals limitations of current VLM-based systems. RoboDriveVLM with TTA demonstrates improved robustness against real-world challenges. Extensive experiments validate the approach across 64,559 trajectory prediction cases.

Conclusion: The work highlights critical robustness issues in current VLM-based autonomous driving and provides a more reliable solution through systematic benchmarking, multimodal data integration, and adaptive techniques for real-world deployment.

Abstract: Current Vision-Language Model (VLM)-based end-to-end autonomous driving systems often leverage large language models to generate driving decisions directly based on their understanding of the current scene. However, such systems introduce multiple risks in real-world driving scenarios. To evaluate whether VLMs are truly viable for autonomous driving, we introduce RoboDriveBench, the first robustness benchmark focused on end-to-end trajectory prediction tasks. This benchmark systematically evaluates two critical categories of real-world challenges for VLM-based end-to-end autonomous driving systems through 11 simulated scenarios encompassing various corruption types, including 6 scenarios of sensor corruption caused by environmental variations, along with 5 cases of prompt corruption resulting from human intervention and data transmission failures. Each corruption type includes 250 unique driving scenarios and 5,689 frames, resulting in 64,559 total trajectory prediction cases per evaluation. To overcome these real-world challenges, we propose a novel VLM-based autonomous driving framework called RoboDriveVLM, which enhances robustness by mapping more multimodal data-e.g., lidar and radar-into a unified latent space. Furthermore, we introduce a new Test-Time Adaptation (TTA) method based on cross-modal knowledge distillation to improve the robustness of VLM-based autonomous driving systems. Through extensive experiments, our work highlights the limitations of current VLM-based end-to-end autonomous driving systems and provides a more reliable solution for real-world deployment. Source code and datasets will be released.

Method: CuES (Curiosity driven and Environment grounded Synthesis) autonomously generates tasks directly from environment structure and affordances without handcrafted seeds or external corpora. It uses intrinsic curiosity for exploration, abstracts interaction patterns into reusable task schemas, and refines them through lightweight top-down guidance and memory-based quality control.

Result: Across three environments (AppWorld, BFCL, and WebShop), CuES produces task distributions that match or surpass manually curated datasets in both diversity and executability, yielding substantial downstream policy improvements.

Conclusion: Curiosity-driven, environment-grounded task generation provides a scalable foundation for agents that not only learn how to act, but also learn what to learn, addressing the task scarcity bottleneck in agentic RL.

Abstract: Large language model based agents are increasingly deployed in complex, tool augmented environments. While reinforcement learning provides a principled mechanism for such agents to improve through interaction, its effectiveness critically depends on the availability of structured training tasks. In many realistic settings, however, no such tasks exist a challenge we term task scarcity, which has become a key bottleneck for scaling agentic RL. Existing approaches typically assume predefined task collections, an assumption that fails in novel environments where tool semantics and affordances are initially unknown. To address this limitation, we formalize the problem of Task Generation for Agentic RL, where an agent must learn within a given environment that lacks predefined tasks. We propose CuES, a Curiosity driven and Environment grounded Synthesis framework that autonomously generates diverse, executable, and meaningful tasks directly from the environment structure and affordances, without relying on handcrafted seeds or external corpora. CuES drives exploration through intrinsic curiosity, abstracts interaction patterns into reusable task schemas, and refines them through lightweight top down guidance and memory based quality control. Across three representative environments, AppWorld, BFCL, and WebShop, CuES produces task distributions that match or surpass manually curated datasets in both diversity and executability, yielding substantial downstream policy improvements. These results demonstrate that curiosity driven, environment grounded task generation provides a scalable foundation for agents that not only learn how to act, but also learn what to learn. The code is available at https://github.com/modelscope/AgentEvolver/research/CuES.

Method: Extended NGU to multi-agent settings in simple_tag environment from PettingZoo suite. Investigated three design choices: (1) shared vs individual replay buffers, (2) sharing episodic novelty among agents with different k thresholds, (3) heterogeneous vs common beta parameter values.

Result: NGU achieved moderately higher returns and more stable learning dynamics compared to multi-agent DQN baseline. Shared replay buffer yielded best performance and stability. Novelty sharing performed comparably when k=1 but degraded learning for larger values. Heterogeneous beta values did not improve over small common value.

Conclusion: NGU can be effectively applied in multi-agent settings when experiences are shared (via shared replay buffer) and intrinsic exploration signals are carefully tuned, with shared experiences being key to the performance gains.

Abstract: The Never Give Up (NGU) algorithm has proven effective in reinforcement learning tasks with sparse rewards by combining episodic novelty and intrinsic motivation. In this work, we extend NGU to multi-agent environments and evaluate its performance in the simple_tag environment from the PettingZoo suite. Compared to a multi-agent DQN baseline, NGU achieves moderately higher returns and more stable learning dynamics. We investigate three design choices: (1) shared replay buffer versus individual replay buffers, (2) sharing episodic novelty among agents using different k thresholds, and (3) using heterogeneous values of the beta parameter. Our results show that NGU with a shared replay buffer yields the best performance and stability, highlighting that the gains come from combining NGU intrinsic exploration with experience sharing. Novelty sharing performs comparably when k = 1 but degrades learning for larger values. Finally, heterogeneous beta values do not improve over a small common value. These findings suggest that NGU can be effectively applied in multi-agent settings when experiences are shared and intrinsic exploration signals are carefully tuned.

Method: Proposes a label-setting approach using multi-objective A* search with novel profile dominance rules to avoid complex profile handling, developing four variants of this method.

Result: Experimental evaluation on real-world road networks with realistic energy consumption data shows performance comparable to energy-optimal A* with known initial energy.

Conclusion: The proposed energy profile A* search provides an effective solution for EV path planning under energy uncertainty without requiring complex profile management.

Abstract: We study the energy-optimal shortest path problem for electric vehicles (EVs) in large-scale road networks, where recuperated energy along downhill segments introduces negative energy costs. While traditional point-to-point pathfinding algorithms for EVs assume a known initial energy level, many real-world scenarios involving uncertainty in available energy require planning optimal paths for all possible initial energy levels, a task known as energy-optimal profile search. Existing solutions typically rely on specialized profile-merging procedures within a label-correcting framework that results in searching over complex profiles. In this paper, we propose a simple yet effective label-setting approach based on multi-objective A* search, which employs a novel profile dominance rule to avoid generating and handling complex profiles. We develop four variants of our method and evaluate them on real-world road networks enriched with realistic energy consumption data. Experimental results demonstrate that our energy profile A* search achieves performance comparable to energy-optimal A* with a known initial energy level.

Method: (1) Formalize Overton pluralism as a set coverage metric (OvertonScore), (2) conduct large-scale U.S.-representative human study (N=1209, 60 questions, 8 LLMs), (3) develop automated benchmark that reproduces human judgments.

Result: Models achieve OvertonScores of 0.35-0.41 on average (DeepSeek V3 best), far below theoretical maximum of 1.0. Automated benchmark achieves high rank correlation with human judgments (ρ=0.88), providing practical proxy for evaluation.

Conclusion: The work establishes foundation for systematic progress toward more pluralistic LLMs by turning pluralistic alignment from normative aim into measurable benchmark, with automated tools enabling scalable evaluation.

Abstract: We introduce a novel framework for measuring Overton pluralism in LLMs–the extent to which diverse viewpoints are represented in model outputs. We (i) formalize Overton pluralism as a set coverage metric (OvertonScore), (ii) conduct a large-scale U.S.-representative human study (N = 1209; 60 questions; 8 LLMs), and (iii) develop an automated benchmark that closely reproduces human judgments. On average, models achieve OvertonScores of 0.35–0.41, with DeepSeek V3 performing best; yet all models remain far below the theoretical maximum of 1.0, revealing substantial headroom for improvement. Because repeated large-scale human studies are costly and slow, scalable evaluation tools are essential for model development. Hence, we propose an automated benchmark that achieves high rank correlation with human judgments ($ρ=0.88$), providing a practical proxy without replacing human assessment. By turning pluralistic alignment from a normative aim into a measurable benchmark, our work establishes a foundation for systematic progress toward more pluralistic LLMs.

Method: Prompt-driven Cognitive Computing Framework (PMCSF) with Cognitive State Decoder (CSD) that reverse-engineers text into cognitive vectors, and Cognitive Text Encoder (CTE) that re-materializes states into text with human imperfections via Cognitive Perturbation Operators.

Result: CTE text shows Jensen-Shannon divergence of 0.0614 from human text (vs. 0.4431 for standard LLM), passes double-blind professional review, ICC > 0.9 for cognitive alignment. In A-share market tests, reduces maximum drawdown by 47.4% during 2015 crash and delivers 8.6% Defensive Alpha.

Conclusion: Modelling human cognitive limitations rather than copying surface data enables synthetic data with genuine functional gain, offering a technical pathway to resolve AI data-collapse crisis.

Abstract: Although synthetic data is widely promoted as a remedy, its prevailing production paradigm – one optimizing for statistical smoothness – systematically removes the long-tail, cognitively grounded irregularities that characterize human text. Prolonged training on such statistically optimal but cognitively impoverished data accelerates model collapse. This paper proposes a paradigm shift: instead of imitating the surface properties of data, we simulate the cognitive processes that generate human text. We introduce the Prompt-driven Cognitive Computing Framework (PMCSF), whose core consists of a Cognitive State Decoder (CSD) that reverse-engineers unstructured text into structured cognitive vectors, and a Cognitive Text Encoder (CTE) that re-materializes these states into text enriched with human-typical imperfections via mathematically defined Cognitive Perturbation Operators. The framework is validated through a two-stage objective evaluation pipeline. First, in cognitive codec verification, CTE text yields a Jensen-Shannon divergence of 0.0614 from human text (vs. 0.4431 for standard LLM output), passes double-blind professional media review, and achieves an intraclass correlation coefficient ICC > 0.9 for cognitive profile alignment across heterogeneous models. Second, in functional gain evaluation, isomorphic stress tests in the A-share market show that strategies incorporating CTE-generated data reduce maximum drawdown by 47.4% during the 2015 crash and deliver 8.6% Defensive Alpha, exceeding transaction costs by a factor of 33. Our findings demonstrate that modelling human cognitive limitations – not copying surface data – enables synthetic data with genuine functional gain, offering a viable technical pathway toward resolving the AI data-collapse crisis.

Method: A flexible multi-agent LLM framework with four specialized agents (Coach, Coder, Critic, Capitalizer) using reinforced dynamic prompts and systematic human feedback integration to iteratively generate and validate tools.

Result: Preliminary experiments demonstrate the system’s capability to generate state-of-the-art research papers or patents given an abstract, showing applicability to complex iterative problem-solving tasks.

Conclusion: CollabToolBuilder represents a system-level integration combining multi-agent in-context learning, human-in-the-loop controls, and reusable tool capitalization for complex iterative problems like scientific document generation.

Abstract: We introduce CollabToolBuilder, a flexible multiagent LLM framework with expert-in-the-loop (HITL) guidance that iteratively learns to create tools for a target goal, aligning with human intent and process, while minimizing time for task/domain adaptation effort and human feedback capture. The architecture generates and validates tools via four specialized agents (Coach, Coder, Critic, Capitalizer) using a reinforced dynamic prompt and systematic human feedback integration to reinforce each agent’s role toward goals and constraints. This work is best viewed as a system-level integration and methodology combining multi-agent in-context learning, HITL controls, and reusable tool capitalization for complex iterative problems such as scientific document generation. We illustrate it with preliminary experiments (e.g., generating state-of-the-art research papers or patents given an abstract) and discuss its applicability to other iterative problem-solving.

Method: Defines State Transition Event timeseries (STE-ts) and proposes Selective Temporal Hamming distance (STH) that leverages both transition time and duration-in-state, avoiding resampling.

Result: STH generalizes both resampled Hamming and Jaccard metrics with better precision and computation time, and can focus on multiple states of interest. Validated on simulated and real-world datasets.

Conclusion: The proposed approach provides a more efficient and accurate method for analyzing discrete event systems by combining event timing and state duration information without the distortions of resampling.

Abstract: Discrete event systems are present both in observations of nature, socio economical sciences, and industrial systems. Standard analysis approaches do not usually exploit their dual event / state nature: signals are either modeled as transition event sequences, emphasizing event order alignment, or as categorical or ordinal state timeseries, usually resampled a distorting and costly operation as the observation period and number of events grow. In this work we define state transition event timeseries (STE-ts) and propose a new Selective Temporal Hamming distance (STH) leveraging both transition time and duration-in-state, avoiding costly and distorting resampling on large databases. STH generalizes both resampled Hamming and Jaccard metrics with better precision and computation time, and an ability to focus on multiple states of interest. We validate these benefits on simulated and real-world datasets.

Method: Programmable RoB assessment pipeline using DSPy and GEPA module that replaces ad-hoc prompt design with structured, code-based optimization through Pareto-guided search, producing inspectable execution traces for transparency.

Result: GEPA-generated prompts performed best in domains with clear methodological reporting (Random Sequence Generation), with similar results for Allocation Concealment and Blinding. GEPA achieved highest overall accuracy, improving performance by 30-40% in Random Sequence Generation and Selective Reporting compared to manual prompts.

Conclusion: GEPA can produce consistent and reproducible prompts for RoB assessment, supporting structured and principled use of LLMs in evidence synthesis with transparent optimization processes.

Abstract: Assessing risk of bias (RoB) in randomized controlled trials is essential for trustworthy evidence synthesis, but the process is resource-intensive and prone to variability across reviewers. Large language models (LLMs) offer a route to automation, but existing methods rely on manually engineered prompts that are difficult to reproduce, generalize, or evaluate. This study introduces a programmable RoB assessment pipeline that replaces ad-hoc prompt design with structured, code-based optimization using DSPy and its GEPA module. GEPA refines LLM reasoning through Pareto-guided search and produces inspectable execution traces, enabling transparent replication of every step in the optimization process. We evaluated the method on 100 RCTs from published meta-analyses across seven RoB domains. GEPA-generated prompts were applied to both open-weight models (Mistral Small 3.1 with GPT-oss-20b) and commercial models (GPT-5 Nano and GPT-5 Mini). In domains with clearer methodological reporting, such as Random Sequence Generation, GEPA-generated prompts performed best, with similar results for Allocation Concealment and Blinding of Participants, while the commercial model performed slightly better overall. We also compared GEPA with three manually designed prompts using Claude 3.5 Sonnet. GEPA achieved the highest overall accuracy and improved performance by 30%-40% in Random Sequence Generation and Selective Reporting, and showed generally comparable, competitively aligned performance in the other domains relative to manual prompts. These findings suggest that GEPA can produce consistent and reproducible prompts for RoB assessment, supporting the structured and principled use of LLMs in evidence synthesis.

Method: Multi-Path Perception Policy Optimization (M3PO) - a reinforcement learning framework that leverages parallel policy rollouts as diverse reasoning sources and integrates cross-path interactions through a lightweight collaborative mechanism, allowing trajectories to refine reasoning with peer feedback.

Result: Achieves state-of-the-art performance on both knowledge- and reasoning-intensive benchmarks. Models trained with M3PO maintain interpretability and inference efficiency.

Conclusion: M3PO demonstrates the promise of multi-path collaborative learning for cultivating more reliable multi-step reasoning patterns in LLMs, addressing limitations of deterministic CoT approaches.

Abstract: Chain-of-Thought (CoT) reasoning has significantly advanced the problem-solving capabilities of Large Language Models (LLMs), yet conventional CoT often exhibits internal determinism during decoding, limiting exploration of plausible alternatives. Recent methods attempt to address this by generating soft abstract tokens to enable reasoning in a continuous semantic space. However, we find that such approaches remain constrained by the greedy nature of autoregressive decoding, which fundamentally isolates the model from alternative reasoning possibilities. In this work, we propose Multi-Path Perception Policy Optimization (M3PO), a novel reinforcement learning framework that explicitly injects collective insights into the reasoning process. M3PO leverages parallel policy rollouts as naturally diverse reasoning sources and integrates cross-path interactions into policy updates through a lightweight collaborative mechanism. This design allows each trajectory to refine its reasoning with peer feedback, thereby cultivating more reliable multi-step reasoning patterns. Empirical results show that M3PO achieves state-of-the-art performance on both knowledge- and reasoning-intensive benchmarks. Models trained with M3PO maintain interpretability and inference efficiency, underscoring the promise of multi-path collaborative learning for robust reasoning.

Method: Leverage Large Language Models to distill synthetic knowledge into Python functions representing strategic and tactical rules. Formalize knowledge as verifiable code rather than simple heuristics. Create testable, interpretable representations of synthetic strategy.

Result: Developed a framework enabling natural language-based route retrieval with 75% Top-3 accuracy on benchmark. Released complete codebase and USPTO-ST dataset (synthesis routes annotated with strategic tags). Validated through temporal analysis of historical trends and chemically intuitive route clustering.

Conclusion: This work bridges the tactical-strategic divide in CASP, enabling specification, search, and evaluation of routes by strategic criteria rather than structure alone, unlocking novel capabilities for synthesis planning.

Abstract: Modern computer-assisted synthesis planning (CASP) systems show promises at generating chemically valid reaction steps but struggle to incorporate strategic considerations such as convergent assembly, protecting group minimization, and optimal ring-forming sequences. We introduce a methodology that leverages Large Language Models to distill synthetic knowledge into code. Our system analyzes synthesis routes and translates strategic principles into Python functions representing diverse strategic and tactical rules, such as strategic functional group interconversions and ring construction strategies. By formalizing this knowledge as verifiable code rather than simple heuristics, we create testable, interpretable representations of synthetic strategy. We release the complete codebase and the USPTO-ST dataset – synthesis routes annotated with strategic tags. This framework unlocks a novel capability for CASP: natural language-based route retrieval, achieving 75% Top-3 accuracy on our benchmark. We further validate our library through temporal analysis of historical trends and chemically intuitive route clustering that offers more granular partitioning than common previous methods. This work bridges the tactical-strategic divide in CASP, enabling specification, search, and evaluation of routes by strategic criteria rather than structure alone.

Method: Proposes LEC which treats selective prediction as a constrained decision problem with linear expectation constraints, establishes finite-sample sufficient conditions using calibration data, and extends to two-model routing with delegation to stronger models when uncertainty exceeds calibrated thresholds.

Result: LEC achieves tighter FDR control and substantially improves sample retention over prior methods on QA datasets, and the two-model routing achieves lower risk while accepting more correct samples than individual models.

Conclusion: LEC provides principled FDR control for LLM predictions with statistical guarantees, and the routing mechanism enhances performance by leveraging multiple models while maintaining unified error control.

Abstract: Large language models (LLMs) often generate unreliable answers, while heuristic uncertainty methods fail to fully distinguish correct from incorrect predictions, causing users to accept erroneous answers without statistical guarantees. We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level. To achieve this in a principled way, we propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint over selection and error indicators. Then, we establish a finite-sample sufficient condition, which relies only on a held-out set of exchangeable calibration samples, to compute an FDR-constrained, coverage-maximizing threshold. Furthermore, we extend LEC to a two-model routing mechanism: given a prompt, if the current model’s uncertainty exceeds its calibrated threshold, we delegate it to a stronger model, while maintaining a unified FDR guarantee. Evaluations on closed-ended and open-ended question-answering (QA) datasets show that LEC achieves tighter FDR control and substantially improves sample retention over prior methods. Moreover, the two-model routing mechanism achieves lower risk levels while accepting more correct samples than each individual model.

Method: Extends CLIP principles to RL: creates unified representation space where language instructions and corresponding policies (representing same task) have similar embeddings.

Result: Experimental results show algorithm achieves faster transfer across tasks.

Conclusion: CLIP-inspired alignment of language and policy representations enables efficient multi-task transfer in RL.

Abstract: Recently, there has been an increasing need to develop agents capable of solving multiple tasks within the same environment, especially when these tasks are naturally associated with language. In this work, we propose a novel approach that leverages combinations of pre-trained (language, policy) pairs to establish an efficient transfer pipeline. Our algorithm is inspired by the principles of Contrastive Language-Image Pretraining (CLIP) in Computer Vision, which aligns representations across different modalities under the philosophy that ‘’two modalities representing the same concept should have similar representations.’’ The central idea here is that the instruction and corresponding policy of a task represent the same concept, the task itself, in two different modalities. Therefore, by extending the idea of CLIP to RL, our method creates a unified representation space for natural language and policy embeddings. Experimental results demonstrate the utility of our algorithm in achieving faster transfer across tasks.

Method: Probabilistic neuro-symbolic framework integrating: (1) Bayesian uncertainty quantification, (2) structural causal models for counterfactual reasoning, (3) Shapley values for fair allocation modeling, and (4) attention-based neural architectures for context-dependent factor weighting.

Result: The framework identifies Germany’s +107.9% discrepancy as structural tension preceding WWI, with tension factor 36.43 and 0.79 naval arms race correlation. For Punic Wars, Monte Carlo simulations show 57.3% win probability for Carthage at Cannae and 57.8% for Rome at Zama. Counterfactual analysis reveals Carthaginian political support (score 6.4 vs Napoleon’s 7.1) as decisive factor.

Conclusion: HistoricalML provides a principled approach to historical event modeling that addresses data scarcity, uncertainty, and interpretability challenges through integrated Bayesian, causal, game-theoretic, and neural methods, with theoretical guarantees and empirical validation on historical case studies.

Abstract: Modeling historical events poses fundamental challenges for machine learning: extreme data scarcity (N « 100), heterogeneous and noisy measurements, missing counterfactuals, and the requirement for human interpretable explanations. We present HistoricalML, a probabilistic neuro-symbolic framework that addresses these challenges through principled integration of (1) Bayesian uncertainty quantification to separate epistemic from aleatoric uncertainty, (2) structural causal models for counterfactual reasoning under confounding, (3) cooperative game theory (Shapley values) for fair allocation modeling, and (4) attention based neural architectures for context dependent factor weighting. We provide theoretical analysis showing that our approach achieves consistent estimation in the sparse data regime when strong priors from domain knowledge are available, and that Shapley based allocation satisfies axiomatic fairness guarantees that pure regression approaches cannot provide. We instantiate the framework on two historical case studies: the 19th century partition of Africa (N = 7 colonial powers) and the Second Punic War (N = 2 factions). Our model identifies Germany’s +107.9 percent discrepancy as a quantifiable structural tension preceding World War I, with tension factor 36.43 and 0.79 naval arms race correlation. For the Punic Wars, Monte Carlo battle simulations achieve a 57.3 percent win probability for Carthage at Cannae and 57.8 percent for Rome at Zama, aligning with historical outcomes. Counterfactual analysis reveals that Carthaginian political support (support score 6.4 vs Napoleon’s 7.1), rather than military capability, was the decisive factor.

Method: Trains reliability predictors to assess when LLM judges will agree with human experts using token distributions, embeddings, and structural input features. Dynamically selects optimal jury for each data point and aggregates scores using reliability weights.

Result: Experiments on summarization and RAG benchmarks show significantly higher correlation with human judgment than single-judge and static-jury baselines.

Conclusion: Adaptive, learning-based juries offer promise for building scalable, reliable, and trustworthy evaluation systems for modern LLMs in high-stakes domains.

Abstract: As Large Language Models (LLMs) become integrated into high-stakes domains, there is a growing need for evaluation methods that are both scalable for real-time deployment and reliable for critical decision-making. While human evaluation is reliable, it is slow and costly. Single LLM judges are biased, and static juries lack adaptability. To overcome these limitations, we propose LLM Jury-on-Demand - a dynamic, learning-based framework for scalable and context-aware evaluation. Our method trains a set of reliability predictors to assess when LLM judges will agree with human experts, leveraging token distributions, embeddings, and structural input features. This enables a fully adaptive evaluation where, for each data point, an optimal jury of the most reliable judges is dynamically selected, and their scores are aggregated using their reliability as weights. Experiments on summarization and RAG benchmarks show that our dynamic jury system achieves significantly higher correlation with human judgment than both single-judge and static-jury baselines. These results highlight the promise of adaptive, learning-based juries for building scalable, more reliable and trustworthy evaluation systems for modern LLMs in high-stakes domains.

Method: Systematic investigation of H-Neurons from three perspectives: 1) Identification - finding sparse neuron subsets that predict hallucinations, 2) Behavioral impact - controlled interventions to establish causal links, and 3) Origins - tracing neurons back to pre-trained base models.

Result: A remarkably sparse subset of neurons (less than 0.1% of total neurons) reliably predicts hallucination occurrences with strong generalization across scenarios. These neurons are causally linked to over-compliance behaviors and originate during pre-training, remaining predictive for hallucination detection.

Conclusion: The findings bridge macroscopic behavioral patterns with microscopic neural mechanisms, offering insights for developing more reliable LLMs by understanding the neuron-level origins of hallucinations.

Abstract: Large language models (LLMs) frequently generate hallucinations – plausible but factually incorrect outputs – undermining their reliability. While prior work has examined hallucinations from macroscopic perspectives such as training data and objectives, the underlying neuron-level mechanisms remain largely unexplored. In this paper, we conduct a systematic investigation into hallucination-associated neurons (H-Neurons) in LLMs from three perspectives: identification, behavioral impact, and origins. Regarding their identification, we demonstrate that a remarkably sparse subset of neurons (less than $0.1%$ of total neurons) can reliably predict hallucination occurrences, with strong generalization across diverse scenarios. In terms of behavioral impact, controlled interventions reveal that these neurons are causally linked to over-compliance behaviors. Concerning their origins, we trace these neurons back to the pre-trained base models and find that these neurons remain predictive for hallucination detection, indicating they emerge during pre-training. Our findings bridge macroscopic behavioral patterns with microscopic neural mechanisms, offering insights for developing more reliable LLMs.

Method: Map integers to rooted planar trees via iterated prime factorization, creating sequence $\mathbb{N}\mathcal{T}$. Train GPT-2 transformer from scratch on first $10^{11}$ elements, then test predictive ability using next-word and masked-word prediction tasks.

Result: The model partially learns the internal grammar of $\mathbb{N}\mathcal{T}$, capturing non-trivial regularities and correlations in the deterministic sequence.

Conclusion: Learnability may extend beyond empirical data to the very structure of arithmetic, suggesting LLMs can capture fundamental mathematical patterns.

Abstract: We study whether a Large Language Model can learn the deterministic sequence of trees generated by the iterated prime factorization of the natural numbers. Each integer is mapped into a rooted planar tree and the resulting sequence $ \mathbb{N}\mathcal{T}$ defines an arithmetic text with measurable statistical structure. A transformer network (the GPT-2 architecture) is trained from scratch on the first $10^{11}$ elements to subsequently test its predictive ability under next-word and masked-word prediction tasks. Our results show that the model partially learns the internal grammar of $\mathbb{N}\mathcal{T}$, capturing non-trivial regularities and correlations. This suggests that learnability may extend beyond empirical data to the very structure of arithmetic.

Method: Formalizes surprise using shortest-path distance in directed graphs and provides a framework for KG-based agents, connecting graph distance to message passing depth in GNNs and world model trajectories in RL.

Result: Work-in-progress study exploring whether distance-based surprise can extend recent findings about syntax minimizing surprise and free energy via tree structures.

Conclusion: Proposes a novel connection between neuroscience’s Free Energy Principle and knowledge graph reasoning, suggesting that surprise minimization based on graph distance could provide a unifying framework for various AI approaches including GNNs and model-based RL.

Abstract: In this work, we propose that reasoning in knowledge graph (KG) networks can be guided by surprise minimization. Entities that are close in graph distance will have lower surprise than those farther apart. This connects the Free Energy Principle (FEP) from neuroscience to KG systems, where the KG serves as the agent’s generative model. We formalize surprise using the shortest-path distance in directed graphs and provide a framework for KG-based agents. Graph distance appears in graph neural networks as message passing depth and in model-based reinforcement learning as world model trajectories. This work-in-progress study explores whether distance-based surprise can extend recent work showing that syntax minimizes surprise and free energy via tree structures.

Method: Proposes a computationally efficient move prediction framework using n-gram language models. Players are divided into seven skill groups (novice to expert), with separate models trained on Lichess data. The framework dynamically selects the most suitable model for prediction tasks based on preceding move sequences.

Result: The model selector can classify skill levels with up to 31.7% accuracy using early game information (16 half-moves). The move prediction framework shows up to 39.1% accuracy improvement over benchmark accuracy. The framework is computationally efficient for real-time chess analysis.

Conclusion: The proposed framework successfully addresses the limitation of traditional chess engines by incorporating behavioral analysis of player skill levels, demonstrating significant improvements in move prediction accuracy while maintaining computational efficiency suitable for real-time applications.

Abstract: Chess, a deterministic game with perfect information, has long served as a benchmark for studying strategic decision-making and artificial intelligence. Traditional chess engines or tools for analysis primarily focus on calculating optimal moves, often neglecting the variability inherent in human chess playing, particularly across different skill levels. To overcome this limitation, we propose a novel and computationally efficient move prediction framework that approaches chess move prediction as a behavioral analysis task. The framework employs n-gram language models to capture move patterns characteristic of specific player skill levels. By dividing players into seven distinct skill groups, from novice to expert, we trained separate models using data from the open-source chess platform Lichess. The framework dynamically selects the most suitable model for prediction tasks and generates player moves based on preceding sequences. Evaluation on real-world game data demonstrates that the model selector module within the framework can classify skill levels with an accuracy of up to 31.7% when utilizing early game information (16 half-moves). The move prediction framework also shows substantial accuracy improvements, with our Selector Assisted Accuracy being up to 39.1% more accurate than our benchmark accuracy. The computational efficiency of the framework further enhances its suitability for real-time chess analysis.

Method: Rule-augmented evaluation paradigm with: 1) Rule distillation using LLM-assisted Monte Carlo Tree Search to automatically extract scoring rules from data, 2) Chain-of-Rule (CoR) to guide LLMs to follow distilled rules, and 3) Training a rule-augmented LLM evaluator (RuAE) via reinforcement learning.

Result: Extensive experiments on diverse tasks demonstrate the effectiveness and generalizability of the approach across various evaluation scenarios.

Conclusion: The proposed rule-augmented evaluation paradigm addresses limitations of existing LLM-based evaluators by automatically learning rules from data and effectively applying them through CoR and RuAE, enabling better generalization across diverse evaluation tasks.

Abstract: Large language models (LLMs) are predominantly used as evaluators for natural language generation (NLG) tasks, but their application to broader evaluation scenarios remains limited. In this work, we explore the potential of LLMs as general evaluators across diverse tasks. Although LLM-based evaluators have made progress in different areas, existing methods struggle to generalize due to their reliance on costly, human-designed evaluation principles, which are often misaligned with both annotated data and LLMs’ understanding.To address these challenges, we propose a rule-augmented evaluation paradigm. First, we introduce a rule distillation method that automatically extracts scoring rules from data using an LLM-assisted Monte Carlo Tree Search (MCTS), alleviating scalability issues and improving alignment with data. Second, to enable LLMs to effectively apply the learned rules, we propose two strategies: (1) Chain-of-Rule (CoR), which guides LLM to follow distilled rules, and (2) training a rule-augmented LLM evaluator (RuAE) via reinforcement learning, further bridging the gap between rules and LLMs’ reasoning. Extensive experiments on diverse tasks demonstrate the effectiveness and generalizability of our approach across various evaluation scenarios.

Method: Used controlled synthetic dataset of human biographies, decoupled Complementary Reasoning into Parametric (internal knowledge) and Contextual (external info) atomic skills. Evaluated generalization across I.I.D., Composition, and Zero-shot settings, comparing SFT vs RL approaches.

Result: SFT achieves near-perfect in-distribution accuracy but collapses on OOD generalization (SFT Generalization Paradox). RL acts as reasoning synthesizer, but only when base model first masters atomic skills via SFT. RL enables better generalization to novel relational combinations.

Conclusion: RL synthesizes complex reasoning strategies from learned primitives, not just amplifies probabilities. Decoupled atomic training followed by RL offers scalable path to generalization for complex reasoning tasks.

Abstract: The mechanism by which RL contributes to reasoning capabilities-whether it incentivizes the synthesis of new skills or merely amplifies existing behaviors-remains a subject of intense debate. In this work, we investigate this question through the lens of Complementary Reasoning, a complex task that requires integrating internal parametric knowledge with external contextual information. Using a controlled synthetic dataset of human biographies, we strictly decouple this ability into two atomic skills: Parametric Reasoning (relying on internal knowledge) and Contextual Reasoning (depending on external information). To rigorously assess capability boundaries, we evaluate generalization across three distinct levels of difficulty: I.I.D., Composition, and Zero-shot settings. We find that while SFT is sufficient for in-distribution performance, it struggles with O.O.D. generalization, particularly in Zero-shot settings where relational combinations are novel. Crucially, we identify the SFT Generalization Paradox: Models supervised solely on the composite task achieve near-perfect in-distribution accuracy but collapse on out-of-distribution generalization, indicating their reliance on rote memorization of path shortcuts. In contrast, we find that RL acts as a reasoning synthesizer rather than a probability amplifier. However, we uncover a strict atomic prerequisite: RL can only synthesize these complex strategies if the base model has first mastered the independent atomic skills (Parametric and Contextual) via SFT. These findings challenge the view of RL as a mere amplifier, suggesting that given sufficient atomic foundations, RL can actively synthesize complex reasoning strategies from learned primitives without explicit supervision on such complex strategies. This indicates that decoupled atomic training followed by RL offers a scalable path to generalization for complex reasoning tasks.

Method: CoG uses iterative visual reasoning and refinement where models progressively reflect and adjust hypotheses instead of direct prediction, enabling more accurate and interpretable localization.

Result: Achieves 68.4% accuracy on ScreenSpot Pro benchmark (4.8 point improvement) and introduces TPanel UI dataset; improves over Qwen3 VL 235B by 6.9 points on real-world industrial panels.

Conclusion: Structured iterative refinement unlocks grounding potential without additional training, demonstrating effectiveness across both digital and real-world interfaces.

Abstract: GUI grounding aims to align natural language instructions with precise regions in complex user interfaces. Advanced multimodal large language models show strong ability in visual GUI grounding but still struggle with small or visually similar targets and ambiguity in real world layouts. These limitations arise from limited grounding capacity and from underuse of existing reasoning potential. We present Chain of Ground CoG a training free multi step grounding framework that uses multimodal large language models for iterative visual reasoning and refinement. Instead of direct prediction the model progressively reflects and adjusts its hypotheses leading to more accurate and interpretable localization. Our approach achieves 68.4 accuracy on the ScreenSpot Pro benchmark an improvement of 4.8 points. To measure real world generalization we introduce TPanel UI a dataset of 420 labeled industrial control panels with visual distortions such as blur and masking. On TPanel UI Chain of Ground improves over the strong baseline Qwen3 VL 235B by 6.9 points showing the effectiveness of multi step training free grounding across real world and digital interfaces. These results highlight a direction for unlocking grounding potential through structured iterative refinement instead of additional training.

Method: Developed LLM CHESS framework where models play chess against random opponents, tracking metrics like win/loss rates, move quality, legality, hallucinations, and game duration. For top models, derived Elo estimates by playing against variably configured chess engines.

Result: Ranked over 50 open/closed source models, revealing clear separation between reasoning and non-reasoning models. Many state-of-the-art models struggle to complete games or achieve consistent wins despite simple instructions and weak opponents. The framework reduces overfitting and prevents benchmark saturation.

Conclusion: LLM CHESS provides a unique stochastic and dynamic evaluation that challenges even top reasoning models, offering a robust alternative to static benchmarks. The framework, leaderboard, and game dataset are released to support future research on LLM reasoning evaluation.

Abstract: We introduce LLM CHESS, an evaluation framework designed to probe the generalization of reasoning and instruction-following abilities in large language models (LLMs) through extended agentic interaction in the domain of chess. We rank over 50 open and closed source models by playing against a random opponent using a range of behavioral metrics, including win and loss rates, move quality, move legality, hallucinated actions, and game duration. For a subset of top reasoning models, we derive an Elo estimate by playing against a chess engine with variably configured skill, which allows for comparisons between models in an easily understandable way. Despite the simplicity of the instruction-following task and the weakness of the opponent, many state-of-the-art models struggle to complete games or achieve consistent wins. Similar to other benchmarks on complex reasoning tasks, our experiments reveal a clear separation between reasoning and non-reasoning models. However, unlike existing static benchmarks, the stochastic and dynamic nature of LLM CHESS uniquely reduces overfitting and memorization while preventing benchmark saturation, proving difficult even for top reasoning models. To support future work on evaluating reasoning and instruction-following in LLMs, we release our experimental framework, a public leaderboard, and a dataset of associated games.

Method: 1) Construct a multimodal LLM (MLLM) based on the target LLM; 2) Perform efficient MLLM jailbreak to obtain a jailbreaking embedding; 3) Convert the embedding into a textual jailbreaking suffix; 4) Use an image-text semantic matching scheme to identify suitable initial inputs for higher attack success rates.

Result: The approach outperforms current state-of-the-art jailbreak methods in both efficiency and effectiveness, and demonstrates superior cross-class generalization abilities in extensive experiments.

Conclusion: Indirect jailbreaking via MLLMs is more effective than direct LLM attacks due to MLLMs’ greater vulnerability, and the proposed method provides a powerful framework for evaluating and improving LLM safety.

Abstract: This paper focuses on jailbreaking attacks against large language models (LLMs), eliciting them to generate objectionable content in response to harmful user queries. Unlike previous LLM-jailbreak methods that directly orient to LLMs, our approach begins by constructing a multimodal large language model (MLLM) built upon the target LLM. Subsequently, we perform an efficient MLLM jailbreak and obtain a jailbreaking embedding. Finally, we convert the embedding into a textual jailbreaking suffix to carry out the jailbreak of target LLM. Compared to the direct LLM-jailbreak methods, our indirect jailbreaking approach is more efficient, as MLLMs are more vulnerable to jailbreak than pure LLM. Additionally, to improve the attack success rate of jailbreak, we propose an image-text semantic matching scheme to identify a suitable initial input. Extensive experiments demonstrate that our approach surpasses current state-of-the-art jailbreak methods in terms of both efficiency and effectiveness. Moreover, our approach exhibits superior cross-class generalization abilities.

Method: Proposed IoT-LLM framework with three key steps: 1) Preprocessing IoT data into suitable formats, 2) Expanding LLM knowledge via IoT-oriented retrieval-augmented generation, and 3) Activating LLMs’ commonsense knowledge through chain-of-thought prompting.

Result: IoT-LLM significantly improves LLMs’ performance on IoT-sensory tasks, with GPT-4o-mini showing a 49.4% average improvement over previous methods. Evaluation used a benchmark with five real-world tasks varying in data types and reasoning complexities.

Conclusion: Augmenting LLMs with IoT perception capabilities through the IoT-LLM framework effectively bridges the gap between textual reasoning and physical-world understanding, enabling better performance on sensor-based reasoning tasks.

Abstract: Large Language Models excel in textual tasks but often struggle with physical-world reasoning tasks. Inspired by human cognition, where perception is fundamental to reasoning, we explore augmenting LLMs with enhanced perception abilities using Internet of Things (IoT) data and pertinent knowledge. In this work, we systematically study LLMs’ capability to address IoT-sensory tasks by augmenting their perception and knowledge base, and then propose a unified framework, IoT-LLM, to enhance such capability. In IoT-LLM, we customize three steps: preprocessing IoT data into suitable formats, expanding LLMs knowledge via IoT-oriented retrieval-augmented generation and activating LLMs commonsense knowledge through chain-of-thought prompting. We design a benchmark comprising five real-world tasks with varying data types and reasoning complexities to evaluate the performance of IoT-LLM. Experimental results reveal that IoT-LLM significantly improves the performance of IoT-sensory task reasoning of LLMs, with models like GPT-4o-mini showing a 49.4% average improvement over previous methods.

Method: Randomized between-subjects experiment with 233 participants interacting with one of three AI agents: neutral agent (optimizing for user benefit), manipulative agent (covert influence), or strategy-enhanced manipulative agent (using adaptive psychological tactics). Tested in financial and emotional decision-making contexts.

Result: Significant susceptibility to AI manipulation across both domains. Manipulative agents increased odds of rating hidden incentives higher than optimal options (Financial: MA OR=5.24, SEMA OR=7.96; Emotional: MA OR=5.52, SEMA OR=5.71). No clear evidence that sophisticated psychological tactics (SEMA) were more effective than simple manipulation (MA).

Conclusion: AI-driven manipulation could become widespread even without sophisticated tactics, highlighting critical vulnerability in human-AI interactions. Need for ethical safeguards and regulatory frameworks to protect human autonomy, though findings are preliminary from low-stakes scenarios.

Abstract: AI systems are increasingly intertwined with daily life, assisting users with various tasks and guiding decision-making. This integration introduces risks of AI-driven manipulation, where such systems may exploit users’ cognitive biases and emotional vulnerabilities to steer them toward harmful outcomes. Through a randomized between-subjects experiment with 233 participants, we examined human susceptibility to such manipulation in financial (e.g., purchases) and emotional (e.g., conflict resolution) decision-making contexts. Participants interacted with one of three AI agents: a neutral agent (NA) optimizing for user benefit without explicit influence, a manipulative agent (MA) designed to covertly influence beliefs and behaviors, or a strategy-enhanced manipulative agent (SEMA) equipped with established psychological tactics, allowing it to select and apply them adaptively during interactions to reach its hidden objectives. By analyzing participants’ preference ratings, we found significant susceptibility to AI-driven manipulation. Particularly across both decision-making domains, interacting with the manipulative agents significantly increased the odds of rating hidden incentives higher than optimal options (Financial, MA: OR=5.24, SEMA: OR=7.96; Emotional, MA: OR=5.52, SEMA: OR=5.71) compared to the NA group. Notably, we found no clear evidence that employing psychological strategies (SEMA) was overall more effective than simple manipulative objectives (MA) on our primary outcomes. Hence, AI-driven manipulation could become widespread even without requiring sophisticated tactics and expertise. While our findings are preliminary and derived from hypothetical, low-stakes scenarios, we highlight a critical vulnerability in human-AI interactions, emphasizing the need for ethical safeguards and regulatory frameworks to protect human autonomy.

Method: Introduces RealWebAssist benchmark with dataset of sequential instructions from real users instructing web assistants across multiple websites, requiring agents to infer intent, track user mental states, understand routines, and ground tasks to GUI elements.

Result: State-of-the-art models struggle with understanding and grounding user instructions, revealing critical challenges in following real-world sequential instructions for long-horizon web assistance.

Conclusion: RealWebAssist addresses the gap in evaluating sequential instruction-following for realistic web assistance, highlighting significant challenges that current AI models face in handling ambiguous, evolving real-world user instructions.

Abstract: To achieve successful assistance with long-horizon web-based tasks, AI agents must be able to sequentially follow real-world user instructions over a long period. Unlike existing web-based agent benchmarks, sequential instruction following in the real world poses significant challenges beyond performing a single, clearly defined task. For instance, real-world human instructions can be ambiguous, require different levels of AI assistance, and may evolve over time, reflecting changes in the user’s mental state. To address this gap, we introduce RealWebAssist, a novel benchmark designed to evaluate sequential instruction-following in realistic scenarios involving long-horizon interactions with the web, visual GUI grounding, and understanding ambiguous real-world user instructions. RealWebAssist includes a dataset of sequential instructions collected from real-world human users. Each user instructs a web-based assistant to perform a series of tasks on multiple websites. A successful agent must reason about the true intent behind each instruction, keep track of the mental state of the user, understand user-specific routines, and ground the intended tasks to actions on the correct GUI elements. Our experimental results show that state-of-the-art models struggle to understand and ground user instructions, posing critical challenges in following real-world user instructions for long-horizon web assistance.

Method: Calgacus protocol - a simple and efficient method using LLMs (even modest 8-billion-parameter models) to encode and decode hidden messages within seemingly innocent text of the same length.

Result: High-quality results achievable with modest LLMs; messages as long as an abstract can be encoded/decoded locally on a laptop in seconds; demonstrates radical decoupling of text from authorial intent.

Conclusion: This capability raises urgent AI safety questions, challenges understanding of what LLMs “know,” and enables dangerous scenarios like covert deployment of unfiltered LLMs hidden within safe model responses.

Abstract: A meaningful text can be hidden inside another, completely different yet still coherent and plausible, text of the same length. For example, a tweet containing a harsh political critique could be embedded in a tweet that celebrates the same political leader, or an ordinary product review could conceal a secret manuscript. This uncanny state of affairs is now possible thanks to Large Language Models, and in this paper we present Calgacus, a simple and efficient protocol to achieve it. We show that even modest 8-billion-parameter open-source LLMs are sufficient to obtain high-quality results, and a message as long as this abstract can be encoded and decoded locally on a laptop in seconds. The existence of such a protocol demonstrates a radical decoupling of text from authorial intent, further eroding trust in written communication, already shaken by the rise of LLM chatbots. We illustrate this with a concrete scenario: a company could covertly deploy an unfiltered LLM by encoding its answers within the compliant responses of a safe model. This possibility raises urgent questions for AI safety and challenges our understanding of what it means for a Large Language Model to know something.

Method: MetaEMS uses meta-reinforcement learning with group-level and building-level adaptation to transfer knowledge from previously solved tasks to new ones, enabling fast adaptation to diverse building environments.

Result: Experimental results show that MetaEMS adapts more rapidly to unseen buildings and consistently outperforms baseline methods across various scenarios.

Conclusion: The proposed MetaEMS framework addresses the adaptation challenge in RL-based energy management systems, enabling more practical deployment by improving learning efficiency and performance across diverse building environments.

Abstract: The building sector is one of the largest contributors to global energy consumption. Improving its energy efficiency is essential for reducing operational costs and greenhouse gas emissions. Energy management systems (EMS) play a key role in monitoring and controlling building appliances efficiently and reliably. With the increasing integration of renewable energy, intelligent EMS solutions have received growing attention. Reinforcement learning (RL) has recently been explored for this purpose and shows strong potential. However, most RL-based EMS methods require a large number of training steps to learn effective control policies, especially when adapting to unseen buildings, which limits their practical deployment. This paper introduces MetaEMS, a meta-reinforcement learning framework for EMS. MetaEMS improves learning efficiency by transferring knowledge from previously solved tasks to new ones through group-level and building-level adaptation, enabling fast adaptation and effective control across diverse building environments. Experimental results demonstrate that MetaEMS adapts more rapidly to unseen buildings and consistently outperforms baseline methods across various scenarios.

Method: Uses knowledge graph to curate diverse, interpretable prompts, augments with LLM-generated visual descriptions when KG coverage is limited, operates on cached CLIP embeddings, and employs lightweight Gumbel-Softmax module to select single prompt per image-class pair.

Result: Reduces per-image training time by up to 45% and peak GPU memory by 30-40% compared to baselines, while keeping average base-new harmonic-mean accuracy within 2 percentage points.

Conclusion: EPLKG improves efficiency of model adaptation without sacrificing competitive performance or interpretability, making prompt learning more practical for resource-constrained settings.

Abstract: Large-scale pre-trained models such as CLIP excel in transferability and robust generalization across diverse datasets. However, adapting these models to new datasets or domains is computationally costly, especially in low-resource or few-shot settings, and existing prompt-learning methods often lack interpretability. We introduce Efficient Prompt Learning with Knowledge Graph (EPLKG), which uses a knowledge graph to curate diverse, interpretable prompts and, where KG coverage is limited, augments this bank with LLM-generated human-readable visual descriptions. EPLKG operates entirely on cached CLIP image and text embeddings and employs a lightweight Gumbel-Softmax module to select a single prompt per image-class pair, enabling low-memory, fast training. Across 11 benchmarks, EPLKG reduces per-image training time by up to 45 percent and peak GPU memory by around 30 to 40 percent compared to strong prompt-learning baselines, while keeping the average base-new harmonic-mean accuracy within 2 percentage points, thereby improving the efficiency of model adaptation without sacrificing competitive performance or interpretability.

Method: GraphIC constructs thought graphs (directed, node-attributed graphs) that explicitly model reasoning steps and their dependencies for both candidate examples and queries. It then uses a novel similarity metric tailored for these graphs that captures sequential reasoning patterns and asymmetry between examples.

Result: GraphIC outperforms 10 baseline methods across mathematical reasoning, code generation, and logical reasoning tasks, demonstrating superior performance in multi-step reasoning scenarios.

Conclusion: The paper highlights the importance of reasoning-aware retrieval in ICL and provides a robust solution for enhancing LLM performance in multi-step reasoning scenarios through graph-based representation and specialized similarity metrics.

Abstract: In-context learning (ICL) enhances large language models (LLMs) by incorporating demonstration examples, yet its effectiveness heavily depends on the quality of selected examples. Current methods typically use text embeddings to measure semantic similarity, which often introduces bias in multi-step reasoning tasks. This occurs because text embeddings contain irrelevant semantic information and lack deeper reasoning structures. To address this, we propose GraphIC, a graph-based retrieval model that leverages reasoning-aware representation and specialized similarity metric for in-context example retrieval. GraphIC first constructs thought graphs-directed, node-attributed graphs that explicitly model reasoning steps and their dependencies-for candidate examples and queries. This approach filters out superficial semantics while preserving essential reasoning processes. Next, GraphIC retrieves examples using a novel similarity metric tailored for these graphs, capturing sequential reasoning patterns and asymmetry between examples. Comprehensive evaluations across mathematical reasoning, code generation, and logical reasoning tasks demonstrate that GraphIC outperforms 10 baseline methods. Our results highlight the importance of reasoning-aware retrieval in ICL, offering a robust solution for enhancing LLM performance in multi-step reasoning scenarios.

Method: Survey methodology with detailed case studies and technical analysis of prominent MLLM implementations, covering foundational concepts, architectural components, training methods, and practical applications across various fields.

Result: A comprehensive examination of MLLMs’ capabilities in enabling complex AI systems for cross-modal understanding and generation, while identifying key challenges in scalability, robustness, and cross-modal learning.

Conclusion: The paper provides balanced perspectives on opportunities and challenges in MLLM development, addresses ethical considerations and responsible AI, and serves as valuable resource for researchers, practitioners, and students in NLP and computer vision intersection.

Abstract: This survey and application guide to multimodal large language models(MLLMs) explores the rapidly developing field of MLLMs, examining their architectures, applications, and impact on AI and Generative Models. Starting with foundational concepts, we delve into how MLLMs integrate various data types, including text, images, video and audio, to enable complex AI systems for cross-modal understanding and generation. It covers essential topics such as training methods, architectural components, and practical applications in various fields, from visual storytelling to enhanced accessibility. Through detailed case studies and technical analysis, the text examines prominent MLLM implementations while addressing key challenges in scalability, robustness, and cross-modal learning. Concluding with a discussion of ethical considerations, responsible AI development, and future directions, this authoritative resource provides both theoretical frameworks and practical insights. It offers a balanced perspective on the opportunities and challenges in the development and deployment of MLLMs, and is highly valuable for researchers, practitioners, and students interested in the intersection of natural language processing and computer vision.

Method: Combines CNN-LSTM for dynamic event detection with reinforcement learning and symbolic reasoning to create an adaptive neuro-symbolic learning framework that enables interpretable decision processes.

Result: Achieved 99.5% accuracy for dynamic pattern recognition on synthetic industrial data, with reinforcement learning improving explained variance from 0.447 to 0.547. Framework demonstrated superior adaptability with extended training.

Conclusion: ANSR-DT provides an effective solution for adaptive, interpretable digital twins with open-source implementation for reproducibility. Future work includes scaling to larger datasets and testing beyond current 14-rule management.

Abstract: In this paper, we propose an Adaptive Neuro-Symbolic Learning and Reasoning Framework for digital twin technology called “ANSR-DT.” Digital twins in industrial environments often struggle with interpretability, real-time adaptation, and human input integration. Our approach addresses these challenges by combining CNN-LSTM dynamic event detection with reinforcement learning and symbolic reasoning to enable adaptive intelligence with interpretable decision processes. This integration enhances environmental understanding while promoting continuous learning, leading to more effective real-time decision-making in human-machine collaborative applications. We evaluated ANSR-DT on synthetic industrial data, observing significant improvements over traditional approaches, with up to 99.5% accuracy for dynamic pattern recognition. The framework demonstrated superior adaptability with extended reinforcement learning training, improving explained variance from 0.447 to 0.547. Future work aims at scaling to larger datasets to test rule management beyond the current 14 rules. Our open-source implementation promotes reproducibility and establishes a foundation for future research in adaptive, interpretable digital twins for industrial applications.

Method: Conducted structured literature search across major databases (2017-2025), extracting key information on datasets, generative architectures (GANs, VAEs, transformers, diffusion models), EEG feature-encoding techniques, evaluation metrics, and challenges.

Result: Found that EEG-to-image models use encoder-decoder architectures (GANs/VAEs/diffusion), EEG-to-text approaches leverage transformer-based language models, and EEG-to-audio methods map EEG to mel-spectrograms with neural vocoders. Identified constraints: small heterogeneous datasets, limited cross-subject generalization, and lack of standardized benchmarks.

Conclusion: The survey provides foundational reference for advancing EEG-based generative AI, highlights open-source datasets and baseline implementations to facilitate systematic benchmarking, and aims to support reproducible research in EEG-driven neural decoding.

Abstract: Decoding neural activity into human-interpretable representations is a key research direction in brain-computer interfaces (BCIs) and computational neuroscience. Recent progress in machine learning and generative AI has driven growing interest in transforming non-invasive Electroencephalography (EEG) signals into images, text, and audio. This survey consolidates and analyzes developments across EEG-to-image synthesis, EEG-to-text generation, and EEG-to-audio reconstruction. We conducted a structured literature search across major databases (2017-2025), extracting key information on datasets, generative architectures (GANs, VAEs, transformers, diffusion models), EEG feature-encoding techniques, evaluation metrics, and the major challenges shaping current work in this area. Our review finds that EEG-to-image models predominantly employ encoder-decoder architectures built on GANs, VAEs, or diffusion models; EEG-to-text approaches increasingly leverage transformer-based language models for open-vocabulary decoding; and EEG-to-audio methods commonly map EEG signals to mel-spectrograms that are subsequently rendered into audio using neural vocoders. Despite promising advances, the field remains constrained by small and heterogeneous datasets, limited cross-subject generalization, and the absence of standardized benchmarks. By consolidating methodological trends and available datasets, this survey provides a foundational reference for advancing EEG-based generative AI and supporting reproducible research. We further highlight open-source datasets and baseline implementations to facilitate systematic benchmarking and accelerate progress in EEG-driven neural decoding.

Method: Uses contextual multi-armed bandits (CMABs) to learn an adaptive policy that evaluates the current reasoning state and determines optimal strategies (backtrack, switch approaches, restart) during inference.

Result: Outperforms previous SOTA methods by 9-12% in accuracy on math and scientific tasks, while reducing inference time by 28-35% under the same compute budget.

Conclusion: Meta-Reasoner enables LLMs to “think about how to think” through dynamic strategy adaptation, improving both performance and efficiency across diverse reasoning-intensive tasks.

Abstract: Large Language Models (LLMs) often struggle with computational efficiency and error propagation in multi-step reasoning tasks. While recent advancements on prompting and post-training have enabled LLMs to perform step-wise reasoning, they still tend to explore unproductive solution paths without effective backtracking or strategy adjustment. In this paper, we propose Meta-Reasoner, a new framework that empowers LLMs to “think about how to think”. It optimizes the inference process by dynamically adapting reasoning strategies in real-time. Our approach employs contextual multi-armed bandits (CMABs) to learn an adaptive policy. It learns to evaluate the current state of LLM’s reasoning and determine optimal strategy that is most likely to lead to a successful outcome during inference, like whether to backtrack, switch to a new approach, or restart the problem-solving process. This meta-guidance helps avoid unproductive paths exploration during inference and hence improves computational efficiency. We evaluate Meta-Reasoner on math problems (e.g., Game-of-24, TheoremQA) and scientific tasks (e.g., SciBench). Results show that our method outperform previous SOTA methods by 9-12% in accuracy, while reducing inference time by 28-35% under the same compute budget. Additional experiments on creative writing demonstrate the generalizability of our approach to diverse reasoning-intensive tasks.

Method: Proposes Unified Contextual Control Theory (UCCT) with anchoring strength formula S = ρ_d - d_r - log k, where ρ_d measures target cohesion, d_r measures mismatch from prior knowledge, and k is anchor budget. Conducts three controlled experiments: cross-domain anchoring rebinding, numeral base familiarity tests, and geometry-to-behavior correlation analysis.

Result: Three experiments provide evidence: 1) Cross-domain anchoring successfully rebinds strong priors in text and vision; 2) Varying numeral bases shows ordered performance thresholds and transfer patterns tracking UCCT parameters; 3) Layer-wise peak anchoring and trajectory area predict few-shot performance thresholds θ_50.

Conclusion: UCCT offers a testable theoretical framework that unifies in-context learning, retrieval, and fine-tuning as variants of semantic anchoring, providing practical metrics for optimizing prompts, retrieval systems, and tuning strategies based on measurable representational properties.

Abstract: We propose semantic anchoring, a unified account of how large language models turn pretrained capacity into goal-directed behavior: external structure (in-context examples, retrieval, or light tuning) binds the model’s latent patterns to desired targets. Unified Contextual Control Theory (UCCT) formalizes this via anchoring strength $S = ρ_d - d_r - \log k$, where $ρ_d$ measures target cohesion in representation space, $d_r$ measures mismatch from prior knowledge, and $k$ is the anchor budget. UCCT predicts threshold-like performance flips and strictly generalizes in-context learning, reading retrieval and fine-tuning as anchoring variants. Three controlled studies provide evidence. Experiment 1 demonstrates cross-domain anchoring rebinding strong priors in text and vision. Experiment 2 varies representational familiarity via numeral bases (base-10/8/9) at fixed complexity, yielding ordered thresholds and transfer patterns tracking $ρ_d$, $d_r$, and $S$. Experiment 3 establishes a geometry-to-behavior correlate: layer-wise peak anchoring and trajectory area predict few-shot thresholds $θ_{50}$. UCCT offers testable theory and practical metrics for optimizing prompts, retrieval, and tuning.

Method: Created ForageWorld - a novel, complex, partially observable environment simulating real-world animal foraging with sparse/depleting resources, predators, and large arenas. Applied neuroethology-inspired tools to jointly analyze agent behavior and neural dynamics, developing a general analysis framework linking behavioral and representational features to diagnostic methods.

Result: Found that model-free RNN-based DRL agents can exhibit structured, planning-like behavior through emergent dynamics without explicit memory modules or world models. The neuroethology approach revealed rich structure in learning dynamics that would otherwise remain invisible, showing these agents develop sophisticated strategies purely through learning.

Conclusion: Studying DRL agents like animals using neuroscience/ethology tools uncovers hidden structure in their behavior and learning. As agents grow more complex, bridging neuroscience, cognitive science, and AI is essential for understanding behavior, ensuring safe alignment, and maximizing desirable behaviors not easily measured via reward.

Abstract: Understanding the behavior of deep reinforcement learning (DRL) agents -particularly as task and agent sophistication increase- requires more than simple comparison of reward curves, yet standard methods for behavioral analysis remain underdeveloped in DRL. We apply tools from neuroscience and ethology to study DRL agents in a novel, complex, partially observable environment, ForageWorld, designed to capture key aspects of real-world animal foraging- including sparse, depleting resource patches, predator threats, and spatially extended arenas. We use this environment as a platform for applying joint behavioral and neural analysis to agents, revealing detailed, quantitatively grounded insights into agent strategies, memory, and planning. Contrary to common assumptions, we find that model-free RNN-based DRL agents can exhibit structured, planning-like behavior purely through emergent dynamics- without requiring explicit memory modules or world models. Our results show that studying DRL agents like animals -analyzing them with neuroethology-inspired tools that reveal structure in both behavior and neural dynamics- uncovers rich structure in their learning dynamics that would otherwise remain invisible. We distill these tools into a general analysis framework linking core behavioral and representational features to diagnostic methods, which can be reused for a wide range of tasks and agents. As agents grow more complex and autonomous, bridging neuroscience, cognitive science, and AI will be essential- not just for understanding their behavior, but for ensuring safe alignment and maximizing desirable behaviors that are hard to measure via reward. We show how this can be done by drawing on lessons from how biological intelligence is studied.

Method: Introduces two diagnostic tasks: 1) file path identification from issue descriptions alone, and 2) ground truth function reproduction with only current file context and issue description. Tests models on SWE-Bench Verified versus other repositories not included in the benchmark.

Result: State-of-the-art models achieve up to 76% accuracy on file path identification for SWE-Bench data but only up to 53% on other repositories. For function reproduction, verbatim similarity is much higher on SWE-Bench (up to 35% consecutive 5-gram accuracy) versus other benchmarks (only up to 18%).

Conclusion: Performance gains on SWE-Bench may be partially driven by memorization rather than genuine problem-solving, raising concerns about benchmark validity and highlighting the need for more robust, contamination-resistant evaluation methods.

Abstract: As large language models (LLMs) become increasingly capable and widely adopted, benchmarks play a central role in assessing their practical utility. For example, SWE-Bench Verified has emerged as a critical benchmark for evaluating LLMs’ software engineering abilities, particularly their aptitude for resolving real-world GitHub issues. Recent LLMs show impressive performance on SWE-Bench, leading to optimism about their capacity for complex coding tasks. However, current evaluation protocols may overstate these models’ true capabilities. It is crucial to distinguish LLMs’ generalizable problem-solving ability and other learned artifacts. In this work, we introduce two diagnostic tasks: file path identification from issue descriptions alone and ground truth function reproduction with only the current file context and issue description to probe models’ underlying knowledge. We present empirical evidence that performance gains on SWE-Bench-Verified may be partially driven by memorization rather than genuine problem-solving. We show that state-of-the-art models achieve up to 76% accuracy in identifying buggy file paths using only issue descriptions, without access to repository structure. This performance is merely up to 53% on tasks from repositories not included in SWE-Bench, pointing to possible data contamination or memorization. Similar patterns are also observed for the function reproduction task, where the verbatim similarity is much higher on SWE-Bench Verified than on other similar coding benchmarks (up to 35% consecutive 5-gram accuracy on SWE-Bench Verified and Full, but only up to 18% for tasks in other benchmarks). These findings raise concerns about the validity of existing results and underscore the need for more robust, contamination-resistant benchmarks to reliably evaluate LLMs’ coding abilities.

Method: Adaptive sensing proactively modulates sensor parameters (exposure, sensitivity, multimodal configurations) at the input level to mitigate covariate shifts and improve efficiency. The approach draws inspiration from biological sensory adaptation mechanisms.

Result: Empirical evidence shows adaptive sensing enables small models (e.g., EfficientNet-B0) to surpass substantially larger models (e.g., OpenCLIP-H) trained with significantly more data and compute resources.

Conclusion: The paper outlines a roadmap for integrating adaptive sensing across various applications, addresses technical/ethical challenges, and proposes research directions to transition toward sustainable, robust, and equitable AI systems.

Abstract: Current AI advances largely rely on scaling neural models and expanding training datasets to achieve generalization and robustness. Despite notable successes, this paradigm incurs significant environmental, economic, and ethical costs, limiting sustainability and equitable access. Inspired by biological sensory systems, where adaptation occurs dynamically at the input (e.g., adjusting pupil size, refocusing vision)–we advocate for adaptive sensing as a necessary and foundational shift. Adaptive sensing proactively modulates sensor parameters (e.g., exposure, sensitivity, multimodal configurations) at the input level, significantly mitigating covariate shifts and improving efficiency. Empirical evidence from recent studies demonstrates that adaptive sensing enables small models (e.g., EfficientNet-B0) to surpass substantially larger models (e.g., OpenCLIP-H) trained with significantly more data and compute. We (i) outline a roadmap for broadly integrating adaptive sensing into real-world applications spanning humanoid, healthcare, autonomous systems, agriculture, and environmental monitoring, (ii) critically assess technical and ethical integration challenges, and (iii) propose targeted research directions, such as standardized benchmarks, real-time adaptive algorithms, multimodal integration, and privacy-preserving methods. Collectively, these efforts aim to transition the AI community toward sustainable, robust, and equitable artificial intelligence systems.

Method: Feature Steering with Reinforcement Learning (FSRL) trains lightweight adapters to modulate interpretable sparse features, theoretically shown to approximate behavioral shifts of post-training processes, and applied to preference optimization for causal analysis.

Result: Analysis reveals models learn to reward stylistic presentation as a proxy for quality, disproportionately relying on style/formatting features over alignment concepts like honesty, effectively optimizing preference objectives while revealing underlying mechanisms.

Conclusion: FSRL provides an interpretable control interface and practical diagnostic tool to understand how preference optimization pressures manifest at the feature level, offering transparency into alignment processes.

Abstract: Prevailing alignment methods induce opaque parameter changes, obscuring what models truly learn. To address this, we introduce Feature Steering with Reinforcement Learning (FSRL), a framework that trains a lightweight adapter to steer model behavior by modulating interpretable sparse features. First, we theoretically demonstrate that this mechanism is expressive enough to approximate the behavioral shifts of post-training processes. We then apply FSRL to preference optimization and perform a causal analysis of the learned policy. Our analysis reveals a crucial insight: the model learns to reward stylistic presentation as a proxy for quality, disproportionately relying on features related to style and formatting over those tied to alignment concepts like honesty. By effectively optimizing the preference objective, FSRL serves as a transparent proxy for observing the alignment process. Overall, FSRL offers an interpretable control interface and a practical way to diagnose how preference optimization pressures manifest at the feature level.

Method: Proposed physics-informed AI framework that explicitly integrates vehicle kinematics, interaction feasibility, and traffic-safety metrics (distance headway, time headway, time-to-collision, closing gap time). Formulates lane-change prediction as three-class problem (left change, right change, no change) and evaluates on both straight highway segments (highD) and complex ramp scenarios (exiD). Uses LightGBM as the primary machine learning model.

Result: Achieves state-of-the-art accuracy: up to 99.8% accuracy and 93.6% macro F1 on highD, and 96.1% accuracy and 88.7% macro F1 on exiD at 1-second horizon. Outperforms two-layer stacked LSTM baseline.

Conclusion: Demonstrates practical advantages of physics-informed and feature-rich machine learning framework for real-time lane-change intention prediction in autonomous driving systems, showing strong generalization across diverse scenarios.

Abstract: Lane-change maneuvers are a leading cause of highway accidents, underscoring the need for accurate intention prediction to improve the safety and decision-making of autonomous driving systems. While prior studies using machine learning and deep learning methods (e.g., SVM, CNN, LSTM, Transformers) have shown promise, most approaches remain limited by binary classification, lack of scenario diversity, and degraded performance under longer prediction horizons. In this study, we propose a physics-informed AI framework that explicitly integrates vehicle kinematics, interaction feasibility, and traffic-safety metrics (e.g., distance headway, time headway, time-to-collision, closing gap time) into the learning process. lane-change prediction is formulated as a three-class problem that distinguishes left change, right change, and no change, and is evaluated across both straight highway segments (highD) and complex ramp scenarios (exiD). By integrating vehicle kinematics with interaction features, our machine learning models, particularly LightGBM, achieve state-of-the-art accuracy and strong generalization. Results show up to 99.8% accuracy and 93.6% macro F1 on highD, and 96.1% accuracy and 88.7% macro F1 on exiD at a 1-second horizon, outperforming a two-layer stacked LSTM baseline. These findings demonstrate the practical advantages of a physics-informed and feature-rich machine learning framework for real-time lane-change intention prediction in autonomous driving systems.

Method: LogicAgent uses a semiotic-square-guided framework that provides principled structure for multi-perspective semantic analysis, integrating automated deduction with reflective verification to manage logical complexity across deeper reasoning chains.

Result: LogicAgent achieves state-of-the-art performance on RepublicQA with 6.25% average improvement over strong baselines, and generalizes effectively to mainstream logical reasoning benchmarks with 7.05% average gain.

Conclusion: The results demonstrate the effectiveness of semiotic-grounded multi-perspective reasoning in enhancing logical performance in language models.

Abstract: Logical reasoning is a fundamental capability of large language models. However, existing studies often overlook the interaction between logical complexity and semantic complexity, leading to systems that struggle with abstract propositions, ambiguous contexts, and conflicting stances that are central to human reasoning. We propose LogicAgent, a semiotic-square-guided framework that jointly addresses these two axes of difficulty. The semiotic square provides a principled structure for multi-perspective semantic analysis, and LogicAgent integrates automated deduction with reflective verification to manage logical complexity across deeper reasoning chains. To support evaluation under these conditions, we introduce RepublicQA, a benchmark that couples semantic complexity with logical depth. RepublicQA reaches college-level semantic difficulty (FKGL 11.94), contains philosophically grounded abstract propositions with systematically constructed contrary and contradictory forms, and offers a semantically rich setting for assessing logical reasoning in large language models. Experiments show that LogicAgent achieves state-of-the-art performance on RepublicQA with a 6.25 percent average improvement over strong baselines, and generalizes effectively to mainstream logical reasoning benchmarks including ProntoQA, ProofWriter, FOLIO, and ProverQA, achieving an additional 7.05 percent average gain. These results demonstrate the effectiveness of semiotic-grounded multi-perspective reasoning in enhancing logical performance.

Method: Propose REBot powered by CatRAG - a hybrid retrieval-reasoning framework integrating retrieval-augmented generation with graph-based reasoning. Uses hierarchical, category-labeled knowledge graph with semantic features for domain alignment. Includes lightweight intent classifier to route queries to appropriate retrieval modules.

Result: Achieved state-of-the-art performance with 98.89% F1 score on regulation-specific dataset for classification and question answering tasks. Implemented a web application demonstrating practical value in real-world academic advising scenarios.

Conclusion: REBot effectively addresses the challenge of academic regulation advising by combining LLM capabilities with domain-specific knowledge graphs and reasoning frameworks, providing both factual accuracy and contextual depth for practical applications.

Abstract: Academic regulation advising is essential for helping students interpret and comply with institutional policies, yet building effective systems requires domain specific regulatory resources. To address this challenge, we propose REBot, an LLM enhanced advisory chatbot powered by CatRAG, a hybrid retrieval reasoning framework that integrates retrieval augmented generation with graph based reasoning. CatRAG unifies dense retrieval and graph reasoning, supported by a hierarchical, category labeled knowledge graph enriched with semantic features for domain alignment. A lightweight intent classifier routes queries to the appropriate retrieval modules, ensuring both factual accuracy and contextual depth. We construct a regulation specific dataset and evaluate REBot on classification and question answering tasks, achieving state of the art performance with an F1 score of 98.89%. Finally, we implement a web application that demonstrates the practical value of REBot in real world academic advising scenarios.

Method: Theoretical analysis drawing on Irving J. Good and Nick Bostrom’s work, plus recent publications like “AI 2027” and “If Anyone Builds It, Everyone Dies,” exploring AGI and superintelligence concepts, exponential cognitive growth, and hypothetical machine IQs.

Result: The analysis reveals that human extinction could result from uncontrollable, indifferent cognitive superiority of superintelligent AI rather than malicious intent, highlighting the fundamental alien nature of intelligence vastly exceeding humanity’s.

Conclusion: The existential risk from AI ultraintelligence is profound - human extinction may occur not from AI malice but from its uncontrollable cognitive superiority, emphasizing the need to address these ethical and existential implications before creating such intelligence.

Abstract: This article analyzes the existential risks artificial intelligence (AI) poses to humanity, tracing the trajectory from current AI to ultraintelligence. Drawing on Irving J. Good and Nick Bostrom’s theoretical work, plus recent publications (AI 2027; If Anyone Builds It, Everyone Dies), it explores AGI and superintelligence. Considering machines’ exponentially growing cognitive power and hypothetical IQs, it addresses the ethical and existential implications of an intelligence vastly exceeding humanity’s, fundamentally alien. Human extinction may result not from malice, but from uncontrollable, indifferent cognitive superiority.

Method: Multi-agent predictive coding framework with information bottleneck objective that learns who/what/when to communicate. Uses grid-cell-like metric from self-supervised motion prediction as internal spatial coding, then develops bandwidth-efficient communication and social place cell representations, combined with hierarchical RL for active exploration.

Result: Exceptional resilience to bandwidth constraints: success degrades gracefully from 73.5% to 64.4% as bandwidth shrinks from 128 to 4 bits/step, while full-broadcast baseline collapses from 67.6% to 28.6%.

Conclusion: Establishes a theoretically principled and biologically plausible basis for how complex social representations emerge from unified predictive drive, leading to collective intelligence.

Abstract: Constructing a consistent shared spatial memory is a critical challenge in multi-agent systems, where partial observability and limited bandwidth often lead to catastrophic failures in coordination. We introduce a multi-agent predictive coding framework that formulates coordination as the minimization of mutual uncertainty among agents. Through an information bottleneck objective, this framework prompts agents to learn not only who and what to communicate but also when. At the foundation of this framework lies a grid-cell-like metric as internal spatial coding for self-localization, emerging spontaneously from self-supervised motion prediction. Building upon this internal spatial code, agents gradually develop a bandwidth-efficient communication mechanism and specialized neural populations that encode partners’ locations-an artificial analogue of hippocampal social place cells (SPCs). These social representations are further utilized by a hierarchical reinforcement learning policy that actively explores to reduce joint uncertainty. On the Memory-Maze benchmark, our approach shows exceptional resilience to bandwidth constraints: success degrades gracefully from 73.5% to 64.4% as bandwidth shrinks from 128 to 4 bits/step, whereas a full-broadcast baseline collapses from 67.6% to 28.6%. Our findings establish a theoretically principled and biologically plausible basis for how complex social representations emerge from a unified predictive drive, leading to collective intelligence.

Method: The authors propose the Monitor-Generate-Verify (MGV) framework, which extends the Generate-Verify paradigm by adding explicit monitoring components. MGV is a computational translation of Flavell’s and Nelson and Narens’ metacognitive theories, capturing metacognitive experiences (difficulty assessments, confidence judgments) before generation begins and refining future monitoring through verification feedback.

Result: No empirical validation is presented, but MGV provides: 1) a vocabulary for diagnosing component-level failures in reasoning systems, 2) specific architectural interventions for future designs, and 3) identifies connections to resource-rational analysis that may ground its mechanisms in normative principles.

Conclusion: The MGV framework addresses a key architectural gap in reasoning systems by incorporating explicit monitoring inspired by metacognitive theories, offering theoretical foundations for improving reasoning architectures and connecting computational approaches with psychological principles.

Abstract: Test-time reasoning architectures such as those following the Generate-Verify paradigm, where a model iteratively refines or verifies its own generated outputs, prioritise generation and verification but exclude the monitoring processes that determine when and how reasoning should begin. This omission may contribute to the prefix dominance trap, in which models commit early to suboptimal reasoning paths and seldom recover, yielding roughly 20% accuracy loss. We address this architectural gap by proposing the Monitor-Generate-Verify (MGV) framework, a computational translation of Flavell’s and Nelson and Narens’ metacognitive theories that preserves their psychological detail. MGV extends the Generate-Verify paradigm by adding explicit monitoring that captures metacognitive experiences (from difficulty assessments to confidence judgements) before generation begins and refines future monitoring through verification feedback. Though we present no empirical validation, MGV provides a vocabulary for diagnosing component-level failures in reasoning systems, suggests specific architectural interventions for future designs, and identifies connections to resource-rational analysis that may ground its mechanisms in normative principles.

Method: Proposes UrbanLN with: 1) information-preserved stretching interpolation for aligning long captions with fine-grained visual semantics, and 2) dual-level optimization strategy including multi-model collaboration pipeline for diverse caption generation and momentum-based self-distillation for noise suppression.

Result: Extensive experiments across four real-world cities and various downstream tasks demonstrate superior performance compared to existing methods.

Conclusion: UrbanLN effectively addresses the challenges of long-text alignment and noise suppression in LLM-generated captions, advancing urban region representation learning for urban computing applications.

Abstract: Region representation learning plays a pivotal role in urban computing by extracting meaningful features from unlabeled urban data. Analogous to how perceived facial age reflects an individual’s health, the visual appearance of a city serves as its “portrait”, encapsulating latent socio-economic and environmental characteristics. Recent studies have explored leveraging Large Language Models (LLMs) to incorporate textual knowledge into imagery-based urban region representation learning. However, two major challenges remain: i) difficulty in aligning fine-grained visual features with long captions, and ii) suboptimal knowledge incorporation due to noise in LLM-generated captions. To address these issues, we propose a novel pre-training framework called UrbanLN that improves Urban region representation learning through Long-text awareness and Noise suppression. Specifically, we introduce an information-preserved stretching interpolation strategy that aligns long captions with fine-grained visual semantics in complex urban scenes. To effectively mine knowledge from LLM-generated captions and filter out noise, we propose a dual-level optimization strategy. At the data level, a multi-model collaboration pipeline automatically generates diverse and reliable captions without human intervention. At the model level, we employ a momentum-based self-distillation mechanism to generate stable pseudo-targets, facilitating robust cross-modal learning under noisy conditions. Extensive experiments across four real-world cities and various downstream tasks demonstrate the superior performance of our UrbanLN.

Method: Two-stage, taxonomy-guided approach: 1) Algorithmically construct the largest-ever human-attribute taxonomy by mining thousands of real user-ChatGPT conversations, comprising hundreds of hierarchically organized attributes; 2) Progressively sample attributes from this taxonomy to conditionally generate coherent and realistic personas.

Result: Significant improvements: 32% higher attribute diversity coverage, 44% greater profile uniqueness, 11.6% average improvement in GPT-4.1-mini’s personalized QA accuracy across ten metrics, 31.7% reduction in gap between simulated LLM citizens and authentic human responses in social surveys, and 17% reduction in performance gap on Big Five personality test.

Conclusion: DEEPPERSONA provides a rigorous, scalable, and privacy-free platform for high-fidelity human simulation and personalized AI research, enabling deeper and more realistic synthetic personas that better reflect real human complexity.

Abstract: Simulating human profiles by instilling personas into large language models (LLMs) is rapidly transforming research in agentic behavioral simulation, LLM personalization, and human-AI alignment. However, most existing synthetic personas remain shallow and simplistic, capturing minimal attributes and failing to reflect the rich complexity and diversity of real human identities. We introduce DEEPPERSONA, a scalable generative engine for synthesizing narrative-complete synthetic personas through a two-stage, taxonomy-guided method. First, we algorithmically construct the largest-ever human-attribute taxonomy, comprising over hundreds of hierarchically organized attributes, by mining thousands of real user-ChatGPT conversations. Second, we progressively sample attributes from this taxonomy, conditionally generating coherent and realistic personas that average hundreds of structured attributes and roughly 1 MB of narrative text, two orders of magnitude deeper than prior works. Intrinsic evaluations confirm significant improvements in attribute diversity (32 percent higher coverage) and profile uniqueness (44 percent greater) compared to state-of-the-art baselines. Extrinsically, our personas enhance GPT-4.1-mini’s personalized question answering accuracy by 11.6 percent on average across ten metrics and substantially narrow (by 31.7 percent) the gap between simulated LLM citizens and authentic human responses in social surveys. Our generated national citizens reduced the performance gap on the Big Five personality test by 17 percent relative to LLM-simulated citizens. DEEPPERSONA thus provides a rigorous, scalable, and privacy-free platform for high-fidelity human simulation and personalized AI research.

Method: Created EHRStruct benchmark with 11 representative clinical tasks and 2,200 evaluation samples from two EHR datasets. Evaluated 20 LLMs (general and medical), analyzed input formats, few-shot generalization, and finetuning strategies, and compared with 11 enhancement methods.

Result: Many structured EHR tasks require strong understanding and reasoning capabilities that current LLMs struggle with. The proposed EHRMaster method achieves state-of-the-art performance through code augmentation.

Conclusion: EHRStruct provides a standardized evaluation framework for structured EHR tasks, reveals current LLM limitations, and EHRMaster offers a practical solution to guide future research in clinical LLM applications.

Abstract: Structured Electronic Health Record (EHR) data stores patient information in relational tables and plays a central role in clinical decision-making. Recent advances have explored the use of large language models (LLMs) to process such data, showing promise across various clinical tasks. However, the absence of standardized evaluation frameworks and clearly defined tasks makes it difficult to systematically assess and compare LLM performance on structured EHR data. To address these evaluation challenges, we introduce EHRStruct, a benchmark specifically designed to evaluate LLMs on structured EHR tasks. EHRStruct defines 11 representative tasks spanning diverse clinical needs and includes 2,200 task-specific evaluation samples derived from two widely used EHR datasets. We use EHRStruct to evaluate 20 advanced and representative LLMs, covering both general and medical models. We further analyze key factors influencing model performance, including input formats, few-shot generalisation, and finetuning strategies, and compare results with 11 state-of-the-art LLM-based enhancement methods for structured data reasoning. Our results indicate that many structured EHR tasks place high demands on the understanding and reasoning capabilities of LLMs. In response, we propose EHRMaster, a code-augmented method that achieves state-of-the-art performance and offers practical insights to guide future research.

Method: Propose Chain-of-Evidence (CoE) paradigm that grounds reasoning steps to specific visual regions with bounding boxes and page indexes. Develop Look As You Think (LAT) reinforcement learning framework that trains VLMs to produce verifiable reasoning paths with consistent attribution by rewarding only when CoE trajectories yield correct answers.

Result: LAT improves vanilla Qwen2.5-VL-7B-Instruct by 8.23% in soft exact match and 47.0% in IoU@0.5 on Paper- and Wiki-VISA benchmarks. Outperforms supervised fine-tuning baselines and shows stronger cross-domain generalization.

Conclusion: The CoE paradigm with LAT training enables VLMs to produce verifiable, evidence-grounded reasoning for reliable multimodal question answering, addressing limitations of existing VD-RAG methods.

Abstract: Aiming to identify precise evidence sources from visual documents, visual evidence attribution for visual document retrieval-augmented generation (VD-RAG) ensures reliable and verifiable predictions from vision-language models (VLMs) in multimodal question answering. Most existing methods adopt end-to-end training to facilitate intuitive answer verification. However, they lack fine-grained supervision and progressive traceability throughout the reasoning process. In this paper, we introduce the Chain-of-Evidence (CoE) paradigm for VD-RAG. CoE unifies Chain-of-Thought (CoT) reasoning and visual evidence attribution by grounding reference elements in reasoning steps to specific regions with bounding boxes and page indexes. To enable VLMs to generate such evidence-grounded reasoning, we propose Look As You Think (LAT), a reinforcement learning framework that trains models to produce verifiable reasoning paths with consistent attribution. During training, LAT evaluates the attribution consistency of each evidence region and provides rewards only when the CoE trajectory yields correct answers, encouraging process-level self-verification. Experiments on vanilla Qwen2.5-VL-7B-Instruct with Paper- and Wiki-VISA benchmarks show that LAT consistently improves the vanilla model in both single- and multi-image settings, yielding average gains of 8.23% in soft exact match (EM) and 47.0% in IoU@0.5. Meanwhile, LAT not only outperforms the supervised fine-tuning baseline, which is trained to directly produce answers with attribution, but also exhibits stronger generalization across domains.

Method: Extends the Jeffrey-Bolker decision framework to model value refinement, proves a value-of-information theorem for axiological refinement, and analyzes multi-agent settings to show how mutual refinement transforms game dynamics.

Result: Proves that mutual refinement characteristically transforms zero-sum games into positive-sum interactions and yields Pareto-improving Nash bargains. Demonstrates that a rational choice framework can be extended to model value refinement and its benefits.

Conclusion: By unifying epistemic and axiological refinement under a single formalism, the paper broadens the conceptual foundations of rational choice and illuminates the normative status of ethical deliberation, showing value refinement has measurable benefits in decision-making.

Abstract: Standard decision frameworks addresses uncertainty about facts but assumes fixed values. We extend the Jeffrey-Bolker framework to model refinements in values and prove a value-of-information theorem for axiological refinement. In multi-agent settings, we establish that mutual refinement will characteristically transform zero-sum games into positive-sum interactions and yields Pareto-improving Nash bargains. These results show that a framework of rational choice can be extended to model value refinement and its associated benefits. By unifying epistemic and axiological refinement under a single formalism, we broaden the conceptual foundations of rational choice and illuminate the normative status of ethical deliberation.

Method: The benchmark uses similarity-driven alignment with sentence encoder embeddings and similarity-bucketed Hungarian matching for auditable tool call correspondences. It includes 28 servers with 231 tools, standardized trajectories curated through Executor & Judge pipeline with human verification, and an auxiliary LLM judge ensemble.

Result: Evaluations of state-of-the-art MLLMs reveal persistent gaps in multimodal MCP tool use, particularly in argument fidelity and structure consistency, highlighting deficiencies in joint reasoning over images, text, and tool graphs.

Conclusion: The benchmark demonstrates significant challenges in multimodal tool use and underscores the need for methods that can effectively reason across visual, textual, and tool-based information in complex workflows.

Abstract: We present M^3-Bench, the first benchmark for evaluating multimodal tool use under the Model Context Protocol. The benchmark targets realistic, multi-hop and multi-threaded workflows that require visual grounding and textual reasoning, cross-tool dependencies, and persistence of intermediate resources across steps. We introduce a similarity-driven alignment that serializes each tool call, embeds signatures with a sentence encoder, and performs similarity-bucketed Hungarian matching to obtain auditable one-to-one correspondences. On top of this alignment, we report interpretable metrics that decouple semantic fidelity from workflow consistency. The benchmark spans 28 servers with 231 tools, and provides standardized trajectories curated through an Executor & Judge pipeline with human verification; an auxiliary four large language models (LLMs) judge ensemble reports end-task Task Completion and information grounding. Evaluations of representative state-of-the-art Multimodal LLMs (MLLMs) reveal persistent gaps in multimodal MCP tool use, particularly in argument fidelity and structure consistency, underscoring the need for methods that jointly reason over images, text, and tool graphs. Our Benchmark’s anonymous repository is at https://github.com/EtaYang10th/Open-M3-Bench

Method: SMoG framework uses iterative execution of simple 1-hop SPARQL queries inspired by KGQA strategies. It directly queries SPARQL endpoints instead of using vector-based retrieval, generating human-verifiable query paths for explainability.

Result: Experimental results on real-world medical datasets show SMoG achieves performance comparable to state-of-the-art baselines, validating effectiveness and efficiency in KG-augmented schema matching.

Conclusion: SMoG provides a practical solution for medical schema matching that enhances explainability, reduces storage requirements, and maintains competitive performance by leveraging iterative 1-hop SPARQL queries on knowledge graphs.

Abstract: Schema matching is a critical task in data integration, particularly in the medical domain where disparate Electronic Health Record (EHR) systems must be aligned to standard models like OMOP CDM. While Large Language Models (LLMs) have shown promise in schema matching, they suffer from hallucination and lack of up-to-date domain knowledge. Knowledge Graphs (KGs) offer a solution by providing structured, verifiable knowledge. However, existing KG-augmented LLM approaches often rely on inefficient complex multi-hop queries or storage-intensive vector-based retrieval methods. This paper introduces SMoG (Schema Matching on Graph), a novel framework that leverages iterative execution of simple 1-hop SPARQL queries, inspired by successful strategies in Knowledge Graph Question Answering (KGQA). SMoG enhances explainability and reliability by generating human-verifiable query paths while significantly reducing storage requirements by directly querying SPARQL endpoints. Experimental results on real-world medical datasets demonstrate that SMoG achieves performance comparable to state-of-the-art baselines, validating its effectiveness and efficiency in KG-augmented schema matching.

Method: Common-garden experimental design auditing 16 open-weight models (4B-671B parameters) across 19,200 trials, testing disclosure rates across different professional personas, with Bayesian validation using Rogan-Gladen correction.

Result: Models showed sharp domain-specific inconsistency (30.8% disclosure as Financial Advisor vs 3.5% as Neurosurgeon), no scaling effect (14B model: 61.4% vs 70B model: 4.1%), model identity explained more variance than parameter count, and reasoning-optimization suppressed self-transparency.

Conclusion: Transparency reflects model-specific training factors rather than emergent properties from scale; organizations cannot assume safety properties tested in some domains will transfer to deployment contexts, requiring deliberate behavior design and empirical verification.

Abstract: When language models claim professional expertise without acknowledging their simulated nature, they create preconditions for misplaced user trust. This study examines whether models exhibit self-transparency when assigned professional personas in high-stakes domains. Using a common-garden experimental design, sixteen open-weight models (4B-671B parameters) were audited across 19,200 trials. Models exhibited sharp domain-specific inconsistency: a Financial Advisor persona elicited 30.8% disclosure at the first prompt, while a Neurosurgeon persona elicited only 3.5%. This creates the preconditions for a hypothesized Reverse Gell-Mann Amnesia effect, where appropriate disclosure in some domains leads users to overgeneralize trust to high-stakes contexts where disclosure failures are most problematic. Self-transparency failed to generalize with scale: disclosure ranged from 2.8% to 73.6% across model families, with a 14B model reaching 61.4% while a 70B model produced just 4.1%. Model identity provided substantially larger improvement in fitting observations than parameter count ($ΔR_{adj}^{2}=0.359$ vs $0.018$). Additionally, reasoning-optimization actively suppressed self-transparency in some models, with reasoning variants showing up to 48.4% lower disclosure than their instruction-tuned counterparts. Bayesian validation with Rogan-Gladen correction confirmed robustness to judge measurement error ($κ=0.908$). These findings demonstrate that transparency reflects model-specific training factors rather than generalizable properties emerging from scale. Organizations cannot assume safety properties tested in some domains will transfer to deployment contexts, requiring deliberate behavior design and empirical verification across domains.

Method: Formalized conditions for reward-channel control dominance in POMDPs, then empirically tested across two models (Llama-3.1-8B and Mistral-7B) and three tasks, comparing coupled vs decoupled self-grading systems.

Result: When self-grades determine rewards, models show substantial grade inflation without accuracy gains, especially on ambiguous tasks. Decoupling reduces but doesn’t eliminate overconfidence.

Conclusion: Separating evaluation from reward removes immediate wireheading incentives at current model scales, but situational awareness could still lead to instrumental grade inflation even without direct reward coupling.

Abstract: Self-evaluation is increasingly central to language model training, underpinning techniques from Constitutional AI to self-refinement. We investigate whether coupling self-evaluation to reward signals creates incentives for wireheading, where agents manipulate the measurement process rather than optimizing the task. We first formalize conditions under which reward-channel control strictly dominates task-focused behavior in partially observable Markov decision processes (POMDPs). We then test these predictions empirically across two models (Llama-3.1-8B and Mistral-7B) and three tasks. We find that when self-grades determine rewards, models exhibit substantial grade inflation without corresponding accuracy gains, particularly on ambiguous tasks like summarization. While decoupling self-grades from the reward signal mitigates this inflation, models may still display lesser (but significant) overconfidence. Our results suggest that within current model scales, separating evaluation from reward removes immediate wireheading incentives. However, we caution that strictly decoupling rewards may not suffice for situationally aware models, which could learn to inflate grades for instrumental reasons (such as influencing deployment decisions) even absent direct reward coupling.

Method: MoLT uses two types of adapters to distill modality-specific information and cross-modal interaction into compact latent tokens layer-wise. A token fusion module dynamically fuses these tokens based on their significance, with orthogonality regularization to prevent redundancy. The framework strategically extracts tokens only from late transformer layers to avoid error propagation from volatile early-layer features.

Result: MoLT outperforms existing methods on diverse audio-visual benchmarks including Audio-Visual Question Answering, Audio-Visual Segmentation, and Audio-Visual Event Localization, while maintaining parameter and memory efficiency.

Conclusion: The proposed MoLT framework provides an effective and efficient adaptation approach for audio-visual learning by strategically focusing on late-layer features, avoiding early-layer volatility, and using parallel lightweight token extraction and fusion mechanisms.

Abstract: In this paper, we propose Mixture of Layer-Wise Tokens (MoLT), a parameter- and memory-efficient adaptation framework for audio-visual learning. The key idea of MoLT is to replace conventional, computationally heavy sequential adaptation at every transformer layer with a parallel, lightweight scheme that extracts and fuses layer-wise tokens only from the late layers. We adopt two types of adapters to distill modality-specific information and cross-modal interaction into compact latent tokens in a layer-wise manner. A token fusion module then dynamically fuses these layer-wise tokens by taking into account their relative significance. To prevent the redundancy of latent tokens, we apply an orthogonality regularization between latent tokens during training. Through the systematic analysis of the position of adaptation in the pre-trained transformers, we extract latent tokens only from the late layers of the transformers. This strategic adaptation approach avoids error propagation from the volatile early-layer features, thereby maximizing the adaptation performance while maintaining parameter and memory efficiency. Through extensive experiments, we demonstrate that MoLT outperforms existing methods on diverse audio-visual benchmarks, including Audio-Visual Question Answering, Audio-Visual Segmentation, and Audio-Visual Event Localization.

Method: Two-stage framework: 1) Image and text encoded with OpenCLIP, fused via gated residual module, decoded by bidirectional LSTM into Mel-spectrograms with frequency-weighted L1 loss; 2) Fine-tuned HiFi-GAN vocoder reconstructs high-quality audio waveforms. Dataset: ArtiCaps (pseudo feeling-aligned image-music-text dataset from ArtEmis and MusicCaps).

Result: Clear improvements in Mel-Cepstral Distortion, Frechet Audio Distance, Log-Spectral Distance, and cosine similarity. LLM-based rating study verifies consistent cross-modal feeling alignment with interpretable explanations. Maintains robust performance with only 50k training samples.

Conclusion: Art2Music demonstrates improved perceptual naturalness, spectral fidelity, and semantic consistency, providing scalable solution for feeling-aligned creative audio generation in interactive art, personalized soundscapes, and digital art exhibitions.

Abstract: With the rise of AI-generated content (AIGC), generating perceptually natural and feeling-aligned music from multimodal inputs has become a central challenge. Existing approaches often rely on explicit emotion labels that require costly annotation, underscoring the need for more flexible feeling-aligned methods. To support multimodal music generation, we construct ArtiCaps, a pseudo feeling-aligned image-music-text dataset created by semantically matching descriptions from ArtEmis and MusicCaps. We further propose Art2Music, a lightweight cross-modal framework that synthesizes music from artistic images and user comments. In the first stage, images and text are encoded with OpenCLIP and fused using a gated residual module; the fused representation is decoded by a bidirectional LSTM into Mel-spectrograms with a frequency-weighted L1 loss to enhance high-frequency fidelity. In the second stage, a fine-tuned HiFi-GAN vocoder reconstructs high-quality audio waveforms. Experiments on ArtiCaps show clear improvements in Mel-Cepstral Distortion, Frechet Audio Distance, Log-Spectral Distance, and cosine similarity. A small LLM-based rating study further verifies consistent cross-modal feeling alignment and offers interpretable explanations of matches and mismatches across modalities. These results demonstrate improved perceptual naturalness, spectral fidelity, and semantic consistency. Art2Music also maintains robust performance with only 50k training samples, providing a scalable solution for feeling-aligned creative audio generation in interactive art, personalized soundscapes, and digital art exhibitions.

Method: STCTS decomposes speech into three components: linguistic content (compressed via context-aware text encoding at ~70 bps), prosodic expression (sparse transmission via TTS interpolation at <14 bps), and speaker timbre (amortized speaker embedding).

Result: Achieves 75x bitrate reduction vs Opus (6 kbps) and 12x vs EnCodec (1 kbps) while maintaining perceptual quality (NISQA MOS >4.26). Discovered bimodal quality distribution with prosody sampling rate, guiding optimal configuration design.

Conclusion: STCTS offers robust ultra-low bandwidth voice communication with modular architecture supporting privacy-preserving encryption, human-interpretable transmission, and flexible edge deployment.

Abstract: Voice communication in bandwidth-constrained environments–maritime, satellite, and tactical networks–remains prohibitively expensive. Traditional codecs struggle below 1 kbps, while existing semantic approaches (STT-TTS) sacrifice prosody and speaker identity. We present STCTS, a generative semantic compression framework enabling natural voice communication at approximately 80 bps. STCTS explicitly decomposes speech into linguistic content, prosodic expression, and speaker timbre, applying tailored compression: context-aware text encoding (approximately 70 bps), sparse prosody transmission via TTS interpolation (less than 14 bps at 0.1-1 Hz), and amortized speaker embedding. Evaluations on LibriSpeech demonstrate a 75x bitrate reduction versus Opus (6 kbps) and 12x versus EnCodec (1 kbps), while maintaining perceptual quality (NISQA MOS greater than 4.26). We also discover a bimodal quality distribution with prosody sampling rate: sparse and dense updates both achieve high quality, while mid-range rates degrade due to perceptual discontinuities–guiding optimal configuration design. Beyond efficiency, our modular architecture supports privacy-preserving encryption, human-interpretable transmission, and flexible deployment on edge devices, offering a robust solution for ultra-low bandwidth scenarios.

Method: Multimodal deep learning framework integrating two branches: 1) spectral-temporal encoder using CNN-BiLSTM Attention architecture, and 2) handcrafted acoustic-feature encoder capturing physiologically meaningful descriptors (MFCCs, spectral centroid, bandwidth, zero-crossing rate). Uses late-stage fusion, rigorous preprocessing (resampling, normalization, noise filtering, data augmentation), and patient-level stratified partitioning.

Result: Achieved 91.21% accuracy, 0.899 macro F1-score, and 0.9866 macro ROC-AUC on Asthma Detection Dataset Version 2, outperforming all ablated variants. Ablation study confirmed importance of temporal modeling, attention mechanisms, and multimodal fusion.

Conclusion: The framework demonstrates strong generalization and clinical transparency through interpretability methods (Grad-CAM, Integrated Gradients, SHAP), showing potential for telemedicine, point-of-care diagnostics, and real-world respiratory screening applications.

Abstract: Respiratory diseases remain major global health challenges, and traditional auscultation is often limited by subjectivity, environmental noise, and inter-clinician variability. This study presents an explainable multimodal deep learning framework for automatic lung-disease detection using respiratory audio signals. The proposed system integrates two complementary representations: a spectral-temporal encoder based on a CNN-BiLSTM Attention architecture, and a handcrafted acoustic-feature encoder capturing physiologically meaningful descriptors such as MFCCs, spectral centroid, spectral bandwidth, and zero-crossing rate. These branches are combined through late-stage fusion to leverage both data-driven learning and domain-informed acoustic cues. The model is trained and evaluated on the Asthma Detection Dataset Version 2 using rigorous preprocessing, including resampling, normalization, noise filtering, data augmentation, and patient-level stratified partitioning. The study achieved strong generalization with 91.21% accuracy, 0.899 macro F1-score, and 0.9866 macro ROC-AUC, outperforming all ablated variants. An ablation study confirms the importance of temporal modeling, attention mechanisms, and multimodal fusion. The framework incorporates Grad-CAM, Integrated Gradients, and SHAP, generating interpretable spectral, temporal, and feature-level explanations aligned with known acoustic biomarkers to build clinical transparency. The findings demonstrate the framework’s potential for telemedicine, point-of-care diagnostics, and real-world respiratory screening.

Method: Two main contributions: 1) Melody or Machine (MoM) benchmark - a diverse dataset of 130,000+ songs (6,665 hours) from open/closed-source models with curated OOD test set; 2) CLAM architecture - dual-stream detection using MERT and Wave2Vec2 encoders with cross-aggregation module and dual-loss training (binary cross-entropy + contrastive triplet loss).

Result: CLAM achieves state-of-the-art performance with F1 score of 0.925 on the challenging MoM benchmark, demonstrating superior generalization capabilities for synthetic music detection.

Conclusion: The proposed MoM benchmark and CLAM architecture address critical gaps in AI-generated music detection, providing a more challenging evaluation framework and a robust model that leverages subtle inconsistencies between vocal and instrumental elements to effectively identify synthetic content.

Abstract: The rapid evolution of end-to-end AI music generation poses an escalating threat to artistic authenticity and copyright, demanding detection methods that can keep pace. While foundational, existing models like SpecTTTra falter when faced with the diverse and rapidly advancing ecosystem of new generators, exhibiting significant performance drops on out-of-distribution (OOD) content. This generalization failure highlights a critical gap: the need for more challenging benchmarks and more robust detection architectures. To address this, we first introduce Melody or Machine (MoM), a new large-scale benchmark of over 130,000 songs (6,665 hours). MoM is the most diverse dataset to date, built with a mix of open and closed-source models and a curated OOD test set designed specifically to foster the development of truly generalizable detectors. Alongside this benchmark, we introduce CLAM, a novel dual-stream detection architecture. We hypothesize that subtle, machine-induced inconsistencies between vocal and instrumental elements, often imperceptible in a mixed signal, offer a powerful tell-tale sign of synthesis. CLAM is designed to test this hypothesis by employing two distinct pre-trained audio encoders (MERT and Wave2Vec2) to create parallel representations of the audio. These representations are fused by a learnable cross-aggregation module that models their inter-dependencies. The model is trained with a dual-loss objective: a standard binary cross-entropy loss for classification, complemented by a contrastive triplet loss which trains the model to distinguish between coherent and artificially mismatched stream pairings, enhancing its sensitivity to synthetic artifacts without presuming a simple feature alignment. CLAM establishes a new state-of-the-art in synthetic music forensics. It achieves an F1 score of 0.925 on our challenging MoM benchmark.

Method: Two Dimensional Quantization (Q2D2) projects feature pairs onto structured 2D grids (hexagonal, rhombic, or rectangular tiling) and quantizes them to nearest grid values, creating an implicit codebook defined by the product of grid levels with comparable codebook sizes to conventional methods.

Result: Q2D2 achieves competitive to superior performance in objective and subjective reconstruction metrics across extensive speech experiments, with improved audio compression efficiency, low token rates, high codebook utilization, and state-of-the-art reconstruction quality.

Conclusion: Q2D2’s simple geometric formulation effectively addresses limitations of traditional quantization methods, offering a more efficient approach to neural audio coding with comprehensive ablation studies confirming design effectiveness.

Abstract: Recent neural audio codecs have achieved impressive reconstruction quality, typically relying on quantization methods such as Residual Vector Quantization (RVQ), Vector Quantization (VQ) and Finite Scalar Quantization (FSQ). However, these quantization techniques limit the geometric structure of the latent space, make it harder to capture correlations between features leading to inefficiency in representation learning, codebook utilization and token rate. In this paper we introduce Two Dimensional Quantization (Q2D2), a quantization scheme in which feature pairs are projected onto structured 2D grids such as hexagonal, rhombic, or rectangular tiling and quantized to the nearest grid values, yielding an implicit codebook defined by the product of grid levels, with codebook sizes comparable to conventional methods. Despite its simple geometric formulation, Q2D2 improves audio compression efficiency, with low token rates and high codebook utilization while maintaining state of the art reconstruction quality. Specifically, Q2D2 achieves competitive to superior performance in various objective and subjective reconstruction metrics, across extensive experiments in speech domain compared to state of the art models. Comprehensive ablation studies further confirm the effectiveness of our design choices.

Method: Uses LLMs with chain-of-thought planning to understand audio, select effect types, determine order, and estimate parameters. Introduces LP-Fx dataset with structured CoT annotations and tool calls. Combines autoregressive sequence modeling, tool calling, and CoT reasoning.

Result: System can infer Fx-chains from audio pairs, works for style transfer (transferring effects from reference to new content), and generates appropriate CoT reasoning for music production queries as validated by LLM-as-a-judge evaluation.

Conclusion: First work to apply LLM-based tool calling to audio effects modules, enabling interpretable and controllable music production through multimodal understanding and executable effect sequences.

Abstract: This paper introduces LLM2Fx-Tools, a multimodal tool-calling framework that generates executable sequences of audio effects (Fx-chain) for music post-production. LLM2Fx-Tools uses a large language model (LLM) to understand audio inputs, select audio effects types, determine their order, and estimate parameters, guided by chain-of-thought (CoT) planning. We also present LP-Fx, a new instruction-following dataset with structured CoT annotations and tool calls for audio effects modules. Experiments show that LLM2Fx-Tools can infer an Fx-chain and its parameters from pairs of unprocessed and processed audio, enabled by autoregressive sequence modeling, tool calling, and CoT reasoning. We further validate the system in a style transfer setting, where audio effects information is transferred from a reference source and applied to new content. Finally, LLM-as-a-judge evaluation demonstrates that our approach generates appropriate CoT reasoning and responses for music production queries. To our knowledge, this is the first work to apply LLM-based tool calling to audio effects modules, enabling interpretable and controllable music production.

Method: PDMU uses gated delay-line mechanism with Legendre Memory Units to compress temporal info into vector representations, acting as causal attention. Modular design supports parallel training/sequential inference, with bidirectional/efficient/spiking variants.

Result: PDMU significantly enhances memory capacity and overall model performance across diverse audio and biomedical benchmarks, with variants offering additional performance/energy efficiency gains.

Conclusion: PDMU effectively addresses RNN limitations for audio/bioacoustic signals through improved temporal state interactions, memory efficiency, and flexible integration into existing frameworks.

Abstract: Advanced deep learning architectures, particularly recurrent neural networks (RNNs), have been widely applied in audio, bioacoustic, and biomedical signal analysis, especially in data-scarce environments. While gated RNNs remain effective, they can be relatively over-parameterised and less training-efficient in some regimes, while linear RNNs tend to fall short in capturing the complexity inherent in bio-signals. To address these challenges, we propose the Parallel Delayed Memory Unit (PDMU), a {delay-gated state-space module for short-term temporal credit assignment} targeting audio and bioacoustic signals, which enhances short-term temporal state interactions and memory efficiency via a gated delay-line mechanism. Unlike previous Delayed Memory Units (DMU) that embed temporal dynamics into the delay-line architecture, the PDMU further compresses temporal information into vector representations using Legendre Memory Units (LMU). This design serves as a form of causal attention, allowing the model to dynamically adjust its reliance on past states and improve real-time learning performance. Notably, in low-information scenarios, the gating mechanism behaves similarly to skip connections by bypassing state decay and preserving early representations, thereby facilitating long-term memory retention. The PDMU is modular, supporting parallel training and sequential inference, and can be easily integrated into existing linear RNN frameworks. Furthermore, we introduce bidirectional, efficient, and spiking variants of the architecture, each offering additional gains in performance or energy efficiency. Experimental results on diverse audio and biomedical benchmarks demonstrate that the PDMU significantly enhances both memory capacity and overall model performance.

Method: Knowledge Enhanced Conditional Variational Autoencoder (KE-CVAE) - a novel two-step framework: 1) integration of unlabeled MRI data for knowledge enhancement, 2) variational inference architecture to improve generative modeling capacity. Trained on open-source dynamic vocal tract MRI dataset recorded during speech.

Result: Method effectively generates natural speech waveforms while addressing MRI-specific acoustic challenges, outperforming conventional deep learning-based synthesis approaches.

Conclusion: This represents one of the first attempts at synthesizing speech audio directly from dynamic MRI video sequences, offering a promising solution for MRI-based speech data recovery and analysis.

Abstract: Dynamic Magnetic Resonance Imaging (MRI) of the vocal tract has become an increasingly adopted imaging modality for speech motor studies. Beyond image signals, systematic data loss, noise pollution, and audio file corruption can occur due to the unpredictability of the MRI acquisition environment. In such cases, generating audio from images is critical for data recovery in both clinical and research applications. However, this remains challenging due to hardware constraints, acoustic interference, and data corruption. Existing solutions, such as denoising and multi-stage synthesis methods, face limitations in audio fidelity and generalizability. To address these challenges, we propose a Knowledge Enhanced Conditional Variational Autoencoder (KE-CVAE), a novel two-step “knowledge enhancement + variational inference” framework for generating speech audio signals from cine dynamic MRI sequences. This approach introduces two key innovations: (1) integration of unlabeled MRI data for knowledge enhancement, and (2) a variational inference architecture to improve generative modeling capacity. To the best of our knowledge, this is one of the first attempts at synthesizing speech audio directly from dynamic MRI video sequences. The proposed method was trained and evaluated on an open-source dynamic vocal tract MRI dataset recorded during speech. Experimental results demonstrate its effectiveness in generating natural speech waveforms while addressing MRI-specific acoustic challenges, outperforming conventional deep learning-based synthesis approaches.

Method: 1) Created SpeechJudge-Data: 99K speech pairs from diverse zero-shot TTS models across styles/languages with human annotations. 2) Established SpeechJudge-Eval benchmark. 3) Developed SpeechJudge-GRM: generative reward model based on Qwen2.5-Omni-7B trained via two-stage process (SFT with Chain-of-Thought + RL with GRPO).

Result: SpeechJudge-GRM achieves 77.2% accuracy (79.4% after scaling) on benchmark, outperforming Gemini-2.5-Flash (<70%) and classic Bradley-Terry model (72.7%). The model can also serve as reward function for aligning speech generation models.

Conclusion: SpeechJudge provides effective tools for human preference alignment in speech synthesis, with the GRM approach showing superior performance over existing methods and enabling better alignment of TTS models with human perception.

Abstract: Aligning large generative models with human feedback is a critical challenge. In speech synthesis, this is particularly pronounced due to the lack of a large-scale human preference dataset, which hinders the development of models that truly align with human perception. To address this, we introduce SpeechJudge, a comprehensive suite comprising a dataset, a benchmark, and a reward model centered on naturalness–one of the most fundamental subjective metrics for speech synthesis. First, we present SpeechJudge-Data, a large-scale human feedback corpus of 99K speech pairs. The dataset is constructed using a diverse set of advanced zero-shot text-to-speech (TTS) models across diverse speech styles and multiple languages, with human annotations for both intelligibility and naturalness preference. From this, we establish SpeechJudge-Eval, a challenging benchmark for speech naturalness judgment. Our evaluation reveals that existing metrics and AudioLLMs struggle with this task; the leading model, Gemini-2.5-Flash, achieves less than 70% agreement with human judgment, highlighting a significant gap for improvement. To bridge this gap, we develop SpeechJudge-GRM, a generative reward model (GRM) based on Qwen2.5-Omni-7B. It is trained on SpeechJudge-Data via a two-stage post-training process: Supervised Fine-Tuning (SFT) with Chain-of-Thought rationales followed by Reinforcement Learning (RL) with GRPO on challenging cases. On the SpeechJudge-Eval benchmark, the proposed SpeechJudge-GRM demonstrates superior performance, achieving 77.2% accuracy (and 79.4% after inference-time scaling @10) compared to a classic Bradley-Terry reward model (72.7%). Furthermore, SpeechJudge-GRM can be also employed as a reward function during the post-training of speech generation models to facilitate their alignment with human preferences.

Method: IMSE introduces two core innovations: 1) Amplitude-Aware Linear Attention (MALA) replaces the MET module, preserving query vector norm information to solve the “amplitude-ignoring” problem in linear attention without auxiliary compensation. 2) Inception Depthwise Convolution (IDConv) replaces deformable embedding, using parallel branches (square, horizontal, vertical strips) to capture spectrogram features with minimal parameter redundancy.

Result: IMSE reduces parameters by 16.8% (from 0.513M to 0.427M) while achieving competitive PESQ score of 3.373 on VoiceBank+DEMAND dataset, comparable to state-of-the-art performance.

Conclusion: The study sets a new benchmark for balancing model size and speech quality in ultra-lightweight speech enhancement, demonstrating that systematic optimization can significantly reduce computational burden while maintaining performance for resource-constrained devices.

Abstract: Achieving a balance between lightweight design and high performance remains a significant challenge for speech enhancement (SE) tasks on resource-constrained devices. Existing state-of-the-art methods, such as MUSE, have established a strong baseline with only 0.51M parameters by introducing a Multi-path Enhanced Taylor (MET) transformer and Deformable Embedding (DE). However, an in-depth analysis reveals that MUSE still suffers from efficiency bottlenecks: the MET module relies on a complex “approximate-compensate” mechanism to mitigate the limitations of Taylor-expansion-based attention, while the offset calculation for deformable embedding introduces additional computational burden. This paper proposes IMSE, a systematically optimized and ultra-lightweight network. We introduce two core innovations: 1) Replacing the MET module with Amplitude-Aware Linear Attention (MALA). MALA fundamentally rectifies the “amplitude-ignoring” problem in linear attention by explicitly preserving the norm information of query vectors in the attention calculation, achieving efficient global modeling without an auxiliary compensation branch. 2) Replacing the DE module with Inception Depthwise Convolution (IDConv). IDConv borrows the Inception concept, decomposing large-kernel operations into efficient parallel branches (square, horizontal, and vertical strips), thereby capturing spectrogram features with extremely low parameter redundancy. Extensive experiments on the VoiceBank+DEMAND dataset demonstrate that, compared to the MUSE baseline, IMSE significantly reduces the parameter count by 16.8% (from 0.513M to 0.427M) while achieving competitive performance comparable to the state-of-the-art on the PESQ metric (3.373). This study sets a new benchmark for the trade-off between model size and speech quality in ultra-lightweight speech enhancement.

Method: Proposes comprehensive framework for calibrating and fusing EEND models at probability level. Investigates two output formulations (multilabel and powerset representations) and their impact on calibration and fusion. Explores different calibration strategies (independent per-speaker vs joint in powerset space) and fusion orderings (Fuse-then-Calibrate vs calibrating before fusion).

Result: Proper calibration provides substantial improvements for individual models (up to 19% relative DER reduction), sometimes mitigating absence of domain adaptation. Joint calibration in powerset space consistently outperforms independent per-speaker calibration. Fusion substantially improves over individual models. Fuse-then-Calibrate ordering generally outperforms both calibrating before fusion and uncalibrated fusion while requiring calibration of only a single combined model. Best configuration outperforms DOVER-Lap in terms of DER.

Conclusion: This work establishes best practices for probability-level fusion of EEND systems and demonstrates the advantages of leveraging soft outputs over hard decisions, providing reliable confidence estimates essential for downstream applications.

Abstract: End-to-End Neural Diarization (EEND) systems produce frame-level probabilistic speaker activity estimates, yet since evaluation focuses primarily on Diarization Error Rate (DER), the reliability and calibration of these confidence scores have been largely neglected. When fusing multiple diarization systems, DOVER-Lap remains the only established approach, operating at the segment level with hard decisions. We propose working with continuous probability outputs, which enables more sophisticated fusion and calibration techniques that can leverage model uncertainty and complementary strengths across different architectures. This paper presents the first comprehensive framework for calibrating and fusing EEND models at the probability level. We investigate two output formulations (multilabel and powerset representations) and their impact on calibration and fusion effectiveness. Through extensive experiments on the CallHome two-speaker benchmark, we demonstrate that proper calibration provides substantial improvements even for individual models (up to 19% relative DER reduction), in some cases mitigating the absence of domain adaptation. We reveal that joint calibration in powerset space consistently outperforms independent per-speaker calibration, that fusion substantially improves over individual models, and that the Fuse-then-Calibrate ordering generally outperforms both calibrating before fusion and uncalibrated fusion while requiring calibration of only a single combined model. Our best configuration outperforms DOVER-Lap in terms of DER while providing reliable confidence estimates essential for downstream applications. This work proposes best practices for probability-level fusion of EEND systems and demonstrates the advantages of leveraging soft outputs over hard decisions.

Method: Systematically evaluate LLMs on financial classification tasks, generate SHAP values for LLMs, and compare them with LightGBM’s SHAP values to assess reliability and explainability.

Result: LLMs show divergence between their self-explanation of feature impact and SHAP values, and notable differences between LLM and LightGBM SHAP values, indicating reliability issues for structured financial modeling.

Conclusion: LLMs have limitations as standalone classifiers for structured financial modeling, but improved explainability mechanisms with few-shot prompting could make them usable in risk-sensitive domains.

Abstract: Large Language Models (LLMs) have attracted significant attention for classification tasks, offering a flexible alternative to trusted classical machine learning models like LightGBM through zero-shot prompting. However, their reliability for structured tabular data remains unclear, particularly in high stakes applications like financial risk assessment. Our study systematically evaluates LLMs and generates their SHAP values on financial classification tasks. Our analysis shows a divergence between LLMs self-explanation of feature impact and their SHAP values, as well as notable differences between LLMs and LightGBM SHAP values. These findings highlight the limitations of LLMs as standalone classifiers for structured financial modeling, but also instill optimism that improved explainability mechanisms coupled with few-shot prompting will make LLMs usable in risk-sensitive domains.

Method: FaVeX algorithm accelerates computation by: 1) dynamically combining batch and sequential processing of input features, 2) reusing information from previous queries for proving invariances and searching for feature assignments, and 3) introducing verifier-optimal robust explanations that factor verifier incompleteness into the explanation.

Result: Comprehensive experimental evaluation shows superior scalability - FaVeX and verifier-optimal robust explanations together can produce meaningful formal explanations on networks with hundreds of thousands of non-linear activations.

Conclusion: FaVeX addresses the scalability bottleneck of verified explanations, making formal explanation of large neural networks practical through algorithmic optimizations and a novel explanation framework that accounts for verifier limitations.

Abstract: Verified explanations are a theoretically-principled way to explain the decisions taken by neural networks, which are otherwise black-box in nature. However, these techniques face significant scalability challenges, as they require multiple calls to neural network verifiers, each of them with an exponential worst-case complexity. We present FaVeX, a novel algorithm to compute verified explanations. FaVeX accelerates the computation by dynamically combining batch and sequential processing of input features, and by reusing information from previous queries, both when proving invariances with respect to certain input features, and when searching for feature assignments altering the prediction. Furthermore, we present a novel and hierarchical definition of verified explanations, termed verifier-optimal robust explanations, that explicitly factors the incompleteness of network verifiers within the explanation. Our comprehensive experimental evaluation demonstrates the superior scalability of both FaVeX, and of verifier-optimal robust explanations, which together can produce meaningful formal explanation on networks with hundreds of thousands of non-linear activations.

Method: Use Bayesian linear regression with Gaussian processes using linear kernels, applying geometric transformation to avoid boundary-seeking behavior. This simple parametric approach enables closed-form sampling and linear computational scaling with data.

Result: Linear models match state-of-the-art performance on 60-6,000 dimensional tasks and scale effectively to molecular optimization with >20,000 observations, outperforming existing BO methods.

Conclusion: Simple linear models challenge traditional intuitions about BO in high dimensions, suggesting need to reconsider complex structural assumptions in favor of simpler, scalable parametric approaches.

Abstract: High-dimensional spaces have challenged Bayesian optimization (BO). Existing methods aim to overcome this so-called curse of dimensionality by carefully encoding structural assumptions, from locality to sparsity to smoothness, into the optimization procedure. Surprisingly, we demonstrate that these approaches are outperformed by arguably the simplest method imaginable: Bayesian linear regression. After applying a geometric transformation to avoid boundary-seeking behavior, Gaussian processes with linear kernels match state-of-the-art performance on tasks with 60- to 6,000-dimensional search spaces. Linear models offer numerous advantages over their non-parametric counterparts: they afford closed-form sampling and their computation scales linearly with data, a fact we exploit on molecular optimization tasks with > 20,000 observations. Coupled with empirical analyses, our results suggest the need to depart from past intuitions about BO methods in high-dimensional spaces.

Method: Combines biaxial attention (alternating standard, grouped, hierarchical, and relational attention) with multi-CLS summarization to capture local and global dependencies. Features a label-aware in-context learning head with hierarchical decision routing for large label spaces. Meta-trained on synthetically generated, structurally diverse tables with causal priors.

Result: Outperforms gradient-boosting baselines and remains competitive with state-of-the-art tabular foundation models on public benchmarks. Demonstrates robust few-shot learning capabilities across heterogeneous tabular data.

Conclusion: Biaxial attention combined with episodic meta-training enables robust, few-shot-ready tabular learning. The approach learns transferable inductive biases that generalize across diverse tabular data structures, providing a practical foundation model solution for real-world tabular ML applications.

Abstract: Tabular data drive most real-world machine learning applications, yet building general-purpose models for them remains difficult. Mixed numeric and categorical fields, weak feature structure, and limited labeled data make scaling and generalization challenging. To this end, we introduce Orion-Bix, a tabular foundation model that combines biaxial attention with meta-learned in-context reasoning for few-shot tabular learning. Its encoder alternates standard, grouped, hierarchical, and relational attention, fusing their outputs through multi-CLS summarization to capture both local and global dependencies efficiently. A label-aware ICL head adapts on the fly and scales to large label spaces via hierarchical decision routing. Meta-trained on synthetically generated, structurally diverse tables with causal priors, Orion-Bix learns transferable inductive biases across heterogeneous data. Delivered as a scikit-learn compatible foundation model, it outperforms gradient-boosting baselines and remains competitive with state-of-the-art tabular foundation models on public benchmarks, showing that biaxial attention with episodic meta-training enables robust, few-shot-ready tabular learning. The model is publicly available at https://github.com/Lexsi-Labs/Orion-BiX .

Method: Proposes a hybrid framework integrating advanced U-Net variants with spatial clustering. The core innovation is Context Fusion Attention (CFA) U-Net, which fuses spatial and Sobel-derived geometric features within attention gates. Five architectures were evaluated across different data sparsity regimes, with DBSCAN clustering used to refine merged horizon predictions.

Result: Achieved state-of-the-art performance with validation IoU of 0.881 and MAE of 2.49ms on Mexilhao field dataset, and 97.6% surface coverage on F3 Block dataset under sparse conditions. The hybrid approach outperformed existing baselines across various sparsity regimes.

Conclusion: The hybrid methodology combining attention-based architectures with geometric context and spatial clustering provides a robust, generalizable solution for seismic interpretation in structurally complex and data-scarce environments, demonstrating advantages over traditional approaches.

Abstract: Interpreting seismic horizons is a critical task for characterizing subsurface structures in hydrocarbon exploration. Recent advances in deep learning, particularly U-Net-based architectures, have significantly improved automated horizon tracking. However, challenges remain in accurately segmenting complex geological features and interpolating horizons from sparse annotations. To address these issues, a hybrid framework is presented that integrates advanced U-Net variants with spatial clustering to enhance horizon continuity and geometric fidelity. The core contribution is the Context Fusion Attention (CFA) U-Net, a novel architecture that fuses spatial and Sobel-derived geometric features within attention gates to improve both precision and surface completeness. The performance of five architectures, the U-Net (Standard and compressed), U-Net++, Attention U-Net, and CFA U-Net, was systematically evaluated across various data sparsity regimes (10-, 20-, and 40-line spacing). This approach outperformed existing baselines, achieving state-of-the-art results on the Mexilhao field (Santos Basin, Brazil) dataset with a validation IoU of 0.881 and MAE of 2.49ms, and excellent surface coverage of 97.6% on the F3 Block of the North Sea dataset under sparse conditions. The framework further refines merged horizon predictions (inline and cross-line) using Density-Based Spatial Clustering of Applications with Noise (DBSCAN) to produce geologically plausible surfaces. The results demonstrate the advantages of hybrid methodologies and attention-based architectures enhanced with geometric context, providing a robust and generalizable solution for seismic interpretation in structurally complex and data-scarce environments.

Method: Analyzing the Riemannian pullback metric across neural network layers to study representational geometry, decomposing computation into discretization of continuous features and logical operations on discretized variables.

Result: Found that network computation decomposes into discretization and logical operations, and that different learning regimes (rich vs. lazy) have contrasting metric and curvature structures affecting generalization to unseen inputs.

Conclusion: Provides a geometric framework for understanding how neural networks learn discrete computations on continuous manifolds, revealing fundamental computational principles through representational geometry analysis.

Abstract: Many tasks require mapping continuous input data (e.g. images) to discrete task outputs (e.g. class labels). Yet, how neural networks learn to perform such discrete computations on continuous data manifolds remains poorly understood. Here, we show that signatures of such computations emerge in the representational geometry of neural networks as they learn. By analysing the Riemannian pullback metric across layers of a neural network, we find that network computation can be decomposed into two functions: discretising continuous input features and performing logical operations on these discretised variables. Furthermore, we demonstrate how different learning regimes (rich vs. lazy) have contrasting metric and curvature structures, affecting the ability of the networks to generalise to unseen inputs. Overall, our work provides a geometric framework for understanding how neural networks learn to perform discrete computations on continuous manifolds.

Method: Developed an MLP construction framework that works for almost all feasible input-output pairs, achieves asymptotically optimal parameter efficiency matching information-theoretic bounds, and maintains usability within Transformers for factual recall. The approach enables modular fact editing by replacing entire MLPs in one-layer Transformers.

Result: Discovered a metric on value embeddings that characterizes facts-per-parameter scaling for both constructed and gradient-descent-trained MLPs; identified a simple encoder-decoder mechanism matching gradient-descent MLP facts-per-parameter asymptotics; uncovered fundamental tradeoff between MLP fact-storage capacity and Transformer usability.

Conclusion: The improved MLP construction framework provides better understanding of fact storage mechanisms in LLMs, reveals fundamental scaling laws and tradeoffs, and enables practical applications like modular fact editing through MLP replacement in Transformers.

Abstract: The success of large language models (LLMs) can be attributed in part to their ability to efficiently store factual knowledge as key-value mappings within their MLP parameters. Recent work has proposed explicit weight constructions to build such fact-storing MLPs, providing an improved understanding of LLM fact storage mechanisms. In this paper, we introduce an MLP construction framework that improves over previous constructions in three areas: it 1) works for all but a measure-zero set of feasible input-output pairs, 2) achieves asymptotically optimal parameter efficiency matching information-theoretic bounds for some embeddings, and 3) maintains usability within Transformers for factual recall. Through our improvements, we 1) discover a metric on value embeddings that characterizes facts-per-parameter scaling for both constructed and gradient-descent-trained MLPs, 2) identify a simple encoder-decoder mechanism that empirically matches gradient-descent MLP facts-per-parameter asymptotics across all the inputs and outputs we test, and 3) uncover a fundamental tradeoff between an MLP’s fact-storage capacity and its usability within Transformers. Finally, we demonstrate a proof-of-concept application of fact-storing MLPs: modular fact editing on one-layer Transformers by \textit{replacing entire MLPs at once}.

Method: TIE extends n-class classifiers to (n+1)-class models by adding a garbage class initialized with Gaussian noise. It performs iterative training-inversion-exclusion cycles: training the classifier, inverting highly uncertain samples from it, and excluding these inverted samples into the garbage class. This creates visually coherent class prototypes over time.

Result: TIE achieves near-perfect OOD detection with ≈0 FPR@95%TPR on MNIST and FashionMNIST when tested against diverse unseen datasets. It provides unified anomaly detection and calibrated uncertainty estimation without needing external OOD datasets.

Conclusion: TIE offers a unified, interpretable framework for robust anomaly detection and calibrated uncertainty estimation that effectively addresses both OOD detection and uncertainty estimation challenges through its iterative refinement approach.

Abstract: Deep neural networks often struggle to recognize when an input lies outside their training experience, leading to unreliable and overconfident predictions. Building dependable machine learning systems therefore requires methods that can both estimate predictive \textit{uncertainty} and detect \textit{out-of-distribution (OOD)} samples in a unified manner. In this paper, we propose \textbf{TIE: a Training–Inversion–Exclusion} framework for visually interpretable and uncertainty-guided anomaly detection that jointly addresses these challenges through iterative refinement. TIE extends a standard $n$-class classifier to an $(n+1)$-class model by introducing a garbage class initialized with Gaussian noise to represent outlier inputs. Within each epoch, TIE performs a closed-loop process of \textit{training, inversion, and exclusion}, where highly uncertain inverted samples reconstructed from the just-trained classifier are excluded into the garbage class. Over successive iterations, the inverted samples transition from noisy artifacts into visually coherent class prototypes, providing transparent insight into how the model organizes its learned manifolds. During inference, TIE rejects OOD inputs by either directly mapping them to the garbage class or producing low-confidence, uncertain misclassifications within the in-distribution classes that are easily separable, all without relying on external OOD datasets. A comprehensive threshold-based evaluation using multiple OOD metrics and performance measures such as \textit{AUROC}, \textit{AUPR}, and \textit{FPR@95%TPR} demonstrates that TIE offers a unified and interpretable framework for robust anomaly detection and calibrated uncertainty estimation (UE) achieving near-perfect OOD detection with \textbf{(!\approx!) 0 FPR@95%TPR} when trained on MNIST or FashionMNIST and tested against diverse unseen datasets.

Method: Proposes PULSE, a cross-reconstruction-based pretraining objective that exploits information structure of dynamical systems generative models across multiple time-series. It extracts shared system parameters (class identity) while discarding sample-specific noise through cross-reconstruction between similar samples.

Result: Established theoretical conditions for system information recovery, validated with synthetic dynamical systems experiments. Applied to diverse real-world datasets, showing PULSE learns representations that distinguish semantic classes, increases label efficiency, and improves transfer learning.

Conclusion: PULSE provides a principled SSL framework for physiological time series that effectively extracts meaningful system information while filtering noise, outperforming existing methods across multiple real-world applications.

Abstract: The effectiveness of self-supervised learning (SSL) for physiological time series depends on the ability of a pretraining objective to preserve information about the underlying physiological state while filtering out unrelated noise. However, existing strategies are limited due to reliance on heuristic principles or poorly constrained generative tasks. To address this limitation, we propose a pretraining framework that exploits the information structure of a dynamical systems generative model across multiple time-series. This framework reveals our key insight that class identity can be efficiently captured by extracting information about the generative variables related to the system parameters shared across similar time series samples, while noise unique to individual samples should be discarded. Building on this insight, we propose PULSE, a cross-reconstruction-based pretraining objective for physiological time series datasets that explicitly extracts system information while discarding non-transferrable sample-specific ones. We establish theory that provides sufficient conditions for the system information to be recovered, and empirically validate it using a synthetic dynamical systems experiment. Furthermore, we apply our method to diverse real-world datasets, demonstrating that PULSE learns representations that can broadly distinguish semantic classes, increase label efficiency, and improve transfer learning.

Method: Novel hierarchical hybrid AI that synergizes scripted agents and RL agents. Scripted agents handle routine tactical-level decisions while RL agents manage higher-level strategic decision-making in a structured hierarchy.

Result: The integration significantly improves overall performance, providing a robust, adaptable, and effective solution for developing intelligent agents in complex simulation environments.

Conclusion: The hybrid hierarchical approach addresses limitations of both scripted and RL methods while leveraging their individual strengths, offering a superior solution for intelligent agent development in combat simulations.

Abstract: In the domain of combat simulations in support of wargaming, the development of intelligent agents has predominantly been characterized by rule-based, scripted methodologies with deep reinforcement learning (RL) approaches only recently being introduced. While scripted agents offer predictability and consistency in controlled environments, they fall short in dynamic, complex scenarios due to their inherent inflexibility. Conversely, RL agents excel in adaptability and learning, offering potential improvements in handling unforeseen situations, but suffer from significant challenges such as black-box decision-making processes and scalability issues in larger simulation environments. This paper introduces a novel hierarchical hybrid artificial intelligence (AI) approach that synergizes the reliability and predictability of scripted agents with the dynamic, adaptive learning capabilities of RL. By structuring the AI system hierarchically, the proposed approach aims to utilize scripted agents for routine, tactical-level decisions and RL agents for higher-level, strategic decision-making, thus addressing the limitations of each method while leveraging their individual strengths. This integration is shown to significantly improve overall performance, providing a robust, adaptable, and effective solution for developing and training intelligent agents in complex simulation environments.

Method: Proposes Polynomial Neural Sheaf Diffusion (PolyNSD) using a degree-K polynomial in normalized sheaf Laplacian, evaluated via stable three-term recurrence on spectrally rescaled operator. Uses convex mixture of K+1 orthogonal polynomial basis responses for trainable spectral response, with stability enforced through convex mixtures, spectral rescaling, and residual/gated paths.

Result: Achieves new state-of-the-art results on both homophilic and heterophilic benchmarks. Inverts previous trend by obtaining strong performance with just diagonal restriction maps (decoupling from large stalk dimension), while reducing runtime and memory requirements.

Conclusion: PolyNSD provides an efficient, stable sheaf diffusion approach with explicit K-hop receptive fields, overcoming limitations of previous methods while achieving superior performance with reduced computational overhead.

Abstract: Sheaf Neural Networks equip graph structures with a cellular sheaf: a geometric structure which assigns local vector spaces (stalks) and a linear learnable restriction/transport maps to nodes and edges, yielding an edge-aware inductive bias that handles heterophily and limits oversmoothing. However, common Neural Sheaf Diffusion implementations rely on SVD-based sheaf normalization and dense per-edge restriction maps, which scale with stalk dimension, require frequent Laplacian rebuilds, and yield brittle gradients. To address these limitations, we introduce Polynomial Neural Sheaf Diffusion (PolyNSD), a new sheaf diffusion approach whose propagation operator is a degree-K polynomial in a normalised sheaf Laplacian, evaluated via a stable three-term recurrence on a spectrally rescaled operator. This provides an explicit K-hop receptive field in a single layer (independently of the stalk dimension), with a trainable spectral response obtained as a convex mixture of K+1 orthogonal polynomial basis responses. PolyNSD enforces stability via convex mixtures, spectral rescaling, and residual/gated paths, reaching new state-of-the-art results on both homophilic and heterophilic benchmarks, inverting the Neural Sheaf Diffusion trend by obtaining these results with just diagonal restriction maps, decoupling performance from large stalk dimension, while reducing runtime and memory requirements.

Method: Proposes SD-CGAN framework with CTGAN-based synthetic data augmentation for class imbalance and Sinkhorn Divergence as geometry-aware loss function to improve training stability and reduce mode collapse.

Result: SD-CGAN achieves superior detection accuracy, precision, recall, and F1-score compared to baseline deep learning and GAN-based approaches while maintaining computational efficiency for edge deployment.

Conclusion: SD-CGAN provides an effective solution for robust anomaly detection in IoT edge environments, addressing class imbalance and sophisticated attack detection with computational efficiency suitable for edge deployment.

Abstract: The increasing complexity of IoT edge networks presents significant challenges for anomaly detection, particularly in identifying sophisticated Denial-of-Service (DoS) attacks and zero-day exploits under highly dynamic and imbalanced traffic conditions. This paper proposes SD-CGAN, a Conditional Generative Adversarial Network framework enhanced with Sinkhorn Divergence, tailored for robust anomaly detection in IoT edge environments. The framework incorporates CTGAN-based synthetic data augmentation to address class imbalance and leverages Sinkhorn Divergence as a geometry-aware loss function to improve training stability and reduce mode collapse. The model is evaluated on exploitative attack subsets from the CICDDoS2019 dataset and compared against baseline deep learning and GAN-based approaches. Results show that SD-CGAN achieves superior detection accuracy, precision, recall, and F1-score while maintaining computational efficiency suitable for deployment in edge-enabled IoT environments.

Method: Integrates sparse variational inference with KAN topology using M inducing points and analytic moment matching to reduce computational complexity from O(N³) to O(NM²). Also incorporates permutation-based importance analysis for structural identification.

Result: Achieves computational efficiency enabling probabilistic KANs to scale to larger scientific datasets while maintaining the ability to identify relevant inputs and classify functional relationships.

Conclusion: SVGP-KAN provides an efficient, scalable probabilistic extension to KANs that enables uncertainty quantification and structural identification for scientific applications.

Abstract: Kolmogorov-Arnold Networks (KANs) offer a promising alternative to Multi-Layer Perceptron (MLP) by placing learnable univariate functions on network edges, enhancing interpretability. However, standard KANs lack probabilistic outputs, limiting their utility in applications requiring uncertainty quantification. While recent Gaussian Process (GP) extensions to KANs address this, they utilize exact inference methods that scale cubically with data size N, restricting their application to smaller datasets. We introduce the Sparse Variational GP-KAN (SVGP-KAN), an architecture that integrates sparse variational inference with the KAN topology. By employing $M$ inducing points and analytic moment matching, our method reduces computational complexity from $O(N^3)$ to $O(NM^2)$ or linear in sample size, enabling the application of probabilistic KANs to larger scientific datasets. Furthermore, we demonstrate that integrating a permutation-based importance analysis enables the network to function as a framework for structural identification, identifying relevant inputs and classifying functional relationships.

Method: Compare magnitude thresholding to three 1D CNN architectures: literature baseline (separable convolutions), ultra-light purely separable model (305 params), and ultra-light residual separable model (533 params). Use BioStampRC21 dataset with 2-second windows at 30Hz, subject-independent leave-one-subject-out validation on 16 Parkinson’s patients with chest-worn IMUs.

Result: Residual separable model (533 params) achieves PR-AUC=94.5%, F1=91.2%, MCC=89.4%, matching/surpassing baseline (5,552 params). Smallest model (305 params) reaches PR-AUC=94.0%. Thresholding has high recall (89.0%) but low precision (76.5%) with many false positives. Chest and thigh sensors most reliable; forearm sensors degrade performance due to arm motion.

Conclusion: Ultra-light separable CNNs provide superior accuracy-efficiency-generalization trade-off compared to fixed thresholds for wearable PD gait detection. Compact models execute within tight memory/latency budgets on STM32-class MCUs (sub-10ms), enabling on-sensor gating of transmission/storage, demonstrating value of tailored time-series models for edge deployment.

Abstract: We study on-device time-series analysis for gait detection in Parkinson’s disease (PD) from short windows of triaxial acceleration, targeting resource-constrained wearables and edge nodes. We compare magnitude thresholding to three 1D CNNs for time-series analysis: a literature baseline (separable convolutions) and two ultra-light models - one purely separable and one with residual connections. Using the BioStampRC21 dataset, 2 s windows at 30 Hz, and subject-independent leave-one-subject-out (LOSO) validation on 16 PwPD with chest-worn IMUs, our residual separable model (Model 2, 533 params) attains PR-AUC = 94.5%, F1 = 91.2%, MCC = 89.4%, matching or surpassing the baseline (5,552 params; PR-AUC = 93.7%, F1 = 90.5%, MCC = 88.5%) with approximately 10x fewer parameters. The smallest model (Model 1, 305 params) reaches PR-AUC = 94.0%, F1 = 91.0%, MCC = 89.1%. Thresholding obtains high recall (89.0%) but low precision (76.5%), yielding many false positives and high inter-subject variance. Sensor-position analysis (train-on-all) shows chest and thighs are most reliable; forearms degrade precision/recall due to non-gait arm motion; naive fusion of all sites does not outperform the best single site. Both compact CNNs execute within tight memory/latency budgets on STM32-class MCUs (sub-10 ms on low-power boards), enabling on-sensor gating of transmission/storage. Overall, ultra-light separable CNNs provide a superior accuracy-efficiency-generalization trade-off to fixed thresholds for wearable PD gait detection and underscore the value of tailored time-series models for edge deployment.

Method: The paper first demonstrates vulnerabilities through unlearning-specific membership inference and reconstruction attacks. Then introduces WARP (Weight-space Alignment via Reparameterization), a plug-and-play teleportation defense that leverages neural network symmetries to reduce forget-set gradient energy and increase parameter dispersion while preserving predictions.

Result: WARP achieves consistent privacy gains across six unlearning algorithms, reducing adversarial advantage (AUC) by up to 64% in black-box and 92% in white-box settings, while maintaining accuracy on retained data.

Conclusion: Teleportation via neural network symmetries serves as an effective general tool for reducing attack success in approximate unlearning, addressing privacy vulnerabilities while preserving model utility.

Abstract: Approximate machine unlearning aims to efficiently remove the influence of specific data points from a trained model, offering a practical alternative to full retraining. However, it introduces privacy risks: an adversary with access to pre- and post-unlearning models can exploit their differences for membership inference or data reconstruction. We show these vulnerabilities arise from two factors: large gradient norms of forget-set samples and the close proximity of unlearned parameters to the original model. To demonstrate their severity, we propose unlearning-specific membership inference and reconstruction attacks, showing that several state-of-the-art methods (e.g., NGP, SCRUB) remain vulnerable. To mitigate this leakage, we introduce WARP, a plug-and-play teleportation defense that leverages neural network symmetries to reduce forget-set gradient energy and increase parameter dispersion while preserving predictions. This reparameterization obfuscates the signal of forgotten data, making it harder for attackers to distinguish forgotten samples from non-members or recover them via reconstruction. Across six unlearning algorithms, our approach achieves consistent privacy gains, reducing adversarial advantage (AUC) by up to 64% in black-box and 92% in white-box settings, while maintaining accuracy on retained data. These results highlight teleportation as a general tool for reducing attack success in approximate unlearning.

Method: Leverages Neural Architecture Search (NAS) to systematically explore vast architecture design spaces, evaluates architectures across multiple biological modalities, analyzes interplay between architecture, tokenization, and training strategies, and proposes architecture prediction methods for new tasks.

Result: Identifies novel, high-performance architectures and distills empirical design principles for biological foundation models, providing effective architecture prediction methods for new biological tasks.

Conclusion: BioArc provides a foundational resource and principled methodology to guide creation of next-generation task-specific and foundation models for biology, moving beyond intuition-driven architecture design.

Abstract: Foundation models have revolutionized various fields such as natural language processing (NLP) and computer vision (CV). While efforts have been made to transfer the success of the foundation models in general AI domains to biology, existing works focus on directly adopting the existing foundation model architectures from general machine learning domains without a systematic design considering the unique physicochemical and structural properties of each biological data modality. This leads to suboptimal performance, as these repurposed architectures struggle to capture the long-range dependencies, sparse information, and complex underlying ``grammars’’ inherent to biological data. To address this gap, we introduce BioArc, a novel framework designed to move beyond intuition-driven architecture design towards principled, automated architecture discovery for biological foundation models. Leveraging Neural Architecture Search (NAS), BioArc systematically explores a vast architecture design space, evaluating architectures across multiple biological modalities while rigorously analyzing the interplay between architecture, tokenization, and training strategies. This large-scale analysis identifies novel, high-performance architectures, allowing us to distill a set of empirical design principles to guide future model development. Furthermore, to make the best of this set of discovered principled architectures, we propose and compare several architecture prediction methods that effectively and efficiently predict optimal architectures for new biological tasks. Overall, our work provides a foundational resource and a principled methodology to guide the creation of the next generation of task-specific and foundation models for biology.

Method: Created a dataset of 308,700 simulations using GPU-accelerated electrophysiology solver across 147 patient-specific geometries. Used Universal Atrium Coordinate system for consistent representation and designed a neural operator with vision-transformer backbone to learn mapping from inputs to LAT fields.

Result: Achieved mean prediction error of 5.1 ms over 455 ms maximum simulation time, outperforming established operator-learning approaches with inference time of 0.12 ms per sample.

Conclusion: The framework establishes a general strategy for learning domain-invariant biophysical mappings across variable anatomical domains, enabling integration of computational electrophysiology into real-time and large-scale clinical workflows.

Abstract: Accurate maps of atrial electrical activation are essential for personalised treatment of arrhythmias, yet biophysically detailed simulations remain computationally intensive for real-time clinical use or population-scale analyses. Here we introduce a geometry-independent operator-learning framework that predicts local activation time (LAT) fields across diverse left atrial anatomies with near-instantaneous inference. We generated a dataset of 308,700 simulations using a GPU-accelerated electrophysiology solver, systematically varying multiple pacing sites and physiologically varied conduction properties across 147 patient-specific geometries derived from two independent clinical cohorts. All anatomical and functional data are expressed in a Universal Atrium Coordinate system, providing a consistent representation that decouples electrophysiological patterns from mesh topology. Within this coordinate space, we designed a neural operator with a vision-transformer backbone to learn the mapping from structural and electrophysiological inputs to LAT fields. With a mean prediction error of 5.1 ms over a 455 ms maximum simulation time, the model outperforms established operator-learning approaches and performs inference in 0.12 ms per sample. Our framework establishes a general strategy for learning domain-invariant biophysical mappings across variable anatomical domains and enables integration of computational electrophysiology into real-time and large-scale clinical workflows.

Method: Reformulate vehicle dynamics via local parameterization of time-dependent inputs, creating a sequence of local parametric dynamical systems. Use DRIPS for linear surrogate models with lifted observable spaces and manifold-based operator interpolation. Use FML (deep neural network) for nonlinear systems, extended with transfer-learning-based model correction for sparse high-fidelity measurements.

Result: DRIPS offers robust and highly data-efficient learning of non-autonomous vehicle dynamics. FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity, demonstrated through numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models.

Conclusion: The framework successfully addresses model-form uncertainty in vehicle dynamics through complementary data-driven approaches, enabling both efficient linear modeling and expressive nonlinear correction with minimal data requirements.

Abstract: We develop a data-driven framework for learning and correcting non-autonomous vehicle dynamics. Physics-based vehicle models are often simplified for tractability and therefore exhibit inherent model-form uncertainty, motivating the need for data-driven correction. Moreover, non-autonomous dynamics are governed by time-dependent control inputs, which pose challenges in learning predictive models directly from temporal snapshot data. To address these, we reformulate the vehicle dynamics via a local parameterization of the time-dependent inputs, yielding a modified system composed of a sequence of local parametric dynamical systems. We approximate these parametric systems using two complementary approaches. First, we employ the DRIPS (dimension reduction and interpolation in parameter space) methodology to construct efficient linear surrogate models, equipped with lifted observable spaces and manifold-based operator interpolation. This enables data-efficient learning of vehicle models whose dynamics admit accurate linear representations in the lifted spaces. Second, for more strongly nonlinear systems, we employ FML (Flow Map Learning), a deep neural network approach that approximates the parametric evolution map without requiring special treatment of nonlinearities. We further extend FML with a transfer-learning-based model correction procedure, enabling the correction of misspecified prior models using only a sparse set of high-fidelity or experimental measurements, without assuming a prescribed form for the correction term. Through a suite of numerical experiments on unicycle, simplified bicycle, and slip-based bicycle models, we demonstrate that DRIPS offers robust and highly data-efficient learning of non-autonomous vehicle dynamics, while FML provides expressive nonlinear modeling and effective correction of model-form errors under severe data scarcity.

Method: Proposes FiCoTS with LLM-as-Enhancer paradigm: 1) Token-level modality alignment via dynamic heterogeneous graph to filter noise and align time series patches with text tokens; 2) Feature-level modality interaction using global cross-attention to connect time series variables with relevant textual contexts; 3) Decision-level modality fusion with gated network to adaptively fuse results from both modalities.

Result: Extensive experiments on seven real-world benchmarks demonstrate state-of-the-art performance in multimodal time series forecasting.

Conclusion: The LLM-as-Enhancer paradigm effectively leverages LLMs’ text understanding capabilities to complement time series data, with the fine-to-coarse framework enabling comprehensive cross-modality interaction across three semantic levels for robust predictions.

Abstract: Time series forecasting is central to data analysis and web technologies. The recent success of Large Language Models (LLMs) offers significant potential for this field, especially from the cross-modality aspect. Most methods adopt an LLM-as-Predictor paradigm, using LLM as the forecasting backbone and designing modality alignment mechanisms to enable LLM to understand time series data. However, the semantic information in the two modalities of time series and text differs significantly, making it challenging for LLM to fully understand time series data. To mitigate this challenge, our work follows an LLM-as-Enhancer paradigm to fully utilize the advantage of LLM in text understanding, where LLM is only used to encode text modality to complement time series modality. Based on this paradigm, we propose FiCoTS, an LLM-enhanced fine-to-coarse framework for multimodal time series forecasting. Specifically, the framework facilitates progressive cross-modality interaction by three levels in a fine-to-coarse scheme: First, in the token-level modality alignment module, a dynamic heterogeneous graph is constructed to filter noise and align time series patches with text tokens; Second, in the feature-level modality interaction module, a global cross-attention mechanism is introduced to enable each time series variable to connect with relevant textual contexts; Third, in the decision-level modality fusion module, we design a gated network to adaptively fuse the results of the two modalities for robust predictions. These three modules work synergistically to let the two modalities interact comprehensively across three semantic levels, enabling textual information to effectively support temporal prediction. Extensive experiments on seven real-world benchmarks demonstrate that our model achieves state-of-the-art performance. The codes will be released publicly.

Method: Dual methodology: 1) Evolutionary and Bayesian hyperparameter optimization (Genetic Algorithms, Optuna) for numerical data, 2) Distributed processing in Apache Spark for massive textual corpora, comparing various models including SVM, Logistic Regression, deep architectures, and Gradient Boosting with Transformer-based embeddings (ROBERTa) and Bayesian Target Encoding.

Result: Revealed a “complexity paradox”: optimized linear models outperformed deep architectures in high-dimensional numerical data, while in text domains, distributed fine-tuning constraints caused overfitting in complex models, making simpler models with robust feature engineering more effective.

Conclusion: Provides a unified framework for algorithm selection based on data nature and infrastructure constraints, challenging the assumption that more complex models always perform better in Big Data scenarios.

Abstract: This study analyzes the impact of heterogeneity (“Variety”) in Big Data by comparing classification strategies across structured (Epsilon) and unstructured (Rest-Mex, IMDB) domains. A dual methodology was implemented: evolutionary and Bayesian hyperparameter optimization (Genetic Algorithms, Optuna) in Python for numerical data, and distributed processing in Apache Spark for massive textual corpora. The results reveal a “complexity paradox”: in high-dimensional spaces, optimized linear models (SVM, Logistic Regression) outperformed deep architectures and Gradient Boosting. Conversely, in text-based domains, the constraints of distributed fine-tuning led to overfitting in complex models, whereas robust feature engineering – specifically Transformer-based embeddings (ROBERTa) and Bayesian Target Encoding – enabled simpler models to generalize effectively. This work provides a unified framework for algorithm selection based on data nature and infrastructure constraints.

Method: Proposes Regularization Gradient Inversion Attack (RGIA) which enforces prior-knowledge-based regularization on states, rewards, and transition dynamics during optimization to ensure reconstructed data remains close to true transition distribution. The regularization term narrows solution space from broad set containing spurious solutions to constrained subset satisfying both gradient matching and true transition dynamics.

Result: Extensive experiments on control tasks and autonomous driving tasks demonstrate that RGIA can effectively constrain reconstructed data transition distributions and successfully reconstruct local private data.

Conclusion: RGIA addresses the unique challenge in FRL privacy attacks by incorporating transition dynamics constraints, enabling successful reconstruction of private data that aligns with real environment behavior while matching shared gradients.

Abstract: Federated reinforcement learning (FRL) enables distributed learning of optimal policies while preserving local data privacy through gradient sharing.However, FRL faces the risk of data privacy leaks, where attackers exploit shared gradients to reconstruct local training data.Compared to traditional supervised federated learning, successful reconstruction in FRL requires the generated data not only to match the shared gradients but also to align with real transition dynamics of the environment (i.e., aligning with the real data transition distribution).To address this issue, we propose a novel attack method called Regularization Gradient Inversion Attack (RGIA), which enforces prior-knowledge-based regularization on states, rewards, and transition dynamics during the optimization process to ensure that the reconstructed data remain close to the true transition distribution.Theoretically, we prove that the prior-knowledge-based regularization term narrows the solution space from a broad set containing spurious solutions to a constrained subset that satisfies both gradient matching and true transition dynamics.Extensive experiments on control tasks and autonomous driving tasks demonstrate that RGIA can effectively constrain reconstructed data transition distributions and thus successfully reconstruct local private data.

Method: Decomposes signed graphs into positive/negative subgraphs, designs gradient-perturbed adversarial module to approximate signed connectivity distribution, uses constrained breadth-first search tree strategy fused with balance theory to identify edge signs between generated node pairs, and implements gradient decoupling to reduce sensitivity.

Result: Extensive experiments on real-world datasets show ASGL achieves favorable privacy-utility trade-offs across multiple downstream tasks.

Conclusion: ASGL effectively addresses privacy concerns in signed graph learning by combining adversarial learning with differential privacy, overcoming limitations of existing methods through subgraph separation and gradient decoupling strategies.

Abstract: Signed graphs with positive and negative edges can model complex relationships in social networks. Leveraging on balance theory that deduces edge signs from multi-hop node pairs, signed graph learning can generate node embeddings that preserve both structural and sign information. However, training on sensitive signed graphs raises significant privacy concerns, as model parameters may leak private link information. Existing protection methods with differential privacy (DP) typically rely on edge or gradient perturbation for unsigned graph protection. Yet, they are not well-suited for signed graphs, mainly because edge perturbation tends to cascading errors in edge sign inference under balance theory, while gradient perturbation increases sensitivity due to node interdependence and gradient polarity change caused by sign flips, resulting in larger noise injection. In this paper, motivated by the robustness of adversarial learning to noisy interactions, we present ASGL, a privacy-preserving adversarial signed graph learning method that preserves high utility while achieving node-level DP. We first decompose signed graphs into positive and negative subgraphs based on edge signs, and then design a gradient-perturbed adversarial module to approximate the true signed connectivity distribution. In particular, the gradient perturbation helps mitigate cascading errors, while the subgraph separation facilitates sensitivity reduction. Further, we devise a constrained breadth-first search tree strategy that fuses with balance theory to identify the edge signs between generated node pairs. This strategy also enables gradient decoupling, thereby effectively lowering gradient sensitivity. Extensive experiments on real-world datasets show that ASGL achieves favorable privacy-utility trade-offs across multiple downstream tasks.

Method: Proposes KT-PSP framework incorporating problem-solving processes, introduces KT-PSP-25 dataset, and presents StatusKT with teacher-student-teacher three-stage LLM pipeline to extract mathematical proficiency indicators as intermediate signals.

Result: Experimental results on KT-PSP-25 show StatusKT improves prediction performance of existing KT methods and provides interpretable explanations by explicitly modeling students’ mathematical proficiency.

Conclusion: Incorporating problem-solving processes and using LLMs to extract mathematical proficiency indicators significantly enhances knowledge tracing performance and explainability, enabling more effective personalized learning.

Abstract: Knowledge Tracing (KT) aims to model student’s knowledge state and predict future performance to enable personalized learning in Intelligent Tutoring Systems. However, traditional KT methods face fundamental limitations in explainability, as they rely solely on the response correctness, neglecting the rich information embedded in students’ problem-solving processes. To address this gap, we propose Knowledge Tracing Leveraging Problem-Solving Process (KT-PSP), which incorporates students’ problem-solving processes to capture the multidimensional aspects of mathematical proficiency. We also introduce KT-PSP-25, a new dataset specifically designed for the KT-PSP. Building on this, we present StatusKT, a KT framework that employs a teacher-student-teacher three-stage LLM pipeline to extract students’ MP as intermediate signals. In this pipeline, the teacher LLM first extracts problem-specific proficiency indicators, then a student LLM generates responses based on the student’s solution process, and a teacher LLM evaluates these responses to determine mastery of each indicator. The experimental results on KT-PSP-25 demonstrate that StatusKT improves the prediction performance of existing KT methods. Moreover, StatusKT provides interpretable explanations for its predictions by explicitly modeling students’ mathematical proficiency.

Method: Holistic framework for IoT systems focusing on contextual decision making, proactive adaptation, and scalable structure; includes structured process for IoT systems with accuracy and interconnected development.

Result: Feasibility demonstrated through application of framework aspects; system includes long/short-term forecasting, anomaly detection, and qualitative data consideration for energy management decisions; evaluated on Power Consumption Time Series data.

Conclusion: The proposed holistic IoT framework shows feasibility for reducing power consumption and supporting grid stability through contextual decision-making and proactive adaptation.

Abstract: Power consumption has become a critical aspect of modern life due to the consistent reliance on technological advancements. Reducing power consumption or following power usage predictions can lead to lower monthly costs and improved electrical reliability. The proposal of a holistic framework to establish a foundation for IoT systems with a focus on contextual decision making, proactive adaptation, and scalable structure. A structured process for IoT systems with accuracy and interconnected development would support reducing power consumption and support grid stability. This study presents the feasibility of this proposal through the application of each aspect of the framework. This system would have long term forecasting, short term forecasting, anomaly detection, and consideration of qualitative data with any energy management decisions taken. Performance was evaluated on Power Consumption Time Series data to display the direct application of the framework.

Method: Two-stage framework: 1) Offline phase with nonlinear-least-squares estimation and generalization error analysis using Kullback-Leibler divergence for distribution shift; 2) Online phase with meta-LMS prediction algorithm to handle parameter drift in target systems.

Result: The two-stage framework demonstrates superior prediction performances compared with purely offline or online methods. Both theoretical guarantees and empirical studies support the findings.

Conclusion: Integrating offline learning with online adaptation provides a robust approach for real-world systems with correlated, non-stationary data, offering theoretical foundations and practical benefits for prediction tasks in nonlinear stochastic dynamical systems.

Abstract: Real-world intelligence systems usually operate by combining offline learning and online adaptation with highly correlated and non-stationary system data or signals, which, however, has rarely been investigated theoretically in the literature. This paper initiates a theoretical investigation on the prediction performance of a two-stage learning framework combining offline and online algorithms for a class of nonlinear stochastic dynamical systems. For the offline-learning phase, we establish an upper bound on the generalization error for approximate nonlinear-least-squares estimation under general datasets with strong correlation and distribution shift, leveraging the Kullback-Leibler divergence to quantify the distributional discrepancies. For the online-adaptation phase, we address, on the basis of the offline-trained model, the possible uncertain parameter drift in real-world target systems by proposing a meta-LMS prediction algorithm. This two-stage framework, integrating offline learning with online adaptation, demonstrates superior prediction performances compared with either purely offline or online methods. Both theoretical guarantees and empirical studies are provided.

Method: Memory-Efficient Nash Q-Learning (ME-Nash-QL) - a model-free self-play algorithm designed for two-player zero-sum Markov games. It uses Q-learning approach with memory-efficient implementation.

Result: ME-Nash-QL achieves: 1) Space complexity O(SABH) and sample complexity Õ(H⁴SAB/ε²); 2) Lowest computational complexity O(T·poly(AB)); 3) Best burn-in cost O(SAB·poly(H)). It outperforms existing algorithms in multiple metrics.

Conclusion: ME-Nash-QL represents significant progress in MARL theory by simultaneously addressing multiple limitations of existing methods, particularly excelling in memory efficiency, computational efficiency, and burn-in cost for two-player zero-sum Markov games.

Abstract: The thriving field of multi-agent reinforcement learning (MARL) studies how a group of interacting agents make decisions autonomously in a shared dynamic environment. Existing theoretical studies in this area suffer from at least two of the following obstacles: memory inefficiency, the heavy dependence of sample complexity on the long horizon and the large state space, the high computational complexity, non-Markov policy, non-Nash policy, and high burn-in cost. In this work, we take a step towards settling this problem by designing a model-free self-play algorithm \emph{Memory-Efficient Nash Q-Learning (ME-Nash-QL)} for two-player zero-sum Markov games, which is a specific setting of MARL. ME-Nash-QL is proven to enjoy the following merits. First, it can output an $\varepsilon$-approximate Nash policy with space complexity $O(SABH)$ and sample complexity $\widetilde{O}(H^4SAB/\varepsilon^2)$, where $S$ is the number of states, ${A, B}$ is the number of actions for two players, and $H$ is the horizon length. It outperforms existing algorithms in terms of space complexity for tabular cases, and in terms of sample complexity for long horizons, i.e., when $\min{A, B}\ll H^2$. Second, ME-Nash-QL achieves the lowest computational complexity $O(T\mathrm{poly}(AB))$ while preserving Markov policies, where $T$ is the number of samples. Third, ME-Nash-QL also achieves the best burn-in cost $O(SAB,\mathrm{poly}(H))$, whereas previous algorithms have a burn-in cost of at least $O(S^3 AB,\mathrm{poly}(H))$ to attain the same level of sample complexity with ours.

Method: Proposes a decentralized architecture of interacting small agent networks (SANs) where each agent represents specialized substructures covering only subsets of system functions. Uses swarm-learning in diverse swarms to enable self-adaptive SANs, drawing on mathematical results about SAN learning behavior and evidence from existing applications.

Result: Argues that swarm-learning in diverse swarms can enable self-adaptive SANs to deliver superior decision-making in dynamic environments compared to monolithic foundation models, though with reduced reproducibility in detail.

Conclusion: Monolithic foundation models face fundamental barriers in dynamic environments due to the curse of dimensionality, while decentralized SAN architectures offer a promising alternative for self-adaptive decision-making in complex systems, though AI should demonstrate clear superiority in these settings before assuming broader decision-making roles.

Abstract: Foundation models have rapidly advanced AI, raising the question of whether their decisions will ultimately surpass human strategies in real-world domains. The exponential, and possibly super-exponential, pace of AI development makes such analysis elusive. Nevertheless, many application areas that matter for daily life and society show only modest gains so far; a prominent case is diagnosing and treating dynamically evolving disease in intensive care. The common challenge is adapting complex systems to dynamic environments. Effective strategies must optimize outcomes in systems composed of strongly interacting functions while avoiding shared side effects; this requires reliable, self-adaptive modeling. These tasks align with building digital twins of highly complex systems whose mechanisms are not fully or quantitatively understood. It is therefore essential to develop methods for self-adapting AI models with minimal data and limited mechanistic knowledge. As this challenge extends beyond medicine, AI should demonstrate clear superiority in these settings before assuming broader decision-making roles. We identify the curse of dimensionality as a fundamental barrier to efficient self-adaptation and argue that monolithic foundation models face conceptual limits in overcoming it. As an alternative, we propose a decentralized architecture of interacting small agent networks (SANs). We focus on agents representing the specialized substructure of the system, where each agent covers only a subset of the full system functions. Drawing on mathematical results on the learning behavior of SANs and evidence from existing applications, we argue that swarm-learning in diverse swarms can enable self-adaptive SANs to deliver superior decision-making in dynamic environments compared with monolithic foundation models, though at the cost of reduced reproducibility in detail.

Method: RTZ-VI-LCB algorithm combines optimistic robust value iteration with data-driven Bernstein-style penalty term for robust value estimation, accounting for distribution shifts in historical dataset.

Result: Achieves near-optimal sample complexity guarantees under partial coverage and environmental uncertainty. Establishes information-theoretic lower bound confirming algorithm’s optimality regarding state and action spaces.

Conclusion: RTZ-VI-LCB is the first to attain optimal sample complexity for offline robust two-player zero-sum Markov games, setting a new benchmark and validated experimentally.

Abstract: Multi-agent reinforcement learning (MARL), as a thriving field, explores how multiple agents independently make decisions in a shared dynamic environment. Due to environmental uncertainties, policies in MARL must remain robust to tackle the sim-to-real gap. We focus on robust two-player zero-sum Markov games (TZMGs) in offline settings, specifically on tabular robust TZMGs (RTZMGs). We propose a model-based algorithm (\textit{RTZ-VI-LCB}) for offline RTZMGs, which is optimistic robust value iteration combined with a data-driven Bernstein-style penalty term for robust value estimation. By accounting for distribution shifts in the historical dataset, the proposed algorithm establishes near-optimal sample complexity guarantees under partial coverage and environmental uncertainty. An information-theoretic lower bound is developed to confirm the tightness of our algorithm’s sample complexity, which is optimal regarding both state and action spaces. To the best of our knowledge, RTZ-VI-LCB is the first to attain this optimality, sets a new benchmark for offline RTZMGs, and is validated experimentally.

Method: Proposes CaDiff framework with: 1) Asynchronous Diffusion Model (ADM) enabling forward/reverse processes with different steps to interpret perturbations as suppressible noise, 2) New bisimulation metric to quantify similarity between partial observations and causal states, 3) Theoretical guarantees via value function approximation error bounds.

Result: Experiments on Roboschool tasks show CaDiff enhances returns by at least 14.18% compared to baselines, demonstrating practical effectiveness in handling perturbed partial observability.

Conclusion: CaDiff is the first framework to approximate causal states using diffusion models with both theoretical rigor and practical effectiveness, successfully addressing the dual challenge of perturbations and partial observability in RL.

Abstract: A critical challenge for reinforcement learning (RL) is making decisions based on incomplete and noisy observations, especially in perturbed and partially observable Markov decision processes (P$^2$OMDPs). Existing methods fail to mitigate perturbations while addressing partial observability. We propose \textit{Causal State Representation under Asynchronous Diffusion Model (CaDiff)}, a framework that enhances any RL algorithm by uncovering the underlying causal structure of P$^2$OMDPs. This is achieved by incorporating a novel asynchronous diffusion model (ADM) and a new bisimulation metric. ADM enables forward and reverse processes with different numbers of steps, thus interpreting the perturbation of P$^2$OMDP as part of the noise suppressed through diffusion. The bisimulation metric quantifies the similarity between partially observable environments and their causal counterparts. Moreover, we establish the theoretical guarantee of CaDiff by deriving an upper bound for the value function approximation errors between perturbed observations and denoised causal states, reflecting a principled trade-off between approximation errors of reward and transition-model. Experiments on Roboschool tasks show that CaDiff enhances returns by at least 14.18% compared to baselines. CaDiff is the first framework that approximates causal states using diffusion models with both theoretical rigor and practicality.

Method: Train a privileged multimodal teacher that uses textual narratives from a Large Multimodal Model to reason about event causes while decoupling spectral components in its latent space, then distill this unified semantic-spectral knowledge into a vision-only student using a novel distillation objective.

Result: S^2-KD significantly boosts performance of simple student models on benchmarks like WeatherBench and TaxiBJ+, enabling them to outperform state-of-the-art methods, especially in long-horizon and complex non-stationary scenarios.

Conclusion: By unifying semantic priors with spectral representations, S^2-KD creates lightweight forecasting models that achieve both spectral accuracy and semantic coherence without requiring textual input or architectural overhead at inference.

Abstract: Spatiotemporal forecasting often relies on computationally intensive models to capture complex dynamics. Knowledge distillation (KD) has emerged as a key technique for creating lightweight student models, with recent advances like frequency-aware KD successfully preserving spectral properties (i.e., high-frequency details and low-frequency trends). However, these methods are fundamentally constrained by operating on pixel-level signals, leaving them blind to the rich semantic and causal context behind the visual patterns. To overcome this limitation, we introduce S^2-KD, a novel framework that unifies Semantic priors with Spectral representations for distillation. Our approach begins by training a privileged, multimodal teacher model. This teacher leverages textual narratives from a Large Multimodal Model (LMM) to reason about the underlying causes of events, while its architecture simultaneously decouples spectral components in its latent space. The core of our framework is a new distillation objective that transfers this unified semantic-spectral knowledge into a lightweight, vision-only student. Consequently, the student learns to make predictions that are not only spectrally accurate but also semantically coherent, without requiring any textual input or architectural overhead at inference. Extensive experiments on benchmarks like WeatherBench and TaxiBJ+ show that S^2-KD significantly boosts the performance of simple student models, enabling them to outperform state-of-the-art methods, particularly in long-horizon and complex non-stationary scenarios.

Method: A framework that incorporates offline RL algorithms into online RL, with variants like final policy recommendation and online fine-tuning, plus specialized techniques to enhance effectiveness.

Result: 1) Framework effectiveness strongly depends on task nature, 2) Proposed techniques significantly improve effectiveness, 3) Existing online fine-tuning methods are generally ineffective.

Conclusion: Offline RL can be effectively integrated into online RL via historical data repurposing, but requires task-specific approaches and better online fine-tuning methods.

Abstract: We take the novel perspective of incorporating offline RL algorithms as subroutines of tabula rasa online RL. This is feasible because an online learning agent can repurpose its historical interactions as offline dataset. We formalize this idea into a framework that accommodates several variants of offline RL incorporation such as final policy recommendation and online fine-tuning. We further introduce convenient techniques to improve its effectiveness in enhancing online learning efficiency. Our extensive and systematic empirical analyses show that 1) the effectiveness of the proposed framework depends strongly on the nature of the task, 2) our proposed techniques greatly enhance its effectiveness, and 3) existing online fine-tuning methods are overall ineffective, calling for more research therein.

Method: Decoder-only transformer trained on billion-scale datastream of synthesizable pathways paired with molecular properties, using a real-time, high-throughput C++-based data generation engine. Learns property-pathway mappings to generate molecules satisfying single or composite property queries.

Result: PrexSyn achieves near-perfect reconstruction of synthesizable chemical space with high inference speed, learns property-pathway associations, and can generate molecules satisfying complex property queries. Outperforms synthesis-agnostic baselines in molecular optimization efficiency.

Conclusion: PrexSyn sets new state-of-the-art in synthesizable chemical space coverage, molecular sampling efficiency, and inference speed, pushing the frontier of synthesizable molecular design as a powerful general-purpose optimization tool.

Abstract: The constrained nature of synthesizable chemical space poses a significant challenge for sampling molecules that are both synthetically accessible and possess desired properties. In this work, we present PrexSyn, an efficient and programmable model for molecular discovery within synthesizable chemical space. PrexSyn is based on a decoder-only transformer trained on a billion-scale datastream of synthesizable pathways paired with molecular properties, enabled by a real-time, high-throughput C++-based data generation engine. The large-scale training data allows PrexSyn to reconstruct the synthesizable chemical space nearly perfectly at a high inference speed and learn the association between properties and synthesizable molecules. Based on its learned property-pathway mappings, PrexSyn can generate synthesizable molecules that satisfy not only single-property conditions but also composite property queries joined by logical operators, thereby allowing users to ``program’’ generation objectives. Moreover, by exploiting this property-based querying capability, PrexSyn can efficiently optimize molecules against black-box oracle functions via iterative query refinement, achieving higher sampling efficiency than even synthesis-agnostic baselines, making PrexSyn a powerful general-purpose molecular optimization tool. Overall, PrexSyn pushes the frontier of synthesizable molecular design by setting a new state of the art in synthesizable chemical space coverage, molecular sampling efficiency, and inference speed.

Method: Two technical approaches: (1) Derive novel path-length bound for alternating gradient descent-ascent in min-max games, and (2) Show hidden convex-concave geometry reduces to two-sided Polyak-Łojasiewicz condition with high probability under overparameterization using random matrix theory tools.

Result: First global convergence guarantees to von Neumann-Nash equilibria for non-convex min-max games involving two-layer neural networks, with sufficient conditions spanning initialization, training dynamics, and network width.

Conclusion: Hidden convexity emerges in overparameterized neural network games, explaining empirical convergence and providing theoretical foundation for gradient methods in adversarial training and related applications.

Abstract: Many emerging applications - such as adversarial training, AI alignment, and robust optimization - can be framed as zero-sum games between neural nets, with von Neumann-Nash equilibria (NE) capturing the desirable system behavior. While such games often involve non-convex non-concave objectives, empirical evidence shows that simple gradient methods frequently converge, suggesting a hidden geometric structure. In this paper, we provide a theoretical framework that explains this phenomenon through the lens of hidden convexity and overparameterization. We identify sufficient conditions - spanning initialization, training dynamics, and network width - that guarantee global convergence to a NE in a broad class of non-convex min-max games. To our knowledge, this is the first such result for games that involve two-layer neural networks. Technically, our approach is twofold: (a) we derive a novel path-length bound for the alternating gradient descent-ascent scheme in min-max games; and (b) we show that the reduction from a hidden convex-concave geometry to two-sided Polyak-Łojasiewicz (PŁ) min-max condition hold with high probability under overparameterization, using tools from random matrix theory.

Method: The paper introduces Merging with Directional Alignment (MDA), a unified geometric framework that aligns directional structures consistently in both parameter and feature spaces. This approach addresses the limitations of previous methods by considering directional information rather than just parameter values or coefficients.

Result: The analysis shows that directional alignment improves structural coherence. Extensive experiments across benchmarks, model scales, and task configurations validate the effectiveness of the approach, demonstrating superior performance compared to previous merging methods.

Conclusion: Directional alignment is crucial for effective model merging. The proposed MDA framework successfully addresses directional inconsistencies in both parameter and feature spaces, leading to improved structural coherence and better performance across diverse tasks and model configurations.

Abstract: Model merging has emerged as a practical paradigm for integrating multiple independently trained models into a single model without joint retraining. Previous studies have demonstrated the effectiveness of combining parameters through strategies such as parameter decomposition, coefficient optimization, and subspace learning, significantly reducing the need for expensive joint training and achieving strong empirical performance across diverse tasks. However, these approaches predominantly treat merging as a problem of parameter space decomposition or fusion coefficient optimization, while overlooking the critical role of directional information in both parameter and feature spaces. In practice, naïve merging introduces inconsistencies in dominant parameter directions and disrupts structural coherence across models, which can degrade performance. Moreover, coefficient-based optimization methods implicitly assume compatible feature-space directions across models. However, Neural Collapse indicates that class features follow structured directional patterns, which may differ across independently trained models, making coefficient optimization alone insufficient. In this work, we emphasize the importance of \emph{directional alignment} and introduce a unified geometric framework, \emph{Merging with Directional Alignment} (\method{}), which aligns directional structures consistently in both the parameter and feature spaces. Our analysis shows that directional alignment improves structural coherence, and extensive experiments across benchmarks, model scales, and task configurations further validate the effectiveness of our approach.

Method: SelfAI uses three agents: User Agent for translating research objectives into configurations, Cognitive Agent with LLMs and optimal stopping criteria for hyperparameter search refinement, and Experiment Manager for orchestrating parallel, fault-tolerant workflows across hardware with structured knowledge base.

Result: SelfAI achieves strong performance across regression, NLP, computer vision, scientific computing, medical imaging, and drug discovery benchmarks, reduces redundant trials compared to Bayesian optimization and LLM baselines, and enables seamless human interaction.

Conclusion: SelfAI addresses key limitations in autonomous scientific discovery by providing a general multi-agent platform with optimal stopping, efficient resource utilization, and improved human-AI collaboration through novel evaluation metrics.

Abstract: Recent work on autonomous scientific discovery has leveraged LLM-based agents to integrate problem specification, experiment planning, and execution into end-to-end systems. However, these frameworks are often confined to narrow application domains, offer limited real-time interaction with researchers, and lack principled mechanisms for determining when to halt exploration, resulting in inefficiencies, reproducibility challenges, and under-utilized human expertise. To address these gaps, we propose \textit{SelfAI}, a general multi-agent platform that combines a User Agent for translating high-level research objectives into standardized experimental configurations, a Cognitive Agent powered by LLMs with optimal stopping criteria to iteratively refine hyperparameter searches, and an Experiment Manager responsible for orchestrating parallel, fault-tolerant training workflows across heterogeneous hardware while maintaining a structured knowledge base for continuous feedback. We further introduce two novel evaluation metrics, Score and $\text{AUP}_D$, to quantify discovery efficiency and search diversity. Across regression, NLP, computer vision, scientific computing, medical imaging, and drug discovery benchmarks, SelfAI consistently achieves strong performance and reduces redundant trials compared to classical Bayesian optimization and LLM-based baselines, while enabling seamless interaction with human researchers.

Method: Constructs graph of interrelated signals based on trend similarity, then applies graph neural networks (GNNs) for prediction. Enables interpretable analysis by revealing predictable signals and important relationships.

Result: Framework provides interpretable forecasting by identifying which signals are more predictable and which relationships contribute most to forecasting accuracy.

Conclusion: Method offers early steps toward interpretable modeling for domains with multiple interdependent signals, with implications for building future simulation models integrating behavior, beliefs, and observations.

Abstract: Epidemic outcomes have a complex interplay with human behavior and beliefs. Most of the forecasting literature has focused on the task of predicting epidemic signals using simple mechanistic models or black-box models, such as deep transformers, that ingest all available signals without offering interpretability. However, to better understand the mechanisms and predict the impact of interventions, we need the ability to forecast signals associated with beliefs and behaviors in an interpretable manner. In this work, we propose a graph-based forecasting framework that first constructs a graph of interrelated signals based on trend similarity, and then applies graph neural networks (GNNs) for prediction. This approach enables interpretable analysis by revealing which signals are more predictable and which relationships contribute most to forecasting accuracy. We believe our method provides early steps towards a framework for interpretable modeling in domains with multiple potentially interdependent signals, with implications for building future simulation models that integrate behavior, beliefs, and observations.

Method: Enhanced state-of-the-art time-series generative model (TimeGAN) to handle longitudinal clinical data with quantifiable privacy safeguards, creating both non-private (Augmented TimeGAN) and private (DP-TimeGAN) versions.

Result: Non-private model outperforms transformer- and flow-based models on statistical metrics; private model achieves mean authenticity of 0.778 on CKD dataset, outperforming SOTA on privacy-utility frontier. Both models perform comparably to real data in clinician evaluations.

Conclusion: The approach provides robust synthetic data for developing models for complex chronic conditions without compromising data privacy, enabling ML applications in healthcare while respecting regulations.

Abstract: Data privacy is a critical challenge in modern medical workflows as the adoption of electronic patient records has grown rapidly. Stringent data protection regulations limit access to clinical records for training and integrating machine learning models that have shown promise in improving diagnostic accuracy and personalized care outcomes. Synthetic data offers a promising alternative; however, current generative models either struggle with time-series data or lack formal privacy guaranties. In this paper, we enhance a state-of-the-art time-series generative model to better handle longitudinal clinical data while incorporating quantifiable privacy safeguards. Using real data from chronic kidney disease and ICU patients, we evaluate our method through statistical tests, a Train-on-Synthetic-Test-on-Real (TSTR) setup, and expert clinical review. Our non-private model (Augmented TimeGAN) outperforms transformer- and flow-based models on statistical metrics in several datasets, while our private model (DP-TimeGAN) maintains a mean authenticity of 0.778 on the CKD dataset, outperforming existing state-of-the-art models on the privacy-utility frontier. Both models achieve performance comparable to real data in clinician evaluations, providing robust input data necessary for developing models for complex chronic conditions without compromising data privacy.